> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/ai-runtime-setup.md).

# AI Runtime Setup

This document describes the current Trend Finder runtime foundation for AI OS. The repository has command names, dependency metadata, env placeholders, secret guardrails, a script-only Apify readiness layer, OpenAI account auth storage, a Codex account transport smoke path, a provider boundary, structured AI analyst validation, deterministic scoring, local snapshots, collector integration, dashboard runtime delivery, and a script-only historical backtest CLI.

## Current Behavior

* `apify-client` is installed.
* `apify:smoke` loads the Trend Finder Apify config wrapper, reports redacted readiness, includes reviewed candidate declaration counts, applies compliance and placeholder gates, and can optionally run configured Actors with `--check-sources`.
* Missing `APIFY_TOKEN` is a setup warning, not a script failure.
* Apify Actor and Dataset helper modules are script-only under `scripts/lib/apify/`.
* The Trend Finder collector can merge built-in public source adapters with configured Apify Actor/Dataset output when the extension is enabled and the operator provides reviewed source declarations.
* Apify source declarations carry role, quality, compliance, timeout, memory, item-cap, primary candidate, fallback candidate, and pending validation metadata. Restricted, unknown, disabled, or known placeholder Actor sources are disabled by compliance gates.
* `auth:openai` can report status, login, refresh, reauth, and logout using the script-only auth file path. Codex-account runtimes also attempt automatic refresh when stored account auth is expired; if the refresh token is rejected, readiness stays `expired-auth` with reauth guidance.
* `codex:smoke` can call the Codex account transport with stored account auth and emits sanitized readiness output.
* `agents:codex:smoke` still returns deterministic deferred output for the future Agents SDK provider.
* The AI runtime provider boundary supports disabled, mock, Codex-account, and placeholder OpenAI API modes. Disabled and missing-auth states produce readable fallback guidance instead of crashing the collector.
* AI OS host jobs can opt into the same provider boundary with script-only `AI_OS_AI_RUNTIME_*` names. Host names take precedence for host config, then fall back to existing `FINDTREND_*` compatibility names.
* AI OS Dream uses the host runtime boundary through the material-first scheduler job. Dream is enabled in starter scheduler metadata, keeps explicit activation, refreshes stale or missing material through agent aggregate, stays private-output only, and is manually triggerable from the local dashboard or CLI.
* Analyst output is parsed and validated against structured contracts. Invalid output gets one repair attempt and then falls back deterministically.
* Topic scoring combines AI relevance, novelty, evidence strength, source diversity, niche fit, and creator potential into dashboard score breakdowns.
* Local trend snapshots are written below `.cache/extensions/trend-finder/` for movement classification and are ignored by git.
* The `trend-finder:backtest` CLI replays reviewed historical windows, stores detailed output under `.cache/extensions/trend-finder/backtests/`, and publishes only bounded browser-safe summaries when explicitly requested.
* Each `aggregate` run writes a private structured trace to `logs/aggregate-*.jsonl` and `logs/aggregate.latest.jsonl`, including runtime readiness, AI-analysis request metadata, sanitized request payloads, sanitized responses, validation issues, fallback reasons, and mapped provider error causes.
* The dashboard renders runtime readiness, source health, ranked trends, hidden gems, watchlist state, score breakdowns, evidence links, creator angles, and the Brief view from generated runtime data.
* Dashboard provenance labels distinguish live generated data, fixture/demo data, AI-analyzed runs, mock runtime output, disabled runtime, deterministic fallback analysis, degraded sources, and blocked sources.
* `data/openai-account-auth.example.json` is a sanitized credential template.
* `data/openai-account-auth.json` is private generated data and is ignored by git.
* Runtime env values are script-only. No runtime secret should use a `VITE_` browser prefix.
* `AI_OS_OPENAI_ACCOUNT_AUTH_PATH`, `FINDTREND_OPENAI_ACCOUNT_AUTH_PATH`, and `FINDTREND_AI_RUNTIME_AUTH_PATH` are account-auth path selectors only; never paste account-auth JSON or tokens into env files.
* Secret scanning covers APIFY token values and OpenAI account-auth token fields.

## Deferred Behavior

The following work is intentionally deferred:

* Hosted production runtime operation.
* Direct OpenAI API provider execution beyond placeholder readiness.
* Agents SDK provider execution.
* Marketplace or dynamic extension loading.
* New source adapters, unreviewed Apify Actor selections, or dynamic Actor installation.
* Generated HTML report delivery; the implemented MVP delivery surface is the dashboard Brief view.
* Broad browser Dream controls beyond the local manual run button, a live scheduler API, a gateway, a daemon, remote Dream execution, and automatic Dream timer installation.

## Demo Runtime Paths

Use [Trend Finder Demo Workflow](/ai-os-and-trend-finder-docs/docs/hackathon/trend-finder-demo.md) for the full local walkthrough and [Hackathon Submission](/ai-os-and-trend-finder-docs/docs/hackathon/hackathon-submission.md) for the final 60-second script, proof points, certification commands, and deferrals.

Credential-free fixture/demo path:

```bash
rm -f src/data/live-data.json
bun run seed:data
bun run dev
```

Expected labels:

* `Fixture/demo data`
* `Deterministic fallback analysis`
* `Fixture/demo sources`
* Runtime status `Disabled`

Credentialed reviewed-source path:

```bash
bun run apify:smoke -- --json
bun run apify:smoke -- --tiny-validate --json
bun run auth:openai -- status
bun run aggregate
bun run dev
```

Expected labels depend on local setup:

* `Live generated data` when aggregation writes a new Trend Finder payload.
* `AI-analyzed run` when configured AI analysis succeeds.
* `Mock runtime analysis` when the mock runtime provider is selected.
* `Deterministic fallback analysis` when runtime analysis is unavailable or analyst output fails validation.
* `Reviewed public sources`, `Degraded source path`, or `Blocked source path` depending on source results.

## Final Certification

The Phase 06 local submission certification does not require live Apify or OpenAI credentials. The final command set is:

```bash
bun run typecheck
bun run typecheck:scripts
bun run test -- scripts/lib/apify/__tests__ scripts/lib/ai-runtime/__tests__ scripts/extensions/trend-finder/__tests__ scripts/extensions/trend-finder/sources/__tests__ src/lib/__tests__/trend-finder-schema.test.ts src/lib/__tests__/trend-finder-dashboard.test.tsx src/lib/__tests__/trend-finder-collector.test.ts
bun run test
bun run test:e2e -- tests/e2e/trend-finder.spec.ts
bun run lint
bun run format:check
bun run build
bun run budget:check
bun run runtime:check-private
```

Use missing-token and missing-auth checks only with redacted output. The certified command outputs report readiness states and `secretValuesPrinted: false`; they should not print token values, raw auth JSON, Actor input, Dataset rows, prompts, transcripts, raw responses, or private provenance IDs.

### Credential-Free Smoke Behavior

These optional integration checks are safe to run without private credentials when local env values are suppressed or a missing auth path is used.

| Command                                                                                                           | Expected missing-credential behavior                                                                        |
| ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `APIFY_TOKEN= FINDTREND_APIFY_ACTORS= FINDTREND_APIFY_ACTORS_PATH= bun run apify:smoke -- --json`                 | Exit `0`; `status: "setup-required"` and `readiness.status: "missing-token"`.                               |
| `APIFY_TOKEN= FINDTREND_APIFY_ACTORS= FINDTREND_APIFY_ACTORS_PATH= bun run apify:smoke -- --tiny-validate --json` | Exit `0`; tiny validation does not run live sources and each reviewed source is blocked by `missing-token`. |
| `bun run auth:openai -- status --json --auth-path <missing-path>`                                                 | Exit `0`; reports missing stored account auth without printing secrets.                                     |
| `bun run codex:smoke -- --json --auth-path <missing-path>`                                                        | Exit `1`; reports `missing_auth` with redacted recovery commands.                                           |
| `bun run agents:codex:smoke -- --json`                                                                            | Exit `78`; reports the deferred provider surface without importing `@openai/agents`.                        |

## Provenance Label Definitions

| Label                             | Meaning                                                                  |
| --------------------------------- | ------------------------------------------------------------------------ |
| `Live generated data`             | Local aggregation produced the Trend Finder payload.                     |
| `Fixture/demo data`               | Committed safe fallback or hand-authored demo data is active.            |
| `AI-analyzed run`                 | A configured AI runtime returned validated analyst output.               |
| `Mock runtime analysis`           | The mock provider produced deterministic local runtime output.           |
| `AI runtime disabled`             | No AI analysis was requested.                                            |
| `Deterministic fallback analysis` | Topics were grouped and scored without live AI clustering.               |
| `Reviewed public sources`         | Configured reviewed public sources contributed evidence.                 |
| `Degraded source path`            | At least one source returned partial data or warnings.                   |
| `Blocked source path`             | A source was skipped because credentials or reviewed config are missing. |

## Local Files

| Path                                    | Purpose                                       | Commit? |
| --------------------------------------- | --------------------------------------------- | ------- |
| `.env.local.example`                    | Committed script and browser env template     | Yes     |
| `.env.local`                            | Private local values copied from the template | No      |
| `data/openai-account-auth.example.json` | Sanitized account-auth example                | Yes     |
| `data/openai-account-auth.json`         | Generated account-auth credentials            | No      |
| `.cache/extensions/trend-finder/`       | Local Trend Finder cache and snapshots        | No      |
| `logs/aggregate-*.jsonl`                | Private aggregate and AI-analysis run traces  | No      |
| `logs/aggregate.latest.jsonl`           | Latest private aggregate run trace            | No      |

## Environment Variables

Copy `.env.local.example` to `.env.local` when local runtime values are needed. Use the `AI_OS_*` host names for AI OS scheduler jobs, including Dream. Keep the `FINDTREND_*` compatibility runtime keys in the private script-only section for Trend Finder extension collection and fallback compatibility.

When unset, the runtime boundary defaults to `provider: disabled`, `enabled: false`, `model: gpt-5.4-mini`, `reasoningEffort: medium`, and `originator: ai-os`.

Example AI OS host runtime overrides:

```
AI_OS_AI_RUNTIME_PROVIDER=codex-account
AI_OS_AI_RUNTIME_ENABLED=true
AI_OS_AI_RUNTIME_MODEL=gpt-5.5
AI_OS_AI_RUNTIME_REASONING_EFFORT=xhigh
AI_OS_AI_RUNTIME_BASE_URL=https://chatgpt.com/backend-api
AI_OS_OPENAI_ACCOUNT_AUTH_PATH=data/openai-account-auth.json
OPENAI_API_KEY=<key>
```

Host config precedence is:

```
AI_OS_AI_RUNTIME_PROVIDER -> FINDTREND_AI_RUNTIME_PROVIDER
AI_OS_AI_RUNTIME_ENABLED -> FINDTREND_AI_RUNTIME_ENABLED
AI_OS_AI_RUNTIME_MODEL -> FINDTREND_AI_RUNTIME_MODEL
AI_OS_AI_RUNTIME_REASONING_EFFORT -> FINDTREND_AI_RUNTIME_REASONING_EFFORT
AI_OS_AI_RUNTIME_BASE_URL -> FINDTREND_AI_RUNTIME_BASE_URL
AI_OS_OPENAI_ACCOUNT_AUTH_PATH -> FINDTREND_OPENAI_ACCOUNT_AUTH_PATH -> FINDTREND_AI_RUNTIME_AUTH_PATH
```

Example Trend Finder compatibility overrides:

```
FINDTREND_AI_RUNTIME_PROVIDER=codex-account
FINDTREND_AI_RUNTIME_ENABLED=true
FINDTREND_AI_RUNTIME_MODEL=gpt-5.5
FINDTREND_AI_RUNTIME_REASONING_EFFORT=xhigh
FINDTREND_AI_RUNTIME_ORIGINATOR=ai-os
FINDTREND_AI_RUNTIME_BASE_URL=https://chatgpt.com/backend-api
FINDTREND_AI_RUNTIME_AUTH_PATH=data/openai-account-auth.json
FINDTREND_OPENAI_ACCOUNT_AUTH_PATH=data/openai-account-auth.json
FINDTREND_OPENAI_OAUTH_CALLBACK_HOST=127.0.0.1
FINDTREND_APIFY_ACTORS_PATH=data/trend-finder.apify-actors.json
FINDTREND_APIFY_DEFAULT_MEMORY_MB=1024
FINDTREND_APIFY_DEFAULT_TIMEOUT_SECS=120
FINDTREND_APIFY_MAX_ITEMS_PER_SOURCE=50
APIFY_TOKEN=<token>
```

Implemented Apify defaults:

| Variable                               | Default | Bounds           |
| -------------------------------------- | ------- | ---------------- |
| `FINDTREND_APIFY_DEFAULT_MEMORY_MB`    | `1024`  | `128` to `32768` |
| `FINDTREND_APIFY_DEFAULT_TIMEOUT_SECS` | `120`   | `1` to `900`     |
| `FINDTREND_APIFY_MAX_ITEMS_PER_SOURCE` | `50`    | `1` to `500`     |

Invalid numeric values fall back to the default. Values above the maximum are clamped and reported as warnings.

Copy `data/trend-finder.apify-actors.example.json` to `data/trend-finder.apify-actors.json` for the editable local Apify source list. `FINDTREND_APIFY_ACTORS` remains available as an inline override and accepts either JSON or semicolon-separated entries. JSON is preferred for that override:

```json
[
  {
    "id": "github-ai-repositories",
    "actorId": "automation-lab/github-trending-scraper",
    "role": "trend-source",
    "enabled": true,
    "maxItems": 25,
    "maxTotalChargeUsd": 0.05,
    "timeoutSecs": 120,
    "memoryMb": 1024,
    "input": {
      "query": "AI agents"
    }
  }
]
```

For a quick local config shape, use `source-id=actor-id;other-source=actor-id`. Actor input and per-source `maxTotalChargeUsd` charge caps are only supported by the JSON form.

`OPENAI_API_KEY` remains the script-only key for official OpenAI API backed scripts. It is separate from the Codex account-auth JSON path.

Current behavior: host config can resolve disabled, mock, Codex-account, missing-auth, invalid-auth, and OpenAI API placeholder readiness states. Dream uses those states through the AI OS scheduler. Disabled and missing-auth states produce explicit skipped or fallback behavior; mock can produce credential-free output for validation; degraded, timeout, failed, blocked, stale-lock, and unreadable-state labels remain safe scheduler output.

## Commands

The runtime commands cover current smoke checks, aggregate refreshes, and the implemented Dream scheduler path:

```bash
bun run auth:openai -- login
bun run auth:openai -- status
bun run auth:openai -- refresh
bun run auth:openai -- reauth
bun run auth:openai -- logout
bun run codex:smoke -- --json
bun run agents:codex:smoke -- --json
bun run apify:smoke -- --json
bun run apify:smoke -- --json --check-sources
bun run scheduler:dream:run
bun run scheduler:dream:status
```

Manual `refresh` is still useful for operator diagnostics, but collectors and other AI runtime consumers attempt the same refresh automatically before declaring expired Codex-account auth unavailable.

Current command behavior:

| Command                  | Current result                                                                                                                            |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `auth:openai`            | Reports missing/invalid/expired/valid auth state; supports login, refresh, reauth, and logout without printing credential values          |
| `codex:smoke`            | Runs the account-auth Codex transport smoke path or returns redacted missing-auth recovery guidance                                       |
| `agents:codex:smoke`     | Reports deferred Agents SDK provider behavior and does not import `@openai/agents`                                                        |
| `apify:smoke`            | Reports implemented Trend Finder Apify config/client readiness; `--check-sources` runs configured reviewed Actors when a token is present |
| `aggregate`              | Runs enabled extension collectors, merges Trend Finder runtime output, and writes generated data to `src/data/live-data.json`             |
| `scheduler:dream:run`    | Runs material-first AI OS Dream through the scheduler without printing runtime secrets or private output contents                         |
| `scheduler:dream:status` | Prints safe Dream scheduler status labels, counts, timing, lock outcome, and command hints                                                |

The smoke commands do not print token values, raw account IDs, prompt payloads, Actor inputs, Dataset items, raw responses, or transcripts. The Apify smoke command prints declaration counts rather than raw candidate inputs.

Dream scheduler commands follow the same boundary. They must not print token values, raw account-auth JSON, prompts, provider responses, transcripts, raw source dumps, private output contents, local usernames, emails, or private path contents.

## Credential Handling

Use `data/openai-account-auth.example.json` only as a shape reference. The account auth CLI writes generated credentials to `data/openai-account-auth.json` by default, or to `--auth-path` / `AI_OS_OPENAI_ACCOUNT_AUTH_PATH` / `FINDTREND_OPENAI_ACCOUNT_AUTH_PATH` / `FINDTREND_AI_RUNTIME_AUTH_PATH` when overridden.

Rules:

* Never commit `data/openai-account-auth.json`.
* Never paste access tokens or refresh tokens into docs, tests, or issue text.
* Never expose account-auth data through `VITE_` variables.
* Generated auth writes use owner-only file permissions where the platform supports it.
* Treat readiness checks as metadata-only probes. They do not prove write, edit, or runtime execution capabilities.

## Secret Guardrails

Before committing runtime work, verify:

```bash
git check-ignore data/openai-account-auth.json
git ls-files data/openai-account-auth.json
git check-ignore src/data/live-data.json .cache/extensions/trend-finder/snapshots/latest.json
git ls-files src/data/live-data.json .cache/extensions/trend-finder/snapshots/latest.json
bun run runtime:check-private
bun run test -- src/lib/__tests__/git-hooks.test.ts
```

`git check-ignore` should print the ignored path. `git ls-files` should print nothing for the private auth path, generated live data path, and local snapshot cache path.

## Phase 06 Source Candidates

Phase 06 Session 01 updates the reviewed Apify candidate declarations for the hackathon source set:

| Source ID                   | Primary candidate                        | Fallback candidate                     |
| --------------------------- | ---------------------------------------- | -------------------------------------- |
| `github-ai-repositories`    | `automation-lab/github-trending-scraper` | `crawlerbros/github-repo-intelligence` |
| `reddit-ai-discussions`     | `clearpath/reddit-search-scraper`        | `iskander/fast-reddit-scraper`         |
| `arxiv-ai-papers`           | `easyapi/arxiv-search-scraper`           | `gentle_cloud/arxiv-paper-search`      |
| `producthunt-ai-launches`   | `muzafferkadir/product-hunt-leaderboard` | `thirdwatch/producthunt-scraper`       |
| `youtube-ai-creator-videos` | `api-ninja/youtube-search-scraper`       | `powerai/youtube-video-search-scraper` |
| `rss-ai-news`               | `xtech/feed-extractor`                   | none recorded                          |

These are reviewed static declarations, not proof that the Actors have been run successfully or that their Dataset rows match the normalizers. Session 02 owns tiny capped validation and fixture updates.

Known unresolved placeholder IDs such as `apify/github-public-search` are blocked before collection and reported with `placeholder-actor` warnings.

## Session Ownership

Phase 05 Session 02 owns the script-only Apify config, client, Actor, Dataset, normalization, smoke, and documentation foundation. Phase 05 Session 03 owns script-only OpenAI account auth, local credential storage, and the Codex transport smoke path. Phase 05 Session 04 owns the runtime provider boundary. Phase 05 Sessions 05-10 own analyst contracts, trend snapshots, collector runtime integration, source breadth, dashboard delivery, and certification. Phase 06 Sessions 01-07 own reviewed candidate declarations, tiny validation, Creator Lens persistence, lens-aware run control, watchlist/movement artifacts, source breakdown UI, and demo provenance semantics.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/ai-runtime-setup.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.