> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/extensions/trend-finder/pipeline.md).

# Trend Finder Pipeline

This guide covers the normal Trend Finder aggregate path, browser payload boundary, local run control, state locations, and primary implementation files.

## Pipeline Summary

Each normal Trend Finder aggregate run follows this implemented path:

1. Create a run ID and sanitized Engine Replay trace recorder.
2. Load the active Creator Lens from the Trend Finder extension cache, or use the default lens when no valid saved config exists.
3. Collect source data from built-in source adapters and then from configured reviewed Apify sources.
4. Normalize source items into two related records: browser-visible evidence and analyst-ready evidence.
5. Apply source-local signals, then merge analyst-ready evidence through the private enrichment cache. Unchanged cache hits reuse safe summaries, misses are written atomically, and stale entries outside the active keep set are pruned.
6. Aggregate per-source spend summaries from Apify and Google Trends demand. Exact usage is used when available; otherwise configured charge caps are labeled as estimates.
7. Load the previous Trend Finder snapshot and the 84-day historical context from the private extension cache.
8. Resolve AI runtime readiness.
9. Deduplicate and cap analyst-ready evidence to the top 120 records.
10. Run the AI trend analyst only when the runtime is ready and analyst-ready evidence exists.
11. Use deterministic fallback topic generation when AI analysis is skipped, unavailable, invalid, or fails validation.
12. Resolve topic identity against previous snapshots and historical context.
13. Score topics across six score factors, capped source-local support, and capped burst support.
14. Attach source breakdowns, consensus ratios, role shares, saturation, hidden-gem scores, risk flags, security relevance, velocity dynamics, lifecycle stages, convergence, score trajectories, 12-week summaries, and bounded sparklines to topics.
15. Generate the read-only watchlist. Browser-local watching remains a UI presentation state and is not written into generated data.
16. Generate movement analysis records and movement groups.
17. Generate demand clusters from question-shaped evidence and safe analyst or deterministic fallback copy.
18. Generate industry events from existing `rss-ai-news` and `google-ai-news` evidence only. Event rows publish only after at least two independent publisher identities support the cluster. Deterministic summaries are used unless a validator-gated analyst event brief cites known event and evidence IDs.
19. Generate theme rollups and top-N outlier ideas, reusing the private enrichment cache when possible.
20. Grade pending older predictions, then write new predictions for current topics, including target dates and lifecycle stages when available.
21. Build a sanitized Engine Replay trace.
22. Run required-derived-field closeout after browser-safe diagnostics are slimmed. Generated payloads must include per-topic `attentionPattern`, `receptionSignal`, `corroboration`, `securityRelevance`, and `evidenceRationales`, plus report-level `runNarratives` and `industryEvents`, before snapshots, static Brief readiness, or browser handoff can proceed.
23. Validate the Trend Finder payload, write a private local snapshot, prune private archive caches by retention policy, rebuild the sanitized trace with aggregate retention counts, and let the aggregate writer publish the browser payload to generated live data.

The browser reads only the generated payload under `LiveData.extensions.items["trend-finder"]`. Raw prompts, provider responses, logs, account auth files, and private source internals are not rendered.

One to Watch is projected after the payload is parsed by the browser or static Brief exporter. It does not add a collector stage or persisted branch. The projection reads existing prediction/retro summary rows and current topic fields, emits bounded display rows, and keeps raw archives, prompts, provider responses, private paths, raw source rows, comments, transcripts, and local triage notes out of UI and exports.

## Industry Events

The industry-events rollup is a report-level lens over current reviewed news evidence. It does not collect new sources, read article bodies, inspect comments, read transcripts, or change topic scoring.

The helper selects only normalized evidence whose `sourceId` is `rss-ai-news` or `google-ai-news`. Publisher identity prefers browser-safe `sourceLocal.entityKind: "publisher"` metadata and falls back to the public URL hostname only when publisher metadata is missing. Local/private hosts and Google-hosted redirect hosts are not used as fallback publishers.

Candidate evidence is clustered with deterministic title/entity tokens. A row publishes only when at least two independent publisher identities support the cluster. Single-publisher clusters are suppressed before browser payload publication and are summarized only as safe collector warnings. Published rows contain bounded titles, summaries, publisher labels, reason codes, related topic IDs, source IDs, and cited evidence IDs. They do not include raw article text, raw source payloads, prompts, provider responses, transcripts, comments, credentials, tokens, local paths, or cache paths.

The AI analyst may return optional industry event summaries for known event candidate IDs. Validator checks reject unknown event IDs and evidence IDs. The normal analyst metadata scan still rejects invented URLs, dates, metrics, and source-name fields. Invalid or unavailable AI output falls back to the deterministic event summary.

## Security Lens

`topic.securityRelevance` is a deterministic derived lens over existing browser-safe topic evidence. It reuses the reviewed `security` keyword category from the keyword-pack taxonomy and does not add a source, collector, feed, network call, dependency scanner, remediation step, or policy enforcement path.

The helper reads only normalized topic name, summary, why-now text, evidence title/snippet, and source label/type/role/quality fields that already exist in the generated payload. It skips private-looking text markers such as prompts, transcripts, provider responses, raw response bodies, raw source markers, dataset rows, comment-body markers, redacted strings, credential-shaped strings, and local paths.

Published output is bounded to enum-like display fields:

* `status`: `available` or `unavailable`
* `severity`: `critical`, `high`, `medium`, `low`, or `unavailable`
* `reasonCodes`: capped reviewed reason codes such as `prompt_injection`, `credential_exposure`, `supply_chain`, `guardrails`, `privacy`, or `reviewed_security_keyword`
* `actionItems`: capped informational labels such as `review_exposure`, `monitor_advisory`, `validate_guardrails`, `check_dependency_chain`, `track_policy_change`, or `review_privacy_controls`
* `citedEvidenceIds`: capped IDs that must exist in the generated evidence set and belong to the topic
* `summary` and `unavailableReason`

Non-security topics and topics without usable evidence publish explicit `unavailable` output. Derivation errors degrade to unavailable security relevance without blocking unrelated topic enrichment. The field is additive for legacy parsing, required for generated payload closeout, included in payload-size reports as `data.topics[].securityRelevance`, and projected into Workbench and static Brief surfaces as informational context only.

## Stage Validation Narration

Optional enrichment stages now publish bounded Engine Replay narration when a stage validates, retries, or degrades. The narrated stages are AI analysis, source-breakdown derivation, run narratives, and outlier ideas. Each wrapped stage retries at most once. If it still fails, the collector keeps the existing deterministic fallback: AI analysis falls back to deterministic scoring, source breakdown preserves scored topic output, run narratives are omitted, and outlier ideas preserve evidence output.

Browser-visible narration rows expose only safe status fields:

* stage ID
* status
* issue code
* retry count
* degradation path
* short sanitized message text

The trace sanitizer rejects unsafe narration rows before browser output. Raw model output, raw normalized rows, prompts, provider responses, request IDs, tokens, stack traces, local paths, cache paths, and private diagnostics remain outside Engine Replay and static/browser payloads.

## Phase 28 Release Checks

Phase 28 makes collection health part of release readiness. Normal runs publish dedup and coverage summaries after source normalization and before payload publication. These summaries are browser-safe aggregate labels only: duplicate rates, high-duplicate source counts, source coverage counts, warning counts, and bounded source IDs or labels where the UI already needs them. They do not include raw source rows, raw URLs, private cache paths, Actor inputs, Dataset rows, prompts, or provider responses.

Source-death baseline checks run at the same source boundary, after Source Setup state and accepted-evidence counts are known and before `sources.collected` is written to the sanitized trace. The helper reads and writes a private last-good accepted-evidence baseline under the collector cache. Browser payloads and Engine Replay receive only `sourceDeathAlarmCount`, `sourceDeathAlarmLabel`, and bounded per-source warning labels. They never include the private baseline file path, baseline file name, prior accepted count, raw diagnostics, tokens, or credential-shaped strings.

Source-state transitions stay explicit throughout the run:

| State          | Release Meaning                                                                      |
| -------------- | ------------------------------------------------------------------------------------ |
| `live`         | Reviewed sources contributed current browser-safe evidence.                          |
| `degraded`     | Reviewed sources contributed partial evidence or warnings.                           |
| `blocked`      | Required credentials, enablement, reviewed config, or compliance gates were missing. |
| `fallback`     | Deterministic fallback produced the row or topic.                                    |
| `fixture-demo` | Committed fixture/demo data is active and not proof of live collection.              |
| `unknown`      | Legacy payload did not publish a newer state.                                        |

Closeout validation treats payload, privacy, and static export checks as one release surface:

* Generated payload closeout is stricter than legacy schema parsing. `parseTrendFinderData()` and `validateTrendFinderData()` still preserve defaults for older payloads, but collector closeout and static Brief export reject generated/exported payloads that omit the required derived branches.
* `bun run trend-finder:export-brief` validates the generated payload, projects a smaller report contract, including bounded One-to-Watch rows, runs static Brief QA, scans for private strings, and writes only generated local report files under the ignored cache root.
* `bun run runtime:check-private` verifies private generated artifacts, scheduler state, logs, reports, coverage, and Playwright report paths remain ignored and untracked using git metadata only.
* `bun run build` followed by `bun run budget:check` verifies the shared browser payload and bundle budget remain under the configured release limits.

Validation records sanitized command outcomes only. Generated reports, payloads, trace files, private caches, raw logs, and `src/data/live-data.json` remain local artifacts and stay out of committed documentation.

## Phase 29 Release Validation

Phase 29 closeout treats validation narration, required derived fields, source-death baselines, static Brief export, payload budget, private artifacts, and dependency audit as one release bundle. A closeout record must name the command, sanitized result, and any unrelated drift rather than scattering evidence across generated private files.

The release bundle includes:

* Reference mode and Markdown phrase checks for shipped behavior, deferrals, non-goals, and private-boundary language.
* `bun run trend-finder:export-brief -- --dry-run --json` to prove static Brief projection, required-derived-field closeout, QA, and private-string scanning before output promotion.
* `bun run runtime:check-private` to prove generated payloads, static reports, logs, caches, coverage, Playwright reports, and `src/data/live-data.json` remain ignored and untracked.
* `bun run build` plus `bun run budget:check` to verify the browser bundle and payload budget gate.
* `bun audit` to record dependency posture for the phase.
* TypeScript, script TypeScript, Vitest, Playwright, and scoped Prettier proof for changed documentation and Reference mode tests.

Session 18 does not introduce a new collector stage or schema branch. It documents and validates the behavior already shipped in Sessions 01-16, records Session 17 as deferred by Session 16, and keeps generated artifacts out of committed release notes. If repo-wide formatting drift exists outside changed files, record it as unrelated drift and still prove formatting for touched files.

## Browser Payload Growth Brief

Trend Finder is currently stretching a single generated browser contract across several product jobs. The Trends, Hidden Gems, Watchlist, Brief, Sources, Workbench, Engine Replay, source setup, scheduler status, spend, enrichment, assets, and calibration surfaces all read from `LiveData.extensions.items["trend-finder"]`. That keeps the local app simple: one scoped scheduler run writes one branch, the dev server exposes one generated payload, and every tab renders from the same validated object.

The tradeoff is that payload growth is cumulative. Each new browser-visible capability tends to add bounded fields rather than replace older ones: evidence rows carry source-local labels, asset summaries, quality components, competitor matches, metrics, and topic IDs; topics carry scoring, movement, lifecycle, action, convergence, and creator-angle projections; prediction and retro calibration publishes recent rows plus a story log; Engine Replay adds sanitized decision records; setup and scheduler panels add safe local-control summaries. None of those branches is individually raw-private, but together they create a large all-tabs payload even when a user is looking at only one surface.

The present cap increase to `9_999_999` bytes is a tactical development punt so current local runs can publish while the shape of the product is still moving. It should not be treated as the long-term architecture. A future pass should split the monolith into a small always-loaded summary plus route-scoped payloads or local dev endpoints for heavyweight detail views, with explicit retention budgets for evidence rows, story logs, Engine Replay records, and static Brief support data. That keeps privacy validation and schema safety intact while making payload size an owned design boundary rather than an incidental failure mode.

## Scheduled Refresh

Trend Finder can refresh through its own scheduler job without forcing the full AI OS aggregate path:

```bash
bun run scheduler:trend-finder:run
bun run scheduler:trend-finder:status
```

The scoped scheduler job writes only `LiveData.extensions.items["trend-finder"]` through the generated-data merge writer. Host/local live-data branches and other extension items are preserved.

Trend Finder still follows local extension enablement. Set `VITE_CLAUDE_OS_ENABLED_EXTENSIONS=trend-finder` before running the scoped refresh. If the extension is not enabled, the scheduler records a skipped run without rewriting existing Trend Finder data.

The browser can now read a sanitized scheduler status projection for this scoped job. The status includes reviewed cadence choices, the selected cadence, timer intent, latest scoped run state, warning counts, generated-data freshness, command hints, first-run state, and safe diagnostics. It does not include raw run logs, private scheduler paths, stdout/stderr, tokens, prompts, provider responses, Actor inputs, Dataset rows, or credential values.

Scheduler status also publishes an additive `preRunEstimate` projection before an operator starts a scoped run. It combines the latest browser-safe spend summary, the selected reviewed cadence, source-count confidence, and the latest successful or degraded scheduler duration when available. The estimate labels exact, estimated, mixed, unavailable, not-applicable, and cadence-unavailable states explicitly. It is transparency for planning only; it does not introduce or imply an enforced spend cap.

First-run is an operator state, not a data error. It means the scoped scheduler has enough context to show a setup checklist before a successful Trend Finder payload exists. The UI may show enablement, credential, reviewed source, runtime, run-now, and Engine Replay review checks, but it still cannot inspect private scheduler files or source config paths.

Cadence and timer intent can be changed only through reviewed Trend Finder cadence IDs. The local bridge rejects arbitrary jobs, raw calendar expressions, unsupported fields, malformed JSON, oversized requests, and invalid cadence IDs. Writes are local, atomic, and preserve unrelated scheduler config entries. Operator-managed system timers remain command-line work; the browser records intent and reviewed local config only.

## Run Control And Local State

The Trends tab run button is a local dev-server control, not a hosted job system.

Run control does this:

1. Save unsaved Creator Lens edits through `/__trend_finder_creator_lens` when the draft is dirty.
2. Request a local run token from `/__token`.
3. POST to `/__run_trend_finder`, which runs `scripts/scheduler-runner.ts --job trend-finder`.
4. Poll `/__run_trend_finder_status` while the local run is active.
5. Refresh `/__live-data` after the scoped Trend Finder scheduler job completes.

The run button is intentionally permissive. Runtime not ready, missing Apify credentials, offline sources, or deterministic fallback do not disable the button. Those states are surfaced as readiness notes, provenance labels, source health rows, warnings, and Engine Replay decisions.

Canceling a run cancels the browser request. It may not stop a Bun aggregate process that already started in the local terminal.

Duplicate run attempts are handled as explicit overlap states. The local run bridge and scheduler lock can return an already-running result, and the browser keeps that state visible instead of starting a second write to generated Trend Finder data.

During an active local run, the browser displays sanitized live progress when available. Progress uses the Engine Replay stage taxonomy, elapsed time, completed stage labels, cancellation/overlap states, and bounded per-source counts. If progress cannot be loaded, the UI falls back to the coarse run lifecycle and latest scheduler status rather than exposing process output.

Trend Finder local state is split across these places:

| Location                                           | Purpose                                                                                                    | Browser Visible                                  |
| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| `src/data/live-data.json`                          | Generated AI OS runtime data, including Trend Finder.                                                      | Yes, via `/__live-data`                          |
| `src/data/live-data.example.json`                  | Safe committed fallback payload for fresh clones.                                                          | Yes, when live data is missing                   |
| `.cache/extensions/trend-finder/creator-lens.json` | Saved Creator Lens used by the next aggregate run.                                                         | Indirectly, after the next run                   |
| `.cache/extensions/trend-finder/enrichment-cache/` | Private safe enrichment cache entries.                                                                     | Only aggregate hit/miss/prune counts             |
| `.cache/extensions/trend-finder/snapshots/`        | Private latest and historical Trend Finder snapshots, daily topic series, and model download observations. | Only bounded summaries and retention counts      |
| `.cache/extensions/trend-finder/predictions/`      | Private prediction archives and nested `retros/` archives.                                                 | Only capped recent rows and retention counts     |
| `.cache/extensions/trend-finder/static-brief/`     | Generated opt-in static Brief report.                                                                      | Yes, as a local standalone report                |
| Browser `localStorage` Creator Lens draft          | Unsaved Creator Lens draft for the UI.                                                                     | Only in the local browser                        |
| Browser `ai-os.trend-finder.visibility.v1` storage | Per-view visibility preferences for UI filtering.                                                          | Only in the local browser                        |
| `data/trend-finder.visibility.json`                | Optional local file overlay for visibility settings.                                                       | Only parsed hidden IDs and warnings are returned |
| `data/ai-os.scheduler.json`                        | Optional reviewed scheduler cadence/timer config.                                                          | Only sanitized Trend Finder fields are returned  |

Generated private cache files and `src/data/live-data.json` must stay gitignored. The committed example payload and fixture/demo data are safe demo surfaces, not proof of live source collection.

## Static Brief Export

Trend Finder can export the current generated Brief as a local standalone HTML report:

```bash
bun run trend-finder:export-brief
```

The command reads `src/data/live-data.json` by default, extracts `LiveData.extensions.items["trend-finder"].data`, privacy-scans the raw Trend Finder data, runs required-derived-field closeout before schema defaults can hide omissions, validates it with the Trend Finder schema, projects it into a smaller browser-safe report contract, and writes `index.html` plus `manifest.json` under `.cache/extensions/trend-finder/static-brief/`. The command also supports:

```bash
bun run trend-finder:export-brief -- --dry-run --json
bun run trend-finder:export-brief -- --output .cache/extensions/trend-finder/static-brief/latest
bun run trend-finder:export-brief -- --archive
```

Without `--archive`, the output layout is unchanged: the selected output directory receives `index.html`, `manifest.json`, and any copied report-local `assets/` files. With `--archive`, the selected output directory becomes an archive root. The exporter writes the newest report under a numbered `run-0001/`, `run-0002/`, ... directory, writes that run's `index.html` and `manifest.json` there, and updates `latest.html` in the archive root to point to the newest run. The latest pointer uses a relative `run-NNNN/index.html` link so the generated directory can move without leaking local absolute paths.

Archive writes use the same validation, static Brief QA, private-string scan, asset-copy checks, and duplicate output lock as default exports. QA and privacy failures happen before either a run directory or `latest.html` is promoted. If `latest.html` promotion fails after the run directory was promoted, the new run directory is removed and any previous run backup is restored so the old latest pointer remains usable.

Archive manifests include archive metadata: the run number, run directory, source run ID, relative archived index path, latest pointer path, latest pointer href, and QA status. JSON CLI summaries include the resolved local archive run path and latest pointer path for operator scripts.

Validation fails before final output is written for missing live data, missing Trend Finder extension data, missing or wrong-shaped required derived fields, invalid Trend Finder payloads, path traversal, unsafe output roots, private field names, token-shaped strings, local home paths, raw logs, prompts, provider responses, Actor or Dataset internals, raw source dumps, account auth, credentials, local triage notes, and private cache paths. Output writes use a temporary directory and final rename so an older report remains intact when validation fails before promotion.

The static report includes safe run metadata, source health, today's pick, movement groups, demand centers, industry events, the security lens, polarity and attention rows, top topics, score/actionability labels, attention, aggregate reception, corroboration, bounded sparklines, angle-pack copy, public evidence links, evidence rationale labels, provenance labels, prediction/retro context, Story Log summaries, spend labels, enrichment counts, and asset fallback labels. It does not read browser localStorage, visibility settings, Workbench triage or watching notes, raw source payloads, raw billing payloads, account IDs, prompts, provider responses, or raw logs.

The static Brief keeps the deliberate no-media-embed boundary. It may link to public evidence URLs and copied report-local assets, but it does not inline social embeds, thumbnails, transcripts, video, audio, iframe content, object embeds, image tags, or remote script tags. QA rejects rendered output that adds those tags.

Evidence assets are optional. The exporter can copy only manifest-approved assets from `.cache/extensions/trend-finder/assets/` into a report-local `assets/` directory. Unsupported, blocked, failed, missing, pruned, bridge-only, or manifest-missing assets render fallback labels. Manifest path escapes or private asset references fail closed.

Hosting remains operator-managed. If a generated report is hosted, serve `index.html` with `Cache-Control: public, max-age=0, must-revalidate` and copied report assets with `Cache-Control: public, max-age=31536000, immutable`. Do not commit generated reports or private cache contents by default.

## Enrichment Cache And Spend Accounting

The enrichment cache lives only below `.cache/extensions/trend-finder/enrichment-cache/`. Entries are keyed by source ID, stable source item ID, adapter version, enrichment type, and a sanitized metadata fingerprint. Cache files store safe summaries only. Raw source payloads, Actor inputs, Dataset rows, billing payloads, account identifiers, tokens, and private paths are not written to the browser payload.

The collector builds cache candidates after cheap evidence collection and source-local normalization. Current evidence IDs form the active keep set. Cache hits are reused, stale fingerprints become misses, and stale cache files outside the keep set are pruned after the merge step. Browser output receives only bounded counts such as eligible, hit, miss, skipped, saved, written, retained, pruned, stale, and error counts.

Retention is tied to the active evidence keep set for the current run. The cache is not a durable operator archive, and Reference mode does not read it. Historical summaries continue to come from the separate private snapshot, prediction, and retro archives after those helpers publish bounded browser-safe rows.

## Snapshot And Prediction Archive Retention

Snapshot, prediction, and retro archives are private local history stores. The collector prunes them after snapshot and prediction/retro archive writes finish. The default retention window is 180 days. `TREND_FINDER_ARCHIVE_RETENTION_DAYS` can override the window, but the value is clamped from 90 to 730 days so it stays above the 84-day historical context window and cannot silently become unbounded.

Retention deletes only dated JSON archive files it can safely classify inside these stores:

| Store       | Directory                                            |
| ----------- | ---------------------------------------------------- |
| snapshots   | `.cache/extensions/trend-finder/snapshots/`          |
| predictions | `.cache/extensions/trend-finder/predictions/`        |
| retros      | `.cache/extensions/trend-finder/predictions/retros/` |

Preserved entries include `latest.json`, active Creator Lens config, backtest archives under `.cache/extensions/trend-finder/backtests/`, subdirectories, non-JSON files, and JSON files whose date cannot be classified from the filename. The helpers assert path containment before every deletion.

Engine Replay receives only aggregate retention fields: store name, retention days, cutoff timestamp, scanned count, preserved count, pruned count, skipped count, warning count, and bounded warning codes. Collector warnings use `trend-cache-retention-degraded` with store/count/code summaries only. Deleted filenames, absolute paths, and raw filesystem errors are not copied into browser payloads.

Spend accounting is also summary-only. Apify and Google Trends demand can publish exact `usageTotalUsd` when the runner reports it. If exact usage is not available, configured `maxTotalChargeUsd` caps are shown as estimates. Public non-paid sources are labeled as not applicable.

Recurring spend projection combines the selected scheduler cadence with the latest per-source spend summary. The browser labels exact, estimated, mixed, unavailable, not-applicable, and cadence-unavailable states explicitly. Cadence projection is an operational estimate for local refresh planning; raw billing payloads, account IDs, Actor internals, Dataset rows, and private paths stay out of browser payloads.

Pre-run estimates reuse the same spend summary and cadence facts but stay additive to retrospective accounting. Missing or zero-source spend data renders as unavailable. A `$0` next-run estimate appears only when all contributing sources are explicitly labeled not applicable to paid runners. Wall-clock projection uses only bounded latest-run duration and otherwise renders as unavailable.

## Evidence Asset Manifest And Bridge

Evidence assets use a private manifest below `.cache/extensions/trend-finder/assets/evidence-assets.json`. The manifest is local generated state and is gitignored with the rest of `.cache/`. Entries are keyed by stable evidence IDs and safe asset IDs. Manifest entries can reference only relative paths below the explicit asset root.

The helper rejects absolute paths, path traversal, null bytes, empty path segments, unsupported content types, malformed IDs, and oversized files. Manifest writes are atomic. The collector builds the active keep set from the current evidence IDs, prunes stale manifest entries and files outside that keep set, and reports retained, pruned, failed, blocked, missing, unsupported, and error counts.

The local bridge is mounted at `/__trend_finder_asset`. It is a dev-only, loopback-only endpoint gated by the per-server `x-claude-os-token` refresh token. It serves only manifest-approved asset IDs from explicit roots, with `Cache-Control: no-store`, `X-Content-Type-Options: nosniff`, same-origin resource policy, content-type allowlisting, and bounded file sizes.

Current source normalizers do not expose unapproved media. YouTube audiovisual content, cached thumbnails, transcripts, RSS/news media enclosures, Reddit profile/avatar/comment fields, raw Apify fields, and arbitrary hotlinked URLs remain blocked or unsupported unless a source-specific compliance document later approves a bounded asset type. Static Brief export copies only manifest-approved report-local assets and documents optional hosting cache headers without enabling a deployment workflow.

## Implementation Map

| Area                                  | Primary Files                                                          |
| ------------------------------------- | ---------------------------------------------------------------------- |
| Client registration and tabs          | `src/extensions/trend-finder/client.tsx`                               |
| Browser payload schema                | `src/extensions/trend-finder/schema.ts`                                |
| View-model sorting and display labels | `src/extensions/trend-finder/view-model.ts`                            |
| Engine Replay route                   | `src/routes/extensions.trend-finder.engine.tsx`                        |
| Engine Replay Reference mode          | `src/extensions/trend-finder/views/engine-reference-view.tsx`          |
| In-app Trend Finder manual registry   | `src/extensions/trend-finder/reference-docs.ts`                        |
| Trends tab                            | `src/extensions/trend-finder/views/trends-view.tsx`                    |
| Hidden Gems tab                       | `src/extensions/trend-finder/views/hidden-gems-view.tsx`               |
| Sources tab                           | `src/extensions/trend-finder/views/sources-view.tsx`                   |
| Watchlist tab                         | `src/extensions/trend-finder/views/watchlist-view.tsx`                 |
| Brief tab                             | `src/extensions/trend-finder/views/brief-view.tsx`                     |
| Collector pipeline                    | `scripts/extensions/trend-finder/collector.ts`                         |
| Required derived field closeout       | `scripts/extensions/trend-finder/required-derived-fields.ts`           |
| Static Brief export CLI               | `scripts/trend-finder-export-brief.ts`                                 |
| Static Brief export module            | `scripts/extensions/trend-finder/static-brief-export.ts`               |
| Static Brief renderer                 | `scripts/extensions/trend-finder/static-brief-renderer.ts`             |
| Evidence asset manifest               | `scripts/extensions/trend-finder/evidence-assets.ts`                   |
| Enrichment cache helper               | `scripts/extensions/trend-finder/enrichment-cache.ts`                  |
| Spend accounting helper               | `scripts/extensions/trend-finder/spend-accounting.ts`                  |
| Security relevance lens               | `scripts/lib/ai-runtime/security-lens.ts`                              |
| Hacker News source adapter            | `scripts/extensions/trend-finder/sources/hn-adapter.ts`                |
| Apify source declarations             | `scripts/extensions/trend-finder/sources/apify-source-config.ts`       |
| Apify dataset normalizers             | `scripts/extensions/trend-finder/sources/apify-normalizers.ts`         |
| Source quality and caps               | `scripts/extensions/trend-finder/sources/source-quality.ts`            |
| Source compliance gate                | `scripts/extensions/trend-finder/sources/source-compliance.ts`         |
| Score math                            | `scripts/lib/ai-runtime/scoring.ts`                                    |
| Velocity dynamics                     | `scripts/lib/ai-runtime/velocity-dynamics.ts`                          |
| Risk flags                            | `scripts/lib/ai-runtime/risk-flags.ts`                                 |
| Lifecycle stages                      | `scripts/lib/ai-runtime/lifecycle.ts`                                  |
| Cross-source convergence              | `scripts/lib/ai-runtime/convergence.ts`                                |
| AI analyst prompt and validation      | `scripts/lib/ai-runtime/trend-analyst.ts`                              |
| Topic identity                        | `scripts/lib/ai-runtime/topic-identity.ts`                             |
| Daily series and download baselines   | `scripts/lib/ai-runtime/trend-series.ts`                               |
| Historical context                    | `scripts/lib/ai-runtime/historical-context.ts`                         |
| Watchlist rules                       | `scripts/lib/ai-runtime/watchlist.ts`                                  |
| Movement analysis                     | `scripts/lib/ai-runtime/movement-analyst.ts`                           |
| Prediction writer                     | `scripts/lib/ai-runtime/predictions.ts`                                |
| Retro evaluator                       | `scripts/lib/ai-runtime/retros.ts`                                     |
| Snapshot cache                        | `scripts/lib/ai-runtime/snapshots.ts`                                  |
| Competitor matching                   | `scripts/extensions/trend-finder/competitor-matching.ts`               |
| Demand clusters                       | `scripts/extensions/trend-finder/demand-clusters.ts`                   |
| Industry events                       | `scripts/extensions/trend-finder/industry-events.ts`                   |
| Theme rollups                         | `scripts/extensions/trend-finder/theme-rollups.ts`                     |
| Outlier ideas                         | `scripts/extensions/trend-finder/outlier-ideas.ts`                     |
| Creator Lens browser draft            | `src/extensions/trend-finder/creator-lens-storage.ts`                  |
| Creator Lens cache config             | `scripts/lib/ai-runtime/creator-lens-config.ts`                        |
| Creator Lens local bridge             | `scripts/lib/trend-finder-lens-bridge.ts`                              |
| Evidence asset local bridge           | `scripts/lib/trend-finder-asset-bridge.ts`                             |
| Scheduler status contract             | `src/extensions/trend-finder/scheduler-status.ts`                      |
| Scheduler status hook                 | `src/extensions/trend-finder/use-scheduler-status.ts`                  |
| Scheduler status helper               | `scripts/lib/trend-finder-scheduler-status.ts`                         |
| Scheduler status local bridge         | `scripts/lib/trend-finder-scheduler-status-bridge.ts`                  |
| First-run scheduler panel             | `src/extensions/trend-finder/components/scheduler-first-run-panel.tsx` |
| Live run progress component           | `src/extensions/trend-finder/components/live-run-progress.tsx`         |
| Visibility settings                   | `src/extensions/trend-finder/visibility-config.ts`                     |
| Visibility local bridge               | `scripts/lib/trend-finder-visibility-config-bridge.ts`                 |
| Browser run control                   | `src/extensions/trend-finder/hooks/use-trend-finder-run.ts`            |
| Engine Replay model                   | `src/extensions/trend-finder/engine-replay-model.ts`                   |
| Convergence timeline                  | `src/extensions/trend-finder/components/convergence-timeline.tsx`      |
| Score trajectory                      | `src/extensions/trend-finder/components/score-trajectory.tsx`          |
| Topic sparkline                       | `src/extensions/trend-finder/components/trend-sparkline.tsx`           |
| Story Log panel                       | `src/extensions/trend-finder/components/story-log-panel.tsx`           |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/extensions/trend-finder/pipeline.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.