> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_27/prd_phase_27.md). # PRD Phase 27: Trend Finder Alpha Radar Adoption **Status**: Complete **Sessions**: 12 (initial estimate) **Estimated Duration**: 12-16 days **Progress**: 12/12 sessions (100%) *** ## Overview Phase 27 adopts the objective improvements mapped from the AI Alpha Radar hackathon submission into the Trend Finder extension. The mapping was captured on 2026-06-12 from `docs/ongoing-projects/alpha-radar.md`; this phase PRD and its session stubs fold in the complete mapping so that file can be deleted without losing details. AI Alpha Radar is a judge-facing static product demo. Its backend pipeline is **not** included in its repo, so the mapping is grounded in the artifacts below: | Artifact | Path | What It Proves | | ----------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | | Generated data contract | `/home/aiwithapex/projects/aios/EXAMPLES/ai-alpha-radar-submission/public/data.json` | The full per-trend, prediction, demand, and outlier field model. | | Dashboard behavior | `/home/aiwithapex/projects/aios/EXAMPLES/ai-alpha-radar-submission/public/dashboard.html` | Surfaces, interactions, and render functions (single-file app). | | Landing page | `/home/aiwithapex/projects/aios/EXAMPLES/ai-alpha-radar-submission/public/index.html` | Marketing surface only; not relevant to extension behavior. | | Backend notes | `/home/aiwithapex/projects/aios/EXAMPLES/ai-alpha-radar-submission/technical-appendix/data-flow.md` | Pipeline stages: fetch -> normalize -> cluster -> score -> enrich. | | Scoring notes | `/home/aiwithapex/projects/aios/EXAMPLES/ai-alpha-radar-submission/technical-appendix/scoring-summary.md` | Ranking signal definitions and frontend contract. | | Project overview | `/home/aiwithapex/projects/aios/EXAMPLES/ai-alpha-radar-submission/README.md` | Product framing and section list. | Trend Finder's current behavior is documented in `docs/extensions/trend-finder-*.md` and implemented under `src/extensions/trend-finder/`, `scripts/extensions/trend-finder/`, and `scripts/lib/ai-runtime/`. All paths cited in this phase were verified against the working tree on 2026-06-12. "Objective improvement" means: a capability Alpha Radar demonstrably ships (visible in `data.json` or `dashboard.html`) that Trend Finder lacks or implements more weakly, and that can be adopted without breaking Trend Finder invariants. ### Trend Finder Invariants (Every Session Must Respect) * Browser-safe payload boundary (`docs/extensions/trend-finder-pipeline.md`): no raw prompts, provider responses, logs, billing, or private paths. * Deterministic fallback for every AI stage (`docs/extensions/trend-finder-runtime-and-provenance.md`). * Compliance-gated sources: a new adapter requires `docs/sources/source-compliance-.md` first (`docs/extensions/README_docs-extensions.md`). * Validators reject invented metrics, dates, URLs, and sources (`scripts/lib/ai-runtime/trend-analyst.ts`). Alpha Radar invents estimates in places (e.g. `demand_clusters[].sources: ["inferred"]`, `askers_estimate`); anything ported from those fields must carry explicit `estimated`/`derived` provenance labels instead. ### Calibration Caveat From The Source Alpha Radar's `hit_rate.rate` of `1.0` with `0 verified / 45 verified_early` shows its calibration loop is younger than Trend Finder's retro evaluator. The improvement being adopted is its presentation and metric set, not its rigor. ### Where Trend Finder Is Already Ahead (Do Not Regress) Several Alpha Radar fields would be downgrades; sessions must not regress these while porting: | Area | Trend Finder advantage | | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Baseline math | Median peer-entity baselines with placement exclusion and explicit low-sample states (`scripts/extensions/trend-finder/source-local-signals.ts`) vs. Alpha Radar's single channel-baseline divisor. | | Provenance | Explicit live/fixture/degraded/blocked/fallback labels everywhere; Alpha Radar has a single `data_freshness_status`. | | Validation | Analyst output validators reject invented IDs/URLs/dates/metrics (`scripts/lib/ai-runtime/trend-analyst.ts`); Alpha Radar ships inferred estimates. | | Compliance | Reviewed source declarations, compliance gate, PII boundaries (`scripts/extensions/trend-finder/sources/source-compliance.ts`, `docs/sources/`). | | Cost visibility | Per-source spend accounting and recurring projection (`scripts/extensions/trend-finder/spend-accounting.ts`); absent in Alpha Radar. | | Run proof | Engine Replay sanitized trace + Reference mode (`src/extensions/trend-finder/views/engine-replay-view.tsx`); Alpha Radar has a static methodology modal. | | Retro rigor | hit/partial/miss/pending with stale handling and backtest/future-run modes (`scripts/lib/ai-runtime/retros.ts`); Alpha Radar's loop shows a 100% rate with zero fully-verified calls. | ### Explicit Non-Goals (Not Worth Porting) * **`tbts` / `peak_estimate_days` point estimates** -- invented precision; Alpha Radar itself ships nulls for `peak_estimate_days`. Conflicts with the no-invented-metrics rule. * **X/Twitter source** -- no compliant public collection path under current terms. * **Hotlinked thumbnails** -- Alpha Radar hotlinks `i.ytimg.com` thumbnails; Trend Finder media policy blocks YouTube thumbnails until a compliance doc approves a bounded asset type. The asset manifest/bridge path (`scripts/extensions/trend-finder/evidence-assets.ts`, `scripts/lib/trend-finder-asset-bridge.ts`) already exists for when it does. * **Landing-page media** (`bg-pingpong.mp4`, `cloud-light.mp3`) -- marketing asset choices for a hackathon demo, not extension behavior. * **Single-creator hardcoding** (the Jack Roberts profile) -- Trend Finder's Creator Lens is already the generalized version; Session 09 ports the competitor list field, not the hardcoded profile. ### Deferred: Source Coverage Candidates (Compliance-First Path) Alpha Radar's per-trend `sources` map names feeds Trend Finder does not collect. Every addition goes through the compliance-first path (`docs/sources/apify-source-onboarding.md` plus a new `docs/sources/source-compliance-.md`), so these stay out of Phase 27 sessions (except Hugging Face download deltas, which need no new source and land in Session 04). They are recorded here as candidates for a future phase: | Alpha Radar signal | Trend Finder status | Candidate adapter notes | | ------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | `semantic_scholar_citations_7d` | Not collected; arXiv covers research role | Public API with documented rate limits; would strengthen the research role with citation velocity. | | `bluesky_mentions_7d` | Not collected | Public AppView API; discussion role; needs PII review for author fields like the Reddit docs. | | `replicate_runs_7d_delta` | Not collected; Hugging Face models covers developer role | Public explore/model pages; run-count deltas need snapshot persistence (pairs with Session 04). | | `newsletters` (`newsletter_signals`) | `rss-ai-news` covers feed metadata | Mostly covered; specific newsletter feeds could be added as reviewed `rss-ai-news` targets via Source Setup. | | `x_posts_7d` | Not collected | No compliant public path under current X API terms; explicit non-goal. | | `digg_ai_mentions_7d` | Not collected | Low marginal value; skip unless a creator need appears. | | `hf_downloads` deltas (`meta.hf_downloads`) | `huggingface-ai-models` collects metrics, no run-over-run delta | No new source needed -- persist per-model download counts across runs and emit deltas (Session 04). | ### Tier Sequencing (From The Mapping) | Tier | Sessions | Rationale | | --------------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------- | | 1 -- payload exists, mostly UI/derive | 01-03 | Deterministic, no new sources, no new privacy surface. | | 2 -- needs snapshot/history persistence | 04-08 | Built on per-run persistence; sequence with the SQLite plan in `docs/ongoing-projects/sqlite-observation-store-transition-plan.md`. | | 3 -- new analysis or compliance work | 09-11 | Each needs analyst-prompt changes, an enrichment stage, or a dependency decision. New source adapters stay deferred (table above). | | Closeout | 12 | Docs, validation, security review, release, and safe deletion of `docs/ongoing-projects/alpha-radar.md`. | Within Tier 1, Session 01 items (movement-grouped Brief, hit-rate/Brier) are the highest leverage: both make existing collector output visible (movement records and retro archives are already generated every run and currently underexposed). *** ## Progress Tracker | Session | Name | Status | Est. Tasks | Validated | | ------- | --------------------------------------------- | -------- | ---------- | ---------- | | 01 | Brief Movement Groups And Calibration Metrics | Complete | 20 | 2026-06-12 | | 02 | Deterministic Derived Signals And Risk Flags | Complete | 20 | 2026-06-12 | | 03 | Data-Driven Radar Aliases And Watching State | Complete | 20 | 2026-06-12 | | 04 | Daily Time-Series Persistence And Sparklines | Complete | 20 | 2026-06-12 | | 05 | Velocity Dynamics Upgrade | Complete | 20 | 2026-06-13 | | 06 | Lifecycle Stage Taxonomy | Complete | 20 | 2026-06-13 | | 07 | Convergence Detection And Trajectory Visuals | Complete | 21 | 2026-06-13 | | 08 | Dated Predictions And Story Log | Complete | 22 | 2026-06-13 | | 09 | Competitor Lens And Creator Angle Pack | Complete | 24 | 2026-06-13 | | 10 | Demand Centers | Complete | 24 | 2026-06-13 | | 11 | Theme Rollups And Outlier Ideas | Complete | 24 | 2026-06-13 | | 12 | Documentation Validation And Release | Complete | \~12-25 | 2026-06-13 | *** ## Completed Sessions * Session 01: Brief Movement Groups And Calibration Metrics - Completed 2026-06-12. Added movement-grouped Brief sections, today's pick, confidence bands, and aggregate prediction calibration metrics to live and Static Brief surfaces. * Session 02: Deterministic Derived Signals And Risk Flags - Completed 2026-06-12. Added deterministic saturation, hidden-gem score, healthy-source consensus, role-share/builder signal, risk flags, card/source summaries, Hidden Gems ordering, and Signal Workbench filters/sorts with explicit unavailable states. * Session 03: Data-Driven Radar Aliases And Watching State - Completed 2026-06-12. Added data-driven topic radar projection, bounded alias chips, browser-local watching state, watched filters for Trends and Hidden Gems, and a cross-source outlier preset over existing source-local lift data. * Session 04: Daily Time-Series Persistence And Sparklines - Completed 2026-06-12. Added private daily topic series and Hugging Face model download observation persistence, bounded 14-day browser sparklines, download delta states, card and Workbench rendering, and focused/full validation coverage. * Session 05: Velocity Dynamics Upgrade - Completed 2026-06-13. Added deterministic acceleration, statistical significance, and burst dynamics over historical evidence counts and 14-day sparklines, with bounded browser schema fields, score breakdown rows, Engine Replay proof updates, sanitized trace counters, and focused/full validation coverage. * Session 06: Lifecycle Stage Taxonomy - Completed 2026-06-13. Added deterministic lifecycle classification with `unknown`, `whisper`, `builder`, `creator`, and `saturated` stages, additive browser schema defaults, trend-card chips, Workbench filtering/sorting, radar stage colors, sanitized trace counters, and focused/full validation coverage. * Session 07: Convergence Detection And Trajectory Visuals - Completed 2026-06-13. Added deterministic cross-source convergence over linked evidence timestamps, bounded score trajectory payloads, selected-topic lead-lag and actual-score visuals, additive schema defaults, sanitized trace counters, and focused/full validation coverage. * Session 08: Dated Predictions And Story Log - Completed 2026-06-13. Added dated lifecycle-targeted predictions, target-date-aware retro grading, bounded browser-safe Story Log projection, Watchlist verdict/due-state filters, Brief prediction-history summary, sanitized Engine Replay counters, and focused/full validation coverage. * Session 09: Competitor Lens And Creator Angle Pack - Completed 2026-06-13. Added bounded Creator Lens competitor lists, fail-closed saved-config and bridge validation, collect-time YouTube channel-title matching, competitor evidence chips, analyst angle-pack fields, deterministic fallback copy, Trend Card and Brief rendering, static Brief projection, and focused/full validation coverage. * Session 10: Demand Centers - Completed 2026-06-13. Added deterministic question-shaped discussion evidence extraction, bounded cross-topic demand clusters, observed count and derived growth labels, analyst demand brief validation with deterministic fallback copy, dashboard and static Brief Demand Centers rendering, related-topic jump-links, and focused/full validation coverage. * Session 11: Theme Rollups And Outlier Ideas - Completed 2026-06-13. Added deterministic topic theme rollups, validated analyst theme labels, grouped and flat Signal Workbench modes, top-N source-local outlier creator ideas, enrichment-cache reuse, bounded browser schema fields, and the embedding fallback clustering ADR with focused/full validation coverage. * Session 12: Documentation Validation And Release - Completed 2026-06-13. Updated the committed Trend Finder manuals and Reference mode registry, preserved deferred source candidates and non-goals in durable docs, completed full validation and security closeout, deleted the migrated Alpha Radar planning note, and released Phase 27 with full test coverage. *** ## Upcoming Sessions * None. Phase 27 is complete. Session 01 validation passed on 2026-06-12. Session 02 validation passed on 2026-06-12. Session 03 validation passed on 2026-06-12. Session 04 validation passed on 2026-06-12. Session 05 validation passed on 2026-06-13. Session 06 validation passed on 2026-06-13. Session 07 validation passed on 2026-06-13. Session 08 validation passed on 2026-06-13. Session 09 validation passed on 2026-06-13. Session 10 validation passed on 2026-06-13. Session 11 validation passed on 2026-06-13. Session 12 validation passed on 2026-06-13. The phase is complete, and the next workflow step is `audit`. *** ## Objectives 1. Expose already-generated movement and retro data: movement-grouped Brief ("what moved / what cooled / what is emerging"), an explicit today's pick, banded confidence labels, and aggregate hit-rate plus Brier calibration metrics. 2. Add deterministic derived signals with zero AI cost: explicit saturation, continuous hidden-gem score, consensus ratio, builder/role-share signal, and per-topic risk flags. 3. Make the Signal Radar positions data-driven, render existing topic aliases, add a browser-local watching/pin state, and ship a cross-source outlier ranking preset over the existing source-local lift math. 4. Persist per-topic daily evidence counts and per-model Hugging Face download counts across runs, publishing bounded 14-day sparklines and download deltas. 5. Upgrade velocity math with acceleration, statistical significance, and a bounded burst signal, each with explicit unavailable states below minimum sample sizes. 6. Add a deterministic lifecycle stage taxonomy (absolute maturity labels) wired into cards, Workbench, and the radar, unlocking lifecycle-targeted predictions. 7. Detect time-windowed cross-source convergence with per-source first-seen timestamps and render lead-lag and trajectory visuals. 8. Extend the prediction loop with dated, lifecycle-targeted predictions and a Story Log prediction-history surface with verdict filtering. 9. Extend the Creator Lens with a public competitor/adjacent-channel list and the analyst output with contrarian, plain-language, and ready-title angle fields, plus per-outlier creator ideas. 10. Aggregate question-shaped evidence into demand clusters with strictly labeled estimates, and add analyst theme labels with deterministic fallback grouping. 11. Document, validate, security-review, and release the upgraded surfaces, then delete `docs/ongoing-projects/alpha-radar.md` once coverage is confirmed. *** ## Prerequisites * Phase 26 completed (Knowledge Graph Shared Brain Port). * `EXAMPLES/ai-alpha-radar-submission/` present locally for reference during implementation (read-only source material). * No open security findings (per `.spec_system/SECURITY-COMPLIANCE.md`). *** ## Technical Considerations ### Architecture * All collector-side math stays under `scripts/lib/ai-runtime/` and `scripts/extensions/trend-finder/`; browser surfaces consume validated payload fields through `src/extensions/trend-finder/schema.ts` only. * New schema fields are additive with Zod `.default()` values so legacy payloads, fixtures, and the committed example payload keep parsing. * Browser-local state (watching/pin) extends the existing Signal Workbench triage storage pattern; the generated Watchlist stays read-only. * Persistence work (Sessions 04-08) builds on the private snapshot archives under `.cache/extensions/trend-finder/`; if the SQLite plan in `docs/ongoing-projects/sqlite-observation-store-transition-plan.md` lands first, daily series become SQLite rollups instead of snapshot-file derivations. ### Technologies * Zod schema validation (additive defaults). * Bun scripts for collector/runtime work; Vitest for script and UI tests; Playwright for browser flows. * Existing AI runtime provider abstraction for analyst prompt extensions; all new AI fields need deterministic fallbacks. * Inline SVG for sparklines, trajectory, and convergence timeline visuals (Alpha Radar proves these need no charting library). ### Risks * **Payload growth**: new arrays (sparklines, story log, demand clusters) could press the shared 1 MB extension payload limit. Mitigation: bound every new array in the schema (14 sparkline entries, \~50 story log rows, capped demand clusters) and verify payload size in validation. * **Snapshot sample size**: acceleration/significance need >= 3 snapshots. Mitigation: explicit `unavailable` states below thresholds, matching the source-local low-sample pattern. * **Analyst prompt creep**: angle pack, themes, demand briefs, and outlier ideas expand analyst output. Mitigation: extend the validator for each new field, keep one repair retry, and ship deterministic fallbacks first. * **Enrichment spend**: per-outlier ideas add AI calls. Mitigation: top-N cap and reuse of `scripts/extensions/trend-finder/enrichment-cache.ts` so unchanged outliers do not re-bill. * **Invented-estimate leakage**: demand metrics imitate Alpha Radar's inferred numbers. Mitigation: publish observed counts plus `estimated`-labeled values only, copying the spend-accounting labeling pattern. ### Relevant Considerations * \[P02] **Extension payloads and demo labels stay bounded**: every new field is bounded and additive; explicit live/fallback/degraded states stay intact. * \[P02] **New source adapters need per-source compliance review**: this is why the source coverage candidates table stays deferred out of Phase 27. * \[P15] **Aggregate collection must stay budgeted**: new analyst enrichment stages must respect runtime budgets and degrade to fallbacks. * \[P24] **Browser-safe export and triage boundaries**: watching state and Story Log must keep local triage and private archives out of exports. * \[P24] **Fail-closed bridges need boundary validation**: the Creator Lens bridge change in Session 09 keeps loopback, token, body-size, and schema checks. * \[P24] **Additive defaults keep legacy payloads alive** (lesson): schema defaults preserved old fixtures while phase 24 shipped; repeat that here. * \[P24] **Project before rendering** (lesson): static Brief export gets the new sections through its projection layer, never raw payload reads. * \[P24] **Deterministic ordering helps dense triage** (lesson): new Workbench columns and presets keep stable sort tie-breakers. * \[P23] **Backlog migration into phase PRD** (lesson): this phase migrates `docs/ongoing-projects/alpha-radar.md` with source anchors and explicit no-action decisions so the file can be deleted without losing audit history. * \[P05] **Script-only runtime boundary** (lesson): all new math stays under `scripts/lib/` / `scripts/extensions/`; browser code reads validated payload fields only. *** ## Success Criteria Phase complete when: * [x] All 12 sessions completed and validated * [x] Brief groups topics by movement and shows hit-rate/Brier calibration * [x] Saturation, gem score, consensus ratio, builder signal, and risk flags render with explicit unavailable states * [x] Signal Radar positions encode momentum/saturation instead of decoration * [x] Sparklines, acceleration, significance, and burst derive from persisted daily counts with low-sample guards * [x] Lifecycle stages, convergence records, dated predictions, and the Story Log ship with deterministic derivations and provenance labels * [x] Competitor lens, angle pack, demand clusters, themes, and outlier ideas ship with deterministic fallbacks and validator coverage * [x] No regression in the "already ahead" areas (baseline math, provenance, validation, compliance, spend, Engine Replay, retro rigor) * [x] `docs/extensions/trend-finder-*.md` manuals updated; full suite, e2e, and security review pass * [x] `docs/ongoing-projects/alpha-radar.md` deleted after coverage check *** ## Dependencies ### Depends On * Phase 26: Knowledge Graph Shared Brain Port (current baseline) * Phase 24: Trend Finder Outlier Signal Upgrade (source-local signals, Workbench, static Brief export that this phase extends) * Phase 13: Trend Finder Self-Evaluation Loop (identity, movement, prediction, retro machinery this phase builds on) ### Enables * A future source-expansion phase (Semantic Scholar, Bluesky, Replicate adapters from the deferred candidates table) * The SQLite observation store transition (`docs/ongoing-projects/sqlite-observation-store-transition-plan.md`), which can replace Session 04 snapshot derivations with durable rollups --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_27/prd_phase_27.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.