> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/hackathon/brainstorm-hackathon.md). # Trend Finder Hackathon Brainstorm Date: 2026-05-17 > **Historical snapshot:** This brainstorm predates the final Phase 06 finish path. Creator Lens persistence, run controls, watchlist/movement artifacts, source/evidence UI, demo provenance labels, and the dashboard Brief view are now implemented. Use `docs/hackathon/hackathon-submission.md`, `docs/hackathon/trend-finder-demo.md`, `docs/apify.md`, `docs/ai-runtime-setup.md`, and `README.md` for current behavior. > > **Current audit note, 2026-05-26:** Treat the brainstorm and Phase 05 review below as planning history unless a section explicitly says it is the current implementation. The current demo path is dashboard-first, local-only, and label-driven. ## Hackathon Focus The hackathon is asking for a working dashboard that finds trending AI topics before they become mainstream. The strongest submission is not a generic app foundation or broad productivity dashboard. It is a repeatable trend-discovery system that turns public internet signals into creator-ready opportunities. The core product story: > Trend Finder uses Apify to collect public AI-topic signals, then uses a permissioned ChatGPT/Codex analyst runtime to rank emerging topics by momentum and evidence quality and turn each trend into a creator-ready opportunity with links, score, explanation, and content angles. ## Product Direction Trend Finder should answer: > What is becoming important for my audience before everyone else covers it? This is sharper than: > What is popular in AI? The top of the system should be a lightweight, editable niche or creator profile. Trending only matters relative to the user's audience, content format, goals, and tolerance for early experimental topics. For the hackathon MVP, the default profile should be: > AI vibe-coders, creators and automators looking for early topics they can turn into tutorials, workflows, newsletters, product ideas, or experiments. The MVP should expose this as a small dashboard control, not a full profile system. A user should be able to adjust: * Audience * Content format * Topic focus * Goal * Risk level Those settings should influence niche fit, scoring labels, and creator angles. They do not need accounts, onboarding, saved profile management, or long-term preference learning for the hackathon version. A later version can support richer creator lenses, such as: * AI Automator * Developer / Builder * Creator / Newsletter * Product Researcher ## System Shape Trend Finder should be built as the first vertical app on a permissioned AI runtime. The dashboard is the product surface, but the deeper asset is the runtime that can coordinate approved extensions, produce typed artifacts, and turn noisy public signals into evidence-backed opportunities. The hackathon version should use only the runtime primitives needed for Trend Finder. The broader direction is an AI OS/platform, but developed vertical-first through this concrete workflow. Top-to-bottom platform shape: ``` User Intent / Product App -> Creator Lens -> Agent Workflow -> Permissioned Extension Runtime -> Apify Source Platform -> Apify Actors / Datasets -> Trend Finder Evidence Normalizers -> ChatGPT/Codex AI Analysis Layer -> Scoring / Ranking Layer -> Product Output -> Feedback / Memory ``` Trend Finder workflow: ``` User opens dashboard -> edits Creator Lens -> starts Trend Finder run -> runtime runs approved Apify source actors -> Apify stores structured results in datasets -> Trend Finder normalizers convert dataset items into evidence -> AI clusters evidence into trend topics -> scoring layer ranks topics -> AI explains why each trend matters -> dashboard shows ranked opportunities -> user feedback improves future runs ``` Artifact flow: ``` Apify Actors -> Dataset Items -> Normalized Signals -> Topic Clusters -> Trend Scores -> Creator Opportunities -> Dashboard ``` Key architectural principle: > The runtime does not produce UI directly. It produces trusted, typed artifacts. Apps render those artifacts into useful workflows. Useful artifact types: * EvidenceItem * SourceRun * TrendCluster * TrendScore * CreatorAngle * Snapshot * FeedbackSignal ## Runtime Scope The runtime should be built at the highest useful abstraction, then restricted through explicit permissions and narrow workflow steps. The target is not a general autonomous OS for the hackathon. The target is a constrained AI analyst runtime that can become the foundation for a broader OS later. Minimum useful runtime capabilities: * register approved Apify source configurations * declare extension capabilities and permissions * run configured Apify Actors and read Dataset items * run a repeatable agent workflow * call AI only at defined analysis steps * require structured JSON artifacts * preserve evidence links for every generated insight * log enough trace data to explain why a trend was ranked Example collection capability shape: ``` extension: trend-finder capabilities: - runApifyActors - readApifyDatasets - normalizeEvidence - analyzeTrends permissions: - network:apify - apify:runActor - apify:readDataset - ai:analyzeEvidence - write:evidence - read:snapshots ``` The ChatGPT/Codex-account auth and transport layer is the primary local AI runtime path. Apify is the primary data collection platform. Trend Finder owns normalization, scoring, snapshots, and dashboard artifacts. ## Workstreams ### 1. Choosing Sources Apify is the finalized collection path. Sources should be chosen based on what signal they reveal, then implemented through configured Apify Actors and Dataset normalizers. Selected source roles: * GitHub: builder and developer momentum. * Hacker News / Reddit: discussion momentum and emerging debates. * arXiv / papers: technical novelty. * Product Hunt / launch directories: productization and market movement. * YouTube / newsletters / blogs: creator and content momentum. The MVP should target 5-7 source roles because Apify centralizes collection through Actors, Datasets, API, and MCP. The constraint is not "how many sources can we scrape?" The constraint is "which sources add distinct signal?" Apify collects. Trend Finder interprets. Actor output should become normalized evidence and then pass through the same AI analyst, deterministic scoring, snapshot, and dashboard pipeline. Important source rules: * Do not treat every source equally. * Weight sources by evidence quality and source role. * Reward cross-source confirmation. * Allow unusually strong single-source signals when the source is high quality. * Avoid turning the dashboard into an undifferentiated firehose. ### 2. Gathering Data The ingestion layer should preserve evidence, not just summaries. Apify Actor runs and Datasets provide the acquisition/provenance layer; Trend Finder normalizers turn the dataset items into consistent evidence. Useful fields: * title * URL * source * source role * timestamp * engagement metrics * tags or keywords * excerpt or summary, where available * Apify actor ID and dataset/run ID for debugging The dashboard is credible only if each trend points back to real source evidence. ### 3. Transferring Data To Value Raw posts are not the product. The value is the interpretation. Trend Finder should use AI for judgment-heavy analysis: * group similar posts into topics * name trend topics clearly * filter for real AI relevance * detect acceleration * identify whether a topic is already mainstream or still early * explain why the trend matters now * estimate niche fit for the creator lens * generate creator angles and hooks Deterministic code should still handle measurable scoring inputs: * recency * volume * engagement * velocity * source diversity * prior snapshot movement This is where the project becomes useful instead of being a feed reader. AI should synthesize and judge, but every conclusion should point back to evidence links and source metrics. ### 4. Delivering The Value The output should be creator-first, not data-first. A user should be able to make a content decision in a few minutes. Each trend should show: * topic name * trend score * hidden-gem or early-signal label * why it is trending * evidence links * source breakdown * creator angle * suggested hook or title * watchlist status Suggested dashboard sections: * Top Trending AI Topics * Hidden Gems * Evidence * Trend Score * Creator Angle * Source Breakdown * Watchlist ### 5. Self-Improvement Over Time The hackathon version should keep this simple but visible. Minimum viable version: * save each run as a dated snapshot * compare the current run against previous runs * track score changes * label topics as new, rising, cooling, or watchlist Later version: * collect user feedback, such as useful / not useful * learn preferred topics and formats * adjust scoring weights based on feedback * track which early predictions later became mainstream * build a historical trend archive ## Scoring Direction A useful score should combine trend signal with niche fit. Scoring model: ``` Opportunity Score = Momentum + Novelty / Under-the-radar Status + Evidence Strength + Source Diversity + Niche Fit + Creator Potential ``` Important distinction: * A general trend score says what is moving. * A creator opportunity score says what is worth making content about. ## Hackathon MVP The MVP scope: * one strong default niche profile * a lightweight editable creator lens at the top of the dashboard * Apify-backed collection for 5-7 public source roles * a constrained ChatGPT/Codex AI analyst runtime underneath the dashboard * permissioned source configuration rather than hardcoded one-off scrapers * repeatable run workflow * normalized local data * typed trend artifacts for the dashboard to render * explainable scoring model * polished dashboard using the existing app surface * evidence links for each trend * dated snapshots for movement over time * a clean 60-second Loom story ## Historical Phase 05 Implementation Review With Current Audit Notes Date reviewed: 2026-05-17 Phase 05 is marked complete in the project records as the AI Runtime Platform. It delivered the local runtime/dashboard path that this brainstorm called for, but several product-facing pieces still need a final hackathon pass before the demo should claim the full MVP story. ### Current Bottom Line The project is no longer just an inherited dashboard shell. It now has a real Trend Finder runtime path: ``` configured sources / HN adapter -> normalized evidence -> optional Codex-account AI analyst -> deterministic scoring and movement snapshots -> LiveData.extensions.items["trend-finder"] -> dashboard views ``` The implementation is strongest as a local, permissioned, dashboard-centered MVP. It should not be described as a hosted platform, dynamic marketplace, general AI OS, generated HTML report tool, or fully live 5-7 source product yet. ### Goal-By-Goal Reconciliation This is the tightest reading of the original brainstorm against the current implementation: | Brainstorm goal | Current state | Nuance to preserve | | -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Editable Creator Lens that changes niche fit, scoring labels, and creator angles | Implemented as editable browser draft fields with save, reset, risk level, localStorage migration, and token-gated cache save. | Edits affect the next saved-and-run aggregate; they do not instantly rescore the already-rendered payload or select sources. | | User starts a Trend Finder run from the dashboard | Implemented through the Trends tab and top-bar run control. It saves dirty Creator Lens edits, requests a local token, and posts to `/__run_trend_finder`. | This is a local dev-server scoped Trend Finder scheduler refresh, not a hosted job system. Canceling can stop the browser request but may not stop a Bun process that already started. | | Apify-backed collection for 5-7 public source roles | Apify helpers, reviewed declarations, compliance gates, and normalizers exist. | Live Apify collection requires source JSON or an inline override plus `APIFY_TOKEN`; declarations are metadata, not active default sources. | | Permissioned source configuration | Collector capabilities, reviewed source IDs, compliance gates, and disabled/blocked source states exist. | Permissioning is declarative and config-gated, not a full user approval workflow or marketplace permission system. | | AI clusters evidence into topics | Codex-account analysis can cluster evidence when runtime is ready and output validates. | Disabled/missing/invalid runtime falls back to deterministic grouping, which should not be described as AI clustering. | | Explainable score | Score breakdown exists and combines the intended dimensions. | Demo copy should call it an opportunity score or explainable score, not a proof of truth. | | Evidence links for each trend | Public evidence links, missing-reference handling, source labels, dates, relevance labels, and metric chips exist. | Metric chips show browser-safe public metrics when source data includes them; missing metrics are not invented. | | Source breakdown | Per-topic source context rows show source name, role, quality, status, warning state, and evidence count. | Global source health still matters; a topic-level row does not prove every configured source ran live. | | Dated snapshots and movement over time | Local snapshots, previous score, velocity, score delta, movement labels, 12-week watchlist history, predictions, and retros exist. | There is still no durable hosted run picker or full historical replay UI; historical backtests remain script-only. | | Watchlist | Generated read-only watchlist rows exist for analyst reasons, hidden gems, rising deltas, creator fit, novelty, or weak-evidence signals. | It is not a manual saved-items feature and has no user add/remove persistence. | | Feedback / memory | Snapshots, topic identity, movement, prediction archives, and retro grading exist in private local cache. | There are still no useful/not-useful controls, preference learning, or adaptive scoring weights. | | Clean 60-second Loom story | Dashboard-centered proof points and deferral guardrails are documented. | The recording must use either matching credentialed-source labels or clearly labeled fixture/demo and fallback labels. | ### Accomplished In Current Path Runtime and guardrails: * `apify-client` is installed and script-side Apify helpers exist under `scripts/lib/apify/`. * `bun run apify:smoke` checks Apify readiness with redacted output and can run configured actors with `--check-sources`. * `data/openai-account-auth.json`, `.env.local`, `src/data/live-data.json`, logs, coverage, test results, Playwright output, and `.cache/` snapshots are ignored as private/generated data. * `bun run auth:openai` supports local account auth status, login, refresh, reauth, and logout. * `bun run codex:smoke` can exercise the Codex account transport with stored auth and sanitized output. * The official OpenAI API provider exists only as a future/placeholder slot; this is intentional for the hackathon MVP. Apify and sources: * The runtime has generic Apify config, Actor run, Dataset read, and dataset normalization modules. * Trend Finder declares Apify source roles for GitHub, Reddit, arXiv, Product Hunt, YouTube, and RSS/news. * Source declarations carry role, quality tier, compliance status, docs link, timeout, memory, and item cap metadata. * Source declaration gates disable unknown or explicitly disabled sources before collection. * Reviewed concrete Actor declarations exist for GitHub, Reddit, arXiv, Product Hunt, YouTube, and RSS/news. * The existing Hacker News public adapter still runs as a built-in source adapter and acts as a useful fallback/supplement. * Apify normalizers preserve public URLs, titles, snippets, timestamps, source roles, source IDs, and engagement metrics such as points, comments, downloads, stars, forks, upvotes, and views when available. * Apify actor ID, run ID, dataset ID, and item index are preserved in internal normalized item provenance, but the browser-safe evidence currently focuses on public evidence links rather than exposing Apify internal URLs. AI runtime and analyst contracts: * Runtime providers are now `disabled`, `mock`, `codex-account`, and placeholder `openai-api`. * Missing credentials and disabled runtime states produce readable fallback status instead of crashing collection. * The analyst contract accepts a Creator Lens, normalized evidence, and a previous snapshot summary. * The analyst output contract requires topic IDs, names, slugs, summaries, `whyNow`, `creatorAngle`, `suggestedHooks`, `audienceQuestions`, evidence IDs, source IDs, relevance/novelty/niche/creator scores, confidence, rejected evidence, and run notes. * Analyst validation rejects invented evidence IDs, unknown source IDs, invented URLs, invented dates, invented source names, and invented metrics. * Invalid AI output gets a repair attempt and then falls back deterministically. * Aggregate traces record runtime readiness, request metadata, sanitized payload summaries, validation issues, fallback reasons, and mapped provider errors without printing secrets. Scoring, snapshots, and artifacts: * The runtime produces typed Trend Finder artifacts rather than UI. * Browser-safe schema fields now include `creatorLens`, `runtime`, `runId`, enriched topics, score breakdowns, evidence IDs, source summaries, source breakdowns, watchlist entries, movement analysis, and `lastUpdatedAt`. * Opportunity score is deterministic and combines: momentum, novelty, evidence strength, source diversity, niche fit, and creator potential. * Scoring uses recency, engagement, volume, source status, source quality, source role, source diversity, and prior snapshot movement. * Source roles are weighted; research/developer/launch signals carry more scoring weight than lower-quality discussion or creator-only signals. * Hidden-gem labeling exists through deterministic criteria. * Movement labels exist for new, rising, cooling, steady, hidden-gem, and watchlist status. * Local trend snapshots are read and written below `.cache/extensions/trend-finder/` so score movement can be calculated across runs. Collector integration: * The Trend Finder collector now declares capabilities for network access, Apify Actor runs, Apify Dataset reads, AI analysis, trend snapshot reads, and trend snapshot writes. * Collection combines built-in source adapters with configured Apify sources. * A failed source no longer fails the whole collection; healthy source evidence is preserved and warnings are emitted. * The collector loads the active saved Creator Lens, loads previous snapshots, checks AI runtime readiness, prepares bounded analyst evidence, runs AI analysis only when ready, scores topics, assigns topic IDs back to evidence, validates data, and writes a snapshot. * When AI is unavailable, deterministic fallback keeps the product usable but groups evidence by a simple source/title-based path instead of semantic trend clustering. * The collector generates read-only watchlist rows from analyst reasons, hidden-gem status, movement, creator fit, novelty, and weak-evidence signals. * The collector generates movement analysis, prediction, retro, and sanitized Engine Replay artifact summaries when the relevant inputs are available. Dashboard delivery: * The Trend Finder extension now has dashboard views for Trends, Hidden Gems, Sources, Watchlist, and Brief. * The old report framing was replaced with a dashboard-native Brief view. * Runtime readiness is visible in the dashboard with redacted provider, credential, account, status, warnings, and setup next steps. * Source health shows active, degraded, and offline states plus warning copy. * Trend cards show rank, score, movement, momentum, confidence, topic status, summary, why-now copy, creator angle, hooks, audience questions, score breakdown, novelty, source count, evidence count, source context rows, evidence metric chips, and evidence links. * Missing referenced evidence is surfaced instead of silently hidden. * Hidden Gems filters early lower-volume topics. * The Creator Lens editor can save local drafts for the next collector run. * The Trend Finder run control saves dirty Creator Lens edits, calls the local aggregate refresh, shows progress, and keeps fallback/source degradation visible instead of disabling the run. * The Watchlist view renders generated monitoring rows with movement, historical context, and prediction/retro context when available. * The Brief view gives a compact handoff: source health, top creator angles, and evidence to verify. * Engine Replay has Replay and Reference modes: Replay shows latest-run browser-safe proof, while Reference renders committed Trend Finder manuals. Validation: * Phase 05 Session 10 recorded passing results for typecheck, script typecheck, focused runtime/dashboard tests, full unit/component tests, Playwright e2e tests, lint, format check, build, and bundle budget. * Re-run the current certification commands in `docs/hackathon/hackathon-submission.md` before recording or sharing a new branch. * Fallback checks passed for missing Apify token, missing account auth, invalid account auth, disabled provider, invalid AI output, and one-source failure. ### Current Remaining Gaps Creator Lens: * The Creator Lens is editable and saved for the next collector run. * It does not immediately rescore the current payload after a field changes. * It does not select, approve, or mutate source adapter configuration. * There is no cloud account profile sync or long-term preference-learning loop. Live source breadth: * The architecture supports Apify-backed multi-source collection. * Static declarations cover the intended 5-7 source roles. * The static declarations are reviewed metadata used to enrich matching env source IDs; they are not auto-loaded as runnable default sources. * Live Apify collection requires operator-provided source JSON or inline override entries plus `APIFY_TOKEN`. Matching reviewed source IDs can run; unknown sources still require a static declaration before the runtime treats them as approved. * A strong credentialed demo still needs a locally configured reviewed source run with labels that prove which source paths actually produced data. ### Apify Worker Research Date researched: 2026-05-17 Method: queried current Apify Store and Actor metadata with `apify-client` using `client.store().list(...)` and `client.actor(id).get()`. No Actors were run, no Datasets were read, and no API token or dataset item content was printed. The usage numbers below are point-in-time Store metadata, not guarantees. Important correction: * The previous placeholder declaration Actor IDs used as source-role examples did not resolve as public Apify Actors: `apify/github-public-search`, `apify/reddit-public-search`, `apify/arxiv-public-search`, `apify/producthunt-public-search`, `apify/youtube-public-search`, and `apify/rss-public-feed`. * The static declarations should target concrete Apify Actor candidates. Their sample Dataset shapes still need a tiny validation run before the demo relies on them. * Direct Reddit API and direct YouTube API notes should not be used as blanket blockers for Apify Actor adaptation. Trend Finder integrates with Apify Actor runs and Apify Datasets, then normalizes browser-safe public evidence fields. Selection criteria: * Prefer public, non-deprecated Actors with recent usage and clear structured metadata output. * Prefer metadata, public API, RSS, leaderboard, or search-result collection over full-content extraction, transcripts, comments, contact enrichment, or lead-generation oriented scrapers. * Prefer output that maps cleanly into existing Trend Finder evidence fields: public URL, title/name, short description/abstract/tagline, timestamp, source role, and metrics such as stars, forks, points, comments, downloads, upvotes, or views. * Avoid adapting actors whose primary value is private contact enrichment, account data, full-content capture, comments, transcripts, or media download. Recommended hackathon-safe candidates: | Source role | Source ID fit | Apify Actor candidate | Why adapt it | Notes | | --------------------- | -------------------------------------------- | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | Developer momentum | `github-ai-repositories` | `automation-lab/github-trending-scraper` | Public, non-deprecated, recently active; reports GitHub Trending repositories with stars, forks, star gains, language | Good first demo worker for builder momentum; input example includes `language` and `since`. | | Developer enrichment | future or secondary GitHub ID | `crawlerbros/github-repo-intelligence` | Rich repository metadata, topics, README, stars, forks, languages, license | Better as a second-pass enrichment worker after the trending worker is proven. | | Research novelty | `arxiv-ai-papers` | `easyapi/arxiv-search-scraper` | Public, non-deprecated, recently updated; extracts arXiv titles, abstracts, categories, identifiers, and PDF links | Use only public metadata and canonical `arxiv.org/abs/...` links in normalized evidence. | | Research fallback | `arxiv-ai-papers` | `gentle_cloud/arxiv-paper-search` | Simpler arXiv search/extraction path with public paper metadata | Lower usage than `easyapi/arxiv-search-scraper`; keep as fallback if input shape is cleaner. | | Launch/productization | `producthunt-ai-launches` | `muzafferkadir/product-hunt-leaderboard` | Leaderboard-focused output with products, descriptions, upvotes, comments, categories, and time windows | Prefer this over Product Hunt actors that emphasize emails or founder contact enrichment. | | Launch/productization | `producthunt-ai-launches` | `thirdwatch/producthunt-scraper` | Broader launch/profile/leaderboard coverage with topics, votes, comments, website URLs, and date ranges | Lower usage but clearer product-intelligence fit; useful fallback if leaderboard-only is thin. | | RSS/news | `rss-ai-news` | `xtech/feed-extractor` | Parses RSS, Atom, and JSON Feed into one structured row per feed item | Best fit for curated public AI blogs/newsletters because it avoids broad scraping. | | Reddit discussion | `reddit-ai-discussions` | `clearpath/reddit-search-scraper` | Rich keyword search output for posts/comments with scores, timestamps, permalinks, subreddit context | Prefer post-level public metadata; drop author/profile/comment-body fields in normalization. | | Reddit discussion | `reddit-ai-discussions` | `iskander/fast-reddit-scraper` | High-usage no-login Reddit search candidate with public post metadata and engagement | Good fallback if Clearpath's output shape is harder to map. | | YouTube creators | `youtube-ai-creator-videos` | `api-ninja/youtube-search-scraper` | High-usage YouTube search actor with video/channel/playlists metadata and result filters | Prefer title, watch URL, channel label, publish time, views/likes/comments only. | | YouTube creators | `youtube-ai-creator-videos` | `powerai/youtube-video-search-scraper` | Video-search focused output with titles, descriptions, direct URLs, statistics, upload timestamps | Good fallback if API Ninja output shape is less convenient. | | Discussion momentum | add Apify HN declaration or keep built-in HN | `gentle_cloud/hacker-news-scraper` | Uses HN Firebase and Algolia search; exposes stories, URLs, scores, comment counts, timestamps | Good candidate if we want HN under Apify; current built-in HN adapter can remain the fallback. | | Discussion momentum | add Apify HN declaration or keep built-in HN | `nexgendata/hacker-news-scraper` | Focused HN stories and trend tracking with top/new/best/ask/show/jobs feeds | More explicit example input, but less usage than `gentle_cloud/hacker-news-scraper`. | Lower-priority candidates: | Source role | Apify Actor candidate | Why lower priority | | ------------------ | ---------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Newsletter/content | `cloud9_ai/substack-scraper` | Safer than email-enrichment scrapers because it uses RSS-style post metadata, but RSS feeds cover the demo need. | | Newsletter/content | `hata1234/substack-scraper` | Useful later for public Substack metadata, but full article bodies should be avoided in the hackathon demo path. | | AI news aggregator | `buseta/ai-news-scraper` | Domain-relevant, but Store metadata showed a high recent failure/timed-out ratio compared with curated RSS. | Practical adaptation path: 1. For the demo source set, start with GitHub Trending, Reddit search, arXiv, Product Hunt leaderboard, YouTube search, and curated RSS/news. Add Hacker News through either the existing built-in adapter or a new reviewed Apify HN declaration. 2. Run each Actor once with tiny caps before trusting it: 5-10 items, short timeout, no full-content options, no comments/transcripts/contact fields. 3. Compare sample Dataset fields against `apify-normalizers.ts`; update field maps and tests only for public evidence fields that we want in browser-safe artifacts. 4. Do not let direct API notes block Apify Actor adaptation. Keep the browser artifact boundary tight: public URLs, labels, timestamps, snippets, and aggregate metrics only. Phase 06 Session 01 decision: keep Hacker News on the existing built-in public adapter for the hackathon MVP. Do not add a separate Apify Hacker News declaration until a later session can prevent duplicate discussion-source activation and validate a tiny capped HN Actor output shape. The reviewed Apify source set for Session 02 validation is therefore GitHub, Reddit, arXiv, Product Hunt, YouTube, and RSS/news. Run workflow: * The user can edit the Creator Lens and start a Trend Finder run from the dashboard. * Unsaved lens edits are saved through the token-gated local bridge before the run starts. * The run control requests a local token and calls `/__run_trend_finder`, which runs `scripts/scheduler-runner.ts --job trend-finder` through local dev middleware. * The UI shows saving/running/canceled/succeeded/failed states and elapsed run time. Canceling is browser-side and may not stop a Bun scheduler process that already started. Watchlist: * The schema, collector, and view generate and render watchlist items. * Rows can come from analyst `watchlistReason`, hidden-gem status, high novelty/low evidence combinations, creator fit, or movement thresholds. * The watchlist is read-only generated output, not user-authored saved items. Per-topic source breakdown: * Trend cards show source count, evidence count, evidence links, global source health, and compact per-topic source rows. * Source context rows include source role, source name, evidence count, quality tier, status, and warning state. Evidence metrics: * Apify normalizers preserve metrics such as points, comments, stars, forks, downloads, upvotes, and views for analysis and scoring. * Browser evidence cards show source label, relevance, date, snippet, evidence ID, public link, and metric chips when browser-safe source metrics exist. * Missing metrics are left absent rather than invented. Movement and snapshots: * Snapshot comparison supports `previousScore`, `velocity`, and statuses such as new, rising, cooling, steady, hidden-gem, and watchlist. * Trend cards show movement as new or score delta, plus previous-score summary text when a previous match exists. * Watchlist rows can show 12-week historical context and prediction/retro context when those private cache summaries exist. * There is still no durable hosted run picker or full browser historical replay UI. Historical backtests remain script-only. Feedback and learning: * The brainstorm's "Feedback / Memory" box is represented by local snapshots, topic identity, movement analysis, predictions, retros, and historical backtests. * There are still no useful/not-useful controls, feedback storage, scoring weight adaptation, or creator preference learning loops. Fallback semantics: * Missing Apify, missing auth, disabled runtime, invalid AI output, and one-source failure all have certified fallback paths. * Those fallbacks prove resilience, but they are not equivalent to a fully credentialed Apify-plus-Codex run. * Next step: be explicit in docs and Loom narration when the screen is showing fallback-generated topics versus AI-analyzed topics. Demo proof: * The docs describe a dashboard-centered Loom path. * A strong final demo needs either a real credentialed run with matching labels or a clearly labeled fixture/demo run that does not claim live source or AI output. * Use the 60-second script in `docs/hackathon/hackathon-submission.md` and read visible provenance labels aloud. Docs cleanup: * This audit corrected the archived Phase 05 PRD completed-session list so it includes Session 10 as well as the progress tracker. ### Off-Track Or Overclaim Risks * Do not claim that Trend Finder currently runs 5-7 live public source roles by default. It has a multi-source Apify framework and reviewed declarations; live collection still depends on configured actors, credentials, and compliance. * Do not claim any Apify Actor candidate is demo-ready until its tiny validation run succeeds and the Dataset output maps cleanly through the normalizers. * Do not present the official OpenAI API provider or Agents SDK provider as implemented analysis paths. The primary implemented local AI path is the Codex-account provider, with mock and disabled fallback paths. * Do not frame the project as a generated HTML report workflow. The implemented MVP surface is the dashboard, especially Trends, Hidden Gems, Sources, Watchlist, and Brief. * Do not spend the remaining hackathon time on a broad AI OS, dynamic marketplace, hosted runtime, full Job-Hunt API/orchestration port, durable accounts, or long-term preference learning. * Do not hide fallback states in the demo. Missing tokens, disabled runtime, source failures, and missing evidence references are useful proof of the permissioned runtime boundary. * Do not claim that Creator Lens edits instantly rescore the current payload or select live source adapters. Edits affect collector output only after save and rerun. * Do not call fallback output "AI-clustered" unless the runtime state is ready and the analyst succeeded. * Do not imply that missing evidence metrics were unavailable upstream; the UI only shows metrics that survived the browser-safe evidence boundary. ### Recommended Hackathon Finish Path 1. Configure a reviewed Apify run with a small safe source set from the Apify worker research above and capture the resulting `src/data/live-data.json` locally for demo use. 2. Save the intended Creator Lens, run Trend Finder from the dashboard, and verify the resulting provenance labels before recording. 3. Show generated watchlist rows, source context, evidence metric chips, and movement/previous-score labels only when the current payload includes them. 4. Record the 60-second Loom using the local dashboard path and exact language: "configured/reviewed public sources", "Codex-account runtime when enabled", "deterministic fallback when unavailable", and "dashboard Brief as the MVP delivery surface." ## Demo Narrative The Loom should quickly show: 1. Trend Finder can run configured reviewed source collection when credentials and reviewed source config are present. 2. In credentialed runs, Apify returns structured datasets from configured public source roles. 3. Trend Finder normalizes the evidence and, when the runtime is ready, the AI analyst groups related items into trend topics. 4. It ranks them by momentum, evidence quality, source diversity, earlyness, niche fit, and creator potential. 5. It turns the strongest topics into creator-ready opportunities. 6. It shows one or two example trends with evidence and suggested content angles. ## Non-Goals For Hackathon Focus Avoid spending time on: * broad foundation-app polish that does not support the Trend Finder story * complex personalization engines * full account/profile management * a general autonomous agent OS detached from the Trend Finder workflow * unconstrained agents that can act without declared permissions * too many weak data sources * UI features that do not help explain or act on a trend * generic dashboards that show activity without a creator decision The product should stay aimed at the competition question: > Can this help an AI creator find a useful early topic and decide what to make next? Runtime non-responsibilities for MVP: * no general autonomous desktop control * no open-ended file editing by the analyst * no full Job-Hunt API service port * no prompt bootstrap system * no workflow session database * no UI generation * no HTML report generation * no browser-visible secrets ## Pieces Not To Port Do not port these Job-Hunt areas for the hackathon: ``` /apps/api/src/agent-runtime/* /apps/api/src/orchestration/* /apps/api/src/prompt/* /apps/api/src/workspace/* /apps/api/src/server/routes/* /apps/api/src/runtime/service-container.ts ``` Reasons: * AI OS already has a Vite/TanStack Start app surface. * AI OS already has a collector extension system. * AI OS already writes generated runtime data through `scripts/aggregate.ts`. * Job-Hunt orchestration sessions are broader than this MVP needs. * Porting the full API service would consume the hackathon instead of serving the Trend Finder demo. No prompt-loader, workspace adapter, session DB, or HTTP orchestration layer is needed for the first implementation. Do not build long-term preference learning before the hackathon. A later version can add explicit user feedback such as useful/not useful and use that to tune weights. ## Run Surface For the hackathon, no new API service is required. The existing local dev middleware and aggregation flow are enough: * `bun run aggregate` produces `src/data/live-data.json` * app reads extension runtime data from the existing live data branch * the Trend Finder run control can save the current Creator Lens and invoke the scoped refresh path through `/__run_trend_finder` * the run control remains a local dev-server surface, not a hosted scheduler or remote execution API ## Broader OS Position This architecture helps the broader OS direction without building the whole OS now. What becomes reusable: * extension capability declarations * Apify Actor/Dataset source acquisition layer * AI runtime provider boundary * readiness state * typed artifact contracts * source adapter pattern * snapshot/memory pattern * traceable evidence-to-output pipeline What remains intentionally future work: * hard permission enforcement across all tools * extension marketplace * cross-app memory * user approval flows for side effects * multi-agent orchestration * hosted service runtime * durable user accounts and cloud sync The right position is vertical-first: > Build Trend Finder as the first useful vertical on top of permissioned runtime primitives. Do not build an abstract OS first and then look for a product. Avoid: ``` full Job-Hunt API/orchestration port ``` That is a post-hackathon platform investment, not the MVP path. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/hackathon/brainstorm-hackathon.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.