> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/runbooks/trend-finder-backtests.md). # Trend Finder Backtests Trend Finder historical backtests are script-only. The Product Hunt backtest CLI replays reviewed historical source windows, filters future evidence before scoring, reuses the existing snapshot, prediction, and retro contracts, and keeps full output private by default. Trend Finder also has a private backhistory replay script for already captured local cache files. It is script-only and writes the same private backtest archive class, but it reads local `.cache/extensions/trend-finder/backhistory/` inputs instead of calling the reviewed Product Hunt historical source path. Backtests are not browser-run controls, scheduler jobs, Engine Replay sessions, or future-run retros. ## Requirements * Run from the repository root. * Use Bun scripts. * Keep optional secrets in `.env.local` or the shell environment. * For `trend-finder:backtest`, use only sources with reviewed historical support. The first supported source is `producthunt-ai-launches`. * For `trend-finder:replay-backhistory`, use only private backhistory cache inputs that already exist on the local machine. * Keep `.cache/extensions/trend-finder/` private and gitignored. Optional credentialed runs need: ``` APIFY_TOKEN= ``` Dry-run planning and unit tests do not require an Apify token. ## Dry Run Dry-run mode validates the source allowlist, window bounds, caps, and output confinement. It does not call adapters and does not write files. ```bash bun run trend-finder:backtest -- --sources producthunt-ai-launches --window-start 2026-01-01 --window-end 2026-01-07 --windows 4 --dry-run ``` Expected output names only source IDs, counts, and private-cache status. ## Private Run Run without `--publish-aggregate` to keep browser data untouched: ```bash bun run trend-finder:backtest -- --sources producthunt-ai-launches --window-start 2026-01-01 --window-end 2026-01-07 --windows 4 ``` Private files are written under: ``` .cache/extensions/trend-finder/backtests// ``` The run writes: * `plan.json` * `aggregate.json` * `windows/window-*/snapshots/latest.json` * `windows/window-*/predictions/latest.json` * `windows/window-*/predictions/retros/latest.json` * `.cache/extensions/trend-finder/backtests/latest.json` These files may contain full per-window artifacts and must not be committed. ## Backhistory Replay Backhistory replay is for local private cache evidence that has already been captured outside the browser. It defaults to the latest private backhistory pointer plus manual RSS and YouTube evidence paths under `.cache/extensions/trend-finder/backhistory/`. Inspect flags: ```bash bun run trend-finder:replay-backhistory -- --help ``` Run with default private inputs: ```bash bun run trend-finder:replay-backhistory ``` Run with explicit private inputs: ```bash bun run trend-finder:replay-backhistory -- --input-dir .cache/extensions/trend-finder/backhistory/ --rss-path .cache/extensions/trend-finder/backhistory/.json --youtube-path .cache/extensions/trend-finder/backhistory/.json ``` The replay writes under `.cache/extensions/trend-finder/backtests//` and adds `backhistory-replay-summary.json`. Optional flags: * `--seed-current-history` also seeds normal private snapshot, prediction, and retro caches. * `--publish-aggregate` publishes only the bounded browser aggregate summary. * `--output ` must remain inside `.cache/extensions/trend-finder/backtests/`. Do not open, paste, or commit private backhistory inputs or replay archives. ## Publish Aggregate Use `--publish-aggregate` only when the dashboard should receive a bounded summary: ```bash bun run trend-finder:backtest -- --sources producthunt-ai-launches --window-start 2026-01-01 --window-end 2026-01-07 --windows 4 --publish-aggregate ``` Browser data receives only `backtestAggregateSummary`: * `mode: "backtest"` * generated timestamp and latest run ID * source and window labels * accepted and rejected evidence counts * topic, prediction, retro, and outcome counts It does not include evidence IDs, topic details, Actor metadata, Dataset IDs, request IDs, private paths, prompts, raw provider responses, or raw logs. ## Source And Window Rules * Unknown source IDs fail before collection. * Current-only sources fail before collection. * Window sizes must fit each selected source's reviewed bounds. * Product Hunt currently allows only `dateFrom` and `dateTo` overrides. * Evidence with `publishedAt > windowEnd` is rejected before identity, scoring, snapshots, predictions, or retro input. * Evidence with `publishedAt == windowEnd` is accepted. * Missing or invalid `publishedAt` values are rejected with typed warnings. ## Validation Use the focused checks while developing: ```bash bun run test -- scripts/lib/ai-runtime/__tests__/backtests.test.ts bun run test -- scripts/extensions/trend-finder/__tests__/historical-backhistory-flow.test.ts bun run test -- src/lib/__tests__/trend-finder-schema.test.ts scripts/lib/ai-runtime/__tests__/snapshots.test.ts scripts/lib/ai-runtime/__tests__/retros.test.ts ``` The historical backhistory flow test builds a temporary private cache, replays a small public-metadata-shaped fixture, seeds current history, runs the live collector against the dated snapshot series, and checks that the bounded published summary survives the aggregate preservation path. Use help checks when auditing command drift: ```bash bun run trend-finder:backtest -- --help bun run trend-finder:replay-backhistory -- --help ``` Before handoff, run: ```bash bun run typecheck bun run typecheck:scripts bun run test bun run runtime:check-private ``` Run `bun run test:e2e` only when browser-visible UI behavior changes beyond schema parsing. ## Demo Language Say "historical backtest CLI" for the reviewed Product Hunt historical source path. Say "private backhistory replay" for replaying local cached evidence. Say "future-run retro" for normal prediction grading between generated runs. Say "Engine Replay" only for the latest sanitized Trend Finder run surface. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/runbooks/trend-finder-backtests.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.