> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/extensions/trend-finder/history.md).

# Trend Finder History

This guide covers topic continuity, movement, historical context, prediction and retro calibration, and script-only historical backtests.

## Topic Identity

Trend Finder tries to keep topic continuity across runs because scores, movement, prediction grading, and historical context are only useful when "same topic" is stable enough.

Identity resolution compares current analyst/fallback topics with the previous snapshot and the 84-day historical context. It can match by:

| Match Reason       | Meaning                                                                                                                                               |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `exact-id`         | Current topic ID matches a previous canonical ID.                                                                                                     |
| `alias`            | Current topic ID appears in a previous alias set.                                                                                                     |
| `slug`             | Current slug matches a previous slug.                                                                                                                 |
| `normalized-name`  | Normalized current name matches a previous name.                                                                                                      |
| `evidence-overlap` | Current and previous evidence IDs overlap enough.                                                                                                     |
| `similarity`       | Names or historical token context are similar enough.                                                                                                 |
| `ai-hint`          | A validated AI continuity hint agrees with evidence/name checks. The resolver supports this reason, but the current collector does not pass AI hints. |
| `reviewed-seed`    | No prior row matched, but the topic strongly matched a reviewed evergreen AI topic seed.                                                              |
| `new`              | No previous or historical match was accepted.                                                                                                         |

The resolved `canonicalTopicId`, aliases, match reason, and confidence travel with the topic in the browser payload. New topics use their current topic ID as the canonical ID. Existing topics keep the older canonical ID so movement and retro logic can connect current and previous rows.

Reviewed canonical seeds are committed metadata for evergreen AI topics such as MCP, RAG, GraphRAG, reasoning models, local LLMs, and AI coding agents. They are used only after previous snapshots, historical context, and validated AI hints fail to resolve a topic. Seed matching requires exact phrase or strong alias evidence and does not override run-specific identity decisions.

## Movement And History

Trend Finder tracks two related but separate movement concepts.

| Concept           | Source                                                              | Meaning                                           |
| ----------------- | ------------------------------------------------------------------- | ------------------------------------------------- |
| Topic movement    | Deterministic scoring comparison against previous snapshot/history. | `new`, `rising`, `cooling`, or `steady`.          |
| Movement analysis | AI or deterministic explanation of evidence churn between runs.     | `new`, `rising`, `cooling`, `noisy`, or `stable`. |

Topic movement uses these score-delta thresholds:

| Delta From Previous Score | Movement |
| ------------------------- | -------- |
| No previous match         | new      |
| >= +10                    | rising   |
| <= -10                    | cooling  |
| Otherwise                 | steady   |

Movement analysis uses a smaller +/-5 score threshold and also checks evidence churn. A topic can be "noisy" when enough previous evidence drops while new evidence appears.

The 12-week history panels are built from private local snapshots. They show appearance count, first/last seen dates, trailing average score, latest score, peak score, score slope, source count, and evidence count. Historical context also reduces novelty for recurring topics.

## Daily Series And Sparklines

Trend Finder also persists private daily topic series below the Trend Finder snapshot cache. The browser receives only bounded summaries:

| Field                  | Browser Meaning                                                  |
| ---------------------- | ---------------------------------------------------------------- |
| `sparkline.state`      | `available`, `missing-history`, `no-volume`, or `unknown`.       |
| `sparkline.points`     | At most 14 daily evidence-count buckets for the canonical topic. |
| `sparkline.totalCount` | Sum of the bounded points published to the browser.              |
| `sparkline.peakCount`  | Largest daily count inside the bounded series.                   |

Sparklines are history context, not raw archives. They do not expose private snapshot paths, raw source rows, or historical evidence bodies. Missing history and no-volume states render explicit labels on cards, Workbench rows, and static Brief output.

Hugging Face model download deltas use the same private observation pattern. The collector stores prior public model download counts locally, compares the current run with the prior observation, and publishes only a bounded `downloadDelta` label on matching evidence rows. No-baseline, unavailable, and invalid-metric states are explicit; Trend Finder does not invent run-over-run download movement when no prior public observation exists.

## Prediction And Retro

Prediction and retro is Trend Finder's feedback loop.

Prediction means: "Given this run, what direction should this topic move by the next run or next week?"

Retro means: "When a later run happens, did an older prediction hit, partially hit, miss, or remain pending?"

The significance is calibration. It gives Trend Finder a memory of whether its near-term calls were useful, instead of only showing fresh scores every run.

### Prediction Writer

After scoring, Trend Finder writes prediction records for the scored topics. When the AI runtime is ready, the prediction writer can produce AI-backed records. Otherwise it uses deterministic fallback.

Fallback expected direction is derived in this order:

1. Movement `rising` predicts rising.
2. Movement `cooling` predicts cooling.
3. Movement `new` predicts rising.
4. Previous-score delta >= +5 predicts rising.
5. Previous-score delta <= -5 predicts cooling.
6. Historical baseline delta >= +5 predicts rising.
7. Historical baseline delta <= -5 predicts cooling.
8. Historical slope >= +4 predicts rising.
9. Historical slope <= -4 predicts cooling.
10. Otherwise predicts stable.

Prediction records include expected direction, forecast horizon, optional UTC target date, optional target date label, lifecycle at filing, optional target lifecycle, rationale, watch-next instruction, cited current evidence IDs, and provenance (`ai` or `fallback`).

Forecast horizons are `next-run`, `next-week`, `30-day`, `60-day`, and `90-day`. AI predictions can choose a dated target when lifecycle context is available. Deterministic fallback predictions also produce bounded target dates from the run date, then derive a target lifecycle from the filing lifecycle and expected direction.

The prediction writer validates target dates as UTC `YYYY-MM-DD` dates on or after the run date, validates lifecycle values against Trend Finder lifecycle stages, caps strings, cites only current evidence IDs, and rejects private paths or raw provider/source data.

### Retro Evaluator

At the next run, Trend Finder reads pending previous predictions and grades them against current topics.

Observed direction is based on current score versus the best available baseline: previous score, historical latest score, or historical trailing average.

| Observation | Rule                                              |
| ----------- | ------------------------------------------------- |
| rising      | current score is at least 5 points above baseline |
| cooling     | current score is at least 5 points below baseline |
| stable      | current score is within +/-5 of baseline          |
| unknown     | no matching topic or baseline is available        |

Outcome rules:

| Outcome | Meaning                                                                                                    |
| ------- | ---------------------------------------------------------------------------------------------------------- |
| hit     | Expected direction equals observed direction.                                                              |
| partial | Directional prediction flattened, stable prediction moved, or another non-opposite partial match happened. |
| miss    | Expected rising but observed cooling, or expected cooling but observed rising.                             |
| pending | Observed direction is unknown.                                                                             |

A pending retro becomes stale after three pending runs.

Target-date predictions are not graded early. If a record has a target date, the retro evaluator keeps it pending until the current run date is on or after that target date. Due and not-yet-due states are visible in the bounded browser projection so an operator can distinguish "waiting for the target window" from "unable to match the topic."

Only a bounded browser-safe summary is surfaced in the UI. Prediction and retro archives are private local cache files.

## Calibration And Story Log

The prediction/retro summary now includes aggregate calibration metrics:

| Metric              | Meaning                                                              |
| ------------------- | -------------------------------------------------------------------- |
| `hitRate`           | Hits divided by resolved, calibration-eligible retros.               |
| `brierScore`        | Directional probability calibration score over resolved predictions. |
| `resolvedCount`     | Retros that have hit, partial, or miss outcomes.                     |
| `pendingCount`      | Retros still waiting for an observation or target date.              |
| `stalePendingCount` | Pending retros that exceeded the stale pending run threshold.        |
| `notYetDueCount`    | Dated predictions excluded from calibration until their target date. |

The metrics are accountability context, not a guarantee of future accuracy. They are computed from bounded retro records and avoid Alpha Radar's weaker "100% hit rate with zero fully verified calls" pattern.

Story Log is the browser-safe history projection for the feedback loop. It combines pending predictions and resolved retros into capped rows with verdict, due-state, target date, lifecycle, expected direction, observed direction, rationale, lesson, provenance, and cited current evidence IDs. It deliberately excludes archive paths, cache roots, raw prompts, provider responses, raw source rows, and private logs.

Watchlist can filter Story Log rows by verdict/due state. Brief and static Brief render only compact summary rows so the creator surface remains scannable.

## One To Watch

One to Watch is a presentation layer over the prediction/retro loop. It does not add a new model, source collector, or forecast. It reads the current `predictionRetroSummary.storyLog`, `recentPredictions`, and current topic rows, then ranks bounded rows for the next review cycle.

The ranking is deterministic. It considers due state, expected direction, watch-next copy, rationale, current score, evidence count, source count, reception, corroboration, visibility, stale pending state, generated date, topic name, and stable IDs. Current topic context joins by topic ID, canonical topic ID, and aliases. If no current topic matches, the row remains visible with support unavailable instead of borrowing evidence from another topic.

Calibration is intentionally honest. A row can say `Uncalibrated`, `Accuracy unknown`, `Stale pending`, or `Aggregate calibrated`, but it never shows a per-row accuracy percentage. Aggregate calibration, when available, is section context only and comes from resolved retro records. Pending, not-yet-due, due-unresolved, and stale-pending rows do not infer accuracy from the aggregate.

## Historical Backtests

Historical backtests are script-only. They are not the normal Trend Finder run and they are not a browser route.

The CLI is:

```bash
bun run trend-finder:backtest -- --sources <source-id[,source-id]> --window-start <iso> --window-end <iso>
```

A backtest runs reviewed historical source windows, scores topics for those windows, writes private window archives under the Trend Finder cache, and can optionally publish only a bounded aggregate summary into generated browser data with `--publish-aggregate`.

At the moment, do not assume every source can be backtested. The first reviewed historical source is `producthunt-ai-launches`, because its reviewed Actor input supports bounded `dateFrom` and `dateTo` overrides. Current-only sources fail before collection in the backtest CLI.

`google-ai-news` was rechecked on 2026-05-27 and remains current-only: the reviewed `thescrappa/google-news-scraper` input exposes query, locale, pagination, sort, and token fields, but no bounded start/end date fields.

Backtest retros use `mode: "backtest"`. Future-run retros use `mode: "future-run"`. The distinction matters: future-run retros grade real predictions from earlier normal runs, while backtest retros grade simulated historical windows.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/extensions/trend-finder/history.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.