> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase28-session07-research-only-calibration-and-cache-retention/spec.md).

# Session Specification

**Session ID**: `phase28-session07-research-only-calibration-and-cache-retention` **Phase**: 28 - Trend Finder Trends-Finderz Adoption **Status**: Not Started **Created**: 2026-06-14

***

## 1. Session Overview

This session closes the remaining Tier 2 derived-layer work in Phase 28. Trend Finder already has source roles, role shares, source-quality scoring, topic visibility gates, signal aging, lifecycle multipliers, and deterministic risk flags. What it still lacks is an explicit caution flag for topics that look diverse only because multiple research feeds agree with each other while developer, discussion, or launch adoption evidence is absent.

The session adds a `research-only` risk flag to the existing Phase 27 risk flag set. It must use role-composition inputs that are already derivable from the collected evidence and source summaries, keep legacy payload parsing additive, and surface the flag through the existing Trend card and Signal Workbench risk flag paths. This extends the current flag framework instead of creating a new evidence-level or verdict-level system.

The session also adds retention pruning for private Trend Finder snapshot and prediction archives. Long-lived local installs currently write dated snapshot, prediction, and retro files without a bounded cleanup pass. The new retention helpers must preserve `latest.json`, respect the 84-day historical context, protect reviewed backtest windows and active Creator Lens state, report only aggregate deletion counts, and run at the end of collection like the existing enrichment-cache prune.

***

## 2. Objectives

1. Add a deterministic `research-only` risk flag when role composition is dominated by research evidence and adoption roles are absent.
2. Surface the new flag through schema parsing, Trend card chips, Signal Workbench facets, fixtures, and scoring documentation.
3. Add snapshot, prediction, and retro archive retention helpers with clamped configurable retention and strict path safety.
4. Invoke retention after collection and archive writes, reporting per-store pruned counts through sanitized trace and warning summaries.

***

## 3. Prerequisites

### Required Sessions

* [x] `phase27-session02-deterministic-derived-signals-and-risk-flags` - provides the current risk flag schema, scoring integration, and chip surfaces this session extends.
* [x] `phase28-session02-signal-quality-score-and-collection-health` - provides source-quality inputs and collection-health patterns used by risk derivation and trace reporting.
* [x] `phase28-session03-calibration-version-and-confidence-dampener` - provides scoring version stamps so pruned historical comparisons remain auditable across score changes.
* [x] `phase28-session06-lifecycle-multiplier-and-named-contributions` - completes Tier 2 score adjustment work before this final Tier 2 caution and hygiene pass.

### Required Tools/Knowledge

* Bun 1.3.14 package/runtime commands.
* TypeScript, Zod, Vitest, React 19, Tailwind CSS 4, and existing AI OS path aliases.
* Existing Trend Finder risk flag, source breakdown, scoring, snapshot, prediction, retro, collector trace, fixture, and documentation patterns.
* Trends-Finderz reference files: `EXAMPLES/trends-finderz/lib/product/research-signal-calibration.ts`, `EXAMPLES/trends-finderz/lib/cron/retention-cleanup.ts`, and `EXAMPLES/trends-finderz/README.md`.

### Environment Requirements

* Run commands from the repository root.
* Keep generated private runtime data, cache files, traces, screenshots, and source dumps out of commits.
* Do not add source adapters, network calls, storage backends, dependencies, analyst calls, credential flows, browser-visible private paths, or a new risk flag framework.

***

## 4. Scope

### In Scope (MVP)

* Trend Finder can derive a `research-only` risk flag from per-topic role composition when research evidence is present and developer, discussion, and launch roles are absent.
* Trend Finder can keep `research-only` as an additive risk flag enum with legacy-safe Zod defaults and no migration requirement for old payloads.
* Trend Finder can show the flag as a bounded chip and include it in Signal Workbench risk flag filters without changing the existing Workbench model.
* Trend Finder can prune dated private snapshot, prediction, and retro archive files past a configurable retention window with a default of 180 days and a lower bound above the 84-day history window.
* Trend Finder can preserve `latest.json`, active Creator Lens config/state, malformed or protected files it cannot safely classify, and reviewed backtest windows.
* Trend Finder can report only aggregate pruned counts and sanitized warning codes in Engine Replay trace and collector warnings; browser payloads must never receive deleted file paths.
* Tests cover role-composition flagging, legacy parsing, Workbench facets, retention clamps, path-safety guards, preserve-list behavior, collector invocation, aggregate counts, and sanitized output.

### Out of Scope (Deferred)

* A new risk flag framework - *Reason: this session extends the existing Phase 27 risk flag set.*
* Per-topic action verdicts, action consistency QA, or action wording caps - *Reason: Session 08 consumes `research-only` as an input.*
* The Action Queue surface - *Reason: Session 09 owns grouped action UI.*
* Enrichment-cache or evidence-asset pruning changes - *Reason: those stores already have prune passes.*
* SQLite-backed retention - *Reason: the SQLite observation store plan owns durable storage; this session records policy-compatible file retention only.*
* New source collection, compliance docs, API adapters, analyst calls, or third-party dependencies - *Reason: this is deterministic over existing local data.*

***

## 5. Technical Approach

### Architecture

Extend `scripts/lib/ai-runtime/risk-flags.ts` with bounded role-composition input. The helper should continue to return the existing `RiskFlagDerivationResult` shape and issue mapping. Add role counts or shares to the input schema, normalize unknown or malformed roles to safe defaults, and append `research-only` only when research evidence exists while developer, discussion, and launch adoption roles are absent. Keep flag ordering controlled by `TREND_TOPIC_RISK_FLAGS`.

Wire the new derivation through `scripts/lib/ai-runtime/scoring.ts` using the same evidence and source summaries already available during scoring. If source-breakdown role shares are not yet attached at that point, derive the minimal role counts locally from scoring evidence and source metadata rather than moving the whole source-breakdown pass earlier. The public topic payload continues to carry one `riskFlags` array.

Add the enum value and labels in `src/extensions/trend-finder/schema.ts` and `src/extensions/trend-finder/view-model.ts`. Existing Trend card chip rendering and Signal Workbench facets should pick up the new enum path; update tests and fixtures to prove that a research-only topic can be filtered and displayed.

For retention, add pure path-safe helpers to `scripts/lib/ai-runtime/snapshots.ts`, `scripts/lib/ai-runtime/predictions.ts`, and `scripts/lib/ai-runtime/retros.ts`. Each helper should list only its own archive directory, skip `latest.json`, skip subdirectories and non-JSON entries it cannot classify safely, compare dated filenames or file metadata against the clamped retention window, and return aggregate counts plus sanitized warning messages. Deletions must use the existing path-containment guard pattern before `unlink`.

Invoke the retention pass from `scripts/extensions/trend-finder/collector.ts` after snapshot and prediction/retro writes are complete. Trace events should record store names and counts only, and collector warnings should use bounded codes such as `trend-cache-retention-degraded`. Browser data may include aggregate counts only if routed through an existing sanitized summary; no private cache path should enter payloads, warnings, docs examples, or tests.

### Design Patterns

* Pure function, then tests, then wire: derive role composition and retention eligibility independently before collector integration.
* Zod additive defaults: old generated payloads and fixtures parse without a migration.
* Path containment before deletion: every unlink target is asserted inside its expected private cache directory.
* Browser-safe projection: expose bounded labels, counts, enums, and warning codes only.
* Compatibility by extension: keep `riskFlags` as the single public flag array and keep existing retention helpers for enrichment cache and assets intact.

### Technology Stack

* Bun 1.3.14 for scripts and test execution.
* TypeScript for application and script code.
* Zod for payload parsing and legacy defaults.
* React 19 and Tailwind CSS 4 for existing chip and Workbench surfaces.
* Vitest for script, schema, view-model, component, and collector coverage.

***

## 6. Deliverables

### Files to Create

| File                                                                                                         | Purpose                                                                                      | Est. Lines |
| ------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------- | ---------- |
| `.spec_system/specs/phase28-session07-research-only-calibration-and-cache-retention/implementation-notes.md` | Record retention constants, preserve-list decisions, validation commands, and residual risks | \~90       |

### Files to Modify

| File                                                                   | Changes                                                                                | Est. Lines |
| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ---------- |
| `scripts/lib/ai-runtime/risk-flags.ts`                                 | Add role-composition schema and `research-only` flag derivation                        | \~140      |
| `scripts/lib/ai-runtime/__tests__/risk-flags.test.ts`                  | Cover research-only detection, adoption-role exclusions, malformed roles, and ordering | \~130      |
| `scripts/lib/ai-runtime/scoring.ts`                                    | Pass source-role inputs into risk flag derivation without exposing raw source details  | \~80       |
| `scripts/lib/ai-runtime/__tests__/scoring.test.ts`                     | Cover scoring integration and coexistence with existing risk flags                     | \~100      |
| `src/extensions/trend-finder/schema.ts`                                | Add additive `research-only` enum support and legacy-safe parsing                      | \~30       |
| `src/lib/__tests__/trend-finder-schema.test.ts`                        | Cover legacy payloads and research-only enum parsing                                   | \~60       |
| `src/extensions/trend-finder/view-model.ts`                            | Add chip label, description, and tone for `research-only`                              | \~35       |
| `src/extensions/trend-finder/__tests__/view-model.test.ts`             | Cover risk flag label, aria text, tone, and bounded copy                               | \~70       |
| `src/extensions/trend-finder/signal-workbench-model.ts`                | Verify risk flag facet support and filter behavior for the new enum                    | \~30       |
| `src/extensions/trend-finder/__tests__/signal-workbench-model.test.ts` | Cover Workbench research-only facet counts and filtering                               | \~90       |
| `src/extensions/trend-finder/fixtures.ts`                              | Add representative research-only fixture data                                          | \~80       |
| `scripts/lib/ai-runtime/snapshots.ts`                                  | Add path-safe snapshot retention helper with clamp and preserve-list behavior          | \~170      |
| `scripts/lib/ai-runtime/__tests__/snapshots.test.ts`                   | Cover retention window clamps, `latest.json`, malformed files, and path safety         | \~170      |
| `scripts/lib/ai-runtime/predictions.ts`                                | Add path-safe prediction archive retention helper                                      | \~120      |
| `scripts/lib/ai-runtime/__tests__/predictions.test.ts`                 | Cover prediction archive pruning and latest preservation                               | \~120      |
| `scripts/lib/ai-runtime/retros.ts`                                     | Add path-safe retro archive retention helper under `predictions/retros`                | \~120      |
| `scripts/lib/ai-runtime/__tests__/retros.test.ts`                      | Cover retro archive pruning and latest preservation                                    | \~120      |
| `scripts/extensions/trend-finder/collector.ts`                         | Invoke retention after writes and emit sanitized trace/count warnings                  | \~160      |
| `scripts/extensions/trend-finder/__tests__/collector.test.ts`          | Cover retention invocation, aggregate counts, warnings, and no private path leakage    | \~160      |
| `docs/extensions/trend-finder-scoring.md`                              | Document shipped `research-only` risk flag and action-layer caveat input               | \~45       |
| `docs/extensions/trend-finder-pipeline.md`                             | Document retention policy, stores, preserve list, and sanitized reporting              | \~70       |

***

## 7. Success Criteria

### Functional Requirements

* [ ] A topic supported only by research-role evidence is flagged `research-only`.
* [ ] A topic with developer, discussion, or launch adoption evidence is not flagged `research-only` even when research evidence is also present.
* [ ] `research-only` appears as a bounded Trend card chip and as a Signal Workbench risk flag filter.
* [ ] Legacy payloads parse when they omit the new flag, and invalid flag data still falls back safely.
* [ ] Snapshot, prediction, and retro archives prune dated JSON files beyond a clamped retention window while preserving `latest.json`.
* [ ] The retention pass respects the 84-day history window, reviewed backtest windows, and active Creator Lens state.
* [ ] Collector trace and warnings report per-store counts and sanitized codes only; no private file paths enter browser payloads.
* [ ] Existing risk flag, scoring, history, prediction, retro, collector, and UI tests continue to pass.

### Testing Requirements

* [ ] Unit tests written and passing for role-composition risk derivation, flag ordering, malformed inputs, and adoption-role exclusions.
* [ ] Scoring integration tests cover research-only payload output alongside existing risk flags.
* [ ] Schema, view-model, and Workbench tests cover additive parsing, chip copy, facet counts, and filter behavior.
* [ ] Snapshot, prediction, and retro archive tests cover retention clamps, latest preservation, deletion counts, malformed files, and path safety.
* [ ] Collector tests cover end-of-collect retention invocation, trace counts, warning behavior, and no private path leakage.
* [ ] Manual testing completed for Trend card risk chips, Signal Workbench risk flag filter, and a dry-run local cache retention scenario.

### Non-Functional Requirements

* [ ] No new dependencies, source adapters, storage backends, network calls, analyst calls, credential flows, or admin write surfaces are introduced.
* [ ] Browser-visible payloads remain bounded and do not expose private cache paths, raw source rows, prompts, logs, billing, provider responses, or deleted filenames.
* [ ] Generated private cache files remain out of git.
* [ ] The shared Trend Finder payload remains within the existing 1 MB release budget.
* [ ] Retention improves GDPR data minimization while preserving documented local deletion paths.

### Quality Gates

* [ ] All files ASCII-encoded.
* [ ] Unix LF line endings.
* [ ] Code follows project conventions.
* [ ] Focused tests pass before full validation.

***

## 8. Implementation Notes

### Key Considerations

* The `research-only` flag should be a caution input for Session 08 action verdicts, not a verdict by itself.
* Role-composition thresholds must be calibrated against Trend Finder source roles, not copied blindly from Trends-Finderz.
* Retention should delete only private dated archives it can safely classify; unknown files should produce sanitized warnings instead of risky deletion.
* The retention default should be 180 days, with the lower clamp kept above the 84-day historical context window.

### Potential Challenges

* Risk derivation currently runs before `sourceBreakdowns` are attached: mitigate by deriving minimal role counts from scoring evidence and source summaries in `scoring.ts`.
* Prediction and retro archives share the `predictions/` cache tree: mitigate by giving each helper a strict directory resolver and independent tests.
* Backtest windows may use private directories: mitigate by preserving known backtest directories and documenting SQLite as the future durable policy.
* Retention warnings can leak paths if reused directly: mitigate by returning bounded codes/counts and testing serialized browser payloads.

### Relevant Considerations

* \[P02] **Extension payloads and labels stay bounded**: Research-only chips and retention summaries must keep bounded labels and counts.
* \[P05] **Script-only runtime boundary**: Retention and role derivation stay in script/runtime helpers, with browser surfaces receiving safe projections.
* \[P24] **Browser-safe export and triage boundaries**: Retention must never expose private cache paths or deleted filenames in browser-visible output.
* \[P27] **Trend Finder payload growth needs release checks**: Additive flags and retention summaries must be covered by payload/privacy checks.
* \[P01] **Extract pure functions, then test, then wire**: Apply this to role composition and archive retention helpers before collector integration.
* \[P27] **Deterministic fallback before AI enrichment**: The flag and retention policy remain deterministic and do not add AI calls.

### Behavioral Quality Focus

Checklist active: Yes

Top behavioral risks for this session:

* Unsafe deletion of dated archives that history, retros, or backtests still need.
* Private cache paths or filenames leaking through warning, trace, fixture, or browser payload fields.
* `research-only` overtriggering on mixed evidence and incorrectly feeding the next action verdict layer.
* Additive schema changes breaking older generated data or fixtures.

***

## 9. Testing Strategy

### Unit Tests

* Test `deriveTopicRiskFlags` for research-only role composition, mixed-role exclusions, malformed role data, existing flag coexistence, and stable order.
* Test snapshot, prediction, and retro retention helpers for clamp boundaries, date classification, `latest.json` preservation, path containment, malformed entries, and deletion counts.

### Integration Tests

* Test `scoreTopics` output for a research-only topic and a mixed research-plus-adoption topic.
* Test collector retention invocation after archive writes with sanitized trace counts and warning codes.
* Test schema, view-model, and Signal Workbench paths using fixture payloads.

### Manual Testing

* Run a focused local Trend Finder payload with a research-only fixture and verify the Trend card chip and Signal Workbench risk flag filter.
* Run a dry-run or temporary-directory retention scenario and verify counts are reported without private paths.
* Inspect docs and generated payload samples for planned-feature language and private-path leakage.

### Edge Cases

* Empty evidence, missing sources, unknown roles, all offline sources, duplicate evidence IDs, and low-quality research sources.
* Boundary retention values below the lower clamp, above the upper clamp, invalid env values, future-dated archive files, and invalid JSON files.
* `latest.json`, nested directories, non-JSON files, backtest directories, and archive files whose timestamps cannot be parsed.
* Legacy payloads with no `riskFlags`, empty `roleShares`, or old scoring versions.

***

## 10. Dependencies

### External Libraries

* None. This session must not add dependencies.

### Other Sessions

* **Depends on**: `phase27-session02-deterministic-derived-signals-and-risk-flags`, `phase28-session02-signal-quality-score-and-collection-health`, `phase28-session03-calibration-version-and-confidence-dampener`, and `phase28-session06-lifecycle-multiplier-and-named-contributions`.
* **Depended by**: `phase28-session08-action-verdicts-and-consistency-qa`, `phase28-session09-action-queue-surface`, and `phase28-session15-documentation-validation-and-release`.

***

## Next Steps

Run the implement workflow step to begin AI-led implementation.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase28-session07-research-only-calibration-and-cache-retention/spec.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.