> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase29-session07-per-stage-validation-narration/spec.md).

# Session Specification

**Session ID**: `phase29-session07-per-stage-validation-narration` **Phase**: 29 - Trend Finder TrendingAI Comparison Adoption **Status**: Not Started **Created**: 2026-06-20

***

## 1. Session Overview

This session makes partial Trend Finder enrichment failures explicit in Engine Replay. Today the collector already emits sanitized trace records for source collection, runtime readiness, analyst analysis, scoring, artifacts, warnings, and decisions. Some optional enrichment paths can still fall back or degrade in ways that are only visible as generic warnings. Session 07 adds a small, bounded narration path so Engine Replay can show which enrichment stage failed, whether it retried, and which safe fallback path was used.

The implementation should add retry-once-then-degrade behavior around selected collector enrichment stages that depend on AI output or bounded derived projections. It must not change the underlying enrichment algorithms, scoring formulas, source adapters, source setup behavior, or generated report semantics beyond recording safe status details.

The result should preserve Trend Finder's privacy boundary. Engine Replay may show stage ID, safe status, issue code, retry count, degradation path, and short sanitized message text. It must not expose raw model output, raw normalized rows, prompts, provider responses, request IDs, cache paths, private diagnostics, tokens, stack traces, or local filesystem paths.

***

## 2. Objectives

1. Add a bounded stage-validation narration contract for enrichment stages.
2. Implement retry-once-then-degrade execution for selected collector enrichment stages without changing their core logic.
3. Record sanitized stage status, issue code, retry count, and degradation path in Engine Replay trace output.
4. Surface the narrated degraded state in Engine Replay with clear accessible UI and safe empty/error behavior.
5. Add focused tests proving retry limits, degraded narration, trace sanitization, payload budget, and privacy safety.

***

## 3. Prerequisites

### Required Sessions

* [x] `phase28-session15-documentation-validation-and-release` - Provides the certified Phase 28 Engine Replay and payload-safety baseline.
* [x] `phase29-session06-cross-topic-substrate-narratives` - Confirms the latest Phase 29 derived-field and analyst-output paths are present before reliability guardrails continue.

### Required Tools/Knowledge

* Bun 1.3.14 workflow through `bun run test`.
* Existing collector orchestration in `scripts/extensions/trend-finder/collector.ts`.
* Existing trace recorder and sanitizer in `scripts/extensions/trend-finder/engine-trace.ts`.
* Existing browser-safe trace schema in `src/extensions/trend-finder/engine-trace.ts`.
* Existing Engine Replay model and view projection patterns.
* Static Brief privacy scan and payload-size validation expectations.

### Environment Requirements

* Project dependencies installed with Bun.
* `src/data/live-data.example.json` available for fixture and schema validation.
* No new source, media, credential, database, hosted storage, or third-party transfer approval is required.

***

## 4. Scope

### In Scope (MVP)

* Trend Finder can narrate degraded enrichment stages - Add bounded stage narration rows for selected enrichment paths.
* Enrichment stages retry at most once - Add a shared wrapper that retries a failed stage one time and then returns the explicit deterministic degradation result.
* Engine Replay records safe trace details - Publish only stage ID, status, issue code, retry count, degradation path, and sanitized message text.
* Sanitization remains strict - Reject or strip unsafe keys and unsafe values before trace output reaches browser-visible data.
* Operators can inspect partial failures - Render stage narration in Engine Replay without hiding existing timeline, source, runtime, scoring, or artifact proof.
* Existing fallback behavior remains intact - Keep current deterministic fallback outputs and warnings; add narration instead of changing scoring or source behavior.
* Release checks prove behavior - Add tests for forced failure, retry count, sanitized trace output, legacy payload parsing, UI rendering, payload budget, and private leak prevention.

### Out of Scope (Deferred)

* Changing the underlying enrichment logic of any stage - Reason: this session is a reliability narration pass, not a scoring or enrichment redesign.
* Source-death baseline alarming - Reason: Session 09 reuses this narrated trace path for private last-good source count baselines.
* Required-derived-field closeout assertions - Reason: Session 08 owns the blocking closeout gate.
* New sources, social reach, comments, podcasts, transcripts, media, hosted storage, database state, or source compliance decisions - Reason: those remain explicit Phase 29 boundaries.
* Exposing raw stage input/output to make debugging easier - Reason: Engine Replay is browser-safe proof, not private diagnostics.

***

## 5. Technical Approach

### Architecture

Create a focused stage-validation helper under `scripts/extensions/trend-finder/`. The helper should own the stage narration contract, safe issue codes, retry cap, degradation path labels, abort-signal handling, and sanitized warning construction. It should accept a stage runner and a deterministic degradation function. On the first failure it retries once. On the second failure, or when the retry is not allowed, it returns the degraded value plus a safe narration record.

Extend the script-side `engine-trace.ts` event allowlist and mapper to accept a single stage narration trace event. The mapper should discard unsafe values and cap output. Extend `src/extensions/trend-finder/engine-trace.ts` with legacy-safe `stageNarrations` defaults and strict Zod parsing. The browser schema must remain additive so old payloads parse with `stageNarrations: []`.

Wire the wrapper around collector enrichment paths that are optional, AI-dependent, or derived-projection-dependent and already have deterministic fallback behavior. Candidate paths include AI analysis result handling, source-breakdown derivation, run narrative generation, and outlier idea projection. The implementation should prefer the narrowest wrapper points that record useful degraded state without duplicating warnings or changing published topic scoring.

Project the safe narration rows through `engine-replay-model.ts` and render them in the Engine Replay view as compact runtime notices or a bounded stage validation section. The UI should use existing tone classes, support empty and legacy states, remain accessible, and avoid layout shifts.

### Design Patterns

* Additive schema evolution: Legacy payloads parse with `stageNarrations: []`.
* Boundary-nearest sanitization: Stage details are normalized before Engine Replay receives them.
* Deterministic fallback before AI enrichment: Degraded output must be explicit and stable without optional AI output.
* Retry cap by construction: The wrapper owns the single retry limit and exposes retry count in the trace.
* Browser-safe projection: Engine Replay receives small safe labels and IDs, not diagnostic payloads.

### Technology Stack

* TypeScript and Zod for trace contracts, schema defaults, and exhaustive enum handling.
* Bun and Vitest for script/runtime tests.
* React 19 with existing Engine Replay view-model and component patterns.
* Existing static Brief privacy and payload-size utilities for boundary checks.

***

## 6. Deliverables

### Files to Create

| File                                                                 | Purpose                                                                                                         | Est. Lines |
| -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------- |
| `scripts/extensions/trend-finder/stage-validation.ts`                | Retry-once-then-degrade helper, stage narration contract, safe issue/degradation codes, and warning mapping.    | \~220      |
| `scripts/extensions/trend-finder/__tests__/stage-validation.test.ts` | Unit coverage for retry limit, degraded result, sanitized messages, abort handling, and unsafe value rejection. | \~220      |

### Files to Modify

| File                                                             | Changes                                                                                                          | Est. Lines |
| ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ---------- |
| `scripts/extensions/trend-finder/collector.ts`                   | Wrap selected enrichment paths, emit stage narration trace events, and preserve existing fallback outputs.       | \~160      |
| `scripts/extensions/trend-finder/engine-trace.ts`                | Add stage narration trace event, sanitization mapper, output caps, warnings, and run-note integration if needed. | \~180      |
| `src/extensions/trend-finder/engine-trace.ts`                    | Add `stageNarrations` types, enums, defaults, Zod input schema, and legacy-safe parsing.                         | \~160      |
| `src/extensions/trend-finder/engine-replay-model.ts`             | Project stage narration rows into Engine Replay view models with tones, labels, counts, and safe empty states.   | \~130      |
| `src/extensions/trend-finder/views/engine-replay-view.tsx`       | Render compact stage validation notices with accessible labels, stable layout, and offline/legacy-safe behavior. | \~80       |
| `src/extensions/trend-finder/fixtures.ts`                        | Add fixture rows for accepted, degraded, retry-once, and unsafe-dropped stage narration states.                  | \~70       |
| `scripts/extensions/trend-finder/__tests__/collector.test.ts`    | Cover forced enrichment-stage failure, retry count, degraded result preservation, and safe warnings.             | \~120      |
| `scripts/extensions/trend-finder/__tests__/engine-trace.test.ts` | Cover stage narration mapping, unsafe key/value stripping, caps, and legacy missing-field behavior.              | \~120      |
| `src/lib/__tests__/trend-finder-engine-replay.test.tsx`          | Cover model/view rendering for degraded stage narration, empty legacy payloads, and private leak prevention.     | \~110      |
| `scripts/extensions/trend-finder/measure-payload-size.ts`        | Keep new Engine Replay trace rows visible in payload-budget reporting if needed.                                 | \~20       |
| `docs/extensions/trend-finder-pipeline.md`                       | Document shipped stage narration behavior and retained privacy boundaries after implementation.                  | \~40       |

***

## 7. Success Criteria

### Functional Requirements

* [ ] A forced enrichment-stage failure records a narrated degraded state in Engine Replay.
* [ ] A failing stage retries at most once and exposes retry count as a safe integer.
* [ ] Stage narration identifies only safe stage ID, status, issue code, degradation path, and short sanitized message text.
* [ ] Legacy payloads without stage narration parse and render without crashing.
* [ ] Existing deterministic fallback output remains intact when narration records a degraded state.
* [ ] Source-death alarms and required-field closeout gates remain unimplemented in this session.

### Testing Requirements

* [ ] Unit tests written and passing for retry-once behavior, degraded result construction, abort handling, and unsafe value rejection.
* [ ] Collector tests prove a forced failure emits a safe trace row and preserves deterministic fallback behavior.
* [ ] Engine trace tests prove unsafe keys and values do not reach `stageNarrations`.
* [ ] Engine Replay model/view tests prove degraded, empty, legacy, and offline states render safely.
* [ ] Payload-size and privacy scan checks remain green.

### Non-Functional Requirements

* [ ] Browser payload stays under the 1 MB extension budget.
* [ ] Engine Replay and static/browser payloads do not expose raw model output, raw rows, prompts, provider responses, request IDs, tokens, credentials, stack traces, cache paths, private diagnostics, comments, transcripts, or local filesystem paths.
* [ ] No new source, collector boundary, media boundary, database, hosted storage path, dependency, or third-party transfer is introduced.
* [ ] Retry behavior is bounded, deterministic, abort-aware, and cannot create unbounded loops.

### Quality Gates

* [ ] All files ASCII-encoded.
* [ ] Unix LF line endings.
* [ ] Code follows project conventions.
* [ ] New enum or schema branches use explicit fallback behavior.

***

## 8. Implementation Notes

### Key Considerations

* The browser-facing field should be a single additive concept, `stageNarrations`, to avoid scattering similar information across warnings, run notes, and decision rows.
* Stage IDs and issue codes must be enums or safely normalized identifiers, not raw exception names or arbitrary provider messages.
* Retry count should be clamped to 0 or 1 in output even if a caller misuses the helper.
* If an existing stage already emits a warning, add narration without duplicating user-visible warning noise where possible.
* Private diagnostics can still contain deeper local information through the existing private path, but Engine Replay must stay browser-safe.

### Potential Challenges

* Some enrichment paths fail by returning fallback status rather than throwing. Mitigation: support both thrown errors and explicit degraded results in the wrapper.
* The trace sanitizer may silently drop unsafe values, making tests brittle. Mitigation: test positive safe fields and negative sentinel strings instead of exact raw error text.
* UI density may suffer if many stages degrade at once. Mitigation: cap row count and show aggregate labels for overflow.
* Retrying a stage could repeat expensive work. Mitigation: apply the wrapper only to bounded enrichment paths and keep retry count fixed at one.

### Relevant Considerations

* \[P02] **Extension payloads and labels stay bounded**: Stage narration rows must be capped, defaultable, and measured against the 1 MB payload budget.
* \[P24] **Browser-safe export and triage boundaries**: Engine Replay and browser-visible output must exclude raw evidence, private paths, prompts, provider responses, and credentials.
* \[P27] **Deterministic fallback before AI enrichment**: Degraded stage output must remain deterministic when optional analyst output is unavailable or invalid.
* \[P28] **Direct public source scope is narrow**: This session adds no source and does not widen deferred social or media candidates.
* \[P28] **Trend Finder release validation is bundled**: Payload, privacy, docs/reference checks, and scoped formatting evidence should be recorded during implementation and closeout.
* \[P00] **Do not document planned features as implemented**: Docs should be updated only to describe behavior that actually lands during implementation.

### Behavioral Quality Focus

Checklist active: Yes Top behavioral risks for this session:

* A thrown provider or projection error leaks raw prompt, response, token, stack, request ID, or local path through Engine Replay.
* Retry behavior loops or reruns an expensive stage more than once.
* A degraded stage is silently hidden, causing operators to trust a fallback output as fully successful.

***

## 9. Testing Strategy

### Unit Tests

* Add `stage-validation` helper tests for successful first try, retry success, retry failure, explicit degraded result, abort handling, retry count clamp, message sanitization, and unsafe value rejection.
* Add engine trace tests for stage narration mapping, field caps, unsafe key/value stripping, legacy missing field defaults, and overflow capping.
* Add collector tests for forced source-breakdown or outlier-idea failure that records degraded narration and keeps fallback output available.

### Integration Tests

* Add Engine Replay model/view tests for degraded stage rows, empty legacy payloads, offline parsed state, accessible labels, and no sentinel/private text leakage.
* Add fixture parsing tests that prove old payloads without `stageNarrations` still render.
* Add payload-size validation to confirm the trace addition stays within the extension budget.

### Manual Testing

* Run a focused collector fixture with a forced enrichment-stage failure and inspect Engine Replay for a degraded stage narration row.
* Open `/extensions/trend-finder/engine` against fixture data and confirm the stage narration does not displace the decision timeline or primary run proof.
* Inspect generated browser/static payloads for private-path, prompt, response, token, and stack-trace sentinel strings.

### Edge Cases

* A stage succeeds on retry after one failure.
* A stage throws twice and returns deterministic degraded output.
* A stage returns an explicit fallback status without throwing.
* An error message contains unsafe keys, local paths, token-shaped text, request IDs, or stack traces.
* Legacy data has no stage narration field.
* More stage narration rows are emitted than the browser cap allows.

***

## 10. Dependencies

### External Libraries

* None.

### Other Sessions

* **Depends on**: Phase 28 Engine Replay baseline and completed Phase 29 sessions 01-06 for current derived-field context.
* **Depended by**: `phase29-session09-source-death-baseline-alarm` and `phase29-session18-documentation-validation-and-release`.

***

## Next Steps

Run the implement workflow step to begin AI-led implementation.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase29-session07-per-stage-validation-narration/spec.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.