> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase28-session02-signal-quality-score-and-collection-health/spec.md).

# Session Specification

**Session ID**: `phase28-session02-signal-quality-score-and-collection-health` **Phase**: 28 - Trend Finder Trends-Finderz Adoption **Status**: Not Started **Created**: 2026-06-14

***

## 1. Session Overview

This session adds one shared cross-source quality score to every Trend Finder evidence row. Today the collector and scoring path still lean on source-specific `relevanceScore` shapes, plus source quality tiers, which means evidence selection and the evidence-strength factor can compare unlike values across Hacker News, Apify-backed sources, RSS-like feeds, and degraded fixture paths.

The work uses Session 01's identity and duplicate counters as the foundation. Session 01 made duplicate and syndication pressure measurable; this session turns those counters into run-level collection-health metadata and displays the duplicate-rate plus "N of M sources produced evidence" rollup in the Sources view and Engine Replay source stage.

The implementation stays deterministic, local, and additive. It derives quality from existing evidence, source, and source-local metrics only; it does not add source calls, AI calls, dependencies, durable storage, or new collection surfaces.

***

## 2. Objectives

1. Compute one bounded 0-100 quality score per evidence row from existing source tier, recency, source-local metric strength, title sanity, and URL presence.
2. Use the shared quality score for evidence selection and the evidence-strength factor while preserving per-source relevance as a component.
3. Publish run-level collection-health metadata with duplicate rate, high-duplicate warning state, and source coverage.
4. Render collection-health rollups in the Sources view, Engine Replay source rail, and optional Signal Workbench evidence rows.

***

## 3. Prerequisites

### Required Sessions

* [x] `phase28-session01-cross-source-signal-identity-and-dedup` - provides deterministic within-source duplicate counts, high-duplicate warnings, cross-source syndication counters, and sanitized Engine Replay trace fields consumed by this session.

### Required Tools/Knowledge

* Bun 1.3.14 and Vitest for focused TypeScript and React tests.
* Zod additive-default patterns in `src/extensions/trend-finder/schema.ts`.
* Existing source-quality helper conventions in `scripts/extensions/trend-finder/sources/source-quality.ts`.
* Existing collector trace and Engine Replay mapping patterns in `scripts/extensions/trend-finder/engine-trace.ts` and `src/extensions/trend-finder/engine-trace.ts`.
* Trends-Finderz quality and scan-health references under `EXAMPLES/trends-finderz/lib/signals/`, `EXAMPLES/trends-finderz/lib/scan/`, and `EXAMPLES/trends-finderz/components/dashboard/`.

### Environment Requirements

* Work from the repo root.
* Do not add runtime dependencies.
* Do not introduce new public data collection paths or AI runtime calls.
* Keep generated private Trend Finder data and raw source payloads out of git and out of browser-visible output.

***

## 4. Scope

### In Scope (MVP)

* Trend Finder collector derives a single cross-source evidence quality score from existing inputs - source quality tier, published-at recency, source-local metric signal, title length sanity, URL presence, and existing relevance as one bounded component.
* Trend Finder analyst evidence and browser evidence carry the shared quality score through additive schema defaults so legacy payloads keep parsing.
* Evidence selection uses the shared quality score before source-specific relevance, while preserving deterministic tie-breakers.
* The evidence-strength factor consumes the shared quality score and keeps relevance as an explicit component, without changing the six-factor score weights.
* Collection-health metadata publishes duplicate rate, duplicate counts, high-duplicate state, and source coverage from Session 01 counters and current source summaries.
* Sources view header and health tiles show duplicate pressure and "N of M sources produced evidence" with explicit empty, degraded, and warning states.
* Engine Replay source stage shows sanitized duplicate and coverage counters.
* Optional Signal Workbench evidence rows expose the quality score as a stable sort/display signal.
* Focused tests cover quality scoring, source selection, evidence-strength behavior, collection-health rollups, legacy parsing, and UI labels.

### Out of Scope (Deferred)

* Changing the six-factor opportunity score weights - *Reason: this session changes inputs to evidence selection and evidence-strength, not the score weight contract.*
* Reworking Session 01 signal-identity or dedup internals - *Reason: those counters are completed and validated inputs.*
* Topic-level noise gating or visibility bands - *Reason: Session 04 owns topic-quality/noise decisions.*
* New source adapters, keyword packs, or network collection paths - *Reason: Sessions 13 and 14 own compliance-gated collection changes.*
* Durable storage for quality scores - *Reason: the SQLite observation-store plan owns persisted signal metadata.*

***

## 5. Technical Approach

### Architecture

Extend `scripts/extensions/trend-finder/sources/source-quality.ts` instead of creating a competing helper. Add a pure `calculateEvidenceQualityScore()` contract that accepts the existing analyst evidence shape plus the matching source summary and returns a bounded 0-100 score with component values. The collector applies this once, after source enrichment and Session 01 identity metadata are available, and writes the bounded score to both analyst and browser evidence.

Add a top-level additive `collectionHealth` payload object in `src/extensions/trend-finder/schema.ts`. It should default to an unavailable state for legacy payloads and include safe aggregate values only: evidence counts, duplicate counts, duplicate rate, high-duplicate state, sources scanned or configured for the run, sources with accepted evidence, and a preformatted coverage label.

Update scoring to treat the quality score as the shared evidence-selection and evidence-strength input. Per-source relevance remains a component in the helper and the evidence-strength math, but it is no longer the cross-source interface. Use existing low-quality source caps and syndication grouping after quality ordering so Session 01 behavior stays intact.

### Design Patterns

* Pure helper first: implement deterministic quality math and tests before collector or scoring wiring.
* Additive schema defaults: new payload fields parse old fixtures and generated data without migration.
* Script-only raw metadata: no raw normalized URLs, private paths, source payloads, or unbounded arrays reach browser state.
* Count-only trace projection: Engine Replay receives sanitized aggregate counters, not source payload details.
* UI through view models: format duplicate-rate and coverage labels in `view-model.ts` so Sources, Engine Replay, and tests share language.

### Technology Stack

* TypeScript on Bun.
* Zod for browser payload parsing and bounded defaults.
* React 19, Tailwind CSS 4, and existing Trend Finder components for UI.
* Vitest and Testing Library for focused script and component coverage.

***

## 6. Deliverables

### Files to Create

| File                                                                           | Purpose                                                   | Est. Lines |
| ------------------------------------------------------------------------------ | --------------------------------------------------------- | ---------- |
| `scripts/extensions/trend-finder/sources/__tests__/source-quality.test.ts`     | Tests for evidence quality score components and ordering  | \~180      |
| `src/extensions/trend-finder/components/__tests__/engine-source-rail.test.tsx` | Component coverage for Engine Replay collection-health UI | \~100      |

### Files to Modify

| File                                                                | Changes                                                                  | Est. Lines |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------ | ---------- |
| `scripts/extensions/trend-finder/sources/source-quality.ts`         | Add shared quality score helper, component weights, and quality ordering | \~170      |
| `scripts/lib/ai-runtime/trend-analyst.ts`                           | Add optional evidence quality score metadata with safe defaults          | \~35       |
| `scripts/extensions/trend-finder/collector.ts`                      | Apply quality scores and build collection-health run metadata            | \~140      |
| `scripts/lib/ai-runtime/scoring.ts`                                 | Use quality-selected evidence and quality score in evidence strength     | \~80       |
| `src/extensions/trend-finder/schema.ts`                             | Add evidence quality and collection-health schemas/defaults              | \~120      |
| `src/extensions/trend-finder/view-model.ts`                         | Add collection-health and quality label view models                      | \~80       |
| `src/extensions/trend-finder/views/sources-view.tsx`                | Render duplicate-rate and source-coverage rollups                        | \~100      |
| `src/extensions/trend-finder/components/engine-source-rail.tsx`     | Render sanitized collection-health counters in Engine Replay             | \~70       |
| `src/extensions/trend-finder/signal-workbench-model.ts`             | Add optional evidence quality row value and stable sort key              | \~60       |
| `src/extensions/trend-finder/components/signal-workbench-table.tsx` | Render optional quality pill/column                                      | \~50       |
| `scripts/extensions/trend-finder/engine-trace.ts`                   | Map collection-health trace data into sanitized replay counters          | \~60       |
| `src/extensions/trend-finder/__tests__/view-model.test.ts`          | Cover legacy parsing, labels, and source coverage view models            | \~90       |
| `scripts/extensions/trend-finder/__tests__/collector.test.ts`       | Cover quality propagation and collection-health rollups                  | \~120      |
| `scripts/lib/ai-runtime/__tests__/scoring.test.ts`                  | Cover quality-selected evidence and evidence-strength behavior           | \~90       |

***

## 7. Success Criteria

### Functional Requirements

* [ ] Every evidence row carries one bounded cross-source quality score derived from existing local inputs and safe defaults.
* [ ] Evidence selection uses the shared quality score while preserving stable deterministic tie-breakers.
* [ ] Evidence-strength scoring consumes the shared quality score, with per-source relevance retained as a component rather than the public interface.
* [ ] Payload run metadata includes duplicate rate, duplicate count, high-duplicate state, sources-with-evidence count, configured source count, and a coverage label.
* [ ] Sources view and Engine Replay show duplicate pressure and source coverage without exposing raw URLs, private paths, or source payloads.
* [ ] Legacy payloads and fixtures parse with unavailable or zero defaults.

### Testing Requirements

* [ ] Unit tests written and passing for quality score components and ordering.
* [ ] Collector tests cover quality propagation, duplicate-rate rollup, high-duplicate warning state, and degraded coverage.
* [ ] Scoring tests prove quality changes evidence-strength and evidence selection without changing six-factor weights.
* [ ] Schema/view-model tests cover defaults and UI labels.
* [ ] Component tests cover Engine Replay collection-health rendering.

### Non-Functional Requirements

* [ ] No new runtime dependencies.
* [ ] Quality score is deterministic across runs for the same evidence and run timestamp.
* [ ] Browser payload additions are bounded and additive.
* [ ] Existing compliance posture is unchanged because no new collection, credential, or third-party transfer path is introduced.

### Quality Gates

* [ ] All files ASCII-encoded.
* [ ] Unix LF line endings.
* [ ] Code follows project conventions.
* [ ] Focused Vitest suites pass.
* [ ] `bun run typecheck:scripts` passes for script-side contracts touched in this session.

***

## 8. Implementation Notes

### Key Considerations

* `source-quality.ts` already exists and is imported by scoring; extend it instead of adding a parallel helper.
* Session 01 already added duplicate and syndication counters to collector and Engine Replay; this session should reuse those counters rather than recomputing identity.
* The quality score must handle missing URL, missing published date, malformed metrics, and unavailable source-local baselines without throwing.
* The Sources view should stay dense and operational: use compact tiles or status text, not a new landing-style panel.
* The optional Workbench quality column must not make the table unstable on smaller viewports; use existing overflow/table patterns.

### Potential Challenges

* Metric scale differences: source-local metrics vary widely by source. Use existing capped ratios, relevance, and normalized metrics as bounded inputs, and test low/missing-sample fallbacks.
* Duplicate-rate semantics: Session 01 counts same-source duplicates before analyst prep. Document and test the denominator so high-duplicate warnings are predictable.
* Evidence-strength drift: quality score will change scoring outputs. Keep the change inside evidence-strength inputs and leave `OPPORTUNITY_SCORE_WEIGHTS` untouched.
* UI overload: Sources already has several health panels. Add collection-health values to existing header/tile patterns rather than creating another heavy section.

### Relevant Considerations

* \[P02] **Extension payloads and labels stay bounded**: new quality and health fields must be scalar or short bounded objects with defaults.
* \[P05] **Script-only runtime boundary**: quality math and collection-health rollups stay in script/helper layers; UI consumes parsed payload fields.
* \[P24] **Browser-safe export and triage boundaries**: no private paths, raw source payloads, or normalized URLs should appear in UI or exports.
* \[P27] **Trend Finder payload growth needs release checks**: additive fields require schema defaults and payload-budget awareness in later closeout.
* \[P01] **Extract pure functions, then test, then wire**: implement quality score tests before collector and scoring changes.
* \[P27] **Deterministic fallback before AI enrichment**: this session is fully deterministic and must work when AI runtime is disabled.

### Behavioral Quality Focus

Checklist active: Yes Top behavioral risks for this session:

* Reopenable Sources and Engine Replay views could show stale health labels if collection-health fields are missing or legacy payloads are loaded.
* State-mutating refresh controls are adjacent to the new health UI, so new labels must not imply a run is complete while refresh is still in-flight.
* Quality score could over-trust one source dialect if relevance is reused without bounding and source-tier calibration.
* Component and schema consumers must exhaustively handle unavailable, degraded, and high-duplicate states.

***

## 9. Testing Strategy

### Unit Tests

* Test quality score components for source tier, recency, title length, URL presence, source-local lift, and fallback relevance.
* Test quality ordering with ties broken by relevance, timestamp, source ID, and evidence ID.
* Test collection-health helper behavior for no sources, no evidence, degraded sources, and high duplicate rate.

### Integration Tests

* Collector test verifies quality score propagation to browser and analyst evidence, collection-health metadata, and sanitized trace events.
* Scoring test verifies higher-quality evidence can change evidence-strength and selected scoring evidence while syndicated grouping still counts one shared story once.
* Schema/view-model tests verify legacy payload defaults and user-facing labels.

### Manual Testing

* Run a Trend Finder fixture or collector pass and inspect Sources plus Engine Replay for duplicate-rate, coverage, and quality labels.
* Inspect generated payload fields for bounded values and absence of raw normalized URLs, private paths, source payloads, or credential-shaped strings.

### Edge Cases

* Evidence with no URL, empty title, malformed date, or zero metrics.
* A high-relevance item from a low-quality source compared with a moderate item from a primary source.
* No configured sources, configured sources with zero evidence, and one-source degraded runs.
* Duplicate count present but source count missing in legacy or partial trace payloads.
* Legacy evidence without `qualityScore` or collection-health metadata.

***

## 10. Dependencies

### External Libraries

* None.

### Other Sessions

* **Depends on**: `phase28-session01-cross-source-signal-identity-and-dedup`
* **Depended by**: `phase28-session03-calibration-version-and-confidence-dampener`, `phase28-session04-topic-noise-gate-and-visibility-bands`, and later Phase 28 decision/UI sessions that rely on quality and collection-health signals.

***

## Next Steps

Run the implement workflow step to begin AI-led implementation.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase28-session02-signal-quality-score-and-collection-health/spec.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.