> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase27-session11-theme-rollups-and-outlier-ideas/spec.md).

# Session Specification

**Session ID**: `phase27-session11-theme-rollups-and-outlier-ideas` **Phase**: 27 - Trend Finder Alpha Radar Adoption **Status**: Not Started **Created**: 2026-06-13

***

## 1. Session Overview

This session adds a theme layer above Trend Finder topics and adds bounded per-outlier creator ideas for the strongest outlier evidence rows. It ports the useful Alpha Radar theme row and creator idea concepts while preserving Trend Finder's stricter requirements: additive schema defaults, deterministic fallbacks, browser-safe payloads, and no invented metrics, sources, dates, or URLs.

Theme rollups make dense topic output easier to scan by grouping related topics under analyst-provided labels when valid, or deterministic labels derived from Creator Lens focus terms and topic keyword overlap when AI output is missing or invalid. Signal Workbench becomes the primary surface for these groups, with a flat-view toggle so operators can still inspect the original row list.

Outlier ideas attach one bounded creator idea to only the top-N source-local outlier rows. The enrichment path reuses the existing private enrichment cache so unchanged outlier rows can reuse prior summaries and avoid repeated spend. The embedding-based fallback clustering idea stays investigate-only in this session and is recorded as an ADR; no embedding model dependency ships here.

***

## 2. Objectives

1. Add validated theme metadata to topic analysis with deterministic fallback labels and stable group ordering.
2. Render Signal Workbench theme group headers with a flat-view toggle and deterministic row ordering.
3. Generate top-N per-outlier creator ideas with bounded text, explicit `ai` or `fallback` provenance, and enrichment-cache reuse.
4. Record an embedding-based fallback clustering decision with a ship or no-ship recommendation and no runtime dependency change.

***

## 3. Prerequisites

### Required Sessions

* [x] `phase27-session03-data-driven-radar-aliases-and-watching-state` - Workbench outlier preset and source-local outlier surface exist.
* [x] `phase27-session09-competitor-lens-and-creator-angle-pack` - Analyst prompt extension, validation, and fallback copy patterns exist.
* [x] `phase27-session10-demand-centers` - Demand brief merge and bounded analyst optional-output patterns are available for reuse.

### Required Tools/Knowledge

* Bun 1.3.14, TypeScript, Vitest, and existing Trend Finder focused tests.
* Zod additive default patterns in `src/extensions/trend-finder/schema.ts`.
* Analyst validation, repair, and optional-output merge patterns in `scripts/lib/ai-runtime/trend-analyst.ts`.
* Private enrichment-cache helper in `scripts/extensions/trend-finder/enrichment-cache.ts`.
* Signal Workbench row, filter, and table model in `src/extensions/trend-finder/signal-workbench-model.ts`.

### Environment Requirements

* No live AI runtime credentials are required; deterministic fallback must produce useful theme labels and outlier ideas without external services.
* No new public source adapters or source calls are introduced.
* Generated private Trend Finder cache artifacts stay out of git.

***

## 4. Scope

### In Scope (MVP)

* Trend Finder can label topic themes - Add bounded `themeLabel` metadata from analyst output and deterministic fallback labels from Creator Lens focus terms, topic names, aliases, and summaries.
* Trend Finder can group Workbench rows by theme - Build stable group headers, counts, top score summaries, and deterministic ordering with a flat-view toggle.
* Trend Finder can enrich top-N outlier rows - Select a capped set of source-local outlier evidence rows, create one bounded creator idea per row, and publish `ai` or `fallback` provenance.
* Trend Finder can reuse outlier idea cache entries - Key unchanged outlier rows through the existing enrichment cache so identical candidates do not re-bill.
* Trend Finder records the embedding fallback decision - Create an ADR that compares local token similarity, local embedding models, and no dependency change, then records the recommendation.
* Focused tests cover theme fallback determinism, analyst validation, outlier idea bounds, cache hits, Workbench grouping, fixtures, and ASCII compliance.

### Out of Scope (Deferred)

* Shipping an embedding model dependency - *Reason: dependency, model size, browser payload, and local runtime costs need a dedicated future decision.*
* Replacing analyst topic clustering - *Reason: this session adds a grouping layer above topics, not a new topic clustering engine.*
* Ideas for every outlier row - *Reason: MVP is spend-bounded top-N only.*
* New public sources or new source calls - *Reason: source expansion requires compliance-first review in a future phase.*
* Hotlinked thumbnails or media ideas - *Reason: media remains compliance gated and outside this session.*

***

## 5. Technical Approach

### Architecture

Add a script-side `theme-rollups` helper that accepts current topics and the Creator Lens, then emits stable theme metadata. Analyst output may provide a bounded theme label per topic, but fallback remains authoritative when output is missing, empty, too long, or too generic. Fallback derives normalized keyword sets from Creator Lens focus terms, topic names, aliases, summaries, and creator copy, then assigns each topic to the highest-overlap theme with stable tie-breakers.

Add a script-side `outlier-ideas` helper that selects top-N source-local outlier evidence rows using the existing Workbench outlier criteria and stable ordering. The helper builds safe cache candidates, deterministic fallback idea text from lens format plus entity/topic context, and bounded browser payload fields. The collector merges cache hits, analyst idea output, and fallback ideas without exposing cache paths, raw prompts, private summaries, or provider responses.

Browser code consumes only validated schema fields. Signal Workbench projects theme groups and outlier ideas through its model, controls, and table components. The flat-view toggle controls grouping without changing the underlying row list, and search/filter behavior remains deterministic across grouped and flat modes.

### Design Patterns

* Additive schema defaults: Legacy payloads parse with empty theme labels and no outlier ideas.
* Script-only runtime boundary: Theme fallback, outlier selection, cache candidates, and fallback idea text stay under `scripts/`.
* Existing cache boundary: Reuse `enrichment-cache.ts` with a distinct enrichment type and sanitized summaries.
* Project before rendering: Workbench renders projected rows and groups, not raw collector intermediates.
* Deterministic ordering: Theme groups, top-N outliers, and flat rows sort with stable score, count, label, and ID tie-breakers.
* Bounded payloads: Theme labels, idea strings, top-N counts, citations, and cache summaries are capped.

### Technology Stack

* TypeScript with existing project path aliases and strict local contracts.
* Zod for runtime schema validation and browser-safe defaults.
* React 19 and Tailwind CSS 4 for Workbench controls and table rendering.
* Bun and Vitest for focused script, schema, collector, model, and component tests.

***

## 6. Deliverables

### Files to Create

| File                                                              | Purpose                                                                                            | Est. Lines |
| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------- |
| `scripts/extensions/trend-finder/theme-rollups.ts`                | Assign validated or fallback theme metadata to topics with stable group ordering.                  | \~220      |
| `scripts/extensions/trend-finder/outlier-ideas.ts`                | Select top-N outlier evidence, build cache candidates, fallback ideas, and bounded payload fields. | \~260      |
| `scripts/extensions/trend-finder/__tests__/theme-rollups.test.ts` | Focused tests for keyword overlap, analyst labels, fallback labels, caps, and ordering.            | \~200      |
| `scripts/extensions/trend-finder/__tests__/outlier-ideas.test.ts` | Focused tests for top-N selection, fallback text, cache summaries, provenance, and bounds.         | \~240      |
| `docs/adr/0002-trend-finder-embedding-fallback-clustering.md`     | Investigate-only decision record for embedding-based fallback clustering.                          | \~90       |

### Files to Modify

| File                                                                              | Changes                                                                                                           | Est. Lines |
| --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ---------- |
| `src/extensions/trend-finder/schema.ts`                                           | Add topic theme metadata, outlier idea schema, caps, defaults, and reference-safe parsing.                        | \~150      |
| `scripts/lib/ai-runtime/trend-analyst.ts`                                         | Add optional theme labels and outlier idea outputs with strict validation and merge helpers.                      | \~180      |
| `scripts/extensions/trend-finder/topics.ts`                                       | Add deterministic fallback theme labels to generated fallback topics.                                             | \~80       |
| `scripts/extensions/trend-finder/collector.ts`                                    | Wire theme assignment, top-N outlier idea candidates, cache reuse, warnings, and payload publication.             | \~220      |
| `src/extensions/trend-finder/signal-workbench-model.ts`                           | Project theme groups, flat/grouped state support, outlier idea labels, search text, and ordering.                 | \~180      |
| `src/extensions/trend-finder/components/signal-workbench-controls.tsx`            | Add accessible grouped/flat view controls.                                                                        | \~70       |
| `src/extensions/trend-finder/components/signal-workbench-table.tsx`               | Render theme group headers and outlier idea details with accessible labels.                                       | \~160      |
| `src/extensions/trend-finder/views/signal-workbench-view.tsx`                     | Wire grouping state, empty states, counts, and grouped table data.                                                | \~120      |
| `src/extensions/trend-finder/view-model.ts`                                       | Project theme and outlier idea labels for shared dashboard consumers.                                             | \~70       |
| `src/extensions/trend-finder/fixtures.ts`                                         | Add bounded fixture data for theme labels, grouped rows, and outlier ideas.                                       | \~90       |
| `src/lib/__tests__/trend-finder-schema.test.ts`                                   | Add schema default, cap, and invalid reference coverage.                                                          | \~100      |
| `scripts/lib/ai-runtime/__tests__/trend-analyst.test.ts`                          | Add analyst theme and outlier idea validation, invented-reference rejection, and fallback acceptance tests.       | \~130      |
| `scripts/extensions/trend-finder/__tests__/collector.test.ts`                     | Add collector integration coverage for theme publication, cache hits, fallback ideas, and no-new-source behavior. | \~150      |
| `src/extensions/trend-finder/__tests__/signal-workbench-model.test.ts`            | Add grouped/flat ordering, facet, search, and top outlier idea projection coverage.                               | \~120      |
| `src/extensions/trend-finder/components/__tests__/signal-workbench-view.test.tsx` | Add Workbench group header, flat toggle, and outlier idea rendering coverage.                                     | \~120      |
| `scripts/extensions/trend-finder/__tests__/enrichment-cache.test.ts`              | Add sanitized summary regression coverage for the outlier idea enrichment type.                                   | \~80       |

***

## 7. Success Criteria

### Functional Requirements

* [ ] Theme labels are validated when analyst-provided and deterministic when fallback-generated.
* [ ] Workbench grouped mode renders stable theme headers and flat mode restores the ungrouped row list without losing filters or sort state.
* [ ] Outlier ideas appear only on top-N source-local outlier rows.
* [ ] Unchanged outlier idea candidates hit the enrichment cache and do not require repeated AI enrichment.
* [ ] Every outlier idea carries explicit `ai` or `fallback` provenance and bounded display text.
* [ ] The embedding fallback clustering ADR records a ship/no-ship recommendation without adding a dependency.

### Testing Requirements

* [ ] Unit tests cover theme fallback, analyst validation, outlier idea selection, cache hit behavior, schema defaults, and payload caps.
* [ ] Component/model tests cover Workbench grouped mode, flat mode, group headers, outlier idea rendering, and empty states.
* [ ] Manual validation covers Workbench grouping, cross-source outlier preset, and fallback-only behavior.

### Non-Functional Requirements

* [ ] Browser payload remains bounded under the shared 1 MB extension limit.
* [ ] No raw prompts, provider responses, private cache paths, local paths, or source payloads enter browser-visible output.
* [ ] No new public source adapter, source call, or embedding dependency is introduced.

### Quality Gates

* [ ] All files ASCII-encoded
* [ ] Unix LF line endings
* [ ] Code follows project conventions

***

## 8. Implementation Notes

### Key Considerations

* Theme fallback should prefer Creator Lens focus terms when they overlap with topic text, then fall back to topic-derived labels such as the strongest shared keyword.
* Outlier idea selection should reuse source-local outlier criteria already used by Signal Workbench so the UI and collector agree on what qualifies.
* Cache summaries must stay sanitized and must not expose private cache roots, raw metadata fingerprints, prompts, provider responses, or local paths.
* The ADR should document why local token similarity remains the fallback for now unless a future phase explicitly accepts embedding dependency costs.

### Potential Challenges

* Analyst output may provide generic theme labels: Reject empty, oversized, or low-signal labels and fall back deterministically.
* Top-N outlier ideas could add spend: Keep the cap small, cache unchanged candidates, and make fallback useful without AI.
* Grouped Workbench rendering could disrupt filters: Keep filtering over the flat row set and group only the filtered rows.
* Payload growth could exceed budget: Cap labels, idea text, and candidate counts, then validate with populated fixtures.

### Relevant Considerations

* \[P02] **Extension payloads and demo labels stay bounded**: Theme metadata and outlier idea fields must be capped and additive.
* \[P15] **Aggregate collection must stay budgeted**: Top-N outlier ideas must respect runtime budgets and degrade to fallback.
* \[P24] **Browser-safe export and triage boundaries**: Outlier ideas and cache state must not expose private paths or raw source payloads.
* \[P24] **Additive defaults keep legacy payloads alive**: Schema changes must preserve old fixtures and generated reports.
* \[P24] **Deterministic ordering helps dense triage**: Theme groups, outlier rows, and Workbench flat/grouped modes need stable tie-breakers.
* \[P05] **Script-only runtime boundary**: Theme and outlier derivation stay under `scripts/`; browser code reads validated payload fields only.

### Behavioral Quality Focus

Checklist active: Yes Top behavioral risks for this session:

* Grouped Workbench mode can hide or reorder rows unexpectedly after filters, sorting, or re-entry.
* Outlier idea enrichment can duplicate spend or publish stale/private cache details if cache boundaries are not strict.
* Analyst theme and idea output can invent unsupported labels or references without explicit schema validation and fallback.

***

## 9. Testing Strategy

### Unit Tests

* Test `theme-rollups.ts` for analyst label acceptance, fallback keyword overlap, generic-label rejection, stable IDs, caps, and deterministic ordering.
* Test `outlier-ideas.ts` for top-N selection, source-local outlier criteria, cache candidate shape, fallback text bounds, provenance, and sanitized summary values.
* Test schema and analyst validation for legacy defaults, invalid references, unknown theme or evidence IDs, and invented metadata rejection.

### Integration Tests

* Test collector output for theme metadata, top-N outlier idea publication, enrichment-cache hit reuse, fallback warnings, and no-new-source behavior.
* Test Workbench model output for grouped and flat modes, filters, search, facets, group counts, and deterministic sort behavior.

### Manual Testing

* Open Signal Workbench with populated fixture data and verify grouped mode, flat mode, cross-source outlier preset, theme headers, and outlier idea details.
* Verify fallback-only output by running tests or fixtures without live AI credentials.
* Review the embedding fallback clustering ADR for a clear recommendation and no dependency changes.

### Edge Cases

* Legacy payloads without theme labels or outlier ideas.
* Analyst output with unknown evidence IDs, unknown topic IDs, generic theme labels, oversized idea text, invented URLs, invented dates, or invented metrics.
* No qualifying source-local outlier rows.
* Cache unavailable, degraded, stale, skipped, or sanitized-summary rejection.
* Filters that produce empty grouped results.

***

## 10. Dependencies

### External Libraries

* None. This session must not add an embedding model or new runtime dependency.

### Other Sessions

* **Depends on**: `phase27-session03-data-driven-radar-aliases-and-watching-state`, `phase27-session09-competitor-lens-and-creator-angle-pack`, `phase27-session10-demand-centers`
* **Depended by**: `phase27-session12-documentation-validation-and-release`

***

## Next Steps

Run the implement workflow step to begin AI-led implementation.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase27-session11-theme-rollups-and-outlier-ideas/spec.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.