> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_27/session_11_theme_rollups_and_outlier_ideas.md).

# Session 11: Theme Rollups And Outlier Ideas

**Session ID**: `phase27-session11-theme-rollups-and-outlier-ideas` **Status**: Not Started **Estimated Tasks**: \~12-25 **Estimated Duration**: 2-4 hours

***

## Objective

Add a theme grouping layer above topics (analyst-provided labels with deterministic keyword-overlap fallback), generate bounded per-outlier creator ideas for top outlier rows with enrichment caching, and complete an investigate-only spike on embedding-based fallback clustering.

***

## Source Mapping (from the Alpha Radar improvement map, items 8.4, 6.2)

### Theme / cluster rollups (item 8.4, lower priority)

* **Alpha Radar:** trends carry `cluster_id` / `cluster_label` (e.g. "LLM evaluation resolution and statistics") and a `meta_trend` slot; the dashboard renders theme rows (`clusterThemeRows()`, `renderThemes()` in `EXAMPLES/ai-alpha-radar-submission/public/dashboard.html`). `cluster_centroids` shows the backend clusters with 384-dim embeddings.
* **Trend Finder today:** topics are flat; the analyst clusters evidence into topics, and the deterministic fallback groups by source ID (`docs/extensions/trend-finder-runtime-and-provenance.md`), which is the weakest part of fallback output. No layer groups related topics.
* **Improvement (two separable pieces):**
  1. Theme labels above topics: analyst-provided `themeLabel` with deterministic fallback from shared topic-focus keyword overlap; render as group headers in Workbench.
  2. Better fallback clustering: replace group-by-source with local text-similarity clustering (token/embedding based). A local embedding model is a new dependency and needs its own decision; flag as investigate-only in this session -- document the outcome, do not ship the dependency.
* **Implement in:** `scripts/lib/ai-runtime/trend-analyst.ts`, `scripts/extensions/trend-finder/topics.ts`, schema in `src/extensions/trend-finder/schema.ts`, Workbench grouping in `src/extensions/trend-finder/signal-workbench-model.ts`.

### Per-outlier creator idea (item 6.2, depends on the Session 03 outlier preset)

* **Alpha Radar:** each outlier carries `creator_idea` and `idea_type` (`adjacent` vs general).
* **Trend Finder today:** creator copy exists per TOPIC, not per evidence row.
* **Improvement:** optional analyst enrichment producing one bounded idea string per top-N outlier rows, with deterministic fallback (template from lens format + entity name). Only worth doing after the outlier surface exists (Session 03).
* **Implement in:** `scripts/lib/ai-runtime/trend-analyst.ts` or a small dedicated enrichment stage in `scripts/extensions/trend-finder/collector.ts`; cache results through `scripts/extensions/trend-finder/enrichment-cache.ts` so unchanged outliers do not re-bill.

***

## Scope

### In Scope (MVP)

* `themeLabel` in analyst output with validation and deterministic keyword-overlap fallback from Creator Lens topic-focus terms
* Workbench theme group headers with stable ordering and a flat-view toggle
* Top-N per-outlier idea enrichment (bounded N, bounded length) with template-based fallback and `ai`/`fallback` provenance
* Enrichment-cache keying for outlier ideas so unchanged rows reuse cached summaries
* Investigate-only spike note on embedding-based fallback clustering (decision doc, no dependency shipped)
* Tests for fallback grouping, validator behavior, cache hits, and bounds

### Out of Scope

* Shipping an embedding model dependency (decision deferred per the mapping)
* Changing how the analyst clusters evidence into topics
* Idea text for every outlier row (top-N only, spend-bounded)

***

## Prerequisites

* [ ] Session 03 completed (outlier preset exists to attach ideas to)
* [ ] Session 09 completed (analyst prompt extension pattern from angle pack)

***

## Deliverables

1. Theme labels end to end with fallback grouping and Workbench headers
2. Per-outlier idea enrichment with caching and fallback templates
3. Embedding-clustering spike writeup with a recommendation
4. Tests for grouping determinism, validation, caching, and bounds

***

## Success Criteria

* [ ] Theme fallback produces stable, deterministic groups from lens focus terms
* [ ] Outlier ideas appear only on top-N rows; unchanged rows hit the enrichment cache (no re-billing)
* [ ] All idea/theme text validates against invented-content rules; fallback provenance is visible
* [ ] Spike concludes with a documented ship/no-ship recommendation for local embeddings
* [ ] Workbench remains performant and deterministic with grouping enabled


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_27/session_11_theme_rollups_and_outlier_ideas.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.