> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase28-session01-cross-source-signal-identity-and-dedup/tasks.md).

# Task Checklist

**Session ID**: `phase28-session01-cross-source-signal-identity-and-dedup` **Total Tasks**: 23 **Estimated Duration**: 3-4 hours **Created**: 2026-06-14

***

## Legend

* `[x]` = Completed
* `[ ]` = Pending
* `[P]` = Parallelizable (can run with other \[P] tasks)
* `[SNNMM]` = Session reference (NN=phase number, MM=session number)
* `TNNN` = Task ID

***

## Progress Summary

| Category       | Total  | Done   | Remaining |
| -------------- | ------ | ------ | --------- |
| Setup          | 3      | 3      | 0         |
| Foundation     | 5      | 5      | 0         |
| Implementation | 10     | 10     | 0         |
| Testing        | 5      | 5      | 0         |
| **Total**      | **23** | **23** | **0**     |

***

## Setup (3 tasks)

Initial configuration and environment preparation.

* [x] T001 \[S2801] Verify Phase 27 collector, scoring, schema, and trace baseline files before editing (`scripts/extensions/trend-finder/collector.ts`)
* [x] T002 \[S2801] Verify Trends-Finderz normalization and fingerprint reference behavior for local adaptation (`EXAMPLES/trends-finderz/lib/signals/signal-fingerprint.ts`)
* [x] T003 \[S2801] Create the identity helper test target and confirm source-helper import conventions (`scripts/extensions/trend-finder/sources/__tests__/signal-identity.test.ts`)

***

## Foundation (5 tasks)

Core structures and base implementations.

* [x] T004 \[S2801] \[P] Create URL normalization helpers with malformed-input fallback and deterministic query cleanup (`scripts/extensions/trend-finder/sources/signal-identity.ts`)
* [x] T005 \[S2801] \[P] Create content hash and per-source fingerprint helpers with stable SHA-256 output and empty-field handling (`scripts/extensions/trend-finder/sources/signal-identity.ts`)
* [x] T006 \[S2801] \[P] Define identity, duplicate summary, and syndication group types with browser-safe count-only outputs (`scripts/extensions/trend-finder/sources/signal-identity.ts`)
* [x] T007 \[S2801] Add focused helper tests for normalization, hash stability, fingerprint source scoping, and malformed input (`scripts/extensions/trend-finder/sources/__tests__/signal-identity.test.ts`)
* [x] T008 \[S2801] Extend analyst evidence types with optional script-only identity metadata and exhaustive fallback handling (`scripts/lib/ai-runtime/trend-analyst.ts`)

***

## Implementation (10 tasks)

Main feature implementation.

* [x] T009 \[S2801] Implement within-source duplicate detection and deterministic first-row retention with counted duplicate summaries (`scripts/extensions/trend-finder/sources/signal-identity.ts`)
* [x] T010 \[S2801] Implement cross-source syndication grouping with source sibling counts and no raw URL browser exposure (`scripts/extensions/trend-finder/sources/signal-identity.ts`)
* [x] T011 \[S2801] Wire identity metadata into collected browser and analyst evidence after source-local enrichment with schema-validated input and explicit warning mapping (`scripts/extensions/trend-finder/collector.ts`)
* [x] T012 \[S2801] Drop same-source duplicate browser and analyst evidence in sync with duplicate-trigger prevention during the collect pass (`scripts/extensions/trend-finder/collector.ts`)
* [x] T013 \[S2801] Emit duplicate and syndication trace events plus high-duplicate-rate warnings with sanitized aggregate counters (`scripts/extensions/trend-finder/collector.ts`)
* [x] T014 \[S2801] Add additive bounded syndication count defaults for evidence and topic payloads with legacy parsing preserved (`src/extensions/trend-finder/schema.ts`)
* [x] T015 \[S2801] Update scoring evidence context to build syndicated story groups for volume, velocity, and diversity-sensitive scoring (`scripts/lib/ai-runtime/scoring.ts`)
* [x] T016 \[S2801] Preserve full evidence rows for evidence links, source breakdowns, lifecycle, and role coverage while using grouped scoring inputs (`scripts/lib/ai-runtime/scoring.ts`)
* [x] T017 \[S2801] Map duplicate and syndication counters into engine trace schemas and builders with private-path and raw URL leak checks (`src/extensions/trend-finder/engine-trace.ts`)
* [x] T018 \[S2801] Map collector trace events into engine replay evidence counters with sanitized aggregate fields (`scripts/extensions/trend-finder/engine-trace.ts`)

***

## Testing (5 tasks)

Verification and quality assurance.

* [x] T019 \[S2801] Add collector tests for same-source dedup, high-rate warnings, syndication counts, and browser-safe output (`scripts/extensions/trend-finder/__tests__/collector.test.ts`)
* [x] T020 \[S2801] Add scoring tests proving syndicated stories do not inflate evidence volume, momentum, or diversity-sensitive score inputs (`scripts/lib/ai-runtime/__tests__/scoring.test.ts`)
* [x] T021 \[S2801] Add engine trace tests for sanitized duplicate and syndicated group counters (`scripts/extensions/trend-finder/__tests__/engine-trace.test.ts`)
* [x] T022 \[S2801] Run focused Vitest suites for identity, collector, scoring, schema, and trace behavior (`package.json`)
* [x] T023 \[S2801] Validate ASCII encoding and record manual verification notes for the session artifacts (`.spec_system/specs/phase28-session01-cross-source-signal-identity-and-dedup/implementation-notes.md`)

***

## Completion Checklist

Before marking session complete:

* [x] All tasks marked `[x]`
* [x] All tests passing
* [x] All files ASCII-encoded
* [x] implementation-notes.md updated
* [x] Ready for the validate workflow step

***

## Next Steps

Run the validate workflow step to verify session completeness.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase28-session01-cross-source-signal-identity-and-dedup/tasks.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.