> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase15-session02-live-data-merge-boundary/spec.md).

# Session Specification

**Session ID**: `phase15-session02-live-data-merge-boundary` **Phase**: 15 - Scheduler Config And Aggregate Split **Status**: Completed **Created**: 2026-05-31

***

## 1. Session Overview

This session establishes the generated live-data write boundary needed before AI OS can split the broad aggregate scheduler job into independently scheduled host/local and Trend Finder refresh jobs. The current full aggregate path writes one `src/data/live-data.json` artifact; future scoped jobs need to update their owned branches without erasing the other branch.

The session defines branch ownership, shared generated-data locking, merge behavior, validation, privacy checks, and runbook guidance. It builds on the Session 01 scheduler config contract and keeps `bun run aggregate` compatible while preparing safe handoff points for Sessions 03 through 05.

Existing live-data write helper work in the repository should be audited and completed against this specification instead of replaced with a parallel writer. The outcome is a single reviewed helper surface that later scheduler handlers can call.

***

## 2. Objectives

1. Document host/local and Trend Finder branch ownership for generated live data.
2. Provide a shared generated-data write lock separate from per-job scheduler run locks.
3. Provide a validation-backed merge writer that preserves untouched branches.
4. Prove branch preservation, invalid-base fallback, lock blocking, stale-lock recovery, and privacy rejection with focused tests.

***

## 3. Prerequisites

### Required Sessions

* [x] `phase15-session01-scheduler-config-contract` - Provides scheduler config parsing, sanitized status behavior, and the reviewed registry contract.

### Required Tools/Knowledge

* Bun script test workflow.
* Current `scripts/aggregate.ts` live-data output shape.
* `src/lib/validate-live-data.ts` read-side validation behavior.
* Existing scheduler lock and private path helpers.

### Environment Requirements

* Repository dependencies installed with Bun.
* No private `src/data/live-data.json` contents committed or copied into docs.
* Tests must use fixtures or temporary files, not operator private data.

***

## 4. Scope

### In Scope (MVP)

* AI OS scheduler jobs can share a generated-data write lock - add or verify a `generated-live-data` lock path under the private scheduler lock root.
* AI OS host/local refresh can own root live-data branches - document and test preservation of `extensions` during host/local scoped writes.
* Trend Finder refresh can own `extensions.items.trend-finder` - document and test preservation of host/local root branches and other extension items.
* Full aggregate remains a compatibility writer - keep existing full output validation behavior unchanged.
* Scoped writers can recover from invalid existing live data - use `src/data/live-data.example.json` as a validated merge base.
* Generated output remains browser-safe - reject secrets, raw prompts, command output, auth payloads, source dumps, and private paths before write.
* Operators can understand the boundary - update scheduled aggregate runbook ownership and recovery guidance.

### Out of Scope (Deferred)

* New scheduler job handlers - *Reason: Session 04 and Session 05 own handler registration and package scripts.*
* Aggregate orchestration extraction - *Reason: Session 03 owns reusable host/local and extension orchestration.*
* SQLite storage or projection changes - *Reason: Future migration work is planned separately.*
* Timer rendering or installation - *Reason: Session 07 is the only timer-management implementation pass.*

***

## 5. Technical Approach

### Architecture

Use a script-side live-data writer under `scripts/lib/` as the only producer gate for generated browser data. Full writes validate and write a complete artifact. Scoped writes load a valid merge base, replace only the branch owned by the requested scope, re-run producer-side validation, acquire the shared generated-data lock, and write the merged file.

The shared generated-data lock reuses scheduler lock semantics for owner records, PID liveness, stale recovery, private path confinement, and release safety, but it uses the `generated-live-data` lock ID so file-level contention is distinct from per-job run-lock contention.

### Design Patterns

* Single producer gate: Prevent later scheduler handlers from writing `src/data/live-data.json` directly.
* Ownership map: Make branch responsibility explicit and testable.
* Validate before write: Reuse `validateLiveData` and producer privacy checks before any output reaches disk.
* Typed degradation: Fall back from invalid existing live data to the committed example shape instead of crashing scoped writes unnecessarily.

### Technology Stack

* TypeScript scripts under `scripts/lib/`.
* Bun and Vitest for focused script tests.
* Existing scheduler path, lock, redaction, and validation helpers.
* LiveData contract from `src/lib/live-data-types.ts`.

***

## 6. Deliverables

### Files to Create

| File          | Purpose                                                                                | Est. Lines |
| ------------- | -------------------------------------------------------------------------------------- | ---------- |
| None expected | Use existing script, test, and docs locations unless implementation finds a clear gap. | \~0        |

### Files to Modify

| File                                                      | Changes                                                                                                                    | Est. Lines |
| --------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------- |
| `scripts/lib/aggregate-live-data-write.ts`                | Finalize ownership map, merge writer, lock wrapper, fallback loading, validation, and privacy rejection behavior.          | \~120      |
| `scripts/lib/scheduler/locks.ts`                          | Verify shared generated-data lock behavior stays compatible with scheduler stale-lock semantics.                           | \~30       |
| `scripts/lib/scheduler/index.ts`                          | Export helpers only if later sessions need a stable import path.                                                           | \~10       |
| `scripts/lib/__tests__/aggregate-live-data-write.test.ts` | Add or tighten branch preservation, invalid base, privacy, lock blocking, stale recovery, and distinct lock path coverage. | \~180      |
| `scripts/lib/__tests__/scheduler-locks.test.ts`           | Add or adjust scheduler lock assertions only if shared lock semantics require it.                                          | \~40       |
| `scripts/aggregate.ts`                                    | Keep full aggregate validation behavior compatible; route direct full writes through the producer gate if needed.          | \~30       |
| `docs/runbooks/scheduled-aggregate.md`                    | Document live-data ownership, shared lock path, invalid-base fallback, and privacy boundaries.                             | \~80       |

***

## 7. Success Criteria

### Functional Requirements

* [ ] Host/local scoped writes preserve existing Trend Finder extension data.
* [ ] Trend Finder scoped writes preserve host/local root data and other extension items.
* [ ] Scoped writes use a generated-data lock distinct from per-job scheduler run locks.
* [ ] Invalid existing live data falls back to `src/data/live-data.example.json` before merging.
* [ ] Full aggregate writes still validate and behave compatibly.

### Testing Requirements

* [ ] Unit tests written and passing.
* [ ] Manual testing completed with focused Bun commands.

### Non-Functional Requirements

* [ ] Private paths, raw command bodies, prompts, auth JSON, source dumps, and credentials are rejected or kept out of emitted status/log surfaces.
* [ ] Lock paths remain confined under the private scheduler root.
* [ ] Merge output is deterministic and bounded to reviewed ownership branches.

### Quality Gates

* [ ] All files ASCII-encoded.
* [ ] Unix LF line endings.
* [ ] Code follows project conventions.

***

## 8. Implementation Notes

### Key Considerations

* Treat existing `scripts/lib/aggregate-live-data-write.ts` as the likely home for this work; remove duplication rather than adding another writer.
* `src/data/live-data.json` is generated and private. Tests should use temp roots and fixture data.
* Keep `src/data/live-data.example.json` as the committed fallback shape for invalid or missing live-data base files.
* Do not expand Trend Finder source support or scheduler handler registration in this session.

### Potential Challenges

* Branch map drift: Mitigate by keeping ownership arrays typed against `keyof LiveData` and testing representative owned branches.
* Validation mutation: Mitigate by failing if read-side validation would repair producer output.
* Lock cleanup on failure: Mitigate with `try/finally` release behavior and explicit tests.
* Existing partial implementation: Mitigate by auditing behavior against this spec before adding code.

### Relevant Considerations

* \[P11] **Scheduler state/log privacy boundary**: Generated-data lock outcomes may be surfaced as safe metadata only.
* \[P12] **Dream private-output boundary**: Dream remains private output; this session must not add browser or service expansion.
* \[P14] **Backtest archives stay private but can grow**: Scoped writes must not widen private archive exposure.
* \[P01] **Extract pure functions, then test, then wire**: Keep merge behavior pure and well-tested before later scheduler handlers call it.

### Behavioral Quality Focus

Checklist active: Yes Top behavioral risks for this session:

* Concurrent scoped jobs overwrite each other's generated data.
* Invalid or missing existing live data causes scoped jobs to erase preserved branches.
* Validation or logs expose private paths, prompts, source dumps, or auth data.

***

## 9. Testing Strategy

### Unit Tests

* Test producer validation accepts complete LiveData output without read-side mutation.
* Test host/local scoped writes preserve Trend Finder extension data.
* Test Trend Finder scoped writes preserve host/local and other extension data.
* Test invalid existing live data falls back to the example file.
* Test shared lock blocking and stale recovery with temp scheduler roots.
* Test privacy rejection for private field names and credential-like strings.

### Integration Tests

* Run focused script tests for live-data writer and scheduler locks.
* Run script typecheck to catch exported type or import drift.

### Manual Testing

* Inspect `docs/runbooks/scheduled-aggregate.md` for accurate current behavior and no claims that Session 04 or Session 05 handlers already exist.
* Confirm `bun run aggregate` remains documented as the compatibility command.

### Edge Cases

* Existing live-data file is missing.
* Existing live-data file is invalid JSON or fails producer validation.
* Example live-data file is invalid or missing.
* Lock owner mismatch is preserved.
* A write fails after lock acquisition and still releases the lock.
* Scoped Trend Finder output omits `extensions.items.trend-finder`.

***

## 10. Dependencies

### External Libraries

* No new external libraries expected.

### Other Sessions

* **Depends on**: `phase15-session01-scheduler-config-contract`
* **Depended by**: `phase15-session03-aggregate-orchestration-refactor`, `phase15-session04-agent-aggregate-scheduler-job`, `phase15-session05-trend-finder-scheduler-job`, `phase15-session06-cross-job-integration-dream-freshness`

***

## Next Steps

Run the implement workflow step to begin AI-led implementation.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase15-session02-live-data-merge-boundary/spec.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.