> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_15/prd_phase_15.md). # PRD Phase 15: Scheduler Config And Aggregate Split **Status**: Complete **Sessions**: 8 completed **Estimated Duration**: 8-12 days **Captured Source Plan**: 2026-05-27 scheduler config and aggregate split plan, folded into this phase record on 2026-05-31 **Progress**: 8/8 sessions complete (100%). *** ## Overview Phase 15 splits the current broad scheduler aggregate path into scoped local-agent and Trend Finder refresh jobs. The phase keeps the static scheduler registry as the reviewed contract, adds repo-local scheduler settings under `data/`, and introduces safe generated-data merge boundaries so independently scheduled jobs do not erase each other's live-data branches. This phase record and the session stubs in this directory are the source of truth for the completed scheduler config, generated-data merge boundary, and aggregate orchestration extraction. The former ongoing-project plan has been folded into this phase so it can be deleted without losing requirements. Phase 15 is finalized after Session 03. The remaining split-job wiring, cross-job validation, timer rendering, and final documentation sweep from the original eight-session plan are carried forward as future phase work instead of remaining open in this phase. *** ## Progress Tracker | Session | Name | Status | Est. Tasks | Validated | | ------- | -------------------------------------------------- | -------- | ---------- | --------- | | 01 | Scheduler Config Contract And Status Read Path | Complete | \~12 | Yes | | 02 | Live Data Merge Boundary And Shared Write Lock | Complete | \~16 | Yes | | 03 | Aggregate Orchestration Refactor | Complete | \~16 | Yes | | 04 | Agent Aggregate Scheduler Job | Complete | \~14 | Yes | | 05 | Trend Finder Scheduler Job | Complete | \~15 | Yes | | 06 | Cross-Job Integration And Dream Freshness Boundary | Complete | \~14 | Yes | | 07 | Timer Unit Rendering And Reconciliation | Complete | \~12 | Yes | | 08 | Documentation Compatibility And Validation Sweep | Complete | \~12 | Yes | *** ## Completion Session Plan Use this run order to complete the phase. The split intentionally separates write-boundary work, aggregate extraction, scheduler jobs, integration validation, and optional timer management so risky changes have clear handoff gates. | Session | Focus | Covers | Exit Gate | | ------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 01 | Scheduler config contract and status read path | Private config parsing, status fallback, warning behavior, docs, and focused config/status tests | `data/ai-os.scheduler.example.json`, private config parsing, status fallback, warning behavior, docs, and focused config/status tests are complete. | | 02 | Live-data ownership map, merge writer, and shared generated-data lock | Detailed live-data write boundary | Host/local and extension branch ownership is documented in code, scoped writes preserve the other branch, shared lock behavior is tested, and full aggregate writes still validate unchanged. | | 03 | Aggregate orchestration refactor with compatibility preserved | First half of the original agent aggregate work | `scripts/aggregate.ts` exposes reusable host/local and extension orchestration without changing `bun run aggregate` output or public behavior. | | 04 | `agent-aggregate` scheduler job | Second half of the original agent aggregate work | Registry metadata, handler wiring, package scripts, status copy, run state, locks, and tests prove the agent job refreshes host/local data without running Trend Finder. | | 05 | Trend Finder scoped scheduler job | Dedicated Trend Finder refresh path | Registry metadata, handler wiring, package scripts, status copy, provider/source readiness warnings, and tests prove Trend Finder refreshes only its owned extension branch. | | 06 | Cross-job integration and Dream freshness boundary | Combined scoped-job behavior plus risk mitigations | Dream can use recent agent aggregate data without forcing Trend Finder, scoped jobs can run in either order without erasing data, and status distinguishes defaults, local timer intent, latest run state, and actual generated-data freshness. | | 07 | Timer unit rendering and reconciliation | Optional timer-management pass | Dry-run rendering, explicit install/uninstall commands, drift reporting, systemd-only docs, and legacy Dream cron separation are complete after the split jobs are stable. | | 08 | Final documentation, compatibility, and validation sweep | Cadence strategy, risks, and validation checklist | Runbooks, `.env.local.example`, data docs, command hints, and validation checklist are synchronized; focused tests, `bun run typecheck:scripts`, and `git diff --check` pass. | ### Sequencing Rules 1. Do not start scoped scheduler job handlers before Session 02 lands the shared generated-data lock and merge writer. 2. Do not change the public `bun run aggregate` compatibility command while extracting orchestration helpers. 3. Keep `scripts/lib/scheduler/registry.ts` as the reviewed job and cadence contract; local config may select only reviewed values. 4. Keep secrets, raw source payloads, prompts, auth paths, private machine paths, and command bodies out of status output, summary logs, browser metadata, and committed templates. 5. Treat Session 07 as the only timer-management implementation pass. Earlier sessions may report desired timer intent but must not install or mutate local timer units. *** ## Completed Sessions * Session 01: Scheduler Config Contract And Status Read Path - completed 2026-05-31 * Session 02: Live Data Merge Boundary And Shared Write Lock - completed 2026-05-31 * Session 03: Aggregate Orchestration Refactor - completed 2026-05-31 *** ## Upcoming Sessions None in Phase 15. Original sessions 04-08 are carried forward to future phase planning. *** ## Deferred Scope The following original Phase 15 items are intentionally deferred and should be used as source material for the next phase build: 1. Register `agent-aggregate` as a scoped host/local scheduler job. 2. Register Trend Finder as a scoped extension scheduler job. 3. Validate cross-job ordering, Dream freshness behavior, and generated-data freshness semantics after both scoped jobs exist. 4. Add systemd user timer rendering, dry-run, inspect, install/uninstall, and drift reporting only after scoped jobs are stable. 5. Synchronize final runbooks, env examples, command hints, and compatibility validation after the scoped-job phase lands. *** ## Objectives 1. Add a deterministic scheduler config contract under `data/` while preserving registry-reviewed job metadata. 2. Add safe scoped writes to `src/data/live-data.json` with a shared lock and branch ownership map. 3. Extract aggregate orchestration paths without changing the public full aggregate command. 4. Add scoped local-agent and Trend Finder scheduler jobs with clear status and cadence behavior. 5. Validate Dream freshness, cross-job ordering, docs, and compatibility gates. *** ## Current State The current AI OS scheduler has two jobs: | Job | Current Role | | ----------- | ------------------------------------------------------------------------------- | | `aggregate` | Runs the full `scripts/aggregate.ts` path and writes `src/data/live-data.json`. | | `dream` | Runs aggregate first, then evaluates Dream activation gates and output. | The full aggregate path currently does both host/local refresh work and extension work: * scans local agent, session, config, memory, and integration data * builds `localAgents` * loads Dream projection data * runs registered extension collectors * writes the single generated `src/data/live-data.json` artifact Trend Finder is currently the registered script-side extension collector. When `VITE_CLAUDE_OS_ENABLED_EXTENSIONS` includes `trend-finder`, the full aggregate run can execute Trend Finder source collection and analysis as part of the same scheduled job. This coupling makes the default every-four-hour aggregate cadence useful for local agent freshness, but too broad for heavier extension work that may call external sources, configured providers, Apify Actors, or AI runtimes. *** ## Target Decisions 1. Keep `scripts/lib/scheduler/registry.ts` as the reviewed static job contract. It continues to define allowed jobs, handlers, timeouts, write targets, network policy, default candidates, and browser-safe metadata. 2. Add a private local scheduler config file under `data/` named `data/ai-os.scheduler.json`. 3. Commit a sanitized template named `data/ai-os.scheduler.example.json`. 4. Keep `.env.local` for secrets, runtime provider settings, and process-level activation such as `AI_OS_DREAM_ENABLED=true`. 5. Avoid arbitrary raw `systemdCalendar` values in local config. Local config selects reviewed cadence IDs or validated time values only. 6. Split scheduler execution into separate jobs: * `agent-aggregate`: refresh local host, agent, and dashboard data * `trend-finder`: refresh only Trend Finder extension data 7. Keep `bun run aggregate` as the full compatibility command unless a later migration explicitly changes that public command. 8. Add a shared generated-data write lock so separate scheduler jobs cannot concurrently overwrite `src/data/live-data.json`. *** ## Config Shape The private local scheduler config is versioned and intentionally narrow: ```json { "version": 1, "jobs": { "agent-aggregate": { "timerEnabled": true, "cadence": "every-4-hours" }, "trend-finder": { "timerEnabled": true, "cadence": "daily" }, "dream": { "timerEnabled": true, "time": "07:00" } } } ``` Config notes: * `timerEnabled` records local operator intent. It does not prove that a systemd timer is installed or active. * `cadence` values must reference reviewed scheduler candidate IDs. * `time` must validate as strict `HH:MM` and should render to safe `*-*-* HH:MM:00` guidance only after validation. * Existing `aggregate` can remain a full manual and backward-compatible job while new scoped jobs become the preferred scheduled timers. *** ## Prerequisites * Phase 14 completed. * Existing Phase 10-12 scheduler, aggregate, and Dream runtime foundations are available. * Current `scripts/aggregate.ts`, scheduler registry, run state, locks, and live-data validation behavior are the source of truth. *** ## Technical Considerations ### Architecture Keep `scripts/lib/scheduler/registry.ts` as the reviewed static job contract. Local config may select reviewed jobs/cadences but must not introduce arbitrary commands, shell fields, raw systemd calendar expressions, private paths, or unreviewed job IDs. Define explicit live-data ownership boundaries before adding scoped job handlers. Host/local branches belong to `agent-aggregate`; Trend Finder extension branches belong to `trend-finder`; full aggregate remains a compatibility writer that validates unchanged output. ### Technologies * Bun scripts and `bun test` style focused tests where existing patterns use it * TypeScript scheduler and aggregate modules under `scripts/lib/` * Generated live data at `src/data/live-data.json` * Private local config under `data/` ### Risks * Scoped jobs can erase each other's generated data if merge boundaries are incomplete: add a shared write lock and validation-backed merge writer before job split handlers. * Trend Finder can be accidentally run on the local-agent cadence: separate job handlers and tests must prove collector isolation. * Status output can leak private config or runtime details: sanitize warnings and keep raw contents, secrets, prompts, command bodies, and private paths out of logs and browser-visible summaries. ### Relevant Considerations * \[P02] Collector timeout of 120s is fragile: Trend Finder should get an independent lower-frequency job and explicit timeout behavior. * \[P11] Scheduler state/log privacy boundary: status surfaces may expose only safe metadata. * \[P12] Dream browser/service surfaces stay narrow: Dream freshness must not imply a browser gateway or force broad refresh behavior. * \[P14] Backtest archives stay private but can grow: new scheduler status must avoid widening private artifact exposure. *** ## Cadence Strategy | Job | Default Cadence | Rationale | | ----------------- | --------------- | -------------------------------------------------------------------------------------------- | | `agent-aggregate` | Every 4 hours | Local sessions, model usage, skills, config, and integration state drift during the workday. | | `trend-finder` | Daily | External collection and AI analysis are heavier and may involve cost or rate limits. | | `dream` | Daily 07:00 | Morning review should use recent aggregate data and explicit activation. | The core reason to split is that host/local telemetry freshness and Trend Finder source intelligence have different operating costs. Agent aggregate is mostly local and benefits from multiple daily refreshes. Trend Finder reaches out to public sources, Apify, and optional AI runtime providers, so it needs an independent cadence and failure surface. *** ## Risk Register | Risk | Mitigation | | ----------------------------------------- | ------------------------------------------------------------------------------------------------------ | | Concurrent writes corrupt live data | Add shared generated-data write lock and validate merged output. | | Partial writes erase another branch | Use explicit ownership map and branch-preserving merge writer. | | Config becomes unsafe command surface | Allow only known job IDs, cadence IDs, booleans, and strict `HH:MM` times. | | Status conflates defaults and real timers | Report registry default, local config intent, latest trigger, and generated-data freshness separately. | | Dream unexpectedly runs heavy collectors | Keep Dream dependent on recent agent aggregate unless explicitly configured otherwise. | | Existing commands break | Preserve `bun run aggregate` and document compatibility scripts. | *** ## Validation Checklist * `bun run test -- scripts/lib/__tests__/scheduler-registry.test.ts` * `bun run test -- scripts/lib/__tests__/scheduler-operator-status.test.ts` * `bun run test -- scripts/lib/__tests__/scheduler-status-cli.test.ts` * `bun run test -- scripts/lib/__tests__/scheduler-runner.test.ts` * Scheduler config parser tests * Live-data merge writer tests * Scoped aggregate handler tests * Trend Finder scheduler handler tests * `bun run typecheck:scripts` * `git diff --check` *** ## Success Criteria Phase complete when: * [x] The closeout scope is narrowed to Sessions 01-03, with original Sessions 04-08 explicitly deferred to a future phase. * [x] `bun run aggregate` remains a compatibility command with unchanged public behavior. * [x] Scoped writer boundaries preserve the other live-data branch during writes; scoped scheduler job handlers remain deferred. * [x] Scheduler status distinguishes registry defaults, local timer intent, latest run state, and generated-data freshness. * [x] Secrets, raw source payloads, prompts, auth paths, private machine paths, and command bodies stay out of status output, logs intended for summaries, browser metadata, and committed templates. * [x] Docs, runbooks, examples, and validation gates are synchronized for the completed closeout scope. * [x] The former ongoing-project plan can be deleted without losing scheduler split requirements or implementation gates. *** ## Dependencies ### Depends On * Phase 10: General Scheduler Foundation * Phase 11: Scheduled Aggregate Path * Phase 12: AI OS Dream Runtime Migration * Phase 14: Trend Finder Historical Backtest Harness ### Enables * Future SQLite observation store transition * Future adaptive Trend Finder source planning * Future managed systemd timer reconciliation --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_15/prd_phase_15.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.