> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase38-session09-intelligence-portal/spec.md). # Session Specification **Session ID**: `phase38-session09-intelligence-portal` **Phase**: 38 - Claude OS v2.8.1 Semantic Port **Status**: Not Started **Created**: 2026-06-30 *** ## 1. Session Overview This session delivers the Hermes Intelligence portal inside the existing AI OS Hermes route. It is next because deterministic project analysis shows Phase 38 Sessions 01 through 08 complete, no active session, and Session 09 as the first unfinished candidate. The work adapts upstream v2.8.1 Intelligence components into AI OS component boundaries instead of copying the upstream route wholesale. The portal must be reachable from `/agents/hermes`, use the existing admin-gated Hermes chat path, and keep `/__hermes_chat` as the only real Hermes backend. The session connects the Session 08 Realtime broker to a voice loop that can call `ask_hermes`, stream the real Hermes answer back through the portal, speak the same answer, and feed Aurora, Cosmos, and Classic visualizers from real chat events. Synthetic output is allowed only for explicit idle or demo states. *** ## 2. Objectives 1. Add a reachable Intelligence portal from `/agents/hermes` with Aurora, Cosmos, and Classic modes under AI OS Hermes component conventions. 2. Add the voice loop and `ask_hermes` tool-call bridge that sends spoken prompts through `admin.chat` and the existing `/__hermes_chat` SSE stream. 3. Map real Hermes chat events into portal activity, transcript, and visualizer state while keeping synthetic motion limited to explicit idle or demo states. 4. Cover mic denied, missing token/key, broker unavailable, provider failure, admin-gated chat, mid-stream abort, and visual fallback paths with focused tests, docs, and browser proof. *** ## 3. Prerequisites ### Required Sessions * [x] `phase38-session05-runtime-bridge-hardening` - Provides loopback, Host-header, token, body-size, and local control-plane guard patterns for `/__hermes_chat` and privileged Vite routes. * [x] `phase38-session08-voice-broker` - Provides the local OpenAI Realtime token broker, `bun run voice`, `/__start_voice`, and env-only provider policy that the portal must use. ### Required Tools Or Knowledge * Bun 1.3.14, Vitest, Playwright, React 19, TanStack Start, Tailwind CSS 4, existing Radix UI primitives, `react-force-graph-3d`, Three.js, and lucide-react. * Upstream v2.8.1 source files under `/claude-os-v2.8.1/src/components/` and `/claude-os-v2.8.1/src/lib/`. * Current AI OS Hermes chat contracts in `src/hooks/use-hermes-admin.ts`, `src/components/hermes/chat/hermes-chat-tab.tsx`, and `scripts/lib/hermes-admin-bridge.ts`. ### Environment Requirements * Repository root is the AI OS checkout root. * Local live voice requires `HERMES_DASHBOARD_ADMIN=1`, the per-run Vite token, `OPENAI_API_KEY`, optional allowlisted `OPENAI_BASE_URL`, and the Session 08 voice broker. * Provider keys, bearer headers, raw provider responses, prompts, transcripts, session IDs, token-shaped strings, local usernames, and home-directory paths must stay out of committed fixtures, public demo data, browser-visible diagnostics, logs, and docs examples. *** ## 4. Scope ### In Scope (MVP) * Local operators can open the Intelligence portal from `/agents/hermes` - integrate the portal into the existing Hermes product route and support a direct `?intel=1` entry state without creating a disconnected route. * Local operators can speak to Hermes - connect the portal to the Session 08 voice broker, request Realtime session credentials with the same-run token, and route `ask_hermes` tool calls through the existing admin chat contract. * Hermes answers are real and visible - use `/__hermes_chat` SSE output for the transcript, spoken response, activity events, and visualizer state. * Aurora, Cosmos, and Classic modes render under AI OS Hermes styling - adapt upstream stages, mind graph, and Oracle visualizers with reduced-motion, lifecycle cleanup, bounded layout, and accessible controls. * Failure states are product-facing - show actionable states for mic denied, no key, no token, broker unavailable, provider down, admin disabled, offline, failed spawn, timeout, and mid-stream abort. * Tests prove the bridge - add hook, mapper, component, and integration tests that drive a mocked-spawn `/__hermes_chat` stream through the voice tool-call handler. ### Out Of Scope (Deferred) * Copying upstream `src/components/agent-core-3d.tsx` - Reason: the Phase 38 plan marks it obsolete/dead and skipped. * Importing upstream `INTEGRATION.md` as product documentation - Reason: it is internal merge guidance, not shippable AI OS docs. * A browser key picker or request-body provider key flow - Reason: Session 08 policy requires provider keys from local environment only. * Upstream `/api/sample` TTS or a disconnected voice demo - Reason: AI OS ships the real product portal, not a standalone sample surface. * A second Hermes chat backend, route, or stored transcript service - Reason: the existing `/__hermes_chat` SSE path is the required backend. * Dream engine endpoints or dashboard Dream work - Reason: Session 07 owns that scope and is already complete. * Public demo live voice or hosted write behavior - Reason: Pages demo remains static-only with no `/__*` bridge calls. *** ## 5. Technical Approach ### Architecture Create `src/components/hermes/intelligence/` as the portal boundary. Keep the route shell in `src/components/hermes/hermes-read-only-page.tsx`, add a launcher from the existing Chat tab, and render the portal as a full-screen Hermes overlay or equivalent route-local state that can be opened by `?intel=1`. Do not add a new TanStack route for the portal. Add a small transcript-returning chat method to `useHermesAdmin` while keeping the existing `sendPrompt(...) => Promise` API for current chat UI callers. The new method should return the final assistant text plus parsed SSE events so the voice tool handler does not scrape React state after the stream finishes. It must preserve duplicate-send prevention, abort handling, query invalidations, and error mapping already present in the hook. Add `useHermesIntelligenceVoice` for broker health, `/__start_voice`, Realtime session creation, microphone lifecycle, WebRTC/data-channel cleanup, function-call handling, and failure-state projection. The hook should request credentials from the Session 08 broker with the same-run token, never accept provider keys from browser input, and keep all media, peer connection, and timer resources scoped to the portal lifecycle. Adapt upstream visualizers under AI OS styling and split pure event mapping into `src/lib/hermes-intelligence-events.ts`. The mapper should convert `HermesChatEvent` records from `admin.chat.events` into portal activity, clusters, transcript phases, and visual intensity with exhaustive handling for known event types and controlled handling for unknown events. ### Design Patterns * Existing route integration: launch from `/agents/hermes` and reuse Hermes route state instead of adding a separate route. * Chat contract compatibility: add a transcript-returning chat method without breaking existing boolean chat callers. * Local control-plane boundary: use the Session 08 broker and `/__hermes_chat` with loopback, token, and admin gates intact. * Explicit state machine: represent idle, starting, listening, thinking, responding, speaking, error, offline, and demo states as typed values. * Product-only UI copy: normal portal surfaces show recovery and status copy, while raw diagnostics stay in tests, docs, and implementation notes. *** ## 6. Deliverables ### Files To Create | File | Purpose | Est. Lines | | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ---------- | | `src/components/hermes/intelligence/index.ts` | Export portal, types, and visualizer components. | \~25 | | `src/components/hermes/intelligence/intelligence-types.ts` | Portal state, design mode, transcript, voice, and activity event types. | \~120 | | `src/components/hermes/intelligence/intelligence-portal.tsx` | Full Intelligence portal shell, controls, transcript, recovery states, and visualizer selection. | \~520 | | `src/components/hermes/intelligence/hermes-mind-3d.tsx` | Adapt upstream Classic 3D Hermes mind graph with lazy-safe lifecycle cleanup. | \~280 | | `src/components/hermes/intelligence/stage-aurora.tsx` | Adapt Aurora canvas stage with reduced-motion and bounded layout behavior. | \~220 | | `src/components/hermes/intelligence/stage-cosmos.tsx` | Adapt Cosmos canvas stage with reduced-motion and bounded layout behavior. | \~240 | | `src/components/hermes/intelligence/oracle-plasma.tsx` | Adapt Plasma visualizer for Classic mode. | \~280 | | `src/components/hermes/intelligence/oracle-rider.tsx` | Adapt Rider visualizer for Classic mode. | \~140 | | `src/components/hermes/intelligence/oracle-sonar.tsx` | Adapt Sonar visualizer for Classic mode. | \~300 | | `src/components/hermes/intelligence/oracle-waveform.tsx` | Adapt Waveform visualizer for Classic mode. | \~300 | | `src/lib/mind-map.ts` | Shared cluster, color, interpolation, and state helpers used by the portal stages. | \~80 | | `src/lib/synthetic-voice.ts` | Explicit idle/demo visual motion driver that never replaces live Hermes output. | \~90 | | `src/lib/hermes-intelligence-events.ts` | Map `HermesChatEvent` streams to portal activities, clusters, and transcript phases. | \~160 | | `src/hooks/use-hermes-intelligence-voice.ts` | Start broker, connect Realtime, handle mic/WebRTC lifecycle, and bridge `ask_hermes`. | \~360 | | `src/components/hermes/intelligence/__tests__/intelligence-portal.test.tsx` | Portal state, controls, transcript, recovery, and accessibility render tests. | \~260 | | `src/components/hermes/intelligence/__tests__/visualizers.test.tsx` | Visualizer render, reduced-motion, cleanup, and canvas fallback tests. | \~220 | | `src/lib/__tests__/hermes-intelligence-events.test.ts` | Event mapper tests for chunks, errors, done events, unknown events, and cluster activity. | \~180 | | `src/hooks/__tests__/use-hermes-intelligence-voice.test.tsx` | Voice hook tests for broker start, token use, tool calls, cleanup, and failures. | \~320 | | `tests/e2e/hermes-intelligence.spec.ts` | Browser proof for opening the portal, product copy, mocked chat stream, and recovery states. | \~220 | ### Files To Modify | File | Changes | Est. Lines | | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ---------- | | `src/hooks/use-hermes-admin.ts` | Add transcript-returning chat method while preserving existing `sendPrompt` behavior. | \~90 | | `src/hooks/__tests__/use-hermes-admin.test.tsx` | Cover transcript-returning chat success, stream error, abort, duplicate prevention, and event capture. | \~140 | | `src/components/hermes/chat/hermes-chat-tab.tsx` | Add Intelligence launcher near the existing chat write surface with disabled states and focus behavior. | \~60 | | `src/components/hermes/chat/__tests__/hermes-chat-tab.test.tsx` | Cover launcher visibility, disabled gates, keyboard activation, and no demo/live confusion. | \~120 | | `src/components/hermes/hermes-read-only-page.tsx` | Add route-local portal state, `?intel=1` opening, lazy loading, and close/reset behavior. | \~110 | | `docs/intelligence-view.md` | Update from Session 08 pending-state language to shipped Session 09 behavior and verification limits. | \~90 | | `docs/local-voice-setup.md` | Add the portal usage path and keep env-only key policy current. | \~35 | | `.spec_system/specs/phase38-session09-intelligence-portal/implementation-notes.md` | Record upstream skips, adaptation choices, test evidence, browser proof, and security notes during implementation. | \~160 | *** ## 7. Success Criteria ### Functional Requirements * [ ] `/agents/hermes` exposes an Intelligence entry point and `?intel=1` opens the same product surface. * [ ] A spoken question starts or reuses the Session 08 broker, creates a Realtime session with the same-run token, and sends `ask_hermes` through `admin.chat` to `/__hermes_chat`. * [ ] A real Hermes answer streams into the portal transcript, is available to Realtime for spoken response, and updates visualizer state from parsed Hermes chat events. * [ ] Aurora, Cosmos, and Classic modes render, switch without remount leaks, and react to real chat events. * [ ] Synthetic motion appears only in explicit idle or demo state and never substitutes for live Hermes answers. * [ ] Mic denied, missing token/key, broker unavailable, provider down, admin disabled, offline, failed spawn, timeout, and mid-stream abort paths show actionable recovery states. * [ ] `src/components/agent-core-3d.tsx` and upstream `INTEGRATION.md` remain recorded as skipped. ### Testing Requirements * [ ] Hook tests cover broker start, token use, session credential fetch, Realtime event handling, `ask_hermes` tool calls, duplicate-trigger prevention, cleanup, and failure projection. * [ ] Admin hook tests cover the transcript-returning chat method without regressing existing boolean `sendPrompt` callers. * [ ] Event mapper tests cover all known `HermesChatEvent` variants and deterministic unknown-event handling. * [ ] Portal and visualizer tests cover design modes, reduced motion, transcript rendering, recovery states, focus behavior, and cleanup. * [ ] Browser proof covers opening the portal from `/agents/hermes`, `?intel=1`, product-surface copy, mocked `/__hermes_chat` stream behavior, and a recovery state. ### Non-Functional Requirements * [ ] All files are ASCII-encoded with Unix LF line endings. * [ ] Code follows project conventions, AI OS naming, Hermes component patterns, and existing test placement. * [ ] No provider key, bearer token, raw provider response, prompt, transcript, local username, private path, or token-shaped string is exposed to committed fixtures, public demo data, logs, or browser-visible diagnostics. * [ ] Visualizers clean up animation frames, media streams, peer connections, data channels, timers, and global listeners when the portal closes. * [ ] Normal user-facing portal surfaces contain product-facing copy only. * [ ] Bundle and route behavior remain compatible with the existing AI OS static public demo boundary. ### Quality Gates * [ ] `bun run test -- src/hooks/__tests__/use-hermes-admin.test.tsx src/hooks/__tests__/use-hermes-intelligence-voice.test.tsx src/lib/__tests__/hermes-intelligence-events.test.ts src/components/hermes/intelligence/__tests__/intelligence-portal.test.tsx src/components/hermes/intelligence/__tests__/visualizers.test.tsx src/components/hermes/chat/__tests__/hermes-chat-tab.test.tsx` * [ ] `bun run test -- tests/e2e/hermes-intelligence.spec.ts` * [ ] `bun run typecheck` * [ ] `bun run lint` * [ ] `bun run build` * [ ] ASCII/LF and secret-pattern sweeps pass for session deliverables. *** ## 8. Implementation Notes ### Working Assumptions * The portal belongs inside the existing `/agents/hermes` route instead of a new route: the Session 09 stub requires integration into the existing Hermes chat surface, and `src/components/hermes/hermes-read-only-page.tsx` already owns chat, admin-gated writes, demo mode, live mode, and route product copy. * `useHermesAdmin` should expose a transcript-returning chat method rather than forcing the voice bridge to scrape React state: the current hook already parses SSE events and owns abort/query invalidation behavior, so extending that contract is safer than duplicating stream parsing elsewhere. * The Session 08 same-run token is the voice broker token for browser requests: Session 08 validation records `AI_OS_VOICE_TOKEN` as the Vite token when `/__start_voice` launches the broker, and the portal already receives that token through the Hermes route data path. * Upstream visualizers are design references, not file-replacement targets: Phase 38 is a semantic port, AI OS has existing Hermes styling and route boundaries, and the upstream docs include browser key-paste and sample TTS behavior that conflicts with AI OS policy. ### Conflict Resolutions * Upstream `docs/intelligence-view.md` describes a browser voice-provider chooser that can paste an OpenAI key. Session 08 AI OS policy requires provider keys from environment only, so Session 09 must use broker readiness and recovery copy instead of a browser key picker. * Upstream `intelligence-portal.tsx` points directly at `http://localhost:8099/api/session` and `/api/sample`. AI OS keeps the session endpoint only, starts the broker through `/__start_voice` when needed, and skips sample TTS because the real product surface is spoken Hermes output. * Upstream route integration targets a large `src/routes/agents.hermes.tsx`. AI OS uses a thin route and component-owned Hermes page, so integration goes through `src/components/hermes/hermes-read-only-page.tsx` and `src/components/hermes/chat/hermes-chat-tab.tsx`. ### Key Considerations * Keep `P04` Hermes bridge guardrails intact: do not weaken loopback, token, admin, body-size, schema, or Host-header protections. * Keep `P21` credential privacy intact: provider keys, bearer headers, and credential-shaped strings stay out of browser state, logs, fixtures, and docs examples. * Keep `P31-P37` public demo boundaries intact: hosted/demo views must not call `/__*` bridge routes or imply live writes. * Follow `P00` stack conventions: use Bun, Vite 8, TanStack Start, React 19, Tailwind CSS 4, Radix UI, and existing Hermes component primitives. ### Potential Challenges * Browser media and WebRTC cleanup: centralize lifecycle ownership in `useHermesIntelligenceVoice` and test cleanup on close, abort, and failure. * Large visualizer surface area: adapt upstream files behind a narrow typed portal state contract and cover reduced-motion and non-blank render states. * Chat result handoff timing: return transcript and parsed events from the admin chat method instead of reading stale hook state after stream completion. * Credential-dependent runtime proof: use mocked provider and chat streams in tests, then record real broker/voice proof when local credentials are available; otherwise record the exact credential blocker without claiming live voice proof. ### Relevant Considerations * \[P04] **Hermes bridge guardrails must stay intact**: The portal must route real chat through the existing admin bridge with loopback and token gates. * \[P21] **Claude OAuth material stays script-only**: Treat provider keys and bearer headers as server/local environment data, never browser product data. * \[P31-P37] **Pages demo stays static-only**: Public demo mode must keep live voice and Hermes writes disabled. * \[P00] **Stack conventions**: Keep implementation in the current React, Vite, Bun, Tailwind, and Hermes component patterns. ### Behavioral Quality Focus Checklist active: Yes Top behavioral risks for this session: * Media/WebRTC resources can leak after closing or switching portal modes. * Voice tool calls can duplicate prompts or speak stale output if the chat stream result is not returned from the owner hook. * Recovery states can overclaim availability when broker, admin gate, token, provider, mic permission, or network prerequisites are missing. *** ## 9. Testing Strategy ### Unit Tests * Test `useHermesAdmin` transcript-returning chat success, stream error, abort, duplicate prevention, event capture, and boolean API compatibility. * Test `useHermesIntelligenceVoice` broker start, token header, session fetch, Realtime event handling, `ask_hermes`, cleanup, permission denial, missing key, bad token, provider failure, and offline states. * Test `hermes-intelligence-events` mapping for chunk, error, done, unknown, and cluster activation events. ### Integration Tests * Test `IntelligencePortal` opening, closing, design switching, transcript rendering, focus return, reduced-motion behavior, demo separation, and recovery states. * Test a mocked `/__hermes_chat` stream through the voice tool-call handler so a spoken request produces the same transcript and event mapping as typed chat. ### Runtime Verification * Run targeted Vitest suites, app typecheck, lint, build, ASCII/LF, and secret-pattern sweeps. * Run Playwright browser proof for `/agents/hermes` Intelligence launcher, `?intel=1`, mocked chat stream behavior, and at least one recovery state. * When local credentials are available, run a real broker health check and a live voice smoke that records only safe readiness and success metadata. ### Edge Cases * Mic denied, revoked, or unavailable after the portal opens. * Broker not running, broker start rejected, broker health timeout, and broker missing provider key. * Admin disabled, missing token, offline browser, malformed chat event, stream error, stream abort, and provider disconnect. * Design mode switch while listening or speaking. * Public demo mode, reduced motion, small viewport, and keyboard-only use. *** ## 10. Dependencies ### Other Sessions * Depends on: `phase38-session05-runtime-bridge-hardening`, `phase38-session08-voice-broker`. * Depended by: `phase38-session10-hunk-reconciliation-and-release-gate`. *** ## Next Steps Run the `implement` workflow step to begin implementation. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/sessions/phase38-session09-intelligence-portal/spec.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.