> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/intelligence-view.md). # Intelligence View This document records the current AI OS Intelligence view behavior after Phase 40 Session 16. ## Current Status AI OS ships the Hermes Intelligence portal inside the existing `/agents/hermes` route. Hermes Intelligence is the portal name; Voice is the speech-control capability inside that portal. Operators can open it from the Hermes Chat tab or directly with: ``` /agents/hermes?intel=1 ``` The portal is route-local; it does not add a second Hermes route or backend. Typed and spoken `ask_hermes` requests use the existing `POST /__hermes_chat` SSE bridge through the Hermes admin hook. The portal maps the parsed chat stream into transcript, activity, and Aurora, Cosmos, or Classic visual state. Session 09 owns the portal product surface. Session 08 owns the local voice broker. Session 16 added the AI OS no-reprompt parity proof through environment-backed broker respawn. Live voice currently requires local configuration: * `HERMES_DASHBOARD_ADMIN=1` * The same-run local Vite token * A configured Session 08 voice broker * `OPENAI_API_KEY` in ignored local environment * Optional allowlisted `OPENAI_BASE_URL` The manual Hermes admin env gate is a current implementation constraint, not the future local access default. Proper delivery for voice means real local execution, visible transcript or spoken result, specific recovery states, and tests; mocked bridge/provider coverage proves boundaries but not a live provider result. Public demo mode can open the portal as a visual product surface, but live voice and local writes stay gated. ## Backend Requirement The real backend for the Intelligence portal is still `POST /__hermes_chat`. That route must remain: * Loopback-only. * Same-run-token gated. * Gated by the current `HERMES_DASHBOARD_ADMIN=1` implementation constraint until the all-access migration replaces manual admin opt-in. * Host-header guarded by the shared local control-plane guard. * Validated for method, JSON shape, session IDs, toolset choices, and unsafe options. * Streaming only sanitized, intentional chat output to the browser. * Free of stack traces, local paths, bearer tokens, raw provider payloads, and secret-shaped strings in user-visible errors. The portal must not invent a second Hermes chat backend. Any future replacement must provide equivalent guard and stream tests. ## Voice Behavior The browser portal starts the local broker through guarded `POST /__start_voice` with `{}`, requests a short-lived Realtime session from `http://127.0.0.1:8099/api/session` with only `voice` and `mode`, and passes only the same-run token in `X-Claude-OS-Token`. Provider keys and provider base URLs remain environment-only and never move through browser inputs, URL params, localStorage, committed fixtures, docs examples, `/__start_voice` bodies, or `/api/session` bodies. `/__start_voice` accepts an empty launch and rejects browser provider config. The parity target is avoiding repeated setup prompts when the local dev server already has `OPENAI_API_KEY`; AI OS satisfies that through broker respawn, not browser key persistence. When the Realtime session calls `ask_hermes`, the portal forwards the request to the Hermes admin hook. The same final answer is returned to the Realtime session for speech and rendered in the portal transcript. When credentials or browser capabilities are missing, the portal shows product-facing recovery states for demo mode, offline browser, missing token, disabled admin mode, missing provider key, broker unavailable, provider failure, timeout, mic denial, unavailable mic/WebRTC, and aborted streams. ## Chat, Compact, And Command Relationship The portal does not add a separate command backend. Typed prompts and Realtime `ask_hermes` tool calls both resolve through the existing Hermes chat admin hook. Parsed `chunk`, `info`, `done`, and `error` events update transcript and visual state. The Chat tab's compact flow and command menu remain Chat-tab behavior. Compact summarizes the current local chat into visible carryover for a fresh chat. Allowlisted Hermes commands route through `/__hermes_cmd` only when current write gates pass. The Intelligence portal may display mapped chat activity, but it does not turn compact, slash actions, or command execution into a new voice backend. ## Visual Behavior The portal includes: * Aurora: warm canvas stage for living-signal motion. * Cosmos: canvas stage for cosmic reasoning-field motion. * Classic: lazy-safe 3D Hermes mind graph with deterministic fallback. * Oracle visualizers: Plasma, Sonar, Waveform, and Rider in Classic mode. Visualizers use parsed Hermes chat events for live activity. Synthetic motion is reserved for explicit demo or empty attract states and must not replace a real Hermes answer. ## Upstream Skips These upstream files remain intentionally skipped: * `src/components/agent-core-3d.tsx` - upstream marked this obsolete/dead. * `INTEGRATION.md` - internal merge guidance, not shippable AI OS product documentation. ## Verification Limits Session 09 and Session 16 tests and browser proof may use mocked `/__hermes_chat`, `/__start_voice`, broker, and provider responses. Mocked tests prove payload shape, recovery states, and trust boundaries. Do not document a real provider voice result unless local credentials were available and the run recorded only safe readiness and success metadata. ## Related Docs * [Hermes Agent Surface](/ai-os-and-trend-finder-docs/readme.md#hermes-agent-surface) * [Local Voice Setup](/ai-os-and-trend-finder-docs/docs/local-voice-setup.md) * [Local API Notes](/ai-os-and-trend-finder-docs/docs/api/readme_api.md) * [Data Contract](/ai-os-and-trend-finder-docs/docs/data-contract.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/docs/intelligence-view.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.