> For the complete documentation index, see [llms.txt](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_38/session_08_voice_broker.md). # Session 08: Voice Broker **Session ID**: `phase38-session08-voice-broker` **Status**: Not Started **Estimated Tasks**: \~12-25 **Estimated Duration**: 2-4 hours *** ## Objective Add the local OpenAI Realtime token broker needed by the Intelligence voice surface. *** ## Non-Negotiable Delivery Bar The broker is done only when a real browser can mint a real Realtime session token through the local broker when configured, and bad origin, bad token, missing key, and non-loopback requests fail with actionable errors. *** ## Source Items * 3.2 `ADAPT`: voice sidecar / Realtime token broker. * 2.4 launch target work is coordinated here with Session 06 metadata. *** ## Scope ### In Scope (MVP) * Add `/home/aiwithapex/projects/aios/voice-lab/server.ts` as a token broker only. * Add optional `/home/aiwithapex/projects/aios/voice-lab/.env.example`. * Add a `voice` script in `/home/aiwithapex/projects/aios/package.json`. * Add `/__start_voice` middleware in `/home/aiwithapex/projects/aios/vite.config.ts`. * Pass provider keys by environment only, not argv. * Use upstream OpenAI Realtime as the default provider path and honor `OPENAI_BASE_URL` for compatible local/proxy endpoints. * Reuse AI OS token gates, loopback guards, cross-origin checks, and Host-header protections. * Add request, rejection, health, and base-url tests. ### Out of Scope * Copying upstream standalone `voice-lab/index.html`. * Shipping a disconnected demo voice UI. * Storing provider keys in browser state. * Wiring the Intelligence portal itself; that is Session 09. *** ## Detailed Port Item ### 3.2 Voice Sidecar / Realtime Token Broker Upstream added a standalone Bun sidecar on port 8099 that mints OpenAI Realtime tokens, returns the configured Realtime base, and proxies TTS. It also added a `voice` script and `/__start_voice` endpoint to launch it with a key passed by environment and a base URL allowlist. * Upstream voice directory: `/home/aiwithapex/projects/claudeos/claude-os-v2.8.1/voice-lab/`. * Upstream server source: `/home/aiwithapex/projects/claudeos/claude-os-v2.8.1/voice-lab/server.ts`. * Upstream env example: `/home/aiwithapex/projects/claudeos/claude-os-v2.8.1/voice-lab/.env.example`. * Upstream README: `/home/aiwithapex/projects/claudeos/claude-os-v2.8.1/voice-lab/README.md`. * Upstream standalone demo: `/home/aiwithapex/projects/claudeos/claude-os-v2.8.1/voice-lab/index.html`. * Patch lines: 7661, 7674, 7715, and 7946. * Upstream Vite source for `/__start_voice`: `/home/aiwithapex/projects/claudeos/claude-os-v2.8.1/vite.config.ts` at patch line 6506. * AI OS targets: `/home/aiwithapex/projects/aios/voice-lab/server.ts`, `/home/aiwithapex/projects/aios/voice-lab/.env.example`, `/home/aiwithapex/projects/aios/package.json`, `/home/aiwithapex/projects/aios/vite.config.ts`, and `/home/aiwithapex/projects/aios/.claude/launch.json`. * Adaptation: align broker loopback/cross-origin rejection with existing AI OS local guard behavior and token gate. Return the Realtime session `base` so local `OPENAI_BASE_URL` works end to end. * Resolved policy: OpenAI Realtime is the default voice path; `OPENAI_BASE_URL` is the compatible override. * Skip upstream defect: do not copy `voice-lab/index.html`; it hardcodes OpenAI's endpoint and is not wired to Hermes. The real AI OS surface is the Session 09 portal. *** ## Prerequisites * [ ] Session 05 completed so Host-header and bridge guards are hardened before voice launch behavior exists. * [ ] OpenAI Realtime default and `OPENAI_BASE_URL` override policy are still resolved. * [ ] AI OS token gate and loopback/cross-origin guard helpers are identified. *** ## Deliverables 1. Local broker with `/api/health` and token-mint behavior. 2. Optional voice-lab env example adapted to AI OS. 3. `voice` package script. 4. `/__start_voice` Vite middleware passing keys through environment only. 5. `.claude/launch.json` target coordinated with Session 06. 6. Tests for missing key, bad token, non-loopback, cross-origin, and `OPENAI_BASE_URL` behavior. *** ## Success Criteria * [ ] `bun run voice` starts a broker whose loopback health endpoint returns ok. * [ ] `/__start_voice` can start the broker from the UI path without exposing keys in argv or browser state. * [ ] A real browser can mint a Realtime session token when configured. * [ ] `OPENAI_BASE_URL` is honored end to end for compatible endpoints. * [ ] Non-loopback, cross-origin, missing-key, and bad-token requests fail with actionable errors. * [ ] Upstream `voice-lab/index.html` is recorded as skipped, with the AI OS portal named as the real surface. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ai-os-and-trend-finder.gitbook.io/ai-os-and-trend-finder-docs/.spec_system/archive/phases/phase_38/session_08_voice_broker.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.