streammm / streamaid

Low-latency native-res 30 fps screen streaming (capped at 4K, never upscaled): capture on your machine, play in any browser, fan out through Cloudflare Workers + Durable Objects.

The origin is a Rust binary (cargo run). It encodes with ffmpeg (VideoToolbox on macOS, libx264 elsewhere) and delivers WebSocket fMP4 — not HTTP progressive download. That is the difference between localhost looking fine and Cloudflare looking like a stalled VOD. Optional microphone AAC muxes into the same fragments when enabled in Settings.

Why WebSocket (not /stream.mp4 through a tunnel)

cloudflared buffers proxied HTTP unless Content-Type: text/event-stream. A live video/mp4 body is held until the connection dies. WebSocket (/stream.ws, and the Worker /publish → /watch path) is not buffered that way.

The Worker also means one upload from home (the publisher) and N viewers at the edge.

Quick start (this computer)

# ffmpeg on PATH. macOS: Screen Recording permission for the capturing app.
cargo run --release
# open http://127.0.0.1:8080

On this Mac the origin is a LaunchAgent (com.streamaid.origin). It stays up across terminal/chat sessions and login, and only stops when you unload it:

# start (and keep running)
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.streamaid.origin.plist

# stop — this is the off switch
launchctl bootout gui/$(id -u)/com.streamaid.origin

Killing the process is not enough — KeepAlive will restart it. Logs: logs/origin.out.log. If capture is black after a reboot, allow Streamaid under System Settings → Privacy & Security → Screen Recording. The origin UI shows a PERMISSION NEEDED banner (with Screen Recording / Accessibility / Input Monitoring buttons) when those grants are missing.

./config.json is created with defaults if missing (gitignored).

Default	Value
Encoder	H.264 (ffmpeg), High profile, no B-frames
Bitrate	20 Mbps CBR, 0.5 s VBV (up to 50 Mbps in Settings). Watch and host headers have a Quality / Balanced / Speed picker that retunes bitrate and scale on the live encode (Speed is 4 Mbps at 0.5×).
GOP	15 frames (~0.5 s join / live edge)
Size	cap 3840×4320, never upscale, lanczos only if the display is larger
Displays	Settings picker + clickable all-monitors map: live thumbnail of the captured screen (animation-frame paint), and ~15 fps JPEG previews of the others from a persistent ffmpeg per inactive display; Capture screen N maps to CGGetActiveDisplayList()[N] so clicks hit the captured screen
fps	30
Audio	Off by default. Settings Capture audio muxes AAC into the live fMP4 (H.264/HEVC, macOS) and reveals an audio device picker (default mic, or BlackHole/Loopback for system audio). Watchers get an Unmute control (autoplay starts muted). Settings Allow watcher to talk reveals a Talk button; their mic plays continuously on the host speakers (ffmpeg/ffplay s16le pipe, afplay fallback). Talk requests browser echo cancellation; if Capture audio is also on, Talk is half-duplex (the live stream mutes while you speak, and the host input is ducked so the mic does not pick up the speakers) and Settings reveals that hint.

Global watch link (OTP PIN)

1. Keep the origin running on the capture machine.
2. Deploy the Worker (below).
3. Set origin cloudflare.publish_url / watch_url (or the Settings fields in the UI). The publish URL still includes STREAM_TOKEN (that is the stream-key secret; do not put it on the public watch page).
4. Share the Worker homepage with no token in the URL:

https://<your-worker>.workers.dev/

The host UI (http://127.0.0.1:8080) shows a live 6-digit PIN (about 5 minutes; regenerate from Settings / POST /api/otp). The remote person types that PIN on the Worker page or localhost login overlay. Redeem issues a short-lived streamaid_session cookie. Wrong PINs are 401 and rate-limited.

Allow unattended access (Settings) reveals a persistent password. After the PIN expires, watchers join with that password on the same unlock field (POST /api/otp/redeem { "pin": "…" }). The origin stores only a SHA-256 hash; GET /api/config reports access.password_set and never the hash. Uncheck to disable. Unattended access, keep-awake, or remote control also reveals Wake-on-LAN: this Mac's MAC address, a Copy MAC button, and a Send wake packet button (POST /api/wol { "mac": "aa:bb:…" }) that broadcasts a magic packet on the LAN. A sleeping origin cannot send that packet itself — copy the MAC and send it from another device on the network. The watch page also gets a Copy MAC control from host flags (persisted in the Worker) so you still have the address when the publisher is down. When the origin is not publishing, the watch page shows host offline (the viewer socket stays up on the Worker) and reveals a Wake-on-LAN hint with that Copy MAC. A sleeping origin cannot send the packet itself — the browser and Worker cannot UDP-broadcast. Firmware Wake-on-LAN must be on; a closed MacBook lid often will not wake.

STREAM_TOKEN stays the publisher secret (npx wrangler secret put STREAM_TOKEN). It is not the human watch password.

The Worker homepage is a player with PIN unlock, Have AI use this computer, and a screen analysis + Ask sidebar. /watch is the viewer WebSocket (session); /publish is ingest from the origin; /health is public.

Remote computer use

Remote control is off by default. On the origin UI, enable:

• Allow remote computer use — the remote watcher can click (left/right/middle), drag, scroll, type, use modifier shortcuts (⌘/Ctrl+C), paste clipboard text, paste files, and drop files onto the session. Enabling it reveals a Send-keys bar for shortcuts the browser would steal (⌘Tab, ⌘W, ⌘Q, Alt+Tab, Alt+F4, Ctrl+Alt+Del, Lock) and a session chat panel so host and watcher can talk without those keystrokes hitting the remote desktop. The AI injector uses the same key actions. Copy on the host is pushed to the watcher (text, PNG images up to 128 MB, and Finder file copies into the inbox with a Save control on the watch page — never truncated; large PNGs go in 24 KB chunks, and a too-large image is rejected rather than truncated). Watchers can paste PNG or other images (JPEG/WebP are converted to PNG) and paste non-image files into the host inbox. Files land in the origin inbox/ folder (next to config.json, up to 2 GB, 24 KB chunks streamed from File.slice and appended to disk, resumes from .part after a drop), are copied onto the host Desktop, and are placed on the host pasteboard as Finder files so Cmd-V pastes them into another folder. Get/download streams the file from disk instead of buffering it in RAM. Folders download as a STORE zip ({name}.zip) with the same 2 GB / 2000-entry cap; Get is revealed on folders as well as files. Checkboxes select multiple items and reveal bulk Copy / Cut / Get / Delete (Get of several items is files.zip; Copy/Cut queue all of them for Paste here). Select all (⌘/Ctrl+A) and Shift-click select a range from the last anchor; ⌘/Ctrl-click toggles. Drag onto a folder row, Up, or an Inbox/Home/Desktop/Documents/Downloads tab to move (⌥/Ctrl copies); dropping OS files onto a folder writes into that folder. Split reveals a second pane (defaults to Desktop) so you can keep two host folders open and drag between them. On the watch page, Get uses showSaveFilePicker so chunks write to disk; Safari/Firefox fall back to GET /api/files/download, which the Worker streams from the origin. Enabling remote control also reveals Inbox / Home / Desktop / Documents / Downloads browse (and Up into subfolders), New folder, Rename, Copy / Cut plus Paste here (move or copy between those folders), Get (files, folders as zip, or a multi-select zip), and Delete (files and folders; folder delete is recursive after confirm, capped at 2000 entries) (file / list|get|delete with root + path, or GET/DELETE /api/files?root=desktop&path=…&name=… on the origin; repeat name= or send names for a selection). Drop / browse-upload writes into the folder you are viewing (Inbox still also copies onto Desktop and the pasteboard). Kill switch: uncheck it.
• Allow AI computer use — the same watcher can submit a task in Have AI use this computer. The origin loops TYPE_SNAP JPEG → vision model JSON actions → the same injector (click/right-click/drag/type/key/paste/clipboard PNG/inbox file). File actions can also list / mkdir / rename / copy / move / delete host folders (same roots as the Files panel); a list result is appended as [files]=JSON on the next step. Cancel AI on the origin UI (POST /api/computer-use/cancel) stops a running loop.

macOS: grant Streamaid Accessibility (and Input Monitoring if prompted) in System Settings → Privacy & Security, in addition to Screen Recording. CI tests use a fake injector; they never post real HID events.

One remote controller at a time. The origin UI shows a REMOTE SESSION banner with End (POST /api/control/release) while someone is driving. Optional Block local input during remote session swallows this Mac's HID until ⌘⇧Esc or End (End cannot be clicked while locked). Optional Blank the host screen during remote session drives the physical displays to black (gamma table plus display brightness) so people nearby cannot see the desktop, including HDR / True Tone panels; capture still streams the real framebuffer. ⌘⇧Esc unblanks and reclaims. Optional Keep this Mac awake for remote access (revealed with unattended access or remote control) holds idle-sleep and display-sleep assertions so the origin stays reachable; a live remote session always holds them. Lid close still sleeps; on wake, capture and the publisher restart immediately. Optional Lock this Mac when the remote session ends injects Control+Command+Q when the watcher disconnects or you press End — not when you reclaim with ⌘⇧Esc. Optional Record remote sessions writes the live fMP4 to recordings/ while someone is driving and reveals a Get list on the host UI. Watchers (and the host live view) also have a Record button that saves this view locally via MediaRecorder (WebM/MP4), independent of the host recordings/ folder, and a Full button (F11) that fullscreen the live view. Escape exits fullscreen without injecting into the remote session. Local host input / the kill switch always wins when unlocked.

Control path: viewer WebSocket JSON → Durable Object → publisher WebSocket → origin injector. Type-4 LLM snapshots still never go to viewers.

Analysis does not call DeepSeek until you attach a key:

cd cloudflare
npx wrangler secret put DEEPSEEK_API_KEY

Until then the APIs return 503 with setup instructions. The origin already publishes a JPEG snapshot (type 4) about every 8 seconds while capturing. Viewers never receive those snapshots — they stay in the Durable Object for the model.

Cloudflare deploy

cd cloudflare
npm install
npx wrangler login          # once
npx wrangler secret put STREAM_TOKEN
npx wrangler deploy

Then on the origin (UI Settings, or POST /api/config):

{
  "cloudflare": {
    "publish_url": "wss://<your-worker>.workers.dev/publish?token=<STREAM_TOKEN>",
    "watch_url": "wss://<your-worker>.workers.dev/watch"
  }
}

Restart the origin after changing publish URL if it was already connected.

Optional: cloudflare/cloudflared.yml if you want the UI on a tunnel hostname. Do not send HTTP /stream.mp4 through the tunnel.

Streams (origin)

Path	Role
GET /stream.ws	Primary. Binary: 1 = fMP4 init, 2 = fragment, 3 = JPEG, 4 = LLM snapshot (origin → Worker only)
GET /stream.mp4	LAN fallback, Transfer-Encoding: chunked (buffered on Tunnel)
GET /stream.mjpeg	MJPEG compatibility

Player live-edge is ~0.45 s with playbackRate catch-up. It appends by type byte, not by sniffing ftyp in a TCP chunk.

API (origin)

GET  /                UI (shows live PIN + kill switches)
GET  /api/status      capture/stream JSON
GET  /api/config      full config        POST /api/config   apply+save
GET  /api/otp         host: current PIN   POST /api/otp     regenerate PIN
POST /api/otp/redeem  `{ "pin": "123456" }` → session cookie
POST /api/computer-use `{ "task": "…" }`  (403 unless AI control enabled)
GET/POST /api/files     folder list (`?root=desktop&path=…`) / `{ "name", "data" }` (base64; 403 unless control on)
POST /api/files/mkdir   `{ "name", "root", "path" }` create a folder in the browsed directory
POST /api/files/rename  `{ "name", "to", "root", "path" }` rename a file or empty-safe folder
POST /api/files/copy    `{ "name", "root", "path", "toRoot", "toPath" }` copy into another browsed folder
POST /api/files/move    same shape; rename across folders (copy+delete if needed)
DELETE /api/files?name=…  remove a file (`root`/`path` optional; same auth as list)
GET  /api/files/download?name=…  (`root`/`path` optional; folders stream as `{name}.zip`)
GET  /api/recordings    host: session fMP4 list   GET /api/recordings/download?name=…
GET  /api/events      SSE
GET  /api/capture-devices

Host token: Authorization: Bearer, ?token=, or streamaid_token cookie (config, PIN mint). Viewer session: streamaid_session cookie or ?session= (watch + computer-use). Empty token keeps open local-dev mode.

Screen analysis (Cloudflare Worker)

The Durable Object (StreamRoom) stores the latest JPEG snapshot, runs vision on it, and serves Q&A on the same watch page.

Path	Role
GET /api/analysis	Latest summary, on-screen Q&A, history, {configured, has_snapshot, model}
GET /api/llm-status	Whether DEEPSEEK_API_KEY is set and a snapshot has arrived
POST /api/analyze-now	Force a vision pass now
POST /api/ask	{ "question": "…" } → { answer, confidence, reasoning }

Model defaults (override with wrangler vars):

• DEEPSEEK_BASE_URL = https://api.deepseek.com
• DEEPSEEK_MODEL = deepseek-v4-flash-vision-exp

Same-origin player polls /api/analysis and also accepts a WebSocket JSON message { "type": "analysis", "data": … } when a pass finishes.

Tests

cargo test                 # unit + HTTP API + player contract + stress
cargo test --test stress   # 30 fps hub budget, 16 WS clients, real 1080p30 ffmpeg encode
cd cloudflare && npm test  # Durable Object: 401, fan-out, late join, publisher replace, snap isolation, 503 without DeepSeek key

Layout

src/            Rust origin (encode, hub, HTTP/WS, publisher)
web/            local UI
cloudflare/     Worker + StreamRoom Durable Object + public player
tests/          cargo tests (HTTP API, player contract, 1080p30 stress)