Skip to content

fix(gateway): hold back half-streamed MEDIA:/[[…]] directives - #138551

Open
santa-sketch wants to merge 1 commit into
NousResearch:mainfrom
santa-sketch:fix/stream-pending-media-directive
Open

santa-sketch wants to merge 1 commit into
NousResearch:mainfrom
santa-sketch:fix/stream-pending-media-directive

Conversation

@santa-sketch

Copy link
Copy Markdown

What does this PR do?

With streaming on (Telegram, transport: edit), a reply that is only a voice note
(MEDIA:/…/cache/audio/tts_….ogg) leaves a visible text bubble with a partial path
(MEDIA:/home/…) above the voice note. Same for [[audio_as_voice]] while it is being typed.

Cause: interim edits fire per chunk, and _clean_for_display only strips complete
deliverable tags (by design, #34517). A tag streamed without its extension yet is shown.
At finish the cleaned final is empty, so _adopt_final_text adopts nothing and the bubble
is never edited again: the partial path stays.

Fix: _should_edit now also defers interim edits while the open (last) line of the cleaned
buffer may still become a directive: an unstripped MEDIA: on that line, a word-start
prefix of MEDIA, or a prefix of [[audio_as_voice]] / [[as_document]]. This mirrors the
existing _is_partial_silence_marker hold-back. The final (non-interim) flush is unchanged,
so undeliverable tags still show up as before.

Related Issue

No existing issue found (searched "MEDIA stream", "streaming media tag", "audio_as_voice").
#73537 fixes the same split-tag class for the API server's SSE endpoints; this is the gateway
stream consumer path.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • gateway/stream_consumer.py: _has_pending_media_directive() + use it in _should_edit().
  • tests/gateway/test_stream_consumer_pending_media.py: voice-only reply streamed in chunks,
    a split [[audio_as_voice]], and prose mentioning "MEDIA" (still streams).

How to Test

  1. scripts/run_tests.sh tests/gateway/test_stream_consumer_pending_media.py — 2 of the 3 tests fail on main, all pass with the fix.
  2. scripts/run_tests.sh over tests/gateway/test_{stream,media,telegram,…voice,…tts,…silence}* (120 files): 1165 passed, 0 failed.
  3. Live: Telegram with streaming on, ask for a voice-only reply; no text bubble above the voice note.

Checklist

Code

  • I've read the Contributing Guide
  • Conventional Commits
  • Searched existing PRs
  • Only changes related to this fix
  • Ran the related gateway test files (above); not the full suite
  • python scripts/check passes

🤖 Generated with Claude Code

Interim stream edits could show a delivery directive before it was complete:
the display cleaner only strips full deliverable tags, so `MEDIA:/home/…`
without its extension, or `[[audio_`, went out as text. For a voice-only
reply the cleaned final is empty, so finalize never edits the bubble and the
partial path stays visible above the voice note on Telegram.

Defer interim edits while the open line may still become a directive, the
same way partial silence markers are deferred. The final flush is unchanged,
so undeliverable tags stay visible (NousResearch#34517).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery tool/tts Text-to-speech and transcription sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Oct 14, 2026
@100yenadmin

Copy link
Copy Markdown

Seen in production use, and this PR fixes it. Details from the Fish Audio plugin side:

Live report (Hermes v0.21.7, Telegram, streaming: true). The plugin tool fish_speak returns [[audio_as_voice]]\nMEDIA:<path>.ogg. Plugin tools aren't on the gateway's auto-append list, so the model's whole reply was that tag. The voice note arrived, but above it a text message stayed frozen at MEDIA ▉; edit_date was null, so it was never edited. The gateway logged Suppressing normal final send … final delivery already confirmed (streamed=True …).

Why the bubble stays. At finish the cleaned final is empty. _send_or_edit takes the if not visible_stripped: branch, which returns True without editing or deleting the preview unless a native stream is open. So the half-streamed prefix becomes the permanent message. With the 24-character default threshold, [[audio_as_voice]]\nMEDIA is exactly the first send.

Probe. I drove GatewayStreamConsumer with a fake Telegram adapter, default config and chunked deltas, against v0.21.7 source:

Case v0.21.7 With this PR applied
media-only, interim edits skipped (Telegram's per-chat edit slot busy) send('\nMEDIA ▉'), never edited no text message
media-only, interim edits allowed send('\nMEDIA ▉'), then edits ending on '\nMEDIA:<local path without extension> ▉', which leaves a local path visible no text message
prose, then the tag the fragment flashes mid-stream; the final edit clears it the final edit is 'Here you go.', with no fragment mid-stream

Possible follow-up, not needed for this PR: the empty-final branch could also delete an existing preview. That would cover other cases where a preview was sent and the final cleans to empty.

On the plugin side we're asking the model for one line of text next to the tag (fish-audio-hermes 1.7.4), but this PR is the real fix.

100yenadmin added a commit to 100yenadmin/fish-audio-hermes that referenced this pull request Oct 14, 2026
…teway tests (#44)

Report a problem: only an explicit yes to publishing, given after the user has seen the final text, approves filing (shared kit v2). fish_speak asks the agent for a short text line next to the audio tag, so a streamed Telegram reply no longer leaves a frozen "MEDIA" message (core fix: NousResearch/hermes-agent#138551). Desktop gateway tests use a temporary Hermes home for the dashboard's own profile.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants