From a18d31a0361f673e6bf5a1ed1022507859e667d5 Mon Sep 17 00:00:00 2001 From: Mish Ushakov <10400064+mishushakov@users.noreply.github.com> Date: Mon, 27 Jul 2026 00:23:07 +0200 Subject: [PATCH] Close the `O=` finding: an object is read once, so it is placed once One object per frame, every tile placed out of it at an offset, was tried a second time -- against kitty this time, since the first attempt had only Ghostty's silence to go on. kitty draws it exactly as Ghostty does, which is to say it does not, and that is what identified the cause: the spec has the terminal "read the data from the memory object and then unlink and close it". One read is all an object gets. A frame that places ninety-one tiles out of one object makes ninety-one commands of it, the first consumes the name, and every later `shm_open` finds nothing -- so the frame's first tile draws and the screen is otherwise black. Two implementations of a rule, not one implementation's gap. Which this codebase already knew. `src/term/shm.rs` sweeps by presence *because* an object that still exists is one the terminal has not read; the sentence that makes the sweep work is the one that makes one object per frame impossible. It was in the tree the whole time, two files from the change. So no code moves here: the emission is an object per tile, byte for byte what it was. What changes is what the tree says about why, in the three places that were guessing -- `place_shm`, the pty assertion, and the finding in RENDERING.md -- and the finding is closed rather than left open. It now says what is not worth doing (probing for it, gating on a terminal's name, trying a third terminal) and what would work: one *transmission* per frame, packing the tiles into a single atlas, `a=t` once, then `a=p` per tile with a source rectangle out of the stored image. That is one read, and legal. Its cost is written down too, because it is not small -- image ids become per-frame rather than per-tile, which drags in the `a=d,d=i` release, `TileGrid`'s id scheme, the cursor and overlay z-order, and deletes for an image store that grows a frame at a time. The numbers are re-measured on this machine rather than carried over, and the benchmark keeps the one-object row so the 0.2 ms still on the table stays a number rather than a memory: shm: object per tile 1.6 ms/frame -> 633 fps ceiling shm: per tile, packed in place 0.9 ms/frame -> 1124 fps ceiling shm: one object per frame 0.7 ms/frame -> 1528 fps ceiling Net of the two attempts: the larger half is in and unaffected -- packing into the mapping the terminal reads rather than into a buffer to be copied in, 1.6 ms/frame to 0.9. The offsets were the smaller half, and they were never available. One thing this does not have: the error reply that would confirm the mechanism instead of predicting it. `q=2` suppresses those, on both terminals. The finding says so, and says the `q=0` reproduction is two lines of shell for whoever wants it certain. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 15 ++++----- RENDERING.md | 80 +++++++++++++++++++++++++++++------------------ src/cli.rs | 14 ++++----- src/term/kitty.rs | 9 ++++-- tests/perf.rs | 12 +++---- tests/pty.rs | 7 +++-- 6 files changed, 81 insertions(+), 56 deletions(-) diff --git a/README.md b/README.md index 6bfd371..f493a71 100644 --- a/README.md +++ b/README.md @@ -207,10 +207,10 @@ Measured on a 1600x832 update in 91 tiles, single threaded | stage | per frame | ceiling | |---|---|---| -| pack BGRA → RGB into a buffer | 1.0 ms | 988 fps | -| `direct`: + zlib + base64 | 13.7 ms | **73 fps** | -| `shm`: an object per tile, packed then copied | 1.5 ms | **654 fps** | -| `shm`: an object per tile, packed in place | 0.9 ms | **1171 fps** | +| pack BGRA → RGB into a buffer | 1.1 ms | 914 fps | +| `direct`: + zlib + base64 | 14.0 ms | **71 fps** | +| `shm`: an object per tile, packed then copied | 1.6 ms | **633 fps** | +| `shm`: an object per tile, packed in place | 0.9 ms | **1124 fps** | zlib is the whole story on the direct path: compression and base64 cost twelve milliseconds that shared memory does not pay at all. On the shared memory path it is the @@ -218,9 +218,10 @@ pack — writing straight into the mapping the terminal will read, rather than i for a copy to follow, which is why the last row comes in below the buffered pack above it. One object for the whole frame instead of one per tile would be faster still, five system -calls rather than five per tile, and the protocol has the keys for it (`O=`/`S=`). Ghostty -draws nothing for a placement carrying them, so that is not what this does — -[`RENDERING.md`](RENDERING.md) has the details. +calls rather than five per tile, and the protocol has the keys for it (`O=`/`S=`): 0.7 ms +a frame, also measured. It cannot be reached a tile at a time, because a terminal unlinks +a shared memory object once it has read it, so only the first tile placed out of one ever +finds it — [`RENDERING.md`](RENDERING.md) has the finding and the shape that would work. So on a local terminal, full-screen motion is not CPU-bound on this side; over SSH, where shared memory cannot work, expect the compression ceiling. Server-side encoding and JPEG diff --git a/RENDERING.md b/RENDERING.md index b500ba6..78774eb 100644 --- a/RENDERING.md +++ b/RENDERING.md @@ -23,7 +23,7 @@ Every design decision below follows from one of these. | `a=d,d=I` frees one image and its placements | a tile the grid no longer reaches is dropped by name, not by erasing the screen | | at equal `z`, the higher image id composites above | the cursor and the overlay backdrops sit above the picture without a second z-layer | | `z=-1` is below text and above the cell background | the status line and the menu stay legible over the picture, and a blank cell does not hide it | -| `t=s` takes `O=` and `S=`, an offset and a length | one object could hold a whole frame -- except that Ghostty draws nothing for a placement carrying them, so it is an object per tile | +| `t=s` takes `O=` and `S=`, an offset and a length | nothing, as it turns out: the terminal unlinks an object once it has read it, so it is an object per tile -- see [the finding](#the-o-finding) | | DEC 2026 brackets an atomic update | a frame is one write, and the screen never shows half of one | | `Fence` synchronises, and `EnableContinuousUpdates` takes a flag | the server's pushing can be turned off and on rather than only declined, and encode time can be told from wire time | @@ -159,8 +159,9 @@ about how far the grid was filled. margin for the filter's support -- not the whole screen because one tile changed. 3. **Pack** each send straight into the shared memory object the terminal will read, so a frame is one pass over its pixels rather than a pack followed by a copy. An object per - tile: the protocol would allow one per frame with an offset per tile, five system calls - instead of five per tile, but see [the offset finding](#the-o-finding). + tile, because the terminal unlinks an object once it has read it -- one object for the + frame is worth 0.2 ms/frame and needs a different shape to get at, see + [the offset finding](#the-o-finding). 4. **Emit**, in this order, into one buffer: drops, the screen erase if this frame adopts a new geometry, the chrome's diff, the re-placements that erase owes, then moves and sends. Anything that clears cells precedes the placements on them, because a terminal may treat @@ -271,7 +272,7 @@ Built: - resize rate limit, leading edge (`src/session.rs`) - tiles packed straight into the shared memory object the terminal reads, rather than into a buffer for a copy to follow (`src/term/shm.rs`, `src/render/framebuffer.rs`) -- - 1.5 ms/frame to 0.9 on a full-screen update, measured by `make perf` + 1.6 ms/frame to 0.9 on a full-screen update, measured by `make perf` - shared memory swept by presence and a byte budget rather than a deadline (`src/term/shm.rs`) - resampling only the region a frame is about to send, grown by the filter's reach @@ -328,35 +329,53 @@ policy behind a flag, default off, until it has been watched for a while. ### The `O=` finding -One object per frame, with every tile placed out of it at an offset, was built and then -taken out again. It is worth writing down so it is not built a second time by accident. +One object per frame, with every tile placed out of it at an offset, was built twice and +taken out twice. The second attempt is what produced the explanation, so the finding is +closed now rather than open: **an object can be placed once, not once per tile.** The protocol has the keys. The spec says a client "can also specify a size and offset to tell the terminal emulator to only read a part of the specified file... using the `S` and `O` keys", and its own example uses them with `t=s`: `_Gs=10,v=2,t=s,S=80,O=10;`. What was emitted matched that shape, with -`S = w * h * 3` for `f=24`. - -Ghostty drew nothing for it. The screen was black with the occasional tile appearing as -the pointer moved -- consistent with the one placement per frame that carries `O=0` being -accepted and every other one being dropped, though that was not confirmed. And it was -silent, because `q=2` suppresses the reply that would have said why. - -To take it up again, in this order: - -1. Reproduce it deliberately with `q=0` and read what the terminal answers. That turns a - guess into a fact, and it is two lines of shell against a hand-made object. -2. Try kitty as well. If it works there, this is a Ghostty gap worth reporting rather than - a misreading of the spec. -3. Whatever the answer, gate it on the capability probe rather than on the terminal's name. - `src/term/caps.rs` already probes `t=s` with `a=q`, which validates a transmission - without storing anything -- the same probe with `S=` and `O=` set answers this question - at startup, and the answer decides whether a frame is one object or one per tile. - -The prize, measured by `make perf` on a full-screen update: 0.9 ms/frame against 0.6, most -of it system calls. Worth having, not worth guessing at. +`S = w * h * 3` for `f=24`. So the keys were right and the shape was wrong. + +What the same spec says about the medium is why the shape was wrong: a terminal "must read +the data from the memory object and then unlink and close it". One read is all an object +gets. A frame that places ninety-one tiles out of one object makes ninety-one commands of +it, and the first one consumes the name, so every later `shm_open` finds nothing. + +That predicts a black screen with the frame's first tile drawing and nothing else, which is +what Ghostty showed, and then what kitty showed too -- two implementations of a rule rather +than one implementation's gap. The prediction was never confirmed against an error reply: +`q=2` suppressed those on both. Anyone who wants it nailed down can reproduce it with `q=0` +against a hand-made object and read the answer, which is two lines of shell. + +That the unlink rule was the mechanism should have been obvious from this codebase, which +depends on it elsewhere: `src/term/shm.rs` sweeps by presence precisely because "an object +that still exists is one the terminal has not read". The same sentence that makes the sweep +work makes one object per frame impossible. + +**What is not worth doing.** Probing for it. Gating on a terminal's name. Trying a third +terminal. The rule is in the spec and both implementations follow it. + +**What would work**, if the 0.7 ms row is ever worth the complexity: one *transmission* per +frame rather than one per tile. Pack the frame's tiles into a single rectangular atlas in +one object, transmit it once with `a=t` -- one read, one unlink, five system calls -- and +then place each tile with `a=p` and a source rectangle (`x=`/`y=`/`w=`/`h=`) out of the +stored image. What that costs, and why it is not free: a tile's image id is how the grid +addresses it today, and this makes the frame the unit instead, so the `a=d,d=i` release +before each placement, the id-per-tile scheme in `TileGrid` and the z-order the cursor and +the overlays rely on all have to be reworked -- and an image store that grows a frame at a +time needs its own deletes, which is the shape of leak this protocol has already produced +once (see the `a=d,d=i` note in `src/term/kitty.rs`). + +The prize, measured by `make perf` on a full-screen update: 0.7 ms/frame against 0.9, most +of it system calls -- and it is the smaller half of what the second attempt was worth. The +larger half, packing into the mapping rather than into a buffer to be copied in, is 1.6 to +0.9 and is in. The benchmark keeps the one-object row so the remaining 0.2 ms stays a +number rather than a memory, and `pixels_travel_through_shared_memory_when_the_terminal_offers_it` asserts that no `O=` -appears in a placement, so this cannot come back without someone reading this first. +appears in a placement, so this cannot come back a third time by accident. ### Considered and rejected @@ -370,6 +389,7 @@ fold-in policy. It profiles well on the case we are already fast at. **Tile size as a constant.** `TILE_TARGET_PX = 128` is a tuning parameter dressed as a law: whole-screen scrolling wants larger tiles to amortise the escape and syscall overhead, a caret wants smaller. Choosing it from the layout, or from the damage pattern, is a real -improvement -- but measure first, and note that the per-tile system calls are still there, -so the argument for larger tiles is stronger than it looked when one object per frame -seemed to be on the table. +improvement -- but measure first, and note that the per-tile system calls are here to stay +unless a frame becomes one transmission (see [the offset finding](#the-o-finding)), so the +argument for larger tiles is stronger than it looked when one object per frame seemed to be +on the table. diff --git a/src/cli.rs b/src/cli.rs index 38293ad..5eb686c 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -151,17 +151,17 @@ impl Transfer { // not small. Measured on a full-screen 1600x832 update in 91 tiles // (`cargo test --release --test perf -- --ignored --nocapture`): // - // pack BGRA->RGB into a buffer 1.0 ms/frame - // direct: +zlib +base64 13.7 ms/frame -> 73 fps ceiling - // shm: object per tile 1.5 ms/frame -> 654 fps ceiling - // shm: per tile, packed in place 0.9 ms/frame -> 1171 fps ceiling + // pack BGRA->RGB into a buffer 1.1 ms/frame + // direct: +zlib +base64 14.0 ms/frame -> 71 fps ceiling + // shm: object per tile 1.6 ms/frame -> 633 fps ceiling + // shm: per tile, packed in place 0.9 ms/frame -> 1124 fps ceiling // // zlib is the whole story on the direct path. On the shared memory one it // is the pack: writing into the mapping rather than into a buffer for a // copy to follow is most of the difference between the last two rows. One - // object for the whole frame would be faster still -- five system calls - // rather than five a tile -- but Ghostty draws nothing for a placement - // carrying an `O=` offset; see RENDERING.md. The probe is what makes this + // object for the whole frame would be faster still -- 0.7 ms/frame, also + // measured -- but an object can be placed only once, because the terminal + // unlinks what it reads; see RENDERING.md. The probe is what makes this // safe to default to -- // frames go out with responses suppressed, so a medium the terminal // cannot handle would fail invisibly, and silence about `t=s` sends us diff --git a/src/term/kitty.rs b/src/term/kitty.rs index 2af066f..183945a 100644 --- a/src/term/kitty.rs +++ b/src/term/kitty.rs @@ -192,9 +192,12 @@ impl KittyEncoder { /// /// The object holds exactly this tile and nothing else, so there are no `O=`/`S=` /// keys. The protocol has them, and one object per frame with an offset per tile - /// would be five system calls a frame rather than five a tile -- but Ghostty draws - /// nothing for a placement carrying them, and with `q=2` it does not say why. Until - /// that is understood, a frame is an object per tile. + /// would be five system calls a frame rather than five a tile -- but a terminal + /// "must read the data from the memory object and then unlink and close it", so an + /// object can only be placed *once*: the first tile of the frame consumes the name + /// and every later one opens nothing. Tried against Ghostty and kitty, and neither + /// draws it. One transmission per frame is the shape that would work; see + /// RENDERING.md. pub fn place_shm(&mut self, out: &mut Vec, at: Placement, name: &str) { let Placement { id, col, row, w, h } = at; if w == 0 || h == 0 { diff --git a/tests/perf.rs b/tests/perf.rs index c6c1cdd..8bf0f5f 100644 --- a/tests/perf.rs +++ b/tests/perf.rs @@ -306,10 +306,10 @@ fn full_screen_compose_throughput() { total, ); - // And the way it would be done if `O=` worked: one object for the frame, every tile - // packed into it at an offset of its own. Five system calls a frame rather than five a - // tile -- but Ghostty draws nothing for a placement carrying an offset, so this is a - // measurement of the road not taken. See RENDERING.md. + // One object for the frame, every tile packed into it at an offset of its own: five + // system calls a frame rather than five a tile. Not what the renderer does -- an object + // can only be read once, because the terminal unlinks it -- but the row is the floor a + // single transmission per frame would be working towards. See RENDERING.md. let frame_bytes: usize = tiles.iter().map(|&(_, _, w, h)| (w * h * 3) as usize).sum(); let start = Instant::now(); let mut total = 0; @@ -324,8 +324,8 @@ fn full_screen_compose_throughput() { println!( "The pack row appends into a buffer, which is what the direct medium still does;\n\ the rows below it pack into the mapping instead, so they come out under it.\n\ - One object per frame is what `O=` would buy: see RENDERING.md for why it is not\n\ - what the renderer does." + The last row is the syscall floor of one object per frame, which needs one\n\ + transmission per frame to reach: see RENDERING.md." ); println!("A partial update costs proportionally less. Server-side encoding, JPEG"); println!("decode and the terminal's own work all come on top."); diff --git a/tests/pty.rs b/tests/pty.rs index e4566ae..da1bc69 100644 --- a/tests/pty.rs +++ b/tests/pty.rs @@ -340,9 +340,10 @@ fn pixels_travel_through_shared_memory_when_the_terminal_offers_it() { // An object per tile, each holding only that tile, and no `O=`/`S=` keys. // // The protocol has them, and one object per frame with an offset per tile would be - // five system calls a frame rather than five a tile -- but Ghostty draws nothing at - // all for a placement carrying them, and with `q=2` it does not say why. This asserts - // the shape that works, so the other one cannot come back without an answer. + // five system calls a frame rather than five a tile -- but the terminal unlinks an + // object once it has read it, so only the first tile placed out of one can find it. + // Ghostty and kitty both draw the frame's first tile and nothing else. This asserts + // the shape that works; RENDERING.md has the shape that would. let frame = frame_containing(&output, b"f=24,t=s,i=").expect("no frame carried a tile"); let placements = shm_placements(frame); assert!(