Skip to content

Replace TransposeDecoder switch with preserve_none tail calls and frameless handlers - #47

Draft
copybara-service[bot] wants to merge 1 commit into
masterfrom
test_1011404129
Draft

copybara-service[bot] wants to merge 1 commit into
masterfrom
test_1011404129

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Replace TransposeDecoder switch with preserve_none tail calls and frameless handlers

On 64-bit Clang x86-64 and AArch64 targets (RIEGELI_TAILCALL), replaces the
central HandleNode() switch dispatch in riegeli::TransposeDecoder::Decode
with 6-register preserve_none + musttail per-handler tail calls across 10
localized hunks (+412 / -1 lines in transpose_decoder.cc, with zero edits to
existing DecodingState callbacks or MaskBuffer overloads, and preserving the
compact 16B/24B StateMachineNode layout and untouched HandleNode() +
TransitionNode() switch loop on 32-bit and non-musttail targets):

  1. Under #if RIEGELI_TAILCALL (defined(__clang__) && ABSL_HAVE_CPP_ATTRIBUTE(clang::musttail) && (defined(__aarch64__) || defined(__x86_64__)) && !defined(__ILP32__)), aligns StateMachineNode to
    32 bytes (alignas(32), zero dead padding) by adding a direct Callback
    function pointer alongside bool is_implicit : 1 and
    CallbackType callback_type (with Context's 128-node inline buffer,
    Context uses ~4.1 KB on Decode's stack). Other targets omit Callback
    and alignas(32), keeping StateMachineNode at 8 + 2 * sizeof(void*).
  2. Dispatches via [[clang::preserve_none]] + [[clang::musttail]] handlers
    (falling back to the standard calling convention + [[clang::musttail]]
    under AddressSanitizer and HWAddressSanitizer and on non-Linux targets),
    pinning state, node, num_iters, transitions_cursor,
    transitions_limit, and dest in registers across transitions, pairing
    frameless fast-path wrappers
    (InvokeVarint, InvokeFixed, InvokeCopyTag, InvokeMessageStart,
    InvokeSubmessageStart, InvokeSubmessageEnd,
    InvokeSkippedSubmessageStart, InvokeSkippedSubmessageEnd) with
    ABSL_ATTRIBUTE_NOINLINE cold/slow helpers and InvokeSlowCallback, and
    #undefing all local RIEGELI_* macros at EOF.

@google-cla

google-cla Bot commented Nov 1, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@copybara-service copybara-service Bot changed the title Dispatch TransposeDecoder callbacks via tail-called table Replace TransposeDecoder switch with preserve_none tail calls Nov 2, 2026
@copybara-service copybara-service Bot changed the title Replace TransposeDecoder switch with preserve_none tail calls Replace TransposeDecoder switch with preserve_none tail calls and frameless handlers Nov 3, 2026
@copybara-service
copybara-service Bot force-pushed the test_1011404129 branch 4 times, most recently from 418c428 to c2e660b Compare November 3, 2026 05:17
…meless handlers

On 64-bit Clang x86-64 and AArch64 targets (`RIEGELI_TAILCALL`), replaces the
central `HandleNode()` `switch` dispatch in `riegeli::TransposeDecoder::Decode`
with 6-register `preserve_none` + `musttail` per-handler tail calls across 10
localized hunks (`+412 / -1` lines in `transpose_decoder.cc`, with zero edits to
existing `DecodingState` callbacks or `MaskBuffer` overloads, and preserving the
compact `16B`/`24B` `StateMachineNode` layout and untouched `HandleNode()` +
`TransitionNode()` `switch` loop on 32-bit and non-`musttail` targets):
1. Under `#if RIEGELI_TAILCALL` (`defined(__clang__) &&
   ABSL_HAVE_CPP_ATTRIBUTE(clang::musttail) && (defined(__aarch64__) ||
   defined(__x86_64__)) && !defined(__ILP32__)`), aligns `StateMachineNode` to
   32 bytes (`alignas(32)`, zero dead padding) by adding a direct `Callback`
   function pointer alongside `bool is_implicit : 1` and
   `CallbackType callback_type` (with `Context`'s 128-node inline buffer,
   `Context` uses ~4.1 KB on `Decode`'s stack). Other targets omit `Callback`
   and `alignas(32)`, keeping `StateMachineNode` at `8 + 2 * sizeof(void*)`.
2. Dispatches via `[[clang::preserve_none]]` + `[[clang::musttail]]` handlers
   (falling back to the standard calling convention + `[[clang::musttail]]`
   under AddressSanitizer and HWAddressSanitizer and on non-Linux targets),
   pinning `state`, `node`, `num_iters`, `transitions_cursor`,
   `transitions_limit`, and `dest` in registers across transitions, pairing
   frameless fast-path wrappers
   (`InvokeVarint`, `InvokeFixed`, `InvokeCopyTag`, `InvokeMessageStart`,
   `InvokeSubmessageStart`, `InvokeSubmessageEnd`,
   `InvokeSkippedSubmessageStart`, `InvokeSkippedSubmessageEnd`) with
   `ABSL_ATTRIBUTE_NOINLINE` cold/slow helpers and `InvokeSlowCallback`, and
   `#undef`ing all local `RIEGELI_*` macros at EOF.

PiperOrigin-RevId: 1011404129
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants