From 3d076beb632dce1cb189dc72dbc23c682661884b Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 04:07:43 +0100 Subject: [PATCH 01/40] feat(ohttp): bound a sealed request by an inner x-date and time attempts by method The gateway now refuses, silently as it does a replay, a sealed request whose inner x-date is missing or unparseable, more than 90 s from its own clock, or more than 15 s before it started. The 15 s is the client's rounding to the nearest 30 s, so a request sealed just after a deploy is still answered. The check runs after decapsulation and before the replay register records the enc, so a stale copy records nothing. The HTTP route shares unseal and answers such a request with an outer 400. The header is x-date, not RFC 9458's Date: Date is a forbidden request header, which a browser's Request strips. Node's undici keeps it, so node tests would pass while every browser request arrived undated. A test pins that the inner request uses no forbidden name. The sealed fetch stamps x-date itself, over any caller value, and both sides now forward x-digest: it is arkd's server-global config digest, and dropping it would blind the SDK's signer-rotation trigger. The gateway never forwards x-date. A GET attempt now gets 9 s, so three fit inside the SDK's 30 s read abort; a non-GET keeps 40 s. A POST stops attempting 45 s after its one sealing and ends outcome-unknown, so every copy of its single x-date (45 s plus 15 s of rounding) reaches the gateway inside the 90 s window. A stale request still takes a rate slot, like every request the gateway opens: the rate check runs before opening, which keeps the rate the bound on decapsulation work. --- docs/DOKPLOY.md | 10 ++--- src/rollup/ohttp-gateway.ts | 11 +++-- src/shielded/sealed-fetch.ts | 22 ++++++---- tests/nostr-carrier.test.ts | 2 +- tests/ohttp-gateway.test.ts | 33 ++++++++++++++- tests/rollup-http.test.ts | 2 +- tests/sealed-fetch.test.ts | 80 ++++++++++++++++++++++++++++-------- 7 files changed, 122 insertions(+), 38 deletions(-) diff --git a/docs/DOKPLOY.md b/docs/DOKPLOY.md index bf5a40c5..eecb6452 100644 --- a/docs/DOKPLOY.md +++ b/docs/DOKPLOY.md @@ -28,7 +28,7 @@ The web app runs from a service worker cache. A browser checks `/sw.js` about on The pool server can open Oblivious HTTP (RFC 9458) requests, so a wallet reaches the pool, arkd, the Arkade indexer and Esplora through a relay that sees its address but not what it asks for. The gateway is off until `SHIELDED_OHTTP_KEY` is set. While it is off, `GET /api/rollup/ohttp-keys` and `POST /api/rollup/ohttp` answer 404 once the pool has opened. Before that they get the pool's 503 instead: "The rollup pool is starting.", or "The rollup pool is not open yet" while the pool is not open, for example while a rollout waits for the volume lock. -**Turn it on.** For testing only for now. Production use waits for three things: a measurement of CPU and memory under a flood on the real host, a replay window, and a way to rotate the key without stranding wallets. The Nostr carrier lets anyone reach the gateway through public relays, so there is no list of callers to hold it to. Every event a listed relay forwards, a flooder's included, costs the pool's main thread work as it arrives, and every answer costs signing and bandwidth going out. Today a repeated sealed request is refused only while an in-memory register remembers it, and a restart clears that register. A time bound needs the wallet to send an inner `Date` (RFC 9458 section 6.5.1), which is wallet-side work. Rotation is a hard cut (see **Rotate**), so before any wallet pins a config, the gateway needs to hold the old and new keys together, or wallets need a stale-key signal they act on. Run `npm run ohttp:keygen -- ` on a trusted machine. It writes the line `SHIELDED_OHTTP_KEY=:` to `` (mode 0600, which holds on Linux and macOS but not on Windows, and it refuses to overwrite an existing file) and prints only the key id, the config and the gateway's Nostr key (`nostr: `), which are public. Paste the line into the application's runtime **Environment**, delete the file and redeploy. Never put the key in a command line, a log, a chat or the repository. Publish the printed config and Nostr key: the wallet build that uses the gateway pins both. +**Turn it on.** For testing only for now. Production use waits for two things: a measurement of CPU and memory under a flood on the real host, and a way to rotate the key without stranding wallets. The Nostr carrier lets anyone reach the gateway through public relays, so there is no list of callers to hold it to. Every event a listed relay forwards, a flooder's included, costs the pool's main thread work as it arrives, and every answer costs signing and bandwidth going out. Rotation is a hard cut (see **Rotate**), so before any wallet pins a config, the gateway needs to hold the old and new keys together, or wallets need a stale-key signal they act on. Run `npm run ohttp:keygen -- ` on a trusted machine. It writes the line `SHIELDED_OHTTP_KEY=:` to `` (mode 0600, which holds on Linux and macOS but not on Windows, and it refuses to overwrite an existing file) and prints only the key id, the config and the gateway's Nostr key (`nostr: `), which are public. Paste the line into the application's runtime **Environment**, delete the file and redeploy. Never put the key in a command line, a log, a chat or the repository. Publish the printed config and Nostr key: the wallet build that uses the gateway pins both. **Verify.** The start log shows `Oblivious HTTP gateway on (key id N, Nostr 3c857963…, 6 relays, HTTP route off).`, with the key id and the first 8 characters of the Nostr key the tool printed. `GET /api/rollup/ohttp-keys` then returns content type `application/ohttp-keys`, and its bytes equal the printed config. This prints the content type and the bytes in the tool's base64url encoding: @@ -59,7 +59,7 @@ If the pool's public origin is also an upstream's, only `/api/rollup/` paths on **What the shared budget costs.** One holder of the public key config can submit at the whole `SHIELDED_OHTTP_RATE`: 20 a second by default, about 1,200 a minute, against 60 a minute per address when calling the pool directly. Each submission costs a Groth16 check. The budget is shared and first come, first served, so that caller crowds out every other sealed caller, reads included. It can also push direct `/spends` requests to a 503 "busy", because the cap on spends being checked at once (8) is shared by everyone. To cut it, lower `SHIELDED_OHTTP_RATE` or unset the key. -**Limits and logs.** A request over 1 MiB gets an outer 413, and one still uploading after 30 seconds is cut off. A response over 4 MiB, or an upstream that takes over 30 seconds, comes back as an inner 502 or 504. Whatever the upstream or the reading client does, the connection is cut after 90 seconds, three times that deadline, so no request holds a slot longer. Of the upstream's headers only the content type, the cache control and `Retry-After` cross, so a sealed client can wait as told. All callers share one budget, `SHIELDED_OHTTP_RATE` a second and 32 requests in flight; past it the gateway answers an outer 429 or 503 with `Retry-After: 1`. It logs counts by kind, such as `[ohttp] forwarded 120, busy 3`, at most once a minute, and never a target or a body. +**Limits and logs.** A request over 1 MiB gets an outer 413, and one still uploading after 30 seconds is cut off. A response over 4 MiB, or an upstream that takes over 30 seconds, comes back as an inner 502 or 504. Whatever the upstream or the reading client does, the connection is cut after 90 seconds, three times that deadline, so no request holds a slot longer. A sealed request carries an inner `X-Date` rounded to the nearest 30 seconds (not `Date`, which a browser strips). One dated more than 90 seconds from the gateway's clock, or more than 15 seconds before the gateway started, is refused like a repeat: an outer 400 on the HTTP route, and no answer over Nostr. Of the client's headers only the content type, `Accept` and `X-Digest` cross to the upstream, and of the upstream's only the content type, the cache control and `Retry-After`, so a sealed client can wait as told. All callers share one budget, `SHIELDED_OHTTP_RATE` a second and 32 requests in flight; past it the gateway answers an outer 429 or 503 with `Retry-After: 1`. It logs counts by kind, such as `[ohttp] forwarded 120, busy 3`, at most once a minute, and never a target or a body. **Rotate.** The gateway holds one key at a time. Run `ohttp:keygen` again. It picks the key id at random, so if it prints the id already in use, run it again. Replace the `SHIELDED_OHTTP_KEY` line, redeploy and publish the new config. The wallet build that uses the gateway pins the config, so a rotation also needs a new app build. Until a client has the new config, its requests are sealed to the old key id, and the gateway answers them with an outer 400. @@ -69,15 +69,15 @@ If the pool's public origin is also an upstream's, only `/api/rollup/` paths on ### Nostr carrier -A wallet can also send its sealed request inside a Nostr event, so no relay of ours or Arkade's has to carry it. The listener subscribes to each relay for kind 21470 events addressed to the gateway's Nostr key and opens each through the same gateway as the HTTP route, with the same rate, in-flight cap and repeat check. Requests are grouped by the hash of their sealed bytes, so every copy of a request, from any relay and from anyone, shares one run. The listener publishes the sealed answer as kind 21471 events signed by that key, to every relay that delivered a copy. A copy from a new relay within 30 seconds of the answer gets the answer re-published there; the listener keeps answers for this up to 16 MiB in total. A request is one event carrying at most 32 KiB of sealed bytes, and an answer is a set of events carrying 32 KiB each: the format allows 32, and the listener sends at most 8. In base64 with the event's fields that is about 44 kB an event, so a relay needs an event size limit above that. The listener asks the gateway for no more than 241,644 bytes (about 236 KiB) of upstream body, the most whose sealed answer fits 8 events whatever headers come with it, so a larger upstream answer comes back as an inner 502. That guarantee assumes Node's default 16 KiB limit on response headers. With a larger `--max-http-header-size`, for example through `NODE_OPTIONS`, an answer can take more than 8 events, though never more than 32. When the gateway is busy the listener publishes a status event instead (503 over 32 requests in flight, 429 over the rate, each with a retry delay of 1 second and the id of the request event it answers) and runs nothing, so the wallet may send again. Answer events, re-published ones included, draw on an outbound budget of 64 publications a second, one event to one relay each. An admitted answer is published whole even if that puts the budget into debt, and a new request that finds the budget spent gets a 503 status before it runs. Answers to requests that ran always go out, so the budget can fall into debt by the answers to the 32 requests in flight, at up to 8 events to each of up to 12 relays that carried a copy (3,072 events), plus re-publishes of answers signed in the last 30 seconds to the 11 other relays: about 4,200 events if all are full-size, which the 16 MiB cache bounds, about 6,600 if all are small, which the default `SHIELDED_OHTTP_RATE` of 20 a second bounds, and about 10,600 for the worst mix, small answers beside two-event answers just over one chunk. That is about 14,000 events at worst, and every new request is then shed with a 503 for up to about 3.5 minutes while the budget repays them at 64 a second. Fewer relays in `SHIELDED_NOSTR_RELAYS`, or a lower `SHIELDED_OHTTP_RATE`, lowers that ceiling; the in-flight cap of 32 is fixed. At most 5 status events go out a second: past that the listener publishes nothing and counts `status-capped`, and the wallet's attempt times out instead, so a GET is sent again and a POST ends `outcome-unknown`. A request the gateway cannot open (not sealed to this key, or a repeat) gets no answer, and neither does any event that is not a valid request. +A wallet can also send its sealed request inside a Nostr event, so no relay of ours or Arkade's has to carry it. The listener subscribes to each relay for kind 21470 events addressed to the gateway's Nostr key and opens each through the same gateway as the HTTP route, with the same rate, in-flight cap, repeat check and date window. Requests are grouped by the hash of their sealed bytes, so every copy of a request, from any relay and from anyone, shares one run. The listener publishes the sealed answer as kind 21471 events signed by that key, to every relay that delivered a copy. A copy from a new relay within 30 seconds of the answer gets the answer re-published there; the listener keeps answers for this up to 16 MiB in total. A request is one event carrying at most 32 KiB of sealed bytes, and an answer is a set of events carrying 32 KiB each: the format allows 32, and the listener sends at most 8. In base64 with the event's fields that is about 44 kB an event, so a relay needs an event size limit above that. The listener asks the gateway for no more than 241,644 bytes (about 236 KiB) of upstream body, the most whose sealed answer fits 8 events whatever headers come with it, so a larger upstream answer comes back as an inner 502. That guarantee assumes Node's default 16 KiB limit on response headers. With a larger `--max-http-header-size`, for example through `NODE_OPTIONS`, an answer can take more than 8 events, though never more than 32. When the gateway is busy the listener publishes a status event instead (503 over 32 requests in flight, 429 over the rate, each with a retry delay of 1 second and the id of the request event it answers) and runs nothing, so the wallet may send again. Answer events, re-published ones included, draw on an outbound budget of 64 publications a second, one event to one relay each. An admitted answer is published whole even if that puts the budget into debt, and a new request that finds the budget spent gets a 503 status before it runs. Answers to requests that ran always go out, so the budget can fall into debt by the answers to the 32 requests in flight, at up to 8 events to each of up to 12 relays that carried a copy (3,072 events), plus re-publishes of answers signed in the last 30 seconds to the 11 other relays: about 4,200 events if all are full-size, which the 16 MiB cache bounds, about 6,600 if all are small, which the default `SHIELDED_OHTTP_RATE` of 20 a second bounds, and about 10,600 for the worst mix, small answers beside two-event answers just over one chunk. That is about 14,000 events at worst, and every new request is then shed with a 503 for up to about 3.5 minutes while the budget repays them at 64 a second. Fewer relays in `SHIELDED_NOSTR_RELAYS`, or a lower `SHIELDED_OHTTP_RATE`, lowers that ceiling; the in-flight cap of 32 is fixed. At most 5 status events go out a second: past that the listener publishes nothing and counts `status-capped`, and the wallet's attempt times out instead, so a GET is sent again and a POST ends `outcome-unknown`. A request the gateway cannot open (not sealed to this key, a repeat, or dated outside the window) gets no answer, and neither does any event that is not a valid request. **Relays.** `SHIELDED_NOSTR_RELAYS` is a comma-separated list of `wss://` URLs. Unset or blank, it is `wss://relay.snort.social`, `wss://nostr.oxtr.dev`, `wss://nostr.data.haus`, `wss://nostr.sathoarder.com`, `wss://relay.mostro.network` and `wss://nostr-01.yakihonne.com`, none of which sat behind Cloudflare when probed on 2026-11-10. Choose relays by the same rule as the HTTP relay: a relay sees who is publishing and the gateway sees what, so no relay may be run by us or Arkade (never `wss://nostr.arkade.sh`), and none may sit behind Cloudflare, which already fronts this domain. Every relay listed must also echo an event to the connection that published it, and its message limit must be 512 KiB or less, as its NIP-11 `max_message_length` or a probe shows, which leaves headroom under the gateway's 1 MiB frame cap for the envelope a relay forwards an event in. The six defaults allow at most 512 KiB. A listed relay must also forward events byte for byte, without re-escaping `<`, `>`, `&`, control characters or non-ASCII characters, because re-escaping can push a forwarded event past the 1 MiB frame cap. A wallet publishes to relays from its own list. A GET that reaches only relays the gateway does not listen on loses an attempt. A POST that does loses the whole request: a relay accepted it, so the wallet never sends it again and it ends `outcome-unknown`. Keep this list equal to the wallet build's or larger. **The key.** The gateway's Nostr key is derived from the OHTTP key (HKDF-SHA256 over the 32-byte secret, with the info `shielded-nostr-carrier-v1`), so there is no second secret to keep and a rotation changes both. `ohttp:keygen` prints its public half as `nostr: `, and the start log shows the first 8 characters. Run one instance per key: two servers with the same `SHIELDED_OHTTP_KEY` both hear every request and both answer it, so a GET gets two conflicting sets of chunks and a POST to arkd runs twice. -**Start and stop.** The listener starts once, the first time the pool has opened, and logs `Nostr carrier listening on N relays.` It closes before the pool does. While a rollout waits for the previous instance to release the volume, the new instance does not listen. It asks each relay only for requests from then on (`limit:0`, and none dated more than about a minute earlier), so a restarted instance does not run again the requests a relay still stores; a relay that ignores `limit:0` can still hand back the last minute's. The same holds when the listener resubscribes after losing a relay: requests that reached that relay during the gap are never heard. They never run, so a GET goes out again through another relay. A POST that only that relay took ends `outcome-unknown`, even though it did not run. +**Start and stop.** The listener starts once, the first time the pool has opened, and logs `Nostr carrier listening on N relays.` It closes before the pool does. While a rollout waits for the previous instance to release the volume, the new instance does not listen. It asks each relay only for requests from then on (`limit:0`, and none dated more than about a minute earlier), so a restarted instance does not run again the requests a relay still stores; a relay that ignores `limit:0` can still hand back the last minute's. The same holds when the listener resubscribes after losing a relay: requests that reached that relay during the gap are never heard. They never run, so a GET goes out again through another relay. A POST that only that relay took ends `outcome-unknown`, even though it did not run. Across a restart, the gateway refuses any request whose `X-Date` is more than 15 seconds before its process started, but its register starts empty: a request dated after that, which the previous instance already answered (it keeps answering while a rollout waits for the volume lock), can run once more if a copy reaches the new instance within 90 seconds of its `X-Date`. -**What a relay and its readers see.** The relay sees the address of the wallet that publishes, and the event: a one-time key as author, the gateway's key in the `p` tag, a time rounded to the nearest 30 seconds and the sealed bytes. Each relay also sees the pool server's own address, which Cloudflare otherwise hides: the listener's connections to the relays go out from the server's egress address. Anyone who reads the relay sees the same events without the address, and the answers: the gateway's key as author, the hash of the sealed request (`x`) and the chunk numbers in the tags, and when. A status event carries, instead of chunk numbers, its status code, its retry delay and the id of the request event it answers. Neither learns the target or the body, which only the gateway can open. The gateway learns them but not the address, because a relay is the only path a request takes to it. Sizes show: a request's length and an answer's chunk count say roughly how much was asked and returned. Anyone who reads the relay can also copy a request's sealed bytes into an event of their own, and the gateway runs them again once its register of opened requests has forgotten them, which a restart does (see the replay window under **Turn it on**). +**What a relay and its readers see.** The relay sees the address of the wallet that publishes, and the event: a one-time key as author, the gateway's key in the `p` tag, a time rounded to the nearest 30 seconds and the sealed bytes. Each relay also sees the pool server's own address, which Cloudflare otherwise hides: the listener's connections to the relays go out from the server's egress address. Anyone who reads the relay sees the same events without the address, and the answers: the gateway's key as author, the hash of the sealed request (`x`) and the chunk numbers in the tags, and when. A status event carries, instead of chunk numbers, its status code, its retry delay and the id of the request event it answers. Neither learns the target or the body, which only the gateway can open. The gateway learns them but not the address, because a relay is the only path a request takes to it. Sizes show: a request's length and an answer's chunk count say roughly how much was asked and returned. Anyone who reads the relay can also copy a request's sealed bytes into an event of their own. The window, not the register of opened requests, is what bounds a replay: a copy that reaches the gateway more than 90 seconds after the original's `X-Date` gets no answer at all, and within those 90 seconds the register refuses it, except across a restart (see **Start and stop**). **Logs.** The listener logs counts by kind, such as `[nostr] answered 12, busy 1, dropped 3`, at most once a minute, and never an event, a relay or a target. `answered` and `busy` are published; `status-capped` is a status event held back because 5 went out in that second; `refused` is a request the gateway would not open; `dropped` is an event that is not a valid request; `duplicate` is another copy of a request already seen; `republished` is a cached answer sent to a relay that delivered a copy late; `failed` is an error after the request was accepted, where nothing is published because the request may have run; `unpublished` is an answer event a relay refused, timed out or could not be reached for; `lost` is a relay whose subscription closed, or that could not be reached, which the listener tries again after 30 seconds; `deaf` is a relay that did not echo the canary within 15 seconds, which the listener closed (so it counts `lost` too) and tries again after 30 seconds; `canary-refused` is a canary a relay refused with a NIP-01 prefix, such as `rate-limited:`, which is not taken for deafness, so the relay stays subscribed, unchecked until a canary goes through. The gateway's own `[ohttp]` line counts `outbound`, a request shed because the carrier's outbound budget was spent. Relay NOTICE messages are not printed. The listener cannot silence one thing: when a relay sends a message the Nostr library cannot parse, or an event it cannot match (a kind 21470 event with no tags, say), the library prints its own warning to stderr, `[nostr] relay error processing message:` with the error, and with the event itself in the second case. For a frame it cannot parse, the error is V8's, which quotes up to 10 characters of the frame on either side of the fault. A relay that does this is misbehaving; take it out of `SHIELDED_NOSTR_RELAYS`. diff --git a/src/rollup/ohttp-gateway.ts b/src/rollup/ohttp-gateway.ts index ebab284d..3d2586b6 100644 --- a/src/rollup/ohttp-gateway.ts +++ b/src/rollup/ohttp-gateway.ts @@ -9,10 +9,10 @@ const MAX_RESPONSE=4<<20,REMEMBERED=100_000; const STREAMS=['/v1/txs','/v1/batch/events','/v1/indexer/script/subscription','/v1/indexer/subscription']; const copied=(from:Headers,names:string[])=>{const kept=new Headers();for(const name of names){const value=from.get(name);if(value!==null)kept.set(name,value);}return kept;}; -export function createOhttpGateway({key,upstreams,poolOrigin,poolLoopback,gatewayToken,rate=20,maxInFlight=32,timeoutMs=30_000,http=true,fetch:send=fetch,now=()=>performance.now()}:{key:OhttpGatewayKey;upstreams:OhttpUpstream[];poolOrigin:string;poolLoopback:string;gatewayToken:string;rate?:number;maxInFlight?:number;timeoutMs?:number;http?:boolean;fetch?:typeof fetch;now?:()=>number}){ +export function createOhttpGateway({key,upstreams,poolOrigin,poolLoopback,gatewayToken,rate=20,maxInFlight=32,timeoutMs=30_000,http=true,fetch:send=fetch,now=()=>performance.now(),clock=()=>Date.now(),windowMs=90_000}:{key:OhttpGatewayKey;upstreams:OhttpUpstream[];poolOrigin:string;poolLoopback:string;gatewayToken:string;rate?:number;maxInFlight?:number;timeoutMs?:number;http?:boolean;fetch?:typeof fetch;now?:()=>number;clock?:()=>number;windowMs?:number}){ if(![rate,maxInFlight].every(count=>Number.isSafeInteger(count)&&count>=1))throw new Error('The Oblivious HTTP rate and in-flight cap are whole numbers of at least 1.'); const allowed=upstreams.map(({origin,prefix})=>{const url=new URL(origin);if(url.protocol!=='https:'||!prefix.startsWith('/')||!prefix.endsWith('/'))throw new Error(`An Oblivious HTTP upstream is https with a path prefix that starts and ends with /, not ${origin} ${prefix}.`);return {origin:url.origin,prefix};}); - const pool=new URL(poolOrigin).origin,loopback=new URL(poolLoopback).origin,tally=new Map(),seen=new Set(); + const pool=new URL(poolOrigin).origin,loopback=new URL(poolLoopback).origin,tally=new Map(),seen=new Set(),startedAt=clock(); let since=now(),second=0,used=0,inFlight=0; const note=(kind:string)=>{tally.set(kind,(tally.get(kind)??0)+1);if(now()-since<60_000)return;console.log(`[ohttp] ${[...tally].map(([name,count])=>`${name} ${count}`).join(', ')}`);tally.clear();since=now();}; const answer=(status:number,kind:string)=>{note(kind);return new Response(null,{status});}; @@ -24,7 +24,7 @@ export function createOhttpGateway({key,upstreams,poolOrigin,poolLoopback,gatewa const lower=routed.toLowerCase(),toPool=url.origin===pool&&lower.startsWith('/api/rollup/'),listed=toPool?!lower.startsWith('/api/rollup/ohttp'):allowed.some(entry=>entry.origin===url.origin&&routed.startsWith(entry.prefix)); if(!listed||!['GET','POST'].includes(request.method)||/%2f|%5c/i.test(path)||path.includes('//'))return answer(403,'refused'); if(/text\/event-stream/i.test(request.headers.get('accept')??'')||STREAMS.some(stream=>lower.startsWith(stream)))return answer(400,'stream'); - const headers=copied(request.headers,['content-type','accept']),timeout=AbortSignal.timeout(timeoutMs),signal=AbortSignal.any([timeout,gone]); + const headers=copied(request.headers,['content-type','accept','x-digest']),timeout=AbortSignal.timeout(timeoutMs),signal=AbortSignal.any([timeout,gone]); if(toPool)headers.set('x-shielded-gateway',gatewayToken); try{ const upstream=await send((toPool?loopback:url.origin)+path+url.search,{method:request.method,headers,redirect:'error',signal,...(request.method==='POST'?{body:await request.arrayBuffer()}:{})}); @@ -46,9 +46,12 @@ export function createOhttpGateway({key,upstreams,poolOrigin,poolLoopback,gatewa const unseal=async(body:Uint8Array,type:string,gone:AbortSignal,cap:number)=>{ const opened=await key.server.decapsulateRequest(new Request('http://gateway.invalid/',{method:'POST',headers:{'content-type':type},body:body as Uint8Array})).catch(()=>undefined); if(!opened){note('unopened');return;} + // Checked before seen.add, so a stale copy records nothing. The 15 s is the client's rounding: a request sealed just after the start can round to before it. + const at=Date.parse(opened.request.headers.get('x-date')??''); + if(!Number.isFinite(at)||Math.abs(clock()-at)>windowMs||atREMEMBERED)seen.delete(seen.values().next().value!); // ponytail: in memory, so a restart or 100 000 newer requests forget a value; an inner Date check (RFC 9458 section 6.5.1) bounds replays for good + seen.add(enc);if(seen.size>REMEMBERED)seen.delete(seen.values().next().value!); // ponytail: in memory, so a restart or 100 000 newer requests forget a value, which can then run once more inside its x-date window return new Uint8Array(await (await opened.context.encapsulateResponse(await forward(opened.request,gone,cap))).arrayBuffer()); }; // No upload and no late reader here, so the upstream deadline alone bounds the slot. diff --git a/src/shielded/sealed-fetch.ts b/src/shielded/sealed-fetch.ts index cb317337..410cefcd 100644 --- a/src/shielded/sealed-fetch.ts +++ b/src/shielded/sealed-fetch.ts @@ -2,7 +2,7 @@ import {AEAD_AES_128_GCM,CipherSuite,KDF_HKDF_SHA256,KEM_DHKEM_X25519_HKDF_SHA25 import {AbstractSimplePool} from 'nostr-tools/abstract-pool'; import {verifyEvent,type Event} from 'nostr-tools/pure'; import {KeyConfig,OHTTPClient} from 'ohttp-ts'; -import {addressOf,NOSTR_ANSWER_KIND,NOSTR_CHUNK_BYTES,NOSTR_RELAYS,readAnswer,refusedByRelay,requestEvent} from '../../packages/protocol/src/nostr-carrier.ts'; +import {addressOf,NOSTR_ANSWER_KIND,NOSTR_CHUNK_BYTES,NOSTR_RELAYS,readAnswer,refusedByRelay,requestEvent,roundedSeconds} from '../../packages/protocol/src/nostr-carrier.ts'; import type {CarrierPool} from '../rollup/nostr-carrier.ts'; const SUITE=new CipherSuite(KEM_DHKEM_X25519_HKDF_SHA256,KDF_HKDF_SHA256,AEAD_AES_128_GCM); @@ -19,9 +19,9 @@ const tooLarge=()=>new SealedFetchError('too-large','The sealed request is large const outcomeUnknown=()=>new SealedFetchError('outcome-unknown','The request went out and no answer came back, so it may have run.'); // No enableReconnect: it resubscribes at since=lastEmitted+1, and the next attempt covers a dropped relay. Pure's verifyEvent caches the verdict readAnswer reads. -export function createSealedFetch({config,gateway,origin,relays=NOSTR_RELAYS,pick=2,timeoutMs=40_000,attempts=3,pool=new AbstractSimplePool({verifyEvent,maxWaitForConnection:3000}),random=Math.random}:{config:Uint8Array;gateway:string;origin:string;relays?:readonly string[];pick?:number;timeoutMs?:number;attempts?:number;pool?:CarrierPool;random?:()=>number}):((input:string|URL|Request,init?:RequestInit)=>Promise)&{close():void} { +export function createSealedFetch({config,gateway,origin,relays=NOSTR_RELAYS,pick=2,timeoutMs=40_000,getTimeoutMs=9_000,sealedForMs=45_000,attempts=3,pool=new AbstractSimplePool({verifyEvent,maxWaitForConnection:3000}),random=Math.random}:{config:Uint8Array;gateway:string;origin:string;relays?:readonly string[];pick?:number;timeoutMs?:number;getTimeoutMs?:number;sealedForMs?:number;attempts?:number;pool?:CarrierPool;random?:()=>number}):((input:string|URL|Request,init?:RequestInit)=>Promise)&{close():void} { const client=new OHTTPClient(SUITE,KeyConfig.select(SUITE,KeyConfig.parseMultiple(config)),{maxMessageSize:5<<20}),urls=[...new Set(relays)],bench=new Map(); - if(!(urls.length&&pick>=1&&attempts>=1&&timeoutMs>0))throw new RangeError('A sealed fetch needs a relay, a pick and attempts of at least 1, and a positive timeout.'); + if(!(urls.length&&pick>=1&&attempts>=1&&timeoutMs>0&&getTimeoutMs>0&&sealedForMs>0))throw new RangeError('A sealed fetch needs a relay, a pick and attempts of at least 1, and positive timeouts.'); // nostr-tools' publish normalizes each URL synchronously, so a bad one would throw unheard inside a later hedge round and leave the attempt to its deadline. if(!urls.every(relay=>{try{return relay.includes('://')&&/^wss?:$/.test(new URL(relay).protocol);}catch{return false;}}))throw new RangeError('Every relay of a sealed fetch is a ws:// or wss:// URL.'); const shut=()=>new SealedFetchError('unavailable','The sealed fetch is closed.'); @@ -29,8 +29,8 @@ export function createSealedFetch({config,gateway,origin,relays=NOSTR_RELAYS,pic const sealedFetch=async(input:string|URL|Request,init?:RequestInit):Promise=>{ if(closed)throw shut(); const signal=init?.signal??(input instanceof Request?input.signal:undefined),source=new Request(input instanceof Request?input:new URL(input,origin),init),headers=new Headers(); - for(const name of ['content-type','accept']){const value=source.headers.get(name);if(value!==null)headers.set(name,value);} - const {url,method}=source,body=source.body?await source.arrayBuffer():undefined,tried=new Set(),cleanup:(()=>void)[]=[],heard=new Map(); + for(const name of ['content-type','accept','x-digest']){const value=source.headers.get(name);if(value!==null)headers.set(name,value);} + const {url,method}=source,attemptMs=method==='GET'?getTimeoutMs:timeoutMs,body=source.body?await source.arrayBuffer():undefined,tried=new Set(),cleanup:(()=>void)[]=[],heard=new Map(); // Sealing only adds bytes, so a body over one event never fits; past 5 MiB ohttp-ts would throw its own error instead. if((body?.byteLength??0)>NOSTR_CHUNK_BYTES)throw tooLarge(); signal?.throwIfAborted(); @@ -48,14 +48,16 @@ export function createSealedFetch({config,gateway,origin,relays=NOSTR_RELAYS,pic return chosen; }; const seal=async()=>{ + // x-date, not Date (RFC 9458 section 6.5.1): a browser's Request drops Date, a forbidden header name. + headers.set('x-date',new Date(roundedSeconds(Date.now())*1000).toUTCString()); const {init:sealed,context}=await client.encapsulateRequest(new Request(url,{method,headers,body})),bytes=sealed.body as Uint8Array; if(bytes.length>NOSTR_CHUNK_BYTES)throw tooLarge(); return {bytes,context}; }; - let once:ReturnType|undefined; + let once:ReturnType|undefined,sealedAt:number|undefined; const attempt=async():Promise=>{ // Every copy of a POST carries one sealing: the gateway opens an enc at most once and never records a shed one, so the POST runs at most once, while the gateway remembers its enc, whatever the relays claim. - const {bytes,context}=await (method==='GET'?seal():(once??=seal())); + const {bytes,context}=await (method==='GET'?seal():(once??=seal()));sealedAt??=Date.now(); // An answer or an abort can end the fetch while this attempt seals; it must not publish after that. if(done)return {kind:'refused'}; if(closed)throw noAnswer(shut()); @@ -70,8 +72,8 @@ export function createSealedFetch({config,gateway,origin,relays=NOSTR_RELAYS,pic }; return new Promise(settle=>{ let maybe=false,ended=false;const resolve=(ending:Ending)=>{ended=true;settle(ending);}; - live={id:event.id,end:resolve,deadline:Date.now()+timeoutMs}; - void after(timeoutMs,{kind:'timeout'} as const).then(resolve); + live={id:event.id,end:resolve,deadline:Date.now()+attemptMs}; + void after(attemptMs,{kind:'timeout'} as const).then(resolve); const send=()=>{ const to=closed?[]:choose(used); if(!to.length){if(!maybe)resolve({kind:'refused'});return;} @@ -93,6 +95,8 @@ export function createSealedFetch({config,gateway,origin,relays=NOSTR_RELAYS,pic }; try{ for(let n=1;n<=attempts;n++){ + // Every copy of a POST carries its one x-date, so it stops while the gateway's window still holds that date. + if(n>1&&method!=='GET'&&Date.now()-sealedAt!>=sealedForMs)break; const end=await Promise.race([answered,aborted,attempt()]); if(end.kind==='sealed')return end.open(); if(end.kind==='timeout'&&method!=='GET')throw outcomeUnknown(); diff --git a/tests/nostr-carrier.test.ts b/tests/nostr-carrier.test.ts index 6aeee45c..a87f924f 100644 --- a/tests/nostr-carrier.test.ts +++ b/tests/nostr-carrier.test.ts @@ -28,7 +28,7 @@ async function rig(t:TestContext,{relays=[A],reply=()=>Response.json({ok:true}), const carrier=startNostrCarrier({open:opener,secret:key.nostrSecret,relays,pool:bus.pool,...(now?{now}:{}),cacheBytes});t.after(()=>carrier.close());await settle(); const client=new OHTTPClient(SUITE,KeyConfig.select(SUITE,KeyConfig.parseMultiple(key.config)),{maxMessageSize:5<<20}); const ask=async(url:string,{to=[A],listen=to}:{to?:string[];listen?:string[]}={})=>{ - const sealed=await client.encapsulateRequest(new Request(url)),bytes=sealed.init.body as Uint8Array,event=requestEvent(bytes,key.nostrPublic,Date.now()),x=addressOf(bytes),heard=new Map(); + const sealed=await client.encapsulateRequest(new Request(url,{headers:{'x-date':new Date().toUTCString()}})),bytes=sealed.init.body as Uint8Array,event=requestEvent(bytes,key.nostrPublic,Date.now()),x=addressOf(bytes),heard=new Map(); for(const relay of listen){heard.set(relay,[]);bus.pool.subscribeMany([relay],{kinds:[NOSTR_ANSWER_KIND],authors:[key.nostrPublic],'#x':[x]},{onevent:answer=>heard.get(relay)!.push(answer)});} await Promise.all(bus.pool.publish(to,event)); const answer=(relay=listen[0]!)=>readAnswer(heard.get(relay)!,x,key.nostrPublic); diff --git a/tests/ohttp-gateway.test.ts b/tests/ohttp-gateway.test.ts index 93df4bcc..389d3f7f 100644 --- a/tests/ohttp-gateway.test.ts +++ b/tests/ohttp-gateway.test.ts @@ -26,7 +26,7 @@ async function gateway(t:TestContext,{reply=()=>Response.json({ok:true}),before, const server=app.listen(0,'127.0.0.1');await once(server,'listening');t.after(()=>new Promise(resolve=>server.close(resolve))); const base=`http://127.0.0.1:${(server.address() as AddressInfo).port}/api/rollup`,keys=await fetch(base+'/ohttp-keys'),config=new Uint8Array(await keys.arrayBuffer()),client=clientOf(config); const post=(body:BodyInit,type='message/ohttp-req')=>fetch(base+'/ohttp',{method:'POST',headers:{'content-type':type},body}); - const seal=(url:string,init?:RequestInit)=>client.encapsulateRequest(new Request(url,init)); + const seal=(url:string,init?:RequestInit)=>{const headers=new Headers(init?.headers);if(!headers.has('x-date'))headers.set('x-date',new Date().toUTCString());return client.encapsulateRequest(new Request(url,{...init,headers}));}; const exchange=async({init,context}:Awaited>)=>{ const outer=await fetch(base+'/ohttp',init); assert.equal(outer.status,200);assert.equal(outer.headers.get('content-type'),'message/ohttp-res');assert.equal(outer.headers.get('cache-control'),'no-store'); @@ -74,7 +74,7 @@ test('a pool and an upstream on one origin are told apart by path',async t=>{ assert.deepEqual(g.calls.map(({url,init})=>[url,new Headers(init.headers).get('x-shielded-gateway')]),[['https://shared.example/v1/info',null],[LOOPBACK+'/api/rollup/status',TOKEN],[LOOPBACK+'/api/ROLLUP/status',TOKEN],['https://shared.example/api/blocks/tip/height',null]],'the token goes to the pool alone, and no spelling of the gateway path leaves'); }); -test('only content-type and accept reach the upstream',async t=>{ +test('hostile headers never reach the upstream',async t=>{ const g=await gateway(t),hostile={'x-forwarded-for':'203.0.113.7',host:'evil.example',cookie:'session=1',authorization:'Bearer stolen','x-shielded-gateway':'guessed'}; const headers={...hostile,'content-type':'application/json',accept:'application/json'},sealed=await g.seal('https://pool.example/api/rollup/spends',{method:'POST',headers,body:'{"id":"ab"}'}); const carried=(await g.key.server.decapsulateRequest(new Request('http://gateway.test/',sealed.init))).request.headers; @@ -306,3 +306,32 @@ test('http:false omits POST /ohttp and keeps the key config',async t=>{ assert.equal((await g.post((await g.seal(ARK)).init.body!)).status,404); assert.equal((await through(g.router,await g.seal(ARK))).status,200,'open() still answers'); }); + +test('a sealed request with no x-date is refused silently, leaving no replay entry',async t=>{ + const lines:string[]=[];t.mock.method(console,'log',(line:string)=>{lines.push(line);}); + let clock=0;const g=await gateway(t,{now:()=>clock}),raw=clientOf(g.config),bare=await raw.encapsulateRequest(new Request(ARK)); + for(const sealed of [bare,bare,await g.seal(ARK,{headers:{'x-date':'not a date'}}),await raw.encapsulateRequest(new Request(ARK,{headers:{date:new Date().toUTCString()}}))])assert.deepEqual(await g.router.open(bytesOf(sealed)),{kind:'refused'}); + assert.equal((await g.post(bare.init.body!)).status,400,'the HTTP route shares the check');assert.equal(g.calls.length,0); + clock=60_000;assert.equal((await g.open(ARK)).status,200); + assert.deepEqual(lines,['[ohttp] stale 5, forwarded 1'],'the second copy is stale again, not a replay: a refusal records nothing'); +}); + +test('an x-date more than 90 s from the gateway clock is refused',async t=>{ + let clock=Date.UTC(2026,10,11,12);const g=await gateway(t,{clock:()=>clock});clock+=300_000; // so the start check is out of reach + const at=(offset:number)=>g.seal(ARK,{headers:{'x-date':new Date(clock+offset).toISOString()}}); + for(const offset of [-91_000,91_000])assert.deepEqual(await g.router.open(bytesOf(await at(offset))),{kind:'refused'},String(offset)); + assert.equal(g.calls.length,0); + for(const offset of [-89_000,89_000])assert.equal((await through(g.router,await at(offset))).status,200,String(offset)); +}); + +test('an x-date before the gateway started is refused, with 15 s of rounding slack',async t=>{ + const T=Date.UTC(2026,10,11,12),g=await gateway(t,{clock:()=>T}),at=(ms:number)=>g.seal(ARK,{headers:{'x-date':new Date(ms).toISOString()}}); + assert.equal((await through(g.router,await at(T-15_000))).status,200); + assert.deepEqual(await g.router.open(bytesOf(await at(T-15_001))),{kind:'refused'});assert.equal(g.calls.length,1); +}); + +test('x-digest crosses to the upstream and x-date does not',async t=>{ + const g=await gateway(t); + assert.equal((await g.open(ARK,{headers:{'x-digest':'abc','x-date':new Date().toUTCString(),'x-sdk-version':'ts-sdk/0.5.3'}})).status,200); + assert.deepEqual(Object.fromEntries(new Headers(g.calls[0]!.init.headers)),{'x-digest':'abc'}); +}); diff --git a/tests/rollup-http.test.ts b/tests/rollup-http.test.ts index 3cfa4a82..163479f0 100644 --- a/tests/rollup-http.test.ts +++ b/tests/rollup-http.test.ts @@ -174,7 +174,7 @@ test('what the gateway forwards to the pool is admitted past the per-address lim app.use('/api/rollup',createRollupRouter(()=>({ready:()=>true,submit:async()=>{}}) as unknown as RollupService,{gatewayToken:TOKEN})); const client=new OHTTPClient(suite,KeyConfig.select(suite,KeyConfig.parseMultiple(key.config)),{maxMessageSize:5<<20}); for(let i=0;i<61;i++){ - const {init,context}=await client.encapsulateRequest(new Request('https://pool.example/api/rollup/spends',{method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify(spend)})); + const {init,context}=await client.encapsulateRequest(new Request('https://pool.example/api/rollup/spends',{method:'POST',headers:{'content-type':'application/json','x-date':new Date().toUTCString()},body:JSON.stringify(spend)})); assert.equal((await context.decapsulateResponse(await fetch(origin+'/api/rollup/ohttp',init))).status,202,`sealed POST ${i+1}`); } }); diff --git a/tests/sealed-fetch.test.ts b/tests/sealed-fetch.test.ts index f13be44e..36c001cf 100644 --- a/tests/sealed-fetch.test.ts +++ b/tests/sealed-fetch.test.ts @@ -4,7 +4,7 @@ import {setTimeout as sleep} from 'node:timers/promises'; import {base64} from '@scure/base'; import {finalizeEvent,generateSecretKey,type Event} from 'nostr-tools/pure'; import {addressOf,NOSTR_ANSWER_KIND,NOSTR_REQUEST_KIND,readRequest,statusEvent} from '../packages/protocol/src/nostr-carrier.ts'; -import {createOhttpGateway} from '../src/rollup/ohttp-gateway.ts'; +import {createOhttpGateway,type OhttpOpen} from '../src/rollup/ohttp-gateway.ts'; import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; import {createSealedFetch,SealedFetchError} from '../src/shielded/sealed-fetch.ts'; @@ -17,16 +17,19 @@ const settle=()=>new Promise(resolve=>setImmediate(resolve)); const holding=(reply=()=>Response.json({ok:true}))=>{let release=()=>{};const held=new Promise(resolve=>{release=()=>resolve(reply());});return {held,release};}; const failsWith=(code:SealedFetchError['code'])=>(error:unknown)=>error instanceof SealedFetchError&&error.code===code; const timers=()=>process.getActiveResourcesInfo().filter(type=>type==='Timeout').length; +const advance=async(t:TestContext,ms:number)=>{for(let at=0;atResponse|Promise;gateway?:Partial[0]>}&Pick[0],'pick'|'timeoutMs'|'attempts'|'random'>; -async function rig(t:TestContext,{relays=[A,B],reply=()=>Response.json({ok:true}),gateway={},...options}:Rig={}){ +type Rig={relays?:string[];reply?:()=>Response|Promise;gateway?:Partial[0]>;open?:OhttpOpen}&Pick[0],'pick'|'timeoutMs'|'getTimeoutMs'|'attempts'|'random'>; +async function rig(t:TestContext,{relays=[A,B],reply=()=>Response.json({ok:true}),gateway={},open,...options}:Rig={}){ const key=(await loadOhttpKey((await generateOhttpKey()).env))!,calls:{url:string;init:RequestInit}[]=[],bus=createNostrBus(); const upstream=(async(url:string,init:RequestInit)=>{calls.push({url,init});return reply();}) as unknown as typeof fetch; const router=createOhttpGateway({key,upstreams:[{origin:'https://ark.example',prefix:'/v1/'}],poolOrigin:POOL,poolLoopback:LOOPBACK,gatewayToken:'token',fetch:upstream,...gateway}); - const carrier=startNostrCarrier({open:router.open,secret:key.nostrSecret,relays,pool:bus.pool});t.after(()=>carrier.close());await settle(); + const carrier=startNostrCarrier({open:open??router.open,secret:key.nostrSecret,relays,pool:bus.pool});t.after(()=>carrier.close());await settle(); const sends=t.mock.method(bus.pool,'publish'),sealedFetch=createSealedFetch({config:key.config,gateway:key.nostrPublic,origin:POOL,relays,pool:bus.pool,random:()=>0,...options});t.after(()=>sealedFetch.close()); const answers=()=>bus.published.filter(event=>event.kind===NOSTR_ANSWER_KIND&&event.pubkey===key.nostrPublic),requests=()=>bus.published.filter(event=>event.kind===NOSTR_REQUEST_KIND); - return {key,calls,bus,fetch:sealedFetch,answers,requests,address:()=>addressOf(readRequest(requests()[0]!,key.nostrPublic)!),statuses:()=>answers().filter(event=>event.tags.some(tag=>tag[0]==='status')), + const inner=async(i=0)=>(await key.server.decapsulateRequest(new Request('http://gateway.test/',{method:'POST',headers:{'content-type':'message/ohttp-req'},body:Buffer.from(readRequest(requests()[i]!,key.nostrPublic)!)}))).request; + return {key,calls,bus,fetch:sealedFetch,answers,requests,inner,address:()=>addressOf(readRequest(requests()[0]!,key.nostrPublic)!),statuses:()=>answers().filter(event=>event.tags.some(tag=>tag[0]==='status')), sentTo:()=>sends.mock.calls.map(call=>call.arguments[0]),sent:()=>sends.mock.calls.map(call=>call.arguments[1])}; } @@ -38,12 +41,12 @@ test('a GET goes out sealed and comes back as a Response',async t=>{ assert.deepEqual([g.bus.openSubscriptions(),timers()],before,'the answer closed every subscription and deadline the fetch opened'); }); -test('relative URLs resolve against the origin and only content-type and accept leave the client',async t=>{ +test('relative URLs resolve against the origin and only content-type, accept and x-digest leave the client, with its own x-date',async t=>{ const g=await rig(t); assert.equal((await g.fetch('/api/rollup/status',{headers:{'X-SDK-VERSION':'ts-sdk/0.5.3',Authorization:'x',Accept:'application/json'}})).status,200); assert.deepEqual(g.calls.map(call=>call.url),[LOOPBACK+'/api/rollup/status']); - const sealed=Buffer.from(readRequest(g.requests()[0]!,g.key.nostrPublic)!),{request}=await g.key.server.decapsulateRequest(new Request('http://gateway.test/',{method:'POST',headers:{'content-type':'message/ohttp-req'},body:sealed})); - assert.equal(request.url,POOL+'/api/rollup/status');assert.deepEqual([...request.headers],[['accept','application/json']]); + const request=await g.inner(); + assert.equal(request.url,POOL+'/api/rollup/status');assert.deepEqual([...request.headers.keys()],['accept','x-date']); }); test('a large answer is reassembled, from chunks in any order, past a chunk another key signed',async t=>{ @@ -77,7 +80,7 @@ test('a POST shed on two attempts within 30 s hears both statuses and runs on it }); test('a GET whose relay never delivers is retried on another relay',async t=>{ - const g=await rig(t,{pick:1,timeoutMs:100});g.bus.relay(A).deliver=false; + const g=await rig(t,{pick:1,getTimeoutMs:100});g.bus.relay(A).deliver=false; assert.equal((await g.fetch(ARK)).status,200); assert.deepEqual(g.sentTo(),[[A],[B]]);assert.equal(g.requests().length,2); assert.notEqual(g.sent()[0]!.content,g.sent()[1]!.content,'each GET attempt is sealed afresh'); @@ -85,7 +88,7 @@ test('a GET whose relay never delivers is retried on another relay',async t=>{ test('a late answer to an earlier attempt counts',async t=>{ // Through A the request and its answer each wait 150 ms, so the answer lands near 300 ms; five attempts keep the fetch open to 500 ms. - let replies=0;const g=await rig(t,{pick:1,timeoutMs:100,attempts:5,reply:()=>Response.json({reply:++replies})}); + let replies=0;const g=await rig(t,{pick:1,getTimeoutMs:100,attempts:5,reply:()=>Response.json({reply:++replies})}); g.bus.relay(A).delayMs=150;g.bus.relay(B).deliver=false; assert.deepEqual(await (await g.fetch(ARK)).json(),{reply:1},'the first attempt\'s answer, the first the gateway gave'); assert.deepEqual(g.sentTo().slice(0,2),[[A],[B]]); @@ -117,7 +120,7 @@ test('a POST through a relay that delivers what it says it never took runs once, }); test('a pick that times out without delivering is hedged at once, and the answer comes from the next relay',{timeout:10_000},async t=>{ - const g=await rig(t,{pick:1,timeoutMs:60_000});Object.assign(g.bus.relay(A),{timesOut:true,deliver:false}); + const g=await rig(t,{pick:1,getTimeoutMs:60_000});Object.assign(g.bus.relay(A),{timesOut:true,deliver:false}); assert.equal((await g.fetch(ARK)).status,200); assert.deepEqual(g.sentTo(),[[A],[B]]);assert.equal(g.sent()[0]!.id,g.sent()[1]!.id,'the same event went to B');assert.equal(g.requests().length,1); }); @@ -162,13 +165,13 @@ test('a round whose OK is still out when a status ends its attempt hedges nothin }); test('attempts run out as unavailable',async t=>{ - const g=await rig(t,{timeoutMs:50,attempts:3}),before=g.bus.openSubscriptions();for(const url of [A,B])g.bus.relay(url).deliver=false; + const g=await rig(t,{getTimeoutMs:50,attempts:3}),before=g.bus.openSubscriptions();for(const url of [A,B])g.bus.relay(url).deliver=false; await assert.rejects(g.fetch(ARK),failsWith('unavailable')); assert.equal(g.requests().length,3);assert.equal(g.bus.openSubscriptions(),before,'every attempt\'s subscription closed'); }); test('a status event cannot hold the client past its attempt deadline',{timeout:10_000},async t=>{ - let replies=0;const g=await rig(t,{relays:[A],timeoutMs:200,reply:()=>++replies===1?new Promise(()=>{}):Response.json({ok:true})}); + let replies=0;const g=await rig(t,{relays:[A],getTimeoutMs:200,reply:()=>++replies===1?new Promise(()=>{}):Response.json({ok:true})}); const started=performance.now(),pending=g.fetch(ARK);await until(()=>g.calls.length===1); await Promise.all(g.bus.pool.publish([A],statusEvent(503,3600,g.address(),g.sent()[0]!.id,g.key.nostrSecret,Date.now()))); assert.equal((await pending).status,200);assert.ok(performance.now()-started<2000,'the hour the status asked for was cut to the deadline'); @@ -184,7 +187,7 @@ test('a stored status from an earlier attempt does not end a later one',async t= }); test('a status for an earlier GET attempt does not end the current attempt',async t=>{ - const {held,release}=holding(),g=await rig(t,{pick:1,timeoutMs:500,attempts:2,reply:()=>held});g.bus.relay(A).deliver=false; + const {held,release}=holding(),g=await rig(t,{pick:1,getTimeoutMs:500,attempts:2,reply:()=>held});g.bus.relay(A).deliver=false; const pending=g.fetch(ARK);await until(()=>g.calls.length===1); g.bus.relay(A).deliver=true;await Promise.all(g.bus.pool.publish([A],statusEvent(503,1,g.address(),g.sent()[0]!.id,g.key.nostrSecret,Date.now())));await sleep(20);release(); assert.equal((await pending).status,200,'the second, last attempt kept waiting for its own answer');assert.deepEqual(g.sentTo().slice(0,2),[[A],[B]]); @@ -221,7 +224,7 @@ test('a request too large for one event is refused before publishing',async t=>{ }); test('close() refuses new fetches and stops one in flight before its next attempt goes out',async t=>{ - const g=await rig(t,{pick:1,timeoutMs:100});g.bus.relay(A).deliver=false; + const g=await rig(t,{pick:1,getTimeoutMs:100});g.bus.relay(A).deliver=false; const pending=g.fetch(ARK);await until(()=>g.sentTo().length===1); g.fetch.close();assert.equal(g.bus.openSubscriptions(),0,'the pool is destroyed'); let refused=false;const late=g.fetch(ARK).catch(error=>{refused=true;throw error;});await null; @@ -247,7 +250,7 @@ test('close() ends a POST that went out as outcome-unknown, and one that never d test('a sealed fetch with no relay, a relay that is not a ws:// or wss:// URL, or a pick, attempt count or timeout under one, is refused when it is made',async()=>{ const key=(await loadOhttpKey((await generateOhttpKey()).env))!; - for(const options of [{relays:[]},{relays:[A,'not a url']},{relays:['https://relay.example']},{relays:['wss:a.example']},{relays:['wss:/a.example']},{pick:0},{attempts:0},{timeoutMs:0},{pick:Number.NaN}])assert.throws(()=>createSealedFetch({config:key.config,gateway:key.nostrPublic,origin:POOL,...options}),RangeError,JSON.stringify(options)); + for(const options of [{relays:[]},{relays:[A,'not a url']},{relays:['https://relay.example']},{relays:['wss:a.example']},{relays:['wss:/a.example']},{pick:0},{attempts:0},{timeoutMs:0},{getTimeoutMs:0},{sealedForMs:0},{pick:Number.NaN}])assert.throws(()=>createSealedFetch({config:key.config,gateway:key.nostrPublic,origin:POOL,...options}),RangeError,JSON.stringify(options)); createSealedFetch({config:key.config,gateway:key.nostrPublic,origin:POOL,relays:['ws://127.0.0.1:7777',A]}).close(); }); @@ -261,3 +264,48 @@ test('abort closes the attempt',async t=>{ await assert.rejects(early,error=>error===reason);await assert.rejects(g.fetch(ARK,{signal:AbortSignal.abort(reason)}),error=>error===reason);await sleep(50); assert.deepEqual([g.requests().length,g.bus.openSubscriptions()],[1,before],'a fetch aborted before it published never publishes');release(); }); + +test('every sealed request carries an x-date rounded to the nearest 30 s, and x-digest crosses',async t=>{ + const g=await rig(t); + assert.equal((await g.fetch(ARK,{headers:{'x-digest':'abc',authorization:'x','x-sdk-version':'ts-sdk/0.5.3'}})).status,200); + const {headers}=await g.inner(),at=Date.parse(headers.get('x-date')!); + assert.ok(at%30_000===0&&Math.abs(Date.now()-at)<=30_000,String(headers.get('x-date'))); + assert.deepEqual([headers.get('x-digest'),headers.has('authorization'),headers.has('x-sdk-version')],['abc',false,false]); +}); + +test('the sealed request uses no header a browser would strip',async t=>{ + const g=await rig(t),stripped=Object.fromEntries([...FORBIDDEN,'proxy-authorization','sec-fetch-mode'].map(name=>[name,'x'])); + assert.equal((await g.fetch('/api/rollup/spends',{method:'POST',headers:{...stripped,'content-type':'application/json',accept:'application/json','x-digest':'abc'},body:'{"id":"ab"}'})).status,200); + const names=[...(await g.inner()).headers.keys()]; + assert.deepEqual(names.filter(name=>FORBIDDEN.includes(name)||/^(proxy|sec)-/.test(name)),[],names.join()); +}); + +test("a caller's own x-date is overwritten, not forwarded",async t=>{ + const g=await rig(t); + assert.equal((await g.fetch(ARK,{headers:{'x-date':'Thu, 01 Jan 1970 00:00:00 GMT'}})).status,200); + assert.ok(Math.abs(Date.now()-Date.parse((await g.inner()).headers.get('x-date')!))<=30_000); +}); + +test('a GET attempt ends after 9 s and a POST attempt after 40 s',async t=>{ + const {held}=holding(),g=await rig(t,{reply:()=>held});t.mock.timers.enable({apis:['setTimeout']}); + const stop=new AbortController(),get=g.fetch(ARK,{signal:stop.signal});await until(()=>g.sentTo().length===1); + t.mock.timers.tick(8_999);await sleep(50);assert.equal(g.sentTo().length,1); + t.mock.timers.tick(1);await until(()=>g.sentTo().length===2);stop.abort();await assert.rejects(get); + let ended=false;const post=assert.rejects(g.fetch('/api/rollup/spends',SPEND),failsWith('outcome-unknown')).finally(()=>{ended=true;});await until(()=>g.sentTo().length===3); + t.mock.timers.tick(39_999);await sleep(50);assert.equal(ended,false); + t.mock.timers.tick(1);await post; +}); + +test('a GET answered on its first attempt resolves well inside the SDK getInfo abort',async t=>{ + const g=await rig(t),before=timers(); + assert.equal((await g.fetch(ARK)).status,200);await sleep(20); + assert.deepEqual([g.requests().length,timers()],[1,before]); +}); + +test('a POST stops attempting 45 s after its sealing and ends outcome-unknown',async t=>{ + const g=await rig(t,{attempts:3,open:async()=>({kind:'busy',status:503,retryAfter:30})});t.mock.timers.enable({apis:['setTimeout','Date'],now:Date.now()}); + const spend=assert.rejects(g.fetch('/api/rollup/spends',SPEND),failsWith('outcome-unknown'));await until(()=>g.sentTo().length===1); + await advance(t,33_000);assert.equal(g.requests().length,2,'the second attempt went after the first 30 s back-off'); + await advance(t,40_000);await spend; + assert.equal(g.requests().length,2,'the second back-off ended past 45 s, so no third attempt went out');assert.equal(new Set(g.requests().map(event=>event.content)).size,1); +}); From 5323a320eacb4d62fdd758abf148d48ef4f0a31d Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 04:50:18 +0100 Subject: [PATCH 02/40] feat(shielded): add the browser's policy-gated sealed transport installPrivateTransport puts the Nostr sealed fetch where the native host puts Tor, reusing the native route policy, isolated providers and payment journal. It never imports src/native/index.ts, which pulls in Tauri. Routing: on the page origin, assets and /api/rollup/proving/* go direct and a pool API path is sealed at the pinned pool origin, so a page served from 127.0.0.1 never names itself; arkd, the indexer and Esplora are classified by NativePolicy and sealed as given; anything else throws. There is no direct fallback. POST /spends is journalled before it is sealed, and the pool's answers to it and to GET /spends/ are observed. The sealed fetch's relay pool gets the WebSocket captured before the override, while the global one admits only the pinned relays, compared through normalizeURL. fetch, EventSource, WebSocket, XMLHttpRequest and RTCPeerConnection (and webkitRTCPeerConnection where present) are defined non-writable and non-configurable. A second install returns the first transport; close() releases that slot. IsolatedIndexerProvider gains a streams option (default 32, the native host's), and the browser passes 0, so every watched script is polled and no subscription stream is opened. The ark provider gets no eventSource, so its batch stream reaches the refused global EventSource and fails closed. journalledRoute names the two journalled pool routes for both the native observer and the transport. --- src/native/mutations.ts | 4 +- src/native/providers.ts | 4 +- src/shielded/private-transport.ts | 56 ++++++++++++++ tests/native-providers.test.ts | 13 ++++ tests/private-transport.test.ts | 122 ++++++++++++++++++++++++++++++ 5 files changed, 195 insertions(+), 4 deletions(-) create mode 100644 src/shielded/private-transport.ts create mode 100644 tests/private-transport.test.ts diff --git a/src/native/mutations.ts b/src/native/mutations.ts index d0dc11b6..7cb30c41 100644 --- a/src/native/mutations.ts +++ b/src/native/mutations.ts @@ -11,6 +11,7 @@ const REFUSED_BODY=new Set(['The request body is not valid JSON.','A spend id is const preAdmissionRefusal=(status:number,result:any)=>typeof result?.error==='string'&&(status===429?result.code==='admission-rate-limit'&&result.error==='Too many requests from this address; try again in a minute.':status===400&&(result.code===undefined?REFUSED_BODY.has(result.error):REFUSED_CODES.has(result.code))); const invalid=()=>new Error('The native payment journal is invalid; funds remain reserved.'); const route=(raw:string)=>{const url=new URL(raw);return {origin:url.origin,path:decodeURIComponent(url.pathname).toLowerCase()};}; +export const journalledRoute=(url:string,method:string):boolean=>{const m=method.toUpperCase(),{path}=route(url);return m==='GET'&&/^\/api\/rollup\/spends\/[0-9a-f]{32}$/.test(path)||m==='POST'&&path==='/api/rollup/spends';}; const parseBody=(body:string)=>{ if(typeof body!=='string'||body.length>MAX_BYTES)throw invalid(); let parsed:any;try{parsed=JSON.parse(body);}catch{throw invalid();} @@ -109,7 +110,6 @@ export const beforeNativeMutation=(request:NativeMutation)=>{ export const pendingPoolNullifiers=(poolOrigin:string)=>current().pendingPoolNullifiers(poolOrigin); export const pendingPoolOutpoints=(poolOrigin:string)=>current().pendingPoolOutpoints(poolOrigin); export const observeNativeResponse=(response:NativeMutationResponse)=>{ - const method=response.method.toUpperCase(),path=route(response.url).path; - if(method==='GET'&&/^\/api\/rollup\/spends\/[0-9a-f]{32}$/.test(path)||method==='POST'&&path==='/api/rollup/spends')current().observeNativeResponse(response); + if(journalledRoute(response.url,response.method))current().observeNativeResponse(response); }; export const reconcileNativeMutations=(fetcher:typeof fetch,poolOrigin:string)=>current().reconcileNativeMutations(fetcher,poolOrigin); diff --git a/src/native/providers.ts b/src/native/providers.ts index a3cbe5ec..ef4b0d9d 100644 --- a/src/native/providers.ts +++ b/src/native/providers.ts @@ -24,7 +24,7 @@ interface LogicalStream {queue:AsyncQueue;attach:(script:s interface LogicalSubscription {entries:Map;streams:Set;tail:Promise;closed:boolean} export class IsolatedIndexerProvider extends RestIndexerProvider { private base:IndexerProvider;private subscriptions=new Map(); - constructor(url:string,eventSource?:EventSourceFactory,base?:IndexerProvider,private pollMs=20_000){super(url,{eventSource});this.base=base??new RestIndexerProvider(url,{eventSource});} + constructor(url:string,eventSource?:EventSourceFactory,base?:IndexerProvider,private pollMs=20_000,private streams=32){super(url,{eventSource});this.base=base??new RestIndexerProvider(url,{eventSource});} override async getVtxos(opts?:GetVtxosOptions):ReturnType{ if(opts?.scripts&&opts?.outpoints)throw new Error('scripts and outpoints are mutually exclusive options'); const items=opts?.scripts??opts?.outpoints; @@ -61,7 +61,7 @@ export class IsolatedIndexerProvider extends RestIndexerProvider { if(record.closed||signal?.aborted)throw signal?.reason??new DOMException('Subscription cancelled','AbortError'); if(record.entries.has(script))continue; let remote:string|null=null; - if(streamReservations<32){streamReservations++;try{remote=await this.base.subscribeForScripts([script],undefined,signal);}catch(error){streamReservations--;throw error;}} + if(streamReservations{});}throw signal?.reason??new DOMException('Subscription cancelled','AbortError');} record.entries.set(script,remote);added.push(script);for(const stream of record.streams)stream.attach(script,remote); } diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts new file mode 100644 index 00000000..3b726957 --- /dev/null +++ b/src/shielded/private-transport.ts @@ -0,0 +1,56 @@ +import {ESPLORA_URL,RestArkProvider,type NetworkName,type OnchainProvider} from '@arkade-os/sdk'; +import {AbstractSimplePool} from 'nostr-tools/abstract-pool'; +import {verifyEvent} from 'nostr-tools/pure'; +import {normalizeURL} from 'nostr-tools/utils'; +import {NOSTR_RELAYS} from '../../packages/protocol/src/nostr-carrier.ts'; +import {journalledRoute,makeNativeMutationJournal} from '../native/mutations.ts'; +import {NativePolicy} from '../native/policy.ts'; +import {IsolatedIndexerProvider,IsolatedOnchainProvider} from '../native/providers.ts'; +import {markPrivateFetch} from './network-privacy.ts'; +import {createSealedFetch} from './sealed-fetch.ts'; + +export const PRIVATE_PAGE=12; +export const PRIVATE_GRACE_MS=180_000; +export interface PrivateTransportConfig {config:Uint8Array;gateway:string;poolOrigin:string;pageOrigin:string;relays?:readonly string[];storage?:Pick;socket?:typeof WebSocket;globals?:Record} +export interface PrivateTransport {fetch:typeof fetch;poolOrigin:string;pageSize:number;journal:ReturnType;providers(arkUrl:string,indexerUrl:string):{arkProvider:RestArkProvider;indexerProvider:IsolatedIndexerProvider;onchainProvider:(network:NetworkName)=>OnchainProvider};close():void} + +let installed:PrivateTransport|undefined; +const refused=(what:string)=>class {constructor(){throw new Error('Private networking does not permit '+what);}}; +export function privateTransport():PrivateTransport|undefined {return installed;} +export function installPrivateTransport(config:PrivateTransportConfig):PrivateTransport { + if(installed)return installed; + const target=(config.globals??globalThis) as Record,direct=target.fetch as typeof fetch,Original=config.socket??(target.WebSocket as typeof WebSocket); + const relays=config.relays??NOSTR_RELAYS,pinned=new Set(relays.map(normalizeURL)),policy=new NativePolicy(config.poolOrigin),{poolOrigin}=policy,pageOrigin=new URL(config.pageOrigin).origin; + const journal=makeNativeMutationJournal(config.storage??globalThis.localStorage); + const sealedFetch=createSealedFetch({config:config.config,gateway:config.gateway,origin:poolOrigin,relays,pool:new AbstractSimplePool({verifyEvent,maxWaitForConnection:3000,websocketImplementation:Original})}); + const Guarded=class extends Original {constructor(url:string|URL,protocols?:string|string[]){if(!pinned.has(normalizeURL(String(url))))throw new Error('Private networking does not permit a WebSocket to '+url);super(url,protocols);}}; + // Order matters: a pool API path is sealed at the pinned origin even when the page is served from it, and a proving path on the page origin goes direct even when it is not. + const route=(url:URL)=>{ + const path=url.pathname,proving=/^\/api\/rollup\/proving\//.test(path),poolApi=/^\/api\/rollup\//.test(path)&&!proving; + if(url.origin===pageOrigin){if(proving||!path.startsWith('/api/'))return undefined;if(poolApi)return poolOrigin+path+url.search;} + else if(url.origin!==poolOrigin||poolApi)return url.href; + throw new Error('Private networking does not permit '+url.origin+path); + }; + const privateFetch:typeof fetch=markPrivateFetch(async(input,init)=>{ + const url=new URL(input instanceof Request?input.url:String(input),pageOrigin),href=route(url); + if(href===undefined)return direct(input instanceof Request?input:url.href,init); + const source=new Request(input instanceof Request?input:url,init),method=source.method,body=method==='GET'?undefined:await source.text(); + policy.classify(href,method,body); + const journalled=journalledRoute(href,method); + if(journalled&&method!=='GET')journal.beforeNativeMutation({url:href,method,body}); + const response=await sealedFetch(href,{method,headers:source.headers,body,signal:source.signal}); + if(journalled)journal.observeNativeResponse({url:href,method,status:response.status,body:await response.clone().text(),requestBody:body}); + return response; + }); + const transport:PrivateTransport={fetch:privateFetch,poolOrigin,pageSize:PRIVATE_PAGE,journal, + providers(arkUrl,indexerUrl){ + policy.registerArk(arkUrl,indexerUrl); + return {arkProvider:new RestArkProvider(arkUrl),indexerProvider:new IsolatedIndexerProvider(indexerUrl,undefined,undefined,undefined,0),onchainProvider:network=>{const url=ESPLORA_URL[network];policy.registerEsplora(url);return new IsolatedOnchainProvider(url);}}; + }, + close(){sealedFetch.close();if(installed===transport)installed=undefined;}, + }; + const define=(name:string,value:unknown)=>Object.defineProperty(target,name,{value,writable:false,configurable:false}); + define('fetch',privateFetch);define('EventSource',refused('EventSource'));define('WebSocket',Guarded);define('XMLHttpRequest',refused('XMLHttpRequest')); + define('RTCPeerConnection',refused('WebRTC'));if('webkitRTCPeerConnection' in target)define('webkitRTCPeerConnection',refused('WebRTC')); + installed=transport;return transport; +} diff --git a/tests/native-providers.test.ts b/tests/native-providers.test.ts index 5d3afba0..1ce03911 100644 --- a/tests/native-providers.test.ts +++ b/tests/native-providers.test.ts @@ -133,6 +133,19 @@ test('ninety watched scripts preserve per-item receive and spend events beyond t spent=true;const changed=await stream.next();assert.equal(changed.value?.spentVtxos[0]?.script,'s69');swept=true;const sweep=await stream.next();assert.equal(sweep.value?.sweptVtxos[0]?.script,'s70');assert.equal(sweep.value?.spentVtxos[0]?.isSwept,true); assert.ok(reads.every(items=>items.length===1));abort.abort();p.retire(); }); +test('an indexer provider built with no stream reservations polls every script instead of subscribing',async()=>{ + const base=new RestIndexerProvider('https://ark.example'),polled=new Map(); + base.subscribeForScripts=async()=>{throw new Error('a stream was reserved');}; + base.getVtxos=async opts=>{const script=opts!.scripts![0]!;return {vtxos:[{...coin('t'+script),script,isSpent:false,isSwept:false,isUnrolled:false}]};}; + const p=new IsolatedIndexerProvider('https://ark.example',undefined,base,20_000,0),abort=new AbortController(),stream=p.getSubscription(await p.subscribeForScripts(['a','b']),abort.signal); + while(polled.size<2){const event=(await stream.next()).value!;polled.set(event.scripts[0]!,event);} + for(const script of ['a','b'])assert.deepEqual(polled.get(script)!.newVtxos.map(c=>c.script),[script]); + abort.abort();p.retire(); + const subscribed:string[][]=[],streaming=new RestIndexerProvider('https://ark.example'); + streaming.subscribeForScripts=async scripts=>{subscribed.push(scripts);return 'remote-1';};streaming.unsubscribeForScripts=async()=>{}; + const q=new IsolatedIndexerProvider('https://ark.example',undefined,streaming);await q.unsubscribeForScripts(await q.subscribeForScripts(['a'])); + assert.deepEqual(subscribed,[['a']],'the three-argument default still reserves a stream'); +}); test('native onchain watches split addresses and release every watch',async()=>{ const original=EsploraProvider.prototype.watchAddresses,calls:string[][]=[],closed:string[]=[]; EsploraProvider.prototype.watchAddresses=async addresses=>{calls.push(addresses);return ()=>{closed.push(...addresses);};}; diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts new file mode 100644 index 00000000..ead2af64 --- /dev/null +++ b/tests/private-transport.test.ts @@ -0,0 +1,122 @@ +import assert from 'node:assert/strict'; +import {readFileSync} from 'node:fs'; +import {test,type TestContext} from 'node:test'; +import {ESPLORA_URL} from '@arkade-os/sdk'; +import {normalizeURL} from 'nostr-tools/utils'; +import {NOSTR_RELAYS,NOSTR_REQUEST_KIND} from '../packages/protocol/src/nostr-carrier.ts'; +import {IsolatedOnchainProvider} from '../src/native/providers.ts'; +import {createOhttpGateway} from '../src/rollup/ohttp-gateway.ts'; +import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; +import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; +import {isPrivateFetch} from '../src/shielded/network-privacy.ts'; +import {installPrivateTransport} from '../src/shielded/private-transport.ts'; +import {createNostrBus} from './fixtures/nostr-bus.ts'; + +const POOL='https://pool.example',PAGE='http://127.0.0.1:5173',LOOPBACK='http://127.0.0.1:9',ARK='https://ark.example',ESPLORA=ESPLORA_URL.mutinynet,ID='ab'.repeat(16); +const settle=()=>new Promise(resolve=>setImmediate(resolve)); +const memory=()=>{const items=new Map();return {getItem:(key:string)=>items.get(key)??null,setItem:(key:string,value:string)=>{items.set(key,value);},removeItem:(key:string)=>{items.delete(key);}};}; + +async function rig(t:TestContext,{pageOrigin=POOL,globals={},reply=()=>Response.json({ok:true})}:{pageOrigin?:string;globals?:Record;reply?:(url:string)=>Response}={}){ + const key=(await loadOhttpKey((await generateOhttpKey()).env))!,bus=createNostrBus(),calls:string[]=[],direct:string[]=[],made:unknown[]=[]; + const router=createOhttpGateway({key,upstreams:[{origin:ARK,prefix:'/v1/'},{origin:new URL(ESPLORA).origin,prefix:'/api/'}],poolOrigin:POOL,poolLoopback:LOOPBACK,gatewayToken:'token',fetch:(async(url:string)=>{calls.push(url);return reply(url);}) as unknown as typeof fetch}); + // The relay pool hands each socket a normalized URL, so the gateway listens on the same strings. + const carrier=startNostrCarrier({open:router.open,secret:key.nostrSecret,relays:NOSTR_RELAYS.map(normalizeURL),pool:bus.pool});t.after(()=>carrier.close());await settle(); + // The sentinel is a WebSocket over the bus, speaking NIP-01 to the sealed fetch's own relay pool. + class Sentinel { + static OPEN=1;static CLOSING=2;static CLOSED=3; + readyState=0;onopen:(()=>void)|null=null;onmessage:((event:{data:string})=>void)|null=null;private subs=new Map(); + constructor(readonly url:string){made.push(new.target);setTimeout(()=>{if(this.readyState)return;this.readyState=1;this.onopen?.();});} + private answer(message:unknown[]){if(this.readyState===1)this.onmessage?.({data:JSON.stringify(message)});} + send(raw:string){ + const [type,first,second]=JSON.parse(raw); + if(type==='EVENT')void Promise.all(bus.pool.publish([this.url],first)).then(([note])=>this.answer(['OK',first.id,true,note]),(error:Error)=>this.answer(['OK',first.id,false,error.message])); + if(type==='REQ'){this.subs.set(first,bus.pool.subscribeMany([this.url],second,{onevent:event=>this.answer(['EVENT',first,event])}));this.answer(['EOSE',first]);} + if(type==='CLOSE'){this.subs.get(first)?.close();this.subs.delete(first);} + } + close(){this.readyState=3;for(const sub of this.subs.values())sub.close();this.subs.clear();} + } + const target:Record={fetch:async(input:string)=>{direct.push(String(input));return new Response('direct');},...globals}; + const transport=installPrivateTransport({config:key.config,gateway:key.nostrPublic,poolOrigin:POOL,pageOrigin,storage:memory(),socket:Sentinel as unknown as typeof WebSocket,globals:target});t.after(()=>transport.close()); + return {transport,globals:target,calls,direct,made,Sentinel,requests:()=>bus.published.filter(event=>event.kind===NOSTR_REQUEST_KIND)}; +} + +test('a relative pool path is rewritten to the pinned pool origin and sealed',async t=>{ + const g=await rig(t,{pageOrigin:PAGE}); + assert.equal((await g.transport.fetch('/api/rollup/status')).status,200); + assert.deepEqual([g.calls,g.direct],[[LOOPBACK+'/api/rollup/status'],[]]); + assert.ok(g.requests().length&&!JSON.stringify(g.requests()).includes('127.0.0.1')); +}); + +test('a pool path is sealed even when the page origin is the pool origin',async t=>{ + const g=await rig(t); + assert.equal((await g.transport.fetch('/api/rollup/status')).status,200); + assert.deepEqual([g.calls,g.direct],[[LOOPBACK+'/api/rollup/status'],[]]); +}); + +test('the page assets and the proving files go direct, and nothing else under /api does',async t=>{ + const g=await rig(t),paths=['/','/assets/app.js','/api/rollup/proving/epoch-spend.wasm?v='+'c'.repeat(64)]; + for(const path of paths)assert.equal(await (await g.transport.fetch(path)).text(),'direct',path); + assert.deepEqual(g.direct,paths.map(path=>POOL+path)); + for(const path of ['/api/native/releases','/api/other'])await assert.rejects(g.transport.fetch(path),/does not permit/,path); + assert.deepEqual([g.requests().length,g.calls.length],[0,0]); +}); + +test('an absolute proving URL at the pool origin is refused rather than sealed',async t=>{ + const g=await rig(t,{pageOrigin:PAGE}); + await assert.rejects(g.transport.fetch(POOL+'/api/rollup/proving/epoch-spend.wasm'),/does not permit/); + assert.deepEqual([g.requests().length,g.direct.length],[0,0]); +}); + +test('arkd, indexer and Esplora requests are classified before they are sealed',async t=>{ + const g=await rig(t),utxos=ESPLORA+'/address/tb1qexample/utxo',p=g.transport.providers(ARK,ARK); + await assert.rejects(g.transport.fetch(utxos),/not approved/,'Esplora is not approved until the SDK asks for its provider'); + p.onchainProvider('mutinynet'); + await assert.rejects(g.transport.fetch(ARK+'/v1/indexer/vtxos?scripts=aa&scripts=bb'),/exactly one item/); + await assert.rejects(g.transport.fetch('https://evil.example/x'),/not approved/); + assert.equal(g.requests().length,0,'nothing refused was published'); + for(const url of [ARK+'/v1/indexer/vtxos?scripts=aa',utxos])assert.equal((await g.transport.fetch(url)).status,200,url); + assert.deepEqual(g.calls,[ARK+'/v1/indexer/vtxos?scripts=aa',utxos]); +}); + +test('the transport fetch is marked private',async t=>{ + const g=await rig(t); + assert.ok(isPrivateFetch(g.transport.fetch));assert.equal(g.globals.fetch,g.transport.fetch); +}); + +test('EventSource, XMLHttpRequest and WebRTC are refused, and WebSocket only reaches the pinned relays',async t=>{ + const g=await rig(t),G=g.globals as Record{close?():void}>; + for(const name of ['EventSource','XMLHttpRequest','RTCPeerConnection'])assert.throws(()=>new G[name]!(ARK),/does not permit/,name); + for(const name of ['fetch','EventSource','WebSocket','XMLHttpRequest','RTCPeerConnection']){const {writable,configurable}=Object.getOwnPropertyDescriptor(g.globals,name)!;assert.deepEqual({writable,configurable},{writable:false,configurable:false},name);} + for(const url of ['wss://relay.snort.social','wss://relay.snort.social/']){const socket=new G.WebSocket!(url);assert.ok(socket instanceof g.Sentinel,url);socket.close!();} + for(const url of ['wss://nostr.arkade.sh','ws://relay.snort.social'])assert.throws(()=>new G.WebSocket!(url),/does not permit/,url); +}); + +test('the sealed fetch is given the original WebSocket, not the guarded one',async t=>{ + const g=await rig(t,{globals:{WebSocket:class {constructor(){throw new Error('the page WebSocket was used');}}}}); + assert.equal((await g.transport.fetch('/api/rollup/status')).status,200); + assert.ok(g.made.length&&g.made.every(made=>made===g.Sentinel),'every relay socket was built by the original constructor itself'); +}); + +test('providers carry no event source and an onchain provider that polls',async t=>{ + const g=await rig(t),p=g.transport.providers(ARK,ARK),onchain=p.onchainProvider('mutinynet'); + assert.deepEqual([Reflect.get(p.arkProvider,'eventSource'),Reflect.get(p.indexerProvider,'eventSource')],[undefined,undefined]); + assert.throws(()=>p.indexerProvider.getFilteredSubscription({} as never,new AbortController().signal),/individual script/); + assert.ok(onchain instanceof IsolatedOnchainProvider&&onchain.forcePolling); + const original=globalThis.fetch,sent:string[]=[];globalThis.fetch=(async(input:string)=>{sent.push(input);return Response.json({subscriptionId:'remote'});}) as unknown as typeof fetch; + try{await p.indexerProvider.unsubscribeForScripts(await p.indexerProvider.subscribeForScripts(['aa']));}finally{globalThis.fetch=original;} + assert.deepEqual(sent,[],'a watched script is polled, never subscribed'); +}); + +test('a pool spend POST is journalled before it goes out and resolved by its own GET',async t=>{ + let reserved:bigint[]=[]; + const g=await rig(t,{reply:url=>{if(url.endsWith('/spends')){reserved=g.transport.journal.pendingPoolNullifiers(POOL);return Response.json({id:ID},{status:202});}return Response.json({status:'included',txid:'d'.repeat(64)});}}); + assert.equal((await g.transport.fetch('/api/rollup/spends',{method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify({id:ID,slot:{nullifiers:['17']}})})).status,202); + assert.deepEqual([reserved,g.transport.journal.pendingPoolNullifiers(POOL)],[[17n],[17n]],'the entry was written before the POST reached the pool'); + assert.equal((await g.transport.fetch('/api/rollup/spends/'+ID)).status,200); + assert.deepEqual([g.calls,g.transport.journal.pendingPoolNullifiers(POOL)],[[LOOPBACK+'/api/rollup/spends',LOOPBACK+'/api/rollup/spends/'+ID],[]]); +}); + +test('the module never imports the native entry point',()=>{ + assert.doesNotMatch(readFileSync('src/shielded/private-transport.ts','utf8'),/native\/index/); + for(const file of ['policy','providers','mutations'])assert.doesNotMatch(readFileSync(`src/native/${file}.ts`,'utf8'),/@tauri-apps/,file); +}); From 5930fe4f2ac0df45be78f981b56174955cd17d4c Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 05:15:16 +0100 Subject: [PATCH 03/40] fix(shielded): route case-insensitively and connect to the relay URL the guard checked Express matches routes case-insensitively, so the three routing checks now read a lower-cased copy of the path, while the sealed or rewritten URL keeps its own case. /API/rollup/... on the page origin is therefore never sent direct. It is rewritten to the pinned pool origin and classified, and the native policy, whose pool routes are case-sensitive, refuses it before anything is published. A proving path in any case still goes direct. The guarded WebSocket now connects to the normalized URL it checked. It used to pass the raw string on, so a bare host such as relay.snort.social passed the check, and a browser resolves a relative WebSocket URL against the page, which would have opened a direct handshake to the pool. Tests: the import test reads six modules for directory, index, side-effect and dynamic imports of the native entry point and for @tauri-apps; a second install returns the first transport; on a page origin other than the pool, assets and proving files go direct; and the rig replaces Node's own WebSocket with one that throws, so a pool that loses its websocketImplementation fails at once instead of reaching the six public relays. --- src/shielded/private-transport.ts | 8 ++++---- tests/private-transport.test.ts | 33 ++++++++++++++++++++++++++----- 2 files changed, 32 insertions(+), 9 deletions(-) diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index 3b726957..9518f3e3 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -23,11 +23,11 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr const relays=config.relays??NOSTR_RELAYS,pinned=new Set(relays.map(normalizeURL)),policy=new NativePolicy(config.poolOrigin),{poolOrigin}=policy,pageOrigin=new URL(config.pageOrigin).origin; const journal=makeNativeMutationJournal(config.storage??globalThis.localStorage); const sealedFetch=createSealedFetch({config:config.config,gateway:config.gateway,origin:poolOrigin,relays,pool:new AbstractSimplePool({verifyEvent,maxWaitForConnection:3000,websocketImplementation:Original})}); - const Guarded=class extends Original {constructor(url:string|URL,protocols?:string|string[]){if(!pinned.has(normalizeURL(String(url))))throw new Error('Private networking does not permit a WebSocket to '+url);super(url,protocols);}}; - // Order matters: a pool API path is sealed at the pinned origin even when the page is served from it, and a proving path on the page origin goes direct even when it is not. + const Guarded=class extends Original {constructor(url:string|URL,protocols?:string|string[]){const u=normalizeURL(String(url));if(!pinned.has(u))throw new Error('Private networking does not permit a WebSocket to '+url);super(u,protocols);}}; + // Express routes ignore case, so these checks do. Order matters: a pool API path is sealed at the pinned origin even when the page is served from it, and a proving path on the page origin goes direct even when it is not. const route=(url:URL)=>{ - const path=url.pathname,proving=/^\/api\/rollup\/proving\//.test(path),poolApi=/^\/api\/rollup\//.test(path)&&!proving; - if(url.origin===pageOrigin){if(proving||!path.startsWith('/api/'))return undefined;if(poolApi)return poolOrigin+path+url.search;} + const path=url.pathname,lower=path.toLowerCase(),proving=/^\/api\/rollup\/proving\//.test(lower),poolApi=/^\/api\/rollup\//.test(lower)&&!proving; + if(url.origin===pageOrigin){if(proving||!lower.startsWith('/api/'))return undefined;if(poolApi)return poolOrigin+path+url.search;} else if(url.origin!==poolOrigin||poolApi)return url.href; throw new Error('Private networking does not permit '+url.origin+path); }; diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index ead2af64..c127fee4 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -9,7 +9,7 @@ import {createOhttpGateway} from '../src/rollup/ohttp-gateway.ts'; import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; import {isPrivateFetch} from '../src/shielded/network-privacy.ts'; -import {installPrivateTransport} from '../src/shielded/private-transport.ts'; +import {installPrivateTransport,privateTransport} from '../src/shielded/private-transport.ts'; import {createNostrBus} from './fixtures/nostr-bus.ts'; const POOL='https://pool.example',PAGE='http://127.0.0.1:5173',LOOPBACK='http://127.0.0.1:9',ARK='https://ark.example',ESPLORA=ESPLORA_URL.mutinynet,ID='ab'.repeat(16); @@ -17,6 +17,8 @@ const settle=()=>new Promise(resolve=>setImmediate(resolve)); const memory=()=>{const items=new Map();return {getItem:(key:string)=>items.get(key)??null,setItem:(key:string,value:string)=>{items.set(key,value);},removeItem:(key:string)=>{items.delete(key);}};}; async function rig(t:TestContext,{pageOrigin=POOL,globals={},reply=()=>Response.json({ok:true})}:{pageOrigin?:string;globals?:Record;reply?:(url:string)=>Response}={}){ + // Node's own WebSocket reaches the six public relays, so a pool that lost its websocketImplementation fails here instead. + const real=globalThis.WebSocket;globalThis.WebSocket=class {constructor(){throw new Error('a test reached for a public relay');}} as unknown as typeof WebSocket;t.after(()=>{globalThis.WebSocket=real;}); const key=(await loadOhttpKey((await generateOhttpKey()).env))!,bus=createNostrBus(),calls:string[]=[],direct:string[]=[],made:unknown[]=[]; const router=createOhttpGateway({key,upstreams:[{origin:ARK,prefix:'/v1/'},{origin:new URL(ESPLORA).origin,prefix:'/api/'}],poolOrigin:POOL,poolLoopback:LOOPBACK,gatewayToken:'token',fetch:(async(url:string)=>{calls.push(url);return reply(url);}) as unknown as typeof fetch}); // The relay pool hands each socket a normalized URL, so the gateway listens on the same strings. @@ -61,6 +63,20 @@ test('the page assets and the proving files go direct, and nothing else under /a assert.deepEqual([g.requests().length,g.calls.length],[0,0]); }); +test('on a page origin other than the pool, the page assets and the proving files still go direct',async t=>{ + const g=await rig(t,{pageOrigin:PAGE}),paths=['/assets/app.js','/api/rollup/proving/epoch-spend.wasm']; + for(const path of paths)assert.equal(await (await g.transport.fetch(path)).text(),'direct',path); + assert.deepEqual([g.direct,g.requests().length],[paths.map(path=>PAGE+path),0]); +}); + +test('a pool API path in any case is classified for sealing, never sent direct, and a proving path in any case goes direct',async t=>{ + const g=await rig(t,{pageOrigin:PAGE}); + await assert.rejects(g.transport.fetch('/API/rollup/status'),/not approved: GET https:\/\/pool\.example\/API\/rollup\/status$/,'rewritten to the pool origin in its own case, which the native policy refuses'); + await assert.rejects(g.transport.fetch('/API/other'),/does not permit/); + assert.equal(await (await g.transport.fetch('/Api/Rollup/proving/epoch-spend.wasm')).text(),'direct'); + assert.deepEqual([g.direct,g.requests().length],[[PAGE+'/Api/Rollup/proving/epoch-spend.wasm'],0]); +}); + test('an absolute proving URL at the pool origin is refused rather than sealed',async t=>{ const g=await rig(t,{pageOrigin:PAGE}); await assert.rejects(g.transport.fetch(POOL+'/api/rollup/proving/epoch-spend.wasm'),/does not permit/); @@ -83,11 +99,17 @@ test('the transport fetch is marked private',async t=>{ assert.ok(isPrivateFetch(g.transport.fetch));assert.equal(g.globals.fetch,g.transport.fetch); }); +test('a second install returns the first transport',async t=>{ + const g=await rig(t); + assert.equal(installPrivateTransport({config:new Uint8Array(),gateway:'',poolOrigin:'https://other.example',pageOrigin:PAGE,globals:g.globals}),g.transport); + assert.equal(privateTransport(),g.transport); +}); + test('EventSource, XMLHttpRequest and WebRTC are refused, and WebSocket only reaches the pinned relays',async t=>{ - const g=await rig(t),G=g.globals as Record{close?():void}>; + const g=await rig(t),G=g.globals as Record{url?:string;close?():void}>; for(const name of ['EventSource','XMLHttpRequest','RTCPeerConnection'])assert.throws(()=>new G[name]!(ARK),/does not permit/,name); for(const name of ['fetch','EventSource','WebSocket','XMLHttpRequest','RTCPeerConnection']){const {writable,configurable}=Object.getOwnPropertyDescriptor(g.globals,name)!;assert.deepEqual({writable,configurable},{writable:false,configurable:false},name);} - for(const url of ['wss://relay.snort.social','wss://relay.snort.social/']){const socket=new G.WebSocket!(url);assert.ok(socket instanceof g.Sentinel,url);socket.close!();} + for(const url of ['wss://relay.snort.social','wss://relay.snort.social/','relay.snort.social']){const socket=new G.WebSocket!(url);assert.ok(socket instanceof g.Sentinel,url);assert.equal(socket.url,'wss://relay.snort.social/',url);socket.close!();} for(const url of ['wss://nostr.arkade.sh','ws://relay.snort.social'])assert.throws(()=>new G.WebSocket!(url),/does not permit/,url); }); @@ -117,6 +139,7 @@ test('a pool spend POST is journalled before it goes out and resolved by its own }); test('the module never imports the native entry point',()=>{ - assert.doesNotMatch(readFileSync('src/shielded/private-transport.ts','utf8'),/native\/index/); - for(const file of ['policy','providers','mutations'])assert.doesNotMatch(readFileSync(`src/native/${file}.ts`,'utf8'),/@tauri-apps/,file); + const entry=/(?:from|import\(?)\s*'(?:\.\.\/native|\.)(?:\/index(?:\.[jt]s)?)?\/?'|@tauri-apps/; + for(const bad of ["import {x} from '../native';","import {x} from '../native/index.ts';","import {x} from '.';","import {x} from './index.js';","import '../native/';","await import('../native/index');","import {invoke} from '@tauri-apps/api/core';"])assert.match(bad,entry,bad); + for(const file of ['src/shielded/private-transport.ts','src/shielded/sealed-fetch.ts','src/shielded/network-privacy.ts','src/native/policy.ts','src/native/providers.ts','src/native/mutations.ts'])assert.doesNotMatch(readFileSync(file,'utf8'),entry,file); }); From eb7f89f55dcb1e5398adaff5d790a1c814759615 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 05:29:15 +0100 Subject: [PATCH 04/40] feat(shielded): size sealed pool pages and stop repeating a sealed 502 The gateway answers at most 241,644 B of upstream body (63*4096-8-16396); over that it answers a sealed 502. A catch-up page of 50 batches runs to about 0.5 MB, so a private browser wallet could never read it. The worst batch record is about 17.8 KB: 10 slots of 1,755 B, every field element 77 digits, a 420-hex ciphertext. A page of 12 of them is 213,345 B, 28,299 B inside the cap; 13 is 231,121 B; 16 is 284,449 B, over by 42,805 B. 12 is the size with real margin, and a test builds that record and pins PRIVATE_PAGE to it. - The private catch-up page size is a brand on the pool api, next to the private-api brand. markPrivatePoolPage/privatePoolPage read it back, and a size outside 1..100 reads as PRIVATE_PAGE_DEFAULT (50). poolApiOf takes an optional page, so a private fetcher without one pages at 50 and the native Tor host is unchanged. - ShieldedWalletConfig gains poolPage, passed to both the wallet api and the claim api (bearer inspection and claiming catch up through the same path). The live() wrapper in create() is a fresh function, so it copies the size across; without that the wrapped api would silently page at 50. - privateCatchUp reads the size from its api instead of a literal 50. Pages stay aligned to the epoch start and clipped at the epoch boundary. - rollupApi gives up at once on a 502 from a private fetcher. The gateway answers 502 both for an oversize page and for an unreachable upstream and they cannot be told apart on the wire, so an unreachable upstream also costs one try instead of five; the 60 s tick repeats the read. Every other status, and every direct fetcher, keeps its retries. The rule keys on isPrivateFetch, which the native Tor fetcher also carries. --- src/shielded/epoch-sync.ts | 4 +-- src/shielded/network-privacy.ts | 9 ++++-- src/shielded/pool.ts | 11 ++++---- src/shielded/wallet.ts | 10 +++---- tests/pool-rate-limit.test.ts | 10 +++++++ tests/tor-content.test.ts | 49 ++++++++++++++++++++++++++++++--- 6 files changed, 75 insertions(+), 18 deletions(-) diff --git a/src/shielded/epoch-sync.ts b/src/shielded/epoch-sync.ts index ca0a72b1..dd543138 100644 --- a/src/shielded/epoch-sync.ts +++ b/src/shielded/epoch-sync.ts @@ -3,7 +3,7 @@ import type {EpochAccount} from '../../packages/protocol/src/rollup/epoch-accoun import type {RollupEpochRecord} from '../../packages/protocol/src/rollup/epoch-record.ts'; import {decodeEpochScan,type EpochScanBatch} from '../../packages/protocol/src/rollup/epoch-scan.ts'; import type {PoolApi} from './pool.ts'; -import {isPrivatePoolApi} from './network-privacy.ts'; +import {isPrivatePoolApi,privatePoolPage} from './network-privacy.ts'; export const WATCHDOG_MS=60_000,EVENT_RETRY_MS=[0,1000,2000,4000,8000],SCAN_FROM=256,SCAN_AFTER=64,SCAN_PAGE=500; export interface ScanPage {from:number;total:number;data:string} @@ -65,7 +65,7 @@ export function epochCatchUp(account:EpochAccount,api:PoolApi,onBatch?:EpochOnBa async function privateCatchUp(account:EpochAccount,api:PoolApi,onBatch?:EpochOnBatch,onPage?:(cursor:number)=>Promise|void):Promise{ let from=Math.floor(account.batchCount/account.epochLen)*account.epochLen,applied=0; for(;;){ - const limit=Math.min(50,account.epochLen-from%account.epochLen),page=await api(`/batches?from=${from}&limit=${limit}`); + const limit=Math.min(privatePoolPage(api),account.epochLen-from%account.epochLen),page=await api(`/batches?from=${from}&limit=${limit}`); if(!Number.isSafeInteger(page.total)||page.total<0)throw new Error('The pool returned an invalid batch count.'); if(page.batches.length>limit)throw new Error('The pool returned more batches than requested.'); if(page.batches.lengthtypeof value==='function'&&(value as unknown as Record)[key]===true; -const mark=(value:T,key:symbol):T=>{Object.defineProperty(value,key,{value:true});return value;}; +const brandOf=(value:unknown,key:symbol)=>typeof value==='function'?(value as unknown as Record)[key]:undefined; +const marked=(value:unknown,key:symbol)=>brandOf(value,key)===true; +const mark=(value:T,key:symbol,brand:unknown=true):T=>{Object.defineProperty(value,key,{value:brand});return value;}; export const isPrivateFetch=(value:unknown):boolean=>marked(value,PRIVATE_FETCH); export const markPrivateFetch=(value:T):T=>mark(value,PRIVATE_FETCH); export const isPrivatePoolApi=(value:unknown):boolean=>marked(value,PRIVATE_POOL_API); export const markPrivatePoolApi=(value:T):T=>mark(value,PRIVATE_POOL_API); +export const markPrivatePoolPage=(value:T,size:number):T=>mark(value,PRIVATE_POOL_PAGE,size); +export const privatePoolPage=(value:unknown):number=>{const size=brandOf(value,PRIVATE_POOL_PAGE) as number;return Number.isSafeInteger(size)&&size>=1&&size<=100?size:PRIVATE_PAGE_DEFAULT;}; diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index e2b53ff3..3d242dec 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -3,16 +3,16 @@ import {base64,hex} from '@scure/base'; import {RawWitness} from '@scure/btc-signer'; import type {OwnedNote} from '../../packages/protocol/src/rollup/account.ts'; import {CARRIER_FREE_SATS,carrierFeeOf,depositFeeOf,parseDepositFee,type DepositFee,type DepositFeeSchedule} from '../../packages/protocol/src/rollup/deposit-fee.ts'; -import {isPrivateFetch,markPrivatePoolApi} from './network-privacy.ts'; +import {PRIVATE_PAGE_DEFAULT,isPrivateFetch,markPrivatePoolApi,markPrivatePoolPage} from './network-privacy.ts'; export interface RollupPoolStatus {phase:'starting'|'keys'|'funding'|'genesis'|'ready'|'blocked';message:string;network?:any;proving?:Record<'spend'|'join'|'batch'|'batchJoin',{wasm:string;zkey:string}>&{batchRecover?:{wasm:string;zkey:string}};pool?:{token:string;notice?:string;address:string;script:string;batches:number;root:string;head:{txid:string;vout:number;value:number};pending:number;padding:number;reserves?:Record;maxReserves?:number; epoch?:number;epochLen?:number;batchesInEpoch?:number;prevNoteRoot?:string;erCount?:number;rsCount?:number;nullifiers?:number;heapBytes?:number;depositFee?:DepositFee}} export interface SpendStatus {status:'pending'|'signing'|'included'|'dropped';batch?:number;txid?:string;reason?:string;arkTx?:string;checkpoint?:string;checkpoints?:string[];vin?:number} export interface Groth16Proof {pi_a:unknown;pi_b:unknown;pi_c:unknown} export type PoolApi=(path:string,body?:unknown,beforeRequest?:()=>void)=>Promise; -export const poolApiOf=(origin:string,fetcher:typeof fetch=fetch):PoolApi=>{ +export const poolApiOf=(origin:string,fetcher:typeof fetch=fetch,page?:number):PoolApi=>{ const api:PoolApi=(path:string,body?:unknown,beforeRequest?:()=>void)=>rollupApi(path,body,fetcher,1000,origin,beforeRequest); - return isPrivateFetch(fetcher)?markPrivatePoolApi(api):api; + return isPrivateFetch(fetcher)?markPrivatePoolPage(markPrivatePoolApi(api),page??PRIVATE_PAGE_DEFAULT):api; }; export const DUST=330; const sleep=(ms:number)=>new Promise(resolve=>setTimeout(resolve,ms)); @@ -29,7 +29,7 @@ export const rejected=(error:unknown,code:string):boolean=>typeof error==='objec /** Retries outages and rate limits; definitive refusals surface at once. */ export async function rollupApi(path:string,body?:unknown,fetcher:typeof fetch=fetch,pauseMs=1000,origin='',beforeRequest?:()=>void):Promise{ const key=new URL(origin||globalThis.location?.origin||'http://localhost').origin; - const privateMutation=body!==undefined&&isPrivateFetch(fetcher); + const isPrivate=isPrivateFetch(fetcher),privateMutation=body!==undefined&&isPrivate; const admission=body!==undefined&&!/^\/spends\/[^/]+\/sign(?:\?|$)/.test(path); const request:RequestInit={method:body===undefined?'GET':'POST',cache:'no-store',headers:body===undefined?{}:{'Content-Type':'application/json'},...(body===undefined?{}:{body:JSON.stringify(body)})}; for(let attempt=0;;attempt++){ @@ -56,7 +56,8 @@ export async function rollupApi(path:string,body?:unknown,fetcher:typeof fetc if(!response.ok)throw Object.assign(new PoolRequestError(metadata.error??`The pool answered ${response.status}.`,response.status),typeof metadata.code==='string'?{code:metadata.code}:{}); return json as T; } - if(privateMutation||attempt>=4)throw new PoolRequestError(`The pool did not answer (${response?.status??'network error'}).`,response?.status); + // A sealed 502 is the gateway's answer to an oversize page and to a dead pool alike; a repeat only repeats the first, so one try, and the 60 s tick reads again. + if(privateMutation||attempt>=4||isPrivate&&response?.status===502)throw new PoolRequestError(`The pool did not answer (${response?.status??'network error'}).`,response?.status); await sleep(pauseMs); } } diff --git a/src/shielded/wallet.ts b/src/shielded/wallet.ts index d4047f97..9c66eacb 100644 --- a/src/shielded/wallet.ts +++ b/src/shielded/wallet.ts @@ -28,11 +28,11 @@ import {DUST,depositFeeFor,pickNote,planFeeOf,poolApiOf,rejected,shieldPlan,shie import {SHIELDED_PRIORITY,poolFundedExit,shieldedRail,shieldedWithdrawRail} from './rails.ts'; import type {SentRecord,ShieldedNoteRepository,ShieldedSnapshot} from './repository.ts'; import {randomSubmissionId,submitAll,type SubmitDeps,type SubmitMember} from './submit.ts'; -import {isPrivatePoolApi,markPrivatePoolApi} from './network-privacy.ts'; +import {isPrivatePoolApi,markPrivatePoolApi,markPrivatePoolPage,privatePoolPage} from './network-privacy.ts'; export interface ShieldedPoolConfig {origin:string;status:RollupPoolStatus} export interface ShieldedProver {prove(built:EpochBuiltSpend,circuit:'spend'|'join'):Promise;joinReady?():boolean} -export type ShieldedWalletConfig=WalletConfig&{pool:ShieldedPoolConfig;secret:string;hash:Hash;prover:ShieldedProver;shieldedRepository?:ShieldedNoteRepository;lightningSpendStore?:LightningSpendStore;bornAt?:number;maxDepositFee?:DepositFeeSchedule;fetcher?:typeof fetch;networkPrivacy?:boolean;pendingNullifiers?:()=>ReadonlySet;pendingOutpoints?:()=>ReadonlySet;lnurlFetch?:typeof fetch;claimFetch?:typeof fetch; +export type ShieldedWalletConfig=WalletConfig&{pool:ShieldedPoolConfig;secret:string;hash:Hash;prover:ShieldedProver;shieldedRepository?:ShieldedNoteRepository;lightningSpendStore?:LightningSpendStore;bornAt?:number;maxDepositFee?:DepositFeeSchedule;fetcher?:typeof fetch;networkPrivacy?:boolean;poolPage?:number;pendingNullifiers?:()=>ReadonlySet;pendingOutpoints?:()=>ReadonlySet;lnurlFetch?:typeof fetch;claimFetch?:typeof fetch; /** Called once with the Arkade address, synchronously and before the pool sync; a throw aborts create. */ onArkAddress?:(address:string)=>void}; @@ -191,14 +191,14 @@ export class ShieldedWallet extends Wallet { // The manager starts up against the indexer while load() reads the pool; start() awaits this same start-up. void wallet.getContractManager().catch(()=>{}); config.onArkAddress?.(await wallet.getAddress()); - wallet.pool=config.pool;wallet.api=poolApiOf(config.pool.origin,config.fetcher);wallet.claimApi=config.claimFetch?poolApiOf(config.pool.origin,config.claimFetch):undefined;wallet.hash=config.hash;wallet.prover=config.prover; + wallet.pool=config.pool;wallet.api=poolApiOf(config.pool.origin,config.fetcher,config.poolPage);wallet.claimApi=config.claimFetch?poolApiOf(config.pool.origin,config.claimFetch,config.poolPage):undefined;wallet.hash=config.hash;wallet.prover=config.prover; wallet.networkPrivacy=config.networkPrivacy??isPrivatePoolApi(wallet.api); if(wallet.networkPrivacy){ - const live=(api:PoolApi):PoolApi=>markPrivatePoolApi(async(path:string,body?:unknown,beforeRequest?:()=>void)=>{ + const live=(api:PoolApi):PoolApi=>markPrivatePoolPage(markPrivatePoolApi(async(path:string,body?:unknown,beforeRequest?:()=>void)=>{ wallet.assertOpen(); const result=await api(path,body,()=>{wallet.assertOpen();beforeRequest?.();}); wallet.assertOpen();return result; - }); + }),privatePoolPage(api)); wallet.api=live(wallet.api);if(wallet.claimApi)wallet.claimApi=live(wallet.claimApi); } wallet.pendingNullifiers=config.pendingNullifiers;wallet.pendingOutpoints=config.pendingOutpoints;wallet.lnurlFetch=config.lnurlFetch; diff --git a/tests/pool-rate-limit.test.ts b/tests/pool-rate-limit.test.ts index 36e39b7c..3ae1ed04 100644 --- a/tests/pool-rate-limit.test.ts +++ b/tests/pool-rate-limit.test.ts @@ -2,6 +2,7 @@ import assert from 'node:assert/strict'; import {setImmediate} from 'node:timers/promises'; import {test} from 'node:test'; import {poolApiOf,rollupApi,waitForSpend} from '../src/shielded/pool.ts'; +import {markPrivateFetch} from '../src/shielded/network-privacy.ts'; const flush=async()=>{for(let i=0;i<8;i++)await setImmediate();}; const outcome=(promise:Promise)=>promise.then(value=>({value,error:undefined}),error=>({value:undefined,error})); @@ -177,6 +178,15 @@ test('definitive 400 and 409 refusals are never retried',async()=>{ } }); +test('a private read does not repeat a sealed 502, and a direct read still does',async()=>{ + for(const [label,isPrivate,status,calls] of [['private 502',true,502,1],['direct 502',false,502,5],['private 503',true,503,5]] as const){ + let count=0; + const raw=(async()=>{count++;return new Response('',{status});}) as typeof fetch; + await assert.rejects(rollupApi('/batches?from=0&limit=12',undefined,isPrivate?markPrivateFetch(raw):raw,0,'https://sealed-502.test'),(error:{status?:number})=>error.status===status,label); + assert.equal(count,calls,label); + } +}); + test('pending polling slows down but observes a new signing round well inside its 30-second window',async t=>{ t.mock.timers.enable({apis:['Date','setTimeout'],now:1_000_000}); const polls:number[]=[],signed:number[]=[]; diff --git a/tests/tor-content.test.ts b/tests/tor-content.test.ts index caceebd4..b186354d 100644 --- a/tests/tor-content.test.ts +++ b/tests/tor-content.test.ts @@ -3,20 +3,23 @@ import {test} from 'node:test'; import {DefaultVtxo,SingleKey,Wallet} from '@arkade-os/sdk'; import {buildPoseidon} from 'circomlibjs'; import type {EpochAccount} from '../packages/protocol/src/rollup/epoch-account.ts'; -import type {RollupEpochRecord} from '../packages/protocol/src/rollup/epoch-record.ts'; +import {BATCH_SLOTS,ROLLUP_FIELD} from '../packages/protocol/src/rollup/constants.ts'; +import type {RollupEpochRecord,RollupEpochRecordSlot} from '../packages/protocol/src/rollup/epoch-record.ts'; +import {ROLLUP_RECORD_BYTES} from '../packages/protocol/src/rollup/wallet.ts'; import {deriveRollupKeys2} from '../packages/protocol/src/wallet-keys.ts'; import {epochCatchUp} from '../src/shielded/epoch-sync.ts'; import {epochSyncAccount} from '../src/shielded/epoch-pool.ts'; import {ShieldedWallet,loadAccount} from '../src/shielded/wallet.ts'; import {lightningRails} from '../src/shielded/lightning.ts'; import {waitForSpend,type PoolApi} from '../src/shielded/pool.ts'; -import {markPrivateFetch,isPrivatePoolApi} from '../src/shielded/network-privacy.ts'; +import {markPrivateFetch,markPrivatePoolPage,privatePoolPage,isPrivatePoolApi} from '../src/shielded/network-privacy.ts'; +import {PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; import {epochLeafPool} from './fixtures/epoch-pool.ts'; import {makeNativeMutationJournal} from '../src/native/mutations.ts'; import {decodeClaimToken,encodeClaimToken} from '../packages/protocol/src/rollup/bearer.ts'; const privateApi=(api:PoolApi)=>Object.assign(api,{[Symbol.for('shielded.private.pool-api')]:true}); -const fakeHistory=(cursor:number,total:number,epochLen=128)=>{ +const fakeHistory=(cursor:number,total:number,epochLen=128,page?:number)=>{ const asked:string[]=[],applied:number[]=[],records=Array.from({length:total},(_,n)=>n as unknown as RollupEpochRecord); let batchCount=cursor; const account={get batchCount(){return batchCount;},epochLen,canSkip:true,async apply(batch:RollupEpochRecord){assert.equal(batch as unknown as number,batchCount);applied.push(batchCount++);}} as unknown as EpochAccount; @@ -25,7 +28,7 @@ const fakeHistory=(cursor:number,total:number,epochLen=128)=>{ const from=Number(url.searchParams.get('from')),limit=Number(url.searchParams.get('limit')); return {total,batches:records.slice(from,from+limit)}; }) as PoolApi); - return {account,api,asked,applied}; + return {account,api:page===undefined?api:markPrivatePoolPage(api,page),asked,applied}; }; test('private catch-up reads complete aligned pages and applies unseen batches with their true cursors',async()=>{ @@ -60,6 +63,44 @@ test('a truncated private page cannot force a tagged follow-up cursor',async()=> assert.equal(account.batchCount,83); }); +test('a private catch-up pages at the size its pool api carries, and the default stays 50',async()=>{ + const sized=fakeHistory(0,40,128,12),plain=fakeHistory(0,40,128); + assert.equal(await epochCatchUp(sized.account,sized.api),40); + assert.deepEqual(sized.asked,[0,12,24,36].map(from=>`/batches?from=${from}&limit=12`)); + await epochCatchUp(plain.account,plain.api); + assert.deepEqual(plain.asked,['/batches?from=0&limit=50']); +}); + +test('a private page never crosses an epoch boundary',async()=>{ + const {account,api,asked}=fakeHistory(120,135,128,12); + await epochCatchUp(account,api); + assert.deepEqual(asked.slice(-3),['/batches?from=108&limit=12','/batches?from=120&limit=8','/batches?from=128&limit=12']); +}); + +test('a size outside 1 to 100 reads back as 50, and a branded one as itself',()=>{ + for(const size of [0,-1,1.5,101,NaN,Infinity,2**53])assert.equal(privatePoolPage(markPrivatePoolPage(()=>{},size)),50,String(size)); + assert.deepEqual([1,12,100].map(size=>privatePoolPage(markPrivatePoolPage(()=>{},size))),[1,12,100]); + assert.equal(privatePoolPage(()=>{}),50);assert.equal(privatePoolPage(undefined),50); +}); + +test('a re-wrapped private pool api keeps its page size',async t=>{ + const sizes:number[]=[],sentinel=new Error('creation inspected'),wallet=Object.assign(Object.create(ShieldedWallet.prototype),{ + stopped:false,unwatch:[],changeListeners:new Set(),actionListeners:new Set(),getContractManager:async()=>({}),getAddress:async()=>'unused', + async load(this:{api:PoolApi;claimApi:PoolApi}){sizes.push(privatePoolPage(this.api),privatePoolPage(this.claimApi));throw sentinel;}, + }) as ShieldedWallet; + t.mock.method(Wallet,'create',async()=>wallet as never);t.mock.method(Wallet.prototype,'dispose',async()=>{}); + const fetcher=markPrivateFetch((async()=>Response.json({})) as typeof fetch); + await assert.rejects(ShieldedWallet.create({identity:undefined as never,pool:{origin:'https://pool.test',status:{pool:{token:'pool',script:'5120',address:'unused'}} as never},secret:'11'.repeat(32),hash:()=>1n,prover:undefined as never,fetcher,claimFetch:fetcher,networkPrivacy:true,poolPage:12}),error=>error===sentinel); + assert.deepEqual(sizes,[12,12]); +}); + +test('a worst-case page of 12 batches stays inside the gateway answer cap',()=>{ + const f='9'.repeat(String(ROLLUP_FIELD).length),slot:RollupEpochRecordSlot={anchor:f,rCur:{epoch:99_999,root:f},nullifiers:[f,f],commitments:[f,f],ctDigest:f,groupId:f,groupSize:10,publics:Array(7).fill(f),ciphertext:'f'.repeat(2*ROLLUP_RECORD_BYTES)}; + const record:RollupEpochRecord={kind:'spend',slots:Array(BATCH_SLOTS).fill(slot),noteBlockRoot:f,txid:'f'.repeat(64),at:1_700_000_000_000}; + const page=(n:number)=>JSON.stringify({from:0,total:n,batches:Array(n).fill(record)}).length,cap=63*4096-8-16_396; + assert.ok(page(PRIVATE_PAGE)<=cap,String(page(PRIVATE_PAGE)));assert.ok(page(16)>cap,String(page(16))); +}); + test('private fresh account avoids a birthday checkpoint and verifies the same final root',async()=>{ const poseidon=await buildPoseidon(),hash=(v:bigint[])=>BigInt(poseidon.F.toObject(poseidon(v))); const keys=deriveRollupKeys2('11'.repeat(32),'mutinynet'),{api,asked}=epochLeafPool(hash,256,140); From 8becf89579bbcd2a97273f519dd0dd062c03452a Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 05:38:38 +0100 Subject: [PATCH 05/40] fix(shielded): give up on a 502 only for the browser's sealed fetch 5bbdc06 keyed the no-repeat-502 rule on isPrivateFetch, which the native Tor fetchers carry as well. The rule exists for the OHTTP gateway's sealed answer, which is how an oversize page and an unreachable pool both arrive; the native host reaches the pool over Tor, not through that gateway, and its behaviour was to stay unchanged. A native 502 lost four of its five tries. - network-privacy.ts: a SEALED_FETCH brand with isSealedFetch and markSealedFetch, beside the private-fetch helpers. - private-transport.ts: the transport's fetch, which is both PrivateTransport.fetch and the page's installed global fetch, carries both marks: private as before, and now sealed. The native fetchers stay marked private only. - pool.ts: rollupApi gives up at once on a 502 only when isSealedFetch(fetcher). Every other fetcher, a private but unsealed native one included, keeps its five tries. A sealed 503 is still repeated, and a sealed POST is unchanged. --- src/shielded/network-privacy.ts | 3 +++ src/shielded/pool.ts | 7 ++++--- src/shielded/private-transport.ts | 6 +++--- tests/pool-rate-limit.test.ts | 27 ++++++++++++++++++++------- tests/private-transport.test.ts | 6 +++--- 5 files changed, 33 insertions(+), 16 deletions(-) diff --git a/src/shielded/network-privacy.ts b/src/shielded/network-privacy.ts index 0e1100de..a3ce71b4 100644 --- a/src/shielded/network-privacy.ts +++ b/src/shielded/network-privacy.ts @@ -1,4 +1,5 @@ export const PRIVATE_FETCH=Symbol.for('shielded.private.fetch'); +export const SEALED_FETCH=Symbol.for('shielded.sealed.fetch'); export const PRIVATE_POOL_API=Symbol.for('shielded.private.pool-api'); export const PRIVATE_POOL_PAGE=Symbol.for('shielded.private.pool-page'); export const PRIVATE_PAGE_DEFAULT=50; @@ -8,6 +9,8 @@ const marked=(value:unknown,key:symbol)=>brandOf(value,key)===true; const mark=(value:T,key:symbol,brand:unknown=true):T=>{Object.defineProperty(value,key,{value:brand});return value;}; export const isPrivateFetch=(value:unknown):boolean=>marked(value,PRIVATE_FETCH); export const markPrivateFetch=(value:T):T=>mark(value,PRIVATE_FETCH); +export const isSealedFetch=(value:unknown):boolean=>marked(value,SEALED_FETCH); +export const markSealedFetch=(value:T):T=>mark(value,SEALED_FETCH); export const isPrivatePoolApi=(value:unknown):boolean=>marked(value,PRIVATE_POOL_API); export const markPrivatePoolApi=(value:T):T=>mark(value,PRIVATE_POOL_API); export const markPrivatePoolPage=(value:T,size:number):T=>mark(value,PRIVATE_POOL_PAGE,size); diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 3d242dec..05481001 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -3,7 +3,7 @@ import {base64,hex} from '@scure/base'; import {RawWitness} from '@scure/btc-signer'; import type {OwnedNote} from '../../packages/protocol/src/rollup/account.ts'; import {CARRIER_FREE_SATS,carrierFeeOf,depositFeeOf,parseDepositFee,type DepositFee,type DepositFeeSchedule} from '../../packages/protocol/src/rollup/deposit-fee.ts'; -import {PRIVATE_PAGE_DEFAULT,isPrivateFetch,markPrivatePoolApi,markPrivatePoolPage} from './network-privacy.ts'; +import {PRIVATE_PAGE_DEFAULT,isPrivateFetch,isSealedFetch,markPrivatePoolApi,markPrivatePoolPage} from './network-privacy.ts'; export interface RollupPoolStatus {phase:'starting'|'keys'|'funding'|'genesis'|'ready'|'blocked';message:string;network?:any;proving?:Record<'spend'|'join'|'batch'|'batchJoin',{wasm:string;zkey:string}>&{batchRecover?:{wasm:string;zkey:string}};pool?:{token:string;notice?:string;address:string;script:string;batches:number;root:string;head:{txid:string;vout:number;value:number};pending:number;padding:number;reserves?:Record;maxReserves?:number; epoch?:number;epochLen?:number;batchesInEpoch?:number;prevNoteRoot?:string;erCount?:number;rsCount?:number;nullifiers?:number;heapBytes?:number;depositFee?:DepositFee}} @@ -29,7 +29,7 @@ export const rejected=(error:unknown,code:string):boolean=>typeof error==='objec /** Retries outages and rate limits; definitive refusals surface at once. */ export async function rollupApi(path:string,body?:unknown,fetcher:typeof fetch=fetch,pauseMs=1000,origin='',beforeRequest?:()=>void):Promise{ const key=new URL(origin||globalThis.location?.origin||'http://localhost').origin; - const isPrivate=isPrivateFetch(fetcher),privateMutation=body!==undefined&&isPrivate; + const privateMutation=body!==undefined&&isPrivateFetch(fetcher); const admission=body!==undefined&&!/^\/spends\/[^/]+\/sign(?:\?|$)/.test(path); const request:RequestInit={method:body===undefined?'GET':'POST',cache:'no-store',headers:body===undefined?{}:{'Content-Type':'application/json'},...(body===undefined?{}:{body:JSON.stringify(body)})}; for(let attempt=0;;attempt++){ @@ -57,7 +57,8 @@ export async function rollupApi(path:string,body?:unknown,fetcher:typeof fetc return json as T; } // A sealed 502 is the gateway's answer to an oversize page and to a dead pool alike; a repeat only repeats the first, so one try, and the 60 s tick reads again. - if(privateMutation||attempt>=4||isPrivate&&response?.status===502)throw new PoolRequestError(`The pool did not answer (${response?.status??'network error'}).`,response?.status); + // Only the browser's sealed fetch: a private native fetch is not the gateway, and keeps its retries. + if(privateMutation||attempt>=4||response?.status===502&&isSealedFetch(fetcher))throw new PoolRequestError(`The pool did not answer (${response?.status??'network error'}).`,response?.status); await sleep(pauseMs); } } diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index 9518f3e3..ab9fb2e2 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -6,7 +6,7 @@ import {NOSTR_RELAYS} from '../../packages/protocol/src/nostr-carrier.ts'; import {journalledRoute,makeNativeMutationJournal} from '../native/mutations.ts'; import {NativePolicy} from '../native/policy.ts'; import {IsolatedIndexerProvider,IsolatedOnchainProvider} from '../native/providers.ts'; -import {markPrivateFetch} from './network-privacy.ts'; +import {markPrivateFetch,markSealedFetch} from './network-privacy.ts'; import {createSealedFetch} from './sealed-fetch.ts'; export const PRIVATE_PAGE=12; @@ -31,7 +31,7 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr else if(url.origin!==poolOrigin||poolApi)return url.href; throw new Error('Private networking does not permit '+url.origin+path); }; - const privateFetch:typeof fetch=markPrivateFetch(async(input,init)=>{ + const privateFetch:typeof fetch=markSealedFetch(markPrivateFetch(async(input,init)=>{ const url=new URL(input instanceof Request?input.url:String(input),pageOrigin),href=route(url); if(href===undefined)return direct(input instanceof Request?input:url.href,init); const source=new Request(input instanceof Request?input:url,init),method=source.method,body=method==='GET'?undefined:await source.text(); @@ -41,7 +41,7 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr const response=await sealedFetch(href,{method,headers:source.headers,body,signal:source.signal}); if(journalled)journal.observeNativeResponse({url:href,method,status:response.status,body:await response.clone().text(),requestBody:body}); return response; - }); + })); const transport:PrivateTransport={fetch:privateFetch,poolOrigin,pageSize:PRIVATE_PAGE,journal, providers(arkUrl,indexerUrl){ policy.registerArk(arkUrl,indexerUrl); diff --git a/tests/pool-rate-limit.test.ts b/tests/pool-rate-limit.test.ts index 3ae1ed04..ab91ea44 100644 --- a/tests/pool-rate-limit.test.ts +++ b/tests/pool-rate-limit.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict'; import {setImmediate} from 'node:timers/promises'; import {test} from 'node:test'; import {poolApiOf,rollupApi,waitForSpend} from '../src/shielded/pool.ts'; -import {markPrivateFetch} from '../src/shielded/network-privacy.ts'; +import {markPrivateFetch,markSealedFetch} from '../src/shielded/network-privacy.ts'; const flush=async()=>{for(let i=0;i<8;i++)await setImmediate();}; const outcome=(promise:Promise)=>promise.then(value=>({value,error:undefined}),error=>({value:undefined,error})); @@ -178,12 +178,25 @@ test('definitive 400 and 409 refusals are never retried',async()=>{ } }); -test('a private read does not repeat a sealed 502, and a direct read still does',async()=>{ - for(const [label,isPrivate,status,calls] of [['private 502',true,502,1],['direct 502',false,502,5],['private 503',true,503,5]] as const){ - let count=0; - const raw=(async()=>{count++;return new Response('',{status});}) as typeof fetch; - await assert.rejects(rollupApi('/batches?from=0&limit=12',undefined,isPrivate?markPrivateFetch(raw):raw,0,'https://sealed-502.test'),(error:{status?:number})=>error.status===status,label); - assert.equal(count,calls,label); +test('a sealed read gives up on a 502 at once, without waiting or asking again',async t=>{ + t.mock.timers.enable({apis:['Date','setTimeout'],now:1_000_000}); + let calls=0,settled=false; + const fetcher=markSealedFetch(markPrivateFetch((async()=>{calls++;return new Response('',{status:502});}) as typeof fetch)); + const pending=outcome(rollupApi('/batches?from=0&limit=12',undefined,fetcher,1000,'https://sealed-502.test')).then(result=>{settled=true;return result;}); + await flush(); + assert.equal(settled,true);assert.equal(calls,1);assert.equal((await pending).error?.status,502); +}); + +test('a private read that is not sealed still repeats a 502, as does a direct one, and a sealed read still repeats a 503',async t=>{ + for(const [label,priv,sealed,status] of [['private 502',true,false,502],['direct 502',false,false,502],['sealed 503',true,true,503]] as const){ + t.mock.timers.enable({apis:['Date','setTimeout'],now:1_000_000}); + let calls=0; + const raw=(async()=>{calls++;return new Response('',{status});}) as typeof fetch,fetcher=sealed?markSealedFetch(markPrivateFetch(raw)):priv?markPrivateFetch(raw):raw; + const pending=outcome(rollupApi('/batches?from=0&limit=12',undefined,fetcher,1000,'https://sealed-502.test')); + await flush();assert.equal(calls,1,label); + for(let i=1;i<5;i++){t.mock.timers.tick(1000);await flush();assert.equal(calls,i+1,label);} + assert.equal((await pending).error?.status,status,label); + t.mock.timers.reset(); } }); diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index c127fee4..80c82b75 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -8,7 +8,7 @@ import {IsolatedOnchainProvider} from '../src/native/providers.ts'; import {createOhttpGateway} from '../src/rollup/ohttp-gateway.ts'; import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; -import {isPrivateFetch} from '../src/shielded/network-privacy.ts'; +import {isPrivateFetch,isSealedFetch} from '../src/shielded/network-privacy.ts'; import {installPrivateTransport,privateTransport} from '../src/shielded/private-transport.ts'; import {createNostrBus} from './fixtures/nostr-bus.ts'; @@ -94,9 +94,9 @@ test('arkd, indexer and Esplora requests are classified before they are sealed', assert.deepEqual(g.calls,[ARK+'/v1/indexer/vtxos?scripts=aa',utxos]); }); -test('the transport fetch is marked private',async t=>{ +test('the transport fetch is marked private and sealed',async t=>{ const g=await rig(t); - assert.ok(isPrivateFetch(g.transport.fetch));assert.equal(g.globals.fetch,g.transport.fetch); + assert.ok(isPrivateFetch(g.transport.fetch));assert.ok(isSealedFetch(g.transport.fetch));assert.equal(g.globals.fetch,g.transport.fetch); }); test('a second install returns the first transport',async t=>{ From 2f6aefeebdc5429b4a13688ce2b77beaf09a327a Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 06:10:59 +0100 Subject: [PATCH 06/40] feat(native): let a journal release a spend a 404 proves lost makeNativeMutationJournal(storage, {graceMs, now}) takes an optional grace. A journal built with one releases an entry when GET /spends/:id answers 404 and graceMs < now - createdAt < 1 h. The pool tracks a spend for an hour after admitting it (service.ts:255,361), so a 404 inside that hour means the POST never arrived; past the hour a 404 proves nothing and the entry stays for Plan 4. The 404 is read before the body is parsed, since reconcileNativeMutations passes an empty 404 body through. There is no default grace. The native host builds its journal without options (current()), so it never releases on a 404: no total request deadline was found in its Rust Tor transport (Ruling 2). createdAt now comes from the journal's clock, which defaults to Date.now, so native behaviour is unchanged. A dropped spend keeps its reservation, as before. --- src/native/mutations.ts | 16 +++++++++------- tests/tor-mutations.test.ts | 29 ++++++++++++++++++++++++++++- 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/src/native/mutations.ts b/src/native/mutations.ts index 7cb30c41..47d00cbd 100644 --- a/src/native/mutations.ts +++ b/src/native/mutations.ts @@ -4,7 +4,7 @@ type JournalStorage=Pick; export interface NativeMutation {url:string;method:string;body?:string} export interface NativeMutationResponse extends NativeMutation {status:number;requestBody?:string} interface PendingMutation {origin:string;id:string;body:string;nullifiers:string[];coin?:string;createdAt:number} -const KEY='shielded-native-pool-mutations-v1',MAX_BYTES=4*1024*1024,MAX_ENTRIES=256; +const KEY='shielded-native-pool-mutations-v1',MAX_BYTES=4*1024*1024,MAX_ENTRIES=256,TRACKED_MS=3_600_000; const REFUSED_CODES=new Set(['slot-shape','group-invalid','recover-unarchived','window-tag','nullifier-range','stale-anchor','fee-missing']); const REFUSED_BODY=new Set(['The request body is not valid JSON.','A spend id is 8 to 32 random bytes in hex.','A spend carries its seven public inputs.','Malformed epoch slot.','An epoch slot names its anchor, not a root.','An epoch slot names exactly one of rCur, for a fast-path spend, and origin, for a recovery.','A recovery names its origin epoch as a whole number.','A window leaf names a whole epoch and its root.','A group has a nonzero id and two or three members; a lone spend has neither.','Expected a decimal field element.','An asset id is 68 lowercase hex characters.','Expected lowercase hex of the right length.','Malformed Groth16 proof.']); // http.ts refuses these before service.submit; named operator refusals remove their tentative admission before throwing. @@ -27,7 +27,7 @@ const feeGroup=(body:string)=>{ return typeof slot.groupId==='string'&&/^[1-9]\d{0,77}$/.test(slot.groupId)&&BigInt(slot.groupId)Date.now()}:{graceMs?:number;now?:()=>number}={}){ const load=():PendingMutation[]=>{ const raw=storage.getItem(KEY);if(raw===null)return []; if(raw.length>MAX_BYTES)throw invalid(); @@ -49,6 +49,7 @@ export function makeNativeMutationJournal(storage:JournalStorage){ if(entries.length>MAX_ENTRIES||new TextEncoder().encode(encoded).byteLength>MAX_BYTES)throw new Error('The native payment journal is full; no payment was sent.'); storage.setItem(KEY,encoded); }; + const drop=(gone:(row:PendingMutation)=>boolean)=>{const entries=load(),remaining=entries.filter(row=>!gone(row));if(remaining.length!==entries.length)save(remaining);}; const beforeNativeMutation=(request:NativeMutation)=>{ if(request.method.toUpperCase()!=='POST')return; const {origin,path}=route(request.url);if(path!=='/api/rollup/spends')return; @@ -58,7 +59,7 @@ export function makeNativeMutationJournal(storage:JournalStorage){ if(kept.id===parsed.id)throw new Error('This payment has an uncertain outcome; its funds remain reserved until reconciled.'); if(parsed.nullifiers.some(n=>kept.nullifiers.includes(n))||parsed.coin&&kept.coin===parsed.coin)throw new Error('These funds remain reserved by an uncertain payment.'); } - save([...entries,{origin,...parsed,body:request.body,createdAt:Date.now()}]); + save([...entries,{origin,...parsed,body:request.body,createdAt:now()}]); }; const pendingPoolNullifiers=(poolOrigin:string):bigint[]=>{ const origin=new URL(poolOrigin).origin; @@ -70,7 +71,9 @@ export function makeNativeMutationJournal(storage:JournalStorage){ }; const observeNativeResponse=(response:NativeMutationResponse)=>{ if(response.body===undefined)return; - const method=response.method.toUpperCase(),{origin,path}=route(response.url); + const method=response.method.toUpperCase(),{origin,path}=route(response.url),spend=method==='GET'?/^\/api\/rollup\/spends\/([0-9a-f]{32})$/.exec(path)?.[1]:undefined; + // The pool forgets a spend an hour after admitting it, so only a 404 inside that hour proves it never arrived; its body may be empty. + if(spend&&response.status===404){if(graceMs!==undefined){const at=now();drop(row=>row.origin===origin&&row.id===spend&&at-row.createdAt>graceMs&&at-row.createdAtrow.origin!==origin||row.id!==refused.id&&(!group||feeGroup(row.body)!==group)); save(remaining);return; } - const match=/^\/api\/rollup\/spends\/([0-9a-f]{32})$/.exec(path);if(method!=='GET'||response.status!==200||!match)return; + if(!spend||response.status!==200)return; // An abandoned batch can still land, correcting its spend from dropped to included. if(!result||!(result.status==='included'&&typeof result.txid==='string'&&/^[0-9a-f]{64}$/.test(result.txid)&&(result.batch===undefined||Number.isSafeInteger(result.batch)&&result.batch>=0)))return; - const entries=load(),remaining=entries.filter(row=>row.origin!==origin||row.id!==match[1]); - if(remaining.length!==entries.length)save(remaining); + drop(row=>row.origin===origin&&row.id===spend); }; const reconcileNativeMutations=async(fetcher:typeof fetch,poolOrigin:string)=>{ const origin=new URL(poolOrigin).origin,entries=load().filter(row=>row.origin===origin),controller=new AbortController();let next=0; diff --git a/tests/tor-mutations.test.ts b/tests/tor-mutations.test.ts index f15fb0c4..21e67b1a 100644 --- a/tests/tor-mutations.test.ts +++ b/tests/tor-mutations.test.ts @@ -63,7 +63,7 @@ test('uncertain deposits reserve the same coin across differently named submissi assert.throws(()=>journal.beforeNativeMutation({...mutation(other,['23']),body:body(other,['23'],coin)}),/reserved/i); }); -test('404, network errors, nonterminal states and malformed terminal evidence retain reservations',async()=>{ +test('network errors, nonterminal states, malformed terminal evidence and a 404 inside the grace retain reservations',async()=>{ for(const response of [new Response('',{status:404}),Response.json({status:'pending'}),Response.json({status:'signing'}),Response.json({status:'included'}),Response.json({status:'dropped',reason:42})]){ const journal=makeNativeMutationJournal(storage());journal.beforeNativeMutation(mutation()); await journal.reconcileNativeMutations((async()=>response) as typeof fetch,origin); @@ -74,6 +74,33 @@ test('404, network errors, nonterminal states and malformed terminal evidence re assert.deepEqual(journal.pendingPoolNullifiers(origin),[17n]); }); +const UNKNOWN=JSON.stringify({error:'Unknown spend; it may have expired or the server restarted.'}); +const lost=(journal:ReturnType,answer='')=>journal.reconcileNativeMutations((async()=>new Response(answer,{status:404})) as typeof fetch,origin); +test('a 404 inside the grace keeps a spend reserved, and past it releases',async()=>{ + const coin={txid:'c'.repeat(64),vout:1}; + for(const answer of ['',UNKNOWN]){ + let at=1_000_000;const journal=makeNativeMutationJournal(storage(),{graceMs:180_000,now:()=>at}); + journal.beforeNativeMutation({...mutation(),body:body(id,['17'],coin)}); + at+=179_999;await lost(journal,answer); + assert.deepEqual([journal.pendingPoolNullifiers(origin),journal.pendingPoolOutpoints(origin)],[[17n],[`${coin.txid}:1`]],answer); + at+=2;await lost(journal,answer); + assert.deepEqual([journal.pendingPoolNullifiers(origin),journal.pendingPoolOutpoints(origin)],[[],[]],answer); + } +}); + +test('a 404 after the pool stops tracking leaves the entry pending',async()=>{ + let at=1_000_000;const journal=makeNativeMutationJournal(storage(),{graceMs:180_000,now:()=>at}); + journal.beforeNativeMutation(mutation()); + at+=3_600_001;await lost(journal,UNKNOWN); + assert.deepEqual(journal.pendingPoolNullifiers(origin),[17n]); +}); + +test('a journal built without a grace never releases on a 404',async t=>{ + let at=1_000_000;t.mock.method(Date,'now',()=>at); + const journal=makeNativeMutationJournal(storage());journal.beforeNativeMutation(mutation()); + for(const later of [300_001,3_599_999]){at=1_000_000+later;await lost(journal,UNKNOWN);assert.deepEqual(journal.pendingPoolNullifiers(origin),[17n],String(later));} +}); + test('only a matching definitive status response retires a reservation durably',async()=>{ for(const answer of [{status:'included',txid:'d'.repeat(64),batch:1},{status:'included',txid:'d'.repeat(64)}]){ const persisted=storage(),journal=makeNativeMutationJournal(persisted);journal.beforeNativeMutation(mutation()); From 689ab5339bd1cddf2aaee23655e2ed5a434cb1dd Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 06:14:17 +0100 Subject: [PATCH 07/40] feat(shielded): give the sealed transport a grace, a page brand and an unsent release The browser transport builds its journal with graceMs PRIVATE_GRACE_MS (180 s), so a sealed GET /spends/:id that answers 404 past the grace, and inside the pool's hour, frees the spend's notes and coin. A sealed request can land at most ~135 s after its entry was written: the 90 s date window, 15 s of rounding and the 30 s upstream deadline. That fits the grace while wallet and gateway clocks are within 45 s. A journalled POST whose sealed fetch throws too-large or unavailable provably never left: too-large is refused before sealing goes out, and for a non-GET unavailable means publish never began (Plan 2b Ruling 11). Its entry is released at once through the journal's new cancelNativeMutation, matched on origin and exact body, and the error is rethrown. outcome-unknown keeps the entry for the 404 rule. Ruling 6: the transport fetch also carries its page size (markPrivatePoolPage(fetch, PRIVATE_PAGE)) beside its private and sealed marks, and poolApiOf takes its default page from the fetcher's brand. So every poolApiOf('') in a sealed page (fullViewOf on /watch, home, claim) pages at 12 with no per-site wiring. An explicit page still wins, and a native fetcher, marked private only, keeps 50. --- src/native/mutations.ts | 4 +++- src/shielded/pool.ts | 4 ++-- src/shielded/private-transport.ts | 17 ++++++++------ tests/private-transport.test.ts | 38 +++++++++++++++++++++++++++---- 4 files changed, 49 insertions(+), 14 deletions(-) diff --git a/src/native/mutations.ts b/src/native/mutations.ts index 47d00cbd..1c849f2c 100644 --- a/src/native/mutations.ts +++ b/src/native/mutations.ts @@ -61,6 +61,8 @@ export function makeNativeMutationJournal(storage:JournalStorage,{graceMs,now=() } save([...entries,{origin,...parsed,body:request.body,createdAt:now()}]); }; + /** For a request that provably never left: its entry goes at once. */ + const cancelNativeMutation=(request:NativeMutation)=>{const {origin}=route(request.url);drop(row=>row.origin===origin&&row.body===request.body);}; const pendingPoolNullifiers=(poolOrigin:string):bigint[]=>{ const origin=new URL(poolOrigin).origin; return [...new Set(load().filter(row=>row.origin===origin).flatMap(row=>row.nullifiers))].map(BigInt); @@ -102,7 +104,7 @@ export function makeNativeMutationJournal(storage:JournalStorage,{graceMs,now=() }))]);}finally{clearTimeout(deadline);controller.abort();} return {pending:load().filter(row=>row.origin===origin).length}; }; - return {beforeNativeMutation,pendingPoolNullifiers,pendingPoolOutpoints,observeNativeResponse,reconcileNativeMutations}; + return {beforeNativeMutation,cancelNativeMutation,pendingPoolNullifiers,pendingPoolOutpoints,observeNativeResponse,reconcileNativeMutations}; } const current=()=>{if(!globalThis.localStorage)throw new Error('The native payment journal is unavailable; no payment was sent.');return makeNativeMutationJournal(globalThis.localStorage);}; diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 05481001..87be4b44 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -3,7 +3,7 @@ import {base64,hex} from '@scure/base'; import {RawWitness} from '@scure/btc-signer'; import type {OwnedNote} from '../../packages/protocol/src/rollup/account.ts'; import {CARRIER_FREE_SATS,carrierFeeOf,depositFeeOf,parseDepositFee,type DepositFee,type DepositFeeSchedule} from '../../packages/protocol/src/rollup/deposit-fee.ts'; -import {PRIVATE_PAGE_DEFAULT,isPrivateFetch,isSealedFetch,markPrivatePoolApi,markPrivatePoolPage} from './network-privacy.ts'; +import {isPrivateFetch,isSealedFetch,markPrivatePoolApi,markPrivatePoolPage,privatePoolPage} from './network-privacy.ts'; export interface RollupPoolStatus {phase:'starting'|'keys'|'funding'|'genesis'|'ready'|'blocked';message:string;network?:any;proving?:Record<'spend'|'join'|'batch'|'batchJoin',{wasm:string;zkey:string}>&{batchRecover?:{wasm:string;zkey:string}};pool?:{token:string;notice?:string;address:string;script:string;batches:number;root:string;head:{txid:string;vout:number;value:number};pending:number;padding:number;reserves?:Record;maxReserves?:number; epoch?:number;epochLen?:number;batchesInEpoch?:number;prevNoteRoot?:string;erCount?:number;rsCount?:number;nullifiers?:number;heapBytes?:number;depositFee?:DepositFee}} @@ -12,7 +12,7 @@ export interface Groth16Proof {pi_a:unknown;pi_b:unknown;pi_c:unknown} export type PoolApi=(path:string,body?:unknown,beforeRequest?:()=>void)=>Promise; export const poolApiOf=(origin:string,fetcher:typeof fetch=fetch,page?:number):PoolApi=>{ const api:PoolApi=(path:string,body?:unknown,beforeRequest?:()=>void)=>rollupApi(path,body,fetcher,1000,origin,beforeRequest); - return isPrivateFetch(fetcher)?markPrivatePoolPage(markPrivatePoolApi(api),page??PRIVATE_PAGE_DEFAULT):api; + return isPrivateFetch(fetcher)?markPrivatePoolPage(markPrivatePoolApi(api),page??privatePoolPage(fetcher)):api; }; export const DUST=330; const sleep=(ms:number)=>new Promise(resolve=>setTimeout(resolve,ms)); diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index ab9fb2e2..38352019 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -6,8 +6,8 @@ import {NOSTR_RELAYS} from '../../packages/protocol/src/nostr-carrier.ts'; import {journalledRoute,makeNativeMutationJournal} from '../native/mutations.ts'; import {NativePolicy} from '../native/policy.ts'; import {IsolatedIndexerProvider,IsolatedOnchainProvider} from '../native/providers.ts'; -import {markPrivateFetch,markSealedFetch} from './network-privacy.ts'; -import {createSealedFetch} from './sealed-fetch.ts'; +import {markPrivateFetch,markPrivatePoolPage,markSealedFetch} from './network-privacy.ts'; +import {createSealedFetch,SealedFetchError} from './sealed-fetch.ts'; export const PRIVATE_PAGE=12; export const PRIVATE_GRACE_MS=180_000; @@ -21,7 +21,7 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr if(installed)return installed; const target=(config.globals??globalThis) as Record,direct=target.fetch as typeof fetch,Original=config.socket??(target.WebSocket as typeof WebSocket); const relays=config.relays??NOSTR_RELAYS,pinned=new Set(relays.map(normalizeURL)),policy=new NativePolicy(config.poolOrigin),{poolOrigin}=policy,pageOrigin=new URL(config.pageOrigin).origin; - const journal=makeNativeMutationJournal(config.storage??globalThis.localStorage); + const journal=makeNativeMutationJournal(config.storage??globalThis.localStorage,{graceMs:PRIVATE_GRACE_MS}); const sealedFetch=createSealedFetch({config:config.config,gateway:config.gateway,origin:poolOrigin,relays,pool:new AbstractSimplePool({verifyEvent,maxWaitForConnection:3000,websocketImplementation:Original})}); const Guarded=class extends Original {constructor(url:string|URL,protocols?:string|string[]){const u=normalizeURL(String(url));if(!pinned.has(u))throw new Error('Private networking does not permit a WebSocket to '+url);super(u,protocols);}}; // Express routes ignore case, so these checks do. Order matters: a pool API path is sealed at the pinned origin even when the page is served from it, and a proving path on the page origin goes direct even when it is not. @@ -31,17 +31,20 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr else if(url.origin!==poolOrigin||poolApi)return url.href; throw new Error('Private networking does not permit '+url.origin+path); }; - const privateFetch:typeof fetch=markSealedFetch(markPrivateFetch(async(input,init)=>{ + const privateFetch:typeof fetch=markPrivatePoolPage(markSealedFetch(markPrivateFetch(async(input,init)=>{ const url=new URL(input instanceof Request?input.url:String(input),pageOrigin),href=route(url); if(href===undefined)return direct(input instanceof Request?input:url.href,init); const source=new Request(input instanceof Request?input:url,init),method=source.method,body=method==='GET'?undefined:await source.text(); policy.classify(href,method,body); const journalled=journalledRoute(href,method); - if(journalled&&method!=='GET')journal.beforeNativeMutation({url:href,method,body}); - const response=await sealedFetch(href,{method,headers:source.headers,body,signal:source.signal}); + const mutation=journalled&&method!=='GET'?{url:href,method,body}:undefined; + if(mutation)journal.beforeNativeMutation(mutation); + let response:Response; + try{response=await sealedFetch(href,{method,headers:source.headers,body,signal:source.signal});} + catch(error){if(mutation&&error instanceof SealedFetchError&&(error.code==='too-large'||error.code==='unavailable'))journal.cancelNativeMutation(mutation);throw error;} if(journalled)journal.observeNativeResponse({url:href,method,status:response.status,body:await response.clone().text(),requestBody:body}); return response; - })); + })),PRIVATE_PAGE); const transport:PrivateTransport={fetch:privateFetch,poolOrigin,pageSize:PRIVATE_PAGE,journal, providers(arkUrl,indexerUrl){ policy.registerArk(arkUrl,indexerUrl); diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index 80c82b75..b66af4a5 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -3,13 +3,15 @@ import {readFileSync} from 'node:fs'; import {test,type TestContext} from 'node:test'; import {ESPLORA_URL} from '@arkade-os/sdk'; import {normalizeURL} from 'nostr-tools/utils'; -import {NOSTR_RELAYS,NOSTR_REQUEST_KIND} from '../packages/protocol/src/nostr-carrier.ts'; +import {NOSTR_CHUNK_BYTES,NOSTR_RELAYS,NOSTR_REQUEST_KIND} from '../packages/protocol/src/nostr-carrier.ts'; import {IsolatedOnchainProvider} from '../src/native/providers.ts'; import {createOhttpGateway} from '../src/rollup/ohttp-gateway.ts'; import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; -import {isPrivateFetch,isSealedFetch} from '../src/shielded/network-privacy.ts'; -import {installPrivateTransport,privateTransport} from '../src/shielded/private-transport.ts'; +import {isPrivateFetch,isSealedFetch,markPrivateFetch,privatePoolPage} from '../src/shielded/network-privacy.ts'; +import {poolApiOf} from '../src/shielded/pool.ts'; +import {installPrivateTransport,privateTransport,PRIVATE_GRACE_MS,PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; +import {SealedFetchError} from '../src/shielded/sealed-fetch.ts'; import {createNostrBus} from './fixtures/nostr-bus.ts'; const POOL='https://pool.example',PAGE='http://127.0.0.1:5173',LOOPBACK='http://127.0.0.1:9',ARK='https://ark.example',ESPLORA=ESPLORA_URL.mutinynet,ID='ab'.repeat(16); @@ -39,7 +41,7 @@ async function rig(t:TestContext,{pageOrigin=POOL,globals={},reply=()=>Response. } const target:Record={fetch:async(input:string)=>{direct.push(String(input));return new Response('direct');},...globals}; const transport=installPrivateTransport({config:key.config,gateway:key.nostrPublic,poolOrigin:POOL,pageOrigin,storage:memory(),socket:Sentinel as unknown as typeof WebSocket,globals:target});t.after(()=>transport.close()); - return {transport,globals:target,calls,direct,made,Sentinel,requests:()=>bus.published.filter(event=>event.kind===NOSTR_REQUEST_KIND)}; + return {transport,globals:target,bus,calls,direct,made,Sentinel,requests:()=>bus.published.filter(event=>event.kind===NOSTR_REQUEST_KIND)}; } test('a relative pool path is rewritten to the pinned pool origin and sealed',async t=>{ @@ -99,6 +101,11 @@ test('the transport fetch is marked private and sealed',async t=>{ assert.ok(isPrivateFetch(g.transport.fetch));assert.ok(isSealedFetch(g.transport.fetch));assert.equal(g.globals.fetch,g.transport.fetch); }); +test('a pool api on the transport fetch pages at the transport size, unless told otherwise, and a native private fetcher keeps 50',async t=>{ + const g=await rig(t),native=markPrivateFetch((async()=>Response.json({})) as typeof fetch); + assert.deepEqual([poolApiOf('',g.transport.fetch),poolApiOf('',g.transport.fetch,30),poolApiOf('',native)].map(api=>privatePoolPage(api)),[PRIVATE_PAGE,30,50]); +}); + test('a second install returns the first transport',async t=>{ const g=await rig(t); assert.equal(installPrivateTransport({config:new Uint8Array(),gateway:'',poolOrigin:'https://other.example',pageOrigin:PAGE,globals:g.globals}),g.transport); @@ -138,6 +145,29 @@ test('a pool spend POST is journalled before it goes out and resolved by its own assert.deepEqual([g.calls,g.transport.journal.pendingPoolNullifiers(POOL)],[[LOOPBACK+'/api/rollup/spends',LOOPBACK+'/api/rollup/spends/'+ID],[]]); }); +const spendPost=(id:string,nf:string,pad='')=>({method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify({id,slot:{nullifiers:[nf]},pad})}); +test('a sealed 404 frees a journalled spend only once the grace has passed',async t=>{ + const g=await rig(t,{reply:url=>url.endsWith('/spends')?Response.json({id:ID},{status:202}):Response.json({error:'Unknown spend; it may have expired or the server restarted.'},{status:404})}); + const start=Date.now();let at=start;t.mock.method(Date,'now',()=>at); + assert.equal((await g.transport.fetch('/api/rollup/spends',spendPost(ID,'17'))).status,202); + for(const [later,left] of [[PRIVATE_GRACE_MS-1,[17n]],[PRIVATE_GRACE_MS+1,[]]] as const){ + at=start+later;assert.equal((await g.transport.fetch('/api/rollup/spends/'+ID)).status,404); + assert.deepEqual(g.transport.journal.pendingPoolNullifiers(POOL),left,String(later)); + } +}); + +test('a spend POST that never went out frees its reservation at once, and one that may have run keeps it',async t=>{ + const g=await rig(t),fails=(code:string)=>(error:unknown)=>error instanceof SealedFetchError&&error.code===code; + await assert.rejects(g.transport.fetch('/api/rollup/spends',spendPost(ID,'17','0'.repeat(NOSTR_CHUNK_BYTES))),fails('too-large')); + assert.deepEqual(g.transport.journal.pendingPoolNullifiers(POOL),[]); + for(const relay of NOSTR_RELAYS)g.bus.relay(normalizeURL(relay)).accept=false; + await assert.rejects(g.transport.fetch('/api/rollup/spends',spendPost(ID,'17')),fails('outcome-unknown'),'the freed id goes out again, and every relay refuses it'); + assert.deepEqual(g.transport.journal.pendingPoolNullifiers(POOL),[17n]); + g.transport.close(); + await assert.rejects(g.transport.fetch('/api/rollup/spends',spendPost('cd'.repeat(16),'23')),fails('unavailable')); + assert.deepEqual([g.transport.journal.pendingPoolNullifiers(POOL),g.calls],[[17n],[]]); +}); + test('the module never imports the native entry point',()=>{ const entry=/(?:from|import\(?)\s*'(?:\.\.\/native|\.)(?:\/index(?:\.[jt]s)?)?\/?'|@tauri-apps/; for(const bad of ["import {x} from '../native';","import {x} from '../native/index.ts';","import {x} from '.';","import {x} from './index.js';","import '../native/';","await import('../native/index');","import {invoke} from '@tauri-apps/api/core';"])assert.match(bad,entry,bad); From 7f361f59f7e9ddc615e152ff636f1860e552db2d Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 06:20:10 +0100 Subject: [PATCH 08/40] feat(shielded): strip the deposit sign request to the depositor's own input service.sign() checks only that the signed arkTx and checkpoint keep their txids (service.ts:565-566), and signDeposits copies only getInput(vin).tapScriptSig from the signed arkTx (:312). So signDeposit signs the full arkTx as before, then returns a rebuilt PSBT carrying the unsigned transaction (every input's txid, index and sequence, and every output), and on the depositor's own input only witnessUtxo, tapLeafScript and tapScriptSig. Inputs and outputs go in before the signature, since scure-btc-signer refuses addOutput once an input is signed. The checkpoint goes whole: signDeposits replaces the operator's with it, and stockCustomerSigned reads its leaf. The full form carried every input's source transaction (PrevArkTxField, covenant.ts:103). At 10 deposits whose coins come from a 5.7 KB transaction, the body was 93 496 B, over a sealed event's 32 KiB and over express.json's 64 KiB, so a direct /sign would also answer 413. It is now 12 692 B. The merged batch still passes stockCustomerSigned and verifyStockCustomerSignatures for every deposit, at the same estimated weight. --- src/shielded/pool.ts | 8 +++++- tests/rollup-epoch-service.test.ts | 39 +++++++++++++++++++++++++++--- 2 files changed, 42 insertions(+), 5 deletions(-) diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 87be4b44..7d6cce6c 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -99,8 +99,14 @@ export function checkSigningRequest(request:SpendStatus,coin:{txid:string;vout:n if(!batchStatements(ark).includes(statement))throw new Error('The batch does not carry this deposit; not signing.'); } +/** The pool checks only the arkTx's id and reads only our input's signature, so the arkTx goes back without every input's source transaction; the checkpoint replaces the operator's whole. */ export async function signDeposit(identity:Identity,request:SpendStatus){ - const ark=await identity.sign(Transaction.fromPSBT(base64.decode(request.arkTx!)),[request.vin!]),checkpoint=await identity.sign(Transaction.fromPSBT(base64.decode(request.checkpoint!)),[0]); + const vin=request.vin!,signed=await identity.sign(Transaction.fromPSBT(base64.decode(request.arkTx!)),[vin]),checkpoint=await identity.sign(Transaction.fromPSBT(base64.decode(request.checkpoint!)),[0]); + const ark=new Transaction({version:signed.version,lockTime:signed.lockTime}); + for(let i=0;i{ + const dir=scratch('rollup-sign-');cpSync(keySet('epoch'),join(dir,'keys'),{recursive:true});writeFileSync(join(dir,'spec.json'),JSON.stringify(rollupSpecOf(token,serverKey))); + const leaves=await loadRollupEpochLeaves(DEFAULT_VM_BINARY,join(dir,'spec.json')),pool=rollupEpochPoolTree(hex.decode(serverKey),hex.decode(emulatorKey),leaves,exitDelay); + const fixture=JSON.parse(readFileSync('tools/vm/testdata/rollup-covenant-snarkjs.json','utf8')) as {newPacket:string;slots:{proof:SnarkProof;publicSignals:string[]}[];batch:{proof:SnarkProof}}; + const depositors=await Promise.all(Array.from({length:BATCH_SLOTS},async(_,i)=>{ + const key=SingleKey.fromHex((0x40+i).toString(16).repeat(32)),user=await key.xOnlyPublicKey(),collab=MultisigTapscript.encode({pubkeys:[user,hex.decode(serverKey)]}).script; + return {key,collab,tree:new VtxoScript([collab,CSVMultisigTapscript.encode({timelock:{type:'seconds',value:2048n},pubkeys:[user]}).script])}; + })); + // About 5.7 KB, as a pool payout's batch transaction is with its witness: the full request then carries one copy per deposit. + const head=offlineNativeFixture([{script:pool.tree.pkScript,amount:100_330n}]),payout=offlineNativeFixture(depositors.map(d=>({script:d.tree.pkScript,amount:1000n})),[new UnknownPacket(0x7e,new Uint8Array(5_200))]); + const built=buildRollupBatchTx({head:{txid:head.id,vout:0,value:100_330,sourceTx:head.toBytes(),tapTree:pool.tree.encode(),leaf:pool.tree.findLeaf(hex.encode(pool.batch))},token,leaves,checkpoint, + deposits:depositors.map((d,vout)=>({txid:payout.id,vout,value:1000,sourceTx:payout.toBytes(),tapTree:d.tree.encode(),leaf:d.tree.findLeaf(hex.encode(d.collab))})),legs:depositors.map(()=>({deposit:1000n,withdraw:0n,asset:false})), + witness:rollupWitness(fixture.batch.proof,fixture.slots.map(s=>({proof:s.proof,publics:s.publicSignals.map(BigInt)}))),newPacket:hex.decode(fixture.newPacket)}); + const arkTx=base64.encode(built.arkTx.toPSBT()),checkpoints=built.checkpoints.map(tx=>base64.encode(tx.toPSBT())),idOf=(psbt:string)=>Transaction.fromPSBT(base64.decode(psbt)).id; + const merged=Transaction.fromPSBT(base64.decode(arkTx)),fromFull=Transaction.fromPSBT(base64.decode(arkTx)),signed=[...checkpoints],first=merged.inputsLength-depositors.length; + for(const [i,{key}] of depositors.entries()){ + const vin=first+i,payload=await signDeposit(key,{status:'signing',arkTx,checkpoint:checkpoints[vin],checkpoints,vin}),stripped=Transaction.fromPSBT(base64.decode(payload.arkTx)); + const full=await key.sign(Transaction.fromPSBT(base64.decode(arkTx)),[vin]); + assert.ok(Buffer.byteLength(JSON.stringify(payload))<=16_384,String(Buffer.byteLength(JSON.stringify(payload)))); + assert.ok(Buffer.byteLength(JSON.stringify({arkTx:base64.encode(full.toPSBT()),checkpoint:payload.checkpoint}))>65_536); + assert.deepEqual([idOf(payload.arkTx),idOf(payload.checkpoint)],[idOf(arkTx),idOf(checkpoints[vin]!)]); + assert.deepEqual(Array.from({length:stripped.inputsLength},(_,n)=>stripped.getInput(n).tapScriptSig?.map(([,sig])=>sig.length)),Array.from({length:stripped.inputsLength},(_,n)=>n===vin?[64]:undefined)); + merged.updateInput(vin,{tapScriptSig:stripped.getInput(vin).tapScriptSig});fromFull.updateInput(vin,{tapScriptSig:full.getInput(vin).tapScriptSig});signed[vin]=payload.checkpoint; + } + const request={arkTx:base64.encode(merged.toPSBT()),checkpoints:signed}; + assert.deepEqual(depositors.map((_,i)=>stockCustomerSigned(request,serverKey,first+i)),depositors.map(()=>true)); + verifyStockCustomerSignatures(request,serverKey); + assert.equal(stockSignedWeights(request,true).ark,stockSignedWeights({arkTx:base64.encode(fromFull.toPSBT()),checkpoints:signed},true).ark); +}); From 4ecf17f6c8f38703fd3ab55aff2288bd6d3b65eb Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 06:21:06 +0100 Subject: [PATCH 09/40] fix(shielded): widen the browser's 404 grace to 15 minutes Ruling 7: a late copy can land until ~135 s plus however far the gateway's clock lags the wallet's, so 180 s held only within 45 s of skew. 900 s holds up to ~12.7 minutes of lag, past the ~105 s at which the gateway already refuses fresh requests as stale. The browser transport test now pins 900 000 literally; the journal's own mechanism tests keep injecting 180 000. --- src/shielded/private-transport.ts | 2 +- tests/private-transport.test.ts | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index 38352019..6f9a3fae 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -10,7 +10,7 @@ import {markPrivateFetch,markPrivatePoolPage,markSealedFetch} from './network-pr import {createSealedFetch,SealedFetchError} from './sealed-fetch.ts'; export const PRIVATE_PAGE=12; -export const PRIVATE_GRACE_MS=180_000; +export const PRIVATE_GRACE_MS=900_000; export interface PrivateTransportConfig {config:Uint8Array;gateway:string;poolOrigin:string;pageOrigin:string;relays?:readonly string[];storage?:Pick;socket?:typeof WebSocket;globals?:Record} export interface PrivateTransport {fetch:typeof fetch;poolOrigin:string;pageSize:number;journal:ReturnType;providers(arkUrl:string,indexerUrl:string):{arkProvider:RestArkProvider;indexerProvider:IsolatedIndexerProvider;onchainProvider:(network:NetworkName)=>OnchainProvider};close():void} diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index b66af4a5..6e44de30 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -10,7 +10,7 @@ import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; import {isPrivateFetch,isSealedFetch,markPrivateFetch,privatePoolPage} from '../src/shielded/network-privacy.ts'; import {poolApiOf} from '../src/shielded/pool.ts'; -import {installPrivateTransport,privateTransport,PRIVATE_GRACE_MS,PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; +import {installPrivateTransport,privateTransport,PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; import {SealedFetchError} from '../src/shielded/sealed-fetch.ts'; import {createNostrBus} from './fixtures/nostr-bus.ts'; @@ -146,11 +146,11 @@ test('a pool spend POST is journalled before it goes out and resolved by its own }); const spendPost=(id:string,nf:string,pad='')=>({method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify({id,slot:{nullifiers:[nf]},pad})}); -test('a sealed 404 frees a journalled spend only once the grace has passed',async t=>{ +test('a sealed 404 frees a journalled spend only once the 15-minute grace has passed',async t=>{ const g=await rig(t,{reply:url=>url.endsWith('/spends')?Response.json({id:ID},{status:202}):Response.json({error:'Unknown spend; it may have expired or the server restarted.'},{status:404})}); const start=Date.now();let at=start;t.mock.method(Date,'now',()=>at); assert.equal((await g.transport.fetch('/api/rollup/spends',spendPost(ID,'17'))).status,202); - for(const [later,left] of [[PRIVATE_GRACE_MS-1,[17n]],[PRIVATE_GRACE_MS+1,[]]] as const){ + for(const [later,left] of [[899_999,[17n]],[900_001,[]]] as const){ at=start+later;assert.equal((await g.transport.fetch('/api/rollup/spends/'+ID)).status,404); assert.deepEqual(g.transport.journal.pendingPoolNullifiers(POOL),left,String(later)); } From 08e1481e1c6e2231bd6309e84b94081bd2677884 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 06:25:28 +0100 Subject: [PATCH 10/40] feat(shielded): refuse batch swaps on a wallet whose transport cannot carry them ShieldedWalletConfig gains batchSwaps (default true). Set to false, settle() and sendSelectedVtxosToSelf() reject before nativeSpend runs, so before any intent is registered: "Renewal and on-chain exits need Arkade's live batch stream, which this private transport cannot carry." That covers renew() and the plain on-chain send rails.ts falls back to, which offboards through this wallet's settle. The overrides are async, so a refusal is always a rejected promise. The pool-funded exit (rails.ts:99) swaps on the companion's settle, through payout.wallet (lightning-wallet.ts:67), which this wallet's guard never sees. So exitFromPool runs the same check first, before the pool withdraws to a fresh key. In the private boot the companion is never opened and rails.ts:97 refuses earlier still. Only an explicit false refuses. A wallet that never set the field keeps today's behaviour, including the tests' Object.create wallets, and send() is untouched. --- src/shielded/wallet.ts | 12 ++++++++---- tests/private-lightning-spends.test.ts | 22 ++++++++++++++++++---- 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/src/shielded/wallet.ts b/src/shielded/wallet.ts index 9c66eacb..ab6f5594 100644 --- a/src/shielded/wallet.ts +++ b/src/shielded/wallet.ts @@ -32,7 +32,7 @@ import {isPrivatePoolApi,markPrivatePoolApi,markPrivatePoolPage,privatePoolPage} export interface ShieldedPoolConfig {origin:string;status:RollupPoolStatus} export interface ShieldedProver {prove(built:EpochBuiltSpend,circuit:'spend'|'join'):Promise;joinReady?():boolean} -export type ShieldedWalletConfig=WalletConfig&{pool:ShieldedPoolConfig;secret:string;hash:Hash;prover:ShieldedProver;shieldedRepository?:ShieldedNoteRepository;lightningSpendStore?:LightningSpendStore;bornAt?:number;maxDepositFee?:DepositFeeSchedule;fetcher?:typeof fetch;networkPrivacy?:boolean;poolPage?:number;pendingNullifiers?:()=>ReadonlySet;pendingOutpoints?:()=>ReadonlySet;lnurlFetch?:typeof fetch;claimFetch?:typeof fetch; +export type ShieldedWalletConfig=WalletConfig&{pool:ShieldedPoolConfig;secret:string;hash:Hash;prover:ShieldedProver;shieldedRepository?:ShieldedNoteRepository;lightningSpendStore?:LightningSpendStore;bornAt?:number;maxDepositFee?:DepositFeeSchedule;fetcher?:typeof fetch;networkPrivacy?:boolean;poolPage?:number;batchSwaps?:boolean;pendingNullifiers?:()=>ReadonlySet;pendingOutpoints?:()=>ReadonlySet;lnurlFetch?:typeof fetch;claimFetch?:typeof fetch; /** Called once with the Arkade address, synchronously and before the pool sync; a throw aborts create. */ onArkAddress?:(address:string)=>void}; @@ -178,6 +178,7 @@ export class ShieldedWallet extends Wallet { private nativeOperations=0; private statusAt=Date.now(); private networkPrivacy=false; + private batchSwaps?:boolean; private pendingNullifiers?:()=>ReadonlySet; private pendingOutpoints?:()=>ReadonlySet; private lnurlFetch?:typeof fetch; @@ -192,7 +193,7 @@ export class ShieldedWallet extends Wallet { void wallet.getContractManager().catch(()=>{}); config.onArkAddress?.(await wallet.getAddress()); wallet.pool=config.pool;wallet.api=poolApiOf(config.pool.origin,config.fetcher,config.poolPage);wallet.claimApi=config.claimFetch?poolApiOf(config.pool.origin,config.claimFetch,config.poolPage):undefined;wallet.hash=config.hash;wallet.prover=config.prover; - wallet.networkPrivacy=config.networkPrivacy??isPrivatePoolApi(wallet.api); + wallet.networkPrivacy=config.networkPrivacy??isPrivatePoolApi(wallet.api);wallet.batchSwaps=config.batchSwaps; if(wallet.networkPrivacy){ const live=(api:PoolApi):PoolApi=>markPrivatePoolPage(markPrivatePoolApi(async(path:string,body?:unknown,beforeRequest?:()=>void)=>{ wallet.assertOpen(); @@ -650,6 +651,7 @@ export class ShieldedWallet extends Wallet { void handle.settled().then(release,error=>{if(canRelease(error))release();try{onFail?.();}catch{}}); } private assertOpen(){if(this.stopped)throw new Error('The wallet was closed.');} + private assertBatchSwaps(){if(this.batchSwaps===false)throw new Error('Renewal and on-chain exits need Arkade\'s live batch stream, which this private transport cannot carry.');} private reserved(coin:{txid:string;vout:number},pending=this.pendingOutpoints?.()){const outpoint=`${coin.txid}:${coin.vout}`;return this.used.has(outpoint)||this.usedTxids.has(coin.txid)||pending?.has(outpoint)===true;} private async nativeSpend(inputs:readonly {txid:string;vout:number}[]|undefined,run:()=>Promise){ this.assertOpen(); @@ -666,8 +668,8 @@ export class ShieldedWallet extends Wallet { override send(...args:Parameters){ return this.nativeSpend('recipients' in args[0]?args[0].selectedVtxos:undefined,()=>super.send(...args)); } - override settle(...args:Parameters){return this.nativeSpend(args[0]?.inputs,()=>super.settle(...args));} - override sendSelectedVtxosToSelf(...args:Parameters){return this.nativeSpend(args[0],()=>super.sendSelectedVtxosToSelf(...args));} + override async settle(...args:Parameters){this.assertBatchSwaps();return this.nativeSpend(args[0]?.inputs,()=>super.settle(...args));} + override async sendSelectedVtxosToSelf(...args:Parameters){this.assertBatchSwaps();return this.nativeSpend(args[0],()=>super.sendSelectedVtxosToSelf(...args));} override async getContractManager(){ this.assertOpen(); const manager=await super.getContractManager(); @@ -806,6 +808,8 @@ export class ShieldedWallet extends Wallet { canExitPrivately():boolean{return typeof this.swaps?.exitPayout==='function';} /** Withdraws `amount` to a fresh never-reused key, saved and watched before the withdrawal, so an interrupted exit still finds the payout. */ exitFromPool(amount:number,title:string,exit:(coin:NormalizedExtendedVirtualCoin,payout:IWallet)=>Promise,dest:{to:string;sats:number}):ShieldedPaymentHandle{ + // Its batch swap runs on the companion's settle, so the guard on this wallet's would come after the pool paid the fresh key. + this.assertBatchSwaps(); return this.announce(shieldedHandle({title,steps:shieldedSteps.exit,railId:'shielded-exit'},async step=>{ this.assertOpen(); const reserve=this.swaps?.exitPayout; diff --git a/tests/private-lightning-spends.test.ts b/tests/private-lightning-spends.test.ts index b01a777b..e7300df6 100644 --- a/tests/private-lightning-spends.test.ts +++ b/tests/private-lightning-spends.test.ts @@ -1,14 +1,14 @@ import assert from 'node:assert/strict'; import {test} from 'node:test'; import {setImmediate} from 'node:timers/promises'; -import {ArkAddress,DefaultVtxo,InMemoryContractRepository,InMemoryWalletRepository,SingleKey,Transaction,VtxoScript,type ExtendedVirtualCoin} from '@arkade-os/sdk'; +import {ArkAddress,DefaultVtxo,InMemoryContractRepository,InMemoryWalletRepository,SingleKey,Transaction,VtxoScript,Wallet,type ExtendedVirtualCoin} from '@arkade-os/sdk'; import {base64,hex} from '@scure/base'; import {Script} from '@scure/btc-signer'; import {buildPoseidon} from 'circomlibjs'; import info from './fixtures/ark-info.json' with {type:'json'}; import {lightningSpendConflict,sameLightningSpendRequest,validateLightningSpend,type LightningSpendRecord,type LightningSpendStore} from '../src/shielded/lightning-spend.ts'; import {InMemoryShieldedNoteRepository} from '../src/shielded/repository.ts'; -import {ShieldedWallet,shieldedKeysOf,type ShieldedProver} from '../src/shielded/wallet.ts'; +import {SHIELD_FLOOR_MS,ShieldedWallet,shieldedKeysOf,type ShieldedProver,type ShieldedWalletConfig} from '../src/shielded/wallet.ts'; import {openRollupNotes} from '../packages/protocol/src/rollup/wallet.ts'; import {rollupApi,type PoolApi,type RollupPoolStatus} from '../src/shielded/pool.ts'; import {batchWitnessOutput} from './fixtures/batch-witness.ts'; @@ -38,14 +38,14 @@ class Store implements LightningSpendStore { async finish(owner:string,pool:string,id:string,result:{state:'included'|'dropped';txid?:string;reason?:string}){const r=(await this.get(owner,pool,id))!;this.rows.set(id,{...r,...result});} } /** `head` is the pool the fetcher serves, so a reopened wallet can meet the batch and root its snapshot saved. */ -async function open(store:LightningSpendStore,repository=new InMemoryShieldedNoteRepository(),p=prover,head={batches:0,root:genesis!.root}){ +async function open(store:LightningSpendStore,repository=new InMemoryShieldedNoteRepository(),p=prover,head={batches:0,root:genesis!.root},extra:Partial={}){ return ShieldedWallet.create({identity:SingleKey.fromHex('11'.repeat(32)),walletMode:'static',settlementConfig:false, storage:{walletRepository:new InMemoryWalletRepository(),contractRepository:new InMemoryContractRepository()}, arkProvider:{serverUrl:'https://mutinynet.arkade.sh',getInfo:async()=>info} as never, indexerProvider:{getVtxos:async()=>({vtxos:[]}),getVirtualTxs:async()=>({txs:[]}),subscribeForScripts:async()=>'sub',getSubscription:async function*(){},unsubscribeForScripts:async()=>{}} as never, onchainProvider:{getCoins:async()=>[],getFeeRate:async()=>1,broadcastTransaction:async()=>'x',getChainTip:async()=>({height:1,time:1}),watchAddresses:async()=>()=>{}} as never, pool:{origin:'',status},secret:'11'.repeat(32),hash,prover:p,shieldedRepository:repository,lightningSpendStore:store, - fetcher:(async(url:string)=>new Response(JSON.stringify(url.endsWith('/status')?{...status,pool:{...status.pool!,...head}}:{total:head.batches,batches:[]}))) as unknown as typeof fetch}); + fetcher:(async(url:string)=>new Response(JSON.stringify(url.endsWith('/status')?{...status,pool:{...status.pool!,...head}}:{total:head.batches,batches:[]}))) as unknown as typeof fetch,...extra}); } async function fund(wallet:ShieldedWallet,amounts=[900n]){const deposits=await Promise.all(amounts.map(deposit=>wallet.account.spend({deposit},wallet.self)));await wallet.account.apply(epochPool(hash,EPOCH_LEN).land('spend',deposits.map(posted)));await wallet.save();} const missing=()=>Object.assign(new Error('Unknown spend after restart.'),{status:404}); @@ -561,6 +561,20 @@ test('a fee refusal on a Lightning shield reads the fee again, so the next attem }finally{await wallet.dispose();} }); +test('a wallet with batch swaps off refuses renewal and the on-chain exit before any intent is registered',async t=>{ + const settle=t.mock.method(Wallet.prototype,'settle',async()=>'renewed');t.mock.method(Wallet.prototype,'send',async()=>'sent'); + const near=async()=>[{txid:TXID,vout:0,value:1000,expiresAt:Date.now()+SHIELD_FLOOR_MS-60_000}]; + const off=await open(new Store(),undefined,prover,undefined,{batchSwaps:false}),on=await open(new Store()); + try{ + off.coins=near;on.coins=near; + for(const swap of [()=>off.renew(),()=>off.settle(),()=>off.sendSelectedVtxosToSelf([])])await assert.rejects(swap(),/batch stream/); + assert.throws(()=>off.exitFromPool(1000,'Sending 1,000 sats on-chain',async()=>{throw new Error('the exit ran');},{to:'tb1qexit',sats:1000}),/batch stream/,'before the pool pays a fresh key'); + assert.equal(settle.mock.callCount(),0); + assert.equal(await off.send({recipients:[{address:destination,amount:330}]}),'sent'); + assert.equal(await on.renew(),'renewed');assert.equal(settle.mock.callCount(),1); + }finally{await off.dispose();await on.dispose();} +}); + test('a stale-anchor refusal still drops at once, on the pool verdict alone and without reading a batch',async()=>{ const store=new Store(),{wallet,pool}=await funded(store);const reason='The spend opens against another previous-epoch root than the next batch does; prove it again.';let batches=0; try{ From 97728f11d8759bb107368c49f0828e70a50b4330 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 07:13:44 +0100 Subject: [PATCH 11/40] fix(shielded): refuse an on-chain send at quote time when batch swaps are off A wallet built with batchSwaps:false (the browser's private transport) failed an on-chain send two different ways. When one pool note covered it, the quote threw that the companion wallet "is not open. Reload and try again", which no reload fixes, because a private wallet never opens one. When the Arkade coins paid it instead, the quote passed and the send failed later in settle(). The pool-funded exit now asks the wallet's existing batch-swap guard first, so both branches, and a send with selected coins, refuse at quote time with the batch-stream message. The guard becomes public for that; a wallet with the default config passes it, so its quotes are unchanged. --- src/shielded/rails.ts | 1 + src/shielded/wallet.ts | 2 +- tests/private-lightning-spends.test.ts | 10 ++++++++++ tests/shielded-rails.test.ts | 2 +- 4 files changed, 13 insertions(+), 2 deletions(-) diff --git a/src/shielded/rails.ts b/src/shielded/rails.ts index 9cc88ae1..1c3b3663 100644 --- a/src/shielded/rails.ts +++ b/src/shielded/rails.ts @@ -86,6 +86,7 @@ function freshPayoutFor(total:number,fees:FeeInfo):number{ export function poolFundedExit(wallet:ShieldedWallet,rail:PaymentRail,feeInfo:()=>Promise):PaymentRail{ const sats=(value:number)=>value.toLocaleString('en-US'); return {...rail,async quote(req,ctx){ + wallet.assertBatchSwaps(); const quote=await rail.quote(req,ctx); if(req.selectedVtxos)return quote; const gross=freshPayoutFor(quote.total,await feeInfo()); diff --git a/src/shielded/wallet.ts b/src/shielded/wallet.ts index ab6f5594..e205a844 100644 --- a/src/shielded/wallet.ts +++ b/src/shielded/wallet.ts @@ -651,7 +651,7 @@ export class ShieldedWallet extends Wallet { void handle.settled().then(release,error=>{if(canRelease(error))release();try{onFail?.();}catch{}}); } private assertOpen(){if(this.stopped)throw new Error('The wallet was closed.');} - private assertBatchSwaps(){if(this.batchSwaps===false)throw new Error('Renewal and on-chain exits need Arkade\'s live batch stream, which this private transport cannot carry.');} + assertBatchSwaps(){if(this.batchSwaps===false)throw new Error('Renewal and on-chain exits need Arkade\'s live batch stream, which this private transport cannot carry.');} private reserved(coin:{txid:string;vout:number},pending=this.pendingOutpoints?.()){const outpoint=`${coin.txid}:${coin.vout}`;return this.used.has(outpoint)||this.usedTxids.has(coin.txid)||pending?.has(outpoint)===true;} private async nativeSpend(inputs:readonly {txid:string;vout:number}[]|undefined,run:()=>Promise){ this.assertOpen(); diff --git a/tests/private-lightning-spends.test.ts b/tests/private-lightning-spends.test.ts index e7300df6..a06c7951 100644 --- a/tests/private-lightning-spends.test.ts +++ b/tests/private-lightning-spends.test.ts @@ -575,6 +575,16 @@ test('a wallet with batch swaps off refuses renewal and the on-chain exit before }finally{await off.dispose();await on.dispose();} }); +test('a wallet with batch swaps off refuses an on-chain send at quote time, whether a pool note or its Arkade coins would pay it',async()=>{ + const off=await open(new Store(),undefined,prover,undefined,{batchSwaps:false}),on=await open(new Store()),BTC='tb1qmt3ue2senlg6ddgmr76hwsk0rdvdk4rgeaen7l'; + try{ + for(const wallet of [off,on]){await fund(wallet,[30_000n]);wallet.getSpendableVtxos=(async()=>[{value:60_000}]) as never;} + for(const amount of [20_000,50_000])await assert.rejects(off.router().route({raw:BTC,amount}),/batch stream/,`${amount} sats`); + await assert.rejects(on.router().route({raw:BTC,amount:20_000}),/needs the recoverable companion wallet/,'a note covers it'); + assert.equal((await on.router().route({raw:BTC,amount:50_000})).railId,'onchain','the Arkade coins pay it'); + }finally{await off.dispose();await on.dispose();} +}); + test('a stale-anchor refusal still drops at once, on the pool verdict alone and without reading a batch',async()=>{ const store=new Store(),{wallet,pool}=await funded(store);const reason='The spend opens against another previous-epoch root than the next batch does; prove it again.';let batches=0; try{ diff --git a/tests/shielded-rails.test.ts b/tests/shielded-rails.test.ts index 92eac42a..1804deac 100644 --- a/tests/shielded-rails.test.ts +++ b/tests/shielded-rails.test.ts @@ -415,7 +415,7 @@ const YOUNG_FEE={intentFee:{offchainInput:"inputType == 'vtxo' && now() - birth const exitWallet=(o:{covers:boolean;arkade?:number[];privately?:boolean})=>{ const exits:{sats:number;title:string;dest:{to:string;sats:number}}[]=[],filters:unknown[]=[],settled:unknown[]=[],payout={txid:'cd'.repeat(32),vout:1,value:20_041,createdAt:new Date(),isSwept:false}; const view={getAddress:async()=>ELSEWHERE,settle:async(params:unknown)=>{settled.push(params);return 'ef'.repeat(32);}}; - const wallet={canWithdraw:()=>o.covers,canExitPrivately:()=>o.privately??true, + const wallet={canWithdraw:()=>o.covers,canExitPrivately:()=>o.privately??true,assertBatchSwaps:()=>{}, getSpendableVtxos:async(filter:unknown)=>{filters.push(filter);return (o.arkade??[]).map(value=>({value}));}, exitFromPool:(sats:number,title:string,exit:(coin:unknown,view:unknown)=>Promise,dest:{to:string;sats:number})=>{exits.push({sats,title,dest});return exit(payout,view);}} as unknown as ShieldedWallet; return {wallet,exits,filters,payout,view,settled}; From 08b2f49f10f21a2ba2cd81dbd92313257fd3be56 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 07:23:03 +0100 Subject: [PATCH 12/40] feat(app): boot the browser private transport behind an internal flag A build pins the gateway with SHIELDED_OHTTP_CONFIG and SHIELDED_NOSTR_GATEWAY (plus optional SHIELDED_NOSTR_RELAYS and SHIELDED_PUBLIC_ORIGIN), each an app-build ARG and a Vite define. A browser profile opts in with ?private=1, stored under shielded-private-transport-v1, and out with ?private=0. Only a pinned build with the flag on installs the transport. app/src/private-boot.ts is main.tsx's first import, because nostr-tools/pool captures WebSocket as it loads. Its module-level call sits behind a typeof check on the config define, so Node tests can import the module and an unpinned build constant-folds the call away; the install path then never reaches the bundle. If the install throws, #root shows why and the module rethrows, so main.tsx never renders and no half-private wallet opens. Reading localStorage itself can throw where site data is blocked; the boot reads it through calls that privateFlag already treats as off. bootPrivateTransport takes an optional last globals object, passed to installPrivateTransport, because the globals it defines are non-configurable: every boot test passes a plain object and the real globals stay untouched. RollupWallet hands PRIVATE.fetch to the wallet unwrapped (fetcher, claimFetch, the prover), so its private and sealed marks and its page brand still key the no-repeat-502 rule and the 12-batch page. In private mode it takes the transport's providers, so IsolatedOnchainProvider is used and the SDK never opens Esplora's /v1/ws; it reconciles and reads unresolved outcomes from the transport's journal; it passes poolPage 12 and batchSwaps:false; and it builds no Lightning companion. The startup watchdog stays unarmed too, since with no Lightning to start it would reload the page. The native host keeps precedence in every branch. A txid renders as plain in private mode, in the wallet and on /verify and /watch, since a link would be a direct explorer visit, and the /watch view-key read pages at the transport's size. The pool-origin define falls back with ||, as src/server.ts does for the same variable, so an empty build argument pins the default origin instead of '', which would fail the transport's https check. --- Dockerfile.dokploy | 6 ++++- app/src/Disclose.tsx | 10 ++++--- app/src/RollupWallet.tsx | 23 +++++++++++------ app/src/main.tsx | 2 ++ app/src/private-boot.ts | 30 +++++++++++++++++++++ app/src/vite-env.d.ts | 4 +++ app/vite.config.ts | 3 ++- tests/app-private-build.test.ts | 36 ++++++++++++++++++++++++++ tests/private-boot.test.ts | 46 +++++++++++++++++++++++++++++++++ 9 files changed, 146 insertions(+), 14 deletions(-) create mode 100644 app/src/private-boot.ts create mode 100644 tests/app-private-build.test.ts create mode 100644 tests/private-boot.test.ts diff --git a/Dockerfile.dokploy b/Dockerfile.dokploy index b2274f03..bd64d7d2 100644 --- a/Dockerfile.dokploy +++ b/Dockerfile.dokploy @@ -18,9 +18,13 @@ WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci COPY . . -# The app cache's escape hatches (docs/DOKPLOY.md, App cache); RUN only sees build arguments declared here. +# The app cache's escape hatches and the private transport's pins (docs/DOKPLOY.md: App cache, Nostr carrier); RUN only sees build arguments declared here. ARG SHIELDED_APP_CACHE_FORCE ARG SHIELDED_APP_CACHE_OFF +ARG SHIELDED_OHTTP_CONFIG +ARG SHIELDED_NOSTR_GATEWAY +ARG SHIELDED_NOSTR_RELAYS +ARG SHIELDED_PUBLIC_ORIGIN RUN npm run build FROM --platform=linux/amd64 golang:1.26.9-bookworm AS vm-build diff --git a/app/src/Disclose.tsx b/app/src/Disclose.tsx index 20f3b219..1ab00287 100644 --- a/app/src/Disclose.tsx +++ b/app/src/Disclose.tsx @@ -10,10 +10,12 @@ import type {Hash} from '../../packages/protocol/src/rollup/notes.ts'; import {rollupAddressOf} from '../../packages/protocol/src/rollup/wallet.ts'; import {epochMatchesPool,epochPoolOf} from '../../src/shielded/epoch-checkpoint.ts'; import {epochIncomingNotes} from '../../src/shielded/epoch-disclosure.ts'; +import {privateTransport} from '../../src/shielded/private-transport.ts'; import {epochSyncAccount,poolApiOf,rollupApi,type PoolApi,type RollupPoolStatus} from './rollup-client.ts'; const EXPLORER='https://explorer.mutinynet.arkade.sh'; const sats=(value:bigint|number)=>Number(value).toLocaleString('en-US'),short=(id:string)=>`${id.slice(0,8)}…${id.slice(-4)}`; +const TxLink=({txid}:{txid:string})=>privateTransport()?{txid}:{short(txid)} ↗; const when=(at?:number)=>at?new Date(at).toLocaleString(undefined,{dateStyle:'medium',timeStyle:'short'}):''; const unit=(asset:bigint)=>asset===0n?'sats':'asset units'; const totals=(items:{asset:bigint;amount:bigint}[])=>{const m=new Map();for(const i of items)m.set(i.asset,(m.get(i.asset)??0n)+i.amount);return [...m].map(([asset,amount])=>`${sats(amount)} ${unit(asset)}`).join(' + ');}; @@ -51,19 +53,19 @@ export function Results({checked,watched}:{checked?:Checked;watched?:Watched}){

{checked.ok.every(Boolean)?'Every note below is exactly the one the pool published at its position, owned by this address.':'At least one note differs from what the pool published, so this payment is not proven. A link minted against a different pool, such as an earlier genesis, cannot verify here.'}

Pool
{checked.pool?{checked.pool}:'Not reported'}
To
{checked.disclosure.to}
{checked.disclosure.notes.map((note,i)=>{const batch=Math.floor(note.index/32),record=checked.batches[batch]; - return
Note {i+1}
{checked.ok[i]?'✓':'✗'} {sats(note.amount)} {unit(note.asset)} · batch #{batch}{record?.at?` · ${when(record.at)}`:''}{record?.txid&&<> · {short(record.txid)} ↗}
;})} + return
Note {i+1}
{checked.ok[i]?'✓':'✗'} {sats(note.amount)} {unit(note.asset)} · batch #{batch}{record?.at?` · ${when(record.at)}`:''}{record?.txid&&<> · }
;})}
} {watched&&'entries' in watched&&

{watched.entries.length?'Notes and spends':'No activity yet'}

For {watched.address}

    {watched.entries.slice().reverse().map(e=>
  • {kinds[e.kind]}{e.amounts.length?totals(e.amounts):'No net change'} -
    Batch
    #{e.batch}{e.txid&&<> · {short(e.txid)} ↗}
    +
    Batch
    #{e.batch}{e.txid&&<> · }
    {e.created.map(n=>
    Note {n.epoch}/{n.index}
    {sats(n.amount)} {unit(n.asset)} · {watched.state(n)}
    )}
    • )}
} {watched&&'notes' in watched&&

{watched.notes.length?`${watched.notes.length} ${watched.notes.length===1?'note':'notes'} seen`:'No notes seen yet'}

For {watched.address}. A refreshed or recovered note appears again as a new note, so this list does not add up to a balance.

    {watched.notes.slice().reverse().map(note=>
  • {note.deposit?'Shielded':'Received'}+{sats(note.amount)} {unit(note.asset)} -
    Batch
    #{note.batch}{note.txid&&<> · {short(note.txid)} ↗}
    Position
    {note.index}
    • )}
+
Batch
#{note.batch}{note.txid&&<> · }
Position
{note.index}
)}
} ; } @@ -83,7 +85,7 @@ export default function Disclose(){ if(live())setWatched({address:rollupAddressOf({owner:key.owner,viewPublic:x25519.getPublicKey(key.viewSecret)}),entries:[...account.history],state:noteStateOf(account)}); }else if(watch){ const key=parseViewKey(value),all:RollupEpochRecord[]=[]; - for(;;){const page=await rollupApi<{total:number;batches:RollupEpochRecord[]}>(`/batches?from=${all.length}&limit=100`);all.push(...page.batches);if(all.length>=page.total||!page.batches.length)break;} + for(;;){const page=await rollupApi<{total:number;batches:RollupEpochRecord[]}>(`/batches?from=${all.length}&limit=${privateTransport()?.pageSize??100}`);all.push(...page.batches);if(all.length>=page.total||!page.batches.length)break;} const notes=await epochIncomingNotes(hash,all,key); if(live())setWatched({address:rollupAddressOf({owner:key.owner,viewPublic:x25519.getPublicKey(key.viewSecret)}),notes}); }else{const result=await checkDisclosure(hash,decodeDisclosure(value.includes('#')?value.slice(value.indexOf('#')+1):value));if(live())setChecked(result);} diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 4f8086cd..681cb923 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -37,8 +37,10 @@ import {WalletCreatedNotice,WalletSecurityMenu,type WalletSecurityView} from './ import {isNativeHost,nativeClaimFetch,nativeOperationFetch,nativeSessionFetch,nativeSwapConfig,nativeWalletProviders} from '../../src/native/index.ts'; import {loadWalletPoseidon} from './wallet-crypto.ts'; import {pendingPoolNullifiers,pendingPoolOutpoints,reconcileNativeMutations} from '../../src/native/mutations.ts'; +import {privateTransport} from '../../src/shielded/private-transport.ts'; const INJECTED_NATIVE=isNativeHost(); +const PRIVATE=privateTransport(); const POOL_ORIGIN=INJECTED_NATIVE?(globalThis as typeof globalThis&{__SHIELDED_NATIVE__:{poolOrigin:string}}).__SHIELDED_NATIVE__.poolOrigin:''; const BACKED_UP='shielded-rollup-wallet-backed-up-v1'; @@ -55,7 +57,7 @@ function Copyable({value,qr,label}:{value:string;qr?:string;label?:string}){ {qr!==undefined&&canShare&&} {shown&&qr!==undefined&&}; } -const TxLink=({txid,path='tx'}:{txid:string;path?:string})=>{short(txid)} ↗; +const TxLink=({txid,path='tx'}:{txid:string;path?:string})=>PRIVATE?{txid}:{short(txid)} ↗; // The resolver's kinds, mapped onto the swatches the history list already has. const SWATCH:Record={shield:'shield',unshield:'withdraw','onchain-exit':'withdraw','shielded-send':'send','shielded-receive':'receive','lightning-receive':'receive',merge:'merge',list:'arkade','claim-link':'send'}; const LIGHTNING_STATUS:Record={received:'Received; waiting to shield',shielding:'Received; shielding',shielded:'Shielded in the pool',retrying:'Received; shielding will retry','below-minimum':'Received outside the pool; below the shielding minimum','onchain-recovery':'Onchain recovery; shielding paused','onchain-recovered':'Recovered onchain'}; @@ -317,11 +319,12 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= journal=new IndexedDbLightningRepository();companion=undefined;lnurl=undefined; let created:ShieldedWallet|undefined; const privateFetch=nativeHost?nativeSessionFetch():undefined,lnurlFetch=nativeHost?nativeSessionFetch(nativeOperationFetch('lnurl',crypto.randomUUID(),lnurlPaymentScope.signal)):undefined,lnurlFetchFor=nativeHost?(signal:AbortSignal)=>nativeSessionFetch(nativeOperationFetch('lnurl',crypto.randomUUID(),signal)):undefined,claimFetch=nativeHost?nativeSessionFetch(nativeClaimFetch):undefined; - const providers=nativeHost?nativeWalletProviders(network.arkUrl,network.indexerUrl??network.arkUrl):{arkProvider:new RestArkProvider(network.arkUrl),indexerProvider:new RestIndexerProvider(network.indexerUrl??network.arkUrl)}; - const pairing=createLightningWallet(secret,{...providers,...(nativeHost?{lazyInitialization:true}:{})});pairing.catch(()=>{}); + const providers=nativeHost?nativeWalletProviders(network.arkUrl,network.indexerUrl??network.arkUrl):PRIVATE?PRIVATE.providers(network.arkUrl,network.indexerUrl??network.arkUrl):{arkProvider:new RestArkProvider(network.arkUrl),indexerProvider:new RestIndexerProvider(network.indexerUrl??network.arkUrl)}; + const pairing=PRIVATE&&!nativeHost?undefined:createLightningWallet(secret,{...providers,...(nativeHost?{lazyInitialization:true}:{})});pairing?.catch(()=>{}); try{ reportStartupStage('Connecting to the private wallet'); if(privateFetch)await reconcileNativeMutations(privateFetch,POOL_ORIGIN); + else if(PRIVATE)await PRIVATE.journal.reconcileNativeMutations(PRIVATE.fetch,PRIVATE.poolOrigin); reportStartupStage('Catching up with the private pool'); const bornAt=bornAtFor(storage,BORN,{token:status.pool!.token,batches:status.pool!.batches}); created=await ShieldedWallet.create({ @@ -329,8 +332,9 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= ...providers, storage:{walletRepository:new InMemoryWalletRepository(),contractRepository:new InMemoryContractRepository()}, settlementConfig:false,walletMode:'static', - pool:{origin:POOL_ORIGIN,status},secret,hash,prover:browserProver(()=>poolRef.current??status,privateFetch), - ...(nativeHost?{networkPrivacy:true,fetcher:privateFetch,lnurlFetch,claimFetch,pendingNullifiers:()=>new Set(pendingPoolNullifiers(POOL_ORIGIN)),pendingOutpoints:()=>new Set(pendingPoolOutpoints(POOL_ORIGIN))}:{}), + pool:{origin:POOL_ORIGIN,status},secret,hash,prover:browserProver(()=>poolRef.current??status,privateFetch??PRIVATE?.fetch), + ...(nativeHost?{networkPrivacy:true,fetcher:privateFetch,lnurlFetch,claimFetch,pendingNullifiers:()=>new Set(pendingPoolNullifiers(POOL_ORIGIN)),pendingOutpoints:()=>new Set(pendingPoolOutpoints(POOL_ORIGIN))} + :PRIVATE?{networkPrivacy:true,fetcher:PRIVATE.fetch,claimFetch:PRIVATE.fetch,poolPage:PRIVATE.pageSize,batchSwaps:false,pendingNullifiers:()=>new Set(PRIVATE.journal.pendingPoolNullifiers(PRIVATE.poolOrigin)),pendingOutpoints:()=>new Set(PRIVATE.journal.pendingPoolOutpoints(PRIVATE.poolOrigin))}:{}), shieldedRepository:cacheRef.current!.repository,bornAt,lightningSpendStore:journal, onArkAddress:ark=>{if(mounted.current&&!restoring.current)setEarly(e=>e?.secret===secret?{...e,ark}:e);}, }); @@ -386,6 +390,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= void activate(); service.start(error=>{if(current()&&receiveCurrent(service))lnFail(error);}); }; + if(pairing){ startupWatchdog.arm(); starting=startPageLightning({ pairing, @@ -398,9 +403,10 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= if(startingRef.current===starting)startingRef.current=undefined; if(!parts&&lnurlRef.current===lnurl)lnurlRef.current=undefined; }); + } return created; }catch(error){ - void pairing.then(c=>c.dispose()).catch(()=>{}); + void pairing?.then(c=>c.dispose()).catch(()=>{}); try{if(created)await dispose(created);else await journal[Symbol.asyncDispose]();} catch(cleanup){throw new Error('The previous wallet could not be closed. Reload before trying again.',{cause:cleanup});} throw error; @@ -459,8 +465,9 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= },[wallet]); useEffect(()=>{refreshRef.current?.request();},[wallet,tick]); useEffect(()=>{ - if(!nativeHost||!wallet)return; - try{setNativePending(pendingPoolNullifiers(POOL_ORIGIN).length>0||pendingPoolOutpoints(POOL_ORIGIN).length>0);} + if(!wallet||!nativeHost&&!PRIVATE)return; + const journal=nativeHost?{pendingPoolNullifiers,pendingPoolOutpoints}:PRIVATE!.journal,origin=nativeHost?POOL_ORIGIN:PRIVATE!.poolOrigin; + try{setNativePending(journal.pendingPoolNullifiers(origin).length>0||journal.pendingPoolOutpoints(origin).length>0);} catch(error){setPoolError(errorText(error));} },[wallet,tick,nativeHost]); // Local reads of the receive state: after every Lightning action, on a new receiver, and on each tick. diff --git a/app/src/main.tsx b/app/src/main.tsx index 13c57e4c..8e632a72 100644 --- a/app/src/main.tsx +++ b/app/src/main.tsx @@ -1,3 +1,5 @@ +// First, so the private transport's guards stand before any other module runs: nostr-tools/pool captures WebSocket as it loads. +import './private-boot'; import React,{lazy,Suspense} from 'react'; import {createRoot} from 'react-dom/client'; import {AppUpdatesProvider,NativeUpdateLink} from './AppUpdates.tsx'; diff --git a/app/src/private-boot.ts b/app/src/private-boot.ts new file mode 100644 index 00000000..fd1325c0 --- /dev/null +++ b/app/src/private-boot.ts @@ -0,0 +1,30 @@ +import {base64urlnopad} from '@scure/base'; +import {installPrivateTransport,type PrivateTransport} from '../../src/shielded/private-transport.ts'; + +export const PRIVATE_FLAG='shielded-private-transport-v1'; +type FlagStorage=Pick; + +export function privateFlag(search:string,storage:FlagStorage):boolean{ + try{ + const value=new URLSearchParams(search).get('private'); + if(value==='1'){storage.setItem(PRIVATE_FLAG,'1');return true;} + if(value==='0'){storage.removeItem(PRIVATE_FLAG);return false;} + return storage.getItem(PRIVATE_FLAG)==='1'; + }catch{return false;} +} + +export function bootPrivateTransport(env:{config:string;gateway:string;relays:string;poolOrigin:string},search:string,storage:FlagStorage,pageOrigin:string,globals?:Record):PrivateTransport|undefined{ + if(!env.config||!env.gateway||!privateFlag(search,storage))return undefined; + const relays=env.relays.split(',').map(relay=>relay.trim()).filter(Boolean); + return installPrivateTransport({config:base64urlnopad.decode(env.config),gateway:env.gateway,poolOrigin:env.poolOrigin,pageOrigin,storage,globals,...(relays.length?{relays}:{})}); +} + +if(typeof __SHIELDED_OHTTP_CONFIG__!=='undefined'&&__SHIELDED_OHTTP_CONFIG__&&__SHIELDED_NOSTR_GATEWAY__){ + // Reading localStorage throws where site data is blocked; through these calls that reads as off rather than as a failed boot. + const storage:FlagStorage={getItem:key=>localStorage.getItem(key),setItem:(key,value)=>localStorage.setItem(key,value),removeItem:key=>localStorage.removeItem(key)}; + try{bootPrivateTransport({config:__SHIELDED_OHTTP_CONFIG__,gateway:__SHIELDED_NOSTR_GATEWAY__,relays:__SHIELDED_NOSTR_RELAYS__,poolOrigin:__SHIELDED_POOL_ORIGIN__},location.search,storage,location.origin);} + catch(error){ + document.getElementById('root')?.replaceChildren(`Private networking could not start, so the wallet stays closed: ${String(error instanceof Error?error.message:error).replace(/\.$/,'')}. Open this page with ?private=0 to turn it off.`); + throw error; + } +} diff --git a/app/src/vite-env.d.ts b/app/src/vite-env.d.ts index 9a87f08f..917e878e 100644 --- a/app/src/vite-env.d.ts +++ b/app/src/vite-env.d.ts @@ -4,3 +4,7 @@ declare const __NATIVE_BUILD__:boolean; declare const __APP_COMMIT__:string; declare const __NATIVE_BUILD_NUMBER__:number; declare const __NATIVE_PLATFORM__:string; +declare const __SHIELDED_OHTTP_CONFIG__:string; +declare const __SHIELDED_NOSTR_GATEWAY__:string; +declare const __SHIELDED_NOSTR_RELAYS__:string; +declare const __SHIELDED_POOL_ORIGIN__:string; diff --git a/app/vite.config.ts b/app/vite.config.ts index 869a1b31..6f33e749 100644 --- a/app/vite.config.ts +++ b/app/vite.config.ts @@ -23,7 +23,8 @@ const appCache = (): Plugin => ({ export default defineConfig({ root: fileURLToPath(new URL('.', import.meta.url)), - define: { __APP_CACHE_OFF__: JSON.stringify(off || native), __NATIVE_BUILD__: JSON.stringify(native), __APP_COMMIT__: JSON.stringify(identity.commit), __NATIVE_BUILD_NUMBER__: JSON.stringify(identity.buildNumber), __NATIVE_PLATFORM__: JSON.stringify(identity.platform) }, + define: { __APP_CACHE_OFF__: JSON.stringify(off || native), __NATIVE_BUILD__: JSON.stringify(native), __APP_COMMIT__: JSON.stringify(identity.commit), __NATIVE_BUILD_NUMBER__: JSON.stringify(identity.buildNumber), __NATIVE_PLATFORM__: JSON.stringify(identity.platform), + __SHIELDED_OHTTP_CONFIG__: JSON.stringify(process.env.SHIELDED_OHTTP_CONFIG ?? ''), __SHIELDED_NOSTR_GATEWAY__: JSON.stringify(process.env.SHIELDED_NOSTR_GATEWAY ?? ''), __SHIELDED_NOSTR_RELAYS__: JSON.stringify(process.env.SHIELDED_NOSTR_RELAYS ?? ''), __SHIELDED_POOL_ORIGIN__: JSON.stringify(process.env.SHIELDED_PUBLIC_ORIGIN || 'https://shielded.mutinynet.arkade.sh') }, plugins: [react(), ...(native ? [] : [appCache()])], resolve: {alias: {circomlibjs:fileURLToPath(new URL('./src/circom-browser.ts',import.meta.url))}}, server: { host: '0.0.0.0', port: 5173, proxy: { '/api': process.env.SHIELDED_API_ORIGIN??'http://127.0.0.1:8792' } }, diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts new file mode 100644 index 00000000..f021ec04 --- /dev/null +++ b/tests/app-private-build.test.ts @@ -0,0 +1,36 @@ +import assert from 'node:assert/strict'; +import {readFileSync} from 'node:fs'; +import {test} from 'node:test'; + +const read=(path:string)=>readFileSync(path,'utf8'); +const FLAGS=['SHIELDED_OHTTP_CONFIG','SHIELDED_NOSTR_GATEWAY','SHIELDED_NOSTR_RELAYS','SHIELDED_PUBLIC_ORIGIN']; + +test('every private build flag is declared as an app-build ARG',()=>{ + const lines=read('Dockerfile.dokploy').split(/\r?\n/),start=lines.findIndex(line=>/ AS app-build$/.test(line)),end=lines.indexOf('RUN npm run build',start); + assert.ok(start>=0&&end>start,'the app-build stage runs npm run build'); + const stage=lines.slice(start,end); + for(const flag of FLAGS)assert.ok(stage.some(line=>new RegExp(`^ARG ${flag}(=|$)`).test(line)),flag); +}); + +test('every private define is declared for TypeScript',()=>{ + const block=/define:\s*\{([^}]*)\}/.exec(read('app/vite.config.ts'))?.[1]??'',declared=read('app/src/vite-env.d.ts'); + const names=[...block.matchAll(/(__SHIELDED_\w+__)\s*:/g)].map(m=>m[1]!).sort(); + assert.deepEqual(names,['__SHIELDED_NOSTR_GATEWAY__','__SHIELDED_NOSTR_RELAYS__','__SHIELDED_OHTTP_CONFIG__','__SHIELDED_POOL_ORIGIN__']); + for(const name of names)assert.match(declared,new RegExp(`^declare const ${name}:string;$`,'m'),name); +}); + +test('the transport is the first module main.tsx imports',()=>{ + assert.match(/^import\b.*$/m.exec(read('app/src/main.tsx'))?.[0]??'',/^import '\.\/private-boot(\.ts)?';/); +}); + +// Source-level: /watch and the wallet are read for their literals and branches, not rendered. +test('the pool-reading pages take their page size and their links from the transport',()=>{ + const disclose=read('app/src/Disclose.tsx'); + assert.ok(disclose.includes('privateTransport()?.pageSize')); + assert.doesNotMatch(disclose,/limit=100\b/); + for(const file of ['app/src/RollupWallet.tsx','app/src/Disclose.tsx']){ + const links=read(file).split(/\r?\n/).filter(line=>line.includes('${EXPLORER}')); + assert.ok(links.length,file); + for(const line of links)assert.match(line,/(PRIVATE|privateTransport\(\))\?(?!\.).*\$\{EXPLORER\}/,file); + } +}); diff --git a/tests/private-boot.test.ts b/tests/private-boot.test.ts new file mode 100644 index 00000000..1111af16 --- /dev/null +++ b/tests/private-boot.test.ts @@ -0,0 +1,46 @@ +import assert from 'node:assert/strict'; +import {test} from 'node:test'; +import {NOSTR_RELAYS} from '../packages/protocol/src/nostr-carrier.ts'; +import {generateOhttpKey} from '../src/rollup/ohttp-keys.ts'; +import {isPrivateFetch} from '../src/shielded/network-privacy.ts'; +import {privateTransport} from '../src/shielded/private-transport.ts'; +import {PRIVATE_FLAG,bootPrivateTransport,privateFlag} from '../app/src/private-boot.ts'; + +const PAGE='http://127.0.0.1:5173',POOL='https://shielded.mutinynet.arkade.sh'; +const memory=()=>{const items=new Map();return {items,getItem:(key:string)=>items.get(key)??null,setItem:(key:string,value:string)=>{items.set(key,value);},removeItem:(key:string)=>{items.delete(key);}};}; +const pinned=async()=>{const {config,nostrPublic}=await generateOhttpKey();return {config:Buffer.from(config).toString('base64url'),gateway:nostrPublic,relays:'',poolOrigin:POOL};}; + +test('?private=1 stores the flag, ?private=0 clears it, and nothing else reads it back',()=>{ + const s=memory(),blocked=()=>{throw new Error('blocked');}; + assert.equal(privateFlag('?private=1',s),true);assert.equal(s.items.get(PRIVATE_FLAG),'1'); + assert.equal(privateFlag('',s),true);assert.equal(privateFlag('?private=yes',s),true); + assert.equal(privateFlag('?private=0',s),false);assert.equal(s.items.has(PRIVATE_FLAG),false); + assert.equal(privateFlag('',s),false); + assert.equal(privateFlag('?private=1',{getItem:blocked,setItem:blocked,removeItem:blocked}),false); +}); + +test('a build with no pinned gateway has no private transport, whatever the flag says',async()=>{ + const env=await pinned(),s=memory(); + assert.equal(bootPrivateTransport({...env,config:''},'?private=1',s,PAGE,{}),undefined); + assert.equal(bootPrivateTransport({...env,gateway:''},'?private=1',s,PAGE,{}),undefined); + assert.equal(privateTransport(),undefined); +}); + +test('a pinned build with the flag on installs the transport, with the six default relays when none is named',async()=>{ + const env=await pinned(); + const boot=(relays:string)=>{ + const globals:Record={fetch:async()=>new Response('direct'),WebSocket:class {constructor(readonly url:string){}}}; + const transport=bootPrivateTransport({...env,relays},'?private=1',memory(),PAGE,globals)!; + const permits=(url:string)=>{try{new (globals.WebSocket as typeof WebSocket)(url);return true;}catch{return false;}}; + return {transport,globals,permits}; + }; + const defaults=boot(''); + try{ + assert.equal(defaults.transport.pageSize,12);assert.ok(isPrivateFetch(defaults.transport.fetch)); + assert.equal(defaults.globals.fetch,defaults.transport.fetch);assert.notEqual(globalThis.fetch,defaults.transport.fetch); + assert.deepEqual(['wss://a.example',...NOSTR_RELAYS].filter(defaults.permits),[...NOSTR_RELAYS]); + }finally{defaults.transport.close();} + const named=boot('wss://a.example, wss://b.example'); + try{assert.deepEqual(['wss://a.example','wss://b.example',...NOSTR_RELAYS].filter(named.permits),['wss://a.example','wss://b.example']);} + finally{named.transport.close();} +}); From 6f05b04787cacf604e0ab168fed4e5294d2b27ef Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 07:23:09 +0100 Subject: [PATCH 13/40] perf(shielded): let a build without the private transport drop its cipher suite The app entry now imports the private transport, so an unpinned build kept sealed-fetch.ts's top-level `new CipherSuite(...)` as a possible side effect, and with it all of hpke: the entry grew from 294.24 kB to 310.03 kB (93.98 to 98.69 kB gzip) for every visitor, though nothing reads the suite. Marking the construction pure lets the bundler drop it when createSealedFetch is unused: the unpinned entry is 294.44 kB (94.06 kB gzip). A pinned build reads SUITE and keeps it. --- src/shielded/sealed-fetch.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/shielded/sealed-fetch.ts b/src/shielded/sealed-fetch.ts index 410cefcd..9ca1ee07 100644 --- a/src/shielded/sealed-fetch.ts +++ b/src/shielded/sealed-fetch.ts @@ -5,7 +5,7 @@ import {KeyConfig,OHTTPClient} from 'ohttp-ts'; import {addressOf,NOSTR_ANSWER_KIND,NOSTR_CHUNK_BYTES,NOSTR_RELAYS,readAnswer,refusedByRelay,requestEvent,roundedSeconds} from '../../packages/protocol/src/nostr-carrier.ts'; import type {CarrierPool} from '../rollup/nostr-carrier.ts'; -const SUITE=new CipherSuite(KEM_DHKEM_X25519_HKDF_SHA256,KDF_HKDF_SHA256,AEAD_AES_128_GCM); +const SUITE=/*#__PURE__*/new CipherSuite(KEM_DHKEM_X25519_HKDF_SHA256,KDF_HKDF_SHA256,AEAD_AES_128_GCM); // nostr-tools fulfils "connection failure: …" for a relay it never reached, and otherwise with the relay's own OK text, which may be absent. It rejects an OK false with NIP-01's reason; a timeout or a dropped connection rejects though the event may be out, and "duplicate:" means the relay has it. const acked=(result:PromiseSettledResult)=>result.status==='fulfilled'&&!(typeof result.value==='string'&&result.value.startsWith('connection failure')); const accepted=(result:PromiseSettledResult)=>result.status==='fulfilled'?acked(result):!refusedByRelay(result.reason); From 3abfd55811c8ab57e3ee44e1ae8cc687adc99c5b Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 07:24:18 +0100 Subject: [PATCH 14/40] docs(dokploy): describe the private wallet build and its flag A subsection under Nostr carrier: the four app-build arguments and where ohttp:keygen prints the first two, that a build missing either has no private transport, that SHIELDED_PUBLIC_ORIGIN is read by the server and pinned by the build, how ?private=1 and ?private=0 turn the transport on and off for one browser profile and what a private wallet cannot do, and why the pinned key and relays must follow every gateway rotation. --- docs/DOKPLOY.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/DOKPLOY.md b/docs/DOKPLOY.md index eecb6452..6f24af7e 100644 --- a/docs/DOKPLOY.md +++ b/docs/DOKPLOY.md @@ -81,4 +81,21 @@ A wallet can also send its sealed request inside a Nostr event, so no relay of o **Logs.** The listener logs counts by kind, such as `[nostr] answered 12, busy 1, dropped 3`, at most once a minute, and never an event, a relay or a target. `answered` and `busy` are published; `status-capped` is a status event held back because 5 went out in that second; `refused` is a request the gateway would not open; `dropped` is an event that is not a valid request; `duplicate` is another copy of a request already seen; `republished` is a cached answer sent to a relay that delivered a copy late; `failed` is an error after the request was accepted, where nothing is published because the request may have run; `unpublished` is an answer event a relay refused, timed out or could not be reached for; `lost` is a relay whose subscription closed, or that could not be reached, which the listener tries again after 30 seconds; `deaf` is a relay that did not echo the canary within 15 seconds, which the listener closed (so it counts `lost` too) and tries again after 30 seconds; `canary-refused` is a canary a relay refused with a NIP-01 prefix, such as `rate-limited:`, which is not taken for deafness, so the relay stays subscribed, unchecked until a canary goes through. The gateway's own `[ohttp]` line counts `outbound`, a request shed because the carrier's outbound budget was spent. Relay NOTICE messages are not printed. The listener cannot silence one thing: when a relay sends a message the Nostr library cannot parse, or an event it cannot match (a kind 21470 event with no tags, say), the library prints its own warning to stderr, `[nostr] relay error processing message:` with the error, and with the event itself in the second case. For a frame it cannot parse, the error is V8's, which quotes up to 10 characters of the frame on either side of the fault. A relay that does this is misbehaving; take it out of `SHIELDED_NOSTR_RELAYS`. -**Limits.** Liveness: every 30 seconds the listener sends each relay a canary, a kind 21470 event from its own key with empty content, and expects it back within 15 seconds, so a relay that keeps the socket open but stops forwarding events is closed and subscribed again. The canary also keeps open a relay that drops idle connections (one dropped them after 60 seconds). A frame over 1 MiB closes the connection, a connection whose upgrade stalls for 10 seconds is dropped, and a relay with more than 4 MiB unsent gets no more events until it drains. Each relay message is handled on its own, so a copy of a request that arrives together with a copy the gateway sheds is opened afresh, and is answered or gets a status naming its own event. Restarting the service reopens every subscription. The status cap: under a flood, a request that was shed but not told (past the 5 a second) looks to the wallet like one that may have run. \ No newline at end of file +**Limits.** Liveness: every 30 seconds the listener sends each relay a canary, a kind 21470 event from its own key with empty content, and expects it back within 15 seconds, so a relay that keeps the socket open but stops forwarding events is closed and subscribed again. The canary also keeps open a relay that drops idle connections (one dropped them after 60 seconds). A frame over 1 MiB closes the connection, a connection whose upgrade stalls for 10 seconds is dropped, and a relay with more than 4 MiB unsent gets no more events until it drains. Each relay message is handled on its own, so a copy of a request that arrives together with a copy the gateway sheds is opened afresh, and is answered or gets a status naming its own event. Restarting the service reopens every subscription. The status cap: under a flood, a request that was shed but not told (past the 5 a second) looks to the wallet like one that may have run. + +#### The private wallet build + +The browser wallet can send its requests to the pool, arkd, the Arkade indexer and Esplora through this carrier instead of calling them directly; only the page itself and the proving files still load from the site. For now this is an internal flag, for fresh test wallets only. + +**Build arguments.** The app build takes four. Set them in **Environment → Build Time Arguments**, not in the runtime environment, and redeploy: + +- `SHIELDED_OHTTP_CONFIG`: the `config:` value `ohttp:keygen` printed. +- `SHIELDED_NOSTR_GATEWAY`: the `nostr:` value it printed. +- `SHIELDED_NOSTR_RELAYS`: the relays the wallet publishes to, separated by commas. Unset or blank uses the six defaults under **Relays**. +- `SHIELDED_PUBLIC_ORIGIN`: the pool's origin. Unset or blank uses `https://shielded.mutinynet.arkade.sh`. + +A build missing either of the first two has no private transport, whatever a browser asks for. The server reads `SHIELDED_PUBLIC_ORIGIN` at runtime and the app build reads it too: the build pins it as the origin every sealed pool request names, so set the same value in both places. + +**Turn it on.** On a pinned build, `?private=1` on any page turns the transport on for that browser profile and `?private=0` turns it off. Use it only with a fresh test wallet. A private wallet has no Lightning, no renewal, no on-chain exit and no explorer links; a transaction id shows as plain text. If the transport cannot start, the page says why and the wallet stays closed; `?private=0` opens it again without the transport. + +**Keep the pins equal to the gateway's.** The key config and relays a build pins must match the gateway's `SHIELDED_OHTTP_KEY` and `SHIELDED_NOSTR_RELAYS`. After a rotation, a pinned build seals to the old key, and over Nostr the gateway answers that with silence, not a stale-key error, so its wallets stop syncing without saying why. Rebuild the app with the new config at every rotation, and keep the gateway's relay list equal to the build's or larger. \ No newline at end of file From 69fb5cc84d761a022dc64e382df102afb1dc89a5 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 07:30:04 +0100 Subject: [PATCH 15/40] fix(app): let no link turn the private transport off ?private=0 removed the stored flag, so anyone could send a private tester a link that silently returned their wallet to direct requests, exposing their IP; now only ?private=1 is read, and turning it off means deleting the shielded-private-transport-v1 key. --- app/src/private-boot.ts | 6 ++---- docs/DOKPLOY.md | 4 +++- tests/private-boot.test.ts | 8 ++++---- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/app/src/private-boot.ts b/app/src/private-boot.ts index fd1325c0..b5fd0b3c 100644 --- a/app/src/private-boot.ts +++ b/app/src/private-boot.ts @@ -6,9 +6,7 @@ type FlagStorage=Pick; export function privateFlag(search:string,storage:FlagStorage):boolean{ try{ - const value=new URLSearchParams(search).get('private'); - if(value==='1'){storage.setItem(PRIVATE_FLAG,'1');return true;} - if(value==='0'){storage.removeItem(PRIVATE_FLAG);return false;} + if(new URLSearchParams(search).get('private')==='1'){storage.setItem(PRIVATE_FLAG,'1');return true;} return storage.getItem(PRIVATE_FLAG)==='1'; }catch{return false;} } @@ -24,7 +22,7 @@ if(typeof __SHIELDED_OHTTP_CONFIG__!=='undefined'&&__SHIELDED_OHTTP_CONFIG__&&__ const storage:FlagStorage={getItem:key=>localStorage.getItem(key),setItem:(key,value)=>localStorage.setItem(key,value),removeItem:key=>localStorage.removeItem(key)}; try{bootPrivateTransport({config:__SHIELDED_OHTTP_CONFIG__,gateway:__SHIELDED_NOSTR_GATEWAY__,relays:__SHIELDED_NOSTR_RELAYS__,poolOrigin:__SHIELDED_POOL_ORIGIN__},location.search,storage,location.origin);} catch(error){ - document.getElementById('root')?.replaceChildren(`Private networking could not start, so the wallet stays closed: ${String(error instanceof Error?error.message:error).replace(/\.$/,'')}. Open this page with ?private=0 to turn it off.`); + document.getElementById('root')?.replaceChildren(`Private networking could not start, so the wallet stays closed: ${String(error instanceof Error?error.message:error).replace(/\.$/,'')}. To turn private networking off, remove ${PRIVATE_FLAG} from this site's local storage.`); throw error; } } diff --git a/docs/DOKPLOY.md b/docs/DOKPLOY.md index 6f24af7e..6c864069 100644 --- a/docs/DOKPLOY.md +++ b/docs/DOKPLOY.md @@ -96,6 +96,8 @@ The browser wallet can send its requests to the pool, arkd, the Arkade indexer a A build missing either of the first two has no private transport, whatever a browser asks for. The server reads `SHIELDED_PUBLIC_ORIGIN` at runtime and the app build reads it too: the build pins it as the origin every sealed pool request names, so set the same value in both places. -**Turn it on.** On a pinned build, `?private=1` on any page turns the transport on for that browser profile and `?private=0` turns it off. Use it only with a fresh test wallet. A private wallet has no Lightning, no renewal, no on-chain exit and no explorer links; a transaction id shows as plain text. If the transport cannot start, the page says why and the wallet stays closed; `?private=0` opens it again without the transport. +**Turn it on.** On a pinned build, `?private=1` on any page turns the transport on for that browser profile. Use it only with a fresh test wallet. A private wallet has no Lightning, no renewal, no on-chain exit and no explorer links; a transaction id shows as plain text. If the transport cannot start, the page says why and the wallet stays closed. + +**Turn it off.** No link turns it off, so nobody can send a tester a link that quietly returns their wallet to direct requests; `?private=0` does nothing. Delete the `localStorage` key `shielded-private-transport-v1` in the browser's developer tools, or clear the site's data. Clearing the site's data deletes the wallet as well, so back up its recovery phrase first. **Keep the pins equal to the gateway's.** The key config and relays a build pins must match the gateway's `SHIELDED_OHTTP_KEY` and `SHIELDED_NOSTR_RELAYS`. After a rotation, a pinned build seals to the old key, and over Nostr the gateway answers that with silence, not a stale-key error, so its wallets stop syncing without saying why. Rebuild the app with the new config at every rotation, and keep the gateway's relay list equal to the build's or larger. \ No newline at end of file diff --git a/tests/private-boot.test.ts b/tests/private-boot.test.ts index 1111af16..5e6a8e73 100644 --- a/tests/private-boot.test.ts +++ b/tests/private-boot.test.ts @@ -10,12 +10,12 @@ const PAGE='http://127.0.0.1:5173',POOL='https://shielded.mutinynet.arkade.sh'; const memory=()=>{const items=new Map();return {items,getItem:(key:string)=>items.get(key)??null,setItem:(key:string,value:string)=>{items.set(key,value);},removeItem:(key:string)=>{items.delete(key);}};}; const pinned=async()=>{const {config,nostrPublic}=await generateOhttpKey();return {config:Buffer.from(config).toString('base64url'),gateway:nostrPublic,relays:'',poolOrigin:POOL};}; -test('?private=1 stores the flag, ?private=0 clears it, and nothing else reads it back',()=>{ +test('?private=1 stores the flag, and no other value of private clears it',()=>{ const s=memory(),blocked=()=>{throw new Error('blocked');}; - assert.equal(privateFlag('?private=1',s),true);assert.equal(s.items.get(PRIVATE_FLAG),'1'); - assert.equal(privateFlag('',s),true);assert.equal(privateFlag('?private=yes',s),true); assert.equal(privateFlag('?private=0',s),false);assert.equal(s.items.has(PRIVATE_FLAG),false); - assert.equal(privateFlag('',s),false); + assert.equal(privateFlag('?private=1',s),true);assert.equal(s.items.get(PRIVATE_FLAG),'1'); + for(const search of ['','?private=0','?private=yes'])assert.equal(privateFlag(search,s),true,search); + assert.equal(s.items.get(PRIVATE_FLAG),'1'); assert.equal(privateFlag('?private=1',{getItem:blocked,setItem:blocked,removeItem:blocked}),false); }); From cd271831cc9de4f01b0f30201bcb212f78bf0baf Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 08:56:14 +0100 Subject: [PATCH 16/40] fix(app): keep the wallet closed when an unpinned build sees the private flag An unpinned build ignored ?private=1 and a stored shielded-private-transport-v1, so a tester on a build whose pins went missing opened a wallet that sent every request direct. Such a build now shows the closed-wallet message with the reason "this build has no private gateway pinned" and throws, so main.tsx never renders, and it does not store a flag it read from the URL. A native build takes neither branch, whatever pins or flag exist: the host's Tor transport is its private networking. ?private=1 is now read before storage: where storage throws, the URL still turns private mode on for that load (a pinned build installs the transport) and only the persisted flag is lost. A stored flag that cannot be read is off. The top-level branch moves into bootPrivate(native, install, search, storage), which the tests drive. The install closure comes from a ternary on the defines at the call site rather than from a function taking them: Rolldown does not fold constants through a call, and with the defines passed as arguments the unpinned entry kept the transport and HPKE (376.46 kB). With the ternary the unpinned entry is 295.20 kB (94.41 kB gzip) against 294.44 kB (94.06 kB) at f6c2c67, the difference being the closed-wallet branch, and it holds none of HPKE-v1, message/ohttp-req, DHKEM or x-date. A pinned entry is 376.56 kB. --- app/src/private-boot.ts | 23 ++++++++++++++++------- tests/private-boot.test.ts | 28 +++++++++++++++++++++++++--- 2 files changed, 41 insertions(+), 10 deletions(-) diff --git a/app/src/private-boot.ts b/app/src/private-boot.ts index b5fd0b3c..cf8c139d 100644 --- a/app/src/private-boot.ts +++ b/app/src/private-boot.ts @@ -4,11 +4,10 @@ import {installPrivateTransport,type PrivateTransport} from '../../src/shielded/ export const PRIVATE_FLAG='shielded-private-transport-v1'; type FlagStorage=Pick; -export function privateFlag(search:string,storage:FlagStorage):boolean{ - try{ - if(new URLSearchParams(search).get('private')==='1'){storage.setItem(PRIVATE_FLAG,'1');return true;} - return storage.getItem(PRIVATE_FLAG)==='1'; - }catch{return false;} +/** The URL is read before storage, so storage that throws loses only the stored flag. */ +export function privateFlag(search:string,storage:FlagStorage,store=true):boolean{ + if(new URLSearchParams(search).get('private')==='1'){if(store)try{storage.setItem(PRIVATE_FLAG,'1');}catch{}return true;} + try{return storage.getItem(PRIVATE_FLAG)==='1';}catch{return false;} } export function bootPrivateTransport(env:{config:string;gateway:string;relays:string;poolOrigin:string},search:string,storage:FlagStorage,pageOrigin:string,globals?:Record):PrivateTransport|undefined{ @@ -17,10 +16,20 @@ export function bootPrivateTransport(env:{config:string;gateway:string;relays:st return installPrivateTransport({config:base64urlnopad.decode(env.config),gateway:env.gateway,poolOrigin:env.poolOrigin,pageOrigin,storage,globals,...(relays.length?{relays}:{})}); } -if(typeof __SHIELDED_OHTTP_CONFIG__!=='undefined'&&__SHIELDED_OHTTP_CONFIG__&&__SHIELDED_NOSTR_GATEWAY__){ +/** A native build has its own transport. Without `install`, which only a pinned build makes, the flag keeps the wallet closed: opening would send every request direct. */ +export function bootPrivate(native:boolean,install:(()=>PrivateTransport|undefined)|undefined,search:string,storage:FlagStorage):PrivateTransport|undefined{ + if(native)return undefined; + if(install)return install(); + if(privateFlag(search,storage,false))throw new Error('this build has no private gateway pinned'); + return undefined; +} + +if(typeof __NATIVE_BUILD__!=='undefined'){ // Reading localStorage throws where site data is blocked; through these calls that reads as off rather than as a failed boot. const storage:FlagStorage={getItem:key=>localStorage.getItem(key),setItem:(key,value)=>localStorage.setItem(key,value),removeItem:key=>localStorage.removeItem(key)}; - try{bootPrivateTransport({config:__SHIELDED_OHTTP_CONFIG__,gateway:__SHIELDED_NOSTR_GATEWAY__,relays:__SHIELDED_NOSTR_RELAYS__,poolOrigin:__SHIELDED_POOL_ORIGIN__},location.search,storage,location.origin);} + // A ternary on the defines, not a call taking them: only constant folding drops the transport and its cipher suite from an unpinned build. + const install=__SHIELDED_OHTTP_CONFIG__&&__SHIELDED_NOSTR_GATEWAY__?()=>bootPrivateTransport({config:__SHIELDED_OHTTP_CONFIG__,gateway:__SHIELDED_NOSTR_GATEWAY__,relays:__SHIELDED_NOSTR_RELAYS__,poolOrigin:__SHIELDED_POOL_ORIGIN__},location.search,storage,location.origin):undefined; + try{bootPrivate(__NATIVE_BUILD__,install,location.search,storage);} catch(error){ document.getElementById('root')?.replaceChildren(`Private networking could not start, so the wallet stays closed: ${String(error instanceof Error?error.message:error).replace(/\.$/,'')}. To turn private networking off, remove ${PRIVATE_FLAG} from this site's local storage.`); throw error; diff --git a/tests/private-boot.test.ts b/tests/private-boot.test.ts index 5e6a8e73..a1136c7c 100644 --- a/tests/private-boot.test.ts +++ b/tests/private-boot.test.ts @@ -4,19 +4,27 @@ import {NOSTR_RELAYS} from '../packages/protocol/src/nostr-carrier.ts'; import {generateOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {isPrivateFetch} from '../src/shielded/network-privacy.ts'; import {privateTransport} from '../src/shielded/private-transport.ts'; -import {PRIVATE_FLAG,bootPrivateTransport,privateFlag} from '../app/src/private-boot.ts'; +import {PRIVATE_FLAG,bootPrivate,bootPrivateTransport,privateFlag} from '../app/src/private-boot.ts'; const PAGE='http://127.0.0.1:5173',POOL='https://shielded.mutinynet.arkade.sh'; const memory=()=>{const items=new Map();return {items,getItem:(key:string)=>items.get(key)??null,setItem:(key:string,value:string)=>{items.set(key,value);},removeItem:(key:string)=>{items.delete(key);}};}; const pinned=async()=>{const {config,nostrPublic}=await generateOhttpKey();return {config:Buffer.from(config).toString('base64url'),gateway:nostrPublic,relays:'',poolOrigin:POOL};}; +const blocked=()=>{throw new Error('blocked');},throwing={getItem:blocked,setItem:blocked,removeItem:blocked}; +const flagged=()=>{const s=memory();s.setItem(PRIVATE_FLAG,'1');return s;}; test('?private=1 stores the flag, and no other value of private clears it',()=>{ - const s=memory(),blocked=()=>{throw new Error('blocked');}; + const s=memory(); assert.equal(privateFlag('?private=0',s),false);assert.equal(s.items.has(PRIVATE_FLAG),false); assert.equal(privateFlag('?private=1',s),true);assert.equal(s.items.get(PRIVATE_FLAG),'1'); for(const search of ['','?private=0','?private=yes'])assert.equal(privateFlag(search,s),true,search); assert.equal(s.items.get(PRIVATE_FLAG),'1'); - assert.equal(privateFlag('?private=1',{getItem:blocked,setItem:blocked,removeItem:blocked}),false); +}); + +test('?private=1 turns private mode on for the load where storage throws, and a flag storage cannot read is off',async()=>{ + assert.equal(privateFlag('?private=1',throwing),true);assert.equal(privateFlag('',throwing),false); + const env=await pinned(),globals:Record={fetch:async()=>new Response('direct'),WebSocket:class {}}; + const transport=bootPrivate(false,()=>bootPrivateTransport(env,'?private=1',throwing,PAGE,globals),'?private=1',throwing); + try{assert.ok(transport);assert.equal(globals.fetch,transport.fetch);}finally{transport?.close();} }); test('a build with no pinned gateway has no private transport, whatever the flag says',async()=>{ @@ -26,6 +34,20 @@ test('a build with no pinned gateway has no private transport, whatever the flag assert.equal(privateTransport(),undefined); }); +test('an unpinned build that sees the flag stays closed, and stores no flag from the URL',()=>{ + const fresh=memory(),closed=/^Error: this build has no private gateway pinned$/; + assert.throws(()=>bootPrivate(false,undefined,'?private=1',fresh),closed);assert.equal(fresh.items.has(PRIVATE_FLAG),false); + assert.throws(()=>bootPrivate(false,undefined,'',flagged()),closed); + assert.throws(()=>bootPrivate(false,undefined,'?private=1',throwing),closed); + for(const storage of [memory(),throwing])assert.equal(bootPrivate(false,undefined,'?private=0',storage),undefined); +}); + +test('a native build neither installs the browser transport nor stays closed, whatever its pins and flag',()=>{ + let installs=0; + for(const install of [()=>{installs++;return undefined;},undefined])assert.equal(bootPrivate(true,install,'?private=1',flagged()),undefined); + assert.equal(installs,0); +}); + test('a pinned build with the flag on installs the transport, with the six default relays when none is named',async()=>{ const env=await pinned(); const boot=(relays:string)=>{ From 8fc3c6a4d1785b323f4f5663ed70121fa9e17f8d Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 08:57:03 +0100 Subject: [PATCH 17/40] fix(shielded): end the browser's 404 release at 45 minutes, not the pool's hour The 404 release ran while the entry's age, on the wallet clock, was under the hour the pool tracks a spend on its own clock. The pool stamps a spend after the wallet wrote createdAt, so a constant offset between the two clocks cancels; what it does not cover is a clock that steps between the POST and the GET. The upper bound is now TRACKED_MS - graceMs, 45 minutes with the browser's 15-minute grace, so the margin on that side equals the grace on the other. Age 2 699 999 ms releases, 2 700 000 ms keeps the entry for Plan 4. The native journal, built without graceMs, still never releases on a 404. --- src/native/mutations.ts | 2 +- tests/tor-mutations.test.ts | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/src/native/mutations.ts b/src/native/mutations.ts index 1c849f2c..f79e83dc 100644 --- a/src/native/mutations.ts +++ b/src/native/mutations.ts @@ -75,7 +75,7 @@ export function makeNativeMutationJournal(storage:JournalStorage,{graceMs,now=() if(response.body===undefined)return; const method=response.method.toUpperCase(),{origin,path}=route(response.url),spend=method==='GET'?/^\/api\/rollup\/spends\/([0-9a-f]{32})$/.exec(path)?.[1]:undefined; // The pool forgets a spend an hour after admitting it, so only a 404 inside that hour proves it never arrived; its body may be empty. - if(spend&&response.status===404){if(graceMs!==undefined){const at=now();drop(row=>row.origin===origin&&row.id===spend&&at-row.createdAt>graceMs&&at-row.createdAtrow.origin===origin&&row.id===spend&&at-row.createdAt>graceMs&&at-row.createdAt{ assert.deepEqual(journal.pendingPoolNullifiers(origin),[17n]); }); +test('with the browser grace, a 404 releases until 45 minutes after the POST and keeps the entry from then on',async()=>{ + for(const [age,left] of [[2_699_999,[]],[2_700_000,[17n]]] as const){ + let at=1_000_000;const journal=makeNativeMutationJournal(storage(),{graceMs:PRIVATE_GRACE_MS,now:()=>at}); + journal.beforeNativeMutation(mutation()); + at+=age;await lost(journal,UNKNOWN); + assert.deepEqual(journal.pendingPoolNullifiers(origin),left,String(age)); + } +}); + test('a journal built without a grace never releases on a 404',async t=>{ let at=1_000_000;t.mock.method(Date,'now',()=>at); const journal=makeNativeMutationJournal(storage());journal.beforeNativeMutation(mutation()); From 5410468d3294cc49aa8c3ee5db286eda8cf9f132 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 08:58:16 +0100 Subject: [PATCH 18/40] fix(app): release a lost private spend on a wallet tick, without reopening A browser journal entry whose POST ended outcome-unknown was reconciled only when the wallet opened, so its notes stayed reserved, and the unresolved-outcomes line stayed up, until the user reopened the wallet, even once a 404 past the 15-minute grace proved the spend lost. The transport gains reconcile(): while its journal holds an entry for the pool origin it runs reconcileNativeMutations through the sealed fetch, then says whether any entry remains. It is single-flight, so a call made while one runs gets the same promise. The wallet page calls it on every tick in private mode and sets the unresolved-outcomes line from its answer; the open path calls the same method, so a tick and an open never reconcile at once. The native host keeps its synchronous read on a tick and its reconcile at open, unchanged. --- app/src/RollupWallet.tsx | 6 +++--- src/shielded/private-transport.ts | 8 +++++++- tests/private-transport.test.ts | 11 +++++++++++ 3 files changed, 21 insertions(+), 4 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 681cb923..99b9e883 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -324,7 +324,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= try{ reportStartupStage('Connecting to the private wallet'); if(privateFetch)await reconcileNativeMutations(privateFetch,POOL_ORIGIN); - else if(PRIVATE)await PRIVATE.journal.reconcileNativeMutations(PRIVATE.fetch,PRIVATE.poolOrigin); + else if(PRIVATE)await PRIVATE.reconcile(); reportStartupStage('Catching up with the private pool'); const bornAt=bornAtFor(storage,BORN,{token:status.pool!.token,batches:status.pool!.batches}); created=await ShieldedWallet.create({ @@ -466,8 +466,8 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= useEffect(()=>{refreshRef.current?.request();},[wallet,tick]); useEffect(()=>{ if(!wallet||!nativeHost&&!PRIVATE)return; - const journal=nativeHost?{pendingPoolNullifiers,pendingPoolOutpoints}:PRIVATE!.journal,origin=nativeHost?POOL_ORIGIN:PRIVATE!.poolOrigin; - try{setNativePending(journal.pendingPoolNullifiers(origin).length>0||journal.pendingPoolOutpoints(origin).length>0);} + if(!nativeHost){let live=true;void PRIVATE!.reconcile().then(pending=>{if(live)setNativePending(pending);},error=>{if(live)setPoolError(errorText(error));});return()=>{live=false;};} + try{setNativePending(pendingPoolNullifiers(POOL_ORIGIN).length>0||pendingPoolOutpoints(POOL_ORIGIN).length>0);} catch(error){setPoolError(errorText(error));} },[wallet,tick,nativeHost]); // Local reads of the receive state: after every Lightning action, on a new receiver, and on each tick. diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index 6f9a3fae..21bbb596 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -12,7 +12,7 @@ import {createSealedFetch,SealedFetchError} from './sealed-fetch.ts'; export const PRIVATE_PAGE=12; export const PRIVATE_GRACE_MS=900_000; export interface PrivateTransportConfig {config:Uint8Array;gateway:string;poolOrigin:string;pageOrigin:string;relays?:readonly string[];storage?:Pick;socket?:typeof WebSocket;globals?:Record} -export interface PrivateTransport {fetch:typeof fetch;poolOrigin:string;pageSize:number;journal:ReturnType;providers(arkUrl:string,indexerUrl:string):{arkProvider:RestArkProvider;indexerProvider:IsolatedIndexerProvider;onchainProvider:(network:NetworkName)=>OnchainProvider};close():void} +export interface PrivateTransport {fetch:typeof fetch;poolOrigin:string;pageSize:number;journal:ReturnType;reconcile():Promise;providers(arkUrl:string,indexerUrl:string):{arkProvider:RestArkProvider;indexerProvider:IsolatedIndexerProvider;onchainProvider:(network:NetworkName)=>OnchainProvider};close():void} let installed:PrivateTransport|undefined; const refused=(what:string)=>class {constructor(){throw new Error('Private networking does not permit '+what);}}; @@ -45,7 +45,13 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr if(journalled)journal.observeNativeResponse({url:href,method,status:response.status,body:await response.clone().text(),requestBody:body}); return response; })),PRIVATE_PAGE); + const pending=()=>journal.pendingPoolNullifiers(poolOrigin).length>0||journal.pendingPoolOutpoints(poolOrigin).length>0; + let reconciling:Promise|undefined; const transport:PrivateTransport={fetch:privateFetch,poolOrigin,pageSize:PRIVATE_PAGE,journal, + reconcile(){ + reconciling??=(async()=>{if(pending())await journal.reconcileNativeMutations(privateFetch,poolOrigin);return pending();})().finally(()=>{reconciling=undefined;}); + return reconciling; + }, providers(arkUrl,indexerUrl){ policy.registerArk(arkUrl,indexerUrl); return {arkProvider:new RestArkProvider(arkUrl),indexerProvider:new IsolatedIndexerProvider(indexerUrl,undefined,undefined,undefined,0),onchainProvider:network=>{const url=ESPLORA_URL[network];policy.registerEsplora(url);return new IsolatedOnchainProvider(url);}}; diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index 6e44de30..f4c114a3 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -156,6 +156,17 @@ test('a sealed 404 frees a journalled spend only once the 15-minute grace has pa } }); +test('a pending entry whose GET answers 404 clears at 900 001 ms of age through reconcile, which never runs twice at once',async t=>{ + const g=await rig(t,{reply:url=>url.endsWith('/spends')?Response.json({id:ID},{status:202}):Response.json({error:'Unknown spend; it may have expired or the server restarted.'},{status:404})}); + const start=Date.now();let at=start;t.mock.method(Date,'now',()=>at); + assert.equal((await g.transport.fetch('/api/rollup/spends',spendPost(ID,'17'))).status,202); + at=start+899_999;assert.equal(await g.transport.reconcile(),true); + at=start+900_001;const first=g.transport.reconcile();assert.equal(g.transport.reconcile(),first); + assert.deepEqual([await first,g.transport.journal.pendingPoolNullifiers(POOL)],[false,[]]); + assert.equal(await g.transport.reconcile(),false); + assert.equal(g.calls.filter(url=>url.endsWith('/spends/'+ID)).length,2,'one read per reconcile that found the entry'); +}); + test('a spend POST that never went out frees its reservation at once, and one that may have run keeps it',async t=>{ const g=await rig(t),fails=(code:string)=>(error:unknown)=>error instanceof SealedFetchError&&error.code===code; await assert.rejects(g.transport.fetch('/api/rollup/spends',spendPost(ID,'17','0'.repeat(NOSTR_CHUNK_BYTES))),fails('too-large')); From 2a0da992071b72f3e8d9de6c27aef2dc6ac2be8f Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 08:59:57 +0100 Subject: [PATCH 19/40] fix(shielded): send a GET of the native release list direct in private mode The download list (GET /api/native/releases on the page origin) was refused in private mode, so a private profile's download section came up empty. It now goes direct, like the proving files: it is a public list carrying no wallet data, and the page origin already sees the visit. Only a GET of exactly that path does, matching the server's handler, which answers nothing else; a POST or HEAD to it, and /api/other, are still refused. The route takes the request method for this, read as the Request constructor will normalize it. The transport tests gain one with no injected socket: the page's WebSocket is the sentinel, and every relay socket must be built by that constructor, as captured before the guard replaced it, not by the guard (whose new.target would show). Making the pool build its sockets through the page's current WebSocket fails it. The existing test, which injects a socket, is renamed to say so. --- src/shielded/private-transport.ts | 9 +++++---- tests/private-transport.test.ts | 24 ++++++++++++++++-------- 2 files changed, 21 insertions(+), 12 deletions(-) diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index 21bbb596..934ca841 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -3,6 +3,7 @@ import {AbstractSimplePool} from 'nostr-tools/abstract-pool'; import {verifyEvent} from 'nostr-tools/pure'; import {normalizeURL} from 'nostr-tools/utils'; import {NOSTR_RELAYS} from '../../packages/protocol/src/nostr-carrier.ts'; +import {NATIVE_RELEASE_ENDPOINT} from '../native-release.ts'; import {journalledRoute,makeNativeMutationJournal} from '../native/mutations.ts'; import {NativePolicy} from '../native/policy.ts'; import {IsolatedIndexerProvider,IsolatedOnchainProvider} from '../native/providers.ts'; @@ -24,15 +25,15 @@ export function installPrivateTransport(config:PrivateTransportConfig):PrivateTr const journal=makeNativeMutationJournal(config.storage??globalThis.localStorage,{graceMs:PRIVATE_GRACE_MS}); const sealedFetch=createSealedFetch({config:config.config,gateway:config.gateway,origin:poolOrigin,relays,pool:new AbstractSimplePool({verifyEvent,maxWaitForConnection:3000,websocketImplementation:Original})}); const Guarded=class extends Original {constructor(url:string|URL,protocols?:string|string[]){const u=normalizeURL(String(url));if(!pinned.has(u))throw new Error('Private networking does not permit a WebSocket to '+url);super(u,protocols);}}; - // Express routes ignore case, so these checks do. Order matters: a pool API path is sealed at the pinned origin even when the page is served from it, and a proving path on the page origin goes direct even when it is not. - const route=(url:URL)=>{ + // Express routes ignore case, so these checks do; the release list matches exactly, as its handler does. Order matters: a pool API path is sealed at the pinned origin even when the page is served from it, and a proving path on the page origin goes direct even when it is not. + const route=(url:URL,method:string)=>{ const path=url.pathname,lower=path.toLowerCase(),proving=/^\/api\/rollup\/proving\//.test(lower),poolApi=/^\/api\/rollup\//.test(lower)&&!proving; - if(url.origin===pageOrigin){if(proving||!lower.startsWith('/api/'))return undefined;if(poolApi)return poolOrigin+path+url.search;} + if(url.origin===pageOrigin){if(proving||!lower.startsWith('/api/')||method==='GET'&&path===NATIVE_RELEASE_ENDPOINT)return undefined;if(poolApi)return poolOrigin+path+url.search;} else if(url.origin!==poolOrigin||poolApi)return url.href; throw new Error('Private networking does not permit '+url.origin+path); }; const privateFetch:typeof fetch=markPrivatePoolPage(markSealedFetch(markPrivateFetch(async(input,init)=>{ - const url=new URL(input instanceof Request?input.url:String(input),pageOrigin),href=route(url); + const url=new URL(input instanceof Request?input.url:String(input),pageOrigin),href=route(url,(init?.method??(input instanceof Request?input.method:'GET')).toUpperCase()); if(href===undefined)return direct(input instanceof Request?input:url.href,init); const source=new Request(input instanceof Request?input:url,init),method=source.method,body=method==='GET'?undefined:await source.text(); policy.classify(href,method,body); diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index f4c114a3..3c62f3da 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -18,7 +18,7 @@ const POOL='https://pool.example',PAGE='http://127.0.0.1:5173',LOOPBACK='http:// const settle=()=>new Promise(resolve=>setImmediate(resolve)); const memory=()=>{const items=new Map();return {getItem:(key:string)=>items.get(key)??null,setItem:(key:string,value:string)=>{items.set(key,value);},removeItem:(key:string)=>{items.delete(key);}};}; -async function rig(t:TestContext,{pageOrigin=POOL,globals={},reply=()=>Response.json({ok:true})}:{pageOrigin?:string;globals?:Record;reply?:(url:string)=>Response}={}){ +async function rig(t:TestContext,{pageOrigin=POOL,globals={},reply=()=>Response.json({ok:true}),inject=true}:{pageOrigin?:string;globals?:Record;reply?:(url:string)=>Response;inject?:boolean}={}){ // Node's own WebSocket reaches the six public relays, so a pool that lost its websocketImplementation fails here instead. const real=globalThis.WebSocket;globalThis.WebSocket=class {constructor(){throw new Error('a test reached for a public relay');}} as unknown as typeof WebSocket;t.after(()=>{globalThis.WebSocket=real;}); const key=(await loadOhttpKey((await generateOhttpKey()).env))!,bus=createNostrBus(),calls:string[]=[],direct:string[]=[],made:unknown[]=[]; @@ -39,8 +39,8 @@ async function rig(t:TestContext,{pageOrigin=POOL,globals={},reply=()=>Response. } close(){this.readyState=3;for(const sub of this.subs.values())sub.close();this.subs.clear();} } - const target:Record={fetch:async(input:string)=>{direct.push(String(input));return new Response('direct');},...globals}; - const transport=installPrivateTransport({config:key.config,gateway:key.nostrPublic,poolOrigin:POOL,pageOrigin,storage:memory(),socket:Sentinel as unknown as typeof WebSocket,globals:target});t.after(()=>transport.close()); + const target:Record={fetch:async(input:string)=>{direct.push(String(input));return new Response('direct');},...(inject?{}:{WebSocket:Sentinel}),...globals}; + const transport=installPrivateTransport({config:key.config,gateway:key.nostrPublic,poolOrigin:POOL,pageOrigin,storage:memory(),...(inject?{socket:Sentinel as unknown as typeof WebSocket}:{}),globals:target});t.after(()=>transport.close()); return {transport,globals:target,bus,calls,direct,made,Sentinel,requests:()=>bus.published.filter(event=>event.kind===NOSTR_REQUEST_KIND)}; } @@ -57,12 +57,13 @@ test('a pool path is sealed even when the page origin is the pool origin',async assert.deepEqual([g.calls,g.direct],[[LOOPBACK+'/api/rollup/status'],[]]); }); -test('the page assets and the proving files go direct, and nothing else under /api does',async t=>{ - const g=await rig(t),paths=['/','/assets/app.js','/api/rollup/proving/epoch-spend.wasm?v='+'c'.repeat(64)]; +test('the page assets, the proving files and a GET of the release list go direct, and nothing else under /api does',async t=>{ + const g=await rig(t),paths=['/','/assets/app.js','/api/rollup/proving/epoch-spend.wasm?v='+'c'.repeat(64),'/api/native/releases']; for(const path of paths)assert.equal(await (await g.transport.fetch(path)).text(),'direct',path); assert.deepEqual(g.direct,paths.map(path=>POOL+path)); - for(const path of ['/api/native/releases','/api/other'])await assert.rejects(g.transport.fetch(path),/does not permit/,path); - assert.deepEqual([g.requests().length,g.calls.length],[0,0]); + await assert.rejects(g.transport.fetch('/api/other'),/does not permit/); + for(const method of ['POST','HEAD'])await assert.rejects(g.transport.fetch('/api/native/releases',{method}),/does not permit/,method); + assert.deepEqual([g.requests().length,g.calls.length,g.direct.length],[0,0,paths.length]); }); test('on a page origin other than the pool, the page assets and the proving files still go direct',async t=>{ @@ -120,12 +121,19 @@ test('EventSource, XMLHttpRequest and WebRTC are refused, and WebSocket only rea for(const url of ['wss://nostr.arkade.sh','ws://relay.snort.social'])assert.throws(()=>new G.WebSocket!(url),/does not permit/,url); }); -test('the sealed fetch is given the original WebSocket, not the guarded one',async t=>{ +test('an injected socket builds every relay socket, not the page WebSocket nor the guarded one',async t=>{ const g=await rig(t,{globals:{WebSocket:class {constructor(){throw new Error('the page WebSocket was used');}}}}); assert.equal((await g.transport.fetch('/api/rollup/status')).status,200); assert.ok(g.made.length&&g.made.every(made=>made===g.Sentinel),'every relay socket was built by the original constructor itself'); }); +test('with no socket injected, the sealed fetch gets the page WebSocket captured before the guard replaced it',async t=>{ + const g=await rig(t,{inject:false}); + assert.notEqual(g.globals.WebSocket,g.Sentinel); + assert.equal((await g.transport.fetch('/api/rollup/status')).status,200); + assert.ok(g.made.length&&g.made.every(made=>made===g.Sentinel),'a socket the guard built would carry its own new.target'); +}); + test('providers carry no event source and an onchain provider that polls',async t=>{ const g=await rig(t),p=g.transport.providers(ARK,ARK),onchain=p.onchainProvider('mutinynet'); assert.deepEqual([Reflect.get(p.arkProvider,'eventSource'),Reflect.get(p.indexerProvider,'eventSource')],[undefined,undefined]); From d8b307d24b0621f0f5a2ebae916899e680fe6cae Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:01:30 +0100 Subject: [PATCH 20/40] fix(shielded): give up on a sealed read that came back unavailable rollupApi retried a GET whose sealed fetch threw unavailable four more times, though each try had already made three relay attempts of up to 9 s: a pool read that no relay answered held its caller for minutes before failing. It now rethrows at once, as it does for a sealed 502. A private fetcher that is not sealed (the native host's Tor fetchers) keeps its five tries, and the test covers both. The 502 comment said "the 60 s tick reads again", which holds only for a wallet, not for / or /watch; it now says only why one try is enough, for both cases. A native test pins that the real nativeFetch is private but not sealed, which the rate-limit tests' unsealed stand-in assumes. --- src/shielded/pool.ts | 4 ++-- tests/native-egress.test.ts | 6 ++++++ tests/pool-rate-limit.test.ts | 14 ++++++++++++++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 7d6cce6c..32173107 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -42,7 +42,7 @@ export async function rollupApi(path:string,body?:unknown,fetcher:typeof fetc response=await fetcher(origin+'/api/rollup'+path,read?{...request,signal:read.signal}:request); // An unreadable GET is read again; an admitted POST is not resent (the pool refuses a duplicate id), its caller polls it by id. if(response.status<500)json=await response.json().catch((error:unknown)=>{if(response!.ok&&read)throw error;return {};}); - }catch(error){response=undefined;if(privateMutation||attempt>=4)throw new PoolRequestError(error instanceof Error?error.message:'The pool did not answer (network error).',undefined,{cause:error});} + }catch(error){response=undefined;if(privateMutation||attempt>=4||isSealedFetch(fetcher)&&rejected(error,'unavailable'))throw new PoolRequestError(error instanceof Error?error.message:'The pool did not answer (network error).',undefined,{cause:error});} finally{clearTimeout(timer);} if(response&&response.status<500){ const metadata=json&&typeof json==='object'?json:{}; @@ -56,7 +56,7 @@ export async function rollupApi(path:string,body?:unknown,fetcher:typeof fetc if(!response.ok)throw Object.assign(new PoolRequestError(metadata.error??`The pool answered ${response.status}.`,response.status),typeof metadata.code==='string'?{code:metadata.code}:{}); return json as T; } - // A sealed 502 is the gateway's answer to an oversize page and to a dead pool alike; a repeat only repeats the first, so one try, and the 60 s tick reads again. + // A sealed 502 (an oversize page or a dead pool) and a sealed unavailable (its three relay attempts spent, caught above) would only repeat, so each gets one try. // Only the browser's sealed fetch: a private native fetch is not the gateway, and keeps its retries. if(privateMutation||attempt>=4||response?.status===502&&isSealedFetch(fetcher))throw new PoolRequestError(`The pool did not answer (${response?.status??'network error'}).`,response?.status); await sleep(pauseMs); diff --git a/tests/native-egress.test.ts b/tests/native-egress.test.ts index 32bc87b6..ee6be3c8 100644 --- a/tests/native-egress.test.ts +++ b/tests/native-egress.test.ts @@ -1,5 +1,6 @@ import assert from 'node:assert/strict'; import {test} from 'node:test'; +import {isPrivateFetch,isSealedFetch} from '../src/shielded/network-privacy.ts'; test('native renderer disables WebRTC constructors before an ICE connection can start',async()=>{ let direct=0; @@ -14,3 +15,8 @@ test('native renderer disables WebRTC constructors before an ICE connection can } assert.equal(direct,0); }); + +test('the native fetch is private but not sealed, so it keeps the retries a sealed fetch gives up',async()=>{ + const {nativeFetch}=await import('../src/native/index.ts'); + assert.deepEqual([isPrivateFetch(nativeFetch),isSealedFetch(nativeFetch)],[true,false]); +}); diff --git a/tests/pool-rate-limit.test.ts b/tests/pool-rate-limit.test.ts index ab91ea44..dad93801 100644 --- a/tests/pool-rate-limit.test.ts +++ b/tests/pool-rate-limit.test.ts @@ -3,6 +3,7 @@ import {setImmediate} from 'node:timers/promises'; import {test} from 'node:test'; import {poolApiOf,rollupApi,waitForSpend} from '../src/shielded/pool.ts'; import {markPrivateFetch,markSealedFetch} from '../src/shielded/network-privacy.ts'; +import {SealedFetchError} from '../src/shielded/sealed-fetch.ts'; const flush=async()=>{for(let i=0;i<8;i++)await setImmediate();}; const outcome=(promise:Promise)=>promise.then(value=>({value,error:undefined}),error=>({value:undefined,error})); @@ -200,6 +201,19 @@ test('a private read that is not sealed still repeats a 502, as does a direct on } }); +test('a sealed read gives up on unavailable at once, and a private read that is not sealed still asks five times',async t=>{ + for(const [label,sealed,calls] of [['sealed',true,1],['private',false,5]] as const){ + t.mock.timers.enable({apis:['Date','setTimeout'],now:1_000_000}); + let count=0,settled=false; + const raw=(async()=>{count++;throw new SealedFetchError('unavailable','No relay brought an answer in time.');}) as typeof fetch; + const pending=outcome(rollupApi('/batches?from=0&limit=12',undefined,sealed?markSealedFetch(markPrivateFetch(raw)):markPrivateFetch(raw),1000,'https://sealed-unavailable.test')).then(result=>{settled=true;return result;}); + await flush();assert.deepEqual([count,settled],[1,sealed],label); + for(let i=1;i{ t.mock.timers.enable({apis:['Date','setTimeout'],now:1_000_000}); const polls:number[]=[],signed:number[]=[]; From 38d7c37d3080f2f1a9855f2b2bef12ceca965d33 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:03:41 +0100 Subject: [PATCH 21/40] fix(shielded): let a wallet with batch swaps off still send to itself offchain Ruling 8. sendSelectedVtxosToSelf is an offchain Arkade transaction, which the signer migration uses; it needs no batch stream, so the batchSwaps:false guard on it would have blocked a private wallet's migration for nothing. The guard goes from that method only; settle (renewal), exitFromPool and the on-chain quote still refuse. The method still refuses coins reserved for shielding. --- src/shielded/wallet.ts | 2 +- tests/private-lightning-spends.test.ts | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/shielded/wallet.ts b/src/shielded/wallet.ts index e205a844..fff7b985 100644 --- a/src/shielded/wallet.ts +++ b/src/shielded/wallet.ts @@ -669,7 +669,7 @@ export class ShieldedWallet extends Wallet { return this.nativeSpend('recipients' in args[0]?args[0].selectedVtxos:undefined,()=>super.send(...args)); } override async settle(...args:Parameters){this.assertBatchSwaps();return this.nativeSpend(args[0]?.inputs,()=>super.settle(...args));} - override async sendSelectedVtxosToSelf(...args:Parameters){this.assertBatchSwaps();return this.nativeSpend(args[0],()=>super.sendSelectedVtxosToSelf(...args));} + override async sendSelectedVtxosToSelf(...args:Parameters){return this.nativeSpend(args[0],()=>super.sendSelectedVtxosToSelf(...args));} override async getContractManager(){ this.assertOpen(); const manager=await super.getContractManager(); diff --git a/tests/private-lightning-spends.test.ts b/tests/private-lightning-spends.test.ts index a06c7951..f06ad912 100644 --- a/tests/private-lightning-spends.test.ts +++ b/tests/private-lightning-spends.test.ts @@ -561,13 +561,14 @@ test('a fee refusal on a Lightning shield reads the fee again, so the next attem }finally{await wallet.dispose();} }); -test('a wallet with batch swaps off refuses renewal and the on-chain exit before any intent is registered',async t=>{ - const settle=t.mock.method(Wallet.prototype,'settle',async()=>'renewed');t.mock.method(Wallet.prototype,'send',async()=>'sent'); +test('a wallet with batch swaps off refuses renewal and the on-chain exit before any intent is registered, and still sends to itself offchain',async t=>{ + const settle=t.mock.method(Wallet.prototype,'settle',async()=>'renewed');t.mock.method(Wallet.prototype,'send',async()=>'sent');t.mock.method(Wallet.prototype,'sendSelectedVtxosToSelf',async()=>'migrated'); const near=async()=>[{txid:TXID,vout:0,value:1000,expiresAt:Date.now()+SHIELD_FLOOR_MS-60_000}]; const off=await open(new Store(),undefined,prover,undefined,{batchSwaps:false}),on=await open(new Store()); try{ off.coins=near;on.coins=near; - for(const swap of [()=>off.renew(),()=>off.settle(),()=>off.sendSelectedVtxosToSelf([])])await assert.rejects(swap(),/batch stream/); + for(const swap of [()=>off.renew(),()=>off.settle()])await assert.rejects(swap(),/batch stream/); + assert.equal(await off.sendSelectedVtxosToSelf([]),'migrated','signer migration sends offchain, with no batch'); assert.throws(()=>off.exitFromPool(1000,'Sending 1,000 sats on-chain',async()=>{throw new Error('the exit ran');},{to:'tb1qexit',sats:1000}),/batch stream/,'before the pool pays a fresh key'); assert.equal(settle.mock.callCount(),0); assert.equal(await off.send({recipients:[{address:destination,amount:330}]}),'sent'); From 6b797630f511d36cd63e06ede59672674c21c964 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:04:32 +0100 Subject: [PATCH 22/40] test: import the carrier's ANSWER_BYTES instead of copying it tor-content's 12-batch page check copied 63*4096-8-16_396, and two carrier tests wrote out 241 644, so a change to the cap would have left them checking the old number. ANSWER_BYTES is now exported from src/rollup/nostr-carrier.ts and all three import it; the 8-event fit test still checks that the value itself is right. --- src/rollup/nostr-carrier.ts | 3 ++- tests/nostr-carrier.test.ts | 6 +++--- tests/tor-content.test.ts | 3 ++- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/src/rollup/nostr-carrier.ts b/src/rollup/nostr-carrier.ts index e6d7534d..88617e80 100644 --- a/src/rollup/nostr-carrier.ts +++ b/src/rollup/nostr-carrier.ts @@ -14,7 +14,8 @@ export type RelayPool={ensureRelay(url:string):Promise;destroy():void}; // The most body whose sealed answer fits 8 chunks: 63 Padmé steps of 4 KiB (a 64th plus 32 bytes of nonce and tag passes 256 KiB), less 8 bytes of BHTTP framing and 16 396 for the copied headers at their largest, as Node's fetch refuses 16 KiB of header names and values. // ponytail: a flat 30 s retry, so a relay that keeps refusing us is asked every 30 s; back off if an operator minds. -const ANSWER_BYTES=63*4096-8-16_396,REMEMBERED=10_000,STATUSES_PER_SECOND=5,RESUBSCRIBE_MS=30_000,PUBLICATIONS_PER_SECOND=64,REPUBLISH_MS=30_000,MAX_RELAYS=12,CANARY_MS=30_000,ECHO_MS=15_000; +export const ANSWER_BYTES=63*4096-8-16_396; +const REMEMBERED=10_000,STATUSES_PER_SECOND=5,RESUBSCRIBE_MS=30_000,PUBLICATIONS_PER_SECOND=64,REPUBLISH_MS=30_000,MAX_RELAYS=12,CANARY_MS=30_000,ECHO_MS=15_000; // ponytail: ws's handshakeTimeout only fails an upgrade that stalls, so a relay trickling its reply holds the socket open; add a hard deadline if one does. export class CappedSocket extends WebSocket { diff --git a/tests/nostr-carrier.test.ts b/tests/nostr-carrier.test.ts index a87f924f..3be67f3e 100644 --- a/tests/nostr-carrier.test.ts +++ b/tests/nostr-carrier.test.ts @@ -9,7 +9,7 @@ import {KeyConfig,OHTTPClient} from 'ohttp-ts'; import {addressOf,NOSTR_ANSWER_KIND,NOSTR_RELAYS,NOSTR_REQUEST_KIND,readAnswer,requestEvent} from '../packages/protocol/src/nostr-carrier.ts'; import {createOhttpGateway,type OhttpOpen} from '../src/rollup/ohttp-gateway.ts'; import {generateOhttpKey,loadOhttpKey,type OhttpGatewayKey} from '../src/rollup/ohttp-keys.ts'; -import {nostrRelaysFromEnv,startNostrCarrier,type RelayPool} from '../src/rollup/nostr-carrier.ts'; +import {ANSWER_BYTES,nostrRelaysFromEnv,startNostrCarrier,type RelayPool} from '../src/rollup/nostr-carrier.ts'; import {createNostrBus} from './fixtures/nostr-bus.ts'; const SUITE=new CipherSuite(KEM_DHKEM_X25519_HKDF_SHA256,KDF_HKDF_SHA256,AEAD_AES_128_GCM); @@ -316,7 +316,7 @@ test('close() closes every subscription and destroys the pool, and a late event }); test('an upstream body at the cap fits 8 answer events beside the largest headers fetch passes, and one byte more is a sealed 502',async t=>{ - const cap=241_644,headers={'content-type':'a'.repeat(16_219),'cache-control':'b'.repeat(64),'retry-after':'c'.repeat(64)}; + const cap=ANSWER_BYTES,headers={'content-type':'a'.repeat(16_219),'cache-control':'b'.repeat(64),'retry-after':'c'.repeat(64)}; for(const [size,status] of [[cap,200],[cap+1,502]] as const){ const g=await rig(t,{reply:()=>new Response(new Uint8Array(size),{headers})}),sent=await g.ask(ARK);await until(()=>sent.answer()?.kind==='sealed'); const inner=await sent.inner(); @@ -332,7 +332,7 @@ test('open() is told the answer cap unless the caller sets one',async t=>{ const carrier=startNostrCarrier({open,secret,relays:[A],pool:bus.pool,...(maxAnswer?{maxAnswer}:{})}),before=seen.length;t.after(()=>carrier.close());await settle(); await Promise.all(bus.pool.publish([A],requestEvent(randomBytes(40),gateway,Date.now())));await until(()=>seen.length>before);carrier.close(); } - assert.deepEqual(seen,[241_644,1234]); + assert.deepEqual(seen,[ANSWER_BYTES,1234]); }); test('an answer for a relay that cannot be reached is counted unpublished, and the other relay still gets it',async t=>{ diff --git a/tests/tor-content.test.ts b/tests/tor-content.test.ts index b186354d..04eadda8 100644 --- a/tests/tor-content.test.ts +++ b/tests/tor-content.test.ts @@ -14,6 +14,7 @@ import {lightningRails} from '../src/shielded/lightning.ts'; import {waitForSpend,type PoolApi} from '../src/shielded/pool.ts'; import {markPrivateFetch,markPrivatePoolPage,privatePoolPage,isPrivatePoolApi} from '../src/shielded/network-privacy.ts'; import {PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; +import {ANSWER_BYTES} from '../src/rollup/nostr-carrier.ts'; import {epochLeafPool} from './fixtures/epoch-pool.ts'; import {makeNativeMutationJournal} from '../src/native/mutations.ts'; import {decodeClaimToken,encodeClaimToken} from '../packages/protocol/src/rollup/bearer.ts'; @@ -97,7 +98,7 @@ test('a re-wrapped private pool api keeps its page size',async t=>{ test('a worst-case page of 12 batches stays inside the gateway answer cap',()=>{ const f='9'.repeat(String(ROLLUP_FIELD).length),slot:RollupEpochRecordSlot={anchor:f,rCur:{epoch:99_999,root:f},nullifiers:[f,f],commitments:[f,f],ctDigest:f,groupId:f,groupSize:10,publics:Array(7).fill(f),ciphertext:'f'.repeat(2*ROLLUP_RECORD_BYTES)}; const record:RollupEpochRecord={kind:'spend',slots:Array(BATCH_SLOTS).fill(slot),noteBlockRoot:f,txid:'f'.repeat(64),at:1_700_000_000_000}; - const page=(n:number)=>JSON.stringify({from:0,total:n,batches:Array(n).fill(record)}).length,cap=63*4096-8-16_396; + const page=(n:number)=>JSON.stringify({from:0,total:n,batches:Array(n).fill(record)}).length,cap=ANSWER_BYTES; assert.ok(page(PRIVATE_PAGE)<=cap,String(page(PRIVATE_PAGE)));assert.ok(page(16)>cap,String(page(16))); }); From 9746ee72722435d58342b656865f2a80b0d09a3a Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:09:53 +0100 Subject: [PATCH 23/40] fix(shielded): cap a private wallet's shield at 20 coins, soonest to expire first A shield of several plain coins is one offchain send to self, and its /v1/tx/submit body carries the signed ark transaction plus one user-signed checkpoint per input, so it grows by about 1.4 kB a coin. Through the sealed transport a body over one Nostr event (32 768 B sealed) is refused before it goes out, so a private wallet holding many coins could never shield them. shieldPlan and nextShieldPlan take an optional coin cap, unlimited by default, so direct and native plans are unchanged. Past the cap, plain coins go soonest to expire first, coins without an expiry last, and the fee logic then runs as before. ShieldedWallet takes maxShieldCoins and planFor passes it; the private wallet passes PRIVATE_SHIELD_COINS = 20, exported beside PRIVATE_PAGE. Any coins over the cap go on the next pass. Measured through the private transport's own fetch and sealing, with SDK offchain transactions over DefaultVtxo coins built from the Mutinynet arkd fixture (checkpoint tapscript, 2048 s exit delay, x-digest): 20 coins seal to 28 727 B for submit (27 884 B body) and 19 511 B for finalize (18 561 B body); 22 coins seal to 31 799 B, and 23 are refused as too large. A test seals both bodies at the cap and fails at 23. --- app/src/RollupWallet.tsx | 4 ++-- src/shielded/pool.ts | 5 +++-- src/shielded/private-transport.ts | 1 + src/shielded/wallet.ts | 11 ++++++----- tests/private-transport.test.ts | 22 ++++++++++++++++++++-- tests/tor-content.test.ts | 14 +++++++++++--- 6 files changed, 43 insertions(+), 14 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 99b9e883..7b14c605 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -37,7 +37,7 @@ import {WalletCreatedNotice,WalletSecurityMenu,type WalletSecurityView} from './ import {isNativeHost,nativeClaimFetch,nativeOperationFetch,nativeSessionFetch,nativeSwapConfig,nativeWalletProviders} from '../../src/native/index.ts'; import {loadWalletPoseidon} from './wallet-crypto.ts'; import {pendingPoolNullifiers,pendingPoolOutpoints,reconcileNativeMutations} from '../../src/native/mutations.ts'; -import {privateTransport} from '../../src/shielded/private-transport.ts'; +import {privateTransport,PRIVATE_SHIELD_COINS} from '../../src/shielded/private-transport.ts'; const INJECTED_NATIVE=isNativeHost(); const PRIVATE=privateTransport(); @@ -334,7 +334,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= settlementConfig:false,walletMode:'static', pool:{origin:POOL_ORIGIN,status},secret,hash,prover:browserProver(()=>poolRef.current??status,privateFetch??PRIVATE?.fetch), ...(nativeHost?{networkPrivacy:true,fetcher:privateFetch,lnurlFetch,claimFetch,pendingNullifiers:()=>new Set(pendingPoolNullifiers(POOL_ORIGIN)),pendingOutpoints:()=>new Set(pendingPoolOutpoints(POOL_ORIGIN))} - :PRIVATE?{networkPrivacy:true,fetcher:PRIVATE.fetch,claimFetch:PRIVATE.fetch,poolPage:PRIVATE.pageSize,batchSwaps:false,pendingNullifiers:()=>new Set(PRIVATE.journal.pendingPoolNullifiers(PRIVATE.poolOrigin)),pendingOutpoints:()=>new Set(PRIVATE.journal.pendingPoolOutpoints(PRIVATE.poolOrigin))}:{}), + :PRIVATE?{networkPrivacy:true,fetcher:PRIVATE.fetch,claimFetch:PRIVATE.fetch,poolPage:PRIVATE.pageSize,batchSwaps:false,maxShieldCoins:PRIVATE_SHIELD_COINS,pendingNullifiers:()=>new Set(PRIVATE.journal.pendingPoolNullifiers(PRIVATE.poolOrigin)),pendingOutpoints:()=>new Set(PRIVATE.journal.pendingPoolOutpoints(PRIVATE.poolOrigin))}:{}), shieldedRepository:cacheRef.current!.repository,bornAt,lightningSpendStore:journal, onArkAddress:ark=>{if(mounted.current&&!restoring.current)setEarly(e=>e?.secret===secret?{...e,ark}:e);}, }); diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 32173107..30f29aae 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -142,10 +142,11 @@ export const shieldableWithFee=(plan:ShieldPlan,fee:DepositFeeSchedule|undefined const carrier=BigInt(plan.coin.value),free=BigInt(CARRIER_FREE_SATS); return carrier<=free||carrier-owed>=free; }; -/** What auto-shield moves next: every plain coin as one deposit, else one coin of a listed asset. The pool refuses coins near expiry, and a deposit the fee would eat is left out. While a fee charges, pool payouts (`exempt`) go one at a time: a merged coin is a plain send and would pay. */ -export function shieldPlan(coins:readonly ArkCoin[],listed:ReadonlySet,now:number,floorMs:number,fee:{schedule?:DepositFeeSchedule;exempt?:(coin:ArkCoin)=>boolean}={}):ShieldPlan|undefined { +/** What auto-shield moves next: every plain coin as one deposit, else one coin of a listed asset. The pool refuses coins near expiry, and a deposit the fee would eat is left out. While a fee charges, pool payouts (`exempt`) go one at a time: a merged coin is a plain send and would pay. Past `maxCoins` plain coins, the soonest to expire go first. */ +export function shieldPlan(coins:readonly ArkCoin[],listed:ReadonlySet,now:number,floorMs:number,fee:{schedule?:DepositFeeSchedule;exempt?:(coin:ArkCoin)=>boolean}={},maxCoins=Infinity):ShieldPlan|undefined { const {schedule,exempt}=fee,usable=coins.filter(c=>c.expiresAt===undefined||c.expiresAt-now>floorMs); let plain=usable.filter(c=>!c.assets?.length); + if(plain.length>maxCoins)plain=plain.sort((a,b)=>(a.expiresAt??Number.MAX_SAFE_INTEGER)-(b.expiresAt??Number.MAX_SAFE_INTEGER)).slice(0,maxCoins); if(exempt&&plain.length>1&&(schedule?.fixedSats||schedule?.bps)&&plain.every(exempt))plain=[plain.reduce((a,b)=>b.value>a.value?b:a)]; const btc:ShieldPlan={kind:'btc',coins:plain,amount:plain.reduce((sum,c)=>sum+c.value,0)}; if(shieldableWithFee(btc,schedule,exempt))return btc; diff --git a/src/shielded/private-transport.ts b/src/shielded/private-transport.ts index 934ca841..94125f06 100644 --- a/src/shielded/private-transport.ts +++ b/src/shielded/private-transport.ts @@ -12,6 +12,7 @@ import {createSealedFetch,SealedFetchError} from './sealed-fetch.ts'; export const PRIVATE_PAGE=12; export const PRIVATE_GRACE_MS=900_000; +export const PRIVATE_SHIELD_COINS=20; export interface PrivateTransportConfig {config:Uint8Array;gateway:string;poolOrigin:string;pageOrigin:string;relays?:readonly string[];storage?:Pick;socket?:typeof WebSocket;globals?:Record} export interface PrivateTransport {fetch:typeof fetch;poolOrigin:string;pageSize:number;journal:ReturnType;reconcile():Promise;providers(arkUrl:string,indexerUrl:string):{arkProvider:RestArkProvider;indexerProvider:IsolatedIndexerProvider;onchainProvider:(network:NetworkName)=>OnchainProvider};close():void} diff --git a/src/shielded/wallet.ts b/src/shielded/wallet.ts index fff7b985..36ee5d69 100644 --- a/src/shielded/wallet.ts +++ b/src/shielded/wallet.ts @@ -32,7 +32,7 @@ import {isPrivatePoolApi,markPrivatePoolApi,markPrivatePoolPage,privatePoolPage} export interface ShieldedPoolConfig {origin:string;status:RollupPoolStatus} export interface ShieldedProver {prove(built:EpochBuiltSpend,circuit:'spend'|'join'):Promise;joinReady?():boolean} -export type ShieldedWalletConfig=WalletConfig&{pool:ShieldedPoolConfig;secret:string;hash:Hash;prover:ShieldedProver;shieldedRepository?:ShieldedNoteRepository;lightningSpendStore?:LightningSpendStore;bornAt?:number;maxDepositFee?:DepositFeeSchedule;fetcher?:typeof fetch;networkPrivacy?:boolean;poolPage?:number;batchSwaps?:boolean;pendingNullifiers?:()=>ReadonlySet;pendingOutpoints?:()=>ReadonlySet;lnurlFetch?:typeof fetch;claimFetch?:typeof fetch; +export type ShieldedWalletConfig=WalletConfig&{pool:ShieldedPoolConfig;secret:string;hash:Hash;prover:ShieldedProver;shieldedRepository?:ShieldedNoteRepository;lightningSpendStore?:LightningSpendStore;bornAt?:number;maxDepositFee?:DepositFeeSchedule;fetcher?:typeof fetch;networkPrivacy?:boolean;poolPage?:number;batchSwaps?:boolean;maxShieldCoins?:number;pendingNullifiers?:()=>ReadonlySet;pendingOutpoints?:()=>ReadonlySet;lnurlFetch?:typeof fetch;claimFetch?:typeof fetch; /** Called once with the Arkade address, synchronously and before the pool sync; a throw aborts create. */ onArkAddress?:(address:string)=>void}; @@ -113,8 +113,8 @@ const sats=(value:bigint|number)=>Number(value).toLocaleString('en-US'),short=(i export const shieldTitle=(plan:ShieldPlan,fee=0n)=>(plan.kind==='btc'?`Shielding ${sats(plan.amount)} sats`:`Shielding ${sats(plan.units)} units of asset ${short(plan.assetId)}`)+(fee>0n?` with a ${sats(fee)}-sat deposit fee`:''); const feePaused=(fee:DepositFeeSchedule)=>`Deposits are paused: this pool now charges a deposit fee of ${sats(fee.fixedSats)} sats or ${fee.bps} bps of the deposit, whichever is more, which is above what this wallet has accepted. Once it is shown, accept it with acceptDepositFee({fixedSats:${fee.fixedSats},bps:${fee.bps}}), or pass maxDepositFee of at least that.`; const validSchedule=(value:unknown,name:string)=>{const schedule=parseDepositFeeSchedule(value);if(!schedule)throw new Error(`${name} must be {fixedSats,bps}: whole, non-negative numbers, with bps at most 10000.`);return schedule;}; -export const nextShieldPlan=(coins:readonly ArkCoin[],o:{skip:(c:ArkCoin)=>boolean;used:ReadonlySet;listed:ReadonlySet;now:number;floorMs:number;fee?:Parameters[4]})=> - shieldPlan(coins.filter(c=>!o.skip(c)&&!o.used.has(`${c.txid}:${c.vout}`)),o.listed,o.now,o.floorMs,o.fee); +export const nextShieldPlan=(coins:readonly ArkCoin[],o:{skip:(c:ArkCoin)=>boolean;used:ReadonlySet;listed:ReadonlySet;now:number;floorMs:number;fee?:Parameters[4];maxCoins?:number})=> + shieldPlan(coins.filter(c=>!o.skip(c)&&!o.used.has(`${c.txid}:${c.vout}`)),o.listed,o.now,o.floorMs,o.fee,o.maxCoins); export const needsRenewal=(coins:readonly ArkCoin[],now:number,floorMs:number)=>coins.some(c=>c.expiresAt!==undefined&&c.expiresAt-now<=floorMs); export const mergeAssetFields=(reserves:Record|undefined)=>[BTC_ASSET,...Object.keys(reserves??{}).map(assetFieldOfId)]; export const programOf=(address:string)=>ArkAddress.decode(address).pkScript.subarray(2); @@ -179,6 +179,7 @@ export class ShieldedWallet extends Wallet { private statusAt=Date.now(); private networkPrivacy=false; private batchSwaps?:boolean; + private maxShieldCoins?:number; private pendingNullifiers?:()=>ReadonlySet; private pendingOutpoints?:()=>ReadonlySet; private lnurlFetch?:typeof fetch; @@ -193,7 +194,7 @@ export class ShieldedWallet extends Wallet { void wallet.getContractManager().catch(()=>{}); config.onArkAddress?.(await wallet.getAddress()); wallet.pool=config.pool;wallet.api=poolApiOf(config.pool.origin,config.fetcher,config.poolPage);wallet.claimApi=config.claimFetch?poolApiOf(config.pool.origin,config.claimFetch,config.poolPage):undefined;wallet.hash=config.hash;wallet.prover=config.prover; - wallet.networkPrivacy=config.networkPrivacy??isPrivatePoolApi(wallet.api);wallet.batchSwaps=config.batchSwaps; + wallet.networkPrivacy=config.networkPrivacy??isPrivatePoolApi(wallet.api);wallet.batchSwaps=config.batchSwaps;wallet.maxShieldCoins=config.maxShieldCoins; if(wallet.networkPrivacy){ const live=(api:PoolApi):PoolApi=>markPrivatePoolPage(markPrivatePoolApi(async(path:string,body?:unknown,beforeRequest?:()=>void)=>{ wallet.assertOpen(); @@ -613,7 +614,7 @@ export class ShieldedWallet extends Wallet { /** `skipPayouts` is what auto-shield wants and the "Shield again" button does not: a payout is only shielded on request. */ planFor(coins:readonly ArkCoin[],skipPayouts=true){ const pending=this.pendingOutpoints?.(),used=pending?new Set([...this.used,...pending]):this.used; - return nextShieldPlan(coins,{skip:c=>this.usedTxids.has(c.txid)||(skipPayouts&&this.isPoolPayout(c)),used,listed:new Set(Object.keys(this.pool.status.pool?.reserves??{})),now:Date.now(),floorMs:SHIELD_FLOOR_MS,fee:{schedule:this.depositFee,exempt:c=>this.isPoolPayout(c)}}); + return nextShieldPlan(coins,{skip:c=>this.usedTxids.has(c.txid)||(skipPayouts&&this.isPoolPayout(c)),used,listed:new Set(Object.keys(this.pool.status.pool?.reserves??{})),now:Date.now(),floorMs:SHIELD_FLOOR_MS,fee:{schedule:this.depositFee,exempt:c=>this.isPoolPayout(c)},maxCoins:this.maxShieldCoins}); } private deps():SubmitDeps{return {api:this.api,prover:this.prover,identity:this.identity,poolScript:this.poolScript};} /** Every held note of `assets` but `mine`, as an in-flight set: a rebuild can then choose only the notes its action already holds. */ diff --git a/tests/private-transport.test.ts b/tests/private-transport.test.ts index 3c62f3da..5a679d31 100644 --- a/tests/private-transport.test.ts +++ b/tests/private-transport.test.ts @@ -1,7 +1,8 @@ import assert from 'node:assert/strict'; import {readFileSync} from 'node:fs'; import {test,type TestContext} from 'node:test'; -import {ESPLORA_URL} from '@arkade-os/sdk'; +import {buildOffchainTx,CSVMultisigTapscript,DefaultVtxo,ESPLORA_URL,SingleKey} from '@arkade-os/sdk'; +import {base64,hex} from '@scure/base'; import {normalizeURL} from 'nostr-tools/utils'; import {NOSTR_CHUNK_BYTES,NOSTR_RELAYS,NOSTR_REQUEST_KIND} from '../packages/protocol/src/nostr-carrier.ts'; import {IsolatedOnchainProvider} from '../src/native/providers.ts'; @@ -10,9 +11,10 @@ import {generateOhttpKey,loadOhttpKey} from '../src/rollup/ohttp-keys.ts'; import {startNostrCarrier} from '../src/rollup/nostr-carrier.ts'; import {isPrivateFetch,isSealedFetch,markPrivateFetch,privatePoolPage} from '../src/shielded/network-privacy.ts'; import {poolApiOf} from '../src/shielded/pool.ts'; -import {installPrivateTransport,privateTransport,PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; +import {installPrivateTransport,privateTransport,PRIVATE_PAGE,PRIVATE_SHIELD_COINS} from '../src/shielded/private-transport.ts'; import {SealedFetchError} from '../src/shielded/sealed-fetch.ts'; import {createNostrBus} from './fixtures/nostr-bus.ts'; +import info from './fixtures/ark-info.json' with {type:'json'}; const POOL='https://pool.example',PAGE='http://127.0.0.1:5173',LOOPBACK='http://127.0.0.1:9',ARK='https://ark.example',ESPLORA=ESPLORA_URL.mutinynet,ID='ab'.repeat(16); const settle=()=>new Promise(resolve=>setImmediate(resolve)); @@ -187,6 +189,22 @@ test('a spend POST that never went out frees its reservation at once, and one th assert.deepEqual([g.transport.journal.pendingPoolNullifiers(POOL),g.calls],[[17n],[]]); }); +// As the SDK's one-shot send posts them: submit carries every checkpoint signed by the wallet, finalize the same checkpoints signed by both. +const offchainSend=async(n:number)=>{ + const user=SingleKey.fromHex('11'.repeat(32)),server=SingleKey.fromHex('22'.repeat(32)); + const vtxo=new DefaultVtxo.Script({pubKey:await user.xOnlyPublicKey(),serverPubKey:await server.xOnlyPublicKey(),csvTimelock:{type:'seconds',value:BigInt(info.unilateralExitDelay)}}); + const coins=Array.from({length:n},(_,i)=>({txid:(i+1).toString(16).padStart(64,'a'),vout:i%3,value:12_345_678+i,tapTree:vtxo.encode(),tapLeafScript:vtxo.forfeit()})); + const {arkTx,checkpoints}=buildOffchainTx(coins,[{script:vtxo.pkScript,amount:BigInt(coins.reduce((sum,coin)=>sum+coin.value,0))}],CSVMultisigTapscript.decode(hex.decode(info.checkpointTapscript))); + const ark=await user.sign(arkTx),mine=await Promise.all(checkpoints.map(checkpoint=>user.sign(checkpoint))),both=await Promise.all(mine.map(checkpoint=>server.sign(checkpoint))); + return {'/v1/tx/submit':JSON.stringify({signedArkTx:base64.encode(ark.toPSBT()),checkpointTxs:mine.map(c=>base64.encode(c.toPSBT()))}),'/v1/tx/finalize':JSON.stringify({arkTxid:ark.id,finalCheckpointTxs:both.map(c=>base64.encode(c.toPSBT()))})}; +}; +test(`a ${PRIVATE_SHIELD_COINS}-coin shield's submit and finalize bodies each seal into one event`,async t=>{ + const g=await rig(t);g.transport.providers(ARK,ARK); + for(const [path,body] of Object.entries(await offchainSend(PRIVATE_SHIELD_COINS)))assert.equal((await g.transport.fetch(ARK+path,{method:'POST',headers:{'content-type':'application/json','x-digest':info.digest},body})).status,200,path); + const sealed=g.requests().map(event=>base64.decode(event.content).length); + assert.ok(sealed.length===2&&sealed.every(size=>size<=NOSTR_CHUNK_BYTES),String(sealed)); +}); + test('the module never imports the native entry point',()=>{ const entry=/(?:from|import\(?)\s*'(?:\.\.\/native|\.)(?:\/index(?:\.[jt]s)?)?\/?'|@tauri-apps/; for(const bad of ["import {x} from '../native';","import {x} from '../native/index.ts';","import {x} from '.';","import {x} from './index.js';","import '../native/';","await import('../native/index');","import {invoke} from '@tauri-apps/api/core';"])assert.match(bad,entry,bad); diff --git a/tests/tor-content.test.ts b/tests/tor-content.test.ts index 04eadda8..dfd7012f 100644 --- a/tests/tor-content.test.ts +++ b/tests/tor-content.test.ts @@ -9,11 +9,11 @@ import {ROLLUP_RECORD_BYTES} from '../packages/protocol/src/rollup/wallet.ts'; import {deriveRollupKeys2} from '../packages/protocol/src/wallet-keys.ts'; import {epochCatchUp} from '../src/shielded/epoch-sync.ts'; import {epochSyncAccount} from '../src/shielded/epoch-pool.ts'; -import {ShieldedWallet,loadAccount} from '../src/shielded/wallet.ts'; +import {SHIELD_FLOOR_MS,ShieldedWallet,loadAccount} from '../src/shielded/wallet.ts'; import {lightningRails} from '../src/shielded/lightning.ts'; -import {waitForSpend,type PoolApi} from '../src/shielded/pool.ts'; +import {waitForSpend,type ArkCoin,type PoolApi} from '../src/shielded/pool.ts'; import {markPrivateFetch,markPrivatePoolPage,privatePoolPage,isPrivatePoolApi} from '../src/shielded/network-privacy.ts'; -import {PRIVATE_PAGE} from '../src/shielded/private-transport.ts'; +import {PRIVATE_PAGE,PRIVATE_SHIELD_COINS} from '../src/shielded/private-transport.ts'; import {ANSWER_BYTES} from '../src/rollup/nostr-carrier.ts'; import {epochLeafPool} from './fixtures/epoch-pool.ts'; import {makeNativeMutationJournal} from '../src/native/mutations.ts'; @@ -175,6 +175,14 @@ test('a restarted native wallet reserves uncertain deposit outpoints for every s assert.equal(await wallet.send({recipients:[{address:'unused',amount:330}],selectedVtxos:[input] as never}),'sent'); }); +test('a 30-coin plan in private mode takes at most the cap, the soonest to expire first and coins without expiry last, and a direct plan takes all',()=>{ + const now=Date.now(),coins:ArkCoin[]=Array.from({length:30},(_,i)=>({txid:i.toString(16).padStart(64,'0'),vout:0,value:1000,...(i%2?{}:{expiresAt:now+SHIELD_FLOOR_MS+(100-i)*60_000})})); + const plan=(cap?:number)=>(Object.assign(Object.create(ShieldedWallet.prototype),{used:new Set(),usedTxids:new Set(),pool:{status:{pool:{reserves:{}}}},isPoolPayout:()=>false,maxShieldCoins:cap}) as ShieldedWallet).planFor(coins,false); + const soonest=[...coins.filter(coin=>coin.expiresAt).reverse(),...coins.filter(coin=>!coin.expiresAt).slice(0,PRIVATE_SHIELD_COINS-15)]; + assert.deepEqual(plan(PRIVATE_SHIELD_COINS),{kind:'btc',coins:soonest,amount:1000*PRIVATE_SHIELD_COINS}); + assert.deepEqual(plan(),{kind:'btc',coins,amount:30_000}); +}); + test('real SDK automatic send and settlement exclude restarted uncertain deposits',async t=>{ const rows=new Map(),storage={getItem:(key:string)=>rows.get(key)??null,setItem:(key:string,value:string)=>{rows.set(key,value);},removeItem:(key:string)=>{rows.delete(key);}}; const origin='https://pool.test',id='a'.repeat(32),identity=SingleKey.fromHex('11'.repeat(32)),server=await SingleKey.fromHex('22'.repeat(32)).xOnlyPublicKey(); From c498d7d5b3525c3e848c8b2bfb790613142a0fd2 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:11:25 +0100 Subject: [PATCH 24/40] fix(app): open a private wallet's receive tab on shielded, with no unified tab MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A private wallet never creates Lightning, yet its receive portal opened on the unified tab, where the address never came and the loading line read "Starting the private Lightning service…". ReceivePortal takes unified (default true); without it the strip offers shielded and arkade only, and swipe, arrow-key and glass-lens positions follow the shorter list. The wallet computes withLightning as nativeHost || !PRIVATE, the same condition that builds the Lightning pairing, starts receive on shielded when it is false and passes it to the portal, so no Lightning stage label can show in private mode. Direct and native wallets are unchanged. --- app/src/RollupWallet.tsx | 5 +++-- app/src/wallet-receive.tsx | 28 +++++++++++++++------------- tests/app-private-build.test.ts | 7 +++++++ tests/receive-amount.test.ts | 6 ++++++ 4 files changed, 31 insertions(+), 15 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 7b14c605..7e7be6f3 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -95,7 +95,8 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= const [wallet,setWallet]=useState(),[tick,setTick]=useState(0); const [coins,setCoins]=useState([]),[arkAvailable,setArkAvailable]=useState(),[arkAddress,setArkAddress]=useState(''),[rows,setRows]=useState([]); const [sheet,setSheet]=useState(),[busy,setBusy]=useState(false),[revealed,setRevealed]=useState>({}); - const [tab,setTab]=useState<'home'|'receive'|'send'|'activity'>('home'),[receiveMode,setReceiveMode]=useState('unified'),[requestOpen,setRequestOpen]=useState(false),[assetId,setAssetId]=useState(''),[to,setTo]=useState(''),[amount,setAmount]=useState(''),[withSats,setWithSats]=useState(''); + const withLightning=nativeHost||!PRIVATE; + const [tab,setTab]=useState<'home'|'receive'|'send'|'activity'>('home'),[receiveMode,setReceiveMode]=useState(withLightning?'unified':'shielded'),[requestOpen,setRequestOpen]=useState(false),[assetId,setAssetId]=useState(''),[to,setTo]=useState(''),[amount,setAmount]=useState(''),[withSats,setWithSats]=useState(''); const [sheetOpen,setSheetOpen]=useState(false),narrow=useMedia('(max-width:760px)'),[scanning,setScanning]=useState(false),[codeError,setCodeError]=useState(''); const [pullRefresh,setPullRefresh]=useState({distance:0,ready:false,refreshing:false}); const pageNode=useRef(null); @@ -740,7 +741,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= {paymentTabs&&
} {tab==='home'?(!docked?
{balanceCard()}{recentPreview}
:null):tab==='activity'?

Activity

{historyContent(historyItems)}{synced&&!rows.length&&}
:<> {tab==='receive'?
- + {receivePaymentValue&&requestedAmount!==undefined&&

Requested {satsText(requestedAmount)}.

} {!receiveBlocked&&receiveMode==='unified'&&<>
Had an address before?
diff --git a/app/src/wallet-receive.tsx b/app/src/wallet-receive.tsx index b9795291..fc6a26d8 100644 --- a/app/src/wallet-receive.tsx +++ b/app/src/wallet-receive.tsx @@ -11,6 +11,7 @@ export type ReceivePayload='unified'|'shielded'|'lightning'|'invoice'|'arkade'; type ReceivePortalProps={ active?:boolean; + unified?:boolean; mode:ReceiveMode; payload:ReceivePayload; onModeChange:(mode:ReceiveMode)=>void; @@ -46,13 +47,14 @@ export function amountQrValue(mode:ReceiveMode,value:string,amount:string):strin return `bitcoin:?${mode==='unified'?'lightning':mode==='shielded'?'shielded':'ark'}=${destination}&amount=${btc}`; } -export function ReceivePortal({active=true,mode,payload,onModeChange,value,qrValue,amount,onAmount,loading,description,action,children}:ReceivePortalProps){ +export function ReceivePortal({active=true,unified=true,mode,payload,onModeChange,value,qrValue,amount,onAmount,loading,description,action,children}:ReceivePortalProps){ const [expanded,setExpanded]=useState(false); const buttons=useRef<(HTMLButtonElement|null)[]>([]); const strip=useRef(null),lens=useRef(null); const start=useRef<{x:number;y:number}|null>(null); - const selected=MODES.findIndex(([id])=>id===mode); - const navigation=useRef({selected,onModeChange});navigation.current={selected,onModeChange}; + const modes=unified?MODES:MODES.slice(1); + const selected=modes.findIndex(([id])=>id===mode); + const navigation=useRef({selected,onModeChange,modes});navigation.current={selected,onModeChange,modes}; const canShare=typeof navigator!=='undefined'&&typeof navigator.share==='function'; const invoice=payload==='invoice'; const invalidAmount=!!amount&&receiveAmount(amount)===undefined; @@ -61,20 +63,20 @@ export function ReceivePortal({active=true,mode,payload,onModeChange,value,qrVal useEffect(()=>{ if(!active||!strip.current)return; return bindSwipeGesture(strip.current,direction=>{ - const view=navigation.current,index=view.selected+(direction==='next'?1:-1),next=MODES[index]; + const view=navigation.current,index=view.selected+(direction==='next'?1:-1),next=view.modes[index]; if(next){view.onModeChange(next[0]);buttons.current[index]?.focus();} },{allowButtons:true}); },[active]); const keyTab=(event:KeyboardEvent,index:number)=>{ let next:number; - if(event.key==='ArrowRight')next=(index+1)%MODES.length; - else if(event.key==='ArrowLeft')next=(index+MODES.length-1)%MODES.length; + if(event.key==='ArrowRight')next=(index+1)%modes.length; + else if(event.key==='ArrowLeft')next=(index+modes.length-1)%modes.length; else if(event.key==='Home')next=0; - else if(event.key==='End')next=MODES.length-1; + else if(event.key==='End')next=modes.length-1; else return; event.preventDefault(); - onModeChange(MODES[next]![0]); + onModeChange(modes[next]![0]); buttons.current[next]?.focus(); }; const touchStart=(event:TouchEvent)=>{ @@ -89,19 +91,19 @@ export function ReceivePortal({active=true,mode,payload,onModeChange,value,qrVal const dx=touch.clientX-origin.x,dy=touch.clientY-origin.y; if(Math.abs(dx)<48||Math.abs(dx) -
+
{start.current=null;}}>
-

{TITLES[payload]}

+

{TITLES[payload]}

{displayValue?
- :
{invalidAmount?'Enter a valid amount to show your payment QR.':loading??'Your address will appear here.'}
} + :
{invalidAmount?'Enter a valid amount to show your payment QR.':loading??'Your address will appear here.'}
} {displayValue&&
{LABELS[payload]}{short(displayValue)===displayValue?{displayValue}:}
}

{description}

diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index f021ec04..c9058e96 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -23,6 +23,13 @@ test('the transport is the first module main.tsx imports',()=>{ assert.match(/^import\b.*$/m.exec(read('app/src/main.tsx'))?.[0]??'',/^import '\.\/private-boot(\.ts)?';/); }); +test('a private wallet, which has no Lightning, opens receive on shielded and offers no unified tab',()=>{ + const wallet=read('app/src/RollupWallet.tsx'),lightning=/const (\w+)=nativeHost\|\|!PRIVATE;/.exec(wallet)?.[1]; + assert.ok(lightning); + assert.ok(wallet.includes(`useState(${lightning}?'unified':'shielded')`)); + assert.match(wallet,new RegExp(`]*unified=\\{${lightning}\\}`)); +}); + // Source-level: /watch and the wallet are read for their literals and branches, not rendered. test('the pool-reading pages take their page size and their links from the transport',()=>{ const disclose=read('app/src/Disclose.tsx'); diff --git a/tests/receive-amount.test.ts b/tests/receive-amount.test.ts index 61a75ecc..d67dbc4d 100644 --- a/tests/receive-amount.test.ts +++ b/tests/receive-amount.test.ts @@ -28,6 +28,12 @@ test('the shared amount form covers every receive tab',()=>{ } }); +test('without unified, the portal offers shielded and arkade only',()=>{ + const tabs=(unified?:boolean)=>{const rendered=renderToStaticMarkup(createElement(ReceivePortal,{...(unified===undefined?{}:{unified}),mode:'shielded',payload:'shielded',value:'shrol_example',description:'Test receive',onModeChange:()=>{}}));return [[...rendered.matchAll(/id="receive-tab-(\w+)"/g)].map(m=>m[1]),/--glass-count:(\d)/.exec(rendered)?.[1]];}; + assert.deepEqual(tabs(false),[['shielded','arkade'],'2']); + assert.deepEqual(tabs(),[['unified','shielded','arkade'],'3']); +}); + test('payment URIs encode exact sats as eight decimal BTC amounts',()=>{ for(const [mode,address,parameter]of recipients){ for(const [amount,btc]of amounts){ From c74d22a5f7f64689b9bc62450f8d0144cb7003cf Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:11:59 +0100 Subject: [PATCH 25/40] fix(app): say what main.tsx's first import guarantees, and pin the link branch order MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The note on main.tsx's first import claimed the transport's guards stand before any other module runs. That does not hold: the bundler can hoist shared chunks (library code such as nostr-tools/pool) ahead of the entry's own modules. What holds is that the transport installs before any app code makes a request, which is what the note now says. The explorer-link source check matched any line with a private branch before ${EXPLORER}, so swapping the branches, which would put the explorer link in private mode, still passed. It now only matches PRIVATE?{txid}:line.includes('${EXPLORER}')); assert.ok(links.length,file); - for(const line of links)assert.match(line,/(PRIVATE|privateTransport\(\))\?(?!\.).*\$\{EXPLORER\}/,file); + for(const line of links)assert.match(line,/(PRIVATE|privateTransport\(\))\?\{txid\}<\/code>:]*\$\{EXPLORER\}/,file); } }); From 52cc5f77095e028fbcb2747021515df82d667ce4 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:15:55 +0100 Subject: [PATCH 26/40] test: pin the sealed POST's 45 s bound and the 30 s rounding The 45 s test ran only retryAfter 30 with three attempts, whose third check falls at 60 s, so any bound between 32 and 60 s passed it. It now also runs retryAfter 22 with three attempts and retryAfter 15 with four, and asserts when each later attempt went out: 22 s and 44 s (so the bound is over 44 000 ms), and 15 s and 30 s with no fourth at 45 s (so it is at most 45 000 ms). With sealedForMs at 44 000 or 45 001 the old test still passed and the new one fails. For that the clock has to move only for real waits. The bus delivers on zero-delay timers, which the old step fired only on the next 1 000 ms tick, so every status round trip cost 2 s of mocked time (the second attempt went at 24 s, not 22 s). The step now fires zero-delay timers in place first. The rounding check allowed 30 s, which a floor or ceiling would also pass; the nearest 30 s is within 15 s, so it allows 16 s. The test said to resolve "well inside the SDK getInfo abort" but measures no time: it checks that a GET answered at once publishes once and leaves no deadline behind, and is renamed to say so. --- tests/sealed-fetch.test.ts | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/tests/sealed-fetch.test.ts b/tests/sealed-fetch.test.ts index 36c001cf..bfa0fbdb 100644 --- a/tests/sealed-fetch.test.ts +++ b/tests/sealed-fetch.test.ts @@ -17,7 +17,9 @@ const settle=()=>new Promise(resolve=>setImmediate(resolve)); const holding=(reply=()=>Response.json({ok:true}))=>{let release=()=>{};const held=new Promise(resolve=>{release=()=>resolve(reply());});return {held,release};}; const failsWith=(code:SealedFetchError['code'])=>(error:unknown)=>error instanceof SealedFetchError&&error.code===code; const timers=()=>process.getActiveResourcesInfo().filter(type=>type==='Timeout').length; -const advance=async(t:TestContext,ms:number)=>{for(let at=0;at{for(let i=0;i<8;i++){t.mock.timers.tick(0);await settle();}}; +const advance=async(t:TestContext,ms:number)=>{await flush(t);for(let at=0;atResponse|Promise;gateway?:Partial[0]>;open?:OhttpOpen}&Pick[0],'pick'|'timeoutMs'|'getTimeoutMs'|'attempts'|'random'>; @@ -269,7 +271,7 @@ test('every sealed request carries an x-date rounded to the nearest 30 s, and x- const g=await rig(t); assert.equal((await g.fetch(ARK,{headers:{'x-digest':'abc',authorization:'x','x-sdk-version':'ts-sdk/0.5.3'}})).status,200); const {headers}=await g.inner(),at=Date.parse(headers.get('x-date')!); - assert.ok(at%30_000===0&&Math.abs(Date.now()-at)<=30_000,String(headers.get('x-date'))); + assert.ok(at%30_000===0&&Math.abs(Date.now()-at)<=16_000,String(headers.get('x-date'))); assert.deepEqual([headers.get('x-digest'),headers.has('authorization'),headers.has('x-sdk-version')],['abc',false,false]); }); @@ -296,16 +298,19 @@ test('a GET attempt ends after 9 s and a POST attempt after 40 s',async t=>{ t.mock.timers.tick(1);await post; }); -test('a GET answered on its first attempt resolves well inside the SDK getInfo abort',async t=>{ +test('a GET answered on its first attempt publishes once and leaves no deadline behind',async t=>{ const g=await rig(t),before=timers(); assert.equal((await g.fetch(ARK)).status,200);await sleep(20); assert.deepEqual([g.requests().length,timers()],[1,before]); }); test('a POST stops attempting 45 s after its sealing and ends outcome-unknown',async t=>{ - const g=await rig(t,{attempts:3,open:async()=>({kind:'busy',status:503,retryAfter:30})});t.mock.timers.enable({apis:['setTimeout','Date'],now:Date.now()}); - const spend=assert.rejects(g.fetch('/api/rollup/spends',SPEND),failsWith('outcome-unknown'));await until(()=>g.sentTo().length===1); - await advance(t,33_000);assert.equal(g.requests().length,2,'the second attempt went after the first 30 s back-off'); - await advance(t,40_000);await spend; - assert.equal(g.requests().length,2,'the second back-off ended past 45 s, so no third attempt went out');assert.equal(new Set(g.requests().map(event=>event.content)).size,1); + for(const [retryAfter,attempts,later] of [[30,3,[30_000]],[22,3,[22_000,44_000]],[15,4,[15_000,30_000]]] as const){ + const g=await rig(t,{attempts,open:async()=>({kind:'busy',status:503,retryAfter})});t.mock.timers.enable({apis:['setTimeout','Date'],now:Date.now()}); + const start=Date.now(),sent:number[]=[],spend=assert.rejects(g.fetch('/api/rollup/spends',SPEND),failsWith('outcome-unknown'));await until(()=>g.sentTo().length===1); + for(let at=0;at<100_000;at+=1000){const before=g.requests().length;await advance(t,1000);if(g.requests().length>before)sent.push(Date.now()-start);} + await spend; + assert.deepEqual(sent,later,`retryAfter ${retryAfter}, attempts ${attempts}`);assert.equal(new Set(g.requests().map(event=>event.content)).size,1); + t.mock.timers.reset(); + } }); From 8d91f5ba7cd236e8e440b3094c68e45bf59d984d Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:17:09 +0100 Subject: [PATCH 27/40] fix(rollup): start the gateway's replay window when the carrier starts listening Ruling 20. The gateway took startedAt at construction, at process start, but a new instance in a rolling deploy waits for the previous one to release the pool before its carrier subscribes. Meanwhile the previous instance answers requests from relays, and relays keep ephemeral events for minutes, so once the new carrier starts it can be handed a copy of a request the old instance already ran. Its replay register is empty, and that request's x-date was after the construction-time start, so the copy would run again. The gateway gains listen(), which resets startedAt to the gateway clock, and server.ts calls it right before startNostrCarrier, once service.ready() holds. Until it is called, the start stays at construction, as before (the HTTP route, when on, is served from process start). The 15 s rounding slack is unchanged: a request dated between construction and listen() minus 15 s is now refused as stale, and one dated listen() minus 15 s is still answered. --- src/rollup/ohttp-gateway.ts | 7 ++++--- src/server.ts | 2 +- tests/ohttp-gateway.test.ts | 12 ++++++++++++ 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/src/rollup/ohttp-gateway.ts b/src/rollup/ohttp-gateway.ts index 3d2586b6..6de0a97e 100644 --- a/src/rollup/ohttp-gateway.ts +++ b/src/rollup/ohttp-gateway.ts @@ -12,8 +12,8 @@ const copied=(from:Headers,names:string[])=>{const kept=new Headers();for(const export function createOhttpGateway({key,upstreams,poolOrigin,poolLoopback,gatewayToken,rate=20,maxInFlight=32,timeoutMs=30_000,http=true,fetch:send=fetch,now=()=>performance.now(),clock=()=>Date.now(),windowMs=90_000}:{key:OhttpGatewayKey;upstreams:OhttpUpstream[];poolOrigin:string;poolLoopback:string;gatewayToken:string;rate?:number;maxInFlight?:number;timeoutMs?:number;http?:boolean;fetch?:typeof fetch;now?:()=>number;clock?:()=>number;windowMs?:number}){ if(![rate,maxInFlight].every(count=>Number.isSafeInteger(count)&&count>=1))throw new Error('The Oblivious HTTP rate and in-flight cap are whole numbers of at least 1.'); const allowed=upstreams.map(({origin,prefix})=>{const url=new URL(origin);if(url.protocol!=='https:'||!prefix.startsWith('/')||!prefix.endsWith('/'))throw new Error(`An Oblivious HTTP upstream is https with a path prefix that starts and ends with /, not ${origin} ${prefix}.`);return {origin:url.origin,prefix};}); - const pool=new URL(poolOrigin).origin,loopback=new URL(poolLoopback).origin,tally=new Map(),seen=new Set(),startedAt=clock(); - let since=now(),second=0,used=0,inFlight=0; + const pool=new URL(poolOrigin).origin,loopback=new URL(poolLoopback).origin,tally=new Map(),seen=new Set(); + let since=now(),second=0,used=0,inFlight=0,startedAt=clock(); const note=(kind:string)=>{tally.set(kind,(tally.get(kind)??0)+1);if(now()-since<60_000)return;console.log(`[ohttp] ${[...tally].map(([name,count])=>`${name} ${count}`).join(', ')}`);tally.clear();since=now();}; const answer=(status:number,kind:string)=>{note(kind);return new Response(null,{status});}; const forward=async(request:Request,gone:AbortSignal,cap:number)=>{ @@ -80,5 +80,6 @@ export function createOhttpGateway({key,upstreams,poolOrigin,poolLoopback,gatewa res.status(200).setHeader('Content-Type','message/ohttp-res').setHeader('Cache-Control','no-store').end(sealed); }); router.use((error:{status?:number},_req:express.Request,res:express.Response,_next:express.NextFunction)=>{note(error.status===413?'too-large':'failed');res.sendStatus(error.status??500);}); - return Object.assign(router,{open}); + // Call as the carrier starts: a request dated before then may have run on the previous instance, whose replay register this one lacks. + return Object.assign(router,{open,listen(){startedAt=clock();}}); } diff --git a/src/server.ts b/src/server.ts index e80cd772..1fc529b2 100644 --- a/src/server.ts +++ b/src/server.ts @@ -57,7 +57,7 @@ const step=async()=>{ await service!.step(); if(service!.ready()){ // Listening starts only now, so a request is never opened while another instance still owns the pool. - if(gateway&&ohttpKey&&!carrier&&!stopping){carrier=startNostrCarrier({open:gateway.open,secret:ohttpKey.nostrSecret,relays:nostrRelays});console.log(`Nostr carrier listening on ${nostrRelays.length} relays.`);} + if(gateway&&ohttpKey&&!carrier&&!stopping){gateway.listen();carrier=startNostrCarrier({open:gateway.open,secret:ohttpKey.nostrSecret,relays:nostrRelays});console.log(`Nostr carrier listening on ${nostrRelays.length} relays.`);} // v1 data goes only once this instance owns the pool, so the previous instance has stopped using it. try{const removed=dropLegacyData(data);if(removed.length)console.log('Removed retired v1 data: '+removed.join(', '));}catch(error){console.error('Could not remove retired v1 data: '+(error as Error).message);} return; diff --git a/tests/ohttp-gateway.test.ts b/tests/ohttp-gateway.test.ts index 389d3f7f..d5a1d8b1 100644 --- a/tests/ohttp-gateway.test.ts +++ b/tests/ohttp-gateway.test.ts @@ -330,6 +330,18 @@ test('an x-date before the gateway started is refused, with 15 s of rounding sla assert.deepEqual(await g.router.open(bytesOf(await at(T-15_001))),{kind:'refused'});assert.equal(g.calls.length,1); }); +test('listen() moves the start to when the carrier listens, so a request dated before that, less 15 s, is stale',async t=>{ + const lines:string[]=[];t.mock.method(console,'log',(line:string)=>{lines.push(line);}); + const T=Date.UTC(2026,10,11,12);let clock=T,tick=0;const g=await gateway(t,{clock:()=>clock,now:()=>tick}),at=(ms:number)=>g.seal(ARK,{headers:{'x-date':new Date(ms).toISOString()}}); + clock=T+60_000; + assert.equal((await through(g.router,await at(T+30_000))).status,200,'until listen(), the gateway started when it was made'); + g.router.listen(); + assert.deepEqual(await g.router.open(bytesOf(await at(T+30_000))),{kind:'refused'}); + assert.equal((await through(g.router,await at(T+45_000))).status,200,'the 15 s of rounding slack holds'); + tick=60_000;assert.equal((await through(g.router,await at(T+60_000))).status,200); + assert.deepEqual(lines,['[ohttp] forwarded 3, stale 1']); +}); + test('x-digest crosses to the upstream and x-date does not',async t=>{ const g=await gateway(t); assert.equal((await g.open(ARK,{headers:{'x-digest':'abc','x-date':new Date().toUTCString(),'x-sdk-version':'ts-sdk/0.5.3'}})).status,200); From a2cdf737ca70237d24962ba4a0e5524c0362c758 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:19:16 +0100 Subject: [PATCH 28/40] docs(dokploy): bring the private gateway and wallet build sections up to date - The replay paragraph names the register's other exit: it keeps the latest 100,000 opened requests, so a copy can run again once that many newer ones have opened (over an hour at the default 20 a second). - Logs name the [ohttp] stale count: a wallet clock off by more than about 75 to 105 seconds has every request refused as stale and gets silence. - The gateway's start is now when its Nostr listener starts, so the restart paragraph says only requests dated in the last 15 seconds before it can run twice, not everything dated after the process started. - Build arguments: SHIELDED_NOSTR_RELAYS and SHIELDED_PUBLIC_ORIGIN are read by the server at runtime too, so they are build arguments in addition to the runtime environment, not instead of it (Ruling 21). - Turn it on: a rollback below the first pinned build, or a tab still running an older cached version, ignores the flag, so testers tap Update first; an unpinned build that sees the flag keeps the wallet closed, and how to open it again. A private wallet shields at most 20 coins at a time. - Turn it off: optionally wait until the unresolved-outcomes line clears. - Rotation: rebuild with both pins, then Update or close every tab. - The release list joins the page and proving files as what loads direct. --- docs/DOKPLOY.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/DOKPLOY.md b/docs/DOKPLOY.md index 6c864069..34943070 100644 --- a/docs/DOKPLOY.md +++ b/docs/DOKPLOY.md @@ -59,7 +59,7 @@ If the pool's public origin is also an upstream's, only `/api/rollup/` paths on **What the shared budget costs.** One holder of the public key config can submit at the whole `SHIELDED_OHTTP_RATE`: 20 a second by default, about 1,200 a minute, against 60 a minute per address when calling the pool directly. Each submission costs a Groth16 check. The budget is shared and first come, first served, so that caller crowds out every other sealed caller, reads included. It can also push direct `/spends` requests to a 503 "busy", because the cap on spends being checked at once (8) is shared by everyone. To cut it, lower `SHIELDED_OHTTP_RATE` or unset the key. -**Limits and logs.** A request over 1 MiB gets an outer 413, and one still uploading after 30 seconds is cut off. A response over 4 MiB, or an upstream that takes over 30 seconds, comes back as an inner 502 or 504. Whatever the upstream or the reading client does, the connection is cut after 90 seconds, three times that deadline, so no request holds a slot longer. A sealed request carries an inner `X-Date` rounded to the nearest 30 seconds (not `Date`, which a browser strips). One dated more than 90 seconds from the gateway's clock, or more than 15 seconds before the gateway started, is refused like a repeat: an outer 400 on the HTTP route, and no answer over Nostr. Of the client's headers only the content type, `Accept` and `X-Digest` cross to the upstream, and of the upstream's only the content type, the cache control and `Retry-After`, so a sealed client can wait as told. All callers share one budget, `SHIELDED_OHTTP_RATE` a second and 32 requests in flight; past it the gateway answers an outer 429 or 503 with `Retry-After: 1`. It logs counts by kind, such as `[ohttp] forwarded 120, busy 3`, at most once a minute, and never a target or a body. +**Limits and logs.** A request over 1 MiB gets an outer 413, and one still uploading after 30 seconds is cut off. A response over 4 MiB, or an upstream that takes over 30 seconds, comes back as an inner 502 or 504. Whatever the upstream or the reading client does, the connection is cut after 90 seconds, three times that deadline, so no request holds a slot longer. A sealed request carries an inner `X-Date` rounded to the nearest 30 seconds (not `Date`, which a browser strips). One dated more than 90 seconds from the gateway's clock, or more than 15 seconds before the gateway's start, is refused like a repeat: an outer 400 on the HTTP route, and no answer over Nostr. The start is when the Nostr listener starts (see **Start and stop**), and the process start until then. Of the client's headers only the content type, `Accept` and `X-Digest` cross to the upstream, and of the upstream's only the content type, the cache control and `Retry-After`, so a sealed client can wait as told. All callers share one budget, `SHIELDED_OHTTP_RATE` a second and 32 requests in flight; past it the gateway answers an outer 429 or 503 with `Retry-After: 1`. It logs counts by kind, such as `[ohttp] forwarded 120, busy 3`, at most once a minute, and never a target or a body. **Rotate.** The gateway holds one key at a time. Run `ohttp:keygen` again. It picks the key id at random, so if it prints the id already in use, run it again. Replace the `SHIELDED_OHTTP_KEY` line, redeploy and publish the new config. The wallet build that uses the gateway pins the config, so a rotation also needs a new app build. Until a client has the new config, its requests are sealed to the old key id, and the gateway answers them with an outer 400. @@ -75,29 +75,29 @@ A wallet can also send its sealed request inside a Nostr event, so no relay of o **The key.** The gateway's Nostr key is derived from the OHTTP key (HKDF-SHA256 over the 32-byte secret, with the info `shielded-nostr-carrier-v1`), so there is no second secret to keep and a rotation changes both. `ohttp:keygen` prints its public half as `nostr: `, and the start log shows the first 8 characters. Run one instance per key: two servers with the same `SHIELDED_OHTTP_KEY` both hear every request and both answer it, so a GET gets two conflicting sets of chunks and a POST to arkd runs twice. -**Start and stop.** The listener starts once, the first time the pool has opened, and logs `Nostr carrier listening on N relays.` It closes before the pool does. While a rollout waits for the previous instance to release the volume, the new instance does not listen. It asks each relay only for requests from then on (`limit:0`, and none dated more than about a minute earlier), so a restarted instance does not run again the requests a relay still stores; a relay that ignores `limit:0` can still hand back the last minute's. The same holds when the listener resubscribes after losing a relay: requests that reached that relay during the gap are never heard. They never run, so a GET goes out again through another relay. A POST that only that relay took ends `outcome-unknown`, even though it did not run. Across a restart, the gateway refuses any request whose `X-Date` is more than 15 seconds before its process started, but its register starts empty: a request dated after that, which the previous instance already answered (it keeps answering while a rollout waits for the volume lock), can run once more if a copy reaches the new instance within 90 seconds of its `X-Date`. +**Start and stop.** The listener starts once, the first time the pool has opened, and logs `Nostr carrier listening on N relays.` It closes before the pool does. While a rollout waits for the previous instance to release the volume, the new instance does not listen. It asks each relay only for requests from then on (`limit:0`, and none dated more than about a minute earlier), so a restarted instance does not run again the requests a relay still stores; a relay that ignores `limit:0` can still hand back the last minute's. The same holds when the listener resubscribes after losing a relay: requests that reached that relay during the gap are never heard. They never run, so a GET goes out again through another relay. A POST that only that relay took ends `outcome-unknown`, even though it did not run. Across a restart, the gateway refuses any request whose `X-Date` is more than 15 seconds before its listener started, and its register starts empty. The previous instance keeps answering while a rollout waits for the volume lock, so a request dated in the last 15 seconds before the new listener started, which the previous instance may already have answered, can run once more if a copy reaches the new instance within 90 seconds of its `X-Date`. -**What a relay and its readers see.** The relay sees the address of the wallet that publishes, and the event: a one-time key as author, the gateway's key in the `p` tag, a time rounded to the nearest 30 seconds and the sealed bytes. Each relay also sees the pool server's own address, which Cloudflare otherwise hides: the listener's connections to the relays go out from the server's egress address. Anyone who reads the relay sees the same events without the address, and the answers: the gateway's key as author, the hash of the sealed request (`x`) and the chunk numbers in the tags, and when. A status event carries, instead of chunk numbers, its status code, its retry delay and the id of the request event it answers. Neither learns the target or the body, which only the gateway can open. The gateway learns them but not the address, because a relay is the only path a request takes to it. Sizes show: a request's length and an answer's chunk count say roughly how much was asked and returned. Anyone who reads the relay can also copy a request's sealed bytes into an event of their own. The window, not the register of opened requests, is what bounds a replay: a copy that reaches the gateway more than 90 seconds after the original's `X-Date` gets no answer at all, and within those 90 seconds the register refuses it, except across a restart (see **Start and stop**). +**What a relay and its readers see.** The relay sees the address of the wallet that publishes, and the event: a one-time key as author, the gateway's key in the `p` tag, a time rounded to the nearest 30 seconds and the sealed bytes. Each relay also sees the pool server's own address, which Cloudflare otherwise hides: the listener's connections to the relays go out from the server's egress address. Anyone who reads the relay sees the same events without the address, and the answers: the gateway's key as author, the hash of the sealed request (`x`) and the chunk numbers in the tags, and when. A status event carries, instead of chunk numbers, its status code, its retry delay and the id of the request event it answers. Neither learns the target or the body, which only the gateway can open. The gateway learns them but not the address, because a relay is the only path a request takes to it. Sizes show: a request's length and an answer's chunk count say roughly how much was asked and returned. Anyone who reads the relay can also copy a request's sealed bytes into an event of their own. The window, not the register of opened requests, is what bounds a replay: a copy that reaches the gateway more than 90 seconds after the original's `X-Date` gets no answer at all, and within those 90 seconds the register refuses it, except across a restart (see **Start and stop**) or once 100,000 newer requests have opened, since the register keeps only the latest 100,000. At the default 20 a second, that takes over an hour. -**Logs.** The listener logs counts by kind, such as `[nostr] answered 12, busy 1, dropped 3`, at most once a minute, and never an event, a relay or a target. `answered` and `busy` are published; `status-capped` is a status event held back because 5 went out in that second; `refused` is a request the gateway would not open; `dropped` is an event that is not a valid request; `duplicate` is another copy of a request already seen; `republished` is a cached answer sent to a relay that delivered a copy late; `failed` is an error after the request was accepted, where nothing is published because the request may have run; `unpublished` is an answer event a relay refused, timed out or could not be reached for; `lost` is a relay whose subscription closed, or that could not be reached, which the listener tries again after 30 seconds; `deaf` is a relay that did not echo the canary within 15 seconds, which the listener closed (so it counts `lost` too) and tries again after 30 seconds; `canary-refused` is a canary a relay refused with a NIP-01 prefix, such as `rate-limited:`, which is not taken for deafness, so the relay stays subscribed, unchecked until a canary goes through. The gateway's own `[ohttp]` line counts `outbound`, a request shed because the carrier's outbound budget was spent. Relay NOTICE messages are not printed. The listener cannot silence one thing: when a relay sends a message the Nostr library cannot parse, or an event it cannot match (a kind 21470 event with no tags, say), the library prints its own warning to stderr, `[nostr] relay error processing message:` with the error, and with the event itself in the second case. For a frame it cannot parse, the error is V8's, which quotes up to 10 characters of the frame on either side of the fault. A relay that does this is misbehaving; take it out of `SHIELDED_NOSTR_RELAYS`. +**Logs.** The listener logs counts by kind, such as `[nostr] answered 12, busy 1, dropped 3`, at most once a minute, and never an event, a relay or a target. `answered` and `busy` are published; `status-capped` is a status event held back because 5 went out in that second; `refused` is a request the gateway would not open; `dropped` is an event that is not a valid request; `duplicate` is another copy of a request already seen; `republished` is a cached answer sent to a relay that delivered a copy late; `failed` is an error after the request was accepted, where nothing is published because the request may have run; `unpublished` is an answer event a relay refused, timed out or could not be reached for; `lost` is a relay whose subscription closed, or that could not be reached, which the listener tries again after 30 seconds; `deaf` is a relay that did not echo the canary within 15 seconds, which the listener closed (so it counts `lost` too) and tries again after 30 seconds; `canary-refused` is a canary a relay refused with a NIP-01 prefix, such as `rate-limited:`, which is not taken for deafness, so the relay stays subscribed, unchecked until a canary goes through. The gateway's own `[ohttp]` line counts `outbound`, a request shed because the carrier's outbound budget was spent, and `stale`, a request dated outside the window. A wallet whose clock is off by more than about 75 to 105 seconds (the 90-second window, give or take the 15 seconds of rounding) has its requests counted `stale` and gets silence, so it stops syncing without saying why. Relay NOTICE messages are not printed. The listener cannot silence one thing: when a relay sends a message the Nostr library cannot parse, or an event it cannot match (a kind 21470 event with no tags, say), the library prints its own warning to stderr, `[nostr] relay error processing message:` with the error, and with the event itself in the second case. For a frame it cannot parse, the error is V8's, which quotes up to 10 characters of the frame on either side of the fault. A relay that does this is misbehaving; take it out of `SHIELDED_NOSTR_RELAYS`. **Limits.** Liveness: every 30 seconds the listener sends each relay a canary, a kind 21470 event from its own key with empty content, and expects it back within 15 seconds, so a relay that keeps the socket open but stops forwarding events is closed and subscribed again. The canary also keeps open a relay that drops idle connections (one dropped them after 60 seconds). A frame over 1 MiB closes the connection, a connection whose upgrade stalls for 10 seconds is dropped, and a relay with more than 4 MiB unsent gets no more events until it drains. Each relay message is handled on its own, so a copy of a request that arrives together with a copy the gateway sheds is opened afresh, and is answered or gets a status naming its own event. Restarting the service reopens every subscription. The status cap: under a flood, a request that was shed but not told (past the 5 a second) looks to the wallet like one that may have run. #### The private wallet build -The browser wallet can send its requests to the pool, arkd, the Arkade indexer and Esplora through this carrier instead of calling them directly; only the page itself and the proving files still load from the site. For now this is an internal flag, for fresh test wallets only. +The browser wallet can send its requests to the pool, arkd, the Arkade indexer and Esplora through this carrier instead of calling them directly; only the page itself, the proving files and the list of native downloads still load from the site. For now this is an internal flag, for fresh test wallets only. -**Build arguments.** The app build takes four. Set them in **Environment → Build Time Arguments**, not in the runtime environment, and redeploy: +**Build arguments.** The app build takes four, set in **Environment → Build Time Arguments**; redeploy after changing any. The first two are build arguments only. The server also reads the last two at runtime, so they are build arguments in addition to the runtime environment, not instead of it: - `SHIELDED_OHTTP_CONFIG`: the `config:` value `ohttp:keygen` printed. - `SHIELDED_NOSTR_GATEWAY`: the `nostr:` value it printed. - `SHIELDED_NOSTR_RELAYS`: the relays the wallet publishes to, separated by commas. Unset or blank uses the six defaults under **Relays**. - `SHIELDED_PUBLIC_ORIGIN`: the pool's origin. Unset or blank uses `https://shielded.mutinynet.arkade.sh`. -A build missing either of the first two has no private transport, whatever a browser asks for. The server reads `SHIELDED_PUBLIC_ORIGIN` at runtime and the app build reads it too: the build pins it as the origin every sealed pool request names, so set the same value in both places. +A build missing either of the first two has no private transport. The build pins `SHIELDED_PUBLIC_ORIGIN` as the origin every sealed pool request names, so give it the same value as the runtime one. It pins `SHIELDED_NOSTR_RELAYS` as the relays the wallet publishes to, which must be the gateway's runtime list or part of it. -**Turn it on.** On a pinned build, `?private=1` on any page turns the transport on for that browser profile. Use it only with a fresh test wallet. A private wallet has no Lightning, no renewal, no on-chain exit and no explorer links; a transaction id shows as plain text. If the transport cannot start, the page says why and the wallet stays closed. +**Turn it on.** On a pinned build, `?private=1` on any page turns the transport on for that browser profile. Use it only with a fresh test wallet. A private wallet has no Lightning, no renewal, no on-chain exit and no explorer links, and it shields at most 20 coins at a time; a transaction id shows as plain text. If the transport cannot start, the page says why and the wallet stays closed. A build from before the private transport, which a rollback can return to, and a tab still running an older version from the app cache both ignore the flag and send every request direct, so tap Update before opening a page with `?private=1`. A build without both pins that sees the flag, in the URL or stored, keeps the wallet closed and says "this build has no private gateway pinned"; to open it, remove the key as under **Turn it off** and load the page without `?private=1`. -**Turn it off.** No link turns it off, so nobody can send a tester a link that quietly returns their wallet to direct requests; `?private=0` does nothing. Delete the `localStorage` key `shielded-private-transport-v1` in the browser's developer tools, or clear the site's data. Clearing the site's data deletes the wallet as well, so back up its recovery phrase first. +**Turn it off.** No link turns it off, so nobody can send a tester a link that quietly returns their wallet to direct requests; `?private=0` does nothing. Delete the `localStorage` key `shielded-private-transport-v1` in the browser's developer tools, or clear the site's data. Clearing the site's data deletes the wallet as well, so back up its recovery phrase first. If the wallet says some payments have unresolved outcomes, you can wait until that line clears before turning it off: without the transport, the wallet no longer keeps those payments' notes reserved. -**Keep the pins equal to the gateway's.** The key config and relays a build pins must match the gateway's `SHIELDED_OHTTP_KEY` and `SHIELDED_NOSTR_RELAYS`. After a rotation, a pinned build seals to the old key, and over Nostr the gateway answers that with silence, not a stale-key error, so its wallets stop syncing without saying why. Rebuild the app with the new config at every rotation, and keep the gateway's relay list equal to the build's or larger. \ No newline at end of file +**Keep the pins equal to the gateway's.** The key config and relays a build pins must match the gateway's `SHIELDED_OHTTP_KEY` and `SHIELDED_NOSTR_RELAYS`. After a rotation, a pinned build seals to the old key, and over Nostr the gateway answers that with silence, not a stale-key error, so its wallets stop syncing without saying why. At every rotation, rebuild the app with both new pins, `SHIELDED_OHTTP_CONFIG` and `SHIELDED_NOSTR_GATEWAY` (the Nostr key is derived from the OHTTP key, so it changes too), and have testers tap Update or close every tab of the site, since the app cache serves the old build until then. Keep the gateway's relay list equal to the build's or larger. \ No newline at end of file From 75d22dcc18144001315ed1cdc24b160d748f6509 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:48:43 +0100 Subject: [PATCH 29/40] fix(app): lay out the receive strip in one column per offered tab A private wallet offers two receive tabs, and the glass lens already sized itself from --glass-count, but the strip kept three grid columns, so the lens covered Shielded and half of Arkade. --- app/src/wallet-receive.css | 2 +- tests/receive-amount.test.ts | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/app/src/wallet-receive.css b/app/src/wallet-receive.css index e42761f7..ab5ea2fa 100644 --- a/app/src/wallet-receive.css +++ b/app/src/wallet-receive.css @@ -1,6 +1,6 @@ .receive-portal{--receive-ink:#103554;--receive-pearl:#faf7f0;--receive-gold:#795712;--receive-gold-fill:#e4c46c;--receive-muted:#536069;position:relative;isolation:isolate;display:grid;gap:18px;padding:18px;border:1px solid rgba(255,255,255,.64);border-radius:28px;background:linear-gradient(145deg,rgba(255,255,255,.24),rgba(247,251,255,.08));color:var(--receive-ink);box-shadow:inset 0 2px 0 rgba(255,255,255,.95),0 25px 52px -34px rgba(16,53,84,.38),0 8px 22px -18px rgba(184,134,43,.3);overflow:hidden} .receive-portal::before{content:'';position:absolute;inset:1px;border-radius:31px;pointer-events:none;border:1px solid rgba(255,255,255,.65);mask-image:linear-gradient(125deg,#000,transparent 42%,transparent 72%,#000)} -.receive-pill{position:relative;z-index:1;display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:4px;padding:5px;border:1px solid rgba(16,53,84,.14);border-radius:999px;background:rgba(16,53,84,.055);box-shadow:inset 0 1px 4px rgba(16,53,84,.07);-webkit-backdrop-filter:blur(22px);backdrop-filter:blur(22px)} +.receive-pill{position:relative;z-index:1;display:grid;grid-template-columns:repeat(var(--glass-count,3),minmax(0,1fr));gap:4px;padding:5px;border:1px solid rgba(16,53,84,.14);border-radius:999px;background:rgba(16,53,84,.055);box-shadow:inset 0 1px 4px rgba(16,53,84,.07);-webkit-backdrop-filter:blur(22px);backdrop-filter:blur(22px)} .receive-portal .receive-pill button{display:flex;align-items:center;justify-content:center;gap:6px;justify-self:stretch;min-width:0;min-height:44px;padding:8px 9px;border:0;border-radius:999px;background:transparent;color:var(--receive-muted);box-shadow:none;font-size:13px;font-weight:600;white-space:nowrap;cursor:pointer;transition:background .28s ease,color .28s ease,transform .28s cubic-bezier(.2,.75,.2,1),box-shadow .28s ease} .receive-portal .receive-pill button::after{display:none} .receive-portal .receive-pill button.is-selected{background:transparent;color:var(--receive-ink);box-shadow:none} diff --git a/tests/receive-amount.test.ts b/tests/receive-amount.test.ts index d67dbc4d..40044328 100644 --- a/tests/receive-amount.test.ts +++ b/tests/receive-amount.test.ts @@ -1,4 +1,5 @@ import assert from 'node:assert/strict'; +import {readFileSync} from 'node:fs'; import {test} from 'node:test'; import {Children,createElement,isValidElement,type ReactNode} from 'react'; import {renderToStaticMarkup} from 'react-dom/server'; @@ -34,6 +35,11 @@ test('without unified, the portal offers shielded and arkade only',()=>{ assert.deepEqual(tabs(),[['unified','shielded','arkade'],'3']); }); +test('the receive strip lays out one column per offered tab, as its lens does',()=>{ + const pill=/\.receive-pill\{[^}]*\}/.exec(readFileSync(new URL('../app/src/wallet-receive.css',import.meta.url),'utf8'))?.[0]??''; + assert.match(pill,/grid-template-columns:repeat\(var\(--glass-count,3\),minmax\(0,1fr\)\)/); +}); + test('payment URIs encode exact sats as eight decimal BTC amounts',()=>{ for(const [mode,address,parameter]of recipients){ for(const [amount,btc]of amounts){ From 2b2a93a9fc819f517fe7fecb7a0dfcd0b5b45aec Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 09:48:45 +0100 Subject: [PATCH 30/40] test: pin the private tick reconcile and the private shield cap wiring The helper tests passed even if the wallet's tick stopped calling reconcile, or if the private options or create dropped maxShieldCoins; each of those four breaks now fails a test. --- tests/app-private-build.test.ts | 7 +++++++ tests/tor-content.test.ts | 10 +++++----- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index a9f38b26..27c6abfb 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -30,6 +30,13 @@ test('a private wallet, which has no Lightning, opens receive on shielded and of assert.match(wallet,new RegExp(`]*unified=\\{${lightning}\\}`)); }); +test('a private wallet reconciles its journal on every tick and caps its shields',()=>{ + const wallet=read('app/src/RollupWallet.tsx'),effect=/useEffect\(\(\)=>\{\s*if\(!wallet\|\|!nativeHost&&!PRIVATE\)return;([\s\S]*?)\},\[([^\]]*)\]\);/.exec(wallet); + assert.match(effect?.[1]??'',/if\(!nativeHost\)\{[^\n]*PRIVATE!\.reconcile\(\)/); + assert.match(effect?.[2]??'',/\btick\b/); + assert.match(wallet,/:PRIVATE\?\{[^\n]*maxShieldCoins:PRIVATE_SHIELD_COINS/); +}); + // Source-level: /watch and the wallet are read for their literals and branches, not rendered. test('the pool-reading pages take their page size and their links from the transport',()=>{ const disclose=read('app/src/Disclose.tsx'); diff --git a/tests/tor-content.test.ts b/tests/tor-content.test.ts index dfd7012f..0494ddec 100644 --- a/tests/tor-content.test.ts +++ b/tests/tor-content.test.ts @@ -84,15 +84,15 @@ test('a size outside 1 to 100 reads back as 50, and a branded one as itself',()= assert.equal(privatePoolPage(()=>{}),50);assert.equal(privatePoolPage(undefined),50); }); -test('a re-wrapped private pool api keeps its page size',async t=>{ - const sizes:number[]=[],sentinel=new Error('creation inspected'),wallet=Object.assign(Object.create(ShieldedWallet.prototype),{ +test('a re-wrapped private pool api keeps its page size, and create keeps the shield cap',async t=>{ + const sizes:(number|undefined)[]=[],sentinel=new Error('creation inspected'),wallet=Object.assign(Object.create(ShieldedWallet.prototype),{ stopped:false,unwatch:[],changeListeners:new Set(),actionListeners:new Set(),getContractManager:async()=>({}),getAddress:async()=>'unused', - async load(this:{api:PoolApi;claimApi:PoolApi}){sizes.push(privatePoolPage(this.api),privatePoolPage(this.claimApi));throw sentinel;}, + async load(this:{api:PoolApi;claimApi:PoolApi;maxShieldCoins?:number}){sizes.push(privatePoolPage(this.api),privatePoolPage(this.claimApi),this.maxShieldCoins);throw sentinel;}, }) as ShieldedWallet; t.mock.method(Wallet,'create',async()=>wallet as never);t.mock.method(Wallet.prototype,'dispose',async()=>{}); const fetcher=markPrivateFetch((async()=>Response.json({})) as typeof fetch); - await assert.rejects(ShieldedWallet.create({identity:undefined as never,pool:{origin:'https://pool.test',status:{pool:{token:'pool',script:'5120',address:'unused'}} as never},secret:'11'.repeat(32),hash:()=>1n,prover:undefined as never,fetcher,claimFetch:fetcher,networkPrivacy:true,poolPage:12}),error=>error===sentinel); - assert.deepEqual(sizes,[12,12]); + await assert.rejects(ShieldedWallet.create({identity:undefined as never,pool:{origin:'https://pool.test',status:{pool:{token:'pool',script:'5120',address:'unused'}} as never},secret:'11'.repeat(32),hash:()=>1n,prover:undefined as never,fetcher,claimFetch:fetcher,networkPrivacy:true,poolPage:12,maxShieldCoins:PRIVATE_SHIELD_COINS}),error=>error===sentinel); + assert.deepEqual(sizes,[12,12,PRIVATE_SHIELD_COINS]); }); test('a worst-case page of 12 batches stays inside the gateway answer cap',()=>{ From 8de5e04d2d148d37c9af109df7e7fd7695f1d7b5 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 11:08:01 +0100 Subject: [PATCH 31/40] fix(shielded): let a capped shield take the largest coins when only they cover the fee Past the private wallet's 20-coin cap, the soonest-to-expire coins go first. If a deposit fee leaves that group under the dust minimum, the largest coins up to the cap are tried next; the fee is charged on the whole amount, so no other group of that size nets more. Without this, a fee larger than the soonest coins left every coin unshielded. --- src/shielded/pool.ts | 16 +++++++++------- tests/rollup-wallet-page.test.ts | 7 +++++++ 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 30f29aae..5886617f 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -142,13 +142,15 @@ export const shieldableWithFee=(plan:ShieldPlan,fee:DepositFeeSchedule|undefined const carrier=BigInt(plan.coin.value),free=BigInt(CARRIER_FREE_SATS); return carrier<=free||carrier-owed>=free; }; -/** What auto-shield moves next: every plain coin as one deposit, else one coin of a listed asset. The pool refuses coins near expiry, and a deposit the fee would eat is left out. While a fee charges, pool payouts (`exempt`) go one at a time: a merged coin is a plain send and would pay. Past `maxCoins` plain coins, the soonest to expire go first. */ +/** What auto-shield moves next: every plain coin as one deposit, else one coin of a listed asset. The pool refuses coins near expiry, and a deposit the fee would eat is left out. While a fee charges, pool payouts (`exempt`) go one at a time: a merged coin is a plain send and would pay. Past `maxCoins` plain coins, the soonest to expire go first, or the largest when only they cover the fee. */ export function shieldPlan(coins:readonly ArkCoin[],listed:ReadonlySet,now:number,floorMs:number,fee:{schedule?:DepositFeeSchedule;exempt?:(coin:ArkCoin)=>boolean}={},maxCoins=Infinity):ShieldPlan|undefined { - const {schedule,exempt}=fee,usable=coins.filter(c=>c.expiresAt===undefined||c.expiresAt-now>floorMs); - let plain=usable.filter(c=>!c.assets?.length); - if(plain.length>maxCoins)plain=plain.sort((a,b)=>(a.expiresAt??Number.MAX_SAFE_INTEGER)-(b.expiresAt??Number.MAX_SAFE_INTEGER)).slice(0,maxCoins); - if(exempt&&plain.length>1&&(schedule?.fixedSats||schedule?.bps)&&plain.every(exempt))plain=[plain.reduce((a,b)=>b.value>a.value?b:a)]; - const btc:ShieldPlan={kind:'btc',coins:plain,amount:plain.reduce((sum,c)=>sum+c.value,0)}; - if(shieldableWithFee(btc,schedule,exempt))return btc; + const {schedule,exempt}=fee,usable=coins.filter(c=>c.expiresAt===undefined||c.expiresAt-now>floorMs),plain=usable.filter(c=>!c.assets?.length); + const expiry=(c:ArkCoin)=>c.expiresAt??Number.MAX_SAFE_INTEGER; + for(const order of plain.length>maxCoins?[(a:ArkCoin,b:ArkCoin)=>expiry(a)-expiry(b),(a:ArkCoin,b:ArkCoin)=>b.value-a.value]:[undefined]){ + let picked=order?[...plain].sort(order).slice(0,maxCoins):plain; + if(exempt&&picked.length>1&&(schedule?.fixedSats||schedule?.bps)&&picked.every(exempt))picked=[picked.reduce((a,b)=>b.value>a.value?b:a)]; + const btc:ShieldPlan={kind:'btc',coins:picked,amount:picked.reduce((sum,c)=>sum+c.value,0)}; + if(shieldableWithFee(btc,schedule,exempt))return btc; + } return usable.filter(c=>c.assets?.length===1&&listed.has(c.assets[0]!.assetId)).map((coin):ShieldPlan=>({kind:'asset',coin,assetId:coin.assets![0]!.assetId,units:coin.assets![0]!.amount})).find(plan=>shieldableWithFee(plan,schedule,exempt)); } diff --git a/tests/rollup-wallet-page.test.ts b/tests/rollup-wallet-page.test.ts index d6c3f811..c8e628e5 100644 --- a/tests/rollup-wallet-page.test.ts +++ b/tests/rollup-wallet-page.test.ts @@ -105,6 +105,13 @@ const flat={fixedSats:100,bps:0},feeAddress=rollupAddressOf({owner:1n,viewPublic const coinOf=(n:number,value:number,assets?:ArkCoin['assets']):ArkCoin=>({txid:String(n).repeat(64).slice(0,64),vout:0,value,...(assets?{assets}:{})}); const planned=(coins:ArkCoin[],fee?:Parameters[4],listed:string[]=[])=>shieldPlan(coins,new Set(listed),0,0,fee); +test('past the coin cap, a fee the soonest coins cannot cover takes the largest coins instead',()=>{ + const soon=Array.from({length:20},(_,i)=>({...coinOf(i+10,330),expiresAt:1_000})),big={...coinOf(9,20_000),expiresAt:2_000}; + const plan=shieldPlan([...soon,big],new Set(),0,0,{schedule:{fixedSats:7_000,bps:0}},20); + assert.ok(plan?.kind==='btc'&&plan.coins.length===20&&plan.coins.includes(big),JSON.stringify(plan?.kind)); + assert.deepEqual(shieldPlan([...soon,big],new Set(),0,0,{},20),{kind:'btc',coins:soon,amount:6_600},'with no fee, the soonest to expire still go first'); +}); + test('with a fee on, a plan whose coins are all pool payouts deposits the largest alone, so it keeps its exemption',()=>{ const [a,b,c]=[coinOf(1,500),coinOf(2,900),coinOf(3,700)],exempt=()=>true; assert.deepEqual(planned([a,b,c],{schedule:flat,exempt}),{kind:'btc',coins:[b],amount:900}); From cdd024a85154e80ebc771fcf064d719ae2f9000e Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 11:08:03 +0100 Subject: [PATCH 32/40] fix(app): offer no Lightning, Bitcoin or renewal action a private wallet cannot carry The browser's private transport starts no Lightning service and refuses batch swaps, so renewal and on-chain exits always failed there. The send hints now name only shielded and Arkade recipients, and the receive view explains why expiring sats cannot be renewed instead of offering a button that only throws. The condition is renamed liveStreams: Lightning and batch swaps both need the live streams this transport does not carry. --- app/src/RollupWallet.tsx | 12 ++++++------ tests/app-private-build.test.ts | 9 +++++++++ 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 7e7be6f3..30d5faed 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -95,8 +95,8 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= const [wallet,setWallet]=useState(),[tick,setTick]=useState(0); const [coins,setCoins]=useState([]),[arkAvailable,setArkAvailable]=useState(),[arkAddress,setArkAddress]=useState(''),[rows,setRows]=useState([]); const [sheet,setSheet]=useState(),[busy,setBusy]=useState(false),[revealed,setRevealed]=useState>({}); - const withLightning=nativeHost||!PRIVATE; - const [tab,setTab]=useState<'home'|'receive'|'send'|'activity'>('home'),[receiveMode,setReceiveMode]=useState(withLightning?'unified':'shielded'),[requestOpen,setRequestOpen]=useState(false),[assetId,setAssetId]=useState(''),[to,setTo]=useState(''),[amount,setAmount]=useState(''),[withSats,setWithSats]=useState(''); + const liveStreams=nativeHost||!PRIVATE; + const [tab,setTab]=useState<'home'|'receive'|'send'|'activity'>('home'),[receiveMode,setReceiveMode]=useState(liveStreams?'unified':'shielded'),[requestOpen,setRequestOpen]=useState(false),[assetId,setAssetId]=useState(''),[to,setTo]=useState(''),[amount,setAmount]=useState(''),[withSats,setWithSats]=useState(''); const [sheetOpen,setSheetOpen]=useState(false),narrow=useMedia('(max-width:760px)'),[scanning,setScanning]=useState(false),[codeError,setCodeError]=useState(''); const [pullRefresh,setPullRefresh]=useState({distance:0,ready:false,refreshing:false}); const pageNode=useRef(null); @@ -741,7 +741,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= {paymentTabs&&
} {tab==='home'?(!docked?
{balanceCard()}{recentPreview}
:null):tab==='activity'?

Activity

{historyContent(historyItems)}{synced&&!rows.length&&}
:<> {tab==='receive'?
- + {receivePaymentValue&&requestedAmount!==undefined&&

Requested {satsText(requestedAmount)}.

} {!receiveBlocked&&receiveMode==='unified'&&<>
Had an address before?
@@ -763,7 +763,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()=
Receive details{feeLine&&

{feeLine}{feePaused&&` ${PAUSED_LINE}`}

}

This unified address pays one Arkade payout address at a time, which this wallet derives and replaces after a payment or once a day. {lightningMinimumLine(fee)} The unified address stays the same and accepts payments while closed. Its service sees each payout address it serves and can link them to this receiver; payers before the next replacement share one address.

} - {!receiveBlocked&&receiveMode==='arkade'&&<>
Fees & test funds

{fundingFee||'Sats sent here move into the pool while this wallet is open.'}{expiring>0&&` ${sats(expiring)} sats expire within a day, so the pool will not take them.`}

Need test sats? Use the Mutinynet faucet after a GitHub sign-in.

{expiring>0&&}} + {!receiveBlocked&&receiveMode==='arkade'&&<>
Fees & test funds

{fundingFee||'Sats sent here move into the pool while this wallet is open.'}{expiring>0&&` ${sats(expiring)} sats expire within a day, so the pool will not take them.`}

Need test sats? Use the Mutinynet faucet after a GitHub sign-in.

{expiring>0&&(liveStreams?:

Private networking cannot renew them yet, because renewal joins a batch swap. Turn private networking off to renew them.

)}} {!receiveBlocked&&paidOutSats>=DUST&&
BACK ON ARKADE FROM THE POOL{sats(paidOutSats)} sats

Withdrawals stay on Arkade until you move them back.{payoutFee}

} {!receiveBlocked&&arkAssetHoldings(coins,c=>!!wallet?.isPoolPayout(c)).map(a=>
{a.payout?'BACK ON ARKADE FROM THE POOL':'ARKADE ASSET'} {short(a.assetId)}{sats(a.units)} units @@ -774,13 +774,13 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= :<>

Not in the pool yet. Listing it sends 1 unit and {DUST} sats to the pool, after which anyone can shield it.

}{!a.payout&&listingError?.id===a.assetId&&

{listingError.message}

}
)}
:
{e.preventDefault();submit();}}> {reserves.length>0&&} -
+
{codeError&&

{codeError}

} {arkAssetRecipient&&}

{synced?`Up to ${sats(maxSend)} ${unit} available privately.`:'Syncing your wallet…'}

-
Where each kind of payment goes

A shielded address pays privately inside the pool. An Arkade address leaves the pool, and the amount and destination become public. A Bitcoin address leaves the pool through a fresh key used only for that payment, so it is not linked to your deposit address. Lightning funds its swap directly from one pool note. A claim link pays whoever opens it first.

{to.trim()&&

Get quote contacts the payment service for Lightning invoices and addresses. It shares the invoice or address and amount; payment happens after you confirm the quote.

}
+
Where each kind of payment goes

A shielded address pays privately inside the pool. An Arkade address leaves the pool, and the amount and destination become public.{liveStreams?' A Bitcoin address leaves the pool through a fresh key used only for that payment, so it is not linked to your deposit address. Lightning funds its swap directly from one pool note.':' Private networking does not carry Bitcoin or Lightning payments yet.'} A claim link pays whoever opens it first.

{liveStreams&&to.trim()&&

Get quote contacts the payment service for Lightning invoices and addresses. It shares the invoice or address and amount; payment happens after you confirm the quote.

}
{quote&&
{routeCopy(quote.railId).title}{routeCopy(quote.railId).detail} {quote.fee>0&&Fee {sats(quote.fee)} sats · total {sats(quote.total)} sats}
} {quoteError&&

{quoteError}

} diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index 27c6abfb..50ad4502 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -30,6 +30,15 @@ test('a private wallet, which has no Lightning, opens receive on shielded and of assert.match(wallet,new RegExp(`]*unified=\\{${lightning}\\}`)); }); +test('a private wallet offers no Lightning, Bitcoin or renewal action its transport cannot carry',()=>{ + const wallet=read('app/src/RollupWallet.tsx'),live=/const (\w+)=nativeHost\|\|!PRIVATE;/.exec(wallet)?.[1]; + assert.ok(live); + assert.ok(wallet.includes(`placeholder={${live}?'shrol21…, tark1…, a Lightning invoice or address':'shrol21… or tark1…'}`)); + assert.ok(wallet.includes(`{${live}?' A Bitcoin address leaves the pool`)); + assert.ok(wallet.includes(`{${live}&&to.trim()&&`)); + assert.match(wallet,new RegExp(`expiring>0&&\\(${live}\\?Renew expiring sats:`)); +}); + test('a private wallet reconciles its journal on every tick and caps its shields',()=>{ const wallet=read('app/src/RollupWallet.tsx'),effect=/useEffect\(\(\)=>\{\s*if\(!wallet\|\|!nativeHost&&!PRIVATE\)return;([\s\S]*?)\},\[([^\]]*)\]\);/.exec(wallet); assert.match(effect?.[1]??'',/if\(!nativeHost\)\{[^\n]*PRIVATE!\.reconcile\(\)/); From f20805adc3253c16a01a80719fea1bf24e2a291f Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 11:20:25 +0100 Subject: [PATCH 33/40] test: give the relay-retry test room for a slow second relay Under a loaded full-suite run, the second relay's answer can take longer than 100 ms, so a third attempt went out and the exact attempt list no longer matched. The budget on main was the same 100 ms; 400 ms keeps the assertion and leaves headroom. --- tests/sealed-fetch.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/sealed-fetch.test.ts b/tests/sealed-fetch.test.ts index bfa0fbdb..68520a0b 100644 --- a/tests/sealed-fetch.test.ts +++ b/tests/sealed-fetch.test.ts @@ -82,7 +82,7 @@ test('a POST shed on two attempts within 30 s hears both statuses and runs on it }); test('a GET whose relay never delivers is retried on another relay',async t=>{ - const g=await rig(t,{pick:1,getTimeoutMs:100});g.bus.relay(A).deliver=false; + const g=await rig(t,{pick:1,getTimeoutMs:400});g.bus.relay(A).deliver=false; assert.equal((await g.fetch(ARK)).status,200); assert.deepEqual(g.sentTo(),[[A],[B]]);assert.equal(g.requests().length,2); assert.notEqual(g.sent()[0]!.content,g.sent()[1]!.content,'each GET attempt is sealed afresh'); From 4ccc208b17203efa4790860da8fedc7cf9f705be Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 11:33:10 +0100 Subject: [PATCH 34/40] fix(shielded): shield the largest pool payout alone when no merged group nets past the fee A pool payout deposited alone pays no fee, but merged with any other coin it pays on the whole amount. When a large fixed fee left every merged group under the dust minimum, auto-shield found no plan, even though the largest payout alone would go in fee-free. That case is reached only when no merged plan works, so nothing changes while the fee is off. --- src/shielded/pool.ts | 10 +++++----- tests/rollup-wallet-page.test.ts | 7 +++++++ 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/src/shielded/pool.ts b/src/shielded/pool.ts index 5886617f..595753fb 100644 --- a/src/shielded/pool.ts +++ b/src/shielded/pool.ts @@ -142,13 +142,13 @@ export const shieldableWithFee=(plan:ShieldPlan,fee:DepositFeeSchedule|undefined const carrier=BigInt(plan.coin.value),free=BigInt(CARRIER_FREE_SATS); return carrier<=free||carrier-owed>=free; }; -/** What auto-shield moves next: every plain coin as one deposit, else one coin of a listed asset. The pool refuses coins near expiry, and a deposit the fee would eat is left out. While a fee charges, pool payouts (`exempt`) go one at a time: a merged coin is a plain send and would pay. Past `maxCoins` plain coins, the soonest to expire go first, or the largest when only they cover the fee. */ +/** What auto-shield moves next: every plain coin as one deposit, else one coin of a listed asset. The pool refuses coins near expiry, and a deposit the fee would eat is left out. While a fee charges, pool payouts (`exempt`) go one at a time: a merged coin is a plain send and would pay. Past `maxCoins` plain coins, the soonest to expire go first, or the largest when only they cover the fee. When no merged group nets past the fee, the largest payout goes alone. */ export function shieldPlan(coins:readonly ArkCoin[],listed:ReadonlySet,now:number,floorMs:number,fee:{schedule?:DepositFeeSchedule;exempt?:(coin:ArkCoin)=>boolean}={},maxCoins=Infinity):ShieldPlan|undefined { const {schedule,exempt}=fee,usable=coins.filter(c=>c.expiresAt===undefined||c.expiresAt-now>floorMs),plain=usable.filter(c=>!c.assets?.length); - const expiry=(c:ArkCoin)=>c.expiresAt??Number.MAX_SAFE_INTEGER; - for(const order of plain.length>maxCoins?[(a:ArkCoin,b:ArkCoin)=>expiry(a)-expiry(b),(a:ArkCoin,b:ArkCoin)=>b.value-a.value]:[undefined]){ - let picked=order?[...plain].sort(order).slice(0,maxCoins):plain; - if(exempt&&picked.length>1&&(schedule?.fixedSats||schedule?.bps)&&picked.every(exempt))picked=[picked.reduce((a,b)=>b.value>a.value?b:a)]; + const expiry=(c:ArkCoin)=>c.expiresAt??Number.MAX_SAFE_INTEGER,largest=(group:ArkCoin[])=>group.reduce((a,b)=>b.value>a.value?b:a),payouts=exempt?plain.filter(exempt):[]; + const groups=[...(plain.length>maxCoins?[[...plain].sort((a,b)=>expiry(a)-expiry(b)),[...plain].sort((a,b)=>b.value-a.value)].map(group=>group.slice(0,maxCoins)):[plain]),...(payouts.length?[[largest(payouts)]]:[])]; + for(let picked of groups){ + if(exempt&&picked.length>1&&(schedule?.fixedSats||schedule?.bps)&&picked.every(exempt))picked=[largest(picked)]; const btc:ShieldPlan={kind:'btc',coins:picked,amount:picked.reduce((sum,c)=>sum+c.value,0)}; if(shieldableWithFee(btc,schedule,exempt))return btc; } diff --git a/tests/rollup-wallet-page.test.ts b/tests/rollup-wallet-page.test.ts index c8e628e5..f00a1bf2 100644 --- a/tests/rollup-wallet-page.test.ts +++ b/tests/rollup-wallet-page.test.ts @@ -112,6 +112,13 @@ test('past the coin cap, a fee the soonest coins cannot cover takes the largest assert.deepEqual(shieldPlan([...soon,big],new Set(),0,0,{},20),{kind:'btc',coins:soon,amount:6_600},'with no fee, the soonest to expire still go first'); }); +test('when no merged group nets past the fee, the largest pool payout goes alone, fee-free',()=>{ + const early=Array.from({length:20},(_,i)=>({...coinOf(i+10,400),expiresAt:1_000})),payout={...coinOf(9,330),expiresAt:2_000}; + assert.deepEqual(shieldPlan([...early,payout],new Set(),0,0,{schedule:{fixedSats:8_000,bps:0},exempt:c=>c===payout},20),{kind:'btc',coins:[payout],amount:330},'past the cap'); + const large={...coinOf(8,5_000)},plainCoin=coinOf(7,400); + assert.deepEqual(shieldPlan([large,plainCoin],new Set(),0,0,{schedule:{fixedSats:10_000,bps:0},exempt:c=>c===large}),{kind:'btc',coins:[large],amount:5_000},'and uncapped'); +}); + test('with a fee on, a plan whose coins are all pool payouts deposits the largest alone, so it keeps its exemption',()=>{ const [a,b,c]=[coinOf(1,500),coinOf(2,900),coinOf(3,700)],exempt=()=>true; assert.deepEqual(planned([a,b,c],{schedule:flat,exempt}),{kind:'btc',coins:[b],amount:900}); From 3e026f3088061e5d372533a293623b8d71ed0099 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 12:46:24 +0100 Subject: [PATCH 35/40] fix(app): say how to turn private networking off when sats cannot be renewed The renewal note told a private wallet's user to turn private networking off but not how, and no link can do it. It now names the local storage key to remove, from the same constant the boot reads, and says that requests then go direct. --- app/src/RollupWallet.tsx | 3 ++- tests/app-private-build.test.ts | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 30d5faed..fd0c660b 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -38,6 +38,7 @@ import {isNativeHost,nativeClaimFetch,nativeOperationFetch,nativeSessionFetch,na import {loadWalletPoseidon} from './wallet-crypto.ts'; import {pendingPoolNullifiers,pendingPoolOutpoints,reconcileNativeMutations} from '../../src/native/mutations.ts'; import {privateTransport,PRIVATE_SHIELD_COINS} from '../../src/shielded/private-transport.ts'; +import {PRIVATE_FLAG} from './private-boot.ts'; const INJECTED_NATIVE=isNativeHost(); const PRIVATE=privateTransport(); @@ -763,7 +764,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()=
Receive details{feeLine&&

{feeLine}{feePaused&&` ${PAUSED_LINE}`}

}

This unified address pays one Arkade payout address at a time, which this wallet derives and replaces after a payment or once a day. {lightningMinimumLine(fee)} The unified address stays the same and accepts payments while closed. Its service sees each payout address it serves and can link them to this receiver; payers before the next replacement share one address.

} - {!receiveBlocked&&receiveMode==='arkade'&&<>
Fees & test funds

{fundingFee||'Sats sent here move into the pool while this wallet is open.'}{expiring>0&&` ${sats(expiring)} sats expire within a day, so the pool will not take them.`}

Need test sats? Use the Mutinynet faucet after a GitHub sign-in.

{expiring>0&&(liveStreams?:

Private networking cannot renew them yet, because renewal joins a batch swap. Turn private networking off to renew them.

)}} + {!receiveBlocked&&receiveMode==='arkade'&&<>
Fees & test funds

{fundingFee||'Sats sent here move into the pool while this wallet is open.'}{expiring>0&&` ${sats(expiring)} sats expire within a day, so the pool will not take them.`}

Need test sats? Use the Mutinynet faucet after a GitHub sign-in.

{expiring>0&&(liveStreams?:

Private networking cannot renew them yet, because renewal joins a batch swap. To renew them, turn private networking off: remove {PRIVATE_FLAG} from this site's local storage, then reload. The wallet's requests then go direct, so the services it uses see this browser's address.

)}} {!receiveBlocked&&paidOutSats>=DUST&&
BACK ON ARKADE FROM THE POOL{sats(paidOutSats)} sats

Withdrawals stay on Arkade until you move them back.{payoutFee}

} {!receiveBlocked&&arkAssetHoldings(coins,c=>!!wallet?.isPoolPayout(c)).map(a=>
{a.payout?'BACK ON ARKADE FROM THE POOL':'ARKADE ASSET'} {short(a.assetId)}{sats(a.units)} units diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index 50ad4502..6957c26f 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -37,6 +37,8 @@ test('a private wallet offers no Lightning, Bitcoin or renewal action its transp assert.ok(wallet.includes(`{${live}?' A Bitcoin address leaves the pool`)); assert.ok(wallet.includes(`{${live}&&to.trim()&&`)); assert.match(wallet,new RegExp(`expiring>0&&\\(${live}\\?Renew expiring sats:`)); + assert.match(wallet,/^import \{PRIVATE_FLAG\} from '\.\/private-boot(\.ts)?';$/m); + assert.ok(wallet.includes('turn private networking off: remove {PRIVATE_FLAG} from this site's local storage, then reload.')); }); test('a private wallet reconciles its journal on every tick and caps its shields',()=>{ From d86fb59e51e78b61e0c58d8787a508800db33a94 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 13:45:38 +0100 Subject: [PATCH 36/40] fix(app): tell testers to drop ?private=1 when turning private networking off The flag is set again on every load whose address carries ?private=1, so removing the stored key and reloading the same link turned private networking straight back on. Both the boot error and the renewal note now say to open the page without it, and so does the runbook. --- app/src/RollupWallet.tsx | 2 +- app/src/private-boot.ts | 2 +- docs/DOKPLOY.md | 2 +- tests/app-private-build.test.ts | 3 ++- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index fd0c660b..895c8f64 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -764,7 +764,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()=
Receive details{feeLine&&

{feeLine}{feePaused&&` ${PAUSED_LINE}`}

}

This unified address pays one Arkade payout address at a time, which this wallet derives and replaces after a payment or once a day. {lightningMinimumLine(fee)} The unified address stays the same and accepts payments while closed. Its service sees each payout address it serves and can link them to this receiver; payers before the next replacement share one address.

} - {!receiveBlocked&&receiveMode==='arkade'&&<>
Fees & test funds

{fundingFee||'Sats sent here move into the pool while this wallet is open.'}{expiring>0&&` ${sats(expiring)} sats expire within a day, so the pool will not take them.`}

Need test sats? Use the Mutinynet faucet after a GitHub sign-in.

{expiring>0&&(liveStreams?:

Private networking cannot renew them yet, because renewal joins a batch swap. To renew them, turn private networking off: remove {PRIVATE_FLAG} from this site's local storage, then reload. The wallet's requests then go direct, so the services it uses see this browser's address.

)}} + {!receiveBlocked&&receiveMode==='arkade'&&<>
Fees & test funds

{fundingFee||'Sats sent here move into the pool while this wallet is open.'}{expiring>0&&` ${sats(expiring)} sats expire within a day, so the pool will not take them.`}

Need test sats? Use the Mutinynet faucet after a GitHub sign-in.

{expiring>0&&(liveStreams?:

Private networking cannot renew them yet, because renewal joins a batch swap. To renew them, turn private networking off: remove {PRIVATE_FLAG} from this site's local storage, then open this page again without ?private=1 in its address. The wallet's requests then go direct, so the services it uses see this browser's address.

)}} {!receiveBlocked&&paidOutSats>=DUST&&
BACK ON ARKADE FROM THE POOL{sats(paidOutSats)} sats

Withdrawals stay on Arkade until you move them back.{payoutFee}

} {!receiveBlocked&&arkAssetHoldings(coins,c=>!!wallet?.isPoolPayout(c)).map(a=>
{a.payout?'BACK ON ARKADE FROM THE POOL':'ARKADE ASSET'} {short(a.assetId)}{sats(a.units)} units diff --git a/app/src/private-boot.ts b/app/src/private-boot.ts index cf8c139d..ad417b17 100644 --- a/app/src/private-boot.ts +++ b/app/src/private-boot.ts @@ -31,7 +31,7 @@ if(typeof __NATIVE_BUILD__!=='undefined'){ const install=__SHIELDED_OHTTP_CONFIG__&&__SHIELDED_NOSTR_GATEWAY__?()=>bootPrivateTransport({config:__SHIELDED_OHTTP_CONFIG__,gateway:__SHIELDED_NOSTR_GATEWAY__,relays:__SHIELDED_NOSTR_RELAYS__,poolOrigin:__SHIELDED_POOL_ORIGIN__},location.search,storage,location.origin):undefined; try{bootPrivate(__NATIVE_BUILD__,install,location.search,storage);} catch(error){ - document.getElementById('root')?.replaceChildren(`Private networking could not start, so the wallet stays closed: ${String(error instanceof Error?error.message:error).replace(/\.$/,'')}. To turn private networking off, remove ${PRIVATE_FLAG} from this site's local storage.`); + document.getElementById('root')?.replaceChildren(`Private networking could not start, so the wallet stays closed: ${String(error instanceof Error?error.message:error).replace(/\.$/,'')}. To turn private networking off, remove ${PRIVATE_FLAG} from this site's local storage, then open this page again without ?private=1 in its address.`); throw error; } } diff --git a/docs/DOKPLOY.md b/docs/DOKPLOY.md index 34943070..72065f88 100644 --- a/docs/DOKPLOY.md +++ b/docs/DOKPLOY.md @@ -98,6 +98,6 @@ A build missing either of the first two has no private transport. The build pins **Turn it on.** On a pinned build, `?private=1` on any page turns the transport on for that browser profile. Use it only with a fresh test wallet. A private wallet has no Lightning, no renewal, no on-chain exit and no explorer links, and it shields at most 20 coins at a time; a transaction id shows as plain text. If the transport cannot start, the page says why and the wallet stays closed. A build from before the private transport, which a rollback can return to, and a tab still running an older version from the app cache both ignore the flag and send every request direct, so tap Update before opening a page with `?private=1`. A build without both pins that sees the flag, in the URL or stored, keeps the wallet closed and says "this build has no private gateway pinned"; to open it, remove the key as under **Turn it off** and load the page without `?private=1`. -**Turn it off.** No link turns it off, so nobody can send a tester a link that quietly returns their wallet to direct requests; `?private=0` does nothing. Delete the `localStorage` key `shielded-private-transport-v1` in the browser's developer tools, or clear the site's data. Clearing the site's data deletes the wallet as well, so back up its recovery phrase first. If the wallet says some payments have unresolved outcomes, you can wait until that line clears before turning it off: without the transport, the wallet no longer keeps those payments' notes reserved. +**Turn it off.** No link turns it off, so nobody can send a tester a link that quietly returns their wallet to direct requests; `?private=0` does nothing. Delete the `localStorage` key `shielded-private-transport-v1` in the browser's developer tools, or clear the site's data, then open the page without `?private=1`, which sets the key again on every load. Clearing the site's data deletes the wallet as well, so back up its recovery phrase first. If the wallet says some payments have unresolved outcomes, you can wait until that line clears before turning it off: without the transport, the wallet no longer keeps those payments' notes reserved. **Keep the pins equal to the gateway's.** The key config and relays a build pins must match the gateway's `SHIELDED_OHTTP_KEY` and `SHIELDED_NOSTR_RELAYS`. After a rotation, a pinned build seals to the old key, and over Nostr the gateway answers that with silence, not a stale-key error, so its wallets stop syncing without saying why. At every rotation, rebuild the app with both new pins, `SHIELDED_OHTTP_CONFIG` and `SHIELDED_NOSTR_GATEWAY` (the Nostr key is derived from the OHTTP key, so it changes too), and have testers tap Update or close every tab of the site, since the app cache serves the old build until then. Keep the gateway's relay list equal to the build's or larger. \ No newline at end of file diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index 6957c26f..fd02d354 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -38,7 +38,8 @@ test('a private wallet offers no Lightning, Bitcoin or renewal action its transp assert.ok(wallet.includes(`{${live}&&to.trim()&&`)); assert.match(wallet,new RegExp(`expiring>0&&\\(${live}\\?Renew expiring sats:`)); assert.match(wallet,/^import \{PRIVATE_FLAG\} from '\.\/private-boot(\.ts)?';$/m); - assert.ok(wallet.includes('turn private networking off: remove {PRIVATE_FLAG} from this site's local storage, then reload.')); + assert.ok(wallet.includes('turn private networking off: remove {PRIVATE_FLAG} from this site's local storage, then open this page again without ?private=1 in its address.')); + assert.ok(read('app/src/private-boot.ts').includes("To turn private networking off, remove ${PRIVATE_FLAG} from this site's local storage, then open this page again without ?private=1 in its address."),'a link with ?private=1 sets the flag again on reload'); }); test('a private wallet reconciles its journal on every tick and caps its shields',()=>{ From 95219833579c3a3151d61c1898bf732c79f0652e Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 19:02:54 +0100 Subject: [PATCH 37/40] fix(app): clear a private reconcile error once a later reconcile succeeds A failed reconcile set the page's pool error, and a later success only updated the unresolved line, so the failure stayed on screen. The tick now remembers the text it set and clears it on success, leaving any other pool error in place. The native branch is unchanged. --- app/src/RollupWallet.tsx | 4 ++-- tests/app-private-build.test.ts | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index 01197397..f31034ee 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -120,7 +120,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= const cacheRef=useRef|undefined>(undefined),sheetRef=useRef|undefined>(undefined); const sessions=useRef(new Set<()=>Promise>()); const refreshRef=useRef<{request:()=>Promise}|undefined>(undefined); - const refreshFailed=useRef(false),handleBusy=useRef(false),unifiedRetry=useRef|undefined>(undefined),unifiedStalledRef=useRef(false); + const refreshFailed=useRef(false),handleBusy=useRef(false),unifiedRetry=useRef|undefined>(undefined),unifiedStalledRef=useRef(false),reconcileError=useRef(''); const actions=useRef|undefined>(undefined); actions.current??=pageActions(()=>{if(mounted.current)setTick(t=>t+1);}); const refreshes=useRef(new WeakMapPromise}>()); @@ -449,7 +449,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= useEffect(()=>{refreshRef.current?.request();},[wallet,tick]); useEffect(()=>{ if(!wallet||!nativeHost&&!PRIVATE)return; - if(!nativeHost){let live=true;void PRIVATE!.reconcile().then(pending=>{if(live)setNativePending(pending);},error=>{if(live)setPoolError(errorText(error));});return()=>{live=false;};} + if(!nativeHost){let live=true;void PRIVATE!.reconcile().then(pending=>{if(!live)return;setNativePending(pending);setPoolError(e=>e===reconcileError.current?'':e);},error=>{if(live)setPoolError(reconcileError.current=errorText(error));});return()=>{live=false;};} try{setNativePending(pendingPoolNullifiers(POOL_ORIGIN).length>0||pendingPoolOutpoints(POOL_ORIGIN).length>0);} catch(error){setPoolError(errorText(error));} },[wallet,tick,nativeHost]); diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index fd02d354..e1f49059 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -45,6 +45,7 @@ test('a private wallet offers no Lightning, Bitcoin or renewal action its transp test('a private wallet reconciles its journal on every tick and caps its shields',()=>{ const wallet=read('app/src/RollupWallet.tsx'),effect=/useEffect\(\(\)=>\{\s*if\(!wallet\|\|!nativeHost&&!PRIVATE\)return;([\s\S]*?)\},\[([^\]]*)\]\);/.exec(wallet); assert.match(effect?.[1]??'',/if\(!nativeHost\)\{[^\n]*PRIVATE!\.reconcile\(\)/); + assert.match(effect?.[1]??'',/setPoolError\(e=>e===reconcileError\.current\?'':e\)[^\n]*setPoolError\(reconcileError\.current=errorText\(error\)\)/,'a reconcile that recovers clears its own error and no other'); assert.match(effect?.[2]??'',/\btick\b/); assert.match(wallet,/:PRIVATE\?\{[^\n]*maxShieldCoins:PRIVATE_SHIELD_COINS/); }); From ae139c296a5aa9912d70e1861cd130a3e8348741 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 19:02:56 +0100 Subject: [PATCH 38/40] test: bound the sealed x-date by the clock around the fetch The check compared the rounded date with the time after the response and allowed one second of slack, so a slow runner could fail it near a rounding boundary. --- tests/sealed-fetch.test.ts | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/sealed-fetch.test.ts b/tests/sealed-fetch.test.ts index 5c9bd055..9d92cb2c 100644 --- a/tests/sealed-fetch.test.ts +++ b/tests/sealed-fetch.test.ts @@ -272,10 +272,10 @@ test('abort closes the attempt',async t=>{ }); test('every sealed request carries an x-date rounded to the nearest 30 s, and x-digest crosses',async t=>{ - const g=await rig(t); + const g=await rig(t),before=Date.now(); assert.equal((await g.fetch(ARK,{headers:{'x-digest':'abc',authorization:'x','x-sdk-version':'ts-sdk/0.5.3'}})).status,200); - const {headers}=await g.inner(),at=Date.parse(headers.get('x-date')!); - assert.ok(at%30_000===0&&Math.abs(Date.now()-at)<=16_000,String(headers.get('x-date'))); + const after=Date.now(),{headers}=await g.inner(),at=Date.parse(headers.get('x-date')!); + assert.ok(at%30_000===0&&at>=before-15_000&&at<=after+15_000,String(headers.get('x-date'))); assert.deepEqual([headers.get('x-digest'),headers.has('authorization'),headers.has('x-sdk-version')],['abc',false,false]); }); From 42a2f9db2245e447a31f9a2e239def5e39e68143 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 21:21:06 +0100 Subject: [PATCH 39/40] fix(app): keep a private journal error apart from the page's pool error The previous fix still wrote the reconcile failure into the pool error, so it could replace a startup error and a later success could clear an unrelated error with the same text. The journal error now has its own state and line, a later success clears only it, and a failure keeps the unresolved line on, since the journal still reserves those inputs. --- app/src/RollupWallet.tsx | 7 ++++--- tests/app-private-build.test.ts | 5 ++++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/app/src/RollupWallet.tsx b/app/src/RollupWallet.tsx index f31034ee..c94a2c7b 100644 --- a/app/src/RollupWallet.tsx +++ b/app/src/RollupWallet.tsx @@ -86,7 +86,7 @@ const methodsFailure=(error:unknown)=>[METHODS_REFUSED,METHODS_NOT_WITHDRAWN].in const focusOnMount=(el:HTMLElement|null)=>{el?.focus();}; export default function RollupWallet({storage=globalThis.localStorage,reload=()=>location.reload(),onShutdown,onReady,onStartupFailure,onStartupStage,onClearWallet,nativeHost=INJECTED_NATIVE,paymentRequest,onPaymentRequestHandled,animateEntrance=true,suspended=false,viewActive=true,hidden=false}:{storage?:Storage;reload?:()=>void;onShutdown?:(shutdown:(()=>Promise)|undefined)=>void;onReady?:()=>void;onStartupFailure?:(message:string)=>void;onStartupStage?:(message:string)=>void;onClearWallet?:()=>Promise;nativeHost?:boolean;paymentRequest?:IncomingPayment;onPaymentRequestHandled?:(id:number)=>void;animateEntrance?:boolean;suspended?:boolean;viewActive?:boolean;hidden?:boolean}={}){ const [sessionFetch]=useState(()=>nativeHost?nativeSessionFetch():undefined); - const [pool,setPool]=useState(),[poolError,setPoolError]=useState(''); + const [pool,setPool]=useState(),[poolError,setPoolError]=useState(''),[journalError,setJournalError]=useState(''); const [nativePending,setNativePending]=useState(false); const [startupError,setStartupError]=useState(''),[startupFailure,setStartupFailure]=useState(''),[refreshError,setRefreshError]=useState(''),[feeNotice,setFeeNotice]=useState(''),[feeConfirmed,setFeeConfirmed]=useState(false); const [startupStage,setStartupStage]=useState('Preparing your private wallet'); @@ -120,7 +120,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= const cacheRef=useRef|undefined>(undefined),sheetRef=useRef|undefined>(undefined); const sessions=useRef(new Set<()=>Promise>()); const refreshRef=useRef<{request:()=>Promise}|undefined>(undefined); - const refreshFailed=useRef(false),handleBusy=useRef(false),unifiedRetry=useRef|undefined>(undefined),unifiedStalledRef=useRef(false),reconcileError=useRef(''); + const refreshFailed=useRef(false),handleBusy=useRef(false),unifiedRetry=useRef|undefined>(undefined),unifiedStalledRef=useRef(false); const actions=useRef|undefined>(undefined); actions.current??=pageActions(()=>{if(mounted.current)setTick(t=>t+1);}); const refreshes=useRef(new WeakMapPromise}>()); @@ -449,7 +449,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= useEffect(()=>{refreshRef.current?.request();},[wallet,tick]); useEffect(()=>{ if(!wallet||!nativeHost&&!PRIVATE)return; - if(!nativeHost){let live=true;void PRIVATE!.reconcile().then(pending=>{if(!live)return;setNativePending(pending);setPoolError(e=>e===reconcileError.current?'':e);},error=>{if(live)setPoolError(reconcileError.current=errorText(error));});return()=>{live=false;};} + if(!nativeHost){let live=true;void PRIVATE!.reconcile().then(pending=>{if(!live)return;setNativePending(pending);setJournalError('');},error=>{if(!live)return;setNativePending(true);setJournalError(errorText(error));});return()=>{live=false;};} try{setNativePending(pendingPoolNullifiers(POOL_ORIGIN).length>0||pendingPoolOutpoints(POOL_ORIGIN).length>0);} catch(error){setPoolError(errorText(error));} },[wallet,tick,nativeHost]); @@ -697,6 +697,7 @@ export default function RollupWallet({storage=globalThis.localStorage,reload=()= {wallet&&feePaused&&
{PAUSED_TITLE}{depositFeeBanner(feePaused,wallet.acceptedDepositFee)}

} {feeConfirmed&&!feePaused&&
New fee accepted.
} {ready&&poolError&&

{poolError}

} + {ready&&journalError&&

{journalError}

} {ready&&pullSyncError&&

{pullSyncError}

} {ready&&feeNotice&&

{feeNotice}

} {ready&&startupError&&

{startupError}

} diff --git a/tests/app-private-build.test.ts b/tests/app-private-build.test.ts index e1f49059..9a49b266 100644 --- a/tests/app-private-build.test.ts +++ b/tests/app-private-build.test.ts @@ -45,7 +45,10 @@ test('a private wallet offers no Lightning, Bitcoin or renewal action its transp test('a private wallet reconciles its journal on every tick and caps its shields',()=>{ const wallet=read('app/src/RollupWallet.tsx'),effect=/useEffect\(\(\)=>\{\s*if\(!wallet\|\|!nativeHost&&!PRIVATE\)return;([\s\S]*?)\},\[([^\]]*)\]\);/.exec(wallet); assert.match(effect?.[1]??'',/if\(!nativeHost\)\{[^\n]*PRIVATE!\.reconcile\(\)/); - assert.match(effect?.[1]??'',/setPoolError\(e=>e===reconcileError\.current\?'':e\)[^\n]*setPoolError\(reconcileError\.current=errorText\(error\)\)/,'a reconcile that recovers clears its own error and no other'); + const privateBranch=/if\(!nativeHost\)\{[^\n]*/.exec(effect?.[1]??'')?.[0]??''; + assert.match(privateBranch,/setNativePending\(pending\);setJournalError\(''\)[^\n]*setNativePending\(true\);setJournalError\(errorText\(error\)\)/,'a failed reconcile keeps the unresolved line and its own error; a later success clears only that'); + assert.doesNotMatch(privateBranch,/setPoolError/,'a journal error never replaces a startup error'); + assert.match(wallet,/\{ready&&journalError&&

\{journalError\}<\/p>\}/); assert.match(effect?.[2]??'',/\btick\b/); assert.match(wallet,/:PRIVATE\?\{[^\n]*maxShieldCoins:PRIVATE_SHIELD_COINS/); }); From 81abbe29e724432bb64b156546917eb6194208a2 Mon Sep 17 00:00:00 2001 From: "Andrew Camilleri (Kukks)" Date: Wed, 11 Nov 2026 21:21:10 +0100 Subject: [PATCH 40/40] docs(dokploy): list replay protection across restarts as a production prerequisite --- docs/DOKPLOY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/DOKPLOY.md b/docs/DOKPLOY.md index 426de6e7..aac264ef 100644 --- a/docs/DOKPLOY.md +++ b/docs/DOKPLOY.md @@ -28,7 +28,7 @@ The browser wallet runs from a service worker cache. Worker registrations revali The pool server can open Oblivious HTTP (RFC 9458) requests, so a wallet reaches the pool, arkd, the Arkade indexer and Esplora through a relay that sees its address but not what it asks for. The gateway is off until `SHIELDED_OHTTP_KEY` is set. While it is off, `GET /api/rollup/ohttp-keys` and `POST /api/rollup/ohttp` answer 404 once the pool has opened. Before that they get the pool's 503 instead: "The rollup pool is starting.", or "The rollup pool is not open yet" while the pool is not open, for example while a rollout waits for the volume lock. -**Turn it on.** For testing only for now. Production use waits for two things: a measurement of CPU and memory under a flood on the real host, and a way to rotate the key without stranding wallets. The Nostr carrier lets anyone reach the gateway through public relays, so there is no list of callers to hold it to. Every event a listed relay forwards, a flooder's included, costs the pool's main thread work as it arrives, and every answer costs signing and bandwidth going out. Rotation is a hard cut (see **Rotate**), so before any wallet pins a config, the gateway needs to hold the old and new keys together, or wallets need a stale-key signal they act on. Run `npm run ohttp:keygen -- ` on a trusted machine. It writes the line `SHIELDED_OHTTP_KEY=:` to `` (mode 0600, which holds on Linux and macOS but not on Windows, and it refuses to overwrite an existing file) and prints only the key id, the config and the gateway's Nostr key (`nostr: `), which are public. Paste the line into the application's runtime **Environment**, delete the file and redeploy. Never put the key in a command line, a log, a chat or the repository. Publish the printed config and Nostr key: the wallet build that uses the gateway pins both. +**Turn it on.** For testing only for now. Production use waits for three things: a measurement of CPU and memory under a flood on the real host, a way to rotate the key without stranding wallets, and replay protection that survives a restart. The Nostr carrier lets anyone reach the gateway through public relays, so there is no list of callers to hold it to. Every event a listed relay forwards, a flooder's included, costs the pool's main thread work as it arrives, and every answer costs signing and bandwidth going out. Rotation is a hard cut (see **Rotate**), so before any wallet pins a config, the gateway needs to hold the old and new keys together, or wallets need a stale-key signal they act on. A new process starts with an empty replay register, so a copied request dated just before it started can run once more (see **Start and stop**); production needs that register kept across restarts, or every request it forwards made safe to run twice. Run `npm run ohttp:keygen -- ` on a trusted machine. It writes the line `SHIELDED_OHTTP_KEY=:` to `` (mode 0600, which holds on Linux and macOS but not on Windows, and it refuses to overwrite an existing file) and prints only the key id, the config and the gateway's Nostr key (`nostr: `), which are public. Paste the line into the application's runtime **Environment**, delete the file and redeploy. Never put the key in a command line, a log, a chat or the repository. Publish the printed config and Nostr key: the wallet build that uses the gateway pins both. **Verify.** The start log shows `Oblivious HTTP gateway on (key id N, Nostr 3c857963…, 6 relays, HTTP route off).`, with the key id and the first 8 characters of the Nostr key the tool printed. `GET /api/rollup/ohttp-keys` then returns content type `application/ohttp-keys`, and its bytes equal the printed config. This prints the content type and the bytes in the tool's base64url encoding: