[williamthome]

Description

Updates roadrunner from v0.13.1 to v0.15.0, replacing #1740 (v0.14.2). From 0.14, roadrunner accepts new connections without waiting for each connection process to start, so many clients connecting at once to a busy server are accepted sooner; WebSocket compression is off by default; and WebSocket sessions read with a buffer that grows only for large frames, so the entry drops its fixed 4 KB ws.recv_buffer and uses the default. From 0.15, plain-TCP connections wait with a 16 KB read buffer instead of 64 KB and grow it only when a large body arrives, and QUIC sockets no longer set reuseaddr, so an HTTP/3 port is never shared with another socket. No option the entry sets changed meaning. The roadrunner-gateway entry builds from frameworks/roadrunner, so it picks up the new version too.

---

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command	Description
/benchmark -f <framework>	Run every test the framework subscribes to
/benchmark -f <framework> -t <test>	Run one test only
/benchmark -f <framework> --save	Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save	Run one test and save results
/benchmark -f <framework> --compare <other>	Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,...	Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save	No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test>	Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp --compare genhttp-kestrel

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.

---

Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk) are built as self-contained Docker images on first run.

[williamthome]

/benchmark-multiple -f roadrunner,roadrunner-gateway

[github-actions]

👋 Benchmark request received. A collaborator will review and approve the run.

[github-actions]

Benchmark Results

Frameworks: 2 | Test: all tests

✅ roadrunner

Test	Conn	RPS	Rate	CPU/req	p99	p99.9	CPU	Mem	Δ RPS	Δ Mem
baseline	4096	1,196,515	—	—	—	—	6440.2%	355MiB	+0.6%	-1.4%
pipelined	4096	5,350,051	—	—	—	—	6265.4%	278MiB	-8.1%	+14.9%
limited-conn	4096	533,883	—	—	—	—	3645.9%	276MiB	+2.9%	+19.0%
json-comp	4096	75,242	—	—	—	—	6251.6%	774MiB	-32.3%	+20.7%
json-comp	16384	101,393	—	—	—	—	6441.8%	766MiB	-3.2%	+10.5%
json-tls	4096	178,866	—	—	—	—	6241.6%	868MiB	+8.0%	+0.6%
8gbit	512	48,494	0.9699 ⚠️	284.63us	837.0us	6786.0us	1325.0%	476MiB	~0%	+0.2%
static-tls	1024	83,842	—	—	—	—	6058.0%	666MiB	+0.5%	-6.3%
async-db	1024	127,766	—	—	—	—	6378.1%	830MiB	+4.1%	+14.6%
fortunes	1024	70,107	—	—	—	—	6340.0%	1.2GiB	-0.6%	+34.3%
baseline-h2	256	605,127	—	—	—	—	6657.6%	349MiB	-0.1%	-3.1%
baseline-h2	1024	607,720	—	—	—	—	6447.7%	479MiB	~0%	~0%
static-h2	256	78,994	—	—	—	—	6079.2%	509MiB	-0.5%	-3.6%
static-h2	1024	72,104	—	—	—	—	6187.2%	1.0GiB	-4.4%	~0%
baseline-h2c	256	963,979	—	—	—	—	6683.2%	264MiB	+0.1%	+0.4%
baseline-h2c	1024	951,170	—	—	—	—	6544.1%	338MiB	+0.2%	-8.9%
baseline-h2c	4096	864,556	—	—	—	—	6478.0%	609MiB	-0.5%	-3.5%
json-h2c	1024	111,080	—	—	—	—	3785.5%	960MiB	+0.8%	+7.3%
json-h2c	4096	107,372	—	—	—	—	4424.7%	2.1GiB	+4.1%	+61.5%
baseline-h3	64	184,480	—	—	—	—	3831.3%	334MiB	+0.2%	+3.7%
static-h3	64	40,969	—	—	—	—	3244.8%	1.3GiB	+9.2%	~0%
echo-ws	512	1,952,649	—	—	—	—	6717.8%	192MiB	-0.9%	-1.5%
echo-ws	4096	1,902,553	—	—	—	—	6508.9%	249MiB	-2.9%	-6.4%
echo-ws	16384	1,778,818	—	—	—	—	6670.6%	385MiB	-4.6%	+12.6%
echo-ws-pipeline	512	16,878,201	—	—	—	—	6581.7%	218MiB	-5.1%	+14.7%
echo-ws-pipeline	4096	16,973,788	—	—	—	—	6677.8%	294MiB	+1.2%	-3.3%
echo-ws-pipeline	16384	15,775,644	—	—	—	—	6693.7%	360MiB	-3.9%	+9.4%
echo-ws-limited	512	534,521	—	—	—	—	3234.0%	206MiB	-0.5%	-13.4%
echo-ws-limited	4096	535,131	—	—	—	—	3130.5%	204MiB	-0.5%	-24.4%
latency-1m	1024	997,314	0.9973	63.06us	23384.0us	41584.0us	6547.9%	411MiB	~0%	-14.7%
latency-10k	1024	9,978	0.9978	110.66us	148.0us	11476.0us	106.1%	199MiB	~0%	+3.1%
latency-500k-8cpu	1024	177,421	0.3548 ⚠️	45.00us	12865536.0us	13094912.0us	772.7%	142MiB	+1.2%	-2.7%
async	32000	1,114,988	—	—	—	—	6310.7%	957MiB	-2.5%	+6.5%

Composite score, main against this run, in the board's default view:

Family	main	this run	Δ
HTTP/1.1	2566.7 (58 of 107)	2526.9 (58 of 107)	-39.8
HTTP/2	273.1 (34 of 40)	272.2 (34 of 40)	-0.9
HTTP/3	166.1 (17 of 19)	178.1 (17 of 19)	+11.9
Gateway	1834.7 (5 of 9)	1807.1 (5 of 9)	-27.6
WebSocket	1018.0 (23 of 38)	995.4 (23 of 38)	-22.5

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

✅ roadrunner-gateway

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
gateway-64	512	155,690	6055.1%	2.8GiB	+0.9%	-3.4%
gateway-64	1024	150,557	5979.3%	4.2GiB	~0%	-8.7%
gateway-h3	64	80,259	5317.1%	777MiB	-1.2%	-0.8%
gateway-h3	256	58,802	4408.3%	2.0GiB	+1.2%	+17.6%
production-stack	256	224,824	5684.8%	1.5GiB	-4.8%	~0%
production-stack	1024	192,817	5721.1%	2.4GiB	-1.2%	+14.3%

[williamthome]

/benchmark -f roadrunner -t json-comp