feat(racer-object): add S3 origin and HTTP sidecar

[Jordan Olshevski (jveski)]

Add the racer-object process with a S3 client and server implementation, more APIs coming soon.

The idea is to bridge between Racer's low-level UDS API and higher level applications using efficient splice() syscalls under the hood.

[Copilot]

🟡 Changes recommended

The origin retains AWS SDK checksum defaults that can break supported S3-compatible backends.

1 open finding

• Disable optional S3 response checksum negotiation · New

What changed in this PR

Adds an S3-backed Racer origin and pod-local HTTP sidecar with deployment, image, diagnostics, and usage support.

Changes:

• Implements conditional, ranged S3 object reads through Racer.
• Adds CLI commands, profiling, packaging, and container builds.
• Provides tested Kubernetes examples and documentation.

File	Description
Makefile	Adds build, test, and image targets.
internal/​racerobject/​racerobject.go	Implements the origin and HTTP sidecar.
internal/​racerobject/​racerobject_test.go	Tests origin and sidecar behavior.
images/​racer-object/​Dockerfile	Defines the runtime image.
docs/​content/​guides/​racer-object.md	Documents deployment and usage.
deploy/​racer-object/​sidecar-pod.yaml	Adds an example sidecar pod.
deploy/​racer-object/​packaging_test.go	Tests image workflow integration.
deploy/​racer-object/​origin.yaml	Adds an example origin DaemonSet.
deploy/​racer-object/​manifests_test.go	Validates deployment security and configuration.
deploy/​racer-object/​cache.yaml	Defines the example ClusterCache.
cmd/​racer-object/​README.md	Documents diagnostics and profiling.
cmd/​racer-object/​main.go	Implements CLI configuration and servers.
cmd/​racer-object/​main_test.go	Tests CLI and lifecycle behavior.
cmd/​racer-object/​listener.go	Adds a connection-limiting listener.
cmd/​racer-object/​listener_test.go	Tests listener lifecycle and streaming.
cmd/​racer-object/​debug.go	Implements the diagnostic server.
cmd/​racer-object/​debug_test.go	Tests diagnostic routes and lifecycle.
.github/​workflows/​images.yaml	Supports Dockerfile-based image builds.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

The large new security-sensitive S3, HTTP, Racer, and Kubernetes integration warrants final human validation despite extensive tests.

0 open findings

1 resolved since last review

• Disable optional S3 response checksum negotiation

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The origin timeout flag does not bound complete AWS operations as documented.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Request timeout ignores retries and the configured operation deadline

cmd/​racer-object/​main.go:250

--request-timeout is described as an origin request timeout, but setting it only on http.Client.Timeout bounds each AWS HTTP attempt. SDK retries can exceed the configured value, while racersdk.ServeOrigin still applies its fixed 60-second operation deadline (pkg/racersdk/origin.go:111-117), so values above one minute have no effect. Propagate this setting through the operation context (including the returned GET body), or rename and validate it as a per-attempt timeout.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Published images currently retain unknown source revision and build metadata, reducing artifact provenance.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Pass commit and build timestamp metadata into published images

images/​racer-object/​Dockerfile:13

The image workflow now discovers this Dockerfile, but it passes only VERSION and CONTAINER_REGISTRY (.github/workflows/images.yaml:143-145). Consequently every image published through that workflow keeps these defaults: racer-object --version reports an unknown commit/build time, and the OCI revision label is unknown. Pass the workflow SHA (and build timestamp, if retained in the version output) into the build so published artifacts have usable provenance.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Request timeouts cannot reliably return 503 responses, and published images lose commit/build provenance.

0 open findings

Previously missed (2)

In code that hasn't changed since last review

Generic build omits Docker image commit and build timestamp

.github/​workflows/​images.yaml:66

This fallback newly sends images/racer-object/Dockerfile through the generic build job, but that job passes only VERSION and CONTAINER_REGISTRY. Consequently the Dockerfile's GIT_COMMIT and BUILD_TIME remain unknown, so published images have an unknown OCI revision and racer-object --version cannot identify the source build. Pass the workflow SHA and a build timestamp as build arguments, as the local Make target already does.

WriteTimeout expires before handler can send 503 response

cmd/​racer-object/​main.go:409

WriteTimeout starts when net/http finishes reading the request headers, before the handler creates requestCtx. Therefore, when requestCtx expires during Stat or Get, the connection's write deadline has already expired and the intended 503 response cannot be sent; clients receive a connection failure instead. Keep the handler timeout shorter than the socket write deadline, or remove the server-level write timeout and rely on Object.WriteTo's context-aware write deadlines.

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Ranged HEAD responses currently diverge from S3 semantics by returning 206 and Content-Range.

1 open finding

• Return 200 for ranged HEAD requests instead of 206 · New

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The new security-sensitive data path spans S3 consistency, HTTP semantics, socket lifecycle, image publishing, and Kubernetes deployment, warranting final human validation.

0 open findings

1 resolved since last review

• Return 200 for ranged HEAD requests instead of 206

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The new security-sensitive S3-to-cache-to-HTTP path warrants human validation, and one troubleshooting-documentation issue remains.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Remove misleading origin logs troubleshooting advice

docs/​content/​guides/​racer-object.md:216

This troubleshooting step points users to logs that the adapter does not emit for read failures: the AWS logger is discarded in cmd/racer-object/main.go:262, and origin callback errors are only classified and returned over Racer. Remove the origin logs advice (or add sanitized request-failure logging) so operators are not sent to an empty diagnostic source.

🧠 Review effort: Balanced

[Jordan Olshevski (jveski)]

Copilot resolve the merge conflicts in this pull request

[Copilot]

🔵 Needs a closer look

The new security-sensitive S3, HTTP streaming, socket, and shutdown paths warrant final human validation despite extensive tests.

0 open findings

1 resolved since last review

• Handle all typed-nil S3Client implementations in NewOrigin

🧠 Review effort: Balanced

[Philip Lombardi (plombardi89)]

Follow-up at 87c7065d. Thanks for the fixes; most of my earlier comments are addressed. The two items below are the Major ones that still need work. One thread was resolved without a code or doc change, so I've re-raised it here as a new thread. I haven't repeated the three threads that are still open (VersionId pinning, delete-marker 405, invalid Range).

Checked by reading the code at this head; I did not run tests.

[Copilot]

🟡 Changes recommended

Private SDK error text can reach logs, final response flushing can lose its deadline, and empty-stream failures bypass reporting.

3 open findings

• Prevent private AWS error details from leaking in debug logs · New
• Restore response deadline before net/http flushes GET responses · New
• Report empty-stream completion failures through the SDK failure handler · New

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The new streaming gateway’s version consistency and lifecycle handling warrant human integration review before approval.

0 open findings

3 resolved since last review

• Prevent private AWS error details from leaking in debug logs
• Report empty-stream completion failures through the SDK failure handler
• Restore response deadline before net/http flushes GET responses

Previously missed (1)

In code that hasn't changed since last review

Reversed byte ranges at int64 boundary incorrectly return 416

internal/​racer/​object/​sidecar.go:599

sidecarRangeNumber saturates both bounds to MaxInt64, so this comparison loses their original ordering. For a 10-byte object, Range: bytes=9223372036854775808-9223372036854775807 therefore returns 416 instead of ignoring the reversed range and serving the full object with 200. This contradicts the parser's documented behavior and the ordinary reversed-range case in sidecar_test.go:319. Compare the validated decimal bounds without saturation (ignoring leading zeros), and add a regression case at the int64 boundary.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The new streaming, transport ownership, and shutdown boundaries warrant human review and runtime validation.

0 open findings

🧠 Review effort: Balanced