State-of-the-art phone-as-light-gun aiming for browser games.
The most complete portable stack for turning a phone into an absolute pointer: cross-browser sensor fusion (including the WebKit gyro-axis bug), gnomonic-plane calibration, latency-aware prediction, and a host session you drop into a game. You bring the transport. Safari has no WebXR AR — this is the path that actually ships on iOS and Android.
import { MotionPipeline } from "gyro-aim/browser";
import { AimSession } from "gyro-aim";
const motion = new MotionPipeline({
onSample: (sample) => sendToHost(sample),
});
await motion.requestPermission(); // must run from a tap on iOS
motion.start();
const aim = new AimSession({ screen: [1920, 1080] });
aim.ingest(sample);
const [x, y] = aim.update(dt).aim;Browser gyro aiming has been stuck at Euler DeviceOrientation wrappers (gymote) or Android-only ARCore apps (point-bang). Neither is a library you can put under a multiplayer TV game on Safari. gyro-aim is.
| gyro-aim | gymote | DeviceOrientation | point-bang | |
|---|---|---|---|---|
| iOS Safari | yes | Euler only | yes | no |
| Android Chrome | yes | Euler only | yes | ARCore |
| Gyro bias learning | yes | ARCore VIO | ||
| WebKit vs spec axis discovery | yes | n/a | ||
| Screen calibration | 9-point DLT + affine fallback | 3-corner WebXR | ||
| Grip / muzzle detection | yes | |||
| Prediction (quadratic rate + quality) | yes | ARCore | ||
| One Euro in aim-degrees | yes | |||
| Shot-time pose reconstruction | yes | |||
| Open-source library | yes | yes | n/a | app |
ARCore visual-inertial tracking is more accurate on phones that have it. It does not exist in Safari. For a game that has to run from a QR code on a TV, fusion plus a calibrated homography is the state of the art.
Every figure below is a gate in npm test, not a brochure measurement. The phone is simulated: 1.8°/−2.4°/0.3°/s gyro bias, 0.35°/s white gyro noise, 1.5°/s hand tremor, 0.12 m/s² accel noise. Screen diagonal ≈ 2203 px. Viewing distance in the model is 3600 px (~3.3 screen-heights — a couch).
| Metric | Gate | What it means |
|---|---|---|
| Hold-out aim error | < 45 px (~2.0% of diagonal, ~0.7°) | After 9-point calib, aim at points never used in the fit |
| Repeatability | < 60 px | Same probe, three independent noise seeds |
| Calibration leave-one-out | < 3% of diagonal | Worst held-out target under bias + tremor |
| Perfect-capture mapping | < 1 px | 9 consistent poses; also untrained interior points |
| Unfiltered hold jitter | < 30 px peak over 3 s | Tremor + noise, before One Euro / still-lock |
| Fusion vs 60 s of 2–3°/s bias | < 0.5° | Unfused integration would be > 200° off |
| Resting phone, 100 s | < 10⁻⁶ °, no NaN | Madgwick’s normalized gradient blows up here |
| Prediction vs truth | < 2% of flick travel | 40 ms horizon; body-frame integrate. Wrong composition is > 40 px off |
| Quadratic rate fit | < 2° at 40 ms | Steady 90°/s yaw; still-hold invents < 1° over 50 ms |
| Trigger interpolation | < 0.5° | Slerp to the shot timestamp, not the last frame |
| Sensor rate | 120 Hz in, ≤ 125 Hz out | Emitted from callbacks, never resampled |
Sources: tests/pipeline.test.ts, tests/fusion.test.ts, tests/calibration.test.ts, tests/aiming.test.ts.
A live AimBench on the host reports static RMS, moving residual, and yaw drift °/min so you can prove the same things on a real phone.
Phone gyro aiming looks simple until you ship it. Safari only grants motion from a user gesture. WebKit labels rotationRate differently from the spec. Gravity cannot hold heading, so yaw drifts. A projective map fitted from four noisy corners can place a singularity just off-screen and slam the crosshair into an edge. Pixel-space smoothing makes the aim feel different in the corners than in the middle.
gyro-aim is the stack that survived those problems in a real TV light-gun game:
| Problem | What the library does |
|---|---|
| iOS / Android sensor chaos | Generic Sensor API first, DeviceOrientation + DeviceMotion fallback, online gyro-axis discovery |
| Gyro bias of 2–4°/s at rest | Learned while stationary, persisted through an injectable storage key |
| Yaw unobservable from gravity | Recentre, plus optional drift learning from unambiguous shots |
| Lag vs jitter | Prediction from a noise-adaptive quadratic rate fit; One Euro in degrees, not pixels |
| Calibration that blows up | Hartley-normalized DLT, affine fallback, safe-domain clamp, grip detection |
It is a library to build games, not a product. No WebRTC, no QR, no HUD. Those belong to the host.
Safari has no WebXR AR, so this is the portable path: complementary fusion that runs on iOS and Android, not ARCore-only tracking.
flowchart LR
subgraph phone [Phone]
Sensors["Sensors API / DeviceOrientation"]
Pipeline[MotionPipeline]
Sensors --> Pipeline
end
subgraph wire [Your transport]
Sample["AimSample { t, q, w, nq }"]
end
subgraph host [TV / desktop]
Session[AimSession]
Screen[Screen pixels]
Pipeline --> Sample --> Session --> Screen
end
Two entry points, so Node tests and the host never import DOM sensors:
| Import | Runs in | Contains |
|---|---|---|
gyro-aim |
Node and browser | Math, fusion, mapping, clock, AimSession |
gyro-aim/browser |
Browser only | MotionPipeline |
npm install gyro-aimESM-only, TypeScript types included. The only runtime dependency is ml-matrix (SVD for the calibration homography).
import { MotionPipeline } from "gyro-aim/browser";
const motion = new MotionPipeline(
{
onSample: (sample) => {
// sample.t — performance.now() at the sensor callback
// sample.q — unit quaternion [x, y, z, w]
// sample.w — body-frame angular velocity, rad/s
// sample.nq — 0–1 trust in the fitted rate
dataChannel.send(JSON.stringify(sample));
},
onMode: (mode) => console.log("sensor mode", mode),
},
{ biasStorageKey: "my-game.gyroBias.v1" },
);
button.addEventListener("click", async () => {
const granted = await motion.requestPermission();
if (!granted) return;
motion.start();
});Call requestPermission() inside the tap handler before any await that is not the permission call itself. iOS will not treat a later continuation as a user gesture.
Samples are emitted from the sensor callbacks, not resampled on a timer. Resampling duplicates and skips readings, and stamping them at timer time injects jitter straight into prediction.
import { AimSession, CALIB_FULL_COUNT } from "gyro-aim";
const aim = new AimSession({
screen: [window.innerWidth, window.innerHeight],
});
function onSample(sample) {
aim.ingest(sample);
}
function onFrame(dt: number) {
const frame = aim.update(dt, performance.now(), ducks);
crosshair.style.transform = `translate(${frame.aim[0]}px, ${frame.aim[1]}px)`;
}
function onTrigger(controllerTime: number) {
const fired = aim.aimAt(controllerTime, performance.now(), ducks);
aim.learnDrift(ducks);
aim.noteShot(fired, ducks, hitscan(fired, ducks));
}update is for the visible crosshair. aimAt reconstructs pose at the trigger timestamp from sample history, so a shot is not late by whatever the last animation frame happened to be.
Uncalibrated aim is a gnomonic plane scaled to the screen — good enough to move a cursor, not good enough to hit a duck in the corner. Calibration fits a mapping from that plane to pixels.
aim.beginCalibration({ total: CALIB_FULL_COUNT }); // 9 points
function onCalibHold(quat, plane) {
aim.recordPoint(quat, plane); // after 3 points the crosshair starts helping
if (aim.calibQuats.length >= aim.calibTotal) {
const result = aim.finishCalibration();
if (result.ok) persist(aim.mapping());
}
}| Pattern | Points | Use |
|---|---|---|
CALIB_POINT_COUNT |
4 | Corners. Fast. No held-out error, so quality is guessed from residuals. |
CALIB_FULL_COUNT |
9 | Corners, edge midpoints, centre. Over-determined; reports real error. |
CALIB_REFRESH_COUNT |
4 | Re-anchor an existing mapping (keepMapping: true). |
After three captures the session switches onto a provisional map so the remaining targets can be aimed with a crosshair instead of by eye. Pointing a phone at a dot unaided is only good to a few degrees — most of the error a rejected calibration reports.
The fit:
- Picks the muzzle axis that best explains the captures (gun grip vs. phone held upright).
- Chooses projective vs. affine on leave-one-out error. Projective is physically correct; affine cannot place a singularity anywhere.
- Rejects the session if worst error exceeds 20% of the screen diagonal; keeps a rough fit between 3.5% and 20%.
Restore a saved mapping with aim.applyStoredMapping({ H, muzzle, label }). Recentre after restore so plane origin is the current pose.
aim.setSettings({ absoluteAiming: true }); // default: where the phone points
aim.setSettings({ absoluteAiming: false }); // integrate deltas, like a gyro mouseAbsolute projects the muzzle ray onto a plane, then through the homography. Recentre defines “screen centre.” This is the light-gun feel.
Relative integrates plane deltas. If a homography exists, the Jacobian supplies orientation and scale, so relative mode inherits calibration instead of inventing a second sensitivity. Otherwise settings.sensitivity is pixels per plane unit.
The host never trusts a raw quaternion as “now”:
- Catch-up — integrate the fitted rate over the sample’s age (capped).
- Display lead — add measured display lag plus one frame, scaled by rate quality.
- One Euro in degrees — the homography stretches pixels unevenly; degrees do not.
- Still-lock — freeze the crosshair after a short hold, release when the live aim walks away.
Trigger time uses the same prediction over interpolated history (sampleAt), not the last update pose.
Heading still drifts. Gravity fixes pitch and roll; yaw has nothing to hold it. learnDrift takes a small fraction of each unambiguous miss (or hit-adjacent shot) and caps the standing correction, so player scatter cancels and a run of strange shots cannot steal the aim.
| Sensors | Notes | |
|---|---|---|
| iOS Safari | DeviceOrientation + DeviceMotion | Permission only from a user gesture. WebKit rotationRate is CoreMotion xyz; GyroAxisDetector discovers this online. Accel sign is often negated. |
| Android Chrome | RelativeOrientationSensor @ 120 Hz preferred |
Falls back to DeviceOrientation/Motion. Secure context required. |
| Desktop | Host only | A laptop is the screen; the controller needs a phone. |
Diagnostics you can show on a debug HUD:
const d = motion.getDiagnostics();
d.mode; // "orientation-sensor" | "fusion" | "device-orientation"
d.converged; // false while the filter is still settling
d.gyroConvention; // "spec" | "webkit" | …
d.headingLocked; // magnetometer attitude trusted as a heading reference
d.rateQuality; // 0–1Wait for converged (or a timeout) before starting calibration.
aim.setSettings({
minCutoff: 1, // Hz — One Euro at rest
beta: 0.12, // Hz per deg/s — how fast the cutoff opens
displayLagMs: 25,
predictionHorizonMs: 0, // extra lead on top of age + display + one frame
filterLeadGain: 0, // keep off; prediction already owns latency
aimAssistRadius: 90,
sensitivity: 900, // relative mode, uncalibrated
predictionEnabled: true,
filteringEnabled: true,
aimAssistEnabled: true,
absoluteAiming: true,
driftLearningEnabled: true,
stillLockEnabled: true,
invertX: false,
invertY: false,
});filterLeadGain is off by default on purpose. Leading by the filter’s own lag overshoots and then unwinds: lag peaks as the hand slows, so the crosshair keeps going and creeps back for over a second.
Transport is yours (WebRTC datachannel, WebSocket, postMessage, …). The library only cares about this payload:
type AimSample = {
t: number; // controller clock, ms
q: [number, number, number, number]; // x, y, z, w
w: [number, number, number]; // rad/s, body frame
nq?: number; // rate quality, 0–1
};If the two devices do not share a clock, estimate an offset with the helpers in gyro-aim (clockOffsetFromExchange, estimateClockOffset) and call aim.setClockOffset(hostMinusController).
new MotionPipeline(handlers, { biasStorageKey?, storage? })
requestPermission(): Promise<boolean>
start() / stop()
getMode() / getDiagnostics()
loadBias() / saveBias()
recentre()storage defaults to localStorage. Pass null to disable persistence.
new AimSession({ screen, settings?, now? })
ingest(sample)
update(dt, now?, targets?) => AimFrame
aimAt(controllerT, now?, targets?) => [x, y]
recentre()
beginCalibration({ total?, keepMapping? })
recordPoint(quat, plane)
finishCalibration() => CalibResult
applyStoredMapping({ H, muzzle, label })
mapping() => StoredAimMapping | null
learnDrift(targets)
noteShot(aim, targets, hitId)
setSettings / setScreen / setClockOffsetAimFrame gives { aim, raw, filtered, predicted, clamped, edge, sampleAge, horizonMs, rateQuality, stillLocked } so a debug HUD can draw ghosts without reaching into internals.
Everything AimSession uses is exported: OrientationFusion, GyroAxisDetector, AngularRateEstimator, OneEuroFilter2D, StillLock, solveHomography, detectAimBasis, aimAssistPull, hitscan, quaternion helpers. Use them if you want a custom session.
Full derivation of the filter, the gnomonic plane, and the DLT is in docs/algorithm.md.
- Not a mouse injector. It does not move the OS cursor or talk to MAME. It returns pixels in your canvas.
- Not transport. Pairing, WebRTC, and clock sync protocol stay in the game. Clock math is included.
- Not WebXR / ARCore. Those are excellent on AR-capable Android and absent on iOS Safari. Fusion is the subset that works in a phone browser.
Games and products built on gyro-aim. Open a PR to add yours.
The reference implementation — a Duck Hunt–style light-gun game for a TV and a living room of phones.
Open ducks.game on the big screen, scan the QR with a phone, and the phone is the Zapper. Several people can join the same session; each gets a coloured crosshair. Aim is absolute: you point the handset at a duck on the TV, pull the trigger, and the host reconstructs pose at the shot timestamp. Nine-point calibration maps that aim onto the particular screen, sofa distance, and grip.
The game owns everything that is not aiming: NES-style rounds and flights, the dog, ammo, scoring, QR pairing, WebRTC with a WebSocket fallback, and the debug HUD. gyro-aim is the sensor-to-pixel core extracted from that stack.
MIT © Maksims Gladkovs