SprayTarih

Formerly SprayHistory.

An interactive digital wall where visitors bring historical scenes back to life by "spraying" colour into them.

🇹🇷 Türkçe özet: SprayTarih, bir bilim merkezi sergisi için geliştirilmiş etkileşimli bir dijital duvardır; ziyaretçi elini (ya da IR lazer işaretçiyi) tablonun üzerinde gezdirerek çizgi halindeki tarihi sahneyi tam renge kavuşturur. Sistem halka açık, gözetimsiz kiosk olarak tasarlanmıştır: çökmez, hata olursa kendini toparlar ve her ziyaretçiden sonra kendini sıfırlar. Ayrıntılı Türkçe kurulum ve işletim talimatları docs/INSTALL.md içindedir.

Overview

SprayTarih is a projection-based exhibit built for a science-centre setting. A historical scene is projected onto a wall as a clean black-and-white line drawing. A camera tracks the visitor's hand (or an IR laser pointer); wherever they "spray", the full-colour painting underneath is revealed.

It uses a reveal model: the visitor does not add paint, they bring colour back. The result therefore looks good regardless of the visitor's skill, and there is no such thing as a wrong spray.

Wall (line art)	Revealed (colour)

This is a public, unattended kiosk, so the number one requirement is sustainability: it must never freeze or crash, it must recover from faults on its own, and it must reset itself for the next visitor. The app also launches on an ordinary desktop with no camera or projector (it falls back to mouse input), so it can be tried anywhere.

Features

• Reveal-mask rendering. A WebGL2 renderer accumulates a soft-brush mask and mixes line-art wall and colour image through it. The reveal is permanent until reset.
• Four-screen kiosk flow. Start, scene select, discover (facts about the scene), and game, with a completion screen. Three scenes ship: Fatih Sultan Mehmet (1453), Selahaddin Eyyubi (1187), Yavuz Sultan Selim (1517).
• Pluggable input. MediaPipe hand tracking, IR laser dot tracking (the production setup, with an Intel RealSense IR camera), or mouse. All trackers emit the same {present, x, y, spraying} contract.
• Hardware auto-detection. RealSense, then webcam, then mouse; no branch of the probe can crash start-up.
• Built-in technician screen. All calibration (4-corner alignment, laser thresholds, grid verification) lives inside the app behind a key chord, not in separate tools.
• Idle auto-reset and anonymous telemetry. The wall clears after inactivity; an HTML visitor report (completion, abandonment, popular scenes) is generated from anonymous local logs.
• Licensed Windows distribution. Standalone Nuitka build, encrypted web content, offline licence keys, Inno Setup installer.

Architecture

Two decoupled halves that talk only over a fixed WebSocket protocol (ws://127.0.0.1:8765, JSON frames).

  ┌────────────┐    ┌──────────────┐    ┌──────────────────┐    ┌──────────────────────┐
  │  Camera    │───▶│  MediaPipe   │───▶│    WebSocket     │───▶│  WebGL2 reveal       │
  │ (OpenCV)   │    │  hand track  │    │  ws://127.0.0.1: │    │  renderer (browser)  │
  │            │    │  + smoothing │    │  8765 (JSON)     │    │  wall -> colour via  │
  │            │    │  (1€ filter) │    │                  │    │  accumulated mask    │
  └────────────┘    └──────────────┘    └──────────────────┘    └──────────────────────┘
        └──────────── Python backend (single process) ──────────┘        frontend

• Backend (Python): reads the camera, finds the hand, resolves open-hand/fist and fingertip, mirrors and calibrates the coordinates, smooths them with a One Euro filter and broadcasts normalised screen coordinates (0..1, top-left origin). The camera and tracker run on a worker thread; the asyncio loop broadcasts the latest state at a fixed rate, so a camera hiccup never stalls the socket.
• Frontend (browser, WebGL2): knows nothing about cameras or calibration. It draws the wall, accumulates the reveal mask where the cursor sprays, and mixes in the colour image through that mask.
• Input is pluggable: tracker in config.yaml selects the hand tracker or the IR laser tracker. Everything after the tracker (smoothing, calibration, WebSocket, frontend) is identical for both.

Messages: backend sends {"type":"hand","present","x","y","spraying","intensity"} on every camera frame, plus reset and scene control messages; the frontend may send a hello with its size. Details, render technique and risks: docs/ARCHITECTURE.md (Turkish).

Tech stack

Layer	Technology
Vision backend	Python, OpenCV, MediaPipe Hands, NumPy, 1€ (One Euro) filter
Production camera	Intel RealSense IR imager (pyrealsense2); RTSP/webcam also supported
Transport	websockets (asyncio), JSON
Frontend	Vanilla ES modules, WebGL2, self-hosted fonts, no build step
Packaging	Nuitka standalone, Inno Setup installer
Licensing	Ed25519 signatures, AES-GCM content pack (cryptography), pywebview licence window
Tests / CI	pytest, GitHub Actions (Python 3.12)

Engineering highlights

• Self-recovering by design. The WebSocket server runs even if the camera fails to open or dies mid-show; the worker retries the camera on a timer and keeps broadcasting present:false. The frontend reconnects with growing back-off, and the wall stays visible with no backend at all.
• Defensive hardware probing. pyrealsense2 may be missing, installed without its native DLL, or installed with no device attached; each case is caught separately. A camera probe that could block (OpenCV has no open timeout and a camera held by another app can hang it) is run on a bounded thread.
• Laser tracker state machines. A dual-brightness laser distinguishes "aiming" from "spraying" with a Schmitt trigger plus frame debounce, and finds the dot with an intensity-weighted sub-pixel centroid. process() never raises; any internal error degrades to present:false.
• Persistent diagnostics. A rotating timestamped log tees stdout/stderr, so an unattended failure at 2am leaves a trace without touching existing call sites.
• Licence and packaging pipeline. Frozen builds require an offline-verifiable licence at launch, before the camera is probed or the server starts. Activation is bound to a machine fingerprint (2 of 3 hardware identifiers, HMAC), demo licences watermark the screen and detect clock rollback, and the front-end ships as an AES-GCM web.pak decrypted in memory. The installer build refuses to run if the dist folder contains activation, calibration, key or log leftovers. Plan and deviations: docs/plans/2026-09-11-lisans-ve-kurucu-entegrasyonu.md.
• Privacy-aware telemetry. No camera frames, landmarks, names, IPs or persistent visitor IDs; a random per-visit session number only. Events are batched and flushed with sendBeacon on teardown, and telemetry can never break the exhibit.
• Tests. 16 pytest modules cover the protocol contract, smoothing, config merge, calibration homography, laser state machines, licensing, fingerprinting, the web pack and the installer guard. CI runs them without camera hardware, because heavy dependencies are imported lazily.

Project status

The exhibit is implemented end to end: kiosk UI, three scenes, in-app calibration, telemetry report, licensed Windows installer. Open roadmap items (from the project docs): a depth camera to resolve the dark-room lighting paradox for RGB hand tracking, and clearing usage rights for high-resolution historical imagery. The crash-recovery watchdog is a PowerShell script (tools/watchdog.ps1) rather than part of the EXE.

Repository layout

Path	Purpose
spray.py	Single entry point: licence gate, static server, kiosk browser, backend
backend/	Camera, hand/laser trackers, smoothing, calibration, WebSocket server, hardware detection
frontend/	WebGL2 reveal renderer, kiosk screens, styles, scene content
pics/	Source images for the scenes (colour / line-art pairs)
tools/	Scene preparation, Nuitka/installer build, static server, visitor report, watchdog
lisans*.py, parmak_izi.py, web_paket.py, lisans_web/	Licence, fingerprint and encrypted content pack
installer/	Inno Setup script
tests/	pytest suite
docs/	Architecture, install guide, delivery checklist, IR laser setup, plans
design_handoff_sprayhistory_/	Design handoff for the kiosk UI (reference prototype, not production code)

Running locally

python -m venv backend\.venv
backend\.venv\Scripts\pip install -r requirements.txt
python tools\prepare_scenes.py     # pics/ -> frontend/scenes/ (once)
python spray.py                    # add --windowed to avoid full screen

With no camera attached the app falls back to mouse input. Open the technician screen with Ctrl+Shift+K. Full setup (Turkish): docs/INSTALL.md.