vigilo-wasm

vigilo-wasm is vigilo-core ported to the web. The Rust engine is the same code compiled to WebAssembly, and only inference and camera capture were replaced. There's no server, no upload and no native app to install.

What runs where

The camera comes in through getUserMedia, gets drawn to a canvas, and hands its RGBA bytes to Rust, which letterboxes, crops and packs the input tensors. onnxruntime-web runs the models on its wasm SIMD backend or on WebGPU, and the raw output tensors go straight back to Rust for anchor decoding, NMS, signal assembly and the fusion engine. What comes out is a stream of violation events.

Everything that isn't a matrix multiply stays in Rust: which face is the primary one, whether gaze is allowed to run, what an absent signal means, and what it all adds up to over time. Rewriting any of that in TypeScript would create a second copy of rules tuned against a recorded corpus, and the two copies would drift the first time one of them was fixed.

Models

Face detection (YuNet) runs in 51 ms, head pose in 13 ms, gaze in 104 ms and YOLOX-Nano object detection in 75 ms, measured warm on a single-threaded wasm backend. The four models come to 14 MB and are kept in the Cache API after the first load.

Identity checking was dropped. ArcFace is 13.6 MB, as much as the other four together, for a 0.2 Hz signal that needs an enrolled reference photo a browser tab has no trustworthy way to obtain. The slot reports not_configured, which tells fusion "never available" rather than "absent right now."

YuNet ships as float rather than int8. The quantized file is under half the download and four times the latency, 209 ms against 51 ms, because ORT-web's wasm backend has no fast quantized-convolution kernel. Gaze is the expensive model: at 448×448 it costs more than face and pose combined, so by default it runs on every other frame.

Behaviour worth knowing

Browsers clamp a hidden tab's timers to roughly 1 Hz, and a page can't opt out. The runtime watches visibilitychange and emits a degraded event, so the gap lands in the session record as time the system couldn't see instead of as a run of clean frames.

Fusion takes the timestamp as a parameter rather than reading a clock, so the same frames replay to the same event sequence every time. It's also exposed on its own (ProctorSession and replay) for signals produced some other way.

No bundler is needed. The package ships a --target web build and takes onnxruntime-web as an argument, so a plain <script type="module"> page works. It has 60 Rust tests and 33 TypeScript tests, including a suite that loads the real ONNX graphs and checks every tensor name and shape the decoder assumes.