How browser-based pose detection works (and why your camera stays private)

"A fitness app that watches you through your webcam" sounds like exactly the kind of thing privacy-conscious developers avoid. So it's worth explaining precisely how VibeFit's rep verification works — because the camera feed never leaves your device, and that's not a marketing promise, it's how the technology is built.

What pose detection actually is

Pose detection is a machine-learning model that looks at an image of a person and returns the coordinates of their body's key points — shoulders, elbows, wrists, hips, knees, ankles, and so on. VibeFit uses Google's MediaPipe, which tracks 33 of these landmarks per frame. The model doesn't recognize who you are; it locates joints. From those joint positions, simple geometry does the rest: the angle at your knee tells us how deep your squat is; the angle at your elbow counts a push-up.

The key detail: it runs in your browser

Traditional computer-vision apps send your camera frames to a server, which runs the model and sends a result back. VibeFit doesn't do that. The MediaPipe model is compiled to WebAssembly and downloaded to your browser, where it runs locally on your own device's CPU/GPU. Every frame is analyzed on your machine and then discarded. Nothing is uploaded, streamed, or stored.

The only things that ever reach our server are two numbers per exercise: how many reps you did and a form score from 0 to 1. Never an image. Never a frame of video.

How a rep gets counted

Each exercise has a small, deterministic detector. For a squat, the detector watches the average angle of your knees across frames: when it passes below a 'down' threshold and then back above an 'up' threshold, that's one rep. A separate check looks at the symmetry and depth of the movement to produce a form score, which is what powers real-time coaching cues like 'knees out' or 'a little deeper.'

Because the math is simple and local, it's also fast — the detector runs many times per second with no network round-trip, which is why the rep counter feels instant.

Why this matters for trust

On-device processing isn't only a privacy nicety; it changes the threat model entirely. There's no video sitting in a bucket to be breached, subpoenaed, or accidentally indexed. There's no third-party processor in the loop. If you unplug your network mid-workout, rep counting keeps working — proof that the camera work is happening where you are, not somewhere else.

For an audience that reads privacy policies for a living, that's the point: you get the accountability of a camera-verified workout without handing your camera to anyone.