Face Tracking Before Snapchat — Visage Technologies & XZImg

What Face Tracking Actually Is

Face tracking in the context of AR means detecting a human face in a camera frame and precisely locating specific points on it — the corners of the eyes, the tip of the nose, the edges of the lips, the outline of the jaw — in real time, fast enough to keep up with natural head movement and facial expression changes.

Once those points are located, two things become possible. First, the face's position and orientation in 3D space can be determined — so the application knows exactly where the face is, which way it is pointing, and how it is tilted. Second, the relative positions of the tracked points can be analysed to detect expressions — whether the mouth is open or closed, whether the eyebrows are raised, whether the eyes are looking left or right.

These two capabilities — pose estimation and expression detection — are the foundation of everything from Snapchat dog ears to medical facial analysis software to face-controlled game interfaces.

The Two SDKs — Visage Technologies and XZImg

What I Was Building — Face Games and Virtual Overlays

My exploration of face tracking SDKs around 2014 was driven by two specific ideas that felt genuinely exciting at the time — and that Snapchat would later validate for hundreds of millions of users.

Idea 1 — A Face-Controlled Game

The concept was simple: use facial expressions or head movement to control a game instead of touch controls. Open your mouth — character jumps. Turn your head left — character moves left. Raise your eyebrows — special action triggers. This kind of face-as-controller interaction felt novel and playful, and face tracking SDKs like Visage Technologies made it technically possible.

The challenge was latency and reliability. A game control system needs to respond consistently and immediately — any lag between the player's expression and the game's response breaks the experience. Getting face tracking fast enough and stable enough on the mobile hardware of 2014 to feel like a real game control mechanism was genuinely difficult. The SDKs were capable, but the gap between "technically works in a demo" and "feels good to play" was significant.

Idea 2 — Virtual Overlays on the Face

The second concept was placing virtual objects on the face — glasses, masks, hats, effects — that stayed precisely anchored to facial features as the person moved their head, changed their expression, and shifted their distance from the camera. This is exactly what Snapchat filters do. In 2014, doing this required integrating a professional face tracking SDK, writing custom rendering code, and solving the alignment and scaling problems manually. Two years later, Snapchat made it a one-tap experience for anyone.

The technical requirements for a convincing face overlay are more demanding than they might appear. The virtual object must stay precisely anchored to the correct facial landmarks — glasses must sit on the nose bridge and over the ears, not drift as the person moves. The scale must update as the person moves closer or further from the camera. The object must respond to expression changes — glasses should not clip through the face when the person smiles. Getting all of this right with the tools available in 2014 required careful implementation.

How Visage Technologies Face Tracking Worked

Visage Technologies' SDK tracked the face by locating a set of facial landmarks — specific anatomical points — in each camera frame. The full SDK could track up to 84 facial feature points, though real-time mobile applications typically used a reduced set for performance.

In Unity integration, these landmark positions were exposed as 3D coordinates that updated every frame. A virtual object could be positioned relative to any landmark — anchor glasses to the nose tip and eye corner points, anchor a hat to the forehead landmarks, anchor a beard to the chin and jaw points. The SDK handled the computer vision; the developer handled the rendering and anchoring logic.

Getting Visage Technologies Running in Unity — The Reality

In 2014, integrating Visage Technologies into a Unity project was not a simple asset store import. It required downloading the SDK directly from Visage Technologies — which meant contacting their sales team — integrating the native libraries into the Unity project, writing bridge code between the SDK's output and Unity's component system, and debugging build pipeline issues that the documentation did not always cover. I have email threads from April 2014 with Visage Technologies' team working through exactly these integration issues on Android builds.

The APK built from Android's build system worked correctly. The same project built through Unity's build pipeline initially produced incorrect output — the camera feed appeared as a texture rather than a proper AR view. Resolving this required direct support from Visage Technologies' engineering team. This was the reality of working with professional AR SDKs in 2014: the capability was there, but reaching it required persistence and direct vendor relationships.

What Developers Were Building With Face Tracking in 2013–2015

Then vs Now — How Face Tracking Changed

When Snapchat Changed Everything — 2015 to 2016

Snapchat launched its first Lenses feature in September 2015 — initially offering a small set of face effects including the now-famous face swap. By 2016, the dog ear filter had become one of the most recognised digital experiences in the world. In two years, face tracking went from a niche capability that required specialist SDKs and significant development effort to something a 12-year-old could apply with a single tap.

What Snapchat did was not invent face tracking — that technology had existed in professional SDKs for years. What Snapchat did was abstract all the complexity away entirely, build a creator platform around it, and distribute it to hundreds of millions of users simultaneously. The technology was the same. The accessibility was completely different.

For developers who had been working with face tracking SDKs before Snapchat, watching this happen was a familiar experience. The technology we had been using in small-scale professional projects was suddenly reaching global scale because a consumer platform had made it frictionless.

Frequently Asked Questions