Computing · 06:45
3D face sensing
Before your eyes are fully open, your phone has already checked that the face in front of it is yours. The way it measures that face in three dimensions goes back to a small company in Tel Aviv.
The everyday moment
Face unlock feels like nothing at all: you lift the phone, glance at it, and the lock screen slides away. In that half-second the phone has projected an invisible pattern onto your face, read it back, built a depth map and compared it with the one it stored when you set it up. It works in the dark, through most glasses, and it isn't fooled by a photo, because a photo is flat.
How it works
- Project a patternA tiny projector casts a pattern of more than 30,000 infrared dots onto your face. Infrared is invisible to you, so you never see it.
- Read the distortionAn infrared camera photographs the dots. On a flat wall they would sit in a neat grid. On a nose, a cheekbone or a chin they shift, and how much each one shifts depends on how far away that part of your face is.
- Build a depth mapSoftware turns those shifts into distances and builds a 3D map of your face. This technique is called structured light.
- CompareThe phone compares the new map with the one it saved during setup. Apple puts the chance that a random person could unlock your phone this way at about one in a million.
The Israeli story
PrimeSense was founded in Tel Aviv in 2005 by Aviad Maizels and Alexander Shpunt. Their idea, which they called light coding, was to read depth with cheap, ordinary parts: a near-infrared projector, a standard camera sensor and a chip that did all the maths itself, so the computer it was plugged into hardly had to work.
They showed their depth cameras to game developers in 2006. Microsoft took notice, and in 2010 named PrimeSense as a partner on the project that became Kinect for Xbox 360, the camera that let you play games by moving your body. PrimeSense licensed the hardware design and the chip inside it.
In November 2013 Apple bought the company for about $360 million. Four years later the iPhone X introduced Face ID, whose TrueDepth camera builds on PrimeSense's infrared depth-sensing work.
Where you meet it today
Face ID is on every iPhone since the iPhone X apart from the SE models, and on iPad Pro since 2018. The same depth sensing drives Animoji, portrait selfies and some accessibility features, and PrimeSense's chips also found their way into 3D scanners and robots.
Did you know?
The first big consumer product built on PrimeSense's chip wasn't a phone at all. It was Kinect, a camera for playing video games with your body.