/>
X

Snoopers beware: Google's AI can now spot shoulder-surfers peeking at your screen

Google researchers develop a fast way to alert you to a stranger spying over your shoulder.
liam-tung.jpg
Written by Liam Tung, Contributor on
googleshouldsurf.png

Researchers show off Google's fast shoulder-surfing detector on a Pixel.

Image: Google

Google researchers have developed a privacy application that can instantly detect when a stranger glances at your screen over your shoulder.

Researchers Hee Jung Ryu and Florian Schroff will demonstrate their shoulder-surfing warning system at next month's Neural Information Processing Systems Conference in California.

The pair haven't released a paper that explains technology, but a video demonstration suggests they've installed a lightweight machine-learning model on a Pixel smartphone that uses its front-facing camera for rapid gaze detection.

When the user holds a phone up to chat or view a private video, say, on a train or other crowded place, the algorithm will detect when someone else from behind begins looking at the screen too. The video shows that the algorithm can detect a stranger's gaze within milliseconds.

The video suggests the researchers are using FaceNet, a facial-recognition neural network developed by Schroff and other Google researchers in 2015, as well as GazeNet, a gaze-estimation neural network recently described by researchers in Japan and Germany.

The two Google researchers call their application an 'electronic screen protector', which combines the two areas of computer vision research to provide enhanced privacy when using a big-screen smartphone in public spaces.

The application builds on face-authentication systems already on smartphones by using the enrolled face to distinguish between the user and a stranger, while the gaze-estimation model helps detect the stranger's gaze almost instantaneously.

The system takes just two milliseconds to detect a stranger's gaze, and 47 milliseconds to recognize a face. It also takes 115 milliseconds to detect a face in each frame.

"Because of the quick, robust, and accurate gaze detection mobile model, we can now easily identify the face identity and gaze simultaneously in real time," they explain in a brief description.

"Hence, the application, an electronic screen protector, can enable its enrolled users to continue reading private and confidential contents on your mobile device, while protecting their privacy from onlookers in a crowded space such as the subway or an elevator."

The system takes two milliseconds to detect a stranger's gaze, and 47 milliseconds to recognize a face.

Google/YouTube

Previous and related coverage

iPhone X has face recognition but this heart-scan authentication goes one better

A heart-based authentication system keeps you logged in until you walk away from the device.

Face ID is so good you won't miss the Home button

The removal of the iPhone's Home button has required some tradeoffs, but the superior usability of Face ID sets a new standard for secure access.

iPhone X: Sorry Apple, but I just can't face using Face ID

The iPhone X is putting facial recognition front and centre -- so why does it leave me frowning?

Read more about facial recognition

Related

He flew American Airlines, she flew United. For both, the unthinkable happened
screen-shot-2022-06-30-at-10-14-36-am.png

He flew American Airlines, she flew United. For both, the unthinkable happened

Business
Southwest Airlines has cancelled 20,000 flights. Now for the really bad news
screen-shot-2021-07-07-at-4-01-12-pm.png

Southwest Airlines has cancelled 20,000 flights. Now for the really bad news

Business
McDonald's and Chick-fil-A both have a big problem. Only one has a solution
screen-shot-2022-06-28-at-6-24-27-pm.png

McDonald's and Chick-fil-A both have a big problem. Only one has a solution

Business