Wearing a mask won’t stop facial recognition anymore

If you’ve been walking around your city recently believing that the face mask you wear to protect you from the coronavirus is also fooling facial recognition cameras, then we have bad news for you: Facial recognition is evolving.

New forms of facial recognition can now recognize not just people wearing masks over their mouths, but also people in scarves and even with fake beards. And the technology is already rolling out in China because of one unexpected event: The coronavirus outbreak.

Over the last several weeks, the deadly Covid-19 disease has forced millions of people across China to don surgical masks and N95 respirators. This has led China’s AI champion SenseTime to adapt its facial recognition product to identify people wearing these masks, according to an announcement from the company last week.

But this technology isn’t exactly new. Stanford University postdoctoral fellow Amarjot Singh and his team published research on disguised face identification (DFI) in 2017. Their algorithm made a breakthrough in recognizing people wearing eyeglasses, fake beards, scarves and hard hats.

“Face recognition identifies a person by locating several key points on the face and connecting them together to form a unique person-specific signature,” Singh explained.

These key points are usually around the eyes, nose and lips. But facial recognition systems developed by Singh and others are able to recognize masked people by getting enough key points from just the eyes and nose.

“Although it would be less accurate, a match still can be established,” Singh said.

When the work was originally published, Singh’s team described disguised face identification as extremely challenging. Their network was trained to detect the 14 key points on photos, but the accuracy would drop depending on the disguise and the complexity of the background behind the individual. But since then, more research has been done to improve facial recognition with masks -- and it seems many now seek to commercialize it.

SenseTime, which announced the rollout of its face mask-busting tech last week, explained that its algorithm is designed to read 240 facial feature key points around the eyes, mouth and nose. It can make a match using just the parts of the face that are visible.

Generally speaking, the more key points the algorithm can include, the better the results. But there are also other factors involved. Yufei Wei, chief marketing officer of facial recognition company Qingfei Technologies, explained that a facial recognition system doesn’t necessarily need more data -- but it does need more accurate data.

“There will be a more specific algorithm design to recognize and match the special database,” Wei said.
Researchers from the University of Bradford led by professor Hassan Ugail improved their facial recognition model by feeding it with partial facial images and achieved 90% correct identification. A similar trick was applied by another Chinese facial recognition company that claims it can now recognize people with masks.

Facing the impact of Covid-19, Nanjing-based Minivision launched an emergency data-gathering campaign. Management urgently mobilized employees and relatives to collect a small-scale dataset in two days. The key information that the system recorded on masked faces was the eyes.

The rush was due to China’s strict measures for battling the epidemic. Many residential communities, especially in areas hit hardest by the virus, have been limiting entry to residents only. Minivision introduced the new algorithm to its facial recognition gate lock systems in communities in Nanjing to quickly recognize residents without the need to take off masks.

But these kinds of systems still have limitations. President of Minivision’s AI research institute Hu Jianguo explained in an interview with the Chinese edition of Synced that expanding this system to a wider group of people would be hard. When a population reaches a certain scale, the system is likely to encounter people with similar eyes.

This might be why most commercial facial recognition systems that can identify masked faces seem limited to small-scale applications. SenseTime said that their own system is being used in offices to clock in employees. Beijing-based FaceGo, another company boasting about its newfound face-screening skills, also notes that its systems are used in attendance software.

But next time you’re walking through the streets with your face mask on, remember that evading the all-seeing eyes of facial recognition might require you to cover your entire face.