Army Trains AI to Identify Faces in the Dark

Facial recognition has already come a long way since U.S. Special Operations Forces used the technology to help identify Osama bin Laden after killing the Al-Qaeda leader in his Pakistani hideout in 2011. The U.S. Army Research Laboratory recently unveiled a dataset of faces designed to help train AI on identifying people even in the dark—a possible expansion of facial recognition capabilities that some experts warn could lead to expanded surveillance beyond the battlefield.

The Army Research Laboratory Visible and Thermal Face dataset contains 500,000 images from 395 people. Despite its modest size as far as facial recognition datasets go, it is one of the largest and most comprehensive datasets that includes matching images of people’s faces taken under both ordinary visible-light conditions and with heat-sensing thermal cameras in low-light conditions.

“Our motivation for this dataset was we wanted to develop a nighttime and low-light face recognition capability for these unconstrained or difficult lighting settings,” says Matthew Thielke, a physicist at the Army Research Laboratory in Adelphi, Maryland.

Facial recognition applications for nighttime or low-light conditions are still not mature enough for deployment, according to the Army Research Laboratory team. Early benchmark testing with the dataset showed that facial recognition algorithms struggled to either identify key facial features or identify unique individual faces from thermal camera images—especially when the normal details visible in faces are reduced to blobby heat patterns. The dataset is described in a paper (PDF) that was presented during the 2021 IEEE Winter Conference on Applications of Computer Vision from 5–9 January.

The algorithms also struggled with “off-pose” images in which the person’s face is angled 20 degrees or more away from center. And they had problems matching the visible-light images of individual faces with their thermal imagery counterparts when the person was wearing glasses in one of the images.

But several independent experts familiar with facial recognition technology warned against feeling any false sense of security about how such technology is currently struggling to identify faces in the dark. After all, well-documented problems of racial bias, gender bias, and other accuracy issues with facial recognition have not stopped companies and law enforcement agencies from deploying the technology. The broad availability of such a dataset to train facial recognition algorithms to work better in low-light conditions—and not just for military scenarios—could also contribute to the broader surveillance creep of technologies designed to monitor individuals.

“It’s another of those steppingstones on the way to removing the ability for us to be anonymous at all,” says Benjamin Boudreaux, a policy researcher working on ethics, emerging technology, and international security at the Rand Corporation in Santa Monica, Calif. “We can’t even, if you will, hide in the dark anymore.”

The development of the dataset is related to ongoing work at the Army Research Laboratory aimed at developing automatic facial recognition that can work with the thermal cameras already deployed by military aircraft, drones, ground vehicles, watch towers, and checkpoints. U.S. Army soldiers and other military personnel may also sometimes use body-worn thermal cameras that might someday be coupled with facial recognition.

“The intention is not for any of these low-light facial recognition technologies to be used on U.S. citizens inside the continental United States, because that is outside the general purview of the U.S. military,” says Shuowen “Sean” Hu, an electronics engineer at the Army Research Laboratory. [READ MORE]