It turns out, confounding software into not realizing what it's looking at is a matter of fooling several different smaller systems at once.
Think about a person, for example. Now think of a person who looks nothing like that. And now do it again. Humanity, after all, contains multitudes, and a person can have many different appearances. A machine learning system needs to understand the diverse array of different inputs that, put together, mean "person." A nose by itself won't do it; an eye alone will not suffice; anything could have a mouth. But put dozens or hundreds of those priors together, and you've got enough for an object detector.
Code does not "think" in terms of facial features, the way a human does, but it does look for and classify features in its own way. To foil it, the "cloaks" need to interfere with most or all of those priors. Simply obscuring some of them is not enough. Facial recognition systems used in China, for example, have been trained to identify people who are wearing medical masks while trying to prevent the spread of COVID-19 or other illnesses.
And of course, to make the task even more challenging, different object detection frameworks all use different mechanisms to detect people, Goldstein explained. "We have different cloaks that are designed for different kinds of detectors, and they transfer across detectors, and so a cloak designed for one detector might also work on another detector," he said.
Science fiction writer Philip K. Dick, in his 1977 novel A Scanner Darkly, described a scramble suit, a device that enables the wearer to defy description, both in person and in surveillance videos:
The scramble suit was an invention of the Bell laboratories, conjured up by accident by an employee named S. A. Powers... Basically, his design consisted of a multifaceted quartz lens hooked up to a million and a half physiognomic fraction-representations of various people: men and women, children, with every variant encoded and then projected outward in all directions equally onto a superthin shroudlike membrane large enough to fit around an average human.
(Read more about Philip K. Dick's scramble suit)
I love this story, take a look at similar instances:
A System To Defeat AI Face Recognition
'...points and patches of light... sliding all over their faces in a programmed manner that had been designed to foil facial recognition systems.' - Neal Stephenson, 2019.
Smart TVs Are Listening!
'You had to live -- did live, from habit that became instinct -- in the assumption that every sound you made was overheard...' - George Orwell, 1948.
Technovelgy (that's tech-novel-gee!)
is devoted to the creative science inventions and ideas of sf authors. Look for
the Invention Category that interests
you, the Glossary, the Invention
Timeline, or see what's New.
A System To Defeat AI Face Recognition
'...points and patches of light... sliding all over their faces in a programmed manner that had been designed to foil facial recognition systems.'
Smart TVs Are Listening!
'You had to live -- did live, from habit that became instinct -- in the assumption that every sound you made was overheard...'