Here’s something to think about: When does a panda cease being a panda?
What sounds like an age-old question for Chinese philosophers is actually a query for modern-day data scientists, National Geospatial-Intelligence Agency (NGA) Director of Research Dr. Peter Highnam suggested Sunday during a keynote address at GEOINT Foreword.
“The example, on the right is a panda to you and I,” Highnam said, gesturing to an image of a panda in a tree on the screens behind him. “To a machine-learning algorithm, it’s a gibbon.”
Indeed, Google researchers in 2015 demonstrated how making subtle, pixel-level changes to an image could imperceptibly distort it, such that an image of one thing could be interpreted as an image of something else entirely. In the case of the panda, when researchers initially applied a machine learning algorithm to the image, it correctly identified the subject as a panda with 57.7 percent certainty. Researchers then altered a few pixels and ran the algorithm again. Although the image still looked like a panda to the naked eye, enough pixels had been displaced to convince the algorithm—with 99.3 percent confidence—that it was a gibbon.
“You and I, with our tacit knowledge of wildlife that mopes around in trees and eats bamboo, see it’s a panda … [but] to the machine learning classifier it’s now a gibbon,” Highnam reiterated. “The point being that machine learning is really, really dumb. There is no intelligence in it. It’s purely about the pixels, and we have to be really careful how we put it to use.”
Mistaking a bear for a monkey might not seem like a big deal, but consider what might happen if an unmanned aerial vehicle were mistaken for a bird, or a tank for a truck. Tech-savvy enemies who hack and distort imagery could wreak havoc on all manner of missions by exploiting weaknesses in machine learning algorithms.
According to Highnam, identifying those weaknesses and determining how to correct them is a major priority for NGA Research, which the agency established last year to champion and drive research across the GEOINT Community by activating external research and development partnerships at national labs, universities, and commercial businesses.
Originally stood up with seven focus areas, or “pods”—Radar, Automation, Geophysics, Spectral, Environment and Culture, Geospatial Cyber, and Geophysics—the office has since added an eighth pod through which it’s pursuing greater fidelity, confidence, and security in the areas of artificial intelligence (AI) and machine learning.
“Clearly, there’s a lot we can do in the NGA space with [machine learning]. The more we can bring these tools to bear … the better off we’re going to be,” Highnam said. “The theme underlying this is: We don’t have the deep knowledge yet in these systems … and that’s an issue.”
As the world inches ever closer to a future filled with algorithmic warfare, it’s an issue NGA must solve—and is determined to solve—before anyone else.
“[AI and machine learning] are available to everybody worldwide. We have to differentiate, and we can’t stop differentiating in our applications and understanding,” Highnam concluded.