The Secret History of Facial Recognition
Sixty years ago, a sharecropper’s son invented a technology to identify faces. Then the record of his role all but vanished. Who was Woody Bledsoe, and who was he working for?
Woody Bledsoe was sitting in a wheelchair in his open garage, waiting. To anyone who had seen him even a few months earlier—anyone accustomed to greeting him on Sundays at the local Mormon church, or to spotting him around town on his jogs—the 74-year-old would have been all but unrecognizable. The healthy round cheeks he had maintained for much of his life were sunken. The degenerative disease ALS had taken away his ability to speak and walk, leaving him barely able to scratch out short messages on a portable whiteboard. But Woody’s mind was still sharp. When his son Lance arrived at the house in Austin, Texas, that morning in early 1995, Woody immediately began to issue instructions in dry-erase ink.
He told Lance to fetch a trash can from the backyard—one of the old metal kinds that Oscar the Grouch lives in. Lance grabbed one and set it down near his father. Then Woody sent him into the house for matches and lighter fluid. When Lance got back, Woody motioned to two large file cabinets inside the garage.
They’d been around ever since Lance could remember. Now in his late thirties, Lance was pretty sure they hadn’t been opened since he was a kid. And he knew they weren’t regular file cabinets.
They were the same kind he’d seen when he worked on sonar equipment for US nuclear submarines—fireproof and very heavy, with a strong combination lock on each drawer. His father slowly began writing numbers on the whiteboard, and to Lance’s astonishment, the combination worked. “As I opened the first drawer,” he tells me almost 25 years later, “I felt like Indiana Jones.”
A thick stack of old, rotting documents lay inside. Lance began removing them and placing them in his father’s hands. Woody looked over the piles of paper two inches at a time, then had his son toss them into the fire he’d started in the burn barrel. Some, Lance noticed, were marked “Classified” or “Eyes only.” The flames kept building until both cabinets were empty. Woody insisted on sitting in the garage until all that remained was ash.
Lance could only guess at what he’d helped to destroy. For nearly three decades, his father had been a professor at the University of Texas at Austin, working to advance the fields of automated reasoning and artificial intelligence. Lance had always known him to be a wide-eyed scientific optimist, the sort of man who, as far back as the late 1950s, dreamed of building a computer endowed with all the capabilities of a human—a machine that could prove computer endowed with all the capabilities of a human—a machine that could prove complex mathematical theorems, engage in conversation, and play a decent game of Ping-Pong.
But early in his career, Woody had been consumed with an attempt to give machines one particular, relatively unsung, but dangerously powerful human capacity: the ability to recognize faces. Lance knew that his father’s work in this area—the earliest research on facial-recognition technology—had attracted the interest of the US government’s most secretive agencies. Woody’s chief funders, in fact, seem to have been front companies for the CIA. Had Lance just incinerated the evidence of Washington’s first efforts to identify individual people on a mass, automated scale?
Today, facial recognition has become a security feature of choice for phones, laptops, passports, and payment apps. It promises to revolutionize the business of targeted advertising and speed the diagnosis of certain illnesses. It makes tagging friends on Instagram a breeze. Yet it is also, increasingly, a tool of state oppression and corporate surveillance. In China, the government uses facial recognition to identify and track members of the Uighur ethnic minority, hundreds of thousands of whom have been interned in “reeducation camps.” In the US, according to The Washington Post, Immigration and Customs Enforcement and the FBI have deployed the technology as a digital dragnet, searching for suspects among millions of faces in state driver’s license databases, sometimes without first seeking a court order. Last year, an investigation by the Financial Times revealed that researchers at Microsoft and Stanford University had amassed, and then publicly shared, huge data sets of facial imagery without subjects’ knowledge or consent. (Stanford’s was called Brainwash, after the defunct café in which the footage was captured.) Both data sets were taken down, but not before researchers at tech startups and one of China’s military academies had a chance to mine them.
Woody’s facial-recognition research in the 1960s prefigured all these technological breakthroughs and their queasy ethical implications. And yet his early, foundational work on the subject is almost entirely unknown. Much of it was never made public.
Fortunately, whatever Woody’s intentions may have been that day in 1995, the bulk of his research and correspondence appears to have survived the blaze in his garage. Thousands of pages of his papers—39 boxes’ worth—reside at the Briscoe Center for American History at the University of Texas. Those boxes contain, among other things, dozens of photographs of people’s faces, some of them marked up with strange mathematical notations—as if their human subjects were afflicted with some kind of geometrical skin disease. In those portraits, you can discern the origin story of a technology that would only grow more fraught, more powerful, and more ubiquitous in the decades to come
When Woody was 12 his father died, plunging the family even deeper into poverty in the middle of the Great Depression. Woody took on work at a chicken ranch while he finished high school. Then he moved to the city of Norman and began attending classes at the University of Oklahoma, only to quit after three months to join the Army on the eve of World War II.
Showing an aptitude for math, Woody was put in charge of a payroll office at Fort Leonard Wood in Missouri, where wave after wave of US soldiers were being trained for combat. (“Our group handled all black troops,” wrote the Oklahoman, “which was a new experience for me.”) Then on June 7, 1944, the day after D-Day, Woody was finally deployed to Europe, where he earned a Bronze Star for devising a way to launch large naval vessels—built for beach landings—into the Rhine.
Having landed in the European theater just as Allied troops were accelerating to victory, Woody seemed to have an unusually positive experience of war. “These were exciting times,” he wrote.
“Each day is equivalent to a month of ordinary living. I can see why men get enamored with war. As long as you are winning and don’t sustain many casualties, everything is fine.” He spent the following summer in liberated Paris, his mind and his experience of the world expanding wildly in an atmosphere of sometimes euphoric patriotism. “The most sensational news I ever heard was that we had exploded an atomic bomb,” Woody wrote. “We were glad that such a weapon was in the hands of Americans and not our enemies.”
Woody couldn’t wait to get back to school once the war ended. He majored in mathematics at the University of Utah and finished in two and a half years, then went off to Berkeley for his PhD. After grad school, he got a job at the Sandia Corporation in New Mexico, working on government-funded nuclear weapons research alongside such luminaries as Stanislaw Ulam, one of the inventors of the hydrogen bomb. In 1956 Woody flew to the Marshall Islands to observe weapons tests over Enewetak Atoll, parts of which to this day suffer worse radioactive contamination than Chernobyl or Fukushima. “It was satisfying to me to be helping my own dear country remain the strongest in the world,” he wrote.
Sandia also offered Woody his first steps into the world of computing, which would consume him for the rest of his career. At first, his efforts at writing code tied directly to the grim calculations of nuclear weapons research. One early effort—“Program for Computing Probabilities of Fallout From a Large-Scale Thermonuclear Attack”—took into account explosive yield, burst points, time of detonation, mean wind velocity, and the like to predict where the fallout would land in the case of an attack.
But as his romance with computing grew, Woody took an interest in automated pattern recognition, especially machine reading—the process of teaching a computer to recognize unlabeled images of written characters. He teamed up with his friend and colleague Iben Browning, a polymath inventor, aeronautical engineer, and biophysicist, and together they created what would become known as the n-tuple method. They started by projecting a printed character—the letter Q, say—onto a rectangular grid of cells, resembling a sheet of graph paper. Then each cell was assigned a binary number according to whether it contained part of the character: Empty got a 0, populated got a 1. Then the cells were randomly grouped into ordered pairs, like sets of coordinates. (The groupings could, in theory, include any number of cells, hence the name n-tuple.) With a few further mathematical manipulations, the computer was able to assign the character’s grid a unique score. When the computer encountered a new character, it simply compared that character’s grid with others in its database until it found the closest match.
The beauty of the n-tuple method was that it could recognize many variants of the same character: Most Qs tended to score pretty close to other Qs. Better yet, the process worked with any pattern, not just text. According to an essay coauthored by Robert S. Boyer, a mathematician and longtime friend of Woody’s, the n-tuple method helped define the field of pattern recognition; it was among the early set of efforts to ask, “How can we make a machine do something like what people do?”
Around the time when he was devising the n-tuple method, Woody had his first daydream about building the machine that he called a “computer person.” Years later, he would recall the “wild excitement” he felt as he conjured up a list of skills for the artificial consciousness:
"I wanted it to read printed characters on a page and handwritten script as well. I could see it, or a part of it, in a small camera that would fit on my glasses, with an attached earplug that would whisper into my ear the names of my friends and acquaintances as I met them on the street … For you see, my computer friend had the ability to recognize faces."
In 1960, Woody struck out with Browning and a third Sandia colleague to found a company of their own. Panoramic Research Incorporated was based, at first, in a small office in Palo Alto, California, in what was not yet known as Silicon Valley. At the time, most of the world’s computers—massive machines that stored data on punch cards or magnetic tape—resided in large corporate offices and government labs. Panoramic couldn’t afford one of its own, so it leased computing time from its neighbors, often late in the evenings, when it was cheaper.
Panoramic’s business, as Woody later described it to a colleague, was “trying out ideas which we hoped would ‘move the world.’ ” According to Nels Winkless, a writer and consultant who collaborated on several Panoramic projects and later became a founding editor of Personal Computing magazine, “Their function was literally to do what other people find just too silly.”
The company attracted an odd and eclectic mix of researchers—many of whom, like Woody, had grown up with nothing during the Great Depression and now wanted to explore everything. Their inclinations ranged from brilliant to feral. Browning, who came from a family of poor farmers and had spent two years of his youth eating almost nothing but cabbage, was a perpetual tinkerer. At one point he worked with another Panoramic researcher, Larry Bellinger, to develop the concept for a canine-powered truck called the Dog-Mobile. They also built something called the Hear-a-Lite, a pen-shaped device for blind people that translated light levels into sound.
Bellinger, who had worked as a wing-walker as a teenager (he kept the pastime secret from his mother by playing off his bruises from bad parachute landings as bicycle injuries), had also helped design the Bell X-1, the sound-barrier-breaking rocket plane made famous in Tom Wolfe’s The Right Stuff. Later he created the Mowbot, a self-propelled lawnmower “for cutting grass in a completely random and unattended manner.” (Johnny Carson featured the device on The Tonight Show.)
Then there was Helen Chan Wolf, a pioneer in robot programming who started at Panoramic a couple of years out of college. She would go on to help program Shakey the Robot, described by the Institute of Electrical and Electronics Engineers as “the world’s first robot to embody artificial intelligence”; she has been called, by one former colleague, “the Lady Ada Lovelace of robotics.” In the early 1960s, when Wolf’s coding efforts could involve stacks of punch cards a foot and a half high, she was awed by the range of ideas her Panoramic colleagues threw at the wall. At one point, she says, Woody decided that he “wanted to unravel DNA, and he figured out that it would take 30 or 37 years to do it on the computers that we had at the time. I said, ‘Well, I guess we won’t do that.’
Perhaps not surprisingly, Panoramic struggled to find adequate commercial funding. Woody did his best to pitch his character-recognition technology to business clients, including the Equitable Life Assurance Society and McCall’s magazine, but never landed a contract. By 1963, Woody was all but certain the company would fold.
But throughout its existence, Panoramic had at least one seemingly reliable patron that helped keep it afloat: the Central Intelligence Agency. If any direct mentions of the CIA ever existed in Woody’s papers, they likely ended up in ashes in his driveway; but fragments of evidence that survived in Woody’s archives strongly suggest that, for years, Panoramic did business with CIA front companies. Winkless, who was friendly with the entire Panoramic staff—and was a lifelong friend of Browning—says the company was likely formed, at least in part, with agency funding in mind.
“Nobody ever told me in so many words,” he recalls, “but that was the case.”
According to records obtained by the Black Vault, a website that specializes in esoteric Freedom of Information Act requests, Panoramic was among 80 organizations that worked on Project MK-Ultra, the CIA’s infamous “mind control” program, best known for the psychological tortures it inflicted on frequently unwilling human subjects. Through a front called the Medical Sciences Research Foundation, Panoramic appears to have been assigned to subprojects 93 and 94, on the study of bacterial and fungal toxins and “the remote directional control of activities of selected species of animals.” Research by David H. Price, an anthropologist at Saint Martin’s University, shows that Woody and his colleagues also received money from the Society for the Investigation of Human Ecology, a CIA front that provided grants to scientists whose work might improve the agency’s interrogation techniques or act as camouflage for that work. (The CIA would neither confirm nor deny any knowledge of or connection to, Woody or Panoramic.)
But it was another front company, called the King-Hurley Research Group, that bankrolled Woody’s most notable research at Panoramic. According to a series of lawsuits filed in the 1970s, King-Hurley was a shell company that the CIA used to purchase planes and helicopters for the agency’s secret Air Force, known as Air America. For a time King-Hurley also funded psychopharmacological research at Stanford. But in early 1963, it was the recipient of a different sort of pitch from one Woody Bledsoe: He proposed to conduct “a study to determine the feasibility of a simplified facial recognition machine.” Building on his and Browning’s work with the n-tuple method, he intended to teach a computer to recognize 10 faces. That is, he wanted to give the computer a database of 10 photos of different people and see if he could get it to recognize new photos of each of them. “Soon one would hope to extend the number of persons to thousands,” Woody wrote. Within a month, King-Hurley had given him the
go-ahead.
Read Complete Editorial Here
Posts
