“But in fact, we found that these cells were responding better to - I guess if I can use poetic license - the dream versions of these natural-world pictures.” “We’re used to thinking that these cells are used to responding to very realistic depictions of the world,” says Ponce. But when they looked closer, the team realized the images resembled real-life objects but didn’t seem quite right.Ī face-like image emerges over the course of several image evolutions. In some instances, the evolved images revealed “features of animal faces, bodies, and even animal-care staff known to the monkeys,” the team writes. The neurons responded well to some mutations and not to others, and as the GAN learned from its mistakes and produced increasingly more stimulating pictures, the scientists began to recognize familiar objects through the fuzz over one to three hours, depending on how long the monkey could sit still. Likening the image-generating process to human reproduction, Ponce said that the most stimulating pictures “had sex” with one another to create increasingly stimulating offspring. The monkeys started off by looking at a black and white texture - to keep it “non-biased,” says Ponce - that evolved, ever so slightly, as the neural network responded to the changes that made its neurons fire the most. "We found that these cells were responding better to … the dream versions of these natural-world pictures." On the right are the "natural images" that the scientists used to identify the face neurons in the first place. The images on the left are what the GAN produced in response to the activity of face-recognizing neurons. “They looked like objects in the world that were not in the world,” says Ponce. As the images evolved, one thing became clear: These cells are into some weird shit. In Ponce’s experiments, the discriminator was the monkey neuron, hooked up to the GAN, which burst with activity if it approved of the image it saw. These generative adversarial networks, or GANs, evolve images based on input from a “discriminator” that determines what’s good and what’s not. used to generate imaginary but uncannily realistic images like DeepFakes and other creepy art. To do the impossible, Ponce and his team took advantage of a powerful new tool. Previously, researchers investigated this by showing subjects countless images to find out what was best at turning their neurons on - an impossible task, since there are an infinite number of images to show. Scientists trying to understand this aspect of our visual systems are trying to understand how it is we evolved to not only see but also recognize complex images like faces, and also objects, places, and animals. “We’ve been stuck with this problem for decades,” first author Carlos Ponce, Ph.D., a neuroscientist at Washington University School of Medicine in St. "They looked like objects in the world that were not in the world." Is it a certain eyes-nose-mouth combination that triggers its frenzy? A particular arrangement of colors? What is a face, to a neuron? In a groundbreaking Cell study, scientists found out through an unusual approach: They asked the cells themselves. For a long time, scientists have pondered what it is, exactly, that tickles the very particular fancies of these neurons. And when the team hooked XDREAM up to another of the monkey’s visual neurons, it produced a distorted image of a face in a white mask.Every time you look at a face, a group of neurons behind your ears goes wild with excitation.
Diane is one of the monkeys’ caretakers, who feeds them while wearing blue scrubs and a white face mask. “And then, a few days later, we evolved Diane,” she adds. “We all looked at it and said, ‘Oh, that’s Anthony,’” says Margaret Livingstone, a neuroscientist at Harvard Medical School.
#XDREAM EVOLVING IMAGES FOR VISUAL PATCH#
Soon a red patch appeared next to it, which reminded the watching researchers of the red collar worn by a monkey who lives in the cage opposite Ringo’s.
Two black dots with a black line beneath them, all against a pale oval. But as time passed, “from this haze, something started staring back at us,” says the neuroscientist Carlos Ponce. As the images evolved, the neuron fired away, and the team behind XDREAM watched from a nearby room.Īt first, the pictures were gray and formless. The pictures were created by an artificial-intelligence algorithm called XDREAM, which gradually tweaked them to stimulate one particular neuron in Ringo’s brain, in a region that’s supposedly specialized for recognizing faces. In April 2018, a monkey named Ringo sat in a Harvard lab, sipping juice, while strange images flickered in front of his eyes. The article is really interesting, here’s the setup:
An Atlantic article describing the work is here.