A test subject enters the fMRI machine. ©ATR

Enlarge photo

The pictures shown to the subject (top row); the images recreated based on blood flow data (bottom row).©ATR

Enlarge photo

A group of Japanese researchers led by the Advanced Telecommunications Research Institute International (ATR) has developed the world's first procedure for recreating images processed by the human eye based on analysis of blood flow inside the brain. The process developed by the ATR, a Japanese  independent organization that carries out basic research in the fields of communications, is based on a fundamentally different principle from other methods for reproducing the brain's visual activity. All eyes are on the research team as it works to further refine and advance the new method.

Snapshots from the Mind's Eye When we look at the world around us, light that enters our eyes is converted into electrical signals in the retina, and from there the signals are sent to the visual cortex located in the rear portion of the brain. Based on these signals, neurons in the visual cortex reconstruct a picture of the outside world.

Understanding this process is what enabled researchers to artificially reproduce this imagery. In order to achieve this breakthrough, they honed in on changes in blood flow patterns in the areas surrounding the neurons in the visual cortex. When these neurons become active, their consumption of oxygen and other substances increases, resulting in changes in the surrounding bloodstream. A functional magnetic resonance imaging (fMRI) scanner is used to measure the changes in blood flow, and these measurements are used to extrapolate the activity of the neurons, allowing researchers to attempt a reproduction of the image being processed by the brain.

In this kind of image reproduction, brain activity is regarded as a kind of code, and interpreting this code is known as "brain activity decoding." A previously developed method for decoding involves showing the subject a large number of images, then measuring the patterns in the bloodstream surrounding neurons in the visual cortex that result when each picture is viewed and entering these results into a computer program. When subjects are shown a new image, the resulting bloodstream patterns are compared with the prerecorded patterns to determine its shape. The drawback to this method is that images cannot be distinguished unless a similar one has been recorded for comparison. Discerning complex images through this method would necessitate prerecording an infinite number of patterns.

An image from the analysis of blood flow changes. ©ATR

Enlarge photo

A Breakthrough InnovationThe process developed by the ATR and its research partners involves showing test subjects simple black-and-white images and using data on the blood vessels in the subject's brain to recreate the viewed image on a ten-by-ten grid of pixels, each of which can be either black, white, or gray. If, for example, a subject is shown a white square centered on a black background, an indistinct yet similar image is recreated on the grid based on the patterns formed by the blood vessels in the subject's brain. This method allows for a wide range of images to be recreated without cataloging numerous images in advance.

The new technique has tremendous potential for applications in fields besides imaging. In the area of brain-computer interface (BCI), for example, it is anticipated that it will help humans manipulate machines and devices using nothing more than their thoughts. It also opens up the possibility that scientists may one day be able to turn our thoughts and dreams into visualized pictures. (March 2009)