Using Brain-Wave Technology to ... Create Art?
We pointed and we clicked. Then we tapped and we swiped. Some of us spoke to Siri, some waved our hands at the Kinect.
Yet the quest to control our technology with minimal effort continued. And though, in 2015, we haven’t quite come up with a system that literally reads our minds, we’re getting closer to one that responds to our brain waves.
It’s a thought that’s been on the minds of many for a while now. This past summer, my Yahoo Tech colleague Dan Tynan braved the existing marketplace of products that point to a future in which we can “control objects — your phone, computer, car stereo, game console, lights inside your house, and more — using only your thoughts.”
Today, the technology’s practical applications remain a tantalizing idea. But brainwave-reading has made the most headway in what is perhaps a surprising field: the arts.
That may sound like the exact opposite of “practical,” but even the most out-there artistic experiments with brainwave technology have a payoff beyond creative expression. In short, they’re helping us understand what a “mind-reading” computer interface can really be — both in its potential and its limits.
HOW IT WORKS
First, a note of context: What are we actually talking about when we talk about brain-wave interfaces? It’s not quite as sci-fi as it sounds. The brain’s normal processes — neurons and ions going about their business — result in electrical activity that medical scientists and others have been able to measure and monitor for decades. The practice of recording this activity has the impressive name electroencephalography (more typically shortened to EEG). Ever seen images of a person’s head strapped up with an outbreak of wired sensors? That’s an EEG setup.
Streamline that mess of wires into a more practical object, and you have something like the “neuroheadsets” made by firms like Emotiv and NeuroSky, among others leading the charge of brain-wave control into the consumer market. And once you have a device that can capture brain activity in the form of data, you have an input that can be translated to some form of output.
We’re a ways from a scenario in which the captured input is as specific as “I want to watch Broad City,” or “Follow Yahoo Tech on Twitter.” But as a variety of artists and technologists are demonstrating, it’s already possible to do some pretty creative things simply using off-the-shelf brainwave technologies — if, of course, you put your mind to it.
BRAINY MUSIC
For a piece a few months back, Lisa Park used an Emotiv headset feeding information to a “brain wave-interpreting algorithm,” according to Creators Project, and surrounded herself with speakers connected to shallow metal bowls filled with water. These responded to brain data assigned to various emotional states, meaning the sounds and rippling water around her reflected (roughly) what was on her mind.
Skip ahead to about the two-minute mark in this video to see what that looks like in practice.
In these early stages of brain-wave innovation, music and sound projects have proven relatively popular. Earlier, for instance, cellist Katinka Kleijn, wearing an EEG monitor, performed a duet with sounds translated from her own mental state.
That’s right: She shared the stage with her brain. And in a very real (if also very unusual) sense, she made music with her thoughts.
And most recently, artist Aiste Noreikaite rigged a motorcycle helmet — dubbed “The Experience Helmet” — with sensors meant to gauge attention and focus. Using that data, it produces sounds that reflect this data back to the user, who in effect “hears the sound that is being generated by his brain waves.” The upshot: aural feedback that reveals one’s own state of mind, and adds an unpredictable dimension to life’s natural sound track.
THE ART OF THINKING HARD
One of the more complicated, and fascinating, brain-wave creativity experiments involved visual art — but nothing as straightforward as using the mind to move a paintbrush over a canvas. Chinese artist Jody Xiong’s “Mind Art” project equipped 16 disabled men and women — many in wheelchairs or missing limbs, or both — with brain-wave-reading headsets. (They were recruited partly via the social networks Weibo and WeChat, both popular in China, according to Wired.)
These were connected to “tiny detonators” on balloons filled with paint. The balloons were lowered in front of canvases. The electric signals produced by participants’ focused concentration triggered the detonators, popping the balloons and releasing wild abstractions of paint.
A moving video documents the process and resulting exhibition.
An earlier thought-provoking visual project brought brain activity to the fashion realm. Dubbed “NeuroKnitting,” this effort involved translating brain-wave measurements corresponding to relaxation, excitement, and “cognitive load,” as measured by an EEG headset on a person listening to Bach’s “Goldberg Variations.”
The unique results served as patterns for knitted garments.
A NeuroKnitter with his finished product. (NeuroKnitting)
And then there’s “Solaris,” a project that involves translating the brain activity of participants wearing Emotiv devices into the patterned movement — “corresponding to the strength and formation of their focus,” according to Creators Project — of magnets across a fluorescent pool of liquids.
“The installation,” the creators of “Solaris” write, “visualizes the temperament of the person. [The magnet pattern] copies your mental organization and echoes it on the liquid’s surface.” It’s hard to pass judgment on that claim, but it sure looks cool:
BRAINWAVE TECHNOLOGY FOR THE REST OF US
Mind-bending as some of this experimentation is, it’s not that hard to imagine how some of it could overlap with more everyday uses. If a headset can detect your mood, maybe it can respond to your brain-wave patterns with appropriate music, for instance.
Similarly, there’s obvious crossover between the feedback of the above-mentioned “Experience Helmet,” and Muse, one of the consumer-focused gadgets my fellow Yahoo Tech-er Tynan tried last year.
Instead of a helmet, the Muse is a fairly simple-looking headband, embedded with a cluster of sensors. The idea is something close to a meditation trainer, translating levels of brain activity into an app graphic, gauging whether you are calm and clear, or your head is buzzing with worries and distractions. The gizmo also offers aural feedback, and with practice, you’re supposed to be able to use these sonic prods to help you learn to achieve a peaceful state of mind.
The Muse headband.
In the past, Muse has included smartphone control among its potential goals — replacing the click and the tap. Samsung, meanwhile, has experimented with an EEG-monitoring cap that could ultimately “bring mind control to its mobile devices,” MIT Technology Review reported in 2013, “with the hope of developing ways for people with mobility impairments to connect to the world.” Researchers found that subjects “could launch an application and make selections within it by concentrating on an icon that was blinking at a distinctive frequency.”
There are brain-wave games and toys, as well, hinting at the ways this kind of technology may become more familiar to the masses. In fact, one of the most potentially useful brain-wave projects is essentially an educational doll designed to teach users … about brain waves.
The idea is to see how your mental activity corresponds to the information EEG sensor can collect — here’s the part of your brain lighting up in connection with movement, or mental calm, and so on. The doll currently exists only an experimental context, but at least it has a cute name: Teegi.
If you think about it, just thinking about it must be the most effortless a way to make a gizmo respond to one’s wishes. But there’s still something a little off-putting and weird about the whole idea of technology “reading” our brain waves and converting them to actions.
So it may be that the most important thing these experiments achieve is simply helping to demystify this process. Maybe they’re helping us, in other words, wrap our brains around this stuff.
Write to me at rwalkeryn@yahoo.com or find me on Twitter, @notrobwalker. RSS lover? Paste this URL into your reader of choice: https://www.yahoo.com/tech/author/rob-walker/rss.
