We Have The Technology

May 28, 2010

We Have The Technology

How amazing is this video of a little baby getting his cochlear implant switched on? Also how amazing is the “YouTube : cochlear implant videos :: photography : hearing aid pictures”analogy? I saw this on Reddit this morning. There are dozens of these videos on YouTube (like this one).

Along with the controversy in the Deaf community, another issue with cochlear implants is that while they allow their users to understand speech and hear other environmental cues like doorbells and car horns, they don’t completely restore what we would generally think of as “hearing,” per se. The way the devices work is pretty simple: take an audio signal from a microphone, feed it to a filterbank with somewhere on the order of 8–20 bands. Take the levels from the filterbank’s outputs and map them to electrodes inserted into the cochlea, and hope the brain can reapply the tonotopic mapping in the cochlea to the pattern of stimulation from the electrodes.

This is essentially what the front half of a vocoder does. Not T-Pain’s Auto-Tune with retune speed set to zero that everyone calls “vocoding,” I’m talking about Roger Troutman and Zapp, California Love, Kraftwerk-style. Instead of using the vocoder front end to drive a filterbank hooked up to a keyboard or something, cochlear implants plug that part straight into the brain. It’s futuristic in a disco way.

Consequently, a cochlear implant does not let its user hear music, at least not in any way like what the hearing community would consider to be listening to music. When you hear Roger Troutman sing I Want To Be Your Man, all the pitch information is coming from the keyboards. The filtering, the harmonic structure of “Wanna Be Yooouuuur MaaAaaAn” comes from his voice and the vocoder, but it tosses out the pitch information, and the keyboard brings it back (which is why it sounds similar to Auto-Tune: the pitch is locked perfectly and has instaneous jumps with no gliss or portamento, because it’s a keyboard. Just like Auto-Tune when the retune speed is zero). This isn’t exactly how the cochlear implant works, but it’s similar.

The processing strategy is a main block upon which one has to choose the implant manufacturer, research shows that patients can understand speech with as at least 4 electrodes, but the obstacle is in music perception, where it returns that fine structure stimulation is an important issue. Some strategies used in Advanced Bionics and Medel strategies make use of fine structure presentation by implementing the Hilbert Transform in the signal processing path, while ACE strategies depends mainly on the Short Time Fourier Transform.

Ignoring for a second how not up to Wikipedia’s quality standards this bit is, how awesome is it that people are even working on this? “What? The electronic device you implanted in that child’s brain allows them to hear speech, ambient environmental sounds, and all the other auditory cues necessary for success in human society but NOT MUSIC? DID YOU TRY THE HILBERT TRANSFORM?” If I was a newspaper editor and I had to hed this story I’d go with “HUMANS: NOT TOTALLY SHITTY?”

(Further reading: a 2005 Wired article by a guy with a cochlear implant trying to improve its performance on music.)