We Have The Technology

How amazing is this video of a little baby getting his cochlear implant switched on? Also how amazing is the “YouTube : cochlear implant videos :: photography : hearing aid pictures”analogy? I saw this on Reddit this morning. There are dozens of these videos on YouTube (like this one).

Along with the controversy in the Deaf community, another issue with cochlear implants is that while they allow their users to understand speech and hear other environmental cues like doorbells and car horns, they don’t completely restore what we would generally think of as “hearing,” per se. The way the devices work is pretty simple: take an audio signal from a microphone, feed it to a filterbank with somewhere on the order of 8–20 bands. Take the levels from the filterbank’s outputs and map them to electrodes inserted into the cochlea, and hope the brain can reapply the tonotopic mapping in the cochlea to the pattern of stimulation from the electrodes.

This is essentially what the front half of a vocoder does. Not T-Pain’s Auto-Tune with retune speed set to zero that everyone calls “vocoding,” I’m talking about Roger Troutman and Zapp, California Love, Kraftwerk-style. Instead of using the vocoder front end to drive a filterbank hooked up to a keyboard or something, cochlear implants plug that part straight into the brain. It’s futuristic in a disco way.

Consequently, a cochlear implant does not let its user hear music, at least not in any way like what the hearing community would consider to be listening to music. When you hear Roger Troutman sing I Want To Be Your Man, all the pitch information is coming from the keyboards. The filtering, the harmonic structure of “Wanna Be Yooouuuur MaaAaaAn” comes from his voice and the vocoder, but it tosses out the pitch information, and the keyboard brings it back (which is why it sounds similar to Auto-Tune: the pitch is locked perfectly and has instaneous jumps with no gliss or portamento, because it’s a keyboard. Just like Auto-Tune when the retune speed is zero). This isn’t exactly how the cochlear implant works, but it’s similar.

The processing strategy is a main block upon which one has to choose the implant manufacturer, research shows that patients can understand speech with as at least 4 electrodes, but the obstacle is in music perception, where it returns that fine structure stimulation is an important issue. Some strategies used in Advanced Bionics and Medel strategies make use of fine structure presentation by implementing the Hilbert Transform in the signal processing path, while ACE strategies depends mainly on the Short Time Fourier Transform.

Ignoring for a second how not up to Wikipedia’s quality standards this bit is, how awesome is it that people are even working on this? “What? The electronic device you implanted in that child’s brain allows them to hear speech, ambient environmental sounds, and all the other auditory cues necessary for success in human society but NOT MUSIC? DID YOU TRY THE HILBERT TRANSFORM?” If I was a newspaper editor and I had to hed this story I’d go with “HUMANS: NOT TOTALLY SHITTY?”

(Further reading: a 2005 Wired article by a guy with a cochlear implant trying to improve its performance on music.)

Pony Glass

Pony Glass

( via MOMA)

The Re-Animator

Here’s an interesting studio visit with Los Angeles filmmaker Lewis Klahr. I had the opportunity to see his collage-noir Two Minutes to Zero Trilogy (2004) last night, and I highly recommend seeking out his work. The trilogy takes it’s source material from the 77 sunset strip comic books, telling the story of a bank robbery gone awry in three narrative speeds. The first film, Two Days to Zero clocks in at 22 minutes, while the second film is a more taught 8 minutes and features a Rhys Chatham soundtrack. Finally the last film, Two Minutes to Zero is roughly a minute long burst set to Glenn Branca’s The Ascension. Sadly, I was unable to find any clips of that particular film online, but this video has clips from several of his other works.