When people think of the next big thing in mobile experiences, they think visual, like virtual reality and augmented reality. Forget strapping your phone to your face, though; all you have to do to experience the world in a new way is plug in your earbuds and download Hear.
Most music apps put your favorite artists or shows in your ear and let you stream them at will. That’s not quite the case with Hear. Built by RjDj, a startup founded by last.fm cofounder Michael Breidenbruecker, the app turns a person’s surroundings into a shifting soundscape that is composed in real-time by the world around them.
It’s part of RjDj’s pioneering genre called reactive music, a type of sound that reacts to the listener and their environment. Hear is the latest distillation of that concept and an expansion on the company’s original RjDj app that, back in 2009, gave users an experience like auditory hallucinogenics.
Hear gives users eight filters through which to hear the world, like Instagram but the lens is the headphone jack. Each one of the effects aims to provide users with a different way to hear the world, emphasizing different sounds to provide a different impact on how a person hears what is happening around them.
Breidenbruecker described the filters, comparing the way it caught and translated sound in terms that one might use to describe a particular on a song. “Sounds or noises become distractive or a source of stress when they are not part of a known sound structure or sound source. Our brain then interprets it as danger,” he explained to the Daily Dot. “Imagine a drum break without the song before or after the break. That would be shocking, but as part of a song it’s totally embedded and perceived as something entirely different than shocking.”
Breidenbruecker emphasized two filters, Relax and Super Hearing, as primary examples of what Hear can do. With Relax, he said, “what we are doing is harmonizing acoustic distractions and making them part of a bigger thing that the brain already knows.” The filter runs sounds through the microphone and transforms them into soothing, harmonic chords that the brain won’t perceive as standing out, but blending in.
With Super Hearing, the effect is almost the exact opposite. “We present sounds to the brain as an unknown and new structure with the effect that the brain puts more attention to those sounds,” Breidenbruecker explained. The mode focuses the brain on the disruptive sounds and gives the effect of extra sensitive hearing.
The underlying technology that powers Hear is complex, and it utilizes a variety of advanced algorithms, adaptive compression and equalization techniques, and dynamic controls to create the filters.
https://www.youtube.com/watch?v=pgCjYk1IV28
Breidenbrucker said the biggest challenge wasn’t necessarily building out the technology but rather making it work with inadequate equipment. “The difficult thing is to get this going with a really shitty microphone,” he said, calling out the cord-bound mic on the Apple’s EarPods. But in just working with what was available, Breidenbrucker said he and his team were “very surprised” by the results.
So too, will you be if you plug in your headphones and let Hear compose for you a song as you go about your day. It’s compared regularly to psychedelics, which seems ridiculous at the outset until the headphones are filling your ear with bizarre echoes and reverberations and suddenly you’re moving and speaking slower to hear the effects. Every creek of floorboards and shuffling of feet becomes a new experience.
Some modes are better than others; Hear works well with music running, and often the sound of the app fades into the soundtrack and you’ll forget it’s even there until the microphone catches some unexpected noise and there’s a sudden jolt of indescribable noise. Other filters are genuinely unnerving and induce a paranoia about the sounds produced and where they’re coming from.
Breidenbrucker said the comparison to drugs was an obvious one, but one that he still welcomed. “When people describe their experiences with hear and reference psychedelics it actually means to me that it works,” he said.
For now, Hear is a fascinating and dreamlike experience and one that, depending on your threshold for the auditory trip it takes you on, is mostly a novelty. But Breidenbrucker and his team see it primarily as a building block to create a truly engrossing audio experience.
H/T Fast Company
