Video: the invention bringing music to the hearing impaired

Social entrepreneur Raphael Silva had already created a tech solution aiming at bridging the gap between music and hearing impaired people. Then he started working with coders and developers at Sónar’s +D offshoot to prototype some new ideas.

How did you end up at Sónar +D? I went last year, to talk about Ludwig. This year they had the Social Innovation Challenge. There were six companies with teams, including Red Bull Amaphiko, and I went to mentor the team.

Had you been to Sónar before? No, I hadn’t. I’d been to Lollapalooza to see Arctic Monkeys. I’m a big music fan. When I’m working I’m listening, when I’m riding the bike, I’m listening. I’m really connected to that. What am I listening to right now? An artist called Israel Subirá.

You were working with artists, coders and a graphic designer. What unexpected elements did they bring to the challenge? Actually the whole experiment was unexpected. I was expecting something like wearable, like Ludwig, but the guys had these idea of a creating these box with some chords that vibrated, with projections to create another experience. This was all unexpected. It was so cool, so different. The result was great.

What did you create? Imagine a box of around 1.5m squared, and that this box is open in the centre. In the side parts we have a lot of strings going through the box and as it’s open you can put your hand in and feel them vibrate. Then there are some projections behind the box so that beyond the feeling, the tactile part, you can see what the music is saying with colours and images.

How was it used? We had music playing through one of the team’s Spotify playlist. The music was translated in real time to motors and projections. The guys created some patterns and colours in the images so the projection would follow the music. The motors would follow the music, to make the strings vibrate more or less.

What reaction did you get from people? Hearing people like it too, because it’s different. We are not using it to enjoying music by feeling it, by closing our eyes and feeling the music in the strings, so it’s a bit strange for us but people were really positive.

What do you want the deaf and hearing comm to get from formats like the one you created, and from Ludwig generally? I really want people to be more empathetic. It was never about ‘let’s bring music to deaf people’. It’s about having more empathy, about being more inclusive and about how we can demonstrate love in another way. How we can make it more inclusive world. It’s all about that. For the team, I tried to suggest this idea: how can we be a community together? An equal community together, to make experiences that we all take part in.

What’s your connection to the deaf community? The connection came after Ludwig. All of the people in the Ludwig team were into music and we started like ‘how can we expand these experiences to people who don’t hear?’ The connection was born there, then we started to interact with the deaf community and hear their ideas and how they felt about Ludwig and how we could improve. Originally, it came from my love of music.

What’s next? It’s not official so I don’t want to say anything yet but there is something going on. But yeah, not much I can tell right now. Something exciting, yes.

Video: Gabriel Tineo. Video edit: Paul Bray.

Related project

Levamos a experiência de ouvir música aos deficientes auditivos/ We bring music to the deaf community

Log in to follow