Kico – Hearing agency for hard-of-hearing people

Copenhagen Institute of Interaction DesignNetherlands

Empowering

Concept / Student

Team

Hiroki Sato : design research, concept development, rapid prototyping (Sketch, Origami, Javascript, ml5js, p5js, Ableton Live, Node.js and Arduino) and video production (filming and Premiere Pro).

Overview

Verbal communication between hard-of-hearing people and normal hearing people can be clunky. Normal hearing people may start speaking less clearly as conversation goes along. And both parties can feel stress when something has to be repeated many times. While there are technical advancements facilitating hard-of-hearing people’s “hearing strategy”, there’s not as much improvement on “speaking strategy” for normal hearing people as them.

Kico is a system to create communal efforts by both hard-of-hearing and normal hearing people to engage in understandable conversation, and is conceptualized to function as agency for hard-of-hearing people by AI understanding individual hearing difficulty.

With ‘Polite Reminder’, normal-hearing people are reminded to speak at appropriate pitch, volume and speed through haptic feedback by the system, on behalf of hard-of-hearing people. They can also learn the proper way to speak through visualization of their voice with ‘Speech Training’.
Hard-of-hearing people play a role in training the system by ‘Rewind’. The function to record and play back recent conversation enables them not only to listen back to what was said, but also to mark the difficult sounds, by which AI figures out individual hearing difficulty.

The introduction of the system in conversation creates a new way of communication, in which a machine adds meaningful value by interacting with and reduces stress for both speakers and listeners. It explores the positive aspect of AI by the humane use of it for the matter AI is capable of.
Regarding the process, the project involved 2 Experts, 8 hard-of-hearing people (2 with cochlear implant, 2 with BAHA, 4 with hearing aid), 2 mothers of them and 11 normal-hearing people in 16 interviews / co-creation sessions.

The project was developed along with 3 key findings.

Firstly, I saw pains in asking for repeating so many times everyday for a BAHA wearing daughter and her mother. The daughter sometimes refuses to understand conversation because of the stress. That insight set the important concept that a machine should play a role which could be stressful or regarded as rude if it is done by humans.

Secondly, I learned from language pathologists that there is no single way to speak properly because each hard-of-hearing person hears differently. And my experience of hard time making myself understood for a person wearing cochlear implant, who understands male voice poorer than female because of the pitch of sound he can hear, reminded me of the necessity of visual targets to find the right tone of voice.

Thirdly, an insight of a person wearing hearing aids that it is hard to ask for repeating when talking with more than 4 people led to a 5-second version of ‘Rewind’. For the first prototype, a mother of a daughter wearing BAHA added on smart ideas to control the duration of repeated sound and change the difficult sound so that it gets easier to understand by the system’s personalization.

All 3 directions / working prototypes were validated passionately by all the hard-of-hearing participants / families and an associate professor in DTU.

ABOUT IXDA

Interaction Awards is an initiative of the Interaction Design Association (IxDA), a global community of over 100,000 individuals worldwide dedicated to the professional practice of Interaction Design. Find out how to join your local group and get involved at www.ixda.org.

FOLLOW US
© 2012 - 2021 Interaction Design Association