72 hours (Student Design Challenge, Interaction 18)
Kevin Ong, Mélodie Jacob
Concept, User Research, Content Strategy, Project Management
Upon arrival, I was matched up with two other finalists: Kevin Ong of Malmö Universitet in Sweden and Mélodie Jacob of the Pôle Supérieur de Design in France; together, we were Team Pirouette. During the 72-hour inclusive design competition, we were challenged to come up with a concept to teach rhythm to a population with one of four disabilities: blindness, deafness, mobile dexterity impairment, or dyscalculia.
Project keyHue teaches students with dyscalculia how to read and play music, using projection mapping to display color on a piano keyboard, and corresponding colors on digital sheet music. These visual guides, and especially the rhythm and repetition of the guides, will teach a student to play the correct keys for the right duration. The system will manage the student’s pace through each song, and gradually transition them to eventually read and play music without color guidance. At that point, the digital sheet music will look exactly like its paper version.
Project keyHue was awarded first place at Interaction 18’s Student Design Challenge.
To kick off the competition, the team from Microsoft, led by Principal Design Strategist Margaret Price, conducted a workshop to share the principles of Microsoft's inclusive design practice. They include recognizing exclusion, learning from diversity, and solving for one to extend for many. Creating a persona spectrum accounts for those with permanent disabilities as well as those with temporary and situational disabilities, resulting in benefits to more people.
The brief challenged us to create an interactive way to teach the concept of rhythm, while ensuring our design provided access and reduced friction for the widest number of people. We started by ideating broad concepts of rhythm, and categorizing them by scale of personal to universal.
We were then presented with four constraints to consider: blindness, deafness, mobile dexterity impairment, and dyscalculia. After conducting secondary research on all four disabilities, and brainstorming how each affects rhythm, our team decided to design for people with dyscalculia for two main reasons: (1) none of us had heard of dyscalculia before the challenge and wanted to learn more, and (2) we wanted to raise awareness about dyscalculia, after learning it is as prevalent as dyslexia, yet relatively unknown and an underfunded area of research.
Dyscalculia is a learning difference that makes it difficult to ascribe meaning to numbers. People with dyscalculia may have trouble with calculations and problem solving, but also with telling time, spatial awareness, or memorizing lists. Recommendations to help address these difficulties include: using fingers for counting, and color to help distinguish between different numbers and develop categorization skills, gradually transitioning away from color to black and white.
Focusing on dyscalculia early on allowed us to conduct primary research. We reached out to all of our networks using social media, and to the 1,200 conference attendees using Slack, to try to connect with people with first-hand experience with dyscalculia.
Through our outreach, we were able to connect with a father of a 14-year-old girl, who hasn’t been diagnosed with dyscalculia, but exhibits many of its symptoms. Some insights gleaned from our Skype interview were:
She lives as independently as any other teenager in New York, despite having trouble with memory, telling time, and directions.
Lots of repetition and confirmation has helped her and her family daily
She was encouraged to learn music from a young age to counter her difficulties learning math. Now, she can read music and plays guitar and piano, and sings.
Referring back to our research was critical as we ideated. We also found value in using real content—in our case, the song “Mary Had A Little Lamb.” Not only was it a familiar and relatively simple song, but it also comprises quarter, half, and whole notes, allowing us to test and strengthen our concept.
Extending for many
Although designed for those with dyscalculia, our concept can benefit many others, including anyone learning a musical instrument for the first time, or someone who has only learned to play by ear and wants to go further.
Presentation and Judges’ Critique
We presented Project keyHue on stage on the last day of Interaction 18, and were thrilled to be awarded first place.
“Project keyHue tackled a challenge most people knew nothing about,” judge Haiyan Zhang, Director of Innovation at Microsoft Research, said. “They were able to bring us into the experience of the user, through their video and their research, and by talking to someone with dyscalculia and speaking to some of the emotional challenges they face. Their idea was very innovative, and I liked the combination of an on-screen app and the physicality of still relying on the piano, so a person can, over time, learn how to play a real piano and learn the muscle memory.”
“They had an idea to solve some of the challenges their research subject had, and you could see how it was extensible,” said judge Neil Churcher, Head of Design at Orange. “They also had a believability to it that came through in their presentation, which gave them a little edge.”
If we had more time, we would have liked to test our prototype with more users, especially those with dyscalculia, and musicians. We also want to research the technical feasibility of our concept further.
Overall, the Student Design Challenge was an incredible experience filled with learning and building meaningful connections. I was reminded of the value of being part of a great team, conducting primary research, and prototyping with real content.