MIT Media Lab
Skills
User Research, Data analytics, Haptics
Collaborative Research
PurrfectPitch
Hypothesis:
1) Multisensory feedback will help those with no prior musical background more easily identify different musical intervals
2) Haptic feedback will help novice learners identify the intervals vs. those without the feedback.
Process overview
Vision
Pair haptics and audio together to help participants quickly and intuitively learn to identify pitch intervals. Musical ear training is the ability to identify the intervals between musical chords by ear. Identifying these intervals requires extensive training and the traditional approach of auditory repetition may be strenuous.
This research explores the power of haptics in short term pitch interval training.
A vest lined with motor casting is fitted onto each study participant. The motors are connected to a bluetooth that translates the musical intervals being played on the computer to the worn device. For example, if the notes 1 and 3 are played, motors one and 3 on the vest (from bottom to top) will vibrate.
Phase 1: the participant first completes a test where they hear two notes and must select what they believe the interval is without the aid of haptics. This is a test conducted to measure the learning post haptics.
Phase 2: The user is given both audio and haptic feedback to train on identifying pitch intervals.
Phase 3: The user takes a final test to identify pitch intervals using only audio. This test is to measure the learning impacts of haptics for interval training.
Wearable improvements
When I joined the Purrfect Pitch study, my primary role was to bring my product sense and user experimentation experience to help improve the study.
One of the main challenges for this wearable device is ensuring that it can universally fit different body types. For this experiment to be run accurately, the motors on the vest need to sit flush against the user, or else the user will be missing some of the haptic experiences.
However, upon joining the group, I noticed that depending on the person's body type, some of the motors would barely hover on one's back. I was able to identify common weak spaces for the device that needed additional support.
Furthermore, I noticed that the experience of putting the vest on was challenging in itself. The vest used buckle clasps to secure the straps on the vest. Although that seems fine on paper, in practice it led to a struggle of tightening and untightening every single strap multiple times for each study participant. Despite seeming small, this added a layer of friction for the study participants before they were even able to begin the real experiment. This would impact the NASA TLX down the line. I proposed switching from buckle clasps to velcro to make vest adjustments easier. This cut down the time needed to adjust/readjust straps so that the user could simply put on the straps to match their body.
Study Collaboration
Developing controls for the study
I wanted to ensure there was a smooth process flow that existed for every user who participated in the study. To prevent any errors from happening on the experimenter side, I developed a step-by-step script with exactly what each study conductor should say and any tasks that they would need to do. Due to the multitude of steps for this experiment, I wanted to proactively prevent user study errors from occurring. Figure 1 shows a small portion of this script. The top of the script shows a high level of what the interview flow would look like before it dives into specifics for each step.
Additionally, another space I saw for adding controls to the study was the training portion. In the original PurrfectPitch study, the training portion was gamified by adding different streaks for the number of correct answers a participant got in a row. If the user missed an answer, the full streak would disappear and they would start from 0. Both from my own experience and from participant feedback in the early rounds, I noticed a common theme of frustration and defeat once large streaks were wiped out. Since we were gathering participants with little to no musical training, the possibility of them maintaining a high streak till the end of each 10-minute training section was extremely unlikely.
I proposed that gamification with almost no chances of 'winning' can lead to disengagement. For the purpose of time, we removed the 'streak counter' element rather than revamping the gamification. Check out Figure 2 to see what the current interface now looks like for correct vs incorrect user input. The number in the middle indicates the real interval being played to the user.
Tracking data
I developed this post-survey feedback excel sheet to easily track and compare different responses participants had to the study. This format enabled us to 1) ensure we are asking the same questions to each participant, and 2) directly compare across question/answers that each participant gave. In the final study that is being tracked within this sheet, 5xx was used to compare nonhaptic users and 6xx indicated haptic users. Check out Figure 3 below.
Key takeaways from qualitative data analysis
1. The Haptics study group would use the haptics sensation in parallel to the audio to identify the correct interval. When they moved from the training portion to the final test portion of the test, the haptic device would be turned off. They would have often attempted to recall the feeling of the haptics while listening to the audio.
2. The Audio-only study group would try to identify patterns from the intervals they heard when trying to select their answer.
3. The Haptics group reported lower levels of stress and frustration than the Audio-only group.
Future Iterations
A few ways to continue improving the study would be:
1. Simplifying the study process.
Pretest 1 Learning session Final test
2. The training session is replaced with a learning session. Some of the feedback indicated that participants had to learn the basic concepts of intervals while they were being given prompts (training). Rather than giving a feedback-oriented learning session, have the users hear and feel the intervals at the same time alongside the correct answers.
3. Move the haptic device to a different body part (ie. along the arm). One of the participants shared the knowledge that the back is one of the least sensitive body parts. This meant at times that the haptic sensation was very faint.