An audio-tactile algorithm developed by scientists from the University of Malaga transmits melodic information through vibration.
A groundbreaking prototype developed by experts from the Department of Electronics at the University of Malaga and members of the Electronics for Instruments and Systems research and development group will allow people with hearing loss to hear music through the sense of touch.
It consists of an audio-tactile algorithm that converts monophonic music into tactile stimuli based on vibrations using “tactile illusions”. According to the researchers, “It’s like ‘hacking’ the nervous system to get a different response to the actual stimulus being sent.”
“The long-term goal is for people who can’t hear to be able to ‘hear’ music,” says researcher Paul Remache, the lead author of this paper, who insists on music’s power to affect mood and its potential as therapy mental disorders and treatment of pain.
The researchers predict that this will result in a portable terminal that can be taken to a concert, as this prototype will be easily transferrable to tech devices such as smartphones.
Researchers at the Faculty of Electronics have developed an audio-tactile algorithm that transmits melodic information through vibration. Credit: University of Malaga
This young researcher, in collaboration with UMA professors Andrés Trujillo and Fernando Vidal, has developed an algorithm capable of converting musical features and structures from MIDI files – Musical Instrument Digital Interface – into “vibrotactile stimuli”.
“It’s sort of like mapping music,” explains Remache, adding that it’s possible because not only can this type of file be played and created, but it also provides “symbolic representations.”
Current models do not guarantee the correspondence between the emotional response to music and the vibrotactile version of it. In light of this, these UMA engineers propose an arrangement of the “tactile illusions” to enhance and broaden the spectrum of musical features by making vibration more dynamic in terms of movement, change of direction and location.
“It’s a challenging process because the perceivable frequency range of the skin is lower than that of the auditory system, which can lead to the loss of some musical characteristics,” they explain.
Different emotional reaction
The results of the first experiments, in which more than fifty volunteers took part, indicate that the arrangement of “tactile illusions” evokes more positive than negative emotions. They are also found to be more enjoyable and stimulating than the audio, and provoke a different emotional response than the original music.
Smart healthcare instrumentation and application
This first prototype was presented at the 11th International Workshop on Haptic & Audio Interaction Design (UK) – the largest international event specializing in these fields of study – after being published in the scientific journal LNCS. The UMA researchers are currently working on a second model and are continuing the experiments.
The research is the result of Paul Remache’s doctoral thesis and is part of the National Plan project “Intelligent Instrumentation and Applications in Healthcare”.
Reference: “Mapping monophonic MIDI tracks to vibrotactile stimuli with tactile illusions” by Byron Remache-Vinueza, Andrés Trujillo-León, Maria-Alena Clim, Fabián Sarmiento-Ortiz, Liliana Topon-Visarrea, Alexander Refsum Jensenius and Fernando Vidal-Verdú, 18 August 2022, Lecture notes in computer science.