Immersive Tech Gives Disabled Musicians Customized VR

A new Virtual Reality musical interface is allowing musicians with disabilities to compose, record, and perform their music in a customized way.

Immersive technology like VR is transforming the music industry, but often requires certain visual and motor skills to use. This makes it difficult for people with disabilities to access. But a team of engineers, scientists, and musicians lead by Dr. Franziska Schroeder of the Sonic Arts Research Center in Queens University Belfast, is working to introduce more assistive technology to musicians with disabilities. 

The group, a part of Performance without Barriers, designs assistive technology musical interfaces, such as a virtual reality studio, that maps itself to the user’s specific abilities. Rather than the standard designs, this technology learns from the musician, adapting to whatever mobility needs or creative interests they have.  

To achieve this, Performance without Barriers’ researchers paired with Software developer Zack Kinstner, creator of the Infinite Instrument, an immersive musical studio that allows users to create music using instruments they design. The Infinite Instrument runs on a 360 VR headset called HTC VIVE, with hand-held controllers for the user to access commands.

Schroeder and her team reworked the immersive studio to be more accessible to musicians with different abilities. For instance, one of their designs allowed a musician with cerebral palsy, who did not have precise motor skills in her arms or fingers, to use her own upper body movements to create music in a way that suited her.

Soon they discovered that with feedback from the controllers, musicians could play music by touch and sound alone, without necessarily seeing what they were doing.

Using their Sonic LAB in Queens University Belfast, Ireland, the team adapted the VIVE technology for musicians with disabilities, allowing a blind clarinetist to make use of the Infinite Instrument immersive studio by strapping a VIVE controller to his clarinet while playing. This allowed him to use the controller and his movements to position sound in the virtual studio similar to the way images are positioned in a 360 VR headset.

Musicians with disabilities debuted the customized instruments alongside the Hard Rian Soloist Ensemble in a showcase concert last November.