The theremin can be a difficult musical instrument to master because the slightest of motion across the whole body, or even nearby people can impact the sound. To create an intuitive theremin-like experience, we present Radarmin, a simple setup consisting of a Leap Motion sensor, the Magic Leap Mixed Reality (MR) headset, and a miniaturized Range-Doppler-based radar for detecting fine grain finger motion. The precise position of the fingers on the right hand is mapped to pitch control, while the left hand’s larger movements uses the Leap Motion for complementary instrumental input. To make playing easier we provide MR guidance cues of virtual images for hand placement and spatially mapped sound visualization. Participants experience Radarmin in a rhythm game, where they can learn how to play the musical instrument with the appropriate visual and audio feedback in MR. Radarmin’s main goal is twofold; to use a miniaturised radar as an interactive input device in XR, and to develop a MR tool to teach how to play the theremin.

Paper link here.