TactualPlot: Spatializing Data as Sound using Sensory Substitution for Touchscreen Accessibility
Pramod Chundury, Yasmin Reyazuddin, J. Bern Jordan, Jonathan Lazar, Niklas Elmqvist
DOI: 10.1109/TVCG.2023.3326937
Room: 105
2023-10-25T23:00:00ZGMT-0600Change your timezone on the schedule page
2023-10-25T23:00:00Z
Fast forward
Full Video
Keywords
Accessibility, sonification, multimodal interaction, crossmodal interaction, visualization
Abstract
Tactile graphics are one of the best ways for a blind person to perceive a chart using touch, but their fabrication is often costly, time-consuming, and does not lend itself to dynamic exploration. Refreshable haptic displays tend to be expensive and thus unavailable to most blind individuals. We propose TactualPlot, an approach to sensory substitution where touch interaction yields auditory (sonified) feedback. The technique relies on embodied cognition for spatial awareness—i.e., individuals can perceive 2D touch locations of their fingers with reference to other 2D locations such as the relative locations of other fingers or chart characteristics that are visualized on touchscreens. Combining touch and sound in this way yields a scalable data exploration method for scatterplots where the data density under the user’s fingertips is sampled. The sample regions can optionally be scaled based on how quickly the user moves their hand. Our development of TactualPlot was informed by formative design sessions with a blind collaborator, whose practice while using tactile scatterplots caused us to expand the technique for multiple fingers. We present results from an evaluation comparing our TactualPlot interaction technique to tactile graphics printed on swell touch paper.