Deconstructing a musical illusion: causality and audio-visual integration

Abstract: My colleagues and I have documented a musical illusion in which percussionists are able to manipulate an audience’s perception of note duration independent of that note’s acoustic properties. Curiously, this illusion is at odds with previous research and current theoretical models of auditory-visual integration. Recent work has shown the perception of a causal, cross-modal link plays a key role in explaining this exception. Impact motions integrate with sounds caused by impact events (such as the sound of a percussion instrument being struck), but not sounds that are produced by different types of events (such as air moving over a reed in the case of a clarinet). Similarly, these sounds integrate with gestures that lead (but not lag) their onset, reflecting the slower speed of sound relative to light. This talk will summarize some of my published work on this illusion, and discuss new unpublished experiments exploring the role of amplitude envelope (the shape of a sound over time) in audio-visual integration. For a demonstration of the musical illusion, please see a brief (2 min) TV segment now posted online at http://www.maplelab.net/press-room

Bio: Michael Schutz is Assistant Professor of Music Cognition/Percussion at McMaster University, where he is a core member of the McMaster Institute for Music and the Mind.  Michael is the founding director of the externally funded MAPLE Lab, researching Music, Perception, Acoustics and LEarning (www.maplelab.net). His research interests include the role of visual information in music, communicating emotion through music, rhythm/timing, and the perception of timbre.  His work on these topics appears in journals such as Journal of Experimental Psychology; Perception; Percussive Notes; Attention, Perception, & Psychophysics; and Empirical Musicology Review.