Multimodal Hong Kong: a multisensory study of environmental perception of places through sight, sound, and smell

Despite the importance of soundscape and smellscape in intangible cultural heritage, little is known about multisensory interactions. To explore the impact of multiple senses on environmental perception, the present study focused on six stalls at Shek Kip Mei wet market in Hong Kong. In the first part of the study, an online survey collected perceptual ratings of whole, auditory, and visual environmental quality, as well as free-form responses of imaginations of smell. Participants (N = 36) were presented with 12 audiovisual captures where audio was either matched with video or mismatched. ANOVA was used to examine the influence of three independent factors - scenes, modes, and a priori valence groups - on the ratings. The imagined smells were analysed through frequency counts of keywords. The results demonstrate participants' capacity to notice changes in audio modes and to perceive the environment through the recognition of scene context and multisensory interactions. In the second part of the study, not discussed in the present paper, we will extend this experimental method to include smells in a laboratory setting. We discuss the importance of understanding multisensory interactions between sight, sound, and smell for environmental perceptions and Hong Kong's intangible cultural heritage and urban studies.