Video visualization for moving sound sources based on binocular vision and short-time beamforming

Visualization of moving sound sources is very important for noise source identification of moving objects. In this paper, we introduce binocular vision technology to sound sources visualization. Binocular technology can not only obtain the visualization, but also is able to achieve the spatial measurement of moving objects and determine the relative position of moving sound sources and the microphone array automatically, so that the sound reconstruction plane can be closely attached to the moving object. Together with the short-time beamforming method, the sound map is accurately registered on to the video images, generating the sound video close to the moving sound sources. Then, by video slow motion, noises originated from moving sound sources in a real world are shown in a sound movie. To verify this method, an experiment for a non-stationary speaker in the anechoic room is conducted, where the speaker sound source is visualized and located accurately. Further verification experiments for an outdoor moving vehicle carrying speakers have also been conducted. Sound sources of the moving speakers are successfully visualized and located. The generated sound movie shows the origin of noises under real world conditions making noise source investigation easier and more accurate. These experimental results suggest that our visualization approach has potential for noise identification and control.