Evaluation of machine learning performance on source separated passive acoustic recordings

The United States National Park Service often deploys passive acoustic monitoring devices whose data is used to characterize the ecological health of the site and quantify the level of anthropogenic sources present. The analysis of the data is then used to inform conservation management policy that protects the natural soundscape of the National Parks. Traditional methods for analyzing the acoustic recordings involve human listeners annotating a subset of this data and labeling the acoustic sources present. Manual data processing is labor intensive and relies on human perception that varies between individuals, these limitations ultimately reduce the amount of data that can be analyzed. By augmenting the manual annotation of acoustic data with machine learning (ML) this research aims to increase the quantity of data that can be annotated while achieving higher annotation confidence for acoustic sources. Previous research has shown that separating the transient acoustic sources from the background acoustic environment reduces data complexity allowing for improved characterization of the sources. This paper will evaluate the performance of machine learning models using source-separated data to understand the effects it has on data annotation accuracy, hyperparameter complexity, and ML model training time.