In this new paper,we examine the nature of sound representations in intermediate layers of convolutional DNNs by means of in silico experiments involving a new sound data set of material and actions. Furthermore, by means of a new methodology based on invertible neural networks, we show that there is a causal relationship between these internal representations and the semantic model output. Deciphering the Transformation of Sounds into Meaning: Insights from Disentangling Intermediate Representations in Sound-to-Event DNNs. Tim Dick, Alexia Briassouli, Enrique Hortal Quesada and Elia Formisano (2024) Available at SSRN: https://ssrn.com/abstract=4979651 or http://dx.doi.org/10.2139/ssrn.4979651.
In this new article in Scientific Report (Open Access), we have learned that deep neural networks (DNN) mapping sounds to distributed language semantics approximate human listeners’ behavior better than standard DNNs with categorical output: Esposito, M., Valente, G., Plasencia-Calaña, Y., Michel Dumontier, Bruno L. Giordano, Elia Formisano. Bridging auditory perception and natural language processing with semantically informed deep neural networks. Scientific Reports 14, 20994 (2024). https://doi.org/10.1038/s41598-024-71693-9