%0 Conference Proceedings
%T Modelling and Analyzing Multimodal Dyadic Interactions Using Social Networks
%A Sergio Escalera
%A Petia Radeva
%A Jordi Vitria
%A Xavier Baro
%A Bogdan Raducanu
%B 12th International Conference on Multimodal Interfaces and 7th Workshop on Machine Learning for Multimodal Interaction.
%D 2010
%F Sergio Escalera2010
%O OR;MILAB;HUPBA;MV
%O exported from refbase (http://refbase.cvc.uab.es/show.php?record=1427), last updated on Thu, 18 Jan 2018 12:02:53 +0100
%X Social network analysis became a common technique used to model and quantify the properties of social interactions. In this paper, we propose an integrated framework to explore the characteristics of a social network extracted frommultimodal dyadic interactions. First, speech detection is performed through an audio/visual fusion scheme based on stacked sequential learning. In the audio domain, speech is detected through clusterization of audio features. Clustersare modelled by means of an One-state Hidden Markov Model containing a diagonal covariance Gaussian Mixture Model. In the visual domain, speech detection is performed through differential-based feature extraction from the segmentedmouth region, and a dynamic programming matching procedure. Second, in order to model the dyadic interactions, we employed the Influence Model whose statesencode the previous integrated audio/visual data. Third, the social network is extracted based on the estimated influences. For our study, we used a set of videos belonging to New York Times’ Blogging Heads opinion blog. The resultsare reported both in terms of accuracy of the audio/visual data fusion and centrality measures used to characterize the social network.
%K Social interaction
%K Multimodal fusion
%K Influence model
%K Social network analysis
%U http://delivery.acm.org/10.1145/1900000/1891967/a52-escalera.pdf?ip=158.109.9.24&acc=ACTIVE%20SERVICE&CFID=105856509&CFTOKEN=16186426&__acm__=1338291304_bf0fcfab7182a4cb79822c4dccd3aa49
%U http://dx.doi.org/10.1145/1891903.1891967