TY  - CONF
AU  - Sergio Escalera
AU  - Petia Radeva
AU  - Jordi Vitria
AU  - Xavier Baro
AU  - Bogdan Raducanu
A2  - ICMI-MLI
PY  - 2010//
TI  - Modelling and Analyzing Multimodal Dyadic Interactions Using Social Networks
BT  - 12th International Conference on Multimodal Interfaces and 7th Workshop on Machine Learning for Multimodal Interaction.
KW  - Social interaction
KW  - Multimodal fusion
KW  - Influence model
KW  - Social network analysis
N2  - Social network analysis became a common technique used to model and quantify the properties of social interactions. In this paper, we propose an integrated framework to explore the characteristics of a social network extracted frommultimodal dyadic interactions. First, speech detection is performed through an audio/visual fusion scheme based on stacked sequential learning. In the audio domain, speech is detected through clusterization of audio features. Clustersare modelled by means of an One-state Hidden Markov Model containing a diagonal covariance Gaussian Mixture Model. In the visual domain, speech detection is performed through differential-based feature extraction from the segmentedmouth region, and a dynamic programming matching procedure. Second, in order to model the dyadic interactions, we employed the Influence Model whose statesencode the previous integrated audio/visual data. Third, the social network is extracted based on the estimated influences. For our study, we used a set of videos belonging to New York Times’ Blogging Heads opinion blog. The resultsare reported both in terms of accuracy of the audio/visual data fusion and centrality measures used to characterize the social network.
UR  - http://delivery.acm.org/10.1145/1900000/1891967/a52-escalera.pdf?ip=158.109.9.24&acc=ACTIVE%20SERVICE&CFID=105856509&CFTOKEN=16186426&__acm__=1338291304_bf0fcfab7182a4cb79822c4dccd3aa49
UR  - http://dx.doi.org/10.1145/1891903.1891967
N1  - OR;MILAB;HUPBA;MV
ID  - Sergio Escalera2010
ER  -