|
|
German Barquero, Sergio Escalera, & Cristina Palmero. (2024). Seamless Human Motion Composition with Blended Positional Encodings.
Abstract: Conditional human motion generation is an important topic with many applications in virtual reality, gaming, and robotics. While prior works have focused on generating motion guided by text, music, or scenes, these typically result in isolated motions confined to short durations. Instead, we address the generation of long, continuous sequences guided by a series of varying textual descriptions. In this context, we introduce FlowMDM, the first diffusion-based model that generates seamless Human Motion Compositions (HMC) without any postprocessing or redundant denoising steps. For this, we introduce the Blended Positional Encodings, a technique that leverages both absolute and relative positional encodings in the denoising chain. More specifically, global motion coherence is recovered at the absolute stage, whereas smooth and realistic transitions are built at the relative stage. As a result, we achieve state-of-the-art results in terms of accuracy, realism, and smoothness on the Babel and HumanML3D datasets. FlowMDM excels when trained with only a single description per motion sequence thanks to its Pose-Centric Cross-ATtention, which makes it robust against varying text descriptions at inference time. Finally, to address the limitations of existing HMC metrics, we propose two new metrics: the Peak Jerk and the Area Under the Jerk, to detect abrupt transitions.
|
|
|
|
Guillem Cucurull, Pau Rodriguez, Vacit Oguz Yazici, Josep M. Gonfaus, Xavier Roca, & Jordi Gonzalez. (2018). Deep Inference of Personality Traits by Integrating Image and Word Use in Social Networks.
Abstract: arXiv:1802.06757
Social media, as a major platform for communication and information exchange, is a rich repository of the opinions and sentiments of 2.3 billion users about a vast spectrum of topics. To sense the whys of certain social user’s demands and cultural-driven interests, however, the knowledge embedded in the 1.8 billion pictures which are uploaded daily in public profiles has just started to be exploited since this process has been typically been text-based. Following this trend on visual-based social analysis, we present a novel methodology based on Deep Learning to build a combined image-and-text based personality trait model, trained with images posted together with words found highly correlated to specific personality traits. So the key contribution here is to explore whether OCEAN personality trait modeling can be addressed based on images, here called MindPics, appearing with certain tags with psychological insights. We found that there is a correlation between those posted images and their accompanying texts, which can be successfully modeled using deep neural networks for personality estimation. The experimental results are consistent with previous cyber-psychology results based on texts or images.
In addition, classification results on some traits show that some patterns emerge in the set of images corresponding to a specific text, in essence to those representing an abstract concept. These results open new avenues of research for further refining the proposed personality model under the supervision of psychology experts.
|
|
|
|
Hannes Mueller, Andre Groger, Jonathan Hersh, Andrea Matranga, & Joan Serrat. (2020). Monitoring War Destruction from Space: A Machine Learning Approach.
Abstract: Existing data on building destruction in conflict zones rely on eyewitness reports or manual detection, which makes it generally scarce, incomplete and potentially biased. This lack of reliable data imposes severe limitations for media reporting, humanitarian relief efforts, human rights monitoring, reconstruction initiatives, and academic studies of violent conflict. This article introduces an automated method of measuring destruction in high-resolution satellite images using deep learning techniques combined with data augmentation to expand training samples. We apply this method to the Syrian civil war and reconstruct the evolution of damage in major cities across the country. The approach allows generating destruction data with unprecedented scope, resolution, and frequency – only limited by the available satellite imagery – which can alleviate data limitations decisively.
|
|
|
|
Hao Wu, Alejandro Ariza-Casabona, Bartłomiej Twardowski, & Tri Kurniawan Wijaya. (2023). MM-GEF: Multi-modal representation meet collaborative filtering.
Abstract: In modern e-commerce, item content features in various modalities offer accurate yet comprehensive information to recommender systems. The majority of previous work either focuses on learning effective item representation during modelling user-item interactions, or exploring item-item relationships by analysing multi-modal features. Those methods, however, fail to incorporate the collaborative item-user-item relationships into the multi-modal feature-based item structure. In this work, we propose a graph-based item structure enhancement method MM-GEF: Multi-Modal recommendation with Graph Early-Fusion, which effectively combines the latent item structure underlying multi-modal contents with the collaborative signals. Instead of processing the content feature in different modalities separately, we show that the early-fusion of multi-modal features provides significant improvement. MM-GEF learns refined item representations by injecting structural information obtained from both multi-modal and collaborative signals. Through extensive experiments on four publicly available datasets, we demonstrate systematical improvements of our method over state-of-the-art multi-modal recommendation methods.
|
|
|
|
Hugo Jair Escalante, Heysem Kaya, Albert Ali Salah, Sergio Escalera, Yagmur Gucluturk, Umut Guclu, et al. (2018). Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos.
Abstract: Explainability and interpretability are two critical aspects of decision support systems. Within computer vision, they are critical in certain tasks related to human behavior analysis such as in health care applications. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of computer vision with an emphasis on looking at people tasks. Specifically, we review and study those mechanisms in the context of first impressions analysis. To the best of our knowledge, this is the first effort in this direction. Additionally, we describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, the evaluation protocol, and summarize the results of the challenge. Finally, derived from our study, we outline research opportunities that we foresee will be decisive in the near future for the development of the explainable computer vision field.
|
|
|
|
Hugo Prol, Vincent Dumoulin, & Luis Herranz. (2018). Cross-Modulation Networks for Few-Shot Learning.
Abstract: A family of recent successful approaches to few-shot learning relies on learning an embedding space in which predictions are made by computing similarities between examples. This corresponds to combining information between support and query examples at a very late stage of the prediction pipeline. Inspired by this observation, we hypothesize that there may be benefits to combining the information at various levels of abstraction along the pipeline. We present an architecture called Cross-Modulation Networks which allows support and query examples to interact throughout the feature extraction process via a feature-wise modulation mechanism. We adapt the Matching Networks architecture to take advantage of these interactions and show encouraging initial results on miniImageNet in the 5-way, 1-shot setting, where we close the gap with state-of-the-art.
|
|
|
|
Ignasi Rius, Dani Rowe, Jordi Gonzalez, & Xavier Roca. (2005). 3D Action Modeling and Reconstruction for 2D Human Body Tracking.
|
|
|
|
Ignasi Rius, Javier Varona, Jordi Gonzalez, & Juan J. Villanueva. (2006). Action Spaces for Efficient Bayesian Tracking of Human Motion.
|
|
|
|
J. Filipe, Juan Andrade, & J.L. Ferrier. (2005). FAF 2005.
|
|
|
|
J. Mauri, E Fernandez-Nofrerias, E. Esplugas, A. Cequier, David Rotger, Ricardo Toledo, et al. (2000). Ecografia Intracoronaria: Navegacion Informatica por el cubo de datos de las imagenes..
|
|
|
|
J. Mauri, E Fernandez-Nofrerias, Petia Radeva, & V. Valle. (2000). Ecografia intracoronaria, una ajuda o un mestre en lintervencionisme coronari?".
|
|
|
|
J. Mauri, E. Esplugas, B. Garcia del Blanco, E Fernandez-Nofrerias, A. Cequier, J.A. Gomez-Hospital, et al. (2000). 3-D Stent and Vessel Reconstruction from IVUS: a Physics-Based Approach.
|
|
|
|
J. Suri, S. Singh, S. Laxminarayan, R. Cesar, H. Jelinek, Petia Radeva, et al. (2003). A Note on Future Research in Vascular and Plaque Segmentation.
|
|
|
|
J. Weickert, Bart M. Ter Haar Romeny, Antonio Lopez, & W. Van Enk. (1997). Orientation Analysis by Coherence-Enhancing Diffusion..
|
|
|
|
J.L.Bruguera, R.Casado, M.Martinez, I.Corral, Enric Marti, & L.A.Branda. (2009). El apoyo institucional como elemento favorecedor de la coordinación docente: experiencias en diferentes universidades.
|
|
