| |
Records |
Links |
|
Author |
Wenjuan Gong; Yue Zhang; Wei Wang; Peng Cheng; Jordi Gonzalez |
|
| |
Title |
Meta-MMFNet: Meta-learning-based Multi-model Fusion Network for Micro-expression Recognition |
Type |
Journal Article |
| |
Year |
2023 |
Publication |
ACM Transactions on Multimedia Computing, Communications, and Applications |
Abbreviated Journal |
TMCCA |
|
| |
Volume |
20 |
Issue |
2 |
Pages |
1–20 |
|
| |
Keywords |
|
|
| |
Abstract |
Despite its wide applications in criminal investigations and clinical communications with patients suffering from autism, automatic micro-expression recognition remains a challenging problem because of the lack of training data and imbalanced classes problems. In this study, we proposed a meta-learning-based multi-model fusion network (Meta-MMFNet) to solve the existing problems. The proposed method is based on the metric-based meta-learning pipeline, which is specifically designed for few-shot learning and is suitable for model-level fusion. The frame difference and optical flow features were fused, deep features were extracted from the fused feature, and finally in the meta-learning-based framework, weighted sum model fusion method was applied for micro-expression classification. Meta-MMFNet achieved better results than state-of-the-art methods on four datasets. The code is available at https://github.com/wenjgong/meta-fusion-based-method. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher  |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
ISE |
Approved |
no |
|
| |
Call Number |
Admin @ si @ GZW2023 |
Serial |
3862 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Wenwen Fu; Zhihong An; Wendong Huang; Haoran Sun; Wenjuan Gong; Jordi Gonzalez |
|
| |
Title |
A Spatio-Temporal Spotting Network with Sliding Windows for Micro-Expression Detection |
Type |
Journal Article |
| |
Year |
2023 |
Publication |
Electronics |
Abbreviated Journal |
ELEC |
|
| |
Volume |
12 |
Issue |
18 |
Pages |
3947 |
|
| |
Keywords |
micro-expression spotting; sliding window; key frame extraction |
|
| |
Abstract |
Micro-expressions reveal underlying emotions and are widely applied in political psychology, lie detection, law enforcement and medical care. Micro-expression spotting aims to detect the temporal locations of facial expressions from video sequences and is a crucial task in micro-expression recognition. In this study, the problem of micro-expression spotting is formulated as micro-expression classification per frame. We propose an effective spotting model with sliding windows called the spatio-temporal spotting network. The method involves a sliding window detection mechanism, combines the spatial features from the local key frames and the global temporal features and performs micro-expression spotting. The experiments are conducted on the CAS(ME)2 database and the SAMM Long Videos database, and the results demonstrate that the proposed method outperforms the state-of-the-art method by 30.58% for the CAS(ME)2 and 23.98% for the SAMM Long Videos according to overall F-scores. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
ISE |
Approved |
no |
|
| |
Call Number |
Admin @ si @ FAH2023 |
Serial |
3864 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Diego Velazquez; Pau Rodriguez; Alexandre Lacoste; Issam H. Laradji; Xavier Roca; Jordi Gonzalez |
|
| |
Title |
Evaluating Counterfactual Explainers |
Type |
Journal |
| |
Year |
2023 |
Publication |
Transactions on Machine Learning Research |
Abbreviated Journal |
TMLR |
|
| |
Volume |
|
Issue |
|
Pages |
|
|
| |
Keywords |
Explainability; Counterfactuals; XAI |
|
| |
Abstract |
Explainability methods have been widely used to provide insight into the decisions made by statistical models, thus facilitating their adoption in various domains within the industry. Counterfactual explanation methods aim to improve our understanding of a model by perturbing samples in a way that would alter its response in an unexpected manner. This information is helpful for users and for machine learning practitioners to understand and improve their models. Given the value provided by counterfactual explanations, there is a growing interest in the research community to investigate and propose new methods. However, we identify two issues that could hinder the progress in this field. (1) Existing metrics do not accurately reflect the value of an explainability method for the users. (2) Comparisons between methods are usually performed with datasets like CelebA, where images are annotated with attributes that do not fully describe them and with subjective attributes such as ``Attractive''. In this work, we address these problems by proposing an evaluation method with a principled metric to evaluate and compare different counterfactual explanation methods. The evaluation method is based on a synthetic dataset where images are fully described by their annotated attributes. As a result, we are able to perform a fair comparison of multiple explainability methods in the recent literature, obtaining insights about their performance. We make the code public for the benefit of the research community. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
|
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
ISE |
Approved |
no |
|
| |
Call Number |
Admin @ si @ VRL2023 |
Serial |
3891 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Carles Fernandez; Pau Baiget; Xavier Roca; Jordi Gonzalez |
|
| |
Title |
Determining the Best Suited Semantic Events for Cognitive Surveillance |
Type |
Journal Article |
| |
Year |
2011 |
Publication |
Expert Systems with Applications |
Abbreviated Journal |
EXSY |
|
| |
Volume |
38 |
Issue |
4 |
Pages |
4068–4079 |
|
| |
Keywords |
Cognitive surveillance; Event modeling; Content-based video retrieval; Ontologies; Advanced user interfaces |
|
| |
Abstract |
State-of-the-art systems on cognitive surveillance identify and describe complex events in selected domains, thus providing end-users with tools to easily access the contents of massive video footage. Nevertheless, as the complexity of events increases in semantics and the types of indoor/outdoor scenarios diversify, it becomes difficult to assess which events describe better the scene, and how to model them at a pixel level to fulfill natural language requests. We present an ontology-based methodology that guides the identification, step-by-step modeling, and generalization of the most relevant events to a specific domain. Our approach considers three steps: (1) end-users provide textual evidence from surveilled video sequences; (2) transcriptions are analyzed top-down to build the knowledge bases for event description; and (3) the obtained models are used to generalize event detection to different image sequences from the surveillance domain. This framework produces user-oriented knowledge that improves on existing advanced interfaces for video indexing and retrieval, by determining the best suited events for video understanding according to end-users. We have conducted experiments with outdoor and indoor scenes showing thefts, chases, and vandalism, demonstrating the feasibility and generalization of this proposal. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
Elsevier |
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
ISE |
Approved |
no |
|
| |
Call Number |
Admin @ si @ FBR2011a |
Serial |
1722 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Carles Fernandez; Pau Baiget; Xavier Roca; Jordi Gonzalez |
|
| |
Title |
Augmenting Video Surveillance Footage with Virtual Agents for Incremental Event Evaluation |
Type |
Journal Article |
| |
Year |
2011 |
Publication |
Pattern Recognition Letters |
Abbreviated Journal |
PRL |
|
| |
Volume |
32 |
Issue |
6 |
Pages |
878–889 |
|
| |
Keywords |
|
|
| |
Abstract |
The fields of segmentation, tracking and behavior analysis demand for challenging video resources to test, in a scalable manner, complex scenarios like crowded environments or scenes with high semantics. Nevertheless, existing public databases cannot scale the presence of appearing agents, which would be useful to study long-term occlusions and crowds. Moreover, creating these resources is expensive and often too particularized to specific needs. We propose an augmented reality framework to increase the complexity of image sequences in terms of occlusions and crowds, in a scalable and controllable manner. Existing datasets can be increased with augmented sequences containing virtual agents. Such sequences are automatically annotated, thus facilitating evaluation in terms of segmentation, tracking, and behavior recognition. In order to easily specify the desired contents, we propose a natural language interface to convert input sentences into virtual agent behaviors. Experimental tests and validation in indoor, street, and soccer environments are provided to show the feasibility of the proposed approach in terms of robustness, scalability, and semantics. |
|
| |
Address |
|
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
Elsevier |
Place of Publication |
|
Editor |
|
|
| |
Language |
|
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
|
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
ISE |
Approved |
no |
|
| |
Call Number |
Admin @ si @ FBR2011b |
Serial |
1723 |
|
| Permanent link to this record |
