Home | [211–220] << 221 222 223 224 225 226 227 228 >> |
![]() |
Records | |||||
---|---|---|---|---|---|
Author | Galadrielle Humblot-Renaux; Sergio Escalera; Thomas B. Moeslund | ||||
Title | Beyond AUROC & co. for evaluating out-of-distribution detection performance | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 3880-3889 | ||
Keywords | |||||
Abstract | While there has been a growing research interest in developing out-of-distribution (OOD) detection methods, there has been comparably little discussion around how these methods should be evaluated. Given their relevance for safe(r) AI, it is important to examine whether the basis for comparing OOD detection methods is consistent with practical needs. In this work, we take a closer look at the go-to metrics for evaluating OOD detection, and question the approach of exclusively reducing OOD detection to a binary classification task with little consideration for the detection threshold. We illustrate the limitations of current metrics (AUROC & its friends) and propose a new metric – Area Under the Threshold Curve (AUTC), which explicitly penalizes poor separation between ID and OOD samples. Scripts and data are available at https://github.com/glhr/beyond-auroc | ||||
Address | Vancouver; Canada; June 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | CVPRW | ||
Notes | HUPBA | Approved | no | ||
Call Number | Admin @ si @ HEM2023 | Serial | 3918 | ||
Permanent link to this record | |||||
Author | Dong Wang; Jia Guo; Qiqi Shao; Haochi He; Zhian Chen; Chuanbao Xiao; Ajian Liu; Sergio Escalera; Hugo Jair Escalante; Zhen Lei; Jun Wan; Jiankang Deng | ||||
Title | Wild Face Anti-Spoofing Challenge 2023: Benchmark and Results | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 6379-6390 | ||
Keywords | |||||
Abstract | Face anti-spoofing (FAS) is an essential mechanism for safeguarding the integrity of automated face recognition systems. Despite substantial advancements, the generalization of existing approaches to real-world applications remains challenging. This limitation can be attributed to the scarcity and lack of diversity in publicly available FAS datasets, which often leads to overfitting during training or saturation during testing. In terms of quantity, the number of spoof subjects is a critical determinant. Most datasets comprise fewer than 2,000 subjects. With regard to diversity, the majority of datasets consist of spoof samples collected in controlled environments using repetitive, mechanical processes. This data collection methodology results in homogenized samples and a dearth of scenario diversity. To address these shortcomings, we introduce the Wild Face Anti-Spoofing (WFAS) dataset, a large-scale, diverse FAS dataset collected in unconstrained settings. Our dataset encompasses 853,729 images of 321,751 spoof subjects and 529,571 images of 148,169 live subjects, representing a substantial increase in quantity. Moreover, our dataset incorporates spoof data obtained from the internet, spanning a wide array of scenarios and various commercial sensors, including 17 presentation attacks (PAs) that encompass both 2D and 3D forms. This novel data collection strategy markedly enhances FAS data diversity. Leveraging the WFAS dataset and Protocol 1 (Known-Type), we host the Wild Face Anti-Spoofing Challenge at the CVPR2023 workshop. Additionally, we meticulously evaluate representative methods using Protocol 1 and Protocol 2 (Unknown-Type). Through an in-depth examination of the challenge outcomes and benchmark baselines, we provide insightful analyses and propose potential avenues for future research. The dataset is released under Insightface 1 . | ||||
Address | Vancouver; Canada; June 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | CVPRW | ||
Notes | HUPBA | Approved | no | ||
Call Number | Admin @ si @ WGS2023 | Serial | 3919 | ||
Permanent link to this record | |||||
Author | Senmao Li; Joost Van de Weijer; Yaxing Wang; Fahad Shahbaz Khan; Meiqin Liu; Jian Yang | ||||
Title | 3D-Aware Multi-Class Image-to-Image Translation with NeRFs | Type | Conference Article | ||
Year | 2023 | Publication | 36th IEEE Conference on Computer Vision and Pattern Recognition | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 12652-12662 | ||
Keywords | |||||
Abstract | Recent advances in 3D-aware generative models (3D-aware GANs) combined with Neural Radiance Fields (NeRF) have achieved impressive results. However no prior works investigate 3D-aware GANs for 3D consistent multiclass image-to-image (3D-aware 121) translation. Naively using 2D-121 translation methods suffers from unrealistic shape/identity change. To perform 3D-aware multiclass 121 translation, we decouple this learning process into a multiclass 3D-aware GAN step and a 3D-aware 121 translation step. In the first step, we propose two novel techniques: a new conditional architecture and an effective training strategy. In the second step, based on the well-trained multiclass 3D-aware GAN architecture, that preserves view-consistency, we construct a 3D-aware 121 translation system. To further reduce the view-consistency problems, we propose several new techniques, including a U-net-like adaptor network design, a hierarchical representation constrain and a relative regularization loss. In exten-sive experiments on two datasets, quantitative and qualitative results demonstrate that we successfully perform 3D-aware 121 translation with multi-view consistency. Code is available in 3DI2I. | ||||
Address | Vancouver; Canada; June 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | CVPR | ||
Notes | LAMP | Approved | no | ||
Call Number | Admin @ si @ LWW2023b | Serial | 3920 | ||
Permanent link to this record | |||||
Author | Hugo Bertiche; Niloy J Mitra; Kuldeep Kulkarni; Chun Hao Paul Huang; Tuanfeng Y Wang; Meysam Madadi; Sergio Escalera; Duygu Ceylan | ||||
Title | Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Images | Type | Conference Article | ||
Year | 2023 | Publication | 36th IEEE Conference on Computer Vision and Pattern Recognition | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 459-468 | ||
Keywords | |||||
Abstract | Cinemagraphs are short looping videos created by adding subtle motions to a static image. This kind of media is popular and engaging. However, automatic generation of cinemagraphs is an underexplored area and current solutions require tedious low-level manual authoring by artists. In this paper, we present an automatic method that allows generating human cinemagraphs from single RGB images. We investigate the problem in the context of dressed humans under the wind. At the core of our method is a novel cyclic neural network that produces looping cinemagraphs for the target loop duration. To circumvent the problem of collecting real data, we demonstrate that it is possible, by working in the image normal space, to learn garment motion dynamics on synthetic data and generalize to real data. We evaluate our method on both synthetic and real data and demonstrate that it is possible to create compelling and plausible cinemagraphs from single RGB images. | ||||
Address | Vancouver; Canada; June 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | CVPR | ||
Notes | HUPBA | Approved | no | ||
Call Number | Admin @ si @ BMK2023 | Serial | 3921 | ||
Permanent link to this record | |||||
Author | Albin Soutif; Antonio Carta; Joost Van de Weijer | ||||
Title | Improving Online Continual Learning Performance and Stability with Temporal Ensembles | Type | Conference Article | ||
Year | 2023 | Publication | 2nd Conference on Lifelong Learning Agents | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | |||
Keywords | |||||
Abstract | Neural networks are very effective when trained on large datasets for a large number of iterations. However, when they are trained on non-stationary streams of data and in an online fashion, their performance is reduced (1) by the online setup, which limits the availability of data, (2) due to catastrophic forgetting because of the non-stationary nature of the data. Furthermore, several recent works (Caccia et al., 2022; Lange et al., 2023) arXiv:2205.13452 showed that replay methods used in continual learning suffer from the stability gap, encountered when evaluating the model continually (rather than only on task boundaries). In this article, we study the effect of model ensembling as a way to improve performance and stability in online continual learning. We notice that naively ensembling models coming from a variety of training tasks increases the performance in online continual learning considerably. Starting from this observation, and drawing inspirations from semi-supervised learning ensembling methods, we use a lightweight temporal ensemble that computes the exponential moving average of the weights (EMA) at test time, and show that it can drastically increase the performance and stability when used in combination with several methods from the literature. | ||||
Address | Montreal; Canada; August 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | COLLAS | ||
Notes | LAMP | Approved | no | ||
Call Number | Admin @ si @ SCW2023 | Serial | 3922 | ||
Permanent link to this record | |||||
Author | Cristina Palmero; Oleg V Komogortsev; Sergio Escalera; Sachin S Talathi | ||||
Title | Multi-Rate Sensor Fusion for Unconstrained Near-Eye Gaze Estimation | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the 2023 Symposium on Eye Tracking Research and Applications | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 1-8 | ||
Keywords | |||||
Abstract | The power requirements of video-oculography systems can be prohibitive for high-speed operation on portable devices. Recently, low-power alternatives such as photosensors have been evaluated, providing gaze estimates at high frequency with a trade-off in accuracy and robustness. Potentially, an approach combining slow/high-fidelity and fast/low-fidelity sensors should be able to exploit their complementarity to track fast eye motion accurately and robustly. To foster research on this topic, we introduce OpenSFEDS, a near-eye gaze estimation dataset containing approximately 2M synthetic camera-photosensor image pairs sampled at 500 Hz under varied appearance and camera position. We also formulate the task of sensor fusion for gaze estimation, proposing a deep learning framework consisting in appearance-based encoding and temporal eye-state dynamics. We evaluate several single- and multi-rate fusion baselines on OpenSFEDS, achieving 8.7% error decrease when tracking fast eye movements with a multi-rate approach vs. a gaze forecasting approach operating with a low-speed sensor alone. | ||||
Address | Tubingen; Germany; May 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | ETRA | ||
Notes | HUPBA | Approved | no | ||
Call Number | Admin @ si @ PKE2023 | Serial | 3923 | ||
Permanent link to this record | |||||
Author | Guillermo Torres; Jan Rodríguez Dueñas; Sonia Baeza; Antoni Rosell; Carles Sanchez; Debora Gil | ||||
Title | Prediction of Malignancy in Lung Cancer using several strategies for the fusion of Multi-Channel Pyradiomics Images | Type | Conference Article | ||
Year | 2023 | Publication | 7th Workshop on Digital Image Processing for Medical and Automotive Industry in the framework of SYNASC 2023 | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | |||
Keywords | |||||
Abstract | This study shows the generation process and the subsequent study of the representation space obtained by extracting GLCM texture features from computer-aided tomography (CT) scans of pulmonary nodules (PN). For this, data from 92 patients from the Germans Trias i Pujol University Hospital were used. The workflow focuses on feature extraction using Pyradiomics and the VGG16 Convolutional Neural Network (CNN). The aim of the study is to assess whether the data obtained have a positive impact on the diagnosis of lung cancer (LC). To design a machine learning (ML) model training method that allows generalization, we train SVM and neural network (NN) models, evaluating diagnosis performance using metrics defined at slice and nodule level. | ||||
Address | |||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | DIPMAI | ||
Notes | IAM | Approved | no | ||
Call Number | Admin @ si @ TRB2023 | Serial | 3926 | ||
Permanent link to this record | |||||
Author | Patricia Suarez; Angel Sappa | ||||
Title | Toward a Thermal Image-Like Representation | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 133-140 | ||
Keywords | |||||
Abstract | This paper proposes a novel model to obtain thermal image-like representations to be used as an input in any thermal image compressive sensing approach (e.g., thermal image: filtering, enhancing, super-resolution). Thermal images offer interesting information about the objects in the scene, in addition to their temperature. Unfortunately, in most of the cases thermal cameras acquire low resolution/quality images. Hence, in order to improve these images, there are several state-of-the-art approaches that exploit complementary information from a low-cost channel (visible image) to increase the image quality of an expensive channel (infrared image). In these SOTA approaches visible images are fused at different levels without paying attention the images acquire information at different bands of the spectral. In this paper a novel approach is proposed to generate thermal image-like representations from a low cost visible images, by means of a contrastive cycled GAN network. Obtained representations (synthetic thermal image) can be later on used to improve the low quality thermal image of the same scene. Experimental results on different datasets are presented. | ||||
Address | Lisboa; Portugal; February 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | VISIGRAPP | ||
Notes | MSIAU | Approved | no | ||
Call Number | Admin @ si @ SuS2023b | Serial | 3927 | ||
Permanent link to this record | |||||
Author | David Dueñas; Mostafa Kamal; Petia Radeva | ||||
Title | Efficient Deep Learning Ensemble for Skin Lesion Classification | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 303-314 | ||
Keywords | |||||
Abstract | Vision Transformers (ViTs) are deep learning techniques that have been gaining in popularity in recent years.
In this work, we study the performance of ViTs and Convolutional Neural Networks (CNNs) on skin lesions classification tasks, specifically melanoma diagnosis. We show that regardless of the performance of both architectures, an ensemble of them can improve their generalization. We also present an adaptation to the Gram-OOD* method (detecting Out-of-distribution (OOD) using Gram matrices) for skin lesion images. Moreover, the integration of super-convergence was critical to success in building models with strict computing and training time constraints. We evaluated our ensemble of ViTs and CNNs, demonstrating that generalization is enhanced by placing first in the 2019 and third in the 2020 ISIC Challenge Live Leaderboards (available at https://challenge.isic-archive.com/leaderboards/live/). |
||||
Address | Lisboa; Portugal; February 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | VISIGRAPP | ||
Notes | MILAB | Approved | no | ||
Call Number | Admin @ si @ DKR2023 | Serial | 3928 | ||
Permanent link to this record | |||||
Author | Christian Keilstrup Ingwersen; Artur Xarles; Albert Clapes; Meysam Madadi; Janus Nortoft Jensen; Morten Rieger Hannemose; Anders Bjorholm Dahl; Sergio Escalera | ||||
Title | Video-based Skill Assessment for Golf: Estimating Golf Handicap | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the 6th International Workshop on Multimedia Content Analysis in Sports | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 31-39 | ||
Keywords | |||||
Abstract | Automated skill assessment in sports using video-based analysis holds great potential for revolutionizing coaching methodologies. This paper focuses on the problem of skill determination in golfers by leveraging deep learning models applied to a large database of video recordings of golf swings. We investigate different regression, ranking and classification based methods and compare to a simple baseline approach. The performance is evaluated using mean squared error (MSE) as well as computing the percentages of correctly ranked pairs based on the Kendall correlation. Our results demonstrate an improvement over the baseline, with a 35% lower mean squared error and 68% correctly ranked pairs. However, achieving fine-grained skill assessment remains challenging. This work contributes to the development of AI-driven coaching systems and advances the understanding of video-based skill determination in the context of golf. | ||||
Address | Otawa; Canada; October 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | MMSports | ||
Notes | HUPBA | Approved | no | ||
Call Number | Admin @ si @ KXC2023 | Serial | 3929 | ||
Permanent link to this record | |||||
Author | Yi Xiao; Felipe Codevilla; Diego Porres; Antonio Lopez | ||||
Title | Scaling Vision-Based End-to-End Autonomous Driving with Multi-View Attention Learning | Type | Conference Article | ||
Year | 2023 | Publication | International Conference on Intelligent Robots and Systems | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | |||
Keywords | |||||
Abstract | On end-to-end driving, human driving demonstrations are used to train perception-based driving models by imitation learning. This process is supervised on vehicle signals (e.g., steering angle, acceleration) but does not require extra costly supervision (human labeling of sensor data). As a representative of such vision-based end-to-end driving models, CILRS is commonly used as a baseline to compare with new driving models. So far, some latest models achieve better performance than CILRS by using expensive sensor suites and/or by using large amounts of human-labeled data for training. Given the difference in performance, one may think that it is not worth pursuing vision-based pure end-to-end driving. However, we argue that this approach still has great value and potential considering cost and maintenance. In this paper, we present CIL++, which improves on CILRS by both processing higher-resolution images using a human-inspired HFOV as an inductive bias and incorporating a proper attention mechanism. CIL++ achieves competitive performance compared to models which are more costly to develop. We propose to replace CILRS with CIL++ as a strong vision-based pure end-to-end driving baseline supervised by only vehicle signals and trained by conditional imitation learning. | ||||
Address | Detroit; USA; October 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | IROS | ||
Notes | ADAS | Approved | no | ||
Call Number | Admin @ si @ XCP2023 | Serial | 3930 | ||
Permanent link to this record | |||||
Author | Siyang Song; Micol Spitale; Cheng Luo; German Barquero; Cristina Palmero; Sergio Escalera; Michel Valstar; Tobias Baur; Fabien Ringeval; Elisabeth Andre; Hatice Gunes | ||||
Title | REACT2023: The First Multiple Appropriate Facial Reaction Generation Challenge | Type | Conference Article | ||
Year | 2023 | Publication | Proceedings of the 31st ACM International Conference on Multimedia | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | 9620–9624 | ||
Keywords | |||||
Abstract | The Multiple Appropriate Facial Reaction Generation Challenge (REACT2023) is the first competition event focused on evaluating multimedia processing and machine learning techniques for generating human-appropriate facial reactions in various dyadic interaction scenarios, with all participants competing strictly under the same conditions. The goal of the challenge is to provide the first benchmark test set for multi-modal information processing and to foster collaboration among the audio, visual, and audio-visual behaviour analysis and behaviour generation (a.k.a generative AI) communities, to compare the relative merits of the approaches to automatic appropriate facial reaction generation under different spontaneous dyadic interaction conditions. This paper presents: (i) the novelties, contributions and guidelines of the REACT2023 challenge; (ii) the dataset utilized in the challenge; and (iii) the performance of the baseline systems on the two proposed sub-challenges: Offline Multiple Appropriate Facial Reaction Generation and Online Multiple Appropriate Facial Reaction Generation, respectively. The challenge baseline code is publicly available at https://github.com/reactmultimodalchallenge/baseline_react2023. | ||||
Address | Otawa; Canada; October 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | MM | ||
Notes | HUPBA | Approved | no | ||
Call Number | Admin @ si @ SSL2023 | Serial | 3931 | ||
Permanent link to this record | |||||
Author | Dipam Goswami; Yuyang Liu ; Bartlomiej Twardowski; Joost Van de Weijer | ||||
Title | FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning | Type | Conference Article | ||
Year | 2023 | Publication | 37th Annual Conference on Neural Information Processing Systems | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | |||
Keywords | |||||
Abstract | Poster | ||||
Address | New Orleans; USA; December 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | NEURIPS | ||
Notes | LAMP | Approved | no | ||
Call Number | Admin @ si @ GLT2023 | Serial | 3934 | ||
Permanent link to this record | |||||
Author | Kai Wang; Fei Yang; Shiqi Yang; Muhammad Atif Butt; Joost Van de Weijer | ||||
Title | Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editing | Type | Conference Article | ||
Year | 2023 | Publication | 37th Annual Conference on Neural Information Processing Systems | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | |||
Keywords | |||||
Abstract | Poster | ||||
Address | New Orleans; USA; December 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | NEURIPS | ||
Notes | LAMP | Approved | no | ||
Call Number | Admin @ si @ WYY2023 | Serial | 3935 | ||
Permanent link to this record | |||||
Author | ChuanMing Fang; Kai Wang; Joost Van de Weijer | ||||
Title | IterInv: Iterative Inversion for Pixel-Level T2I Models | Type | Conference Article | ||
Year | 2023 | Publication | 37th Annual Conference on Neural Information Processing Systems | Abbreviated Journal | |
Volume ![]() |
Issue | Pages | |||
Keywords | |||||
Abstract | Large-scale text-to-image diffusion models have been a ground-breaking development in generating convincing images following an input text prompt. The goal of image editing research is to give users control over the generated images by modifying the text prompt. Current image editing techniques are relying on DDIM inversion as a common practice based on the Latent Diffusion Models (LDM). However, the large pretrained T2I models working on the latent space as LDM suffer from losing details due to the first compression stage with an autoencoder mechanism. Instead, another mainstream T2I pipeline working on the pixel level, such as Imagen and DeepFloyd-IF, avoids this problem. They are commonly composed of several stages, normally with a text-to-image stage followed by several super-resolution stages. In this case, the DDIM inversion is unable to find the initial noise to generate the original image given that the super-resolution diffusion models are not compatible with the DDIM technique. According to our experimental findings, iteratively concatenating the noisy image as the condition is the root of this problem. Based on this observation, we develop an iterative inversion (IterInv) technique for this stream of T2I models and verify IterInv with the open-source DeepFloyd-IF model. By combining our method IterInv with a popular image editing method, we prove the application prospects of IterInv. The code will be released at \url{this https URL}. | ||||
Address | New Orleans; USA; December 2023 | ||||
Corporate Author | Thesis | ||||
Publisher | Place of Publication | Editor | |||
Language | Summary Language | Original Title | |||
Series Editor | Series Title | Abbreviated Series Title | |||
Series Volume | Series Issue | Edition | |||
ISSN | ISBN | Medium | |||
Area | Expedition | Conference | NEURIPS | ||
Notes | LAMP | Approved | no | ||
Call Number | Admin @ si @ FWW2023 | Serial | 3936 | ||
Permanent link to this record |