| Home | << 1 2 3 4 5 6 7 8 9 10 >> [11–14] |
|
| Records | |||||
|---|---|---|---|---|---|
| Author | Debora Gil; Jordi Gonzalez; Gemma Sanchez (eds) | ||||
| Title | Computer Vision: Advances in Research and Development | Type | Book Whole | ||
| Year | 2007 | Publication | Proceedings of the 2nd CVC International Workshop | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | |||||
| Address | |||||
| Corporate Author | Thesis | ||||
| Publisher | UAB | Place of Publication | Bellaterra (Spain) | Editor  |
Debora Gil; Jordi Gonzalez; Gemma Sanchez |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | 2 | Abbreviated Series Title | ||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-935251-4-9 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | IAM; ISE; DAG | Approved | no | ||
| Call Number | IAM @ iam @ GGS2007 | Serial | 1493 | ||
| Permanent link to this record | |||||
| Author | Patricia Marquez | ||||
| Title | A Confidence Framework for the Assessment of Optical Flow Performance | Type | Book Whole | ||
| Year | 2015 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Optical Flow (OF) is the input of a wide range of decision support systems such as car driver assistance, UAV guiding or medical diagnose. In these real situations, the absence of ground truth forces to assess OF quality using quantities computed from either sequences or the computed optical flow itself. These quantities are generally known as Confidence Measures, CM. Even if we have a proper confidence measure we still need a way to evaluate its ability to discard pixels with an OF prone to have a large error. Current approaches only provide a descriptive evaluation of the CM performance but such approaches are not capable to fairly compare different confidence measures and optical flow algorithms. Thus, it is of prime importance to define a framework and a general road map for the evaluation of optical flow performance.
This thesis provides a framework able to decide which pairs “ optical flow – confidence measure” (OF-CM) are best suited for optical flow error bounding given a confidence level determined by a decision support system. To design this framework we cover the following points: Descriptive scores. As a first step, we summarize and analyze the sources of inaccuracies in the output of optical flow algorithms. Second, we present several descriptive plots that visually assess CM capabilities for OF error bounding. In addition to the descriptive plots, given a plot representing OF-CM capabilities to bound the error, we provide a numeric score that categorizes the plot according to its decreasing profile, that is, a score assessing CM performance. Statistical framework. We provide a comparison framework that assesses the best suited OF-CM pair for error bounding that uses a two stage cascade process. First of all we assess the predictive value of the confidence measures by means of a descriptive plot. Then, for a sample of descriptive plots computed over training frames, we obtain a generic curve that will be used for sequences with no ground truth. As a second step, we evaluate the obtained general curve and its capabilities to really reflect the predictive value of a confidence measure using the variability across train frames by means of ANOVA. The presented framework has shown its potential in the application on clinical decision support systems. In particular, we have analyzed the impact of the different image artifacts such as noise and decay to the output of optical flow in a cardiac diagnose system and we have improved the navigation inside the bronchial tree on bronchoscopy. |
||||
| Address | July 2015 | ||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Debora Gil;Aura Hernandez | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-943427-2-1 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | IAM; 600.075 | Approved | no | ||
| Call Number | Admin @ si @ Mar2015 | Serial | 2687 | ||
| Permanent link to this record | |||||
| Author | Antonio Esteban Lansaque | ||||
| Title | An Endoscopic Navigation System for Lung Cancer Biopsy | Type | Book Whole | ||
| Year | 2019 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Lung cancer is one of the most diagnosed cancers among men and women. Actually,
lung cancer accounts for 13% of the total cases with a 5-year global survival rate in patients. Although Early detection increases survival rate from 38% to 67%, accurate diagnosis remains a challenge. Pathological confirmation requires extracting a sample of the lesion tissue for its biopsy. The preferred procedure for tissue biopsy is called bronchoscopy. A bronchoscopy is an endoscopic technique for the internal exploration of airways which facilitates the performance of minimal invasive interventions with low risk for the patient. Recent advances in bronchoscopic devices have increased their use for minimal invasive diagnostic and intervention procedures, like lung cancer biopsy sampling. Despite the improvement in bronchoscopic device quality, there is a lack of intelligent computational systems for supporting in-vivo clinical decision during examinations. Existing technologies fail to accurately reach the lesion due to several aspects at intervention off-line planning and poor intra-operative guidance at exploration time. Existing guiding systems radiate patients and clinical staff,might be expensive and achieve a suboptimlal 70% of yield boost. Diagnostic yield could be improved reducing radiation and costs by developing intra-operative support systems able to guide the bronchoscopist to the lesion during the intervention. The goal of this PhD thesis is to develop an image-based navigation systemfor intra-operative guidance of bronchoscopists to a target lesion across a path previously planned on a CT-scan. We propose a 3D navigation system which uses the anatomy of video bronchoscopy frames to locate the bronchoscope within the airways. Once the bronchoscope is located, our navigation system is able to indicate the bifurcation which needs to be followed to reach the lesion. In order to facilitate an off-line validation as realistic as possible, we also present a method for augmenting simulated virtual bronchoscopies with the appearance of intra-operative videos. Experiments performed on augmented and intra-operative videos, prove that our algorithm can be speeded up for an on-line implementation in the operating room. |
||||
| Address | October 2019 | ||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Debora Gil;Carles Sanchez | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-121011-0-2 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | IAM; 600.139; 600.145 | Approved | no | ||
| Call Number | Admin @ si @ Est2019 | Serial | 3392 | ||
| Permanent link to this record | |||||
| Author | Ferran Poveda | ||||
| Title | Computer Graphics and Vision Techniques for the Study of the Muscular Fiber Architecture of the Myocardium | Type | Book Whole | ||
| Year | 2013 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | |||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Place of Publication | Editor |
Debora Gil;Enric Marti | ||
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | Medium | |||
| Area | Expedition | Conference | |||
| Notes | IAM | Approved | no | ||
| Call Number | Admin @ si @ Pov2013 | Serial | 2417 | ||
| Permanent link to this record | |||||
| Author | David Roche | ||||
| Title | A Statistical Framework for Terminating Evolutionary Algorithms at their Steady State | Type | Book Whole | ||
| Year | 2015 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | As any iterative technique, it is a necessary condition a stop criterion for terminating Evolutionary Algorithms (EA). In the case of optimization methods, the algorithm should stop at the time it has reached a steady state so it can not improve results anymore. Assessing the reliability of termination conditions for EAs is of prime importance. A wrong or weak stop criterion can negatively a ect both the computational e ort and the nal result.
In this Thesis, we introduce a statistical framework for assessing whether a termination condition is able to stop EA at its steady state. In one hand a numeric approximation to steady states to detect the point in which EA population has lost its diversity has been presented for EA termination. This approximation has been applied to di erent EA paradigms based on diversity and a selection of functions covering the properties most relevant for EA convergence. Experiments show that our condition works regardless of the search space dimension and function landscape and Di erential Evolution (DE) arises as the best paradigm. On the other hand, we use a regression model in order to determine the requirements ensuring that a measure derived from EA evolving population is related to the distance to the optimum in xspace. Our theoretical framework is analyzed across several benchmark test functions and two standard termination criteria based on function improvement in f-space and EA population x-space distribution for the DE paradigm. Results validate our statistical framework as a powerful tool for determining the capability of a measure for terminating EA and select the x-space distribution as the best-suited for accurately stopping DE in real-world applications. |
||||
| Address | July 2015 | ||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Debora Gil;Jesus Giraldo | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | Medium | |||
| Area | Expedition | Conference | |||
| Notes | IAM; 600.075 | Approved | no | ||
| Call Number | Admin @ si @ Roc2015 | Serial | 2686 | ||
| Permanent link to this record | |||||
| Author | Sergio Vera | ||||
| Title | Anatomic Registration based on Medial Axis Parametrizations | Type | Book Whole | ||
| Year | 2015 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Image registration has been for many years the gold standard method to bring two images into correspondence. It has been used extensively in the eld of medical imaging in order to put images of di erent patients into a common overlapping spatial position. However, medical image registration is a slow, iterative optimization process, where many variables and prone to fall into the pit traps local minima.
A coordinate system parameterizing the interior of organs is a powerful tool for a systematic localization of injured tissue. If the same coordinate values are assigned to speci c anatomical sites, parameterizations ensure integration of data across different medical image modalities. Harmonic mappings have been used to produce parametric meshes over the surface of anatomical shapes, given their ability to set values at speci c locations through boundary conditions. However, most of the existing implementations in medical imaging restrict to either anatomical surfaces, or the depth coordinate with boundary conditions is given at discrete sites of limited geometric diversity. The medial surface of the shape can be used to provide a continuous basis for the de nition of a depth coordinate. However, given that di erent methods for generation of medial surfaces generate di erent manifolds, not all of them are equally suited to be the basis of radial coordinate for a parameterization. It would be desirable that the medial surface will be smooth, and robust to surface shape noise, with low number of spurious branches or surfaces. In this thesis we present methods for computation of smooth medial manifolds and apply them to the generation of for anatomical volumetric parameterization that extends current harmonic parameterizations to the interior anatomy using information provided by the volume medial surface. This reference system sets a solid base for creating anatomical models of the anatomical shapes, and allows comparing several patients in a common framework of reference. |
||||
| Address | November 2015 | ||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Debora Gil;Miguel Angel Gonzalez Ballester | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-943427-8-3 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | IAM; 600.075 | Approved | no | ||
| Call Number | Admin @ si @ Ver2015 | Serial | 2708 | ||
| Permanent link to this record | |||||
| Author | Lluis Gomez | ||||
| Title | Exploiting Similarity Hierarchies for Multi-script Scene Text Understanding | Type | Book Whole | ||
| Year | 2016 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | This thesis addresses the problem of automatic scene text understanding in unconstrained conditions. In particular, we tackle the tasks of multi-language and arbitrary-oriented text detection, tracking, and script identification in natural scenes.
For this we have developed a set of generic methods that build on top of the basic observation that text has always certain key visual and structural characteristics that are independent of the language or script in which it is written. Text instances in any language or script are always formed as groups of similar atomic parts, being them either individual characters, small stroke parts, or even whole words in the case of cursive text. This holistic (sumof-parts) and recursive perspective has lead us to explore different variants of the “segmentation and grouping” paradigm of computer vision. Scene text detection methodologies are usually based in classification of individual regions or patches, using a priory knowledge for a given script or language. Human perception of text, on the other hand, is based on perceptual organization through which text emerges as a perceptually significant group of atomic objects. In this thesis, we argue that the text detection problem must be posed as the detection of meaningful groups of regions. We address the problem of text detection in natural scenes from a hierarchical perspective, making explicit use of the recursive nature of text, aiming directly to the detection of region groupings corresponding to text within a hierarchy produced by an agglomerative similarity clustering process over individual regions. We propose an optimal way to construct such an hierarchy introducing a feature space designed to produce text group hypothese with high recall and a novel stopping rule combining a discriminative classifier and a probabilistic measure of group meaningfulness based in perceptual organization. Within this generic framework, we design a text-specific object proposals algorithm that, contrary to existing generic object proposals methods, aims directly to the detection of text regions groupings. For this, we abandon the rigid definition of “what is text” of traditional specialized text detectors, and move towards more fuzzy perspective of grouping-based object proposals methods. Then, we present a hybrid algorithm for detection and tracking of scene text where the notion of region groupings plays also a central role. By leveraging the structural arrangement of text group components between consecutive frames we can improve the overall tracking performance of the system. Finally, since our generic detection framework is inherently designed for multi-language environments, we focus on the problem of script identification in order to build a multi-language end-toend reading system. Facing this problem with state of the art CNN classifiers is not straightforward, as they fail to address a key characteristic of scene text instances: their extremely variable aspect ratio. Instead of resizing input images to a fixed size as in the typical use of holistic CNN classifiers, we propose a patch-based classification framework in order to preserve discriminative parts of the image that are characteristic of its class. We describe a novel method based on the use of ensembles of conjoined networks to jointly learn discriminative stroke-parts representations and their relative importance in a patch-based classification scheme. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Place of Publication | Editor |
Dimosthenis Karatzas | ||
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | Medium | |||
| Area | Expedition | Conference | |||
| Notes | DAG | Approved | no | ||
| Call Number | Admin @ si @ Gom2016 | Serial | 2891 | ||
| Permanent link to this record | |||||
| Author | Dena Bazazian | ||||
| Title | Fully Convolutional Networks for Text Understanding in Scene Images | Type | Book Whole | ||
| Year | 2018 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Text understanding in scene images has gained plenty of attention in the computer vision community and it is an important task in many applications as text carries semantically rich information about scene content and context. For instance, reading text in a scene can be applied to autonomous driving, scene understanding or assisting visually impaired people. The general aim of scene text understanding is to localize and recognize text in scene images. Text regions are first localized in the original image by a trained detector model and afterwards fed into a recognition module. The tasks of localization and recognition are highly correlated since an inaccurate localization can affect the recognition task.
The main purpose of this thesis is to devise efficient methods for scene text understanding. We investigate how the latest results on deep learning can advance text understanding pipelines. Recently, Fully Convolutional Networks (FCNs) and derived methods have achieved a significant performance on semantic segmentation and pixel level classification tasks. Therefore, we took benefit of the strengths of FCN approaches in order to detect text in natural scenes. In this thesis we have focused on two challenging tasks of scene text understanding which are Text Detection and Word Spotting. For the task of text detection, we have proposed an efficient text proposal technique in scene images. We have considered the Text Proposals method as the baseline which is an approach to reduce the search space of possible text regions in an image. In order to improve the Text Proposals method we combined it with Fully Convolutional Networks to efficiently reduce the number of proposals while maintaining the same level of accuracy and thus gaining a significant speed up. Our experiments demonstrate that this text proposal approach yields significantly higher recall rates than the line based text localization techniques, while also producing better-quality localization. We have also applied this technique on compressed images such as videos from wearable egocentric cameras. For the task of word spotting, we have introduced a novel mid-level word representation method. We have proposed a technique to create and exploit an intermediate representation of images based on text attributes which roughly correspond to character probability maps. Our representation extends the concept of Pyramidal Histogram Of Characters (PHOC) by exploiting Fully Convolutional Networks to derive a pixel-wise mapping of the character distribution within candidate word regions. We call this representation the Soft-PHOC. Furthermore, we show how to use Soft-PHOC descriptors for word spotting tasks through an efficient text line proposal algorithm. To evaluate the detected text, we propose a novel line based evaluation along with the classic bounding box based approach. We test our method on incidental scene text images which comprises real-life scenarios such as urban scenes. The importance of incidental scene text images is due to the complexity of backgrounds, perspective, variety of script and language, short text and little linguistic context. All of these factors together makes the incidental scene text images challenging. |
||||
| Address | November 2018 | ||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Dimosthenis Karatzas;Andrew Bagdanov | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-948531-1-1 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG; 600.121 | Approved | no | ||
| Call Number | Admin @ si @ Baz2018 | Serial | 3220 | ||
| Permanent link to this record | |||||
| Author | Antonio Clavelli | ||||
| Title | A computational model of eye guidance, searching for text in real scene images | Type | Book Whole | ||
| Year | 2014 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Searching for text objects in real scene images is an open problem and a very active computer vision research area. A large number of methods have been proposed tackling the text search as extension of the ones from the document analysis field or inspired by general purpose object detection methods. However the general problem of object search in real scene images remains an extremely challenging problem due to the huge variability in object appearance. This thesis builds on top of the most recent findings in the visual attention literature presenting a novel computational model of eye guidance aiming to better describe text object search in real scene images.
First are presented the relevant state-of-the-art results from the visual attention literature regarding eye movements and visual search. Relevant models of attention are discussed and integrated with recent observations on the role of top-down constraints and the emerging need for a layered model of attention in which saliency is not the only factor guiding attention. Visual attention is then explained by the interaction of several modulating factors, such as objects, value, plans and saliency. Then we introduce our probabilistic formulation of attention deployment in real scene. The model is based on the rationale that oculomotor control depends on two interacting but distinct processes: an attentional process that assigns value to the sources of information and motor process that flexibly links information with action. In such framework, the choice of where to look next is task-dependent and oriented to classes of objects embedded within pictures of complex scenes. The dependence on task is taken into account by exploiting the value and the reward of gazing at certain image patches or proto-objects that provide a sparse representation of the scene objects. In the experimental section the model is tested in laboratory condition, comparing model simulations with data from eye tracking experiments. The comparison is qualitative in terms of observable scan paths and quantitative in terms of statistical similarity of gaze shift amplitude. Experiments are performed using eye tracking data from both a publicly available dataset of face and text and from newly performed eye-tracking experiments on a dataset of street view pictures containing text. The last part of this thesis is dedicated to study the extent to which the proposed model can account for human eye movements in a low constrained setting. We used a mobile eye tracking device and an ad-hoc developed methodology to compare model simulated eye data with the human eye data from mobile eye tracking recordings. Such setting allow to test the model in an incomplete visual information condition, reproducing a close to real-life search task. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Dimosthenis Karatzas;Giuseppe Boccignone;Josep Llados | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-940902-6-4 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG; 600.077 | Approved | no | ||
| Call Number | Admin @ si @ Cla2014 | Serial | 2571 | ||
| Permanent link to this record | |||||
| Author | Ali Furkan Biten | ||||
| Title | A Bitter-Sweet Symphony on Vision and Language: Bias and World Knowledge | Type | Book Whole | ||
| Year | 2022 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Vision and Language are broadly regarded as cornerstones of intelligence. Even though language and vision have different aims – language having the purpose of communication, transmission of information and vision having the purpose of constructing mental representations around us to navigate and interact with objects – they cooperate and depend on one another in many tasks we perform effortlessly. This reliance is actively being studied in various Computer Vision tasks, e.g. image captioning, visual question answering, image-sentence retrieval, phrase grounding, just to name a few. All of these tasks share the inherent difficulty of the aligning the two modalities, while being robust to language
priors and various biases existing in the datasets. One of the ultimate goal for vision and language research is to be able to inject world knowledge while getting rid of the biases that come with the datasets. In this thesis, we mainly focus on two vision and language tasks, namely Image Captioning and Scene-Text Visual Question Answering (STVQA). In both domains, we start by defining a new task that requires the utilization of world knowledge and in both tasks, we find that the models commonly employed are prone to biases that exist in the data. Concretely, we introduce new tasks and discover several problems that impede performance at each level and provide remedies or possible solutions in each chapter: i) We define a new task to move beyond Image Captioning to Image Interpretation that can utilize Named Entities in the form of world knowledge. ii) We study the object hallucination problem in classic Image Captioning systems and develop an architecture-agnostic solution. iii) We define a sub-task of Visual Question Answering that requires reading the text in the image (STVQA), where we highlight the limitations of current models. iv) We propose an architecture for the STVQA task that can point to the answer in the image and show how to combine it with classic VQA models. v) We show how far language can get us in STVQA and discover yet another bias which causes the models to disregard the image while doing Visual Question Answering. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | IMPRIMA | Place of Publication | Editor |
Dimosthenis Karatzas;Lluis Gomez | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-124793-5-5 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG | Approved | no | ||
| Call Number | Admin @ si @ Bit2022 | Serial | 3755 | ||
| Permanent link to this record | |||||
| Author | Andres Mafla | ||||
| Title | Leveraging Scene Text Information for Image Interpretation | Type | Book Whole | ||
| Year | 2022 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Until recently, most computer vision models remained illiterate, largely ignoring the semantically rich and explicit information contained in scene text. Recent progress in scene text detection and recognition has recently allowed exploring its role in a diverse set of open computer vision problems, e.g. image classification, image-text retrieval, image captioning, and visual question answering to name a few. The explicit semantics of scene text closely requires specific modeling similar to language. However, scene text is a particular signal that has to be interpreted according to a comprehensive perspective that encapsulates all the visual cues in an image. Incorporating this information is a straightforward task for humans, but if we are unfamiliar with a language or scripture, achieving a complete world understanding is impossible (e.a. visiting a foreign country with a different alphabet). Despite the importance of scene text, modeling it requires considering the several ways in which scene text interacts with an image, processing and fusing an additional modality. In this thesis, we mainly focus
on two tasks, scene text-based fine-grained image classification, and cross-modal retrieval. In both studied tasks we identify existing limitations in current approaches and propose plausible solutions. Concretely, in each chapter: i) We define a compact way to embed scene text that generalizes to unseen words at training time while performing in real-time. ii) We incorporate the previously learned scene text embedding to create an image-level descriptor that overcomes optical character recognition (OCR) errors which is well-suited to the fine-grained image classification task. iii) We design a region-level reasoning network that learns the interaction through semantics among salient visual regions and scene text instances. iv) We employ scene text information in image-text matching and introduce the Scene Text Aware Cross-Modal retrieval StacMR task. We gather a dataset that incorporates scene text and design a model suited for the newly studied modality. v) We identify the drawbacks of current retrieval metrics in cross-modal retrieval. An image captioning metric is proposed as a way of better evaluating semantics in retrieved results. Ample experimentation shows that incorporating such semantics into a model yields better semantic results while requiring significantly less data to converge. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | IMPRIMA | Place of Publication | Editor |
Dimosthenis Karatzas;Lluis Gomez | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-124793-6-2 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG | Approved | no | ||
| Call Number | Admin @ si @ Maf2022 | Serial | 3756 | ||
| Permanent link to this record | |||||
| Author | Raul Gomez | ||||
| Title | Exploiting the Interplay between Visual and Textual Data for Scene Interpretation | Type | Book Whole | ||
| Year | 2020 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Machine learning experimentation under controlled scenarios and standard datasets is necessary to compare algorithms performance by evaluating all of them in the same setup. However, experimentation on how those algorithms perform on unconstrained data and applied tasks to solve real world problems is also a must to ascertain how that research can contribute to our society.
In this dissertation we experiment with the latest computer vision and natural language processing algorithms applying them to multimodal scene interpretation. Particularly, we research on how image and text understanding can be jointly exploited to address real world problems, focusing on learning from Social Media data. We address several tasks that involve image and textual information, discuss their characteristics and offer our experimentation conclusions. First, we work on detection of scene text in images. Then, we work with Social Media posts, exploiting the captions associated to images as supervision to learn visual features, which we apply to multimodal semantic image retrieval. Subsequently, we work with geolocated Social Media images with associated tags, experimenting on how to use the tags as supervision, on location sensitive image retrieval and on exploiting location information for image tagging. Finally, we work on a specific classification problem of Social Media publications consisting on an image and a text: Multimodal hate speech classification. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Dimosthenis Karatzas;Lluis Gomez;Jaume Gibert | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-121011-7-1 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG; 600.121 | Approved | no | ||
| Call Number | Admin @ si @ Gom20 | Serial | 3479 | ||
| Permanent link to this record | |||||
| Author | Jaume Gibert | ||||
| Title | Vector Space Embedding of Graphs via Statistics of Labelling Information | Type | Book Whole | ||
| Year | 2012 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Pattern recognition is the task that aims at distinguishing objects among different classes. When such a task wants to be solved in an automatic way a crucial step is how to formally represent such patterns to the computer. Based on the different representational formalisms, we may distinguish between statistical and structural pattern recognition. The former describes objects as a set of measurements arranged in the form of what is called a feature vector. The latter assumes that relations between parts of the underlying objects need to be explicitly represented and thus it uses relational structures such as graphs for encoding their inherent information. Vector spaces are a very flexible mathematical structure that has allowed to come up with several efficient ways for the analysis of patterns under the form of feature vectors. Nevertheless, such a representation cannot explicitly cope with binary relations between parts of the objects and it is restricted to measure the exact same number of features for each pattern under study regardless of their complexity. Graph-based representations present the contrary situation. They can easily adapt to the inherent complexity of the patterns but introduce a problem of high computational complexity, hindering the design of efficient tools to process and analyse patterns.
Solving this paradox is the main goal of this thesis. The ideal situation for solving pattern recognition problems would be to represent the patterns using relational structures such as graphs, and to be able to use the wealthy repository of data processing tools from the statistical pattern recognition domain. An elegant solution to this problem is to transform the graph domain into a vector domain where any processing algorithm can be applied. In other words, by mapping each graph to a point in a vector space we automatically get access to the rich set of algorithms from the statistical domain to be applied in the graph domain. Such methodology is called graph embedding. In this thesis we propose to associate feature vectors to graphs in a simple and very efficient way by just putting attention on the labelling information that graphs store. In particular, we count frequencies of node labels and of edges between labels. Although their locality, these features are able to robustly represent structurally global properties of graphs, when considered together in the form of a vector. We initially deal with the case of discrete attributed graphs, where features are easy to compute. The continuous case is tackled as a natural generalization of the discrete one, where rather than counting node and edge labelling instances, we count statistics of some representatives of them. We encounter how the proposed vectorial representations of graphs suffer from high dimensionality and correlation among components and we face these problems by feature selection algorithms. We also explore how the diversity of different embedding representations can be exploited in order to boost the performance of base classifiers in a multiple classifier systems framework. An extensive experimental evaluation finally shows how the methodology we propose can be efficiently computed and compete with other graph matching and embedding methodologies. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Ernest Valveny | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | Medium | |||
| Area | Expedition | Conference | |||
| Notes | DAG | Approved | no | ||
| Call Number | Admin @ si @ Gib2012 | Serial | 2204 | ||
| Permanent link to this record | |||||
| Author | Suman Ghosh | ||||
| Title | Word Spotting and Recognition in Images from Heterogeneous Sources A | Type | Book Whole | ||
| Year | 2018 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Text is the most common way of information sharing from ages. With recent development of personal images databases and handwritten historic manuscripts the demand for algorithms to make these databases accessible for browsing and indexing are in rise. Enabling search or understanding large collection of manuscripts or image databases needs fast and robust methods. Researchers have found different ways to represent cropped words for understanding and matching, which works well when words are already segmented. However there is no trivial way to extend these for non-segmented documents. In this thesis we explore different methods for text retrieval and recognition from unsegmented document and scene images. Two different ways of representation exist in literature, one uses a fixed length representation learned from cropped words and another a sequence of features of variable length. Throughout this thesis, we have studied both these representation for their suitability in segmentation free understanding of text. In the first part we are focused on segmentation free word spotting using a fixed length representation. We extended the use of the successful PHOC (Pyramidal Histogram of Character) representation to segmentation free retrieval. In the second part of the thesis, we explore sequence based features and finally, we propose a unified solution where the same framework can generate both kind of representations. | ||||
| Address | November 2018 | ||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | Ediciones Graficas Rey | Place of Publication | Editor |
Ernest Valveny | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-948531-0-4 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG; 600.121 | Approved | no | ||
| Call Number | Admin @ si @ Gho2018 | Serial | 3217 | ||
| Permanent link to this record | |||||
| Author | Ruben Perez Tito | ||||
| Title | Exploring the role of Text in Visual Question Answering on Natural Scenes and Documents | Type | Book Whole | ||
| Year | 2023 | Publication | PhD Thesis, Universitat Autonoma de Barcelona-CVC | Abbreviated Journal | |
| Volume | Issue | Pages | |||
| Keywords | |||||
| Abstract | Visual Question Answering (VQA) is the task where given an image and a natural language question, the objective is to generate a natural language answer. At the intersection between computer vision and natural language processing, this task can be seen as a measure of image understanding capabilities, as it requires to reason about objects, actions, colors, positions, the relations between the different elements as well as commonsense reasoning, world knowledge, arithmetic skills and natural language understanding. However, even though the text present in the images conveys important semantically rich information that is explicit and not available in any other form, most VQA methods remained illiterate, largely
ignoring the text despite its potential significance. In this thesis, we set out on a journey to bring reading capabilities to computer vision models applied to the VQA task, creating new datasets and methods that can read, reason and integrate the text with other visual cues in natural scene images and documents. In Chapter 3, we address the combination of scene text with visual information to fully understand all the nuances of natural scene images. To achieve this objective, we define a new sub-task of VQA that requires reading the text in the image, and highlight the limitations of the current methods. In addition, we propose a new architecture that integrates both modalities and jointly reasons about textual and visual features. In Chapter 5, we shift the domain of VQA with reading capabilities and apply it on scanned industry document images, providing a high-level end-purpose perspective to Document Understanding, which has been primarily focused on digitizing the document’s contents and extracting key values without considering the ultimate purpose of the extracted information. For this, we create a dataset which requires methods to reason about the unique and challenging elements of documents, such as text, images, tables, graphs and complex layouts, to provide accurate answers in natural language. However, we observed that explicit visual features provide a slight contribution in the overall performance, since the main information is usually conveyed within the text and its position. In consequence, in Chapter 6, we propose VQA on infographic images, seeking for document images with more visually rich elements that require to fully exploit visual information in order to answer the questions. We show the performance gap of different methods when used over industry scanned and infographic images, and propose a new method that integrates the visual features in early stages, which allows the transformer architecture to exploit the visual features during the self-attention operation. Instead, in Chapter 7, we apply VQA on a big collection of single-page documents, where the methods must find which documents are relevant to answer the question, and provide the answer itself. Finally, in Chapter 8, mimicking real-world application problems where systems must process documents with multiple pages, we address the multipage document visual question answering task. We demonstrate the limitations of existing methods, including models specifically designed to process long sequences. To overcome these limitations, we propose a hierarchical architecture that can process long documents, answer questions, and provide the index of the page where the information to answer the question is located as an explainability measure. |
||||
| Address | |||||
| Corporate Author | Thesis | Ph.D. thesis | |||
| Publisher | IMPRIMA | Place of Publication | Editor |
Ernest Valveny | |
| Language | Summary Language | Original Title | |||
| Series Editor | Series Title | Abbreviated Series Title | |||
| Series Volume | Series Issue | Edition | |||
| ISSN | ISBN | 978-84-124793-5-5 | Medium | ||
| Area | Expedition | Conference | |||
| Notes | DAG | Approved | no | ||
| Call Number | Admin @ si @ Per2023 | Serial | 3967 | ||
| Permanent link to this record | |||||