|
German Ros, Laura Sellart, Joanna Materzynska, David Vazquez and Antonio Lopez. 2016. The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. 29th IEEE Conference on Computer Vision and Pattern Recognition.3234–3243.
Abstract: Vision-based semantic segmentation in urban scenarios is a key functionality for autonomous driving. The irruption of deep convolutional neural networks (DCNNs) allows to foresee obtaining reliable classifiers to perform such a visual task. However, DCNNs require to learn many parameters from raw images; thus, having a sufficient amount of diversified images with this class annotations is needed. These annotations are obtained by a human cumbersome labour specially challenging for semantic segmentation, since pixel-level annotations are required. In this paper, we propose to use a virtual world for automatically generating realistic synthetic images with pixel-level annotations. Then, we address the question of how useful can be such data for the task of semantic segmentation; in particular, when using a DCNN paradigm. In order to answer this question we have generated a synthetic diversified collection of urban images, named SynthCity, with automatically generated class annotations. We use SynthCity in combination with publicly available real-world urban images with manually provided annotations. Then, we conduct experiments on a DCNN setting that show how the inclusion of SynthCity in the training stage significantly improves the performance of the semantic segmentation task
Keywords: Domain Adaptation; Autonomous Driving; Virtual Data; Semantic Segmentation
|
|
|
R. de Nijs, Sebastian Ramos, Gemma Roig, Xavier Boix, Luc Van Gool and K. Kühnlenz. 2012. On-line Semantic Perception Using Uncertainty. International Conference on Intelligent Robots and Systems.4185–4191.
Abstract: Visual perception capabilities are still highly unreliable in unconstrained settings, and solutions might not beaccurate in all regions of an image. Awareness of the uncertainty of perception is a fundamental requirement for proper high level decision making in a robotic system. Yet, the uncertainty measure is often sacrificed to account for dependencies between object/region classifiers. This is the case of Conditional Random Fields (CRFs), the success of which stems from their ability to infer the most likely world configuration, but they do not directly allow to estimate the uncertainty of the solution. In this paper, we consider the setting of assigning semantic labels to the pixels of an image sequence. Instead of using a CRF, we employ a Perturb-and-MAP Random Field, a recently introduced probabilistic model that allows performing fast approximate sampling from its probability density function. This allows to effectively compute the uncertainty of the solution, indicating the reliability of the most likely labeling in each region of the image. We report results on the CamVid dataset, a standard benchmark for semantic labeling of urban image sequences. In our experiments, we show the benefits of exploiting the uncertainty by putting more computational effort on the regions of the image that are less reliable, and use more efficient techniques for other regions, showing little decrease of performance
Keywords: Semantic Segmentation
|
|
|
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez and Vladlen Koltun. 2017. CARLA: An Open Urban Driving Simulator. 1st Annual Conference on Robot Learning. Proceedings of Machine Learning.1–16.
Abstract: We introduce CARLA, an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites and environmental conditions. We use CARLA to study the performance of three approaches to autonomous driving: a classic modular pipeline, an endto-end
model trained via imitation learning, and an end-to-end model trained via
reinforcement learning. The approaches are evaluated in controlled scenarios of
increasing difficulty, and their performance is examined via metrics provided by CARLA, illustrating the platform’s utility for autonomous driving research.
Keywords: Autonomous driving; sensorimotor control; simulation
|
|
|
Idoia Ruiz, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder and Joan Serrat. 2021. Weakly Supervised Multi-Object Tracking and Segmentation. IEEE Winter Conference on Applications of Computer Vision Workshops.125–133.
Abstract: We introduce the problem of weakly supervised MultiObject Tracking and Segmentation, i.e. joint weakly supervised instance segmentation and multi-object tracking, in which we do not provide any kind of mask annotation.
To address it, we design a novel synergistic training strategy by taking advantage of multi-task learning, i.e. classification and tracking tasks guide the training of the unsupervised instance segmentation. For that purpose, we extract weak foreground localization information, provided by
Grad-CAM heatmaps, to generate a partial ground truth to learn from. Additionally, RGB image level information is employed to refine the mask prediction at the edges of the
objects. We evaluate our method on KITTI MOTS, the most representative benchmark for this task, reducing the performance gap on the MOTSP metric between the fully supervised and weakly supervised approach to just 12% and 12.7 % for cars and pedestrians, respectively.
|
|
|
Cristina Cañero and 16 others. 1999. Optimal Stent Implantation: Three-dimensional Evaluation of the Mutual Position of Stent and Vessel via Intracoronary Ecography. Proceedings of International Conference on Computer in Cardiology (CIC´99).
Abstract: We present a new automatic technique to visualize and quantify the mutual position between the stent and the vessel wall by considering their three-dimensional reconstruction. Two deformable generalized cylinders adapt to the image features in all IVUS planes corresponding to the vessel wall and the stent in order to reconstruct the boundaries of the stent and the vessel in space. The image features that characterize the stent and the vessel wall are determined in terms of edge and ridge image detectors taking into account the gray level of the image pixels. We show that the 30 reconstruction by deformable cylinders is accurate and robust due to the spatial data coherence in the considered volumetric IVUS image. The main clinic utility of the stent and vessel reconstruction by deformable’ cylinders consists of its possibility to visualize and to assess the optimal stent introduction.
|
|
|
German Ros, J. Guerrero, Angel Sappa and Antonio Lopez. 2013. VSLAM pose initialization via Lie groups and Lie algebras optimization. Proceedings of IEEE International Conference on Robotics and Automation.5740–5747.
Abstract: We present a novel technique for estimating initial 3D poses in the context of localization and Visual SLAM problems. The presented approach can deal with noise, outliers and a large amount of input data and still performs in real time in a standard CPU. Our method produces solutions with an accuracy comparable to those produced by RANSAC but can be much faster when the percentage of outliers is high or for large amounts of input data. On the current work we propose to formulate the pose estimation as an optimization problem on Lie groups, considering their manifold structure as well as their associated Lie algebras. This allows us to perform a fast and simple optimization at the same time that conserve all the constraints imposed by the Lie group SE(3). Additionally, we present several key design concepts related with the cost function and its Jacobian; aspects that are critical for the good performance of the algorithm.
Keywords: SLAM
|
|
|
Jose Carlos Rubio, Joan Serrat, Antonio Lopez and N. Paragios. 2012. Image Contextual Representation and Matching through Hierarchies and Higher Order Graphs. 21st International Conference on Pattern Recognition.2664–2667.
Abstract: We present a region matching algorithm which establishes correspondences between regions from two segmented images. An abstract graph-based representation conceals the image in a hierarchical graph, exploiting the scene properties at two levels. First, the similarity and spatial consistency of the image semantic objects is encoded in a graph of commute times. Second, the cluttered regions of the semantic objects are represented with a shape descriptor. Many-to-many matching of regions is specially challenging due to the instability of the segmentation under slight image changes, and we explicitly handle it through high order potentials. We demonstrate the matching approach applied to images of world famous buildings, captured under different conditions, showing the robustness of our method to large variations in illumination and viewpoint.
|
|
|
Yainuvis Socarras, Sebastian Ramos, David Vazquez, Antonio Lopez and Theo Gevers. 2013. Adapting Pedestrian Detection from Synthetic to Far Infrared Images. ICCV Workshop on Visual Domain Adaptation and Dataset Bias. Sydney, Australy.
Abstract: We present different techniques to adapt a pedestrian classifier trained with synthetic images and the corresponding automatically generated annotations to operate with far infrared (FIR) images. The information contained in this kind of images allow us to develop a robust pedestrian detector invariant to extreme illumination changes.
Keywords: Domain Adaptation; Far Infrared; Pedestrian Detection
|
|
|
Jiaolong Xu, Sebastian Ramos, Xu Hu, David Vazquez and Antonio Lopez. 2013. Multi-task Bilinear Classifiers for Visual Domain Adaptation. Advances in Neural Information Processing Systems Workshop.
Abstract: We propose a method that aims to lessen the significant accuracy degradation
that a discriminative classifier can suffer when it is trained in a specific domain (source domain) and applied in a different one (target domain). The principal reason for this degradation is the discrepancies in the distribution of the features that feed the classifier in different domains. Therefore, we propose a domain adaptation method that maps the features from the different domains into a common subspace and learns a discriminative domain-invariant classifier within it. Our algorithm combines bilinear classifiers and multi-task learning for domain adaptation.
The bilinear classifier encodes the feature transformation and classification
parameters by a matrix decomposition. In this way, specific feature transformations for multiple domains and a shared classifier are jointly learned in a multi-task learning framework. Focusing on domain adaptation for visual object detection, we apply this method to the state-of-the-art deformable part-based model for cross domain pedestrian detection. Experimental results show that our method significantly avoids the domain drift and improves the accuracy when compared to several baselines.
Keywords: Domain Adaptation; Pedestrian Detection; ADAS
|
|
|
Victor Vaquero, German Ros, Francesc Moreno-Noguer, Antonio Lopez and Alberto Sanfeliu. 2017. Joint coarse-and-fine reasoning for deep optical flow. 24th International Conference on Image Processing.2558–2562.
Abstract: We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets.
|
|