Sparse Depth Map Interpolation using Deep Convolutional Neural Networks

I. Makarov; Alisa Korinevskaya; Vladimir Aliev

doi:10.1109/TSP.2018.8441443

Publications

?

Sparse Depth Map Interpolation using Deep Convolutional Neural Networks

P. 1–5.

Makarov I., Alisa Korinevskaya, Vladimir Aliev

The problem of dense depth map inference from sparse depth values is considered in this paper. We address this issue in situation when one has low-cost sensor data and limited computational resources. We propose a method that performs interpolation and then super-resolution while comparing our approach with the state-of-the-art direct RGB-to-Dense reconstruction solutions. In particular, we use an encoder-decoder model of CNN with loss consisting of standard mean squared error and perceptual loss function. Futhermore, it has been shown that the described approach could be adopted to estimate rough depth map in real-time.

Keywords: depth reconstruction

Publication based on the results of:

Applied network research with big data and new technological advances (2018)

In book

Proceedings of 2018 41st International Conference on Telecommunications and Signal Processing (TSP)

NY: IEEE, 2018.

Self-supervised recurrent depth estimation with attention mechanisms

Makarov I., Bakhanova M., Nikolenko S. et al., PeerJ Computer Science 2022 Vol. 8 Article e865

Depth estimation has been an essential task for many computer vision applications, especially in autonomous driving, where safety is paramount. Depth can be estimated not only with traditional supervised learning but also via a self-supervised approach that relies on camera motion and does not require ground truth depth maps. Recently, major improvements have been introduced ...

Added: February 1, 2022

Learning Loss for Active Learning in Depth Reconstruction Problem

Makarov I., Guschenko-Cheverda I., , in: Proceedings of IEEE 21st International Symposium on Computational Intelligence and Informatics (CINTI'21), 18-20 Nov. 2021.: NY: IEEE, 2021. P. 000115–000120.

Added: January 19, 2022

Fast Depth Reconstruction Using Deep Convolutional Neural Networks

Dmitrii Maslov, Makarov I., , in: Advances in Computational Intelligence: 16th International Work-Conference on Artificial Neural Networks, IWANN 2021, Virtual Event, June 16–18, 2021, Proceedings, Part I* 1. Vol. 12861.: Springer, 2021. Ch. 38 P. 456–467.

In this paper, we study depth reconstruction via RGB-based, Sparse-Depth, and RGBd approaches. We showed that combination of RGB and Sparse Depth approach in RGBd scenario provides the best results. We also proved that the models performance can be further tuned via proper selection of architecture blocks and number of depth points guiding RGB-to-depth reconstruction. ...

Added: September 1, 2021

Online supervised attention-based recurrent depth estimation from monocular video

Dmitrii Maslov, Makarov I., PeerJ Computer Science 2020 Vol. 6 No. e317 P. 1–22

Autonomous driving highly depends on depth information for safe driving. Recently, major improvements have been taken towards improving both supervised and self-supervised methods for depth reconstruction. However, most of the current approaches focus on single frame depth estimation, where quality limit is hard to beat due to limitations of supervised learning of deep neural networks ...

Added: October 27, 2020

Real-time Vision-based Depth Reconstruction with NVidia Jetson

Bokovoy A., Muravyev K., Yakovlev K., , in: Proceedings of the 2019 European Conference on Mobile Robotics (ECMR 2019).: Prague: IEEE, 2019. P. 1–6.

Vision-based depth reconstruction is a challenging problem extensively studied in computer vision but still lacking universal solution. Reconstructing depth from single image is particularly valuable to mobile robotics as it can be embedded to the modern vision-based simultaneous localization and mapping (vSLAM) methods providing them with the metric information needed to construct accurate maps in ...

Added: January 15, 2020

Fast Depth Map Super-Resolution Using Deep Neural Network

Alisa Korinevskaya, Makarov I., , in: Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR'18).: NY: IEEE, 2019. P. 117–122.

Depth map super-resolution is a challenging computer vision problem. In this paper, we present two deep convolutional neural networks solving the problem of single depth map super-resolution. Both networks learn residual decomposition and trained with specific perceptual loss improving sharpness and perceptive quality of the upsampled depth map. Several experiments on various depth super-resolution benchmark ...

Added: July 29, 2019

Fast Semi-dense Depth Map Estimation

Makarov I., Alisa Korinevskaya, Vladimir Aliev, , in: Proceedings of the 2018 ACM ICMR Workshop on Multimedia for Real Estate Tech.: NY: Association for Computing Machinery (ACM), 2018. P. 18–21.

We consider the problem of depth reconstruction from downsampled sparse depth values. We compare our approach with semi-dense depth map interpolation and direct RGB-to-Depth reconstruction solutions on several datasets, including Matterport 3D dataset containing RGB and depth images of 90 building-scale scenes. We demonstrate that the proposed model can produce approximate depth map for over ...

Added: September 3, 2018