Real-time Vision-based Depth Reconstruction with NVidia Jetson

?

Real-time Vision-based Depth Reconstruction with NVidia Jetson

P. 1–6.

Bokovoy A., Muravyev K., Yakovlev K.

Vision-based depth reconstruction is a challenging problem extensively studied in computer vision but still lacking universal solution. Reconstructing depth from single image is particularly valuable to mobile robotics as it can be embedded to the modern vision-based simultaneous localization and mapping (vSLAM) methods providing them with the metric information needed to construct accurate maps in real scale. Typically, depth reconstruction is done nowadays via fully-convolutional neural networks (FCNNs). In this work we experiment with several FCNN architectures and introduce a few enhancements aimed at increasing both the effectiveness and the efficiency of the inference. We experimentally determine the solution that provides the best performance/accuracy tradeoff and is able to run on NVidia Jetson with the framerates exceeding 16FPS for 320 × 240 input. We also evaluate the suggested models by conducting monocular vSLAM of unknown indoor environment on NVidia Jetson TX2 in real-time. Open-source implementation of the models and the inference node for Robot Operating System (ROS) are available at https://github.com/CnnDepth/tx2_fcnn_node.

Language: English

DOI

Text on another site

In book

Proceedings of the 2019 European Conference on Mobile Robotics (ECMR 2019)

Prague: IEEE, 2019.

Deep learning deciphers the related role of master regulators and G-quadruplexes in tissue specification

Artem B., Andreasyan A., Konovalov D. et al., Scientific Reports 2025 Vol. 15 Article 23119

G-quadruplexes (GQs) are non-canonical DNA structures encoded by G-flipons with potential roles in gene regulation and chromatin structure. Here, we explore the role of G-flipons in tissue specification. We present a deep learning-based framework for the genome-wide G-flipon predictions across 14 human tissue types. The model was trained using high-confidence experimental maps of GQ-forming sequences ...

Added: August 8, 2025

DeepZ: A Deep Learning Approach for Z-DNA Prediction

Beknazarov N., , in: Z-DNA: Methods and Protocols.: United States of America: Springer, 2023. P. 217–226.

Here we describe an approach that uses deep learning neural networks such as CNN and RNN to aggregate information from DNA sequence; physical, chemical, and structural properties of nucleotides; and omics data on histone modifications, methylation, chromatin accessibility, and transcription factor binding sites and data from other available NGS experiments. We explain how with the ...

Added: December 26, 2023

Self-supervised recurrent depth estimation with attention mechanisms

Makarov I., Bakhanova M., Nikolenko S. et al., PeerJ Computer Science 2022 Vol. 8 Article e865

Depth estimation has been an essential task for many computer vision applications, especially in autonomous driving, where safety is paramount. Depth can be estimated not only with traditional supervised learning but also via a self-supervised approach that relies on camera motion and does not require ground truth depth maps. Recently, major improvements have been introduced ...

Added: February 1, 2022

Learning Loss for Active Learning in Depth Reconstruction Problem

Makarov I., Guschenko-Cheverda I., , in: Proceedings of IEEE 21st International Symposium on Computational Intelligence and Informatics (CINTI'21), 18-20 Nov. 2021.: NY: IEEE, 2021. P. 000115–000120.

Added: January 19, 2022

Fast Depth Reconstruction Using Deep Convolutional Neural Networks

Dmitrii Maslov, Makarov I., , in: Advances in Computational Intelligence: 16th International Work-Conference on Artificial Neural Networks, IWANN 2021, Virtual Event, June 16–18, 2021, Proceedings, Part I* 1. Vol. 12861.: Springer, 2021. Ch. 38 P. 456–467.

In this paper, we study depth reconstruction via RGB-based, Sparse-Depth, and RGBd approaches. We showed that combination of RGB and Sparse Depth approach in RGBd scenario provides the best results. We also proved that the models performance can be further tuned via proper selection of architecture blocks and number of depth points guiding RGB-to-depth reconstruction. ...

Added: September 1, 2021

Neural Network for Real-Time Object Detection on FPGA

Rzaev E., Khanaev A., Amerikanov A., , in: 2021 International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM).: IEEE, 2021. P. 719–723.

Added: July 4, 2021

Deep learning approach for predicting functional Z-DNA regions using omics data

Beknazarov N., Jin S., Poptsova M., Scientific Reports 2020 Vol. 10 P. 19134

Computational methods to predict Z-DNA regions are in high demand to understand the functional role of Z-DNA. The previous state-of-the-art method Z-Hunt is based on statistical mechanical and energy considerations about B- to Z-DNA transition using sequence information. Z-DNA CHiP-seq experiment results showed little overlap with Z-Hunt predictions implying that sequence information only is not ...

Added: December 11, 2020

Map-Merging Algorithms for Visual SLAM: Feasibility Study and Empirical Evaluation

Bokovoy A., Muravyev K., Yakovlev K., , in: Artificial Intelligence. RCAI 2020.: Switzerland: Springer, 2020. P. 46–60.

Simultaneous localization and mapping, especially the one relying solely on video data (vSLAM), is a challenging problem that has been extensively studied in robotics and computer vision. State-of-the-art vSLAM algorithms are capable of constructing accurate-enough maps that enable a mobile robot to autonomously navigate an unknown environment. In this work, we are interested in an ...

Added: November 2, 2020

Online supervised attention-based recurrent depth estimation from monocular video

Dmitrii Maslov, Makarov I., PeerJ Computer Science 2020 Vol. 6 No. e317 P. 1–22

Autonomous driving highly depends on depth information for safe driving. Recently, major improvements have been taken towards improving both supervised and self-supervised methods for depth reconstruction. However, most of the current approaches focus on single frame depth estimation, where quality limit is hard to beat due to limitations of supervised learning of deep neural networks ...

Added: October 27, 2020

Fast Depth Map Super-Resolution Using Deep Neural Network

Alisa Korinevskaya, Makarov I., , in: Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR'18).: NY: IEEE, 2019. P. 117–122.

Depth map super-resolution is a challenging computer vision problem. In this paper, we present two deep convolutional neural networks solving the problem of single depth map super-resolution. Both networks learn residual decomposition and trained with specific perceptual loss improving sharpness and perceptive quality of the upsampled depth map. Several experiments on various depth super-resolution benchmark ...

Added: July 29, 2019

Fast Semi-dense Depth Map Estimation

Makarov I., Alisa Korinevskaya, Vladimir Aliev, , in: Proceedings of the 2018 ACM ICMR Workshop on Multimedia for Real Estate Tech.: NY: Association for Computing Machinery (ACM), 2018. P. 18–21.

We consider the problem of depth reconstruction from downsampled sparse depth values. We compare our approach with semi-dense depth map interpolation and direct RGB-to-Depth reconstruction solutions on several datasets, including Matterport 3D dataset containing RGB and depth images of 90 building-scale scenes. We demonstrate that the proposed model can produce approximate depth map for over ...

Added: September 3, 2018

Sparse Depth Map Interpolation using Deep Convolutional Neural Networks

Makarov I., Alisa Korinevskaya, Vladimir Aliev, , in: Proceedings of 2018 41st International Conference on Telecommunications and Signal Processing (TSP).: NY: IEEE, 2018. P. 1–5.

The problem of dense depth map inference from sparse depth values is considered in this paper. We address this issue in situation when one has low-cost sensor data and limited computational resources. We propose a method that performs interpolation and then super-resolution while comparing our approach with the state-of-the-art direct RGB-to-Dense reconstruction solutions. In particular, ...

Added: September 3, 2018