Deploying Deep Learning

Welcome to our instructional guide for inference and realtime DNN vision library for NVIDIA Jetson Nano/TX1/TX2/Xavier.

This repo uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded Jetson platform, improving performance and power efficiency using graph optimizations, kernel fusion, and FP16/INT8 precision.

Vision primitives, such as imageNet for image recognition, detectNet for object localization, and segNet for semantic segmentation, inherit from the shared tensorNet object. Examples are provided for streaming from live camera feed and processing images. See the API Reference section for detailed reference documentation of the C++ and Python libraries.

There are multiple tracks of the tutorial that you can choose to follow, including Hello AI World for running inference and transfer learning onboard your Jetson, or the full Two Days to a Demo tutorial for training on a PC or server with DIGITS.

It's recommended to walk through the Hello AI World module first to familiarize yourself with machine learning and inference with TensorRT, before proceeding to training in the cloud with DIGITS.

Table of Contents

• Hello AI World
• Two Days to a Demo
• API Reference
• Code Examples
• Pre-Trained Models
• System Requirements
• Extra Resources

>   Jetson Nano Developer Kit and JetPack 4.3 is now supported in the repo.
>   See our latest technical blog including benchmarks, Jetson Nano Brings AI Computing to Everyone.
>   Hello AI World now supports Python and onboard training with PyTorch!

Hello AI World

Hello AI World can be run completely onboard your Jetson, including inferencing with TensorRT and transfer learning with PyTorch. The inference portion of Hello AI World - which includes coding your own image classification application for C++ or Python, object detection, and live camera demos - can be run on your Jetson in roughly two hours or less, while transfer learning is best left to leave running overnight.

• Setting up Jetson with JetPack
• Building the Project from Source
• Classifying Images with ImageNet
  • Using the Console Program on Jetson
  • Coding Your Own Image Recognition Program (Python)
  • Coding Your Own Image Recognition Program (C++)
  • Running the Live Camera Recognition Demo
• Locating Objects with DetectNet
  • Detecting Objects from the Command Line
  • Running the Live Camera Detection Demo
  • Coding Your Own Object Detection Program
• Semantic Segmentation with SegNet
  • Segmenting Images from the Command Line
  • Running the Live Camera Segmentation Demo
• Transfer Learning with PyTorch
  • Re-training on the Cat/Dog Dataset
  • Re-training on the PlantCLEF Dataset
  • Collecting your own Datasets

Two Days to a Demo (DIGITS)

The full tutorial includes training in the cloud or PC, and inference on the Jetson with TensorRT, and can take roughly two days or more depending on system setup, downloading the datasets, and the training speed of your GPU.

• DIGITS Workflow
• DIGITS System Setup
• Setting up Jetson with JetPack
• Building the Project from Source
• Classifying Images with ImageNet
  • Using the Console Program on Jetson
  • Coding Your Own Image Recognition Program
  • Running the Live Camera Recognition Demo
  • Re-Training the Network with DIGITS
  • Downloading Image Recognition Dataset
  • Customizing the Object Classes
  • Importing Classification Dataset into DIGITS
  • Creating Image Classification Model with DIGITS
  • Testing Classification Model in DIGITS
  • Downloading Model Snapshot to Jetson
  • Loading Custom Models on Jetson
• Locating Objects with DetectNet
  • Detection Data Formatting in DIGITS
  • Downloading the Detection Dataset
  • Importing the Detection Dataset into DIGITS
  • Creating DetectNet Model with DIGITS
  • Testing DetectNet Model Inference in DIGITS
  • Downloading the Detection Model to Jetson
  • DetectNet Patches for TensorRT
  • Detecting Objects from the Command Line
  • Multi-class Object Detection Models
  • Running the Live Camera Detection Demo on Jetson
• Semantic Segmentation with SegNet
  • Downloading Aerial Drone Dataset
  • Importing the Aerial Dataset into DIGITS
  • Generating Pretrained FCN-Alexnet
  • Training FCN-Alexnet with DIGITS
  • Testing Inference Model in DIGITS
  • FCN-Alexnet Patches for TensorRT
  • Running Segmentation Models on Jetson

API Reference

Below are links to reference documentation for the C++ and Python libraries from the repo:

jetson-inference

jetson-utils

• C++
• Python

These libraries are able to be used in external projects by linking to libjetson-inference and libjetson-utils.

Code Examples

Introductory code walkthroughs of using the library are covered during these steps of the Hello AI World tutorial:

• Coding Your Own Image Recognition Program (Python)
• Coding Your Own Image Recognition Program (C++)

Additional C++ and Python samples for running the networks on static images and live camera streams can be found here:

note: for working with numpy arrays, see cuda-from-numpy.py and cuda-to-numpy.py

These examples will automatically be compiled while Building the Project from Source, and are able to run the pre-trained models listed below in addition to custom models provided by the user. Launch each example with --help for usage info.

Pre-Trained Models

The project comes with a number of pre-trained models that are available through the Model Downloader tool:

Image Recognition

Object Detection

Semantic Segmentation

• If the resolution is omitted from the CLI argument, the lowest resolution model is loaded
• Accuracy indicates the pixel classification accuracy across the model's validation dataset
• Performance is measured for GPU FP16 mode with JetPack 4.2.1, nvpmodel 0 (MAX-N)

Recommended System Requirements

Training GPU: Maxwell, Pascal, Volta, or Turing-based GPU (ideally with at least 6GB video memory)
                        optionally, AWS P2/P3 instance or Microsoft Azure N-series
                        Ubuntu 16.04/18.04 x86_64

Deployment:   Jetson Nano Developer Kit with JetPack 4.2 or newer (Ubuntu 18.04 aarch64).
                        Jetson Xavier Developer Kit with JetPack 4.0 or newer (Ubuntu 18.04 aarch64)
                        Jetson TX2 Developer Kit with JetPack 3.0 or newer (Ubuntu 16.04 aarch64).
                        Jetson TX1 Developer Kit with JetPack 2.3 or newer (Ubuntu 16.04 aarch64).

Note that TensorRT samples from the repo are intended for deployment onboard Jetson, however when cuDNN and TensorRT have been installed on the host side, the TensorRT samples in the repo can be compiled for PC.

Extra Resources

In this area, links and resources for deep learning are listed:

• ros_deep_learning - TensorRT inference ROS nodes
• NVIDIA AI IoT - NVIDIA Jetson GitHub repositories
• Jetson eLinux Wiki - Jetson eLinux Wiki

Legacy Links

^{© 2016-2019 NVIDIA | Table of Contents}