HOWTO: NVIDIA Jetson TX2: Productive Docker Development & Benchmarking

My previous post, HOWTO: Setting Up NVIDIA Jetson TX2 <-> Mac Laptop Development Workflow, detailed getting a productive workflow going between a Mac Laptop and an NVIDIA Jetson TX2. This post will contain notes and writeups from my work getting a productive Docker workflow going on the Jetson, as well doing performance, power, thermal, etc. benchmarking.

SSH into your Jetson container. Make sure that Docker is actually running correctly; if it isn't the following command will print an error:

sudo docker version

Using Docker on your Jetson will allow you to create development sandboxes, which is especially useful if you are doing deep learning or other things that might require extensive configuration of the OS. The Docker sandboxes can have different versions of TensorFlow, PyTorch, etc.

For your base Docker image, you should use one of the official NVIDIA L4T base images (L4T=Linux for Tegra). These images have the necessary hooks for the Docker container to integrate well with Jetson. You can see a list of these NVIDIA L4T images here (you will need an NVIDIA Developer Account).

A good workflow is to start with a base Dockerfile of some kind to bootstrap a bash shell, then you can experiment within this container as you install different packages to see what you need, making sure to re-add any environment setup back into your Dockerfile.

In these examples we will assume a test directory for a computer vision oriented deep net being tested with the following file layout:

• jetson-experiments/
• src/ - Python source for some TensorFlow inference graph
• models/ - Trained TensorFlow weight files that will be used for inference
• images/ - Example input images that would be used for this TF network
• mounted/ - Mounted output directory where inference output artifacts will go.

Here's a basic Dockerfile that can be leveraged; in this Dockerfile we are using a base NVIDIA image that has TensorFlow 2.3 and Python 3.6. If you need something different see the NVIDIA L4T base images.

FROM nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3

ENV DEBIAN_FRONTEND=noninteractive

ENV SHELL /bin/bash

WORKDIR jetson-experiments

#

# Install some common useful packages then clean up to save space.

#

RUN apt-get update && \

    apt-get install -y --no-install-recommends \

            software-properties-common \

            cmake \

            python3-opencv \

            htop \

            vim \

            curl \

            less \

    && rm -rf /var/lib/apt/lists/*

# alias python3 -> python

RUN rm /usr/bin/python && \

    ln -s /usr/bin/python3.6 /usr/bin/python && \

    ln -s /usr/bin/pip3 /usr/bin/pip

RUN export PYTHON_PATH=/jetson-experiments/src

RUN export PATH=$PATH:/usr/bin

# Ensure we can get low-level stats on the Jetson.

COPY bin/tegrastats /usr/bin/tegrastats

To build this Dockerfile:

sudo docker pull nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3

sudo docker build -f Dockerfile .

Take note of the new Docker image ID, which you will use to set CONTAINER_IMAGE below.

To run this Dockerfile, dumping you into a Bash shell in the Docker container you can play with:

export CONTAINER_IMAGE=...

export USER_COMMAND=/bin/bash

# If trying to get a VNC window into the container; must be run

# from inside of an X Windows session:

#sudo xhost +si:localuser:root

sudo docker run \

    -it --rm \

    -e DISPLAY=$DISPLAY \

    --runtime nvidia \

    --network host \

    --volume $PWD/mounted:/jetson-experiments/mounted \

    --volume $PWD/src:/jetson-experiments/src \

    --volume $PWD/images:/jetson-experiments/images \

    --volume $PWD/models:/jetson-experiments/models \

    --volume /tmp/.X11-unix/:/tmp/.X11-unix \

    --volume /tmp/argus_socket:/tmp/argus_socket \

    $CONTAINER_IMAGE $USER_COMMAND

Once inside the container you can experiment with other commands, updating your Dockerfile as appropriate. Note that the --volume options for src, images, and models means that the Docker container will inherit these directories from the native Jetson environment -- this means if you rsync over new files while you are developing on your Mac they will seamlessly get picked up inside the Docker container, which can help with quicker development.

As you do this workflow you can build up a significant number of Docker images, which can take up valuable disk space on the Jetson. To clean these up, periodically run this command to clean up unused images:

sudo docker image prune -a -f

It can be useful to start a container into a bash shell, running some process like a deep net that you want profile. If this is true, you will want to open other interactive bash shells into the same container where you can run htop to monitor performance.

To do this, start your first primary container bash shell as given above with the docker run command. Then, run the following to get the container ID for this container instance:

sudo docker container ls

Take note of the value in the CONTAINER ID column for the instance you started already.

Then, assuming you are running tmux for your overall SSH session on the Jetson, open another tmux window with the following:

export CONTAINER_ID=...

sudo docker exec -it $CONTAINER_ID /bin/bash

Now you can run htop in these other Docker sessions to gain visibility into your primary process. Note that the tegrastats command is not available inside a Docker container; instead, you will have to run it outside of the container to get visibility into overall GPU details.