tag:blogger.com,1999:blog-31912912022-01-23T00:46:11.768-08:00Coding In ParadiseBrad Neuberg's WeblogBrad Neuberghttp://www.blogger.com/profile/03274020042497854648noreply@blogger.comBlogger731125tag:blogger.com,1999:blog-3191291.post-40137360978307125312021-01-19T16:03:00.004-08:002021-01-19T16:03:35.960-08:00HOWTO: Adding Keyboard Accelerators to Holoviz Applications for Machine Learning Workflows<p>My team at <a href="https://www.planet.com/">Planet</a> trains deep nets with large amounts of geospatial remote sensing data, for tasks such as semantic segmentation. We've created a set of internal Jupyter Notebooks built above the <a href="https://holoviz.org/">HoloViz</a> ecosystem for quickly jumping through, validating, QAing, etc. this training data, leaning on tools like <a href="http://geoviews.org/">GeoViews</a> to efficiently display and create these UIs. Since we are sometimes dealing with thousands of samples, keyboard accelerators are very important for productively dealing with this data.</p><p>Unfortunately, Bokeh/HoloViews/GeoViews don't have any official mechanisms for adding keyboard accelerators. I recently created a workaround for this that was tricky, and thought sharing it might help others in a similar situation when attempting to create Bokeh/HoloViews powered Jupyter Notebooks for the kinds of workflow situations machine learning often requires.</p><p>To use, you will first need to add a <span style="background-color: #cccccc; font-family: courier;">Grab Keyboard</span> button to your HoloViz UI. Here's an example UI from our own Planet QA work with this button added:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-044UZRdpBkE/YAdw6SeXb7I/AAAAAAAAF0s/Y-NUVzSlogA-Fotfj8Zm6gl1rzH11ZPlgCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="423" data-original-width="1284" height="211" src="https://lh3.googleusercontent.com/-044UZRdpBkE/YAdw6SeXb7I/AAAAAAAAF0s/Y-NUVzSlogA-Fotfj8Zm6gl1rzH11ZPlgCLcBGAsYHQ/w640-h211/image.png" width="640" /></a></div><br /><p></p><p>Clicking the <span style="background-color: #cccccc; font-family: courier;">Grab Keyboard</span> button will "inject" JavaScript keyboard handling into the page; if the user clicks the <span style="background-color: #cccccc; font-family: courier;">Release Keyboard</span> button that toggles or loses focus, then keyboard handling will be deactivated. This is so that Jupyter's own keyboard handling doesn't collide with these keyboard accelerators. Image showing the toggled <span style="background-color: #cccccc; font-family: courier;">Release Keyboard</span> button:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-uFlUz5pxvOA/YAdxaXNwn9I/AAAAAAAAF00/ubtnOvuhjm4A1GXIdYRFOX7TW9tiK3DwACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="425" data-original-width="1288" height="212" src="https://lh3.googleusercontent.com/-uFlUz5pxvOA/YAdxaXNwn9I/AAAAAAAAF00/ubtnOvuhjm4A1GXIdYRFOX7TW9tiK3DwACLcBGAsYHQ/w640-h212/image.png" width="640" /></a></div><br />In your HoloViews app, make sure it has this button; also define a unique instance_class CSS class that will be used to 'silo' your panel when dealing with its buttons in case there are multiple panel instances in the Jupyter notebook:<p></p><p><span style="background-color: #cccccc; font-family: courier;">self.instance_class = 'some_tool'<br />self.grab_keyboard_button = pn.widgets.Button(name="Grab Keyboard")</span></p><p>Pass both of these into your panel when drawing the UI:</p><div style="text-align: left;"><span style="background-color: #cccccc; font-family: courier;">def panel(self):<br />&nbsp; &nbsp; &nbsp; &nbsp; return pn.Column(<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; # ... other parts of your UI<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; pn.Row(<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; pn.Column(<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; # 4 pn.widgets.Buttons that we want to attach keyboard accelerators to.<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; pn.Row(<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.prev_button,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.valid_button,&nbsp;<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.invalid_button,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.next_button,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; pn.layout.HSpacer(),<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <b>pn.Column(<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; # The Grab Keyboard button.<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.grab_keyboard_button,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; align="center",<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),</b><br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; background='lightgrey',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <b># Make sure we have a JavaScript hook to bind onto the buttons for just this<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; # pyviz panel in case there are multiple ones in a Jupyter notebook. Used<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; # so that we can add keybindings.<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; css_classes=[self.instance_class],</b><br />&nbsp; &nbsp; &nbsp; &nbsp; )</span></div><p>Next, you will need to grab the <span style="background-color: #cccccc; font-family: courier;">JSKeyboardAccelerators</span> class <a href="https://gist.github.com/BradNeuberg/ab1e4fd7d687c99b8be1d9ede9905e4d" target="_blank">from this gist I put up</a> and save it to the file js_keyboard_accelerators.py.</p><p>Now instantiate a <span style="background-color: #cccccc; font-family: courier;">JSKeyboardAccelerators</span> instance with the actions you want to bind to:</p><div style="text-align: left;"><span style="background-color: #cccccc; font-family: courier;">self.accelerators = JSKeyboardAccelerators(self.instance_class, self.grab_keyboard_button,<br />&nbsp; &nbsp; &nbsp; actions=[<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'action_name': 'Previous',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'keycode': JSKeycodes.LEFT_ARROW,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'html_button_ordering': 0,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'expected_text': '◀',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; },<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'action_name': 'Next',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'keycode': JSKeycodes.RIGHT_ARROW,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'html_button_ordering': 1,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'expected_text': '▶',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; },<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'action_name': 'Invalid',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'keycode': JSKeycodes.DOWN_ARROW,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'html_button_ordering': 2,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'expected_text': '✖',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; },<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'action_name': 'Valid',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'keycode': JSKeycodes.UP_ARROW,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'html_button_ordering': 3,<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'expected_text': '✔',<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; },<br />&nbsp; &nbsp; &nbsp; &nbsp;])</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">The <span style="background-color: #cccccc; font-family: courier;">action_name</span> and <span style="background-color: #cccccc; font-family: courier;">expected_text</span> fields are debugging fields that will be printed to the JavaScript console when this keyboard accelerator is pressed to aid in ensuring that the right action is being invoked; <span style="background-color: #cccccc; font-family: courier;">keycode</span> is one of the <span style="background-color: #cccccc; font-family: courier;">JSKeycodes</span> enums (add your own if one is not listed that you want to use); and <span style="background-color: #cccccc; font-family: courier;">html_button_ordering</span> is the ordering of the button you want to be 'clicked' on in the background -- basically, the order returned from the CSS selector for your instance_class defined, such as <span style="background-color: #cccccc; font-family: courier;">'.some_tool button'</span>.</div><p>Hopefully this helps your own work!</p>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-80678272659736755842021-01-07T07:05:00.003-08:002021-01-18T17:49:23.640-08:00HOWTO: NVIDIA Jetson TX2: Productive Docker Development & Benchmarking<div style="text-align: left;">My previous post,&nbsp;<a href="http://blog.codinginparadise.org/2021/01/howto-getting-productive-development.html">HOWTO: Setting Up NVIDIA Jetson TX2 &lt;-&gt; Mac Laptop Development Workflow,</a> detailed getting a productive workflow going between a Mac Laptop and an NVIDIA Jetson TX2. This post will contain notes and writeups from my work getting a productive Docker workflow going on the Jetson, as well doing performance, power, thermal, etc. benchmarking.</div><div><br /></div><div>SSH into your Jetson container. Make sure that Docker is actually running correctly; if it isn't the following command will print an error:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker version</span></div><div><br /></div><div>Using Docker on your Jetson will allow you to create development sandboxes, which is especially useful if you are doing deep learning or other things that might require extensive configuration of the OS. The Docker sandboxes can have different versions of TensorFlow, PyTorch, etc.</div><div><br /></div><div>For your base Docker image, you should use one of the official NVIDIA L4T base images (L4T=Linux for Tegra). These images have the necessary hooks for the Docker container to integrate well with Jetson. You can see a list of these NVIDIA L4T images <a href="https://ngc.nvidia.com/catalog/containers?orderBy=scoreDESC&amp;pageNumber=0&amp;query=l4t&amp;quickFilter=&amp;filters=">here</a> (you will need an NVIDIA Developer Account).</div><div><br /></div><div>A good workflow is to start with a base <span style="background-color: #cccccc; font-family: courier;">Dockerfile</span> of some kind to bootstrap a bash shell, then you can experiment within this container as you install different packages to see what you need, making sure to re-add any environment setup back into your Dockerfile.</div><div><br /></div><div>In these examples we will assume a test directory for a computer vision oriented deep net being tested with the following file layout:</div><div><ul style="text-align: left;"><li><span style="background-color: #cccccc; font-family: courier;">jetson-experiments/</span></li><ul><li><span style="background-color: #cccccc; font-family: courier;">src/</span> - Python source for some TensorFlow inference graph</li><li><span style="background-color: #cccccc; font-family: courier;">models/</span> - Trained TensorFlow weight files that will be used for inference</li><li><span style="background-color: #cccccc; font-family: courier;">images/</span> - Example input images that would be used for this TF network</li><li><span style="background-color: #cccccc; font-family: courier;">mounted/</span> - Mounted output directory where inference output artifacts will go.</li></ul></ul></div><div>Here's a basic <span style="background-color: #cccccc; font-family: courier;">Dockerfile</span> that can be leveraged; in this <span style="background-color: #cccccc; font-family: courier;">Dockerfile</span> we are using a base NVIDIA image that has TensorFlow 2.3 and Python 3.6. If you need something different see the <a href="https://ngc.nvidia.com/catalog/containers?orderBy=scoreDESC&amp;pageNumber=0&amp;query=l4t&amp;quickFilter=&amp;filters=">NVIDIA L4T base images</a>.</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">FROM nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">ENV DEBIAN_FRONTEND=noninteractive</span></div><div><span style="background-color: #cccccc; font-family: courier;">ENV SHELL /bin/bash</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">WORKDIR jetson-experiments</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">#</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Install some common useful packages then clean up to save space.</span></div><div><span style="background-color: #cccccc; font-family: courier;">#</span></div><div><span style="background-color: #cccccc; font-family: courier;">RUN apt-get update &amp;&amp; \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; apt-get install -y --no-install-recommends \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; software-properties-common \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; cmake \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; python3-opencv \</span></div><div><span style="background-color: #cccccc; font-family: courier;"><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp; &nbsp; htop \</span><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"><span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp; &nbsp; vim \</span><br /></span></span></div><div><span style="background-color: #cccccc; font-family: courier;"><span><span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp; &nbsp; curl \</span><br /></span></span></span></div><div><span style="background-color: #cccccc; font-family: courier;"><span><span><span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp;&nbsp; &nbsp;</span><span>&nbsp; &nbsp; less \</span><br /></span></span></span></span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &amp;&amp; rm -rf /var/lib/apt/lists/*</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># alias python3 -&gt; python</span></div><div><span style="background-color: #cccccc; font-family: courier;">RUN rm /usr/bin/python &amp;&amp; \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; ln -s /usr/bin/python3.6 /usr/bin/python &amp;&amp; \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; ln -s /usr/bin/pip3 /usr/bin/pip</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">RUN export PYTHON_PATH=/jetson-experiments/src</span></div></div><div><span style="background-color: #cccccc; font-family: courier;">RUN export PATH=$PATH:/usr/bin</span></div><div><div><span style="background-color: #cccccc;"><span style="font-family: courier;"><br /></span></span></div><div><span style="background-color: #cccccc;"><span style="font-family: courier;"># Ensure we can get low-level stats on the Jetson.</span></span></div><div><span style="background-color: #cccccc;"><span style="font-family: courier;">COPY bin/tegrastats /usr/bin/tegrastats</span></span></div></div><div><br /></div><div>To build this Dockerfile:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker pull nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker build -f Dockerfile .</span></div></div><div><br /></div><div>Take note of the new Docker image ID, which you will use to set <span style="background-color: #cccccc; font-family: courier;">CONTAINER_IMAGE</span> below.</div><div><br /></div><div>To run this <span style="background-color: #cccccc; font-family: courier;">Dockerfile</span>, dumping you into a Bash shell in the Docker container you can play with:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">export CONTAINER_IMAGE=...</span></div><div><span style="background-color: #cccccc; font-family: courier;">export USER_COMMAND=/bin/bash</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># If trying to get a VNC window into the container; must be run</span></div><div><span style="background-color: #cccccc; font-family: courier;"># from inside of an X Windows session:</span></div><div><span style="background-color: #cccccc; font-family: courier;">#sudo xhost +si:localuser:root</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker run \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; -it --rm \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; -e DISPLAY=$DISPLAY \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; --runtime nvidia \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; --network host \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; --volume $PWD/mounted:/jetson-experiments/mounted \</span></div><div><span style="font-family: courier;"><div><span style="background-color: #cccccc;">&nbsp;&nbsp; &nbsp;--volume $PWD/src:/jetson-experiments/src \</span></div><div><span style="background-color: #cccccc;">&nbsp; &nbsp; --volume $PWD/images:/jetson-experiments/images \</span></div><div><span style="background-color: #cccccc;">&nbsp; &nbsp; --volume $PWD/models:/jetson-experiments/models \</span></div></span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; --volume /tmp/.X11-unix/:/tmp/.X11-unix \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; --volume /tmp/argus_socket:/tmp/argus_socket \</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; $CONTAINER_IMAGE $USER_COMMAND</span></div></div><div><br /></div><div>Once inside the container you can experiment with other commands, updating your Dockerfile as appropriate. Note that the <span style="background-color: #cccccc; font-family: courier;">--volume</span> options for <span style="background-color: #cccccc; font-family: courier;">src</span>, <span style="background-color: #cccccc; font-family: courier;">images</span>, and <span style="background-color: #cccccc; font-family: courier;">models</span> means that the Docker container will inherit these directories from the native Jetson environment -- this means if you <span style="background-color: #cccccc; font-family: courier;">rsync</span> over new files while you are developing on your Mac they will seamlessly get picked up inside the Docker container, which can help with quicker development.</div><div><br /></div><div>As you do this workflow you can build up a significant number of Docker images, which can take up valuable disk space on the Jetson. To clean these up, periodically run this command to clean up unused images:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker image prune -a -f</span></div><div><br /></div><div>It can be useful to start a container into a bash shell, running some process like a deep net that you want profile. If this is true, you will want to open other interactive bash shells <i>into the same container</i>&nbsp;where you can run <span style="background-color: #cccccc; font-family: courier;">htop</span>&nbsp;to monitor performance.</div><div><br /></div><div>To do this, start your first primary container bash shell as given above with the docker run command. Then, run the following to get the container ID for this container instance:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker container ls</span></div><div><br /></div><div>Take note of the value in the <span style="background-color: #cccccc; font-family: courier;">CONTAINER ID</span> column for the instance you started already.</div><div><br /></div><div>Then, assuming you are running <span style="background-color: #cccccc; font-family: courier;">tmux</span> for your overall SSH session on the Jetson, open another <span style="background-color: #cccccc; font-family: courier;">tmux</span> window with the following:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">export CONTAINER_ID=...</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker exec -it $CONTAINER_ID /bin/bash</span></div><div><br /></div><div>Now you can run <span style="background-color: #cccccc; font-family: courier;">htop</span>&nbsp;in these other Docker sessions to gain visibility into your primary process. Note that the <span style="background-color: #cccccc; font-family: courier;">tegrastats</span> command is <i>not</i>&nbsp;available inside a Docker container; instead, you will have to run it outside of the container to get visibility into overall GPU details.</div><div><br /></div><div><br /></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-32965948833441907182021-01-05T17:49:00.017-08:002021-01-18T17:18:36.572-08:00HOWTO: Setting Up NVIDIA Jetson TX2 <-> Mac Laptop Development Workflow<h1 style="text-align: left;">Introduction</h1><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-8nYGSjFV4UI/X_ULLKQaxZI/AAAAAAAAFyU/Kj7tdjTN8jIuI-9XDFWqvRa5WaMTc9WBQCLcBGAsYHQ/jetson1.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="756" data-original-width="1008" height="300" src="https://lh3.googleusercontent.com/-8nYGSjFV4UI/X_ULLKQaxZI/AAAAAAAAFyU/Kj7tdjTN8jIuI-9XDFWqvRa5WaMTc9WBQCLcBGAsYHQ/w400-h300/jetson1.png" width="400" /></a><br /><br />Jetson TX2 with external monitor connected</div><br /></div><p>This post details how to get a productive development cycle between a Mac and an <a href="https://developer.nvidia.com/embedded/jetson-tx2-developer-kit">NVIDIA Jetson TX2</a> development board, so that you can easily do your coding on your Mac and SSH into your Jetson. By default, the host development computer for a Jetson has to be an Ubuntu Linux install, generally with a USB cable directly plugged into the Jetson development board. At the end of this HOWTO we will be able to directly develop on the Mac without having to have Ubuntu running in a VM at the same time, connecting over a WiFi connection to the Jetson board so you don't need a direct cable. In addition, this HOWTO also details how to get Docker containers running on the Jetson, how to compile some common machine learning oriented packages for the ARM chips on the Jetson, and how to install an NVMe SSD storage device for greater development room on the Jetson.</p><p>Hardware/Software details on what this was tested on:</p><p></p><ul style="text-align: left;"><li>Mac OS X 10.15.7 (Catalina)</li><li>MacBook Pro 2019 (x86 architecture) with 4 Thunderbolt 3 ports</li><li>NVIDIA Jetson TX2 development board carrier <a href="https://developer.nvidia.com/embedded/dlc/jetson-tx1-tx2-developer-kit-carrier-board-c02-design-files">C02 model</a>&nbsp;</li><li>Ubuntu 18.04.5 (Bionic Beaver) LTS</li><li>NVIDIA Jetpack 4.4.1</li><li>CUDA 10.2</li><li>cuDNN 8.0</li><li>NVIDIA Container Runtime with Docker 0.9.0</li></ul><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-U5Eq0jXJZEI/X_ULNDUxYJI/AAAAAAAAFyY/FVeUgi8plOoC3ZixqtuimkPt-IQqL8b9QCLcBGAsYHQ/jetson2.png" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="" data-original-height="756" data-original-width="1008" height="300" src="https://lh3.googleusercontent.com/-U5Eq0jXJZEI/X_ULNDUxYJI/AAAAAAAAFyY/FVeUgi8plOoC3ZixqtuimkPt-IQqL8b9QCLcBGAsYHQ/w400-h300/jetson2.png" width="400" /></a><br /><br />Jetson TX2 with Mac laptop in background and NVMe SSD attached to motherboard</div><div></div><h1 style="text-align: left;">Initial Setup</h1><div></div><p></p><div>Turn on the Jetson, following the printed guide with the board and get Ubuntu setup on the TX2 itself, confirming that everything works. Make sure you have a monitor and keyboard attached to the Jetson development board when you do so. Also ensure that you have an <a href="https://developer.nvidia.com/developer-program">NVIDIA developer account</a> before proceeding, as this will be needed for the full dev workflow in this HOWTO.</div><h1 style="text-align: left;">Get Ubuntu 18 VM Going on Mac OS X</h1><div>By default the Jetson needs Ubuntu running on the host computer for development. We need to get this bootstrapped on our Mac before proceeding.</div><div><br /></div><div>On your Mac, download and install <a href="https://download.virtualbox.org/virtualbox/6.1.16/VirtualBox-6.1.16-140961-OSX.dmg">Virtualbox 6.1.16</a>, then also download and install the <a href="https://download.virtualbox.org/virtualbox/6.1.16/Oracle_VM_VirtualBox_Extension_Pack-6.1.16.vbox-extpack">Virtualbox Extensions</a>.</div><div><br /></div><div>Install Ubuntu 18 into this Virtualbox on your Mac. First download <a href="https://releases.ubuntu.com/18.04/ubuntu-18.04.5-desktop-amd64.iso">Ubuntu 18.04.5 (Bionic Beaver) LTS</a>&nbsp;and install it into Virtualbox:</div><div><br /></div><div><span id="docs-internal-guid-b5785635-7fff-8bb1-9ebf-c74b5885305c"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 376px; overflow: hidden; width: 624px;"><img height="376" src="https://lh6.googleusercontent.com/z9V_JNCO-fj9kkozHkPPWNpmrUmFTVl5eim-LdWSwML0wvwvGrjohZ3oG3AThhbLamHtqQ5tjL1TqVPqsD4dfva0kJMMZQdeEwOljBcHCbZo0gncbI91yGXzPUFRkpuHBQTV_Wym" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></span></div><div><br /></div><div><span id="docs-internal-guid-6932a963-7fff-b2bf-99ec-667dedef90ac"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 356px; overflow: hidden; width: 624px;"><img height="356" src="https://lh5.googleusercontent.com/wdN8c-RELDO2A3FgL15zLWOZxbDgR9pflIsuJB-eYpWWeLyJsJJDz2mQbSJW4i8SGW3eukh5vyLuEshhaqiukCWegD1SLzM0-v3qKVE_Qf7OJWvLisyG8QLwXleR_B7oLs1FLtre" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></span></div><div><br /></div><div>Create an Ubuntu machine with the following settings:</div><div><ul style="text-align: left;"><li>Storage is larger than 50GBs</li><li>Go to Settings -&gt; Network -&gt; Adapter 1, change <span style="background-color: #cccccc; font-family: courier;">Attached to</span> to <span style="background-color: #cccccc; font-family: courier;">Bridged Adapter</span>, and name to whatever under Wi-Fi</li><li>Go to Settings -&gt; Ports -&gt; USB, ensure <span style="font-family: courier;"><span style="background-color: #cccccc;">Enable USB Controller</span></span> is under <span style="font-family: courier;"><span style="background-color: #cccccc;">USB 3.0 (xHCI) Controller</span></span></li></ul><div>Last, load the image that you downloaded and spin up a VM.<br /><br />Right click on our new machine and go to <span style="font-family: courier;"><span style="background-color: #cccccc;">Start &gt; Normal Start</span></span> or select the machine and just click the Green Start Arrow icon. We'll soon be prompted with the following, where we'll now select our downloaded ISO file, and click Start.</div></div><div><span style="background-color: white;"><br /></span></div><div>At first your Ubuntu VM window will be very small. To temporary fix this before we configure our display setting in Ubuntu directly, click the fifth icon from the bottom right and go to&nbsp;<span style="background-color: #cccccc; font-family: courier;">Virtual Screen 1 &gt; Scale to 200% (autoscaled output)</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div>Inside the VM Ubuntu will now do a Linux install. Select a minimal install, plus install third party options. Select erase disk and install Ubuntu. Once its finished, restart the Ubuntu VM.<br /><br />When you are back inside the restarted Ubuntu VM, fix the screen resolution. Go to the 9 dots in the bottom left of the Ubuntu UI and search for <span style="font-family: courier;"><span style="background-color: #cccccc;">Settings</span></span><span style="background-color: white;">:</span></div><div><span style="background-color: white;"><br /></span></div><div><div class="separator" style="background-color: white; clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-Q3QLFAHf8Cg/X_TSdmb23DI/AAAAAAAAFwc/XEe9I1k9jCsJVph7DzFXZP3mUrT97uy8gCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="573" data-original-width="700" height="524" src="https://lh3.googleusercontent.com/-Q3QLFAHf8Cg/X_TSdmb23DI/AAAAAAAAFwc/XEe9I1k9jCsJVph7DzFXZP3mUrT97uy8gCLcBGAsYHQ/w640-h524/image.png" width="640" /></a></div><br />Once settings shows up, scroll to the bottom with the left sidebar until you see <span style="font-family: courier;"><span style="background-color: #cccccc;">Devices</span></span> and click on it.</div><div><span style="background-color: white;"><br /></span></div><div><div class="separator" style="clear: both;"><span><a href="https://lh3.googleusercontent.com/-MGWvx6Y8zpU/X_TS751Ec8I/AAAAAAAAFwk/DHjuvj2QkiMGYSxwd3MuHYHMvZb-zqAHQCLcBGAsYHQ/image.png" style="background-color: white; margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="573" data-original-width="700" height="524" src="https://lh3.googleusercontent.com/-MGWvx6Y8zpU/X_TS751Ec8I/AAAAAAAAFwk/DHjuvj2QkiMGYSxwd3MuHYHMvZb-zqAHQCLcBGAsYHQ/w640-h524/image.png" width="640" /></a></span>Select the new resolution from the dropdown -- note that part of it will be cut off.<span><br /></span></div></div><div><span style="background-color: white;"><br /></span></div><div><div class="separator" style="background-color: white; clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-Jl08h7xQkbA/X_TTELsWr3I/AAAAAAAAFwo/XRmpcehXYJYrClZ92d4WvlYdpDjiYqgQwCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="573" data-original-width="700" height="524" src="https://lh3.googleusercontent.com/-Jl08h7xQkbA/X_TTELsWr3I/AAAAAAAAFwo/XRmpcehXYJYrClZ92d4WvlYdpDjiYqgQwCLcBGAsYHQ/w640-h524/image.png" width="640" /></a></div><br />It will prompt you the top right to apply these changes, but unfortunately you can’t click <span style="font-family: courier;"><span style="background-color: #cccccc;">Apply</span></span> because it’s not within the view of the resolution of the window. To get access it to, double-click the top navigation bar where it says&nbsp;<span style="background-color: #cccccc; font-family: courier;">Displays</span>, and then drag the window from right to left until you see the&nbsp;<span style="background-color: #cccccc; font-family: courier;">Apply</span><span style="background-color: white;">&nbsp;</span>button.</div><div><span style="background-color: white;"><br /></span></div><div><div class="separator" style="background-color: white; clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-WE1eC8nXaus/X_TTemeyytI/AAAAAAAAFw4/-cciKVf_3so7AgLqZPFeJEu_udj_KNtlACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="573" data-original-width="700" height="524" src="https://lh3.googleusercontent.com/-WE1eC8nXaus/X_TTemeyytI/AAAAAAAAFw4/-cciKVf_3so7AgLqZPFeJEu_udj_KNtlACLcBGAsYHQ/w640-h524/image.png" width="640" /></a></div><br />The resolution will change, and you’ll be prompted to confirm if you want to keep these changes. If it doesn’t fit your screen, leave it for a few seconds and it will reset back, otherwise click <span style="font-family: courier;"><span style="background-color: #cccccc;">Keep Changes</span></span>.</div><div><span style="background-color: white;"><br /></span></div><div><span><div class="separator" style="background-color: white; clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-Mu6Hoo5h1vY/X_TToBOMX1I/AAAAAAAAFw8/Gac-SQMj9vge-ZGEYo2j3fAZGqw8e_mBACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="469" data-original-width="700" height="428" src="https://lh3.googleusercontent.com/-Mu6Hoo5h1vY/X_TToBOMX1I/AAAAAAAAFw8/Gac-SQMj9vge-ZGEYo2j3fAZGqw8e_mBACLcBGAsYHQ/w640-h428/image.png" width="640" /></a></div><br /></span>Next, open a browser window inside the Ubuntu VM window on your Mac. <a href="https://www.blogger.com/u/1/#">Download</a> the Jetpack 4.4.1 NVIDIA SDK Manager Method of download (you will need to use your NVIDIA developer portal credentials). When you download Jetpack, save it then double-click it to install or else you will get Ubuntu permission errors.</div><div><span style="background-color: white;"><br /></span></div><div>Once Jetpack is installed inside the Ubuntu VM on your Mac, click the 9 dots menu on the lower-left of Ubuntu's UI, search for the word "<span style="font-family: courier;"><span style="background-color: #cccccc;">SDK</span></span>", then double-click the NVIDIA SDK Manager icon that appears. Sign into the NVIDIA developer portal again through the SDK manager. Inside the SDK manager UI, select the Target Hardware to be the Jetson TX2, and select DeepStream to also be installed. Select the <span style="font-family: courier;"><span style="background-color: #cccccc;">Download now, install later</span></span> option.</div><br /><h1 style="text-align: left;">Flash Over Updated OS and SDK to Jetson</h1><div>Now we will use the host Mac Ubuntu VM to flash over and update the Jetson. Ensure that the Jetson is not plugged in. Shutdown the Ubuntu Virtualbox VM on your Mac, making sure its actually powered off and not on standby. Touch some metal to ensure you don't have static charge on you, then connect the Jetson Micro-USB port and your laptop's USB-c port. Note that the Jetson can be quite sensitive to which USB cable you use for the Jetson Micro-USB to Mac USB-c port. Ultimately I had to use the black cable with a tiny NVIDIA green icon on it that came with the Jetson (not the white adapter one) for this all to work:<div><span style="background-color: white;"><br /></span></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-XGkvsY2wJBM/X_TYlz5o8QI/AAAAAAAAFxs/sUiEF-JTtq0qg8RkjRp0M9DfQoOU208RgCLcBGAsYHQ/jetson_illustration.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1600" data-original-width="1200" height="640" src="https://lh3.googleusercontent.com/-XGkvsY2wJBM/X_TYlz5o8QI/AAAAAAAAFxs/sUiEF-JTtq0qg8RkjRp0M9DfQoOU208RgCLcBGAsYHQ/w480-h640/jetson_illustration.png" width="480" /></a></div></div><div><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></div><div>Now, connect the power to the Jetson development board again, and get <span><span style="font-family: courier;"><span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;">Force Recovery Mode</span></span></span> going on the Jetson by working with the 4 onboard red buttons on the edge of the board. Here they are, annotated with their function:</div><div><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-xPi_0VhyA4E/X_TXYf5t5gI/AAAAAAAAFxc/Y_2_2v57Bg4bW3vrtG2AArKTimq3evJ_gCLcBGAsYHQ/IMG_2866.jpg" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="2048" data-original-width="1536" height="640" src="https://lh3.googleusercontent.com/-xPi_0VhyA4E/X_TXYf5t5gI/AAAAAAAAFxc/Y_2_2v57Bg4bW3vrtG2AArKTimq3evJ_gCLcBGAsYHQ/w480-h640/IMG_2866.jpg" width="480" /></a></div></div><div><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></div><div><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></div><div><br /></div><div>Follow these instructions to get <span><span style="font-family: courier;"><span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;">Force Recovery Mode</span></span></span> going:</div><div><ul style="text-align: left;"><li>Press and release the <span><span style="font-family: courier;"><span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;">Power Switch</span></span></span></li><li>Hold down the <span><span style="font-family: courier;"><span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;">Recovery (REC) Switch</span></span></span></li><li>With <span><span style="font-family: courier;"><span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;">REC</span></span></span> held down, press and release the <span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;"><span style="font-family: courier;">Reset Switch</span></span></li><li>Hold <span><span style="font-family: courier;"><span style="background-color: #cccccc; font-size: 11pt; white-space: pre-wrap;">REC</span></span></span> for 2 more seconds, then release</li></ul>Once the Jetson starts up, you want to make sure that the NVIDIA Jetson board is actually connecting and registering itself with the physical Mac laptop before we even involve the Mac Ubuntu VM. On your Mac open a Terminal window and enter:<div><br /></div></div><div><span style="background-color: #cccccc; font-size: 14.6667px; white-space: pre-wrap;"><span style="font-family: courier;">ioreg -p IOUSB -w0 | sed 's/[^o]*o //; s/@.*$//' | grep -v '^Root.*'</span></span></div><div><span style="font-family: Arial; font-size: 14.6667px; white-space: pre-wrap;"><br /></span></div><div>This should show <span style="font-family: courier;"><span style="background-color: #cccccc;">NVIDIA Corp. APX</span></span> or just <span style="font-family: courier;"><span style="background-color: #cccccc;">APX</span></span>.</div><div><br /></div><div>Now, we will start up the Ubuntu VM on the Mac laptop, but we need to make sure it can actually get access to the USB-connected Jetson. Before you start up the Ubuntu VM, for that VM in Virtualbox go to Settings -&gt; Ports -&gt; USB, and click <span style="font-family: courier;"><span style="background-color: #cccccc;">Add new USB filters with all ....</span></span>, then add <span style="font-family: courier;"><span style="background-color: #cccccc;">NVIDIA Corp. APX</span></span>. Then, go to the VM, click the bottom right corner button with the shape of a USB port and select <span style="font-family: courier;"><span style="background-color: #cccccc;">NVIDIA Corp. APX</span></span>. You <i>won't</i> need to remove other USB devices from your Mac; I was able to get everything working fine with all my other USB devices (webcam, keyboard, etc.) plugged in.</div><div><br /></div><div>Once the Ubuntu VM is started again on the host Mac, double click the NVIDIA SDK Manager install again. Note that if you get a warning like the following you can just ignore it; just press <span style="font-family: courier;"><span style="background-color: #cccccc;">Yes</span></span>:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><span style="border: none; display: inline-block; height: 284px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 535px;"><img height="213" src="https://lh4.googleusercontent.com/r9TNou1jClIvGlW2CN76hnnSCPVaLwtAD1W_XjkcNDzktaD0lKHVWkU59O7QyzDUPq3TELmP3oQ0LVj3oi24wgRCkTp3uOp2zd3oHJzSB4tqdKYrj7bneeF8FwC5jbLSpX5pUDij=w400-h213" style="margin-left: 0px; margin-top: 0px;" width="400" /></span></div>Once you continue with the NVIDIA SDK Manager install, you should now see that the Jetson TX2 is detected. At some point you will be prompted on the Jetson developer board's external monitor to continue installation. Go through that full flow on the Jetson itself until you get to the Jetson's login screen, then go back to your Mac's Ubuntu host VM.<div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both;">You will have to go back to the Mac's Ubuntu VM's settings, select&nbsp;<span style="font-family: courier;"><span style="background-color: #cccccc;">Ports</span></span>, then click the <span style="font-family: courier;"><span style="background-color: #cccccc;">+</span></span> button to add a new USB device. You will see <span style="font-family: courier;"><span style="background-color: #cccccc;">NVIDIA Linux for Tegra</span></span> now -- select that, then press <span style="font-family: courier;"><span style="background-color: #cccccc;">OK</span></span>.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><span id="docs-internal-guid-51912365-7fff-daf2-3b0b-c569878b57db"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span><span style="border: none; display: inline-block; height: 557px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="571" src="https://lh5.googleusercontent.com/hH5YsvItg9Tfs9tYWP_bvszTE9JCoKrUFkCQ5raQqx2q9VYrTeb3TtTYfVOoU6ShhKoufsorEIfq5HJXViiK9Vit5nP0sOK9WgxsoCSiIXwe9hysWciA9fdcatIwAiNNVjoj8GHW=w640-h571" style="margin-left: 0px; margin-top: 0px;" width="640" /></span></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div>Unfortunately this new USB device <i>won't</i> be seen by the host Ubuntu VM, so you will have to press the <span style="font-family: courier;"><span style="background-color: #cccccc;">Skip</span></span> button in the SDK Install Manager, then click <span style="font-family: courier;"><span style="background-color: #cccccc;">Finish and Exit</span></span>, then shut down the Mac's Ubuntu VM via the <span style="font-family: courier;"><span style="background-color: #cccccc;">Power Off</span></span> button again. Also shut down the Jetson itself by selecting the <span style="font-family: courier;"><span style="background-color: #cccccc;">Shutdown</span></span> option in its OS UI.</div><div><br /></div><div>We will have to repeat the entire flow above, but we have now made the Mac's Ubuntu VM aware of the second USB driver (<span style="font-family: courier;"><span style="background-color: #cccccc;">NVIDIA Linux for Tegra</span></span>). Put the device back into <span style="font-family: courier;"><span style="background-color: #cccccc;">Force Recovery Mode</span></span> by following the earlier instructions about which red buttons to push on the Jetson development board, start the Mac's Ubuntu VM again, and click back into the NVIDIA SDK Manager to restart the whole install flow. Once you do the device OS setup on the Jetson and get its sign in window, you should now be able to go back to your Mac host Ubuntu VM, enter your credentials, and the host VM will "push over" more of the Jetpack install.</div><div><br /></div><div><img height="402" src="https://lh3.googleusercontent.com/tD6sN09vUO69GFykDlC6JXREj6nV6wVUE88BA4rt9zEKpedOVXU-eWcpINE-LTnx-FpGSmTdNW6AdFkvJAUO0001xPmFU68lQxjP6BOvEqAxMjEgkq1ao2vU3gcJGUr4ypMUHvRV=w640-h402" style="margin-left: 0px; margin-top: 0px; text-align: center;" width="640" /></div><div><br /></div>It will take awhile to install everything.<div><br /><div class="separator" style="clear: both;"><span style="border: none; display: inline-block; height: 221px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="227" src="https://lh4.googleusercontent.com/5rnLF0eiInehswPvP-ofcs3Zr-2gUbZiZzZtvxR-JwIzV8xdQQXEdD6npugmIA_F_oIaZE-sug5Fysu4XdYCFagw1EjiD1o9h4ws6xymG11A9lFK8XoNeWiol8DiujHc4HBSmHWu=w640-h227" style="margin-left: 0px; margin-top: 0px;" width="640" /></span></div><div><span id="docs-internal-guid-8bbb3914-7fff-25d3-a2ae-ba7dab881b85"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span></div><div><br /></div>Once its installed, go back to your Jetson and confirm things installed correctly by opening a terminal on the external Jetson's monitor and run the following:<div><br /></div><div><span style="background-color: #cccccc; font-family: courier;"># Validate NVCC<br />nvcc -V<br /><br /># Validate CUDA<br />ls /usr/local | grep cuda</span></div><div><br /></div><div>Back on the Mac's Ubuntu VM, pin the Terminal application to make it quicker to access -- click the dots in the lower left corner of Ubuntu's UI, search for Terminal, then start it. Once it's running you will see the Terminal app in the Ubuntu app switcher. Command-Click with your Mac mouse on the Terminal icon to simulate a right click and select <span style="font-family: courier;"><span style="background-color: #cccccc;">Add to Favorites</span></span> to pin it.</div><div><br /></div><div>Turn off the screenlock in the Ubuntu install on your Jetson or else it will get annoying; do this by going to <span style="font-family: courier;"><span style="background-color: #cccccc;">Settings</span></span> then <span style="font-family: courier;"><span style="background-color: #cccccc;">Brightness &amp; Lock</span></span> then turning it off.</div><div><br /></div><div>On your host Mac's Ubuntu VM, you will see an <span style="font-family: courier;"><span style="background-color: #cccccc;">L4T-README</span></span> directory that is exported via the Jetson USB connection. Double-click this to see some README files that are somewhat useful to read and know about:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><span style="border: none; display: inline-block; height: 400px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="410" src="https://lh4.googleusercontent.com/rlyg29zbmVcY7oqzwnsFbJiFVPkArzQTV4dse3eTP6V5rBubCbp20rcRhVQUrrnnPlFP-OoBcwvocOyswYlPkbfe18dTNnSpGYW_5uFVCpD6RpMaggKAhp2LTpeUcWOUxcWwPdTr=w640-h410" style="margin-left: 0px; margin-top: 0px;" width="640" /></span></div><div><span id="docs-internal-guid-d3041079-7fff-d9a8-6af8-5509a2a58ff7"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span></div><div><br /></div><h1 style="text-align: left;">Getting a Development Workflow Working with SSH and VNC</h1><div>The best dev workflow is to SSH directly from your Mac laptop into the Jetson itself, with the Jetson connected via an Ethernet connection to your Internet router/switch. This means you won't have to use either the Ubuntu VM on the Mac or a direct USB connection to the Jetson development board.<br /><br />Find the local IP address of your Jetson by using the external monitor and keyboard of the Jetson, open a terminal, then use the <span style="font-family: courier;"><span style="background-color: #cccccc;">ifconfig</span></span> command. Search for <span style="font-family: courier;"><span style="background-color: #cccccc;">eth0</span></span> in the output, then get the IP address from that. Then, go back to your Mac (not the Mac's Ubuntu VM), open a terminal, then SSH to this IP address. You will be prompted to enter your Jetson password; you will now be dumped into an SSH shell on your Jetson. Congrats!</div><div><br /></div>To prevent having to re-type your password constantly when you SSH or SCP from your Mac to the Jetson board, type the following via the Jetson's external monitor and keyboard into a terminal:<div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">mkdir -p ~/.ssh</span></div><div><br /></div>On your Mac laptop outside of the Ubuntu VM, do the following:<br /><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">export USERNAME=&lt;change to your Jetson's username&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">export JETSON_IP=&lt;change to eth0 IP address&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">scp ~/.ssh/id_rsa.pub $USERNAME@$JETSON_IP:./.ssh/authorized_keys</span></div><div><br /></div><div>At this point you can power down the Jetson, power down the Mac's Ubuntu VM, unplug the Jetson to ensure no static electricity hits the development board, touch some metal to discharge static electricity, then unplug the Jetson's USB from your Mac laptop. Now, power the Jetson back up and just use the <span style="font-family: courier;"><span style="background-color: #cccccc;">eth0</span></span> IP address to SSH from your Mac into the Jetson itself. You don't even have to sign into the external monitor on the Jetson itself. Example SSH command:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">export USERNAME=&lt;change to your Jetson's username&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">export JETSON_IP=&lt;change to eth0 IP address&gt;</span></div></div><div><span style="background-color: #cccccc; font-family: courier;">ssh $USERNAME $JETSON_IP</span></div><div><br /></div>In your SSH session on your Mac, install a few useful things:</div><div><br /><div><span style="background-color: #cccccc; font-family: courier;">sudo apt install -y tmux htop curl vim less</span></div><div><br /></div><div>You will want to change the default Ubuntu <span style="background-color: #cccccc; font-family: courier;">sudo</span> timeout, which is very short and which can get quite annoying in your Jetson development. To do so, run:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">sudo visudo</span></div><div><br /></div><div>This will open a configuration file that has the following existing line:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">Defaults&nbsp; &nbsp; &nbsp; &nbsp; env_reset</span></div></div><div><br /></div><div>Change this to be the following:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">Defaults&nbsp; &nbsp; &nbsp; &nbsp; env_reset, timestamp_timeout=XX</span></div></div><div><br /></div><div>Where <span style="background-color: #cccccc; font-family: courier;">XX</span> is the timeout in minutes. I've set that to <span style="background-color: #cccccc; font-family: courier;">300</span> (or 5 hours) in my own sudo settings.</div><div><br /></div><div>By default <span style="background-color: #cccccc; font-family: courier;">python3</span> is installed; make sure <span style="background-color: #cccccc; font-family: courier;">pip3</span> is installed as well:</div><div><br /></div><div><span style="background-color: #cccccc;"><span style="font-family: courier;">sudo apt install python3-pip -y</span></span></div><div><br /></div><div>You will have to restart your Jetson after installing this:</div><div><br /></div><div><span style="background-color: #cccccc;"><span style="font-family: courier;">sudo shutdown -r now</span></span></div><div><br /></div><div>You can use the <span style="font-family: courier;"><span style="background-color: #cccccc;">tegrastats</span></span> command when SSHed into the Jetson to get CPU/GPU/thermal/power details (the Jetson does not have the common <span style="font-family: courier;"><span style="background-color: #cccccc;">nvidia-smi</span></span> command, using <span style="font-family: courier;"><span style="background-color: #cccccc;">tegrastats</span></span> instead).</div><div><br /></div><div>A useful set of commands to get higher level Jetson stats is to&nbsp;<a href="https://pypi.org/project/jetson-stats/">install the jetson-stats package</a>:</div><div><br /></div><div><div><span style="background-color: #cccccc;"><span style="font-family: courier;">sudo -H pip3 install -U jetson-stats</span></span></div></div><div><br /></div><div>Some example commands you can run after installing this (note that you do <i>not</i>&nbsp;have to use sudo for these): <span style="background-color: #cccccc;"><span style="font-family: courier;">jtop</span></span>, <span style="background-color: #cccccc; font-family: courier;">jetson_config</span>, <span style="background-color: #cccccc; font-family: courier;">jetson_release</span>.</div><div><br /></div>To setup VNC between your Mac host and the Jetson, do the following. First, SSH into your Jetson and run the following:<div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt update</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt install vino</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Configure auto-login with GDM3. Open “/etc/gdm3/custom.conf” and uncomment or change the following lines:</span></div><div><span style="background-color: #cccccc; font-family: courier;">AutomaticLoginEnable=true</span></div><div><span style="background-color: #cccccc; font-family: courier;">AutomaticLogin=nvidia</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Set up the VNC server for the user you wish to log in as. When you run "vncserver" for the first time on your Jetson, it will ask you to set a password. Just use the same password as your primary Jetson account for simplicity. Select NO when prompted if you want a view only password.</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Enable the VNC server to start each time you log in</span></div><div><span style="background-color: #cccccc; font-family: courier;">mkdir -p ~/.config/autostart</span></div><div><span style="background-color: #cccccc; font-family: courier;">cp /usr/share/applications/vino-server.desktop ~/.config/autostart</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Configure the VNC server</span></div><div><span style="background-color: #cccccc; font-family: courier;">gsettings set org.gnome.Vino prompt-enabled false</span></div><div><span style="background-color: #cccccc; font-family: courier;">gsettings set org.gnome.Vino require-encryption false</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Replace thepassword with your desired password, the same as your Jetson account</span></div><div><span style="background-color: #cccccc; font-family: courier;">gsettings set org.gnome.Vino authentication-methods "['vnc']"</span></div><div><span style="background-color: #cccccc; font-family: courier;">gsettings set org.gnome.Vino vnc-password $(echo -n 'thepassword'|base64)</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure we get close/minimize/maximize settings, which you will get when you hover over the window titlebar</span></div><div><span style="background-color: #cccccc; font-family: courier;">gsettings set org.gnome.desktop.wm.preferences button-layout "close,minimize,maximize:"</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Reboot the system so the settings take effect</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo reboot</span></div></div><div><br /></div>On your Mac laptop, start an SSH proxy for VNC:<br /><div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">export USERNAME=&lt;change to your Jetson's username&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">export JETSON_IP=&lt;change to eth0 IP address&gt;</span></div></div><div><span style="background-color: #cccccc; font-family: courier;">ssh -L 5900:127.0.0.1:5900 -C -N -l $USERNAME $JETSON_IP</span></div><div><br /></div><div>Now use a Mac VNC client, such as <a href="https://www.realvnc.com/en/connect/download/viewer/macos/">VNC Viewer,</a>&nbsp;to sign into the eth0 IP address at the 5900 port.</div><div><br /></div><div>Inside of a VNC session on your Jetson, if you have an Ubuntu Terminal open, you can open a new terminal tab by pressing <span style="background-color: #cccccc; font-family: courier;">Shift-Control-T</span>.</div><div><br /></div><div>Note that at this point you can choose to boot your Jetson TX2 in the future <i>without</i>&nbsp;a keyboard or external monitor. It will boot up as normal -- wait a few minutes then you can SSH in. Surprisingly, even VNC will work fine without an external monitor hooked up!</div><h1 style="text-align: left;">Use Jetson From Outside Local Network</h1><div>The flow above only allows you to SSH into your Jetson from within your immediate local area network, since the <span style="background-color: #cccccc; font-family: courier;">eth0</span> address you've been using so far is a private address. If you want to sign in and develop from outside your network, or allow colleagues to also develop on your Jetson, you need to securely make it possible to access the Jetson from outside your network. Please be conscious of the security ramifications of doing this.</div><div><br /></div><div>You will need to use port forwarding to do this. Before you setup the port forwarding, you will need to ensure you have a stable external IP address. If you don't, you will have to use a dynamic DNS service. I used <a href="http://no-ip.com">no-ip.com</a> using their free option and will document how to set it up.</div><div><br /></div><div>In a VNC session on the Jetson, use a web browser to navigate to <a href="https://noip.com/download">https://noip.com/download</a> to download their dynamic DNS updater for Linux. This will compile and install a <span style="background-color: #cccccc; font-family: courier;">noip2</span> command line tool:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;">tar -zxvf noip-duc-linux.tar.gz</span></div><div><span style="background-color: #cccccc; font-family: courier;">cd noip-duc-linux</span></div><div><span style="background-color: #cccccc; font-family: courier;">make</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo make install<br /></span></div></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure the command is present; if not copy it there.</span></div><div><span style="background-color: #cccccc; font-family: courier;">ls /usr/local/bin/noip2</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo chmod a+x /usr/local/bin/noip2</span></div><div><br /></div><div>To make sure noip2 starts on system startup, get a systemd script running. Run the following commands in your SSH session on the Jetson:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;"># Generate configuration file</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo /usr/local/bin/noip2 -C</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Add the following systemd script</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo cat &gt;&gt; /etc/systemd/system/noip2.service &lt;&lt;EOL</span></div><div><div><span style="background-color: #cccccc; font-family: courier;">[Unit]</span></div><div><span style="background-color: #cccccc; font-family: courier;">Description=No-ip.com dynamic IP address updater</span></div><div><span style="background-color: #cccccc; font-family: courier;">After=network.target</span></div><div><span style="background-color: #cccccc; font-family: courier;">After=syslog.target</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">[Install]</span></div><div><span style="background-color: #cccccc; font-family: courier;">WantedBy=multi-user.target</span></div><div><span style="background-color: #cccccc; font-family: courier;">Alias=noip.service</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">[Service]</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Start main service</span></div><div><span style="background-color: #cccccc; font-family: courier;">ExecStart=/usr/local/bin/noip2</span></div><div><span style="background-color: #cccccc; font-family: courier;">Restart=always</span></div><div><span style="background-color: #cccccc; font-family: courier;">Type=forking</span></div></div><div><span style="background-color: #cccccc; font-family: courier;">EOL</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Restart and enable systemd services</span></div><div><div><span style="background-color: #cccccc; font-family: courier;">sudo systemctl daemon-reload</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo systemctl enable noip2</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo systemctl start noip2</span></div></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure everything is running fine</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo systemctl status noip2&nbsp;</span></div><div><br /></div><div>Now we will setup port forwarding for SSH, to "poke" a hole in the external router to allow inbound connections to the Jetson. You will have to look at details for your particular home router on how to this. For my own router, a LinkSys router, I used the&nbsp;<a href="https://www.linksys.com/ru/support-article?articleNum=138535">following instructions</a>. I went to&nbsp;<span style="background-color: #cccccc; font-family: courier;">192.168.1.1</span> to get the router admin screen. Note that its important that you <b>DO NOT</b> use the default SSH port 22 as your external port for security reasons, but choose a random port. In the example admin screenshots below private details, such as the external SSH port chosen, have been fuzzed out.</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-QaPw7XuXT9U/X_UNt97ca0I/AAAAAAAAFys/px9bPPugnPkaa998AN6avGxj9QXNp6tCwCLcBGAsYHQ/admin1.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="162" data-original-width="624" height="166" src="https://lh3.googleusercontent.com/-QaPw7XuXT9U/X_UNt97ca0I/AAAAAAAAFys/px9bPPugnPkaa998AN6avGxj9QXNp6tCwCLcBGAsYHQ/w640-h166/admin1.png" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-wafhP3Y2sHY/X_UOABO3zNI/AAAAAAAAFy8/FMSygy0q6bg2KQx6NrjSWIyV35n14bXMwCLcBGAsYHQ/admin2.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="171" data-original-width="626" height="174" src="https://lh3.googleusercontent.com/-wafhP3Y2sHY/X_UOABO3zNI/AAAAAAAAFy8/FMSygy0q6bg2KQx6NrjSWIyV35n14bXMwCLcBGAsYHQ/w640-h174/admin2.png" width="640" /></a></div><br /><br /></div>Now to SSH from a computer outside of your local network:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">export USERNAME=&lt;Jetson user account name&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">export DYN_HOST_NAME=&lt;Dynamic DNS name from noip.com, such as foobar.hopto.org&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">export EXTERNAL_SSH_PORT=&lt;The random external SSH port you chose, such as 301&gt;</span></div><div><span style="background-color: #cccccc; font-family: courier;">ssh $DYN_HOST_NAME -p $EXTERNAL_SSH_PORT</span></div><div><br /></div><div>Here's how you can use SCP to copy something over:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">scp -P $EXTERNAL_SSH_PORT $USERNAME:/some/path/file.txt .</span></div><div><br /></div><div>Example usage of RSYNC:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">rsync -Pavz -e "ssh -p $EXTERNAL_SSH_PORT" . $USERNAME@$DYN_HOST_NAME:~&nbsp;--exclude .git</span></div><div><br /></div><div>In order to now VNC over this external SSH connection, run a local SSH proxy for the VNC session:</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;">ssh -L 5900:127.0.0.1:5900 -C -N -l $USERNAME -p $EXTERNAL_SSH_PORT $DYN_HOST_NAME</span></div><h1 style="text-align: left;"><span style="font-size: small;"><span style="font-weight: 400;">Use a VNC client on your Mac to connect to <span style="background-color: #cccccc;"><span style="font-family: courier;">localhost:5900</span></span>.</span></span></h1><h1 style="text-align: left;">Mount External SSD</h1><div><br /></div><div>The Jetson TX2 has a 32GB eMMC flash storage; however, this can get full very quickly, especially if you are experimenting with multiple Docker containers being available or having large amounts of machine learning oriented data.</div><div><br /></div><div>This section documents how to install an NVMe SSD drive onto the Jetson board. The Jetson has an M.2 connector, which is normally used for attached SSD drives, but the Jetson's M.2 connector is a type "Key E" port and is therefore not compatible with SSD drives. Instead, we will use the Jetson's on-board PCIe Gen2 x4 connector with an M.2 adapter. PCIe Gen2 x4 has a max IO of 2 GB per second, so it can keep up with NVMe drives (be careful not to purchase an NVMe drive that is <i>faster</i>&nbsp;than 2 GB/s, or else you will just be wasting money on a drive that is faster than the PCIe Gen2 x4 connector on the Jetson can accommodate).</div><div><br /></div><div>I purchased a PCIe to M.2 adapter at a <a href="https://www.centralcomputer.com/">local computer store</a>; it was only $12.95:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><span style="border: none; display: inline-block; height: 832px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="832" src="https://lh3.googleusercontent.com/q2gygficThBfyhXiV0s28wCIk2wgniby1JXjbBEv_gUfutWVIYLAush_NgGvz2Asi8fpbGt9nDTzi_yx7oezq7iMN1yVg_3JU5Qm4TzBd0hZaX2HbDc7agA2K7Gi64BIxv35D8sX" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></div><div><span id="docs-internal-guid-4a10791a-7fff-4bd3-585c-d88c621e30c7"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span></div><div><br /></div><div>I also purchased a <a href="https://www.samsung.com/semiconductor/minisite/ssd/product/consumer/970evoplus/">1 TB NVMe SSD Samsung drive</a> to plug into the adapter:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><span style="border: none; display: inline-block; height: 832px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="832" src="https://lh6.googleusercontent.com/UkLlVTbPmvWoTH1eK82bIi0-UK6bcBvQOU2BJBczGoFiINjuUOSY80o7lW8LyjpVno-OtbXDC8XJyYjPxURW9kNwxLoiHO_EPjQU8vgfjB5-UuNBaAzwT8yeisMJnt_tXOqCKcRA" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></div><div><span id="docs-internal-guid-9a923b22-7fff-e2f8-509d-dbc53c5c87b2"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span></div><div><br /></div><div>Here are both put together and mounted onto the Jetson board (remember to unplug the Jetson and ground any static electricity on yourself before mounting these items!):</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><span style="border: none; display: inline-block; height: 468px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="468" src="https://lh5.googleusercontent.com/yFF-HwMqj6Ln2Oa-4JQgh18am6GYqLny6JOMhlpX-TwO-YoEVVspd5WkWcRqWRV1Sew_y1k0BA-ZwGZrwpmeIRnRAQAHFzozrGJSB7GnAiLHHywbOCEzVR3HGCz1rsWojn33mcHo" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></div><div><span id="docs-internal-guid-e330bb2a-7fff-9664-2edd-9167078df340"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span></div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><span style="border: none; display: inline-block; height: 832px; margin-left: 1em; margin-right: 1em; overflow: hidden; width: 624px;"><img height="832" src="https://lh6.googleusercontent.com/0hXnkYK8T1g05vvEEyhZjR7BG7zVd1UxDCa4ujr9bS6WZlRL9jx6tlD7n4pHI8CFvroBUuXNzWLvulMMfTsjI3c6qYTB9eF2RDSYvsW2Ky0gNJdZwP8ceUznzfltKUDLbskxvzzT" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></div><div><span id="docs-internal-guid-2b002990-7fff-bb2a-c927-2fef48984945"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"></span></span></div><div><br /></div><div>Once the machine is restarted, SSH back into the Jetson and compile and mount the SSD drive:</div><div><br /></div><div><div><span style="background-color: #cccccc; font-family: courier;"># SSD should show up as nvme0n1 on this command:</span></div><div><span style="background-color: #cccccc; font-family: courier;">lsblk</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Format the drive</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo mkfs.ext4 -F /dev/nvme0n1</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo mkdir -p /mnt/disks/ssd</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo mount /dev/nvme0n1 /mnt/disks/ssd</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure the new drive shows up here:</span></div><div><span style="background-color: #cccccc; font-family: courier;">df --h</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Let everyone access it</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo chmod a+rxw /mnt/disks/ssd</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Automount the disk on startup</span></div><div><span style="background-color: #cccccc; font-family: courier;">echo UUID=`sudo blkid -s UUID -o value /dev/nvme0n1` /mnt/disks/ssd ext4 discard,defaults,nofail 0 2 | sudo tee -a /etc/fstab</span></div><div><span style="background-color: #cccccc; font-family: courier;">cat /etc/fstab # Make sure the new line shows up correctly</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Benchmark to see how the IO is</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt install hdparm</span></div><div><span style="background-color: #cccccc; font-family: courier;">for run in {1..3}; do sudo hdparm -Tt /dev/nvme0n1; done</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Example output, pretty good IO for reads:</span></div><div><span style="background-color: #cccccc; font-family: courier;">/dev/nvme0n1:</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp;Timing cached reads:&nbsp; &nbsp;3898 MB in&nbsp; 2.00 seconds = 1950.98 MB/sec</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp;Timing buffered disk reads: 2834 MB in&nbsp; 3.00 seconds = 944.28 MB/sec</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">/dev/nvme0n1:</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp;Timing cached reads:&nbsp; &nbsp;3730 MB in&nbsp; 2.00 seconds = 1867.07 MB/sec</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp;Timing buffered disk reads: 3050 MB in&nbsp; 3.00 seconds = 1016.14 MB/sec</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;">/dev/nvme0n1:</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp;Timing cached reads:&nbsp; &nbsp;3892 MB in&nbsp; 2.00 seconds = 1947.68 MB/sec</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp;Timing buffered disk reads: 2856 MB in&nbsp; 3.00 seconds = 951.78 MB/sec</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Do daily trimming of the SSD (TRIM garbage collection) to keep performance tip-top</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo mkdir -v /etc/systemd/system/fstrim.timer.d</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo touch /etc/systemd/system/fstrim.timer.d/override.conf</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo vim /etc/systemd/system/fstrim.timer.d/override.conf</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Add the following into the document (note the double entries for OnCalendar is intentional!)</span></div><div><span style="background-color: #cccccc; font-family: courier;">[Timer]</span></div><div><span style="background-color: #cccccc; font-family: courier;">OnCalendar=</span></div><div><span style="background-color: #cccccc; font-family: courier;">OnCalendar=daily</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Save and reboot. Confirm TRIM is correct, ensuring the new override is at the bottom:</span></div><div><span style="background-color: #cccccc; font-family: courier;">systemctl cat fstrim.timer</span></div><div><span style="background-color: #cccccc; font-family: courier;"># At any point you can run the following to ensure TRIMing actually has happened; probably</span></div><div><span style="background-color: #cccccc; font-family: courier;"># won't show anything the first time.</span></div><div><span style="background-color: #cccccc; font-family: courier;">journalctl | grep fstrim.service</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Force trim manually - might last for minutes, just be patient</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo fstrim -av</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Configure Docker to store all of its images and containers on the SSD</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker version # Make sure Docker reports its running</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo service docker stop # Now stop it</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo vim /etc/docker/daemon.json</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure this JSON file has the following new "data-root" property:</span></div><div><span style="background-color: #cccccc; font-family: courier;">{</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; "runtimes": {</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; "nvidia": {</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "path": "nvidia-container-runtime",</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "runtimeArgs": []</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; &nbsp; &nbsp; }</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; },</span></div><div><span style="background-color: #cccccc; font-family: courier;">&nbsp; &nbsp; "data-root": "/mnt/disks/ssd/docker"</span></div><div><span style="background-color: #cccccc; font-family: courier;">}</span></div><div><span style="background-color: #cccccc; font-family: courier;"># Save and exit</span></div><div><span style="background-color: #cccccc; font-family: courier;">mkdir -p /mnt/disks/ssd/docker</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Copy current data over to new path</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo rsync -aP /var/lib/docker /mnt/disks/ssd/docker</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Temporarily rename old location for sanity check</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo mv /var/lib/docker /var/lib/docker.old</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Restart docker</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo service docker start</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure the following two commands run without error</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker image ls</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker container ls</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Now you can remove the old data</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo rm -rf /var/lib/docker.old</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo docker image prune -a -f</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Move all home directories to SSD</span></div><div><span style="background-color: #cccccc; font-family: courier;">mkdir -p /mnt/disks/ssd/home</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo rsync -aXS /home/. /mnt/disks/ssd/home/.</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo mv /home /home_old</span></div><div><span style="background-color: #cccccc; font-family: courier;">ls /mnt/disks/ssd/home # Make sure everything is there</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo ln -s /mnt/disks/ssd/home /home</span></div><div><span style="background-color: #cccccc; font-family: courier;">ls -al / # Make sure the symbolic link is correct</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo rm -fr /home_old</span></div></div><h1 style="text-align: left;">Example Machine Learning Package Installation</h1><div>The Jetson has an ARM CPU, so you will have to compile most Python packages yourself for the ARM architecture since there are most likely no ARM pip wheels available. In the example commands below, we setup a virtualenv, Python 3.6, and compile and install several common geospatial and machine learning packages on the Jetson.</div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure various apt tools are available</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt-get install -y software-properties-common</span></div><div><br /></div><div><span style="background-color: #cccccc; font-family: courier;"># Setup a Python 3 virtualenv</span></div><div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt-get install python3-venv</span></div><div><span style="background-color: #cccccc; font-family: courier;">python3 -m venv test-arm-compile</span></div><div><span style="background-color: #cccccc; font-family: courier;">source test-arm-compile/bin/activate</span></div><div><span style="background-color: #cccccc; font-family: courier;">python --version # Should be 3.6.9</span></div></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure certain Python dev files are available for later compilation dependencies.</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt-get install python3.6-dev</span></div><div><span style="background-color: #cccccc; font-family: courier;">pip install wheel</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Make sure Cython is available, which is a Numpy dependency.</span></div><div><span style="background-color: #cccccc; font-family: courier;">pip install cython</span></div><div><span style="background-color: #cccccc; font-family: courier;">python -c "import cython" # Make sure this runs without error</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Install numpy; print out detailed info during compilation as it can take awhile.</span></div><div><span style="background-color: #cccccc; font-family: courier;">pip install numpy -v</span></div><div><span style="background-color: #cccccc; font-family: courier;">python -c "import numpy as np; print(np.array([2, 3, 4]))" # Make sure runs without error</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Install various geospatial ecosystem libraries</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo add-apt-repository ppa:ubuntugis/ppa</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt-get install gdal-bin libgdal-dev python-gdal --yes</span></div><div><span style="background-color: #cccccc; font-family: courier;">pip install rasterio -v</span></div><div><span style="background-color: #cccccc; font-family: courier;"><br /></span></div><div><span style="background-color: #cccccc; font-family: courier;"># Install OpenCV</span></div><div><span style="background-color: #cccccc; font-family: courier;">sudo apt install python3-opencv</span></div><div><span style="background-color: #cccccc; font-family: courier;">pip install --upgrade pip</span></div><div><span style="background-color: #cccccc; font-family: courier;">pip install opencv-python-headless -v</span></div><div><span style="background-color: #cccccc; font-family: courier;">python -c "import cv2; print(cv2.__version__)" # Should print out a version number, like 4.4.0</span></div><h1 style="text-align: left;">Appendix</h1><div><span style="background-color: white;"><br /></span></div>Thanks to the following guides which were used for some of the details creating this HOWTO:<br /><ul style="text-align: left;"><li><a href="https://www.blogger.com/u/1/#">https://github.com/KleinYuan/tx2-flash#2-install-virtual-box-and-extensions</a></li><li><a href="https://www.blogger.com/u/1/#">https://codingwithmanny.medium.com/installing-ubuntu-18-04-on-mac-os-with-virtualbox-ac3b39678602</a></li><li><a href="https://www.blogger.com/u/1/#">https://gist.github.com/NathanGiesbrecht/da6560f21e55178bcea7fdd9ca2e39b5</a></li></ul><div><span style="background-color: white;"><br /></span></div></div></div><div><span style="background-color: white;">Previous post:&nbsp;<a href="http://blog.codinginparadise.org/2021/01/howto-getting-productive-development.html">HOWTO: Setting Up NVIDIA Jetson TX2 &lt;-&gt; Mac Laptop Development Workflow</a></span></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-37637581194616479382020-02-10T12:40:00.000-08:002020-02-10T12:41:45.812-08:00Joining Planet!<div style="text-align: center;"></div><div style="text-align: center;"><a href="https://www.blogger.com/blogger.g?blogID=3191291" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><img alt="Image result for planet labs" src="https://thumbs-prod.si-cdn.com/cMGla9KLS2XC7hSwEVOPxw-3T98=/420x240/https://public-media.si-cdn.com/filer/d3/fb/d3fb245e-af07-4582-a75a-26920d28b16c/planet-deployment-flock.jpg" /></div><br />I'm excited to announce I'm joining the machine learning team at Planet! I will be applying machine and deep learning to remote sensing imagery of the Earth's surface. Planet has built a constellation of nanosats in Low Earth Orbit that image the entirety of the Earth daily to monitor changes and pinpoint trends. The ultimate goal is to enable a <a href="https://www.planet.com/pulse/queryable-earth-our-vision-for-making-daily-global-imagery-accessible-and-actionable/">Queryable Earth</a>, indexing physical change on Earth and making it searchable for all. I'm excited that I'll continue to get working at the intersection of space and machine learning. My first day is Tuesday, February 18th.<br /><br /><br /><div style="text-align: center;"></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com1tag:blogger.com,1999:blog-3191291.post-46558908333158491072018-10-10T09:57:00.001-07:002019-03-26T15:08:55.964-07:00Using machine learning to index text from billions of images<div dir="ltr" style="text-align: left;" trbidi="on"><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-wNJNUgCmnek/XJqg_b0t6BI/AAAAAAAAFIM/Bn6ZpdoBVMEtLMuOmu9HwRRawqGPkowBgCLcBGAs/s1600/autoocr-web.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="760" data-original-width="1300" height="233" src="https://1.bp.blogspot.com/-wNJNUgCmnek/XJqg_b0t6BI/AAAAAAAAFIM/Bn6ZpdoBVMEtLMuOmu9HwRRawqGPkowBgCLcBGAs/s400/autoocr-web.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"></div><br />The last year and a half I've led a project on the Dropbox Machine Learning team to take the <a href="http://blog.codinginparadise.org/2017/04/creating-modern-ocr-pipeline-using.html">computer vision/deep learning OCR pipeline</a> I built the year before and automatically run it and several other advanced machine learning models on billions of images daily in Dropbox to extract text for search. This turned out to be one of the largest computational projects Dropbox has ever done. The feature went live yesterday.<br /><i><br /></i><a href="https://blogs.dropbox.com/tech/2018/10/using-machine-learning-to-index-text-from-billions-of-images/">Dive into the technical details</a> of how we built this on the Dropbox technical blog.<br /><br /><br /></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-29746840153917759312017-04-12T11:00:00.000-07:002017-05-31T11:01:46.685-07:00Creating a Modern OCR Pipeline Using Computer Vision and Deep Learning<div dir="ltr" style="text-align: left;" trbidi="on"><span style="background-color: white; color: #5e676d; font-family: &quot;Open Sans&quot;, sans-serif; font-size: 14px;">In this post we will take you behind the scenes on how we built a state-of-the-art Optical Character Recognition (OCR) pipeline for our&nbsp;</span><a href="https://blogs.dropbox.com/dropbox/2016/06/new-dropbox-productivity-tools/" style="background-color: white; border: 0px; box-sizing: border-box; color: #007ee5; font-family: &quot;Open Sans&quot;, sans-serif; font-size: 14px; margin: 0px; outline: 0px; padding: 0px; text-decoration-line: none; vertical-align: baseline;">mobile document scanner</a><span style="background-color: white; color: #5e676d; font-family: &quot;Open Sans&quot;, sans-serif; font-size: 14px;">. We used computer vision and deep learning advances such as bi-directional Long Short Term Memory (LSTMs), Connectionist Temporal Classification (CTC), convolutional neural nets (CNNs), and more. In addition, we will also dive deep into what it took to actually make our OCR pipeline production-ready at Dropbox scale.</span><br /><span style="background-color: white; color: #5e676d; font-family: &quot;Open Sans&quot;, sans-serif; font-size: 14px;"><br /></span><span style="background-color: white; color: #5e676d; font-family: &quot;Open Sans&quot;, sans-serif; font-size: 14px;"><a href="https://blogs.dropbox.com/tech/2017/04/creating-a-modern-ocr-pipeline-using-computer-vision-and-deep-learning/">Read more...</a></span></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-25831529559255858132016-12-12T18:19:00.000-08:002016-12-12T18:19:04.102-08:00Notes from Neural Network NIPS 2016 conference<div dir="ltr" style="text-align: left;" trbidi="on">I've put up all my notes from the NIPS 2016 conference in Barcelona <a href="https://paper.dropbox.com/doc/Brad-Neubergs-NIPS-2016-Notes-XUFRdpNYyBhau0gWcybRo">here</a>. It includes lots of deep and reinforcement learning talks, paper sessions, and more. Hopefully these notes are useful for you!</div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-43049639712233251552016-01-22T14:37:00.001-08:002016-01-22T14:38:21.369-08:00Some Surprising Results of What it Would Take to Transport a Million People to Mars<div dir="ltr" style="text-align: left;" trbidi="on">Elon Musk of SpaceX has said <a href="http://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html">he wants to send one million people to Mars</a>. What would it take in terms of the transport side of the equation to actually make this happen?<br /><br />My good friend <a href="http://eetd.lbl.gov/people/jeffery-greenblatt">Jeffery Greenblatt </a>analyzes emerging technologies to answer questions exactly like this; for example, he recently published a <a href="http://www.nature.com/nclimate/journal/v5/n9/full/nclimate2685.html">scientific paper in Nature Climate Change</a> on the impact of autonomous cars on greenhouse-gas emissions.<br /><br />He has now turned his attention to the transport side of getting a large population to Mars:<br /><br /><blockquote class="tr_bq">"We perform the first comprehensive assessment of the energy, resource and infrastructure requirements of a large-scale human transport system between Earth and Mars. In it, we develop credible mass estimates for a system consisting of four appropriately-sized reusable spacecraft to move humans, and four additional types of reusable spacecraft for moving propellant (hydrogen/oxygen and methane/oxygen) from the Moon and Mars to in-orbit depots. Human consumables (air, water, food) and cargo mass estimates were included in the analysis. We base our estimates on public sources, and develop scenarios of infrastructure scale-up to achieve a Mars settlement size of 1 million people by the first half of the 22nd century. We do not examine the requirements of the Mars settlement itself."</blockquote><br />His result uncovers some surprising repercussions of such large-scale transport; see <a href="http://tinyurl.com/j7cweyx">his blog post</a>&nbsp;and the <a href="http://tinyurl.com/MarsTransportPaper">paper itself</a> for more details.</div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-62456622958616145122016-01-05T09:00:00.000-08:002016-01-05T12:01:31.432-08:00Cloudless: Open Source Deep Learning Pipeline for Orbital Satellite Data<div dir="ltr" style="text-align: left;" trbidi="on">I'm proud to announce the <a href="https://github.com/BradNeuberg/cloudless">1.0 release of Cloudless</a>, an open source computer vision pipeline for orbital satellite data, powered by data from Planet Labs and using deep learning under the covers. This blog post contains details and a technical report on the project.<br /><br /><a href="http://codinginparadise.org/ebooks/html/blog/introducing_cloudless.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com1tag:blogger.com,1999:blog-3191291.post-83192718989632111222015-12-13T14:19:00.002-08:002015-12-13T14:19:39.699-08:00Ten Deep Learning Trends I Saw at NIPS 2015<div dir="ltr" style="text-align: left;" trbidi="on">I attended the Neural Information Processing Systems (NIPS) 2015 conference this week in Montreal. It was an incredible experience, like drinking from a firehose of information. Special thanks to my employer Dropbox for sending me to the show (<a href="https://www.dropbox.com/jobs">we're hiring!</a>)<br />Here's some of the trends I noticed this week; note that they are biased towards deep and reinforcement learning as those are the tracks I attended at the conference:<br /><br /><a href="http://codinginparadise.org/ebooks/html/blog/ten_deep_learning_trends_at_nips_2015.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-38159782845721745422015-12-10T08:11:00.003-08:002015-12-10T08:11:52.219-08:00NIPS Day 3 Posters<div dir="ltr" style="text-align: left;" trbidi="on">These are the posters that caught my eye on the third day of NIPS. Note that they are HDR images, so zoom in using your computer to see more detail. Some of the posters below have extra images focusing on specific parts of the poster to give more visual detail.<br /><br /><a href="http://codinginparadise.org/ebooks/html/blog/nips_day_3_posters.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-55144593539689107082015-12-09T12:38:00.002-08:002015-12-09T12:38:09.917-08:00NIPS Day 3 Morning Sessions<div dir="ltr" style="text-align: left;" trbidi="on">These are my notes from the morning sessions of day 3 of NIPS.<br /><br /><a href="http://codinginparadise.org/ebooks/html/blog/nips_day_3_morning_sessions.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-56469657595892099782015-12-08T20:41:00.002-08:002015-12-08T20:41:40.908-08:00NIPS Day 1: Tutorials on Scaling Deep Learning, Probabilistic Programming, and Reinforcement Learning<div dir="ltr" style="text-align: left;" trbidi="on"><div data-uuid="c20058eaec2548639a8566177af65038">I'm at my first annual <a href="https://nips.cc/">NIPS conference</a> this year in Montreal, the annual pow-wow for machine learning and deep learning in particular.</div><div data-uuid="c20058eaec2548639a8566177af65038"><br /></div><div data-uuid="185428f0b2b6479b8b6cc5d8da016c20">Monday, the first day, had several multi-hour in-depth tutorials from literally the folks that wrote the textbooks in these areas. I attended sessions on scaling deep learning via TensorFlow, presented by Google folks like Jeff Dean; a deep dive into probabilistic programming (being able to describe a statistical system and allow an inference engine to do the hard work of building a model from it); and an introduction to reinforcement learning (using a scalar reward signal to automatically discover the optimal policy for a behavior).</div><div data-uuid="185428f0b2b6479b8b6cc5d8da016c20"><br /></div><div data-uuid="185428f0b2b6479b8b6cc5d8da016c20"><a href="http://codinginparadise.org/ebooks/html/blog/nips_day_1.html">Read more...</a></div></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-88171699711858712992015-10-12T18:34:00.002-07:002015-10-12T18:34:37.333-07:00Intelligence Augmentation and the Myth of the “Golden Lost Age”<div dir="ltr" style="text-align: left;" trbidi="on"><br /><div><div style="text-align: center;"><img alt="" src="http://codinginparadise.org/ebooks/img/chapter03/255-1968-practice-demo.jpg" data-uuid="3f9895097bc545a5aa49577213f5825d" width="75%" /></div></div><br /><div data-uuid="df4fd65d9b794f3681162b49c633c3eb"><a href="https://vanemden.wordpress.com/2015/10/09/ia-or-ai-2/">Maarten van Emden has a great piece</a> on Douglas Engelbart and Intelligence Augmentation (IA) versus Artificial Intelligence (AI). First go read that piece then come back here :) This post follows up a bit and comments on some of what Maarten says.<br /><br /></div><div data-uuid="c7c9e8e852864446a77cab0baa667761">Most people don't realize that much of the computer industry has been about the struggle between two worlds, IA and AI. <a href="https://twitter.com/markoff">John Markoff</a> has chronicled the tension between both of these well with his books <a href="http://www.amazon.com/What-Dormouse-Said-Counterculture-Personal/dp/0143036769">What the Dormouse Said: How the Sixties Counterculture Shaped the Personal Computer Industry</a> and his newest book, <a href="http://www.amazon.com/Machines-Loving-Grace-Common-Between/dp/0062266683">Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots</a>.</div><div data-uuid="c7c9e8e852864446a77cab0baa667761"><br /></div><div data-uuid="c7c9e8e852864446a77cab0baa667761"><a href="http://codinginparadise.org/ebooks/html/blog/ia_vs__ai.html">Read more...</a></div></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-11511604708689071132015-09-21T22:42:00.001-07:002015-09-21T22:47:05.107-07:00Personal Photos Model Using Deep Learning<div dir="ltr" style="text-align: left;" trbidi="on"><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://codinginparadise.org/ebooks/img/chapter03/photo-256887_1280.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://codinginparadise.org/ebooks/img/chapter03/photo-256887_1280.jpg" height="211" width="320" /></a></div><div data-uuid="e48b2158cac0406aa42a26ceb7e893db"><br /></div><div data-uuid="e48b2158cac0406aa42a26ceb7e893db">I've spent the last year going deep into deep learning, machine vision, and autonomous systems. I've completed two Coursera courses, <a href="https://www.coursera.org/learn/machine-learning">Andrew Ng's Introduction to Machine Learning</a> and <a href="https://www.coursera.org/course/neuralnets">Geoffrey Hinton's Neural Networks</a> course. I've also been trying to stay on top of the (crazy flood) of deep learning literature via <a href="http://arxiv.org/">Arxiv.org</a>. I've been doing this part time in the evenings and weekends while I work at <a href="https://www.dropbox.com/recents">Dropbox</a>, generally via the colearning study groups I host weekly.</div><div data-uuid="e48b2158cac0406aa42a26ceb7e893db"><br /></div><div data-uuid="66097a5d97b1425086f3dd68146a9ae1">A few months ago I wanted to ground the theoretical knowledge I've been gaining with an actual coding project before taking more courses. Nothing like trying to actually apply what you've been learning to humble you :)</div><div data-uuid="66097a5d97b1425086f3dd68146a9ae1"><br /></div><div data-uuid="1bb5dce94c9740a6b3ca106806d3fb97">The attempt was to allow someone to train a neural network over their personal photo collection in order to do face detection on the people in those photos. They could then organize the photos by those people into automatic groups.</div><div data-uuid="1bb5dce94c9740a6b3ca106806d3fb97"><br /></div><div data-uuid="7a0b6fd7c19d4455a7a2d09ef8cdcbb7"><a href="http://codinginparadise.org/ebooks/html/blog/personal_photos_model_using_deep_learning.html">Read more...</a></div></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-78172690243881148232015-08-24T12:28:00.002-07:002015-08-24T12:29:49.564-07:00Where Will Work Be in 10 Years?<div dir="ltr" style="text-align: left;" trbidi="on">Wired.uk recently asked me to write a short 100 word answer on what our workplaces will look like in ten years. One hundred words is pretty small; here's my full response:<br /><br /><blockquote class="tr_bq">It's not about what work <i>will</i> be in 10 years, but what it <i>should</i>&nbsp;be. Service work and the Uber-esque sharing economy support middle-class lifestyles via progressive legislation; these jobs can't be outsourced and support a real living wage. Silicon Valley makes its service workers (janitors, etc.) first class employees. Computers and networks eliminate and subsume middle management; collaborative tools allow teams to self-assemble, communicate, and work nimbly. AI doesn't replace people, but augments them. We learn to harness both analytics <i>and</i> empathy.</blockquote><blockquote class="tr_bq">Today Silicon Valley companies like Google see their offices as exclusive preserves to provide high-end amenities to keep their employees. In the future they will integrate much better into their local communities, providing public spaces and leaving behind infrastructure that raise the quality of life for everyone, not just employees. The physical membrane separating offices from the outside will dissolve a bit.</blockquote><blockquote class="tr_bq">The importance of physical offices will also decline; we are actually at peak "real office". Collaborative tools and stronger individual and organizational skills will allow telecommuting to truly go mainstream. Coworking spaces will provide community for these workers.</blockquote><br /><br /></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-44335702135334175582015-05-07T14:52:00.000-07:002015-05-07T14:54:53.291-07:00Roots of Coworking Keynote at GCUC 2015<div dir="ltr" style="text-align: left;" trbidi="on"><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://twitter.com/maburkejr/status/596347007809114112/photo/1" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="http://4.bp.blogspot.com/-KrCbOVkRMtA/VUvK3N26mII/AAAAAAAAAjM/fDRKnYJr4v8/s400/CEamcQzVEAAWRAt.jpg" width="300" /></a></div><br />I gave a keynote at the <a href="http://gcuc.co/">GCUC 2015 coworking conference</a> on the roots of coworking, giving my personal story on <a href="http://codinginparadise.org/ebooks/html/blog/start_of_coworking.html">why I created coworking</a> and what led up to it.<br /><br />This is a slightly longer version of the talk I recorded before hand, with extra material:<br /><br /><iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="https://player.vimeo.com/video/127201764" webkitallowfullscreen="" width="500"></iframe><br /><br /><br />Some nice tweet reactions from folks in the audience:<br /><br /><blockquote class="twitter-tweet" lang="en"><div dir="ltr" lang="en">"Steal this idea, remix it, and make it your own. Let other people expand your ideas." - <a href="https://twitter.com/bradneuberg">@bradneuberg</a> (creator of Coworking) <a href="https://twitter.com/hashtag/gcucusa?src=hash">#gcucusa</a></div>— Oren Salomon (@iOren) <a href="https://twitter.com/iOren/status/596348297662496768">May 7, 2015</a></blockquote><script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script> <br /><blockquote class="twitter-tweet" lang="en"><div dir="ltr" lang="en"><a href="https://twitter.com/bradneuberg">@bradneuberg</a> Go Ahead - Be Weird remain <a href="https://twitter.com/hashtag/authentic?src=hash">#authentic</a> <a href="https://twitter.com/hashtag/grow?src=hash">#grow</a> your <a href="https://twitter.com/hashtag/coworking?src=hash">#coworking</a> space. Weird becomes the norm. Always be <a href="https://twitter.com/hashtag/creating?src=hash">#creating</a> <a href="https://twitter.com/hashtag/community?src=hash">#community</a></div>— Varela Consulting (@varelacons) <a href="https://twitter.com/varelacons/status/596347815606943744">May 7, 2015</a></blockquote><script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script> <br /><blockquote class="twitter-tweet" lang="en"><div dir="ltr" lang="en">It was all started by <a href="https://twitter.com/bradneuberg">@bradneuberg</a> in a funky space called the Spiral Muse. The movement has come so far! <a href="https://twitter.com/hashtag/coworking?src=hash">#coworking</a> <a href="https://twitter.com/hashtag/gcuc2015?src=hash">#gcuc2015</a></div>— CoLab Orlando (@CoLabOrlando) <a href="https://twitter.com/CoLabOrlando/status/596347636153618433">May 7, 2015</a></blockquote><script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script> </div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-12390293060490070522014-11-06T16:22:00.001-08:002014-11-06T16:22:16.991-08:00Left Inkling<div dir="ltr" style="text-align: left;" trbidi="on"><span style="background-color: rgba(255, 255, 255, 0);">I haven't been public about it, but I left Inkling 2 1/2 months ago. It was a fabulous place to work but after nearly four years it was time to move on. During this time I've been doing part time front end engineering consulting for a startup. I'm interviewing with Dropbox tomorrow to do part time consulting for them as well. I've also been taking this time to learn new skills, in particular taking a Product Management course at General Assembly and teaching myself machine learning and neural networks.</span></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-38077871299748762242014-04-14T08:30:00.000-07:002014-09-19T01:08:41.108-07:00Digital Storytelling Conference 2014<div dir="ltr" style="text-align: left;" trbidi="on"><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://codinginparadise.org/ebooks/img/chapter03/inkling_crew_at_digital_storytelling_conf.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://codinginparadise.org/ebooks/img/chapter03/inkling_crew_at_digital_storytelling_conf.jpg" height="240" width="320" /></a></div><br /><br /><span style="background-color: white; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px;">On Wednesday last week a bunch of us at&nbsp;</span><a href="http://inkling.com/" style="background-color: white; border: 0px; box-sizing: border-box; color: #037bb5; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px; padding: 0px; vertical-align: baseline;">Inkling</a><span style="background-color: white; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px;">&nbsp;jumped in a van in San Francisco and drove eight hours to U.C. Irvine for the&nbsp;</span><a href="http://sites.uci.edu/story/" style="background-color: white; border: 0px; box-sizing: border-box; color: #037bb5; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px; padding: 0px; vertical-align: baseline;">Digital Storytelling conference</a><span style="background-color: white; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px;">. It was a great road trip and a fabulous one-day conference.</span><br /><br /><a href="http://codinginparadise.org/ebooks/html/blog/digital_storytelling_conference_2014_notes.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-68967699623820542892014-03-17T08:30:00.000-07:002014-09-19T12:10:30.405-07:00Next Generation eBook Review: The Glo Bible: Surprising Lessons We Can Learn from Religious eBooks<br/><figure class="one-whole" data-uuid="c0d88f319173473c8032a557d58ecd76" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px auto 1.5em; padding: 0px; position: relative; vertical-align: baseline; width: 685px;"><img src="http://codinginparadise.org/ebooks/img/chapter03/illustration_from_gutenberg_bible.jpg" data-uuid="4d3dfb468cc742c7b7c073461dd3f3fd" style="border: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px auto; max-width: 600px; padding: 0px; vertical-align: baseline; width: 600px;" /></figure><br /><div data-uuid="655906919c3b4c77909799f84582b374" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">Today I'm reviewing a very interesting next generation religious eBook named&nbsp;<a href="http://globible.com/" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Glo</a>. The digital religious and eBook communities don't tend to talk to each other. However, the work happening in these digital religious communities around eBooks is incredibly interesting for several reasons.</div><a href="http://codinginparadise.org/ebooks/html/blog/next_generation_ebook_review_the_glo_bible.html">Read more...</a>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-52955436553035244452014-03-10T09:00:00.000-07:002014-09-19T01:16:50.953-07:00You're Doing Web & eBook Footnotes Wrong<div dir="ltr" style="text-align: left;" trbidi="on"><br /><h2 data-uuid="512db736d72546f8a4da3e71037311dc" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Source Sans Pro', Verdana, Helvetica, sans; font-weight: inherit; margin: 0px 0px 0.95em; padding: 0px; vertical-align: baseline;"><span style="font-weight: inherit;">The Problem</span></h2><div data-uuid="657cf30d14d94810afa224509ff0e3cf" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">Most web pages that have footnotes blindly mimic paper and put them at the bottom of the page, jumping the user to the footnote when clicked on:</div><br /><figure class="one-whole" data-uuid="21d588a6f379419abdd2fb68166dccee" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px auto 1.5em; padding: 0px; position: relative; vertical-align: baseline; width: 685px;"><iframe data-uuid="f50febc2519d419f8eff1713b60073d7" height="281" src="http://player.vimeo.com/video/88620817" style="border-width: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px auto; padding: 0px; vertical-align: baseline;" width="500"></iframe><figcaption style="border: 0px; box-sizing: border-box; font-family: 'Source Sans Pro', Verdana, Helvetica, sans; font-size: 0.8947em; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0.375em auto 0px; max-width: 600px; padding: 0px; position: relative; vertical-align: baseline;"><span class="caption-body" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Picking on&nbsp;<a href="http://paulgraham.com/articles.html" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Paul Graham's</a>&nbsp;footnotes</span></figcaption></figure><div data-uuid="427b5eba2eb743358192cfdf8b99748b" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">This is silly for many reasons. First, computer screens aren't paper; they can easily&nbsp;<a href="http://codinginparadise.org/ebooks/html/blog/stretchtext.html" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">accordion open</a>&nbsp;and show extra information based on user intent. Second, they cause a user to lose context while reading an article, forcing them to jump away from what they are doing; this is annoying.</div><a href="http://codinginparadise.org/ebooks/html/blog/doing_web_footnotes_right.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com1tag:blogger.com,1999:blog-3191291.post-89437846787909389312014-03-03T08:30:00.000-08:002014-09-19T01:20:20.292-07:00Inkling Habitat: How a 100,000 Line JavaScript Application Focused on Digital Publishing is Built<div dir="ltr" style="text-align: left;" trbidi="on"><br /><figure class="one-whole" data-uuid="bf517d7444ee4d09a89d05d26271d031" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px auto 1.5em; padding: 0px; position: relative; vertical-align: baseline; width: 685px;"><img data-uuid="081443003d2f46e9b02ea29e26e4422b" src="http://codinginparadise.org/ebooks/img/chapter03/chapter_overview_2.jpg" style="border: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px auto; max-width: 600px; padding: 0px; vertical-align: baseline; width: 600px;" /></figure><div data-uuid="5a4fd0f58e0b4515abf36cf3448ad37b" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;"><a href="http://codinginparadise.org/ebooks/html/blog/books_as_software.html" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Last week</a>&nbsp;I introduced you to one of the original animating ideas behind&nbsp;<a href="https://www.inkling.com/habitat/" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Inkling Habitat</a>, treating books as software to transform the eBook production process. Today I'd like to take you behind the scenes and show you the technologies and processes we used to build Inkling Habitat itself. How did we build this software?</div><h2 data-uuid="eb58c062d38344689ed79aec45486c5b" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Source Sans Pro', Verdana, Helvetica, sans; font-size: 1.5789em; font-weight: inherit; line-height: 1.3; margin: 0px 0px 0.95em; padding: 0px; vertical-align: baseline;">How is Inkling Habitat Built?</h2><div data-uuid="94c0dfb8c46b4a5287802b55d11f9794" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">First, Inkling Habitat is a client side application that runs inside your web browser, built with JavaScript, HTML, and CSS. The client-side portion is roughly 100,000 lines of JavaScript, which is a big application.</div><a href="http://codinginparadise.org/ebooks/html/blog/technology_behind_habitat.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-52659912452806665142014-02-25T08:00:00.000-08:002014-09-19T01:21:33.416-07:00Transforming eBook Production by Treating Books as Software<div dir="ltr" style="text-align: left;" trbidi="on"><br /><figure class="one-whole" data-uuid="7bb749347db347398abb828fcbe80f44" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px auto 1.5em; padding: 0px; position: relative; vertical-align: baseline; width: 685px;"><a href="http://codinginparadise.org/ebooks/html/blog/books_as_software.html" style="border: 0px; box-sizing: border-box; color: #037bb5; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><img data-uuid="a783a4038b6648e98fc3940065807fa9" src="http://codinginparadise.org/ebooks/img/chapter03/books_as_software.jpg" style="border: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px auto; max-width: 600px; padding: 0px; vertical-align: baseline; width: 600px;" /></a><figcaption style="border: 0px; box-sizing: border-box; font-family: 'Source Sans Pro', Verdana, Helvetica, sans; font-size: 0.8947em; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0.375em auto 0px; max-width: 600px; padding: 0px; position: relative; vertical-align: baseline;"><span class="caption-body" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><a href="http://www.flickr.com/photos/j1x4r/4313734090/" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Photo by Jixar</a></span></figcaption></figure><div data-uuid="3d1de2f15d094c4d91d0b6457691bcfc" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">Digital books are bundles of HTML, CSS, and JavaScript. How do we efficiently convert and create these pieces of software?</div><div data-uuid="36d91c6d90774a1eb369a6f6e2321904" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">These bundles can be very complicated. For example,&nbsp;<a href="https://www.inkling.com/store/book/ganongs-review-medical-physiology-barrett-24th/" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Ganong's Review of Medical Physiology</a>&nbsp;has thirty-nine chapters. Now add in interactive quizzes, 3-D models, high-definition video, educational slide lines, and pop tip glossaries/footnotes, and if you're not careful you will need a small army to produce every eBook. If next generation eBooks require Fabergé&nbsp;egg levels of care and expense we'll never get the scale and quantity we need to make this new world real.</div><figure class="one-half right" data-uuid="744c52b311af46629a527eb98bf137e2" style="border: 0px; box-sizing: border-box; clear: right; color: #333333; float: right; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px 0px 1.5em 21.90625px; padding: 0px; position: relative; vertical-align: baseline; width: 331.53125px;"><img data-uuid="e705f2e2b68c4d0a9075f524585dbbd2" src="http://codinginparadise.org/ebooks/img/chapter03/trs80.png" style="border: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; max-width: 600px; padding: 0px; vertical-align: baseline; width: 331.53125px;" /></figure><div data-uuid="81398bc7565e4968b42f778adb8ffc59" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">It turns out over the last fifty years we've developed an incredible set of techniques and tools for dealing with artifacts of incredible complexity: computer software itself. These tools include:</div><ul data-uuid="eb8d65c5baf2466e956dd47ffd139392" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px 0px 1.5em 2.25em; padding: 0px; vertical-align: baseline;"><li data-uuid="16bff8fe8fe5466380878520dffdad2e" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Source control systems</li><li data-uuid="9e6f13549ef6439d9f761cdc73a72277" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Issue databases</li><li data-uuid="90bd798773084efcac0d5863b7fb519b" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Automated testing</li><li data-uuid="ea993ca2255342a988851c8777b1751c" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Cross compilers</li><li data-uuid="05d3a36f79124a8f91a008d0d1d5d22b" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Integrated Development Environments (IDEs)</li></ul><a href="http://codinginparadise.org/ebooks/html/blog/books_as_software.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-82216944002584224102014-02-17T08:00:00.000-08:002014-09-19T12:10:55.257-07:00You Are Not in the Book Business: You Are in the Long Form Content Business<div dir="ltr" style="text-align: left;" trbidi="on"><br /><figure class="one-whole" data-uuid="c948d8a90f8b47a990ea1d8c26aefe26" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px auto 1.5em; padding: 0px; position: relative; vertical-align: baseline; width: 685px;"><img src="http://codinginparadise.org/ebooks/img/chapter03/telegram_clipped.png" data-uuid="0f0e01293e6a475baa14d5c995109b4e" style="border: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px auto; max-width: 600px; padding: 0px; vertical-align: baseline; width: 600px;" /></figure><br /><div data-uuid="e8e56766a65e4265b1be8095f9789631" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">Books are not a goal unto themselves. Instead, they are a means to an end: the transmission of authoritative long form content that possesses depth and breadth. Note that I'm specifically talking about illustrated non-fiction here; literary fiction is an&nbsp;<a href="http://codinginparadise.org/ebooks/html/blog/brad_neuberg_books_that_never_should_have_been_books.html" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">entirely different animal</a>.</div><div data-uuid="2b0210bf20264a72b59668ab5e3ea10f" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;">Organizations fail when they identify themselves with a particular technology rather than a goal.</div><a href="http://codinginparadise.org/ebooks/html/blog/long_form_not_book_business.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0tag:blogger.com,1999:blog-3191291.post-75180339481532843202014-02-10T08:00:00.000-08:002014-09-19T01:23:44.304-07:00Making EPUB3 Play Nice with HTML5<div dir="ltr" style="text-align: left;" trbidi="on"><br /><figure class="one-whole" data-uuid="c20f19362ecc472ba2cdf3fe61406d30" style="border: 0px; box-sizing: border-box; clear: both; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin: 0px auto 1.5em; padding: 0px; position: relative; vertical-align: baseline; width: 685px;"><a href="http://codinginparadise.org/ebooks/html/blog/making_epub3_play_nice_with_html5.html" style="border: 0px; box-sizing: border-box; color: #037bb5; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;"><img class="cat-and-dog-fight" data-uuid="54cfd377bbfb4c329b0d28950c1936fd" src="http://codinginparadise.org/ebooks/img/chapter03/cat_and_dog_fight.jpg" style="border: 0px; box-sizing: border-box; display: block; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px auto; max-height: 480px; max-width: 640px; padding: 0px; vertical-align: baseline; width: 640px;" /></a><figcaption style="border: 0px; box-sizing: border-box; font-family: 'Source Sans Pro', Verdana, Helvetica, sans; font-size: 0.8947em; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0.375em auto 0px; max-width: 600px; padding: 0px; position: relative; vertical-align: baseline;"><span class="caption-body" style="border: 0px; box-sizing: border-box; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Photo by&nbsp;<a href="http://www.flickr.com/photos/97793800@N00/3916951586/" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">adrigu</a></span></figcaption></figure><div data-uuid="c7876838198242f88ebdbccbf81550e7" style="border: 0px; box-sizing: border-box; color: #333333; font-family: 'Meta Serif Office Pro', Georgia, serif; font-size: 19px; line-height: 28.5px; margin-bottom: 1.5em; padding: 0px; vertical-align: baseline;"><a href="http://codinginparadise.org/ebooks/html/blog/epub3_in_html5_universe.html" style="border: 0px; box-sizing: border-box; color: #037bb5; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; line-height: inherit; margin: 0px; padding: 0px; vertical-align: baseline;">Last week I wrote</a>&nbsp;about how EPUB3 is important even in an HTML5 universe. Today I want to write about how to make EPUB3 play nice with HTML5; as it turns out there are some significant problems when you try to use EPUB3 in HTML5.</div><a href="http://codinginparadise.org/ebooks/html/blog/making_epub3_play_nice_with_html5.html">Read more...</a></div>Brad Neuberghttp://www.blogger.com/profile/03436380878044525337noreply@blogger.com0