HOWTO: Adding Keyboard Accelerators to Holoviz Applications for Machine Learning Workflows

My team at Planet trains deep nets with large amounts of geospatial remote sensing data, for tasks such as semantic segmentation. We've created a set of internal Jupyter Notebooks built above the HoloViz ecosystem for quickly jumping through, validating, QAing, etc. this training data, leaning on tools like GeoViews to efficiently display and create these UIs. Since we are sometimes dealing with thousands of samples, keyboard accelerators are very important for productively dealing with this data.

Unfortunately, Bokeh/HoloViews/GeoViews don't have any official mechanisms for adding keyboard accelerators. I recently created a workaround for this that was tricky, and thought sharing it might help others in a similar situation when attempting to create Bokeh/HoloViews powered Jupyter Notebooks for the kinds of workflow situations machine learning often requires.

To use, you will first need to add a Grab Keyboard button to your HoloViz UI. Here's an example UI from our own Planet QA work with this button added:

Clicking the Grab Keyboard button will "inject" JavaScript keyboard handling into the page; if the user clicks the Release Keyboard button that toggles or loses focus, then keyboard handling will be deactivated. This is so that Jupyter's own keyboard handling doesn't collide with these keyboard accelerators. Image showing the toggled Release Keyboard button:

In your HoloViews app, make sure it has this button; also define a unique instance_class CSS class that will be used to 'silo' your panel when dealing with its buttons in case there are multiple panel instances in the Jupyter notebook:

self.instance_class = 'some_tool'
self.grab_keyboard_button = pn.widgets.Button(name="Grab Keyboard")

Pass both of these into your panel when drawing the UI:

def panel(self):
        return pn.Column(
            # ... other parts of your UI
            pn.Row(
                pn.Column(
                    # 4 pn.widgets.Buttons that we want to attach keyboard accelerators to.
                    pn.Row(
                        self.prev_button,
                        self.valid_button, 
                        self.invalid_button,
                        self.next_button,
                    ),
                ),
                pn.layout.HSpacer(),
                pn.Column(
                    # The Grab Keyboard button.
                    self.grab_keyboard_button,
                    align="center",
                ),
                background='lightgrey',
            ),
            # Make sure we have a JavaScript hook to bind onto the buttons for just this
            # pyviz panel in case there are multiple ones in a Jupyter notebook. Used
            # so that we can add keybindings.
            css_classes=[self.instance_class],
        )

Next, you will need to grab the JSKeyboardAccelerators class from this gist I put up and save it to the file js_keyboard_accelerators.py.

Now instantiate a JSKeyboardAccelerators instance with the actions you want to bind to:

self.accelerators = JSKeyboardAccelerators(self.instance_class, self.grab_keyboard_button,
      actions=[
            {
                'action_name': 'Previous',
                'keycode': JSKeycodes.LEFT_ARROW,
                'html_button_ordering': 0,
                'expected_text': '◀',
            },
            {
                'action_name': 'Next',
                'keycode': JSKeycodes.RIGHT_ARROW,
                'html_button_ordering': 1,
                'expected_text': '▶',
            },
            {
                'action_name': 'Invalid',
                'keycode': JSKeycodes.DOWN_ARROW,
                'html_button_ordering': 2,
                'expected_text': '✖',
            },
            {
                'action_name': 'Valid',
                'keycode': JSKeycodes.UP_ARROW,
                'html_button_ordering': 3,
                'expected_text': '✔',
            },
       ])

The action_name and expected_text fields are debugging fields that will be printed to the JavaScript console when this keyboard accelerator is pressed to aid in ensuring that the right action is being invoked; keycode is one of the JSKeycodes enums (add your own if one is not listed that you want to use); and html_button_ordering is the ordering of the button you want to be 'clicked' on in the background -- basically, the order returned from the CSS selector for your instance_class defined, such as '.some_tool button'.

Hopefully this helps your own work!