Parente's Mindtrove

Jupyter Tidbit: Run a notebook headlessly

September 05, 2018

This post originates from a gist that supports comments, forks, and execution in binder.


Summary

You can use nbconvert to execute a notebook from the command line (aka headlessly) and store the results in a new notebook file, an HTML file, a PDF, etc. Tools based on nbconvert, like papermill and nbflow, take this capability a step further and let you easily parameterize and chain notebooks.

Example

Binder

The command below executes the notebook named run_me.ipynb and outputs a new notebook file named you_ran_me.ipynb with all cell outputs captured. nbconvert executes the run_me.ipynb notebook using the kernel the notebook document declares in its metadata, python3 in this example.

jupyter nbconvert --to notebook --execute run_me.ipynb --output you_ran_me

Why is this useful?

You can take a notebook you developed in Jupyter Notebook, JupyterLab, nteract, etc., execute it using a scheduler or workflow tool like cron, Jenkins, Airflow, etc., and capture the same rich output in a notebook document as if you ran it manually. See Scheduling Notebooks at Netflix for inspiration.

Another Read: Jupyter Tidbit: Run and say "done!" »

Many modern web browsers provide a speech synthesis API for JavaScript.You can write and invoke a function to have your notebook speak when it finishes executing certain cells, whether you're running it in JupyterLab (>=0.34) or classic Jupyter Notebook.