Jupyter Tidbit: Run a notebook headlessly
September 05, 2018
This post originates from a gist that supports comments, forks, and execution in binder.
You can use nbconvert to execute a notebook from the command line (aka headlessly) and store the results in a new notebook file, an HTML file, a PDF, etc. Tools based on nbconvert, like papermill and nbflow, take this capability a step further and let you easily parameterize and chain notebooks.
The command below executes the notebook named
run_me.ipynb and outputs a new notebook file named
you_ran_me.ipynb with all cell outputs captured. nbconvert executes the
run_me.ipynb notebook using the kernel the notebook document declares in its metadata,
python3 in this example.
jupyter nbconvert --to notebook --execute run_me.ipynb --output you_ran_me
Why is this useful?
You can take a notebook you developed in Jupyter Notebook, JupyterLab, nteract, etc., execute it using a scheduler or workflow tool like cron, Jenkins, Airflow, etc., and capture the same rich output in a notebook document as if you ran it manually. See Scheduling Notebooks at Netflix for inspiration.