A few days ago I became interested in using Jupyter Notebooks to generate automatic reports to help me analyze and compare data between experimental conditions. I already use Jupyter’s nbconvert to generate html documentation with figures, based on python code. It is really nice. However, one thing was still missing: how to send parameters to jupyter-nbconvert so that I can fully automate my reports?
I found some interesting solutions on the web such as nbparameterise or nbrun, but for my very simple use case, the best is no external tool at all. Also, I may want to hide the parameters from the report, for example if one of the parameters is a password*.
Then I realized that what I need is really super simple. The trick is to write the parameters to a temporary module that will be imported by the Jupyter notebook. Here, for simplicity, we assume that all files are in a same folder.
Master script (the script that automates report generation)
import subprocess
def generate_arguments(dictionary):
"""Create a 'arguments.py' module to initialize a Jupyter notebook."""
with open('arguments.py', 'w') as fid:
for key in dictionary:
fid.write(f'{key} = {repr(d[key])}\n')
# Prepare the arguments
generate_arguments({
'variable1': 'value1',
'variable2': 1234,
'variable3': ['a', 'b', 'c'],
'variable4': {'a': 1, 'b': 2},
})
# Run the notebook
subprocess.call(['jupyter-nbconvert',
'--execute',
'--to', 'html',
'NAME_OF_THE_NOTEBOOK.ipynb'])
In this example, the generate_arguments function creates a local python module named arguments
with this contents:
variable1 = 'value1'
variable2 = 1234
variable3 = ['a', 'b', 'c']
variable4 = {'a': 1, 'b': 2}
Slave Notebook (the notebook that generates a report based on input arguments)
The only thing to do is to begin the Jupyter Notebook with this line:
import arguments
which will declare the variable needed by the report.
*Obviously, if the parameters contain sensitive information, caution has to be taken (ensure the file is local, not readable by others, not backed up, erase it afterward, etc.)