pytest-notebook#

A pytest plugin for regression testing and regenerating Jupyter Notebooks.

Example Test

Purpose#

The purpose of the plugin is to ensure that changes to the python environment (e.g. updating packages), have not affected the outputs of the notebook. If the notebook has changed, this plugin can also regenerate the notebooks, saving the new outputs.

Features#

  • Recognise, collect, execute then diff input vs. output Jupyter Notebooks.

  • Provides clear and colorized diffs of the notebooks (using nbdime)

  • Regenerate failing notebooks (see Regenerating Notebooks).

  • Integration with coverage and pytest_cov (see Execution and Coverage Reporting).

  • A well defined API allows notebook regression tests to be run in multiple ways (see pytest-notebook by example):

    1. Using the pytest test collection architecture.

    2. As a pytest fixtures.

    3. Using the pytest_notebook python package.

  • All stages are highly configurable (see Configuring pytest-notebook) via:

    1. The pytest command-line interface.

    2. The pytest configuration file.

    3. The notebook and cell level metadata.

  • Post-processor plugin entry-points, allow for customisable modifications of the notebook, including source code formatting with black (see Post-processors).

Configuration Examples

License#

Distributed under the terms of the BSD-3 license, pytest-notebook is free and open source software.

Issues#

If you encounter any problems, please file an issue along with a detailed description.

Acknowledgements#

Indices and tables#