TU Cookbook Python Template

Martin Schobben

Project Status

WP Cookbook Development

  • Phase 1: Developer = Teacher → Student
    • Demonstrator (Early 2026)
    • Focus on infrastructure and development
  • Phase 2: Developer → Teacher → Student
    • Focus on easy content creation

Teaching by example

  • Good and working code
    • Clear and well-designed
    • (Reviewed) open source software (pyopensci)
  • Best scientific practices
    • FAIR data
    • citing & attribution
    • …

Phase 1: Developers

  1. Code clarity
  2. Code quality
  3. Dependencies
  4. Code testing
  5. Attribution
  1. Citing
  2. Reviewing
  3. Packaging
  4. Publication

1. Code clarity

  • Notebook is the documentation
    • with 3:1 ratio prose vs code (Wagemann et al., 2022)
  • Abstraction of superfluous code as custom package
  • Type hinting and """docstrings""" for custom functions

Tools: pip, (mypy, ty?)

Automation: Responsibility of the cook

1. Code clarity

📁 tu-cookbook-python/
├─📄 Dockerfile
├─👉notebooks/
│ ├─✏️notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─👉src/
│ └─👉tucookbook/
│   └─✏️__init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─✏️pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─📄 myst.yml

2. Code quality

  • Code style PEP8
  • Type checking
  • (Reviewed) open source software (pyopensci)
  • Runtime?

Tools: ruff

Automation: Assistance with pre-commit hooks

2. Code quality

📁 tu-cookbook-python/
├─📄 Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─✏️.pre-commit-config.yaml
├─📄 CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─✏️pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─📄 myst.yml

3. Dependencies

  • Prevent package version pinning
  • Conda for cross-platform usage (Windows)
  • Custom code as installable Python package (toml file)
  • Docker
    • complex system requirements
    • JupyterHub compatibility

Tools: conda, docker

Automation: Assistance with conda and docker-compose

3. Dependencies

📁 tu-cookbook-python/
├─✏️Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─✏️docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─✏️pyproject.toml
├─📄 COPYRIGHT.md
├─✏️environment.yml
├─📄 LICENSE
└─📄 myst.yml

4. Code testing

myst build --execute --html
  • GitLab CI
    • Check the README “nightly build” badge

Tools: myst, gitlab-ci

Automation: Fully automated check and badge “nightly build”

4. Code testing

📁 tu-cookbook-python/
├─📄 Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─✏️.gitlab-ci.yml
├─📄 README.md
├─📄 pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─✏️myst.yml

5. Attribution

  • Code license
  • Python package metadata (toml)
  • Content license

Tools: ?

Automation: Responsibility of the cook

5. Attribution

📁 tu-cookbook-python/
├─📄 Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─📄 pyproject.toml
├─✏️COPYRIGHT.md
├─📄 environment.yml
├─✏️LICENSE
└─📄 myst.yml

6. Citing

  • Cite all third-party work
  • Explain how to cite the book
  • Zenodo DOI minting?

Tools: CITATION.cff

Automation: Responsibility of the cook

6. Citing

📁 tu-cookbook-python/
├─📄 Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─✏️how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─✏️CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─📄 pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─📄 myst.yml

7. Reviewing

  • GitLab
    • Commit standard
    • Branching model
  • Never push notebooks with executed cells
  • Prevent pushing large files (50 Mb)
  • Contributing guidelines and changelogs

Tools: git, nbstripout, pre-commit

Automation: Assistance with pre-commit hooks

7. Reviewing

📁 tu-cookbook-python/
├─📄 Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─✏️CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─✏️.pre-commit-config.yaml
├─✏️CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─📄 pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─📄 myst.yml

8. Packaging

  • Creating an image for the TUWEL JupyterHub
  • Creating an image for the dataLAB BinderHub

Tools: docker, gitlab-ci

Automation: Fully automated image push to dataLAB registry

8. Packaging

📁 tu-cookbook-python/
├─✏️Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─📄 .gitlab-ci.yml
├─📄 README.md
├─✏️pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─📄 myst.yml

9. Publication

  • Deployment of myst Jupyter book to GitLab pages
  • Local testing
myst build --execute --html

Tools: myst, gitlab-ci

Automation: Fully automated myst build and publication on GitLab Pages

9. Publication

📁 tu-cookbook-python/
├─📄 Dockerfile
├─📁 notebooks/
│ ├─📄 notebook-template.ipynb
│ └─📄 how-to-cite.md
├─📄 .gitignore
├─📄 docker-compose.yml
├─📄 CITATION.cff
├─📄 CONTRIBUTING.md
├─📁 src/
│ └─📁 tucookbook/
│   └─📄 __init__.py
├─📄 .pre-commit-config.yaml
├─📄 CHANGELOG
├─✏️.gitlab-ci.yml
├─📄 README.md
├─📄 pyproject.toml
├─📄 COPYRIGHT.md
├─📄 environment.yml
├─📄 LICENSE
└─✏️myst.yml

Next Steps

Phase 1:

  • Templates Cookbook/CI for R (and Julia?)
  • The big data issue

Phase 2:

  • More teacher assistance
  • Content creation best practices