Development setup#
Create a virtual environment, activate it and install required packages:
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,test,docs]"
# enable basic style checks once
pre-commit install
Important
Make sure to activate the virtual environment in every new shell session:
source .venv/bin/activate
If you want to automate this install direnv and allow it once via direnv allow
(see .envrc
configuration file).
Windows-specifics
On Windows the syntax for virtual environment activation is a bit different:
# The following may need to be run once. Please check the docs for its consequences:
# https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policiess
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser
# Activate via:
.venv\Scripts\Activate.ps1
CLI scripts can now be simply run:
msh2vtu --help
Using make
for shortcuts!
Development-related tasks can also be done with make
(requires a Bash shell with make
). The above development setup can also be generated with:
make setup
To get all available make
-targets run make help
:
help Show this help
setup Setup a virtual environment and install all development dependencies
# clone Setup a virtual environment and install all development dependencies
setup_headless Install vtk-osmesa and gmsh without X11 dependencies
setup_devcontainer Internal usage
test Runs the unit tests
coverage Runs the unit tests generating code coverage reports
check Runs various checks with pre-commit
clean Cleans up temporary files
docs Builds the documentation
cleandocs Cleans up temporary documentation files
preview Runs an auto-updating web server for the documentation
conda init zsh
conda create --prefix /tmp/ogstools-test-env-py312 python=3.12
conda activate /tmp/ogstools-test-env-py312
Testing with pytest
#
Tests are executed via pytest
(shortcut: make test
):
pytest [--capture=tee-sys]
Test coverage#
The following commands run the tests and create a test coverage report (shortcut: make coverage
):
coverage run -m pytest
coverage combine
coverage report --no-skip-covered
coverage html
...
TOTAL 1698 292 83%
coverage html
Wrote HTML report to htmlcov/index.html
You can view a test coverage report by opening htmlcov/index.html
in a browser.
Build documentation#
For generating the documentation we use Sphinx (shortcut: make docs
):
cd docs
make html
This will create the documentation files in docs/_build/html
.
You can use an auto-generating and -reloading web server (Linux / macOS only) for developing the documentation (shortcut: make preview
):
make html -C docs # Generate docs once
python docs/server.py
# Open http://127.0.0.1:5500 in a web browser
# ...
# You can stop the server in the terminal with CTRL-D
Galleries#
Python files in docs/examples
will be added to the Examples-gallery based on sphinx-gallery.
Please note that text blocks are written reStructuredText-format.
The examples can be downloaded from the final website as Jupyter notebook files.
You can interactively run and debug these files in Visual Studio Code, see the Python Interactive window documentation.
If you want to link to a gallery page from another page use the following syntax (prefix with sphx_glr_
, replace directory separator with _
):
{ref}`meshlib example <sphx_glr_auto_examples_howto_meshlib_plot_meshlib_pyvista_input.py>`
Guidelines for examples#
In order to maintain consistency in style and structure between different examples, please follow those recommendations when creating a new one:
All imports and global settings must be done in first python block.
No heading for import and settings cell.
Examples that cover multiple steps and/or datasets must contain section.
Sections have to be give a title, indicated by highest level heading.
Section titles cannot start with ordinal number or letter (no 1./I./A. Example section title)
Further information#
For syntax references and extension usage see the following links:
Run checks#
We use pre-commit to run various checks (shortcut make check
):
pre-commit run --all-files
Create a package#
pyproject-build
Packages can then be found in dist/
.
Development in a container with VSCode#
A full-featured (including e.g. FEFLOW-functionality), ready-to-use development environment can be used via VSCode’s Dev Container feature. You can do all development-related tasks with it, e.g. testing, previewing documentation or even debugging.
Requirements#
VSCode with Remote Development extension pack installed
How-to#
Open the
ogstools
-project in VSCodeClick the blue button in the left-bottom corner
Click on
Reopen in Container
Now you are inside the container. For example, you can open a new terminal (Terminal
/ New Terminal
) and then run some tests with pytest
or use the Testing
-sidebar to select specific tests to debug.
Container specification#
.devcontainer/devcontainer.json
(see also available settings):
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/python
{
"name": "ogstools with FEFLOW",
"build": {
"dockerfile": "Dockerfile"
},
// Features to add to the dev container. More info: https://containers.dev/features.
"features": {
"ghcr.io/devcontainers/features/git-lfs:1": {}
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
"postCreateCommand": "make setup_devcontainer",
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.black-formatter",
"ms-toolsai.jupyter"
],
"settings": {
"python.defaultInterpreterPath": ".venv-devcontainer/bin/python",
"python.terminal.activateEnvInCurrentTerminal": true,
"python.formatting.provider": "none",
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.formatOnSave": true
},
"python.testing.pytestArgs": [
"."
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
}
},
}
FROM mcr.microsoft.com/devcontainers/python:3.10-bookworm
# FEFLOW, instructions from https://download.feflow.com/download/FEFLOW/linux/
ENV \
# Adapt when changing base image:
DIST_DIR=ub2204x64
RUN apt-get update \
&& apt-get install -yq --no-install-recommends gpg curl ca-certificates
# Error: The following packages have unmet dependencies:
# feflow-console81 : Depends: aspnetcore-runtime-6.0 but it is not installable
# Add dot net repos for debian 12 (bookworm base image)
RUN curl -fsSL https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -o packages-microsoft-prod.deb \
&& dpkg -i packages-microsoft-prod.deb && rm packages-microsoft-prod.deb
RUN curl -fsSL https://download.feflow.com/download/FEFLOW/linux/dhi-feflow-key.asc | gpg --dearmor | sudo tee /usr/share/keyrings/feflow.gpg > /dev/null \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/feflow.gpg] https://download.feflow.com/download/FEFLOW/linux/apt-repo/$DIST_DIR stable main" | sudo tee /etc/apt/sources.list.d/feflow.stable.list \
&& apt-get update \
&& apt list | grep feflow \
&& apt-get install -yq --no-install-recommends feflow-ifm-devel feflow-python81 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
ENV \
PYTHONPATH=/opt/feflow/8.1/python \
LD_LIBRARY_PATH=/opt/feflow/8.1/lib64:/opt/feflow/common/qt/lib64:/opt/feflow/common/lib64 \
FEFLOW81_ROOT=/opt/feflow/8.1
Container usage without VSCode#
Advanced topic
If you are familiar with Docker, you can also start the container manually, e.g. with:
docker run --rm -it -v $PWD:$PWD -w $PWD registry.opengeosys.org/ogs/tools/ogstools/devcontainer-3.10 /bin/bash
# Inside the container:
make setup_devcontainer
pytest
Please also be aware of file permission issues when mounting your working directory.
To prevent these issues we recommend running via Apptainer:
apptainer shell docker://registry.opengeosys.org/ogs/tools/ogstools/devcontainer-3.10
# Inside the container:
make setup_devcontainer
pytest
Release procedure#
Make sure there is a complete changelog at
docs/releases
and added to the correspondingindex.md
.Create a tag.
Wait for the tag pipeline to complete. Copy the output of the
pages
-job to a new directory in the ogs/tools/ogstools-docs-repo.