Development#

Installation for development#

PyGlider uses pixi to manage environments. To get started, clone the repo and install in editable mode:

git clone https://github.com/c-proof/pyglider.git
cd pyglider
pixi install

This installs pyglider itself as an editable package (changes to source files take effect immediately without reinstalling). The default environment includes the library and its dependencies. To also get the test dependencies:

pixi install -e test

If you prefer conda/pip without pixi, install the dependencies with conda and then do an editable pip install:

conda create -n pyglider-dev
conda activate pyglider-dev
conda install -c conda-forge dask netcdf4 xarray numpy scipy gsw
pip install -e .

Running the tests#

PyGlider uses pytest. To run the tests with pixi:

pixi run -e test pytest tests/

Or, if your environment already has the dependencies installed:

pytest tests/

Tests run the full processing pipeline on example data and compare the output against stored NetCDF golden files in tests/expected/. A test failure means the output has changed — either a regression or an intentional change that requires updating the golden files (see below).

How the regression tests work#

Each golden file pair in tests/expected/ consists of:

A cleaned .nc file — the reference dataset used for comparison. Dynamic global attributes (date_created, date_issued, history) are stripped so the file is stable across runs.
A .cdl header file (ncdump -h output) alongside it for human-readable diffs in git.

Tests compare fresh pipeline output against the golden .nc files using xarray.testing.assert_identical (via nc_test_helpers.assert_datasets_equal), stripping dynamic attributes from the actual output before comparison.

The golden files live under tests/expected/, organized by glider type and pipeline stage:

tests/expected/
    example-seaexplorer/
        L0-timeseries/
            dfo-eva035-20190718.nc
            dfo-eva035-20190718.cdl
            dfo-eva035-20190718_adjusted.nc
            dfo-eva035-20190718_adjusted.cdl
        L0-gridfiles/
            dfo-eva035-20190718_grid_adjusted.nc
            dfo-eva035-20190718_grid_adjusted.cdl
    example-seaexplorer-raw/
        L0-timeseries/
            dfo-bb046-20200908.nc
            dfo-bb046-20200908.cdl
    example-slocum/
        L0-timeseries/
            dfo-rosie713-20190615.nc
            dfo-rosie713-20190615.cdl
            dfo-rosie713-20190615_adjusted.nc
            dfo-rosie713-20190615_adjusted.cdl
        L0-gridfiles/
            dfo-rosie713-20190615_grid_adjusted.nc
            dfo-rosie713-20190615_grid_adjusted.cdl

The test files (tests/test_*_nc.py) each run the pipeline once at module load time, writing fresh output to tests/example-data/example-*/L0-timeseries-test/ (and equivalent -test directories for other pipeline stages), then compare that output against the golden NC files in tests/expected/. The process_adjusted tests are an exception — they write directly into tests/expected/ and clean the files in place.

Updating golden files after an intentional change#

If you make a change to pyglider that intentionally alters the output — new variable, changed attribute, fixed a calculation — the golden files need to be updated.

Step 1: Regenerate the golden files:

python tests/_generate_expected_cdl.py

This re-runs the processing pipelines, writes cleaned NC golden files to tests/expected/, and generates .cdl headers alongside each one. For the slocum L0 timeseries pipeline (which requires dbdreader), the golden file is read from a pre-existing NC in tests/example-data/example-slocum/L0-timeseries/; run tests/example-data/example-slocum/process_deploymentRealTime.py to refresh that file if needed.

Step 2: Re-run the tests to confirm they now pass:

pixi run -e test pytest tests/

Step 3: Inspect the diff before committing:

git diff tests/expected/

The diff should show only the changes you intended. If unrelated variables or datasets changed, investigate before committing.

Test file inventory#

Test file	What it covers
`test_seaexplorer_nc.py`	SeaExplorer NRT sub and raw delayed L0 timeseries; interpolation behaviour
`test_slocum_nc.py`	Slocum L0 timeseries and profiles; CF/GliderDAC compliance
`test_seaexplorer_og10_nc.py`	SeaExplorer OG 1.0 timeseries; OG/CF compliance
`test_slocum_og10_nc.py`	Slocum OG 1.0 timeseries; OG/CF compliance
`test_process_adjusted_seaexplorer_nc.py`	SeaExplorer `process_adjusted` timeseries and gridfiles
`test_process_adjusted_slocum_nc.py`	Slocum `process_adjusted` timeseries and gridfiles
`test_seaexplorer.py`	SeaExplorer-specific unit tests
`test_utils.py`	Utility function unit tests

Making a release#

Releases are published to PyPI automatically when a version tag is pushed. The GitHub Actions workflow builds the package, publishes to TestPyPI, then PyPI, and creates a GitHub Release with auto-generated notes.

The version number is derived from git tags via setuptools-scm — there is no version to manually bump.

Step 1: Tag the commit you want to release:

git tag v1.2.3
git push --tags

That’s it. The workflow triggers on any v* tag.

TestPyPI runs first as a smoke test. If it fails (e.g. the package already exists at that version on TestPyPI), you can re-tag after fixing, but note that TestPyPI does not allow re-uploading the same version — use a new tag.

Trusted Publishing must be configured on both PyPI and TestPyPI for the pypi and testpypi GitHub Environments respectively. If the workflow’s publish step fails with a permissions error, check that the environment is configured in the repo settings and that the PyPI trusted publisher entry matches the repo and workflow file name.

Numerical tolerances#

Comparisons use xarray.testing.assert_identical, which requires exact bit-for-bit equality of all variable values and attributes. Time monotonicity is checked as a separate test. The date_created, date_issued, and history global attributes are excluded from comparison because they change on every run.