Skip to content

feature: new reader for .h5-jpk files#152

Merged
ns-rse merged 41 commits intoAFM-SPM:mainfrom
derollins:derollins/h5jpk
Jun 17, 2025
Merged

feature: new reader for .h5-jpk files#152
ns-rse merged 41 commits intoAFM-SPM:mainfrom
derollins:derollins/h5jpk

Conversation

@derollins
Copy link
Copy Markdown
Member

Bruker/ JPK have started using a new file type for high speed systems (incl. Nanoracer) rather than images being saves individually frames from a high speed 'video' are saved together in one HDF5 file.

This reader follows the format of the other readers within the library, adding two extra parameters:

  1. flip_image, also used in the updated .jpk reader, it appears that the new .h5-jpk images also need to be flipped,
  2. frame, an integer to indicates which frame should be accessed.

The output is then the same as for the other loaders.

This PR includes:

  • file reader in h5_jpk.py
  • tests in test_h5jpk.py (with sample_0.h5_jpk added to tests/resources
  • updated README

Tests for the file loading, reading and another added for when the frame number provided is out of range.

Copy link
Copy Markdown
Collaborator

@ns-rse ns-rse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies @derollins I started reviewing this the other week and forgot to finish the job and only just realised.

A few minor comments in-line but looking good.

Appreciate you taking the time to contribute these improvements.

Comment thread tests/test_h5jpk.py
Comment thread AFMReader/h5_jpk.py Outdated
Comment thread AFMReader/h5_jpk.py Outdated
Comment thread AFMReader/h5_jpk.py Outdated
elif self.suffix == ".spm":
image, pixel_to_nanometre_scaling_factor = spm.load_spm(self.filepath, self.channel)
elif self.suffix == ".h5-jpk":
image, pixel_to_nanometre_scaling_factor, _ = h5_jpk.load_h5jpk(self.filepath, self.channel)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will people never want the timestamps that are returned?

Or is it the case that for the general_loader they won't be required?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following the format of the .asd loader here. Since I don't know much about the application of the general loader (outside the reference to napari in the docstring) I thought I would just follow the lead of the existing code.

Both .asd and .h5-jpk are 'high-speed' formats and the temporal information is very important for this sort of data, however I don't know if it is required in the context that the general loader is designed.

P.S. I have a little project dealing with this sort of data within a time-aware framework: playNano

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the general_loader is @MaxGamill-Sheffield work and for now it should consistently just return image and pixel_to_nanometre_scaling_factor so don't worry about this.

playNano looks great, I hadn't realised you were so into your coding. It looks in really good shape from the FAIR for Research Software perspective too with good documentation and I like that you're using pre-commit and linting everything 👍 . If you wanted to have it reviewed in some manner there is a useful tool from Scientific Python Repo Review

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it's still quite new and very much a work in progress, but I'm building it with FAIR in mind. Right now, I'm focused on laying down the final pieces of the foundation. Once that's in place, I plan to do a FAIR review and plan before introducing more features. Many of the linting and testing tools were inspired by my experience working with you and the TopoStats team!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆒 , you could consider publishing it in the Journal of Open Source Software when you're happy with it.

@derollins
Copy link
Copy Markdown
Member Author

@ns-rse Should I go through the tests for AFMReader and add pytest.approx too?

Copy link
Copy Markdown
Collaborator

@ns-rse ns-rse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @derollins

Tests are failing under Python 3.11/3.12 because of precision issues which I've fixed in #155. I would suggest updating your branch and which should update this PR but am wary of possible merge conflicts so am happy to merge this.

The one thing I forgot when reviewing was updating docs/usage.md to include .h5-jpk in the supported file formats but I'm adding to that at the moment myself and will include it so nothing to worry about.

@ns-rse ns-rse merged commit 8f47866 into AFM-SPM:main Jun 17, 2025
1 of 10 checks passed
@derollins
Copy link
Copy Markdown
Member Author

Great, thanks Neil!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants