Live SSB on Dectris ARINA data streams in Nion Swift #139
Comments
|
First of all, thank you for the report!
This should instead read (replacing
The See our docs for the supported kwargs when loading HDF5.
Orthogonal to this issue, depending on your system, it can also make sense to try the default executor, for example: context = Context.make_with('dask')
Just for reference, is this on a system with Intel CPUs, or one of the newer ARM-based ones?
I'm curious, does Nion Swift use its own HDF5 format, or is it internally storing the data compressed, as it comes from the detector/DCU? If the latter, you may need to install Hope this helps, and let me know if there are additional issues. |
|
Ahh... so my mistake – I had first tried the in-memory loader, and must have just copied the code incorrectly. It works perfectly fine when using This is an Intel system (which will be also the case for our Nion user PC), but I can also test on my MacBook if you're interested? Swift datasets work out of the box, I believe they just store them as arrays in a HDF5 container. However, I am quite interested in trying to do live SSB, so it may be that I will need to work with the DECTRIS datastreams, good to know. |
|
Maybe I can use the opportunity to ask a further question: I am getting a warning about the MaskContainer for my test dataset:
How should I think about this? Should I pick a selection of trotters only to use for the mask factory? |
You can ignore that warning with SSB, as long as it works! By the way, for speed you can try the Binned SSB Ptychography-4-0/ptychography#68 It works, the PR is just in the backlog for finishing. |
|
It does work, though reconstruction takes about 3 min on my (admittedly old) iMac Pro. Probably will need to look at GPU acceleration on Linux box (or the User PC itself). The ARINA bins to 96 x 96 pix so not convinced I need to bin that further? |
Good to hear that it works now!
I was mostly asking because ARM-based Macs are currently a blind spot in our QA, so there may be issues related to installation etc. on these systems.
Ok.
That's a tiny bit different, as we don't read the HDF5 files as they are written, but we directly get the data stream via zeromq from the DCU. That is not affected by the above-mentioned issue. |
Ah there may be a nuance here: Swift saves the DECTRIS Arina dataset internally as .h5. If I export the file, loading it into LiberTEM has no problem. However, if I try to directly read the .h5 file that Swift internally stores the data in, I get an error
when trying to plot the sum_result using the default |
The time strongly depends on the scan resolution. From 128x128 to 256x256 you get a 16x increase in computational effort. It also depends a lot on the scan step, size of the primary beam and convergence angle, so hard to say if this is already good or can be improved! The UDF already supports GPUs, if I read the source code and remember correctly.
16x16 is enough for SSB, and it also crops to the primary beam. The binned version also processes whole partitions, which might be faster, in particular on GPUs. It also uses improvements in LiberTEM for handling large shared objects for the trotter stack. We've hit, I think, 50.000 fps live with an ARINA (I don't remember if 128x128 or 256x256), so it is worth a try! |
Oh okay, so not just the binning! Yes, I will definitely try this, is there any example code on the live implementation you could share? |
|
The last section of https://github.com/Ptychography-4-0/ptychography/blob/dec7ce7fd3469decf79ff4b48098cced584c718f/examples/ssb-example.ipynb has an example! |
How is that '/cachedata/weber/ssb/slice_00001_thick_1.9525_nm_blocksz100.raw' created? It still sounds like a local file and not a stream on the DECTRIS server PC? |
|
...for doing it live one switches out the data source: https://libertem.github.io/LiberTEM-live/dectris-acquisition-example.html @sk1p do we have the improved notebooks somewhere? Convenient live processing is very much WIP, contributions welcome! There's also https://github.com/LiberTEM/LiberTEM-live-server where work on a Nion Swift integration was supposed to happen, but I am not sure how that is going ATM. |
|
Now that we have the first ARINA installed on a Nion machine, I believe Andreas Mittelberger is empowered to spend some more time on this with us, and yes we'd obviously be happy to contribute back where it makes sense. |
|
That would be great! The current architectural idea is that LiberTEM is running between the detector and the GUI. LiberTEM receives the full data stream from the detector and control commands from the GUI, and sends a strongly reduced data stream to the GUI for display. It was already used successfully with SerialEM that way: https://github.com/Pr4Et/SavvyScan |
|
...also, @sk1p is working on integration with CEOS' Panta Rhei. |
|
In particular the latter works pretty well! We've done 1 mio fps live interactive 4D STEM with an Amsterdam Scientific Instruments CheeTah T3 (Timepix) using a prototype, and as long as Nion Swift can keep up with plot updates it should also be possible there. |
|
I'll rename this issue to talk about live processing to keep this discussion going. |
TomaSusi
changed the title
The document you linked to is one of our example notebooks, and should be up-to-date. Other than that, the DECTRIS section in our docs should contain further details.
I'll follow up on their status soon.
Using a = np.zeros((128, 128))
plt.imshow(a, norm=LogNorm())
plt.show()You can try to mask out the zero values, for example like this: plt.imshow(np.ma.masked_where(a == 0, a), norm=LogNorm())That will give you "data missing" markers where you have zero values, but should plot without issues. |
|
Right, fair point about the LogNorm but since direct-electron datasets will always have zeros in the data, shouldn't there be some better way to handle this directly..? As a workaround, I find that this works decently for my data:
|
I think we do handle these kind of issues in the live plotting interface (which could be improved, surely, as I don't think it supports log scaling by default yet, for example). When getting back the raw results as arrays, we don't really know about the next processing step, so "punching holes" for zeros for log scaling or similar needs to be handled by the data consumer.
That's a good one, too 👍 |
@uellue, I tried to clone and install this fork, which I believe is where you have the binned SSB: The regular non-binned SSB runs fine on my dataset. However, trying to run the example code at the end of the notebook you pointed to (https://github.com/Ptychography-4-0/ptychography/blob/dec7ce7fd3469decf79ff4b48098cced584c718f/examples/ssb-example.ipynb) on the same data, I get an error: |
|
Sorry you are hitting these issues.
[...]
Hmm, I remember we hit this assertion previously. It's related to partition-by-partition processing with certain data sources. This is on the HDF5 data from nion swift, correct? Could you give some details about the file, such as chunking of the dataset, and/or maybe provide an example file that reproduces the issue (could be just an empty data set, as long as the shapes/chunking is similar, and as long as that hits the same issue)? This would be very helpful! |
The data shape is (256, 256, 96, 96) and chunking seems to be (1, 16, 96, 96). This is still some of the first test data on our ARINA, so the quality is not great, but this should not matter for testing. I do note that I am able to reconstruct a (somewhat lousy) SSB phase image using PyPtychoSTEM, but the LiberTEM SSB does not seem to yield anything sensible for the data despite okay-looking COM plots – probably binning will help. I'll send you a link to the data privately, alongside the metadata parameters used for the reconstruction. |
It may be that the (parameters for...) rotation between detector and scan is off. Or maybe @uellue has another idea?
Thank you, that is very helpful. |
That's possible, but I do calculate the rotation by minimizing the curl using py4DSTEM's DPC functionality, and use the same derived rotation for PyPtychoSTEM. But admittedly the data is simply not great, I'm curious to see if binning helps! |
|
@TomaSusi One has to take care that these software packages understand these parameters in the same way as LiberTEM resp. the SSB implementation! In Thermo Fisher software, what they call "scan rotation" is actually an image rotation, i.e. opposite direction, for example. Having the y axis point up instead of down for scan and/or detector is popular as well, meaning the handedness should be checked, too. The DPC method might give wrong results if the microscope or zone axis are not aligned perfectly, unless the specimen is very thin (graphene and such). |
|
Fair enough! How does one determine the rotation using LiberTEM tools? I will not accept having to change the value manually and looking at what seems best ;-) We almost always work with very thin specimen, in this case bilayer hBN. |
|
You can take an overfocused dataset and check where the detector image moves as a function of scan position using frame picking in the web GUI. |
|
I'm afraid that won't work for me, I do all my analyses in Jupyter. |
|
If you trust DPC, you can use this function: https://libertem.github.io/LiberTEM/reference/udf.html#libertem.udf.com.guess_corrections :-) |
|
Of course we're trying to calibrate this at the instrument and save it to metadata, but it may be tricky to get it exactly right for perpetuity. |
|
Yes, it should be recalibrated and checked frequently. |
|
I have now much less noisy datasets for monolayer graphene, where the guess_corrections seems to agree with the rotations I get from py4DSTEM (and we also aligned the scan axes in hardware on the Nion), but LiberTEM SSB still fails to reconstruct anything useful even though PyPtychoSTEM gives beautiful SSB phase images. |
|
Interesting! As far as I can remember, PyPtychoSTEM has a qualitative reconstruction mode where the trotters are weighted by their noise-normalized transfer function. I'm not sure if it is the default or if one has to activate it. On the high and low end of spatial frequencies the trotters are rather small, so they are sensitive to errors and noise. If the high and low spatial frequencies are filtered out, the result "looks better". LiberTEM SSB can filter out small trotters with the "cutoff" parameter. Maybe results look better with that? Also, binned SSB improves the behavior since it averages. Live WDD also has better noise rejection while being reasonably quantitative. For qualitative phase contrast iCoM is pretty good: Less sensitive to noise and much faster to compute. |
|
I checked in with Christoph, and that qualitative mode was probably only in the original Matlab code, PyPtychoSTEM reconstructs phases quantitatively. I tried increasing the cutoff in LiberTEM but can't seem to improve the result. Note that I am not talking only about a poor reconstruction, but a straight-up failure to produce anything sensible (output from
I think the problem is that generating trotters seems to fail for some reason, at least I presume they should not look like this (output of
|
|
Ahh... I found the issue – your parameters take the scanstep in m, not Å :D Now I am getting something sensible out! |
|
Ah, good to hear. That should explain it! |
|
Got this far on the newer datasets, have to cut off the smallest trotters quite heavily to get the noise in the LiberTEM reconstruction down to a similar level. Still trying to understand with Christoph why the range of phase values is different between the two reconstructions... (btw the py4DSTEM DPC phase is essentially iDPC though they implement an iterative scheme for it) With 8x software binning, the LiberTEM-based reconstruction completes in about 7 seconds on my desktop, this might already run fast enough on our beefy new Nion user PC to do semi-live full-frame reconstructions :)
|
I am trying to adopt the LiberTEM/Ptychography4.0 SSB code to analyze Nion Swift data collected on the Dectris ARINA detector, with the ultimate goal of running live SSB and piping the results to Nion Swift.
I am getting a TypeError when trying to load in the data using the InlineJobExecutor, and at first I though this was due to Swift data format, but I am able to reproduce the error using one of the sample datasets. My minimal code example reads:
This results in a TypeError regardless of what I try to do with the
ds_pathkeyword.Am I doing something wrong, or is there a bug perhaps? I am using LiberTEM 0.13.1 installed from conda-forge in a Python 3.11.7 conda environment on macOS 14.2.1.
The text was updated successfully, but these errors were encountered: