Notebook showing how to use LinearSimulator to generate particle images from a given volume

[bogdantoader]

The linear simulator is a differentiable simulator of cryo-EM images that implements the so-called "weak-phase approximation" of the cryo-EM image formation model.

This simulator is a key piece for deep generative models. Being able to explain its use is fundamental for simSPI, which makes this task a high-priority task.

Checklist

Verify that your PR checks all the following items.

• The pull request (PR) has a clear and explanatory title.
• The description of this PR links to relevant GitHub issues.
• Unit tests have been added in the tests folder:
  • in the test_*.py files corresponding the files modified by this PR,
  • for each function added by this PR.
• The code of this PR is properly documented, with docstrings following simSPI conventions.
• The PR passes the DeepSource GitHub Actions workflow (refer to comment below).
• The PR passes Test and Lint GitHub Actions workflows. (refer to comment below)

If some items are not checked, mark your PR as draft (Look for "Still in progress? Convert to Draft" on your PR) . Only mark the PR as "Ready for review" if all the items above are checked.

If you do not know how to address some items, reach out to a maintainer by requesting reviewers.

If some items cannot be addressed, explain the reason in the Description of your PR, and mark the PR ready for review

Description

Issue

Issue 107

Additional context

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[sonyahanson]

Interested in getting this notebook to run so checked out this branch, and came across a a few errors in the notebook as is, though I am not sure if they relate to why tests are failing, and they could very well relate to a problem on my end...

Problem 1: from tests.test_linear_simulator import init_data:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
/var/folders/pc/2x6kyg8j0fz0tw81xkr31dnr0001ng/T/ipykernel_97136/2230198603.py in <cell line: 1>()
----> 1 from tests.test_linear_simulator import init_data

ModuleNotFoundError: No module named 'tests.test_linear_simulator'

I can move forward by just copying in the function directly to the notebook for now, but clearly this is not ideal.

Problem 2: saved_data, config = init_data(config_path):

---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
~/opt/anaconda3/envs/simSPI-mac/lib/python3.9/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    447             try:
--> 448                 return pickle.load(fid, **pickle_kwargs)
    449             except Exception as e:

UnpicklingError: invalid load key, ' '.

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
/var/folders/pc/2x6kyg8j0fz0tw81xkr31dnr0001ng/T/ipykernel_97136/1679176522.py in <cell line: 2>()
      1 config_path = f"{root_dir}tests/data/linear_simulator_data.npy"
----> 2 saved_data, config = init_data(config_path)

/var/folders/pc/2x6kyg8j0fz0tw81xkr31dnr0001ng/T/ipykernel_97136/453811569.py in init_data(path)
     14     config: object
     15     """
---> 16     saved_data = np.load(path, allow_pickle=True).item()
     17     if "config_dict" in saved_data:
     18         config_dict = saved_data["config_dict"]

~/opt/anaconda3/envs/simSPI-mac/lib/python3.9/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    448                 return pickle.load(fid, **pickle_kwargs)
    449             except Exception as e:
--> 450                 raise IOError(
    451                     "Failed to interpret file %s as a pickle" % repr(file)) from e
    452 

OSError: Failed to interpret file '/Users/shanson/Documents/github/simSPI/tests/data/linear_simulator_data.npy' as a pickle

My config_path is the absolute path to linear_simulator_data.npy.

[ninamiolane]

@bogdantoader thank you for this great PR! Can you follow the PR checklist above?

For example, the tests need to pass (see Test / build (pull request) above) before we can mark it for review. --> I will mark the PR as draft in the meantime.

many thanks!

[bogdantoader]

@ninamiolane Apologies, I was aware the notebook test was failing when we finished on Friday and in the meantime I managed to get it to work on my laptop. I’ll push the working version when I’m back in the office tomorrow

@sonyahanson are you running the notebook in the simSPI anaconda environment and did you run “pip install -e .” on each of the 3 repositories? I wonder if that’s why it can’t see the module and also maybe it’a using a different version of pickle than the one in the environment file. I’ll take a closer look at this tomorrow as well!

[bogdantoader]

Did the access change since the hackathon? I can't push to this branch anymore:

(simspi) bogdan@Bogdans-MacBook-Pro simSPI % git push origin linear_simulator_nb
ERROR: Permission to compSPI/simSPI.git denied to bogdantoader.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

[ninamiolane]

Apologies @bogdantoader , yes we did change some access rights after a PR was merged to master without code review -- but we were too strict! You should have the correct rights now, let me know if the problem persist.

[codecov]

Codecov Report

Merging #119 (e9271d3) into master (1cf74ec) will decrease coverage by 0.27%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #119      +/-   ##
==========================================
- Coverage   98.39%   98.12%   -0.26%     
==========================================
  Files          15       15              
  Lines         744      744              
==========================================
- Hits          732      730       -2     
- Misses         12       14       +2     

Impacted Files	Coverage Δ
simSPI/tem.py	95.91% <0.00%> (-1.63%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1cf74ec...e9271d3. Read the comment docs.

[bogdantoader]

Not sure why the test coverage is lower - the "impacted file" tem.py is not touched by these changes

[ninamiolane]

Very nice, thank you for this addition! LGTM.