Plan for a coding week

[danielhollas]

Here's a rough plan for my work on this package during the AiiDA coding.
I'll see how far I'll get, since I also want to work on AiiDAlab related things.

This is the list of tasks, roughly in the order they need to be done

• Fix CI issues and support new Python versions Support new Python versions #73
• Support new aiida-core versions Support aiida 2.5 #84 WIP: Support AiiDA v2.6 #85
• Rename the package to aiida-test-cache consider name change #60
• Create a GHA release workflow, and publish version 0.1 to PyPI, to be used by the current users of the package. Other breaking changes will be released in v0.2.0 Release automation #72
• Chore tasks: switch from Pylint to ruff
• Deprecate AiiDA v1 support, deprecate Python 3.7 and 3.8
• Support the new test fixtures with sqlite backend from aiida 2.6, ideally in a backward compatible manner. Use new pytest fixtures with SQLite #79
• Test using mock_code in aiida-orca plugin
• Test archive_cache in ATMOSPEC AiiDAlab app
• Extend documentation

[danielhollas]

CC @giovannipizzi @ltalirz @unkcpz Let me know if the plan above sounds good, and please tag people who you know might be interested in this (and potentially available to review the PRs).

[DropD]

What I would like to see is an integrated way to skip running a "mock code". Instead, I would like to get a calcjob node instance, which has the inputs and cached outputs connected to it. This could be used to unit test parsers, workchain logic etc. I've seen custom implementations for this in many plugins for testing parsers, including the two I (co-)authored, aiida-vasp and aiida-icon.

Having this integrated would be nice for two reasons:

• If you set up tests with aiida-test-cache you would get data for parser unit tests "for free"
• Depending on what you want to test, it might completely eliminate the need for an additional script, which mimicks the code

[ltalirz]

You seem to be describing what export_cache/archive_cache does.

Both the mock code and the archive_cache have use cases.

I'll paste here what I wrote to Daniel in reply to his email, perhaps this can be integrated in the repo README

Edit: Maybe I'm wrong about the "cannot test parsers" in export_cache- I've never used it

===
I would say aiida-testing solves the following problem in the testing of AiiDA plugins:
What do you do, if you can't run the executable your plugin is wrapping in your continuous integration tests? (because you need to test runs that are too expensive to run on the CI runners / because the license forbids it / ...)

It solves it in two different ways, each of which have their pros and cons:

• mock_code: documented well in https://aiida-testing.readthedocs.io/en/latest/user_guide/mock_code.html (still allows to test parsers, but is a bit of a hack of the AiiDA infrastructure)
• export_cache: use the AiiDA caching function (cannot test parsers, can only test workflow logic, but uses standard AiiDA functionality)

I do think that this use case continues to exist (aiida-vasp, ...) and requires a working solution.
Besides keeping it compatible with the new backends, perhaps one of the most impactful things to do would be to improve the documentation of aiida-testing to the point where someone can come to the plugin repo and understand what I wrote above.

[danielhollas]

Thanks both!

Just a note that I haven't made as much progress as I wanted as there's a lot of other stuff going on during the coding week, but I am interested in pursuing this beyond as well.

I currently got stuck on trying to get the tests pass for newer aiida-core versions that had quite a few changes to caching infrastructure. I need to find more focused time to dig into the internals.

One immediate thought that I had while digging through the implementation of the fixtures here: they seem to be mucking deep with the caching infrastructure of aiida-core, so even when I fix the issues with the new AiiDA versions, I am afraid that these fixtures would still be brittle. I see two ways to improve it:

1. Move these fixtures to aiida-core and support them there. Any change to caching would be immediately applied.
2. Provide necessary abstractions in aiida-core, so that these fixtures don't need to rely on the internals (or at least not as much).

I guess I am slightly leaning towards 2., since this repo has other stuff in it besides the fixtures (e.g. configuration management) so I don't know if it makes sense to integrate to aiida-core (at least not before things stabilize a bit).

Anyway, this is just my morning rambling, at some point I'll turn this into a separate issue.