The future direction of the flamel

[xiki-tempula]

@orbeckst @harlor @danijoo
I think it might be good to have a discussion about the future of flamel and its relationship with alchemlyb, before committing to it.

My vision of the alchemlyb would be that it would have two parts, the library part and the workflow part.
The workflow part currently only have the ABFE workflow (alchemistry/alchemlyb#114), which provides the same interface as the alchemical-analysis.

>>> workflow = ABFE(units='kT', software='Gromacs', dir=dir,
>>>                 prefix='dhdl', suffix='xvg', T=298, skiptime=10,
>>>                 uncorr='dhdl', threshold=50,
>>>                 methods=('mbar', 'bar', 'ti'), out='./',
>>>                 resultfilename='result.out', overlap='O_MBAR.pdf',
>>>                 breakdown=True, forwrev=10, log='result.log')

The goal of Flamel will be to transform this python interface into CLI.

There are obviously reasons against this assignment as it would be a departure from the alchemlyb philosophy & principles

It includes functions for parsing data from formats common to existing MD engines, subsampling these data, and fitting these data with an estimator to obtain free energies.
These functions are simple in usage and pure in scope, and can be chained together to build customized analyses of data.

alchemlyb seeks to be as boring and simple as possible to enable more complex work.

However, there are also reasons for this assignment.

The first would be to avoid code duplication. From what I can see, the decorrelation part will be a duplication of the alchemistry/alchemlyb#98.

The second is that when more workflows are available, it will be easier to move part of the code that is used by many workflow (e.g. the unit conversion) from the workflow part to the library part, instead of moving the code from one Github repo to another repo.

The third reason is that it will be easier to release Flamel. The current Flamel has a plugin structure that mimics alchemlyb. So there are two questions, do we want to write the doc for each plugins and do we want to test these plugins? Once alchemistry/alchemlyb#114 is merged, the Flamel could be refactored into a simple CLI and we might not need tests and dedicated docs to release it.

[dnlbauer]

fair enough. I wasn't aware of the alchemlyb workflows but I think this is a good direction to circumvent further code duplication.

With the current state of flamel and it's plugin system, there is already a lot of unneccesary code wrapping around what alchemlyb already provides. Therefore, a more lightweight Flamel that simply passes CLI args to workflows sounds like a good idea to me. I think there should still be tests, but ideally they can be limited to integration tests for CLI parsing.

[xiki-tempula]

@danijoo Sorry for the confusion. After I had a review of your PR, I tagged you in that workflows PR (alchemistry/alchemlyb#114 (comment)) but I guess GitHub had another glitch that it didn't reach you.

When the workflow PR is merged or we eventually decided that the better place for it is here, your contribution is very welcomed.

I wonder what is your preferred software? The current workflow has been tested against Gromacs but it would be extremely helpful if users of other software could chip in.

[orbeckst]

Perhaps we should have a conference call for discussion, @xiki-tempula @danijoo (and @dominikwille / @harlor ?).

@alchemistry/alchemlyb (@dotsdl @davidlmobley @mrshirts @jchodera et al) do you have an opinion on how to move forward with alchemlyb/flamel ?

Our alchemlyb roadmap for alchemlyb itself has been broadly completed except the CLI part (see also MobleyLab/alchemical-analysis#111 ). But an immediate question is if we should include more complex analysis workflows in alchemlyb itself, inside flamel, or possibly as their own package (see my current opinion in alchemistry/alchemlyb#114 (review)).

I'd be happy to host a call for a discussion of the next steps for alchemlyb. Please say something here if you want to be included. Please also feel free to simply voice your opinion.

[mrshirts]

Hi, all-

I'd be happy to have a call to discuss future plans for alchemlyb. I have some thoughts an opinions on workflows (some of which I probably should have brought up earlier). If there are other issues/etc that it would be useful for me to look at before then, that would be great.

In terms of workflows - I think it depends what one means. Anything that is adaptive (run simulations, do analysis, then run more simulations) gets very complicated, and very likely to break on corner cases, and will require a lot of support. A reason for this is that the analysis usually requires specific physical information that is relatively easy to abstract from the simulations files, whatever they are, since they are configurations and energies, but running a simulation requires a lot more decisions and is package-dependent. I would prefer alchemlyb to focus on things that are just analysis, and perhaps anything that wraps around distributed separately.

In terms of just analysis, then flamel I could see belonging within alchemlyb, since it just analysis; but it's fine to be in a separate repository as well (and perhaps cleaner). Anything that is adaptive, i.e. runs simulations, should be somewhere else.

I can think of potentially a corner case where one would want to do reweighting, i.e. automate taking a trajectory that has been produced and calculating energies at new states -- but even that requires too much information about running the simulation.

So maybe the most complicated thing that might make sense is reweighting of configurational variables, like for example expectation of radius of gyration - where one would need to call, for example, md_analysis on the trajectory in order to do the reweighting. But that ONLY requires handling the simulation output, not the simulation input.

[orbeckst]

For clarification: The word "workflows" was used in a very limited sense of stringing multiple alchemlyb analysis building blocks together (see PR alchemistry/alchemlyb#114 for context). At this point the discussion did not include directing simulations.

[dnlbauer]

Hi,

Regarding the question if the workflows should be part of flamel or alchemlyb, my opinion is: none.

On one hand, I can definitely see people using these workflows outside of a CLI environment to build their own toolset, ie for adaptive sampling etc. Therefore, I think flamel is not the right place to put them at all. Ideally, flamel can be kept very lightweight with just a few integration tests to see if it uses the logic of the workflow/alchemlyb code in the intended way.
However, as you, @orbeckst mentioned in alchemistry/alchemlyb#114 (review), the workflow code is quite specialized and complex. Therefore, I feel like incorporating the workflows in alchemlyb itself will come with a lot of additional complexity that will make alchemlyb harder to maintain and to extend for future contributors.

I feel like a new repository for the workflows (alchemlyb-workflows?) would be a good approach. It has the benefit of keeping both flamel and alchemlyb lightweight. In a separate library, more specific workflows can be developed without having to think about the question if they are too specialized for alchemlyb or not. From a developer point of view, it also has the benefit that the three projects can be developed independently/in parallel and with different versioning.

[davidlmobley]

I don't have any concrete plans on my end right now, but want to be in the loop, and may get involved in more concrete planning in the next couple of months.

[harlor]

Actually, I don't have a strong option on where the workflows should be implemented. But I agree that it is nice to have it implemented in such a way that one can use them w/o the CLI.

Even-though I'm not sure that I'll be able to contribute much to the future development. I'd be happy to join the workflow discussion call.

[orbeckst]

If you are interested in participating in a discussion please indicate your availability in the Doodle poll: Future of Alchemlyb. Once I have a time I'll add a Zoom link to this issue. Even if you don't participate in the poll you'll still be welcome to join the call.

[dotsdl]

At the cost of the complexity getting shifted to another place, I'm in favor of an alchemlyb-workflows package approach. I think there is value in keeping the scope of alchemlyb fairly limited. As others have stated, for maintainability, ease of use, etc.

It can also make it easier to move quickly for those interested in workflow development, as they won't need to pass as high a barrier to progress for getting things into alchemlyb.

[orbeckst]

I'll close the poll on Monday 8/23 morning. Thanks to everyone who already responded.

[orbeckst]

We will have a Zoom call on Wed 25 August 2021, 21:00 UTC = 2pm AZ = 2pm Los Angeles = 22:00 London (UK) = 23:00 Berlin (Germany).

Everybody with an interest in alchemlyb is welcome.

Below is what I would use as the agenda. Please feel free to add/change/discuss.

Preliminary Agenda

1. introductions & interest in alchemlyb
2. Gather opinions on what alchemlyb ( = sum of alchemlyb, alchemtest, flamel, ...) should be. What is its mission? Do we need to update the Core Philosophy and API Principles?
3. Decide where "workflows" (in the sense of stringing multiple alchemlyb analysis building blocks together, as in PR Implement the workflow pipeline alchemlyb#114 ) should find a home.
4. Develop a rough roadmap for
   • alchemlyb
   • flamel
5. If any work needs to be done, find volunteers to do it.

Zoom call info

Oliver Beckstein is inviting you to a scheduled Zoom meeting.

Topic: future of alchemlyb
Time: Aug 25, 2021 02:00 PM Arizona

Join from PC, Mac, Linux, iOS or Android: https://asu.zoom.us/j/88958628322

Or Telephone:
Dial (for higher quality, dial a number based on your current location):
US: +1 602 753 0140 or +1 669 219 2599 or +1 669 900 6833 or +1 720 928 9299 or +1 971 247 1195 or +1 213 338 8477 or +1 253 215 8782 or +1 346 248 7799 or +1 312 626 6799 or +1 470 250 9358 or +1 470 381 2552 or +1 646 518 9805 or +1 646 876 9923 or +1 651 372 8299 or +1 786 635 1003 or +1 267 831 0333 or +1 301 715 8592
Meeting ID: 889 5862 8322
International numbers available: https://asu.zoom.us/u/kGua2XhAG

Or iPhone one-tap (US Toll): +16027530140,,88958628322# or +16692192599,,88958628322#

[orbeckst]

Please see discussion (not an issue) alchemistry/alchemlyb#159 for the summary of the meeting. Feel free to continue the discussion there.

P.S.: I enabled alchemlyb/discussions as a forum outside the issue tracker.

[orbeckst]

We decided to add "robust and general workflows" to alchemlyb (see Mission and API principles) as discussed in alchemistry/alchemlyb#159 .

Following up on @xiki-tempula 's original #8 (comment) and #8 (comment) , the likely direction is to turn flamel into a thin CLI. I suggest the following, summarizing the discussion and adding my own take:

1. Because flamel can use the soon-to-be-available alchemlyb.workflows module (in particular PR Implement the workflow pipeline alchemlyb#114 ), we don't need the current plugin structure anymore. Instead, workflows just have the appropriate kwargs to choose decorrelation etc and flamel's job is to pass the correct commandline arguments to the workflow functions.
2. I would prefer if output functionality were still be handled by flamel because alchemlyb should remain simple in that it only deals with DataFrames as data.
3. flamel will need some integration tests (add tests #9 ).
4. flamel will remain in its own repository and just install alchemlyb as a dependency (once we have an installable package make flamel pip installable #3 ). We do not want to bundle flamel with alchemlyb so that alchemlyb can be easily installed as a pure library (and so that we can keep the testing separate).