Move core example model back #2349
Conversation
…t of unnecessary dependencies
* Use Pathlib Code by Phil Robare (versilimidude2) * Use pre-commit * Dropped unused imports * used pre-commit to run pyupgrade, trim whitespace * Remove pathlib changes These belong in a separate PR. * remove redundant black, move comment as per rht's suggestions --------- Co-authored-by: Catherine Devlin <[email protected]>
…ojectmesa#30) Co-authored-by: Jeremy Silver <[email protected]>
* fix bug where agent gives money to itself in the boltzmann model example * [pre-commit.ci] auto fixes from pre-commit.com hooks --------- Co-authored-by: Houssam Kherraz <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
… option; modified most other examples in the same way
* Implement Streamlit UI for Boltzmann wealth model * change slider text * change slider text * update readme --------- Co-authored-by: Ankit Kumar <[email protected]>
See PR projectmesa#36 for the app.py implementation if it. * Initialize Game of Life example * set slider text * udpate readme * Implement Streamlit UI for Boltzmann wealth model * resolve merge conflicts --------- Co-authored-by: Ankit Kumar <[email protected]>
* Bump to Mesa 2.0 * Update to Mesa 2.0 API * Update coord_iter to Mesa 2.0 API --------- Co-authored-by: Tom Pike <[email protected]>
|
Performance benchmarks:
|
|
I agree with this move. However, I also agree with the current description of mesa-examples: Seminal agent-based models developed using Mesa. So, what are seminal ABMs?
In my view, we should have a single emblematic version of each model. For some, we can go with the basic version (e.g., Schelling, Boltzmann); for others, we might go a bit beyond the standard. For example, we could to wolf sheep with event scheduling (as the benchmark version) and do sugarscape with traders, and use propery layers for sugar and spice regrowth. I appreciate the massive effort to retain the git history. However, if we keep mesa-examples and turn this into a show and tell gallery that is not actively maintained (perhaps add an environment file to each example?) I am not sure we need to keep this full history here. |
Agreed. The big question is, where do we keep them? My intention is to have not duplicates, or at least the least amount possible.
This might be an interesting distinction. We could have:
This way we don’t have duplicate models, but do have a nice split between stable and experimental. So maybe all models that we maintain to our highest standards should be in Mesa itself. Others, which aren’t maintained (by us), and thus more user showcases, can be in Mesa-examples. We might “degrade” Mesa models to Mesa-examples from time to time if we feel we have too much examples to actively maintain. |
Yes exactly
Yes, this is exactly what I am advocating with one addition: our examples in the main repo should be emblematic ABMs that everyone familiar with ABMs knows and have been covered extensively in e.g., popular scientific books on complexity science. |
|
Overall, I disagree.
Breaking changes are not meant to be frequent. As such, this is not a basic need. Even the NumPy 2.0 migration guide is not that drastic. People need stability, as this is not web frameworks. Otherwise, tutorials, courses, LLM knowledge all are being made obsolete too often.
This is monorepo vs polyrepo again. It's not as clear-cut as you have painted so far, i.e. monorepo good polyrepo bad.
Testing the examples should be localized to these examples, and they already have a weekly CI to catch bugs from projectmesa/mesa changes. While unit testing the visualization elements in general should be automated according to Solara's way of testing.
This is an argument to improve pedagogy, which I could support. This does sound more convenient than I haven't had the time to summarize the pros and cons in #134 and #963. |
|
This is more complex than one might think for the reasons already stated. My hesitation is that people needed help finding the examples. We need to discuss what our users need from examples and how to make them usable. To be clear, I am not saying 'no,' but I also do not approve at this time. |
|
Thanks for both your insights!
This has nothing to do with if a change is breaking or not, having the examples in the main repo is useful for any change. It allows you to directly show how a new feature can be used in an example, even if it also would have worked the old way. This makes the PRs which we link to in the release notes also way more useful, since they now contain the code changes and how it can be used in the examples.
Okay, but could you than argue for the polyrepo?
Thanks!
Looking forward to it!
Could you expand a bit on which reasons or point to something?
I think this will make it easier for people to find the examples, especially if we document it well. Instead of installing something extra, and making sure its in the right place, etc., you can now just directly import them. from mesa.examples import BoltzmanWealthAnd they show up directly in your IDE. @quaquel so that we should make three lists:
My initial division would be: BasicComplexShowcase |
|
Why am I in favor of having examples in the main mesa repo?
Can you give some short motivation for this breakdown? I personally would include epstein (and el farol) also in basic. The main reason for me for keeping them in the mesa repo is that these are canonical ABMs. I would also include a much expanded version of pd_grid (which is just Axelrod emergence of collaboration on a grid) in advanced. Again, this is is a canonical ABM and enables showing off some nice Python / MESA features. You currently have virus on a network under advanced, but the code is very basic ATM. |
|
I agree with putting canonicals model in the main repo. My main reason for this view is slightly altering Kurt Vonnegut's writing quote is "have mercy on the user," which falls under the pedagogical justification. From our current understanding our main user base is coming from academia. The more we can make Mesa an easier tool for them to use the better. Generalizing, however, the user is the target and this same dynamic goes for them. Being able to rapidly reference canonical works which serve as a examples or "How Tos" is critical to enabling education per scaffolded learning. As we expand Mesa and get into higher end models different backends (rust etc), use of HPCs etc, then we can optimize for a more diverse user community. Ideally after Mesa 3.0 is released we will do a push on our docs, best case we end up with something like Solara https://solara.dev/ Regarding the monorepo vs polyrepo, I am intrigued but cant see the path yet. When I can see the specifics as described here, then I can make an informed recommendation. However, I put this in here to emphasize the hard problem for ABMs is how do we build a sustainable ecosystem so ABMs are as easy to build as say the MLs in scikitlearn. If we figure this out then we can get past boutique models and ABMs will become ubiquitous. Either way this is a great discussion! If you are not having hard conversations, you are not talking substantive change, and the great news is we can always change again! :) |
|
Thanks for your extensive reply. Let me clarify one thing: This PR doesn’t make this a “monorepo” or whatsoever. Monorepo have multiple projects/packages in a single repository, this is still one project and one package in one repo. So the discussion about Mesa (mono)repositories is largely unrelated. I might have not communicated that clearly earlier, sorry. If we discuss/decide to move Mesa-geo, Mesa-frames, etc into this repo, that becomes relevant again. Let’s focus on moving the examples back in this PR discussion. @quaquel I generally don’t have strong opinions about which exact examples go where. Can you come up with another division that you can get behind? |
|
My breakdown would be: Basic
Complex
In stand alone show and tell repo
|
It's bikeshedding, and I shouldn't rehash the existing general arguments in this topic. But IIRC the motivation for Mesa-Geo and mesa-frames being a separate repo is analogous to why the scikit-* (as @tpike3 had raised) are a separate repo from SciPy. It's to grow an ABM ecosystem.
Should have been caught by the weekly CI. Also, it's understandable given that Mesa is undergoing lots of breaking changes recently, and shouldn't happen all the time.
I initially thought this PR as a Trojan horse to later justify putting the examples back here (it came from #2330 after all). But it seems to be a fair bargaining session instead. I wonder if we have a common ground that we want to deduplicate effort in maintaining the core/canonical/classic/basic examples (not sure). There are already performance benchmark examples, which sometimes uses experimental code instead of stable code. What is going to happen to them? How would the maintainers decide whether the core examples should use stable or experimental code? In any case, pedagogy should be prioritized first, which means to recommend the stable code. Which means we can't avoid having more than one Boltzmann/Schelling code as long as experimental API exist.
I thought the performance benchmark examples have been used for this purpose so far. |
|
Thanks for all the discussions so far. In general I'm hesitate about moving some examples back here. The main reason is that I share @jackiekazil's concern about the confusion this may cause to the users regarding where to find examples. This was also one of the reasons why examples were separated to its own repo previously. To elaborate further, I don't generally expect users to look into Mesa source code. Instead, they should be redirected to example code and readthedocs site. Having examples in two different places makes this more difficult. I understand that it would be easier to develop and test features with some examples back in. Hence to me, this is a trade-off between user experience vs. maintainer experience. In this case I would choose users over us maintainers. I also agree with @quaquel's concern that currently examples are not well maintained. But, moving some of them back in would still leave rest of the examples at the risk of under-maintaince. Instead I would suggest to look for ways of properly maintain all examples. But I'm not strongly against this PR. Similar to @jackiekazil, I'm not rejecting this PR, but I do not approve it either. |
Which is why I advocate for turning mesa-examples into more of a show-and-tell/gallery with environment files for each example. This has 2 benefits: we can accept examples more liberally, and, because of the environment file, guarantee a minimum level of reproducibilty. As developers, we are already spread thin, so I don't see a simple solution to improve maintenance of the current examples (let alone if they continue to grow as evidenced by the various outstanding PRs). |
In the EMA workbench, all examples are included in the documentation. This makes the examples very easy to find for all users. To avoid duplication, I have done the following there: when building the docs, the example folder is copied to the docs folder and the relevant files for read the docs are automagically generatored (all handled in conf.py). This implies that whenever an example is updated in the code base, or a new example is added, these changes or additions are automatically included in the documentation. For the benchmarks, we can easily do the same. Just copy the relevant folders from mesa.examples to the benchmark folder and run the benchmarks. This avoids duplication. |
And it might have, but that would always be after the actual change is made. Then a follow-up PR is needed to fix it, or revert it, while it could have been catched directly at the beginning in the PR itself.
In my vision those won't be separate, so we merge the example and benchmark models, so there aren't any duplicate ones. We probably benchmark some of the stable and some of the experimental models.
I think we can make a clear split here, like @quaquel also suggested:
A huge advantage from having them in the main repo, is we can (also like Jan suggested) directly in the docs. Even more, we can use them in tutorials and in (future) interactive demos. And of course we will link to the example folder in the Readme and docs. It just makes a ton of use cases possible if we can directly refer to them.
I don't think this is actually the case, and we can optimize what's best for both. There's also a transition area of users that are becoming contributors - which is a hugely important group for us. I think this will also help them, because the git-mess becomes much easier when working in a single repo. @quaquel, I like your division of models. So here's my updated proposal:
|
Clarifying question: benchmark all 9 or just the same as we currently do? |
|
In my vision same as currently, due to run time. Maybe we could add an "full-benchmarks" method at some point that allows running all models as benchmarks (but takes longer). |
|
Great discussion. I'll give my approval on this PR, regardless of how the final folder structure or example selection will look like. Although bringing some examples back into the main repo might at first look like it might have been a mistake to move them outside in the first place. But I think we made everything right there and I think the mesa examples repo is a success story. We now have a lot more examples using mesa and more are coming through open pull requests. But it also is apparent now that we can no longer fully maintain all of them. So moving some examples back now and providing full support for them seems like a sensible step and in case of the benchmarks even reduces example duplication. And I also agree for the examples repo to switch to fixed requirement files so we always have a known state of dependencies for which they work regardless of how mesa and other libraries evolve. I think we can do a lot more in terms of example discoverability and retrieval but this is a separate topic |
|
Thanks for your perspective Corvince! One interesting note is that we got multiple issues / questions filed (projectmesa/mesa-examples#217, projectmesa/mesa-examples#216, #2356) about examples not working. All those came from people trying to use Mesa 3.0 examples with Mesa 2.x. I think we can also improve this experience if examples are always bundled with the working Mesa version. And included in the docs, in which we now have a version selector (#2324). If I'm tracking this correctly, it looks like Tom, Jan, Vincent and I are now in favor of moving the seminal / core examples back into the main Mesa repo. rht is against. Jackie and Wang, in your last replies you were hesitant/undecided, is this still the case? |
|
Yup! |
Once again, this is not happening all the time. This could solved by checking |
|
Superseded by #2358, thanks everyone! |
This PR moves four core examples back to Mesa:
It uses the approach listed in #2330 (comment). It keeps the git history, with the only side effect that the PR numbers now link to the wrong repo (there's no easy way to fix this).
Moving these four core examples back into the main Mesa repository:
After this PR is merged, these examples can be updated further to become fully exemplary.
@projectmesa/maintainers I would request reviews of highest priority, since this git magic is very time intensive and I would like to know ASAP if I need to add other models.
To review:
Not merging before 3 green checkmarks, I want consensus on this.