Issues running the Binder #36
Labels
Comments
|
Thanks for opening your first issue here 💖! If you have any questions, feel free to mention one of the conveners, previous contributors, or attend our weekly meeting (see https://hepsoftwarefoundation.org/workinggroups/training.html). Also, sometimes issues go unnoticed, so don't hesitate to @mention some of us, if we do not come back to you within a few days. |
|
The installing pip dependancies step is what takes exceptionally long. |
|
MyBinder.org had to reduce its computing capacity (which would lead to longer compute times, since it works by heavily oversubscribing, not dedicated resources) because it lost a major funder (Google). We can't say the cause of a single instance is due to a larger trend (similar to pinning a particular hot day on climate change), but that may be the cause. Various tutorial/training projects have been relying on MyBinder.org because it exactly fits our needs: it doesn't need to be long-lasting, can be preloaded with all dependencies, etc. However, we might need to rethink that if MyBinder.org doesn't get back to previous funding levels. I've been investigating the use of GitHub Codespaces instead of Binder, and I'll be trying it out on my next tutorial. There are screenshots there (without instructions) walking through the process of getting a Jupyter notebook with all dependencies installed, creating an environment like Binder, but with dedicated hardware (usually 2 cores). GitHub Codespaces is a paid resource, with a large number of free hours per month. At least at first, it's unlikely that participants would be near their limit and we can do a whole tutorial without spending any money. But if all tutorials do this, then some students might go over threshold. It's also a GitHub thing and not (for example) a GitLab thing. It requires students to have a GitHub account. It's pretty likely that each student would have a GitHub account, but it's an additional restriction that was not the case for MyBinder.org—anyone with a web browser could launch it. I'm at least doing this experiment with my next tutorial, which is not HSF related. (It's at the SciPy conference.) But if it's successful, maybe you'll want to do something similar? |
|
I should have read this more carefully,
Binder shouldn't be installing dependencies for each user. When I set up this repo, I had Binder pinned to a fixed release and then loaded that fixed release once. Binder then made an image with all dependencies included in it, and that cached image should be what users get. It might be slow due to the MyBinder.org reduced capacity, but it shouldn't be installing dependencies. Maybe the Binder URL was pinned to a new release of this repo and then wasn't launched (manually, by whoever updated the URL) to preload the new image into Binder's cache? Or maybe Binder flushed that cache? Either way, the pip-installation should happen exactly once per cached image, which should in principle be once per release of this repo. That, by the way, is one of the downsides of GitHub Codespaces. As far as I can tell, you can't fill an image with a pip/conda environment: it's a first step for users. With the 2 dedicated CPU cores, that takes 4 minutes in GitHub, and I removed heavy dependencies like ROOT. |
|
I see, thanks for the explanation @jpivarski. Would you like for me to post here the Binder log? |
|
I wouldn't be against it, but there's nothing that I would use it for. We're not debugging Binder—if there was a real bug in Binder, that would go to the Binder team. But also, it's either that you were the first to try the Binder URL for a new release1 or Binder dropped it from their cache because it hadn't been touched in a while. (Maybe they have limited disk.) This repo doesn't have any releases! Every time a minor change is made (text in the README or anything), Binder will require a new image. If you're going to continue to rely on Binder, I'd recommend associating the Binder URLs with fixed releases, so that you can make minor changes without invalidating Binder's cache. But if you're moving away from Binder because of its funding problems, then the point is moot. Footnotes
|
|
Good hint with the Binder releases! Especially because we don't have any "real" notebooks, it's very very rare that we actually change something of importance for the binder |
|
But I just reproduce what @GuillermoFidalgo reported, it doesn't even start: The fact that the last pip uninstall message seems to be cut short sounds like there was a timeout. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Seems like there is something wrong with the Binder environment. I'm not sure what exactly it could be, but Binder takes too long to setup and eventually errors out with a "CondaEnvException: Pip failed" message.
The text was updated successfully, but these errors were encountered: