Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError in osys.update_leaves() #123

Closed
Jashcraf opened this issue Sep 30, 2022 · 25 comments
Closed

TypeError in osys.update_leaves() #123

Jashcraf opened this issue Sep 30, 2022 · 25 comments
Labels
bug Something isn't working

Comments

@Jashcraf
Copy link

Hi there! New user to dLux and am trying to run the Phase Retrieval in dLux demo on Google Collab.

I'm just copy-pasting the code into a collab notebook, but when it's time to run this block of code I get a TypeError from osys_update_leaves()
image

Could anyone point me to what might be going wrong?
@Jashcraf
Copy link
Author

Got the same error in the Phase mask design demo, I suspect this is a me problem and not a dLux problem.

image

@Jordan-Dennis
Copy link
Collaborator

Hey @Jashcraf,
This is a strange error 😆 that I've not seen before. Would you mind sharing the python version that you are using (typing python --version in the terminal will work). I just ran the notebook up to that point on my machine with python==3.10.4 and it seemed to work. typing the module causing the error is part of the standard library so we may need to include a python version in requirements.txt, if this is the problem.

Thanks for the issue

Jordan

@Jashcraf
Copy link
Author

Hi @Jordan-Dennis 👋
I'm using python version 3.7.14 right now because I'm trying dLux out on google collab. I can try install 3.10.4 to see if I get the same error.

@Jashcraf
Copy link
Author

Maybe I can't, it looks like Google Collab only supports Python 3.7 and 3.6

Does dLux work on M1 chips? I tried importing Jax and it was a little rude to my poor computer.
image

@Jordan-Dennis
Copy link
Collaborator

Hmmmm, should work on M1, @LouisDesdoigts and @benjaminpope are both running on MAC.

@Jordan-Dennis
Copy link
Collaborator

I just checked and it is limited to CPU is the only downside.

@Jashcraf
Copy link
Author

Oh that's fine I'm not trying to do anything crazy, just optimize some thin films to make finding the aliens a little easier

I saw that I might need to build it separately jax-ml/jax#5501 (comment)

@Jordan-Dennis
Copy link
Collaborator

Jordan-Dennis commented Sep 30, 2022 •
edited
Loading

Yeah, this stack post links in a whole lot of resources but that is the best I can do I'm afraid. We might change our use of the typing module anyway if it will cause problems with Collab. Its mainly a synthetic thing and has a very minimal runtime type checker which you have run into obviously, 😆

@Jashcraf
Copy link
Author

Long ago I accepted my curse of running into trouble installing packages every time I start a new project - I will prevail.

Thanks for the help! I'll give this a shot :)

@Jashcraf
Copy link
Author

Jashcraf commented Oct 3, 2022

Steps I used to build Jax from Source taken from Jax's documentation

Build jaxlib

  1. git clone https://github.com/google/jax
  2. cd jax
  3. Install Xcode & command line utils from App store
  4. Realize that you need to update your OS
  5. Play Assasin's Creed for 4 hours while your OS updates because it's a weekend and you deserve it
  6. Install Xcode
  7. python build/build.py
  8. doesn't execute successfully.
  9. panic

Well let's try the method outlined in this Stack Overflew thread.

  1. Create new conda environment
  2. conda install numpy; conda install six
  3. pip install -U pip pip install -U https://storage.googleapis.com/jax-releases/mac/jaxlib-0.1.75-cp310-none-macosx_11_0_arm64.whl
  4. Not a supported wheel on this platform, panic again
  5. python build/build.py --target_cpu darwin_arm64
  6. Doesn't build successfully
  7. pip install --upgrade "jax[cpu]"
  8. Doesn't build successfully

Going to try coming back to this again later.

@benjaminpope
Copy link
Collaborator

benjaminpope commented Oct 3, 2022 via email

@Jordan-Dennis
Copy link
Collaborator

Jordan-Dennis commented Oct 3, 2022 •
edited
Loading

I know that it is a bit late, but is there any chance you could repeat the error. I'm curious as to the 4 hidden frames since we don't actually use generic types. This seem's like something that might come from equinox perhaps and if it is you might be better chatting to @patrick-kidger from equinox.

@Jashcraf
Copy link
Author

Jashcraf commented Oct 3, 2022 •
edited
Loading

@benjaminpope trying to install from conda forge with a fresh environment. Had to downgrade to < numpy v1.23, so I switched to 1.22.

Tried import jax and it raised an ImportError.

Got an error that I'm having trouble understanding.
image

If I run the cell again it raises an AttributeError
image

I see that I didn't install Python 3.10.4 like @Jordan-Dennis recommended, so I'm upgrading using conda install python=3.10.4. During the installation there were some apparent conflicts with jax, so I'm just going to try start from the top and re-install jax.

Same errors unfortunately with import jax in python 3.10.4.

import jaxlib seems to work just fine still.

If it's any help, it seems that I've installed jax and jaxlib versions 0.3.20

@benjaminpope
Copy link
Collaborator

Let us know how that goes... the error messages indicate you're using Python 3.8 which does seem like the culprit. I would suggest creating a conda environment with Python 3.10, and using conda-forge to install jaxlib then jax

@Jashcraf
Copy link
Author

Jashcraf commented Oct 3, 2022 •
edited
Loading

Here's something weird (that's probably just my conda inexperience), when I create a new conda version and install python 3.10 checking the version verifies that I have the right installation
image
Installing jax and jaxlib from conda-forge work just fine, but after the install when I try to open python to test the jax import, it shows as python version 3.8
image

Not sure where in the pipeline the python version is changing, but I'll keep looking

UPDATE: It happens at least after jaxlib is installed, appears to be changing my python version back to 3.8.8
image

@Jashcraf
Copy link
Author

Jashcraf commented Oct 3, 2022

Installing from pip with a fresh python 3.10 conda environment doesn't change the python version, but we return to the error where my CPU and OS are not supported.
image

@Jashcraf
Copy link
Author

Jashcraf commented Oct 3, 2022

This is turning into maybe a problem where I can't install Jax, rather than being a dLux problem, so maybe it's worth closing the issue so I stop bugging you all?

@Jordan-Dennis
Copy link
Collaborator

Hahaha, I don't mind, we like having users and these are things that we need to think about. If you are able to work it out we will probably use it in an installation troubleshooting guide so it is helpful to us. I would be a little scared of the Mlir errors. This is the intermediate language representation that google uses to compile XLA. Running from ubuntu which is not so different from MacOS I am able to just run pip install jax and pip install jaxlib without any problems in a conda environment, although I think pip install jax[cpu] might be the recommended one.

@Jashcraf
Copy link
Author

Jashcraf commented Oct 3, 2022

So I also have a windows machine, and somehow I've gotten it working on that using this method of installing jax https://github.com/cloudhan/jax-windows-builder

The phase mask design example executed like a charm!
image

I'll continue sleuthing tomorrow, or maybe open an issue on Jax's github to see if they can figure out what might be going wrong.

@Jordan-Dennis
Copy link
Collaborator

Jordan-Dennis commented Oct 3, 2022 •
edited
Loading

Haha, that was a process, good job 👍. I'm super busy at the moment but I want to look into your original problem with the typing module and find out if it was our usage or equinox's that caused the problem. If it is a problem with us then we might just ditch typing in the relevant place. If it is a problem with equinox I will raise and issue with @patrick-kidger.

I'll leave the issue open for now to remind myself, and since you didn't actually get it working on your Mac.

@LouisDesdoigts
Copy link
Owner

So I suspect that @Jashcraf may be running this through a Rosetta version of bash/zsh. I can't see why else it would possibly give a not supported error for the ARM wheel when running on an ARM architecture.

As for the typing, I think this may have been due to running the pip version of ∂Lux with the new tutorials. I could be wrong, I'll need to try and reproduce the error myself (unless @Jashcraf has the notebook handy)

@benjaminpope
Copy link
Collaborator

Time to push an update to PyPI?

@LouisDesdoigts
Copy link
Owner

Its always time to push an update to PyPi, there's too much dev happening 😂.

Next PyPi update is the 1.0, that's my goal.

@benjaminpope
Copy link
Collaborator

I'd probably push a version a bit sooner than that if we're doing collaboration with Jaren et al

@LouisDesdoigts
Copy link
Owner

Will push a new version by the end of the week - The new source/instrument stuff will force another tutorial update anyway.

@Jordan-Dennis Jordan-Dennis added the bug Something isn't working label Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@benjaminpope @Jashcraf @LouisDesdoigts @Jordan-Dennis