tensorflow 2.6.0

[xhochy]

Fixes #130
Fixes #129

Checklist

• Used a personal fork of the feedstock to propose changes
• Bumped the build number (if the version is unchanged)
• Reset the build number to 0 (if the version changed)
• Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• Ensured the license file is being packaged.

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[hmaarrfk]

Seems like OSX Arm64 is failing. Is it due to conda-forge changes or tensorflow changes?

[xhochy]

Seems like OSX Arm64 is failing. Is it due to conda-forge changes or tensorflow changes?

It's been like this all the time as Tensorflow doesn't support cross-compilation. I will manually build on that and upload.

[h-vetinari]

Should this be <2.7?

[xhochy]

No, as far as I understood the ~= operator this is correct: https://github.com/tensorflow/tensorflow/blob/919f693420e35d00c8d0a42100837ae3718f7927/tensorflow/tools/pip_package/setup.py#L106

[h-vetinari]

Aside from a small nit, this looks pretty good so far. Thanks a lot!

[xhochy]

There is one caveat here that is not visible in CI: tensorflow and tensorflow-estimator nowadays have a cyclic dependency. tensorflow-estimator is a run dependency of tensorflow and tensorflow itself is a host dependency of tensorflow-estimator. My suggestion would be to add the build of tensorflow-estimator to the feedstock here to simplify the handling of that. Any opposition from @conda-forge/tensorflow-estimator ?

This has been for this quite a while but nowadays that need the latest version of each other.

[hmaarrfk]

you are somewhat running out of time in the CIs as is

[hmaarrfk]

i don't have a better idea though

[xhochy]

you are somewhat running out of time in the CIs as is

Nothing new, all tensorflow builds were CFEP-03 manual uploads.

[hmaarrfk]

i see

[wolfv]

@xhochy anything I can do to help this? :)

[xhochy]

@xhochy anything I can do to help this? :)

I've pushed my latest changes. I hope this makes pip check passing and then I could start building all variants locally. If it fails though, it would be nice if someone could have a look at the issue.

[xhochy]

so as not to loose the momentum. I can probably build the linux builds, but who can build the osx builds?

I can do all of them. The Linux builds are easy. Sadly, I need ~2 days for the osx-64 builds, this is the big hassle. I also cannot start them before Monday probably.

[xhochy]

Actually, one thing that is a bit problematic to me is that when we merge this PR, a random number of builds may succeed on Linux. How should we handle that?

[izahn]

Merge it and wait to see which ones succeed. No need to manually build them!

[hmaarrfk]

anaconda upload --skip

should do the trick.

I'm just going to start to build in a script sequentially.

[wolfv]

I can start a OSX build as well if you want me to

[xhochy]

I can start a OSX build as well if you want me to

@wolfv Cool, can you do osx-64? I can do the ones on osx-arm64.

FYI, I'm using this script to manually build things in a clean (and automated) way on all OSes:

#!/bin/bash

set -exuo pipefail

VARIANT=$1

gh repo clone conda-forge/tensorflow-feedstock
pushd tensorflow-feedstock
gh pr checkout 131
OSX_SDK_DIR=/opt ./build-locally.py ${VARIANT} 2>&1 | tee ${VARIANT}.log
gh gist create ${VARIANT}.log

[wolfv]

I've started a osx-64 python 3.8 build.

Did you ever try cross-compiling from M1 to x64? My x64 osx is a bit slow(ish) so maybe from the M1 could be faster :)

[xhochy]

I've started a osx-64 python 3.8 build.

Did you ever try cross-compiling from M1 to x64? My x64 osx is a bit slow(ish) so maybe from the M1 could be faster :)

Cross-compiling definitely doesn't work but it could work when building in Rosetta 2. I fear that this will still break some stuff (at least it did with the non-tensorflow parts of jaxlib). This is why I'm looking for someone with an Intel Mac 😂

[hmaarrfk]

I'll probably be done building linux by tonight. Do you want me to use some uploading service to give you the artifact directories?

[hmaarrfk]

Linux Builds, full build artifacts:
https://we.tl/t-KWgTWU0WHL

[wolfv]

I've got the first osx-64 py 3.8 build done, now moving on to 3.9. For now I've uploaded the files to Google Drive and collected the SHA256 sums.
Logs are here: https://gist.github.com/wolfv/b52af0f1a4fd36c3654b8b543545fe5e

[wolfv]

Is there a PR for changes to also build a GPU variant, at least for Linux?

[izahn]

#120 worked but but is very difficult to reconcile with the non-cuda recipe. We discussed making a separate branch for the cuda build but didn't yet do it.

[izahn]

I've been running my own cuda-enabled builds from https://anaconda.org/izahn/repo in case that helps someone. I would much much rather get that into conda-forge, but my effort has stalled out for the moment.

[wolfv]

@izahn thanks for letting me know! I looked at your branch and was wondering if with the latest tensorflow the different config mechanism would still have to be used (or if we could just rely on the exported env vars). I probably need to dig a bit ...

It would be great to get some cuda enabled builds into conda-forge!

[isuruf]

This is why I'm looking for someone with an Intel Mac

@xhochy, we do have one available at conda-forge. Let me know if you need access.

[wolfv]

I'm building the last variant currently (py 36) for OS X 64

[xhochy]

For the CUDA enabled builds: Let's get 2.6.0 out and rebase the changes on top of that. While looking after Tensorflow issues for fixing this release, I saw some changes regarding CUDA and compiler toolchains.

[xhochy]

osx-arm64 builds are up on my anaconda.org account: https://anaconda.org/uwe.korn

Logs for 3.8: https://gist.github.com/xhochy/15e6af69161da1c72936e41f6cca23e2
Logs for 3.9: https://gist.github.com/xhochy/4184b1b0b5a6b5605d9822b1018e1672

[wolfv]

Here are my osx-64 builds https://drive.google.com/drive/folders/18DB55XqRpGwZlCC91SqgoZlzPGdg8DrY?usp=sharing

Logs & Checksums:

https://gist.github.com/9314d1a8f054311d9511815f28c503af

a3772e7b7a5792162058b29b9bab1311f584d6bf5151a033c9be6759c325dfa8  tensorflow-2.6.0-py39h6e9494a_0.tar.bz2
78507d1cf3057e2adce2e37d5860d3c793144ebfc8c737546b51d731f927606a  tensorflow-base-2.6.0-py39h9e0eb93_0.tar.bz2
a54b1aba6c7a3d81d1f716b78656537fb7c87de129f3ecbf1a58027d777b8264  tensorflow-estimator-2.6.0-py39h9e04aea_0.tar.bz2

https://gist.github.com/b52af0f1a4fd36c3654b8b543545fe5e

d13a389ea47061f8bad0bd96b49d7771e84653f5b970f692b5e7ca60f47dd2d7  tensorflow-2.6.0-py38h50d1736_0.tar.bz2
5b842dbb96b7aea7d101fef503681756d29e4810b74934e51f8eb46a7e31729a  tensorflow-base-2.6.0-py38h8860697_0.tar.bz2
a1c08347f4b99e6af665e370d948d1f29926796952063913bd042c1b85faf18a  tensorflow-estimator-2.6.0-py38h5519746_0.tar.bz2

https://gist.github.com/d337cf7fe565a199859b473916591266

e63b493936b3902aac46bc0ac228a447012f40d94f0e08a1b0d926b53a84a4df  tensorflow-2.6.0-py37hf985489_0.tar.bz2
61d6e06b8ebc0eacfbb855b31f148e3443917b7be2e3e6ba5ce5ba37daf2f2d8  tensorflow-base-2.6.0-py37h05ab19a_0.tar.bz2
8a77b94ac5c44c7de9966d81bb16c0b11a2d315cc34bbb3a754150cc73c8cda1  tensorflow-estimator-2.6.0-py37hfc69ec5_0.tar.bz2

https://gist.github.com/c90615d1c4d81bee68ccd2fd417d68e2

96bde8487582006cb437f4aa3b5b2e996e0d10f9d1ce3481eba5258e1fd076f6  tensorflow-2.6.0-py36h79c6626_0.tar.bz2
fa81b25a46465c44fbd9f59b095f9618e99f57430353507a02b50b8b25f0047c  tensorflow-base-2.6.0-py36ha9b9f1f_0.tar.bz2
0edfb0dec4ba450d91f327ba122494168d1652a7e751285979afdb86222b6341  tensorflow-estimator-2.6.0-py36h7641f05_0.tar.bz2

Happy to also upload them to a channel!

[xhochy]

Happy to also upload them to a channel!

That would be quite helpful to easily transfer them later into conda-forge.

[xhochy]

@conda-forge/core Please approve the manual build upload here

I have made the Linux builds with the following logs:

✓ Created gist linux_64_python3.6.____cpython.log
https://gist.github.com/4236435b43f603e5109c8c9242fbefab
✓ Created gist linux_64_python3.7.____cpython.log
https://gist.github.com/f5ebc755de593a0f1d8e644c46facdd3
✓ Created gist linux_64_python3.8.____cpython.log
https://gist.github.com/2527f5740340d46a6fd830baafae04b7
✓ Created gist linux_64_python3.9.____cpython.log
https://gist.github.com/ce59bde517d382cb6235a3294e4e706a

together with @wolfv's osx-64 builds from #131 (comment) and my osx-arm64 ones #131 (comment) they are now part of my personal Anaconda channel.

[beckermr]

Thank you everyone!