Widen cholesky rule to Hermitian and Symmetric matrices

[devmotion]

Seems to fix #1272:

julia> using Enzyme, LinearAlgebra

julia> g(C, X) = sum(C \ X)
g (generic function with 1 method)

julia> g2(A, X) = g(cholesky(A * A' + I), X)
g2 (generic function with 1 method)

julia> A = rand(2, 2);

julia> X = rand(2, 2);

julia> ∂g2_∂A = zero(A);

julia> ∂g2_∂X = zero(X);

julia> autodiff(Reverse, g2, Active, Duplicated(A, ∂g2_∂A), Duplicated(X, ∂g2_∂X))
┌ Warning: Using fallback BLAS replacements, performance may be degraded
└ @ Enzyme.Compiler ~/.julia/packages/GPUCompiler/U36Ed/src/utils.jl:59
warning: didn't implement memmove, using memcpy as fallback which can result in errors
((nothing, nothing),)

julia> g3(A, X) = g(cholesky(Symmetric(A * A' + I)), X)
g3 (generic function with 1 method)

julia> ∂g3_∂A = zero(A);

julia> ∂g3_∂X = zero(X);

julia> autodiff(Reverse, g3, Active, Duplicated(A, ∂g3_∂A), Duplicated(X, ∂g3_∂X))
┌ Warning: Using fallback BLAS replacements, performance may be degraded
└ @ Enzyme.Compiler ~/.julia/packages/GPUCompiler/U36Ed/src/utils.jl:59
warning: didn't implement memmove, using memcpy as fallback which can result in errors
((nothing, nothing),)

julia> ∂g3_∂A == ∂g2_∂A
true

julia> ∂g3_∂X == ∂g2_∂X
true

However, the big caveat is that I don't know if this is the correct approach.

[devmotion]

I'm wondering, should this check be changed as well?

[wsmoses]

Yeah the check should make sure we're not adding to itself

[wsmoses]

Fix that check otherwise lgtm

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1390045) 75.14% compared to head (d9042ad) 75.21%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1273      +/-   ##
==========================================
+ Coverage   75.14%   75.21%   +0.06%     
==========================================
  Files          35       35              
  Lines       10393    10394       +1     
==========================================
+ Hits         7810     7818       +8     
+ Misses       2583     2576       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[devmotion]

I updated the check 🙂

[wsmoses]

The 1.10 failure is new/real, but not caused by this PR -- but rather only caused [and first noticed] by a forward mode test added here.

The fix requires a jll bump which is currently blocked by: JuliaPackaging/Yggdrasil#8017. I've said on that thread that for now we should just disable Windows for LLVM16+ as hopefully one of the libLLVM_jll maintainers can fix windows on 16.