Differential forms, chain rule, and automatic differentiation with mixedbasis

[goballooning]

Hi @chakravala,

In your JuliaCon paper, I noticed the statement "Mixed-symmetry algebra with Leibniz.jl and Grassmann.jl, having the geometric algebraic product chain rule, yields automatic differ- entiation and...", and I found a related simple example in the README.md file, section "Differential forms and Leibniz tangent algebra".

Would you mind providing one or two concrete examples on symbolic differentiation using Grassmann.jl? For example, with respect to a rotor/versor, or a bivector.
I'm thinking that the key part is about a proper setup of a @mixedbasis tangent .... Am I right?

I'm reading Chapter 8 of the book "Geometric Algebra for Computer Science", but there are just a few examples of basic functions. I would like to apply vector/multivector differentiation on some more functions, and verify my results.
I'm trying to differentiate a function like \sum_i weight_i || V Bivector_0 /V \wedge v_i ||^2 with respect to the versor V.

Thank you!

[chakravala]

Indeed, the automatic symbolic differentiation is the final finishing feature I would like to add before finalizing the paper I am writing. Everything is already in place for AD and differential forms, I only need to implement a feature to detect the correct moment to apply the differential operator to the symbolic expression. Due to the abstract generality of the algebra, this requires careful design.

You can see some examples on how to work with the Leibniz prototype in a related discussion. There is another related discussion also regarding the differential combinatorial graph theory.

Please consider donating at liberapay.org/chakravala if you want to support my work. This is a unique and novel computational implementation that has not been created or studied quite like this before. This is nearly a complete feature, for which I am currently designing the finishing touch for differential calculus with symbolic tensor fields.

[chakravala]

Alright, so I got the feature implemented now, need to register a new version of DirectSum first before the commits can be pushed on this repository.

[goballooning]

Thank you! I will definitely update and try it.
What's the next title on your book list? I'd like to buy you a book (as I can not buy you a beer for geographic reason).

Cheers!

[chakravala]

Thanks for your interest! Discrete Morse Theory is a new book I am interested in.

[goballooning]

It seems only available on ams.org. Do you prefer the electronic version or the print one? Drop me an email (lihongying17 @ google mail). Glad to support this project. :)

[chakravala]

Alright, so the functionality is designed around the concept of differential operators and tensor fields.

julia> using Reduce, Grassmann # can also use SymPy or SymEngine, or others if setup

julia> @mixedbasis tangent(ℝ^2,3,2); # 2D space, 3rd order, 2 variables

Here we load the 2D geometric algebra having 3rd order differentiability with respect to 2 variables.

This gives us some new Basis elements, e.g. ∂1 (differential operator) and ϵ2 (tensor field).

julia> ∂1 * (:(x1^2+x2^2)*ϵ2)
(2x1)∂₁ϵ²

julia> ∂1 * ans
2∂₁∂₁ϵ²

julia> ∂1 * ans
0vv

Roughly speaking, multiplying a differential operator and a tensor field results in the application of that particular differential operator on the tensor field with respect to its indices.

This feature is very new and hasn't been completely tested yet. It only correctly applies first order differential operators at the moment, so it will take a few more commits before it's fully correct.

[goballooning]

Repeating this example gives me an error:

julia> using Reduce, Grassmann
Reduce (Free CSL version, revision 4534), 05-Apr-18 ...

julia> @mixedbasis tangent(ℝ^2,3,2)
(⟨++--₁₂¹²⟩*, v, v₁, v₂, w¹, w², ∂₁, ∂₂, ϵ¹, ϵ², v₁₂, v₁w¹, v₁w², ∂₁v₁, ∂₂v₁, ϵ¹v₁, ϵ²v₁, v₂w¹, v₂w², ∂₁v₂, ∂₂v₂, ϵ¹v₂, ϵ²v₂, w¹², ∂₁w¹, ∂₂w¹, ϵ¹w¹, ϵ²w¹, ∂₁w², ∂₂w², ϵ¹w², ϵ²w², ∂₁₂, ∂₁ϵ¹, ∂₁ϵ², ∂₂ϵ¹, ∂₂ϵ², ϵ¹², v₁₂w¹, v₁₂w², ∂₁v₁₂, ∂₂v₁₂, ϵ¹v₁₂, ϵ²v₁₂, v₁w¹², ∂₁v₁w¹, ∂₂v₁w¹, ϵ¹v₁w¹, ϵ²v₁w¹, ∂₁v₁w², ∂₂v₁w², ϵ¹v₁w², ϵ²v₁w², ∂₁₂v₁, ∂₁ϵ¹v₁, ∂₁ϵ²v₁, ∂₂ϵ¹v₁, ∂₂ϵ²v₁, ϵ¹²v₁, v₂w¹², ∂₁v₂w¹, ∂₂v₂w¹, ϵ¹v₂w¹, ϵ²v₂w¹, ∂₁v₂w², ∂₂v₂w², ϵ¹v₂w², ϵ²v₂w², ∂₁₂v₂, ∂₁ϵ¹v₂, ∂₁ϵ²v₂, ∂₂ϵ¹v₂, ∂₂ϵ²v₂, ϵ¹²v₂, ∂₁w¹², ∂₂w¹², ϵ¹w¹², ϵ²w¹², ∂₁₂w¹, ∂₁ϵ¹w¹, ∂₁ϵ²w¹, ∂₂ϵ¹w¹, ∂₂ϵ²w¹, ϵ¹²w¹, ∂₁₂w², ∂₁ϵ¹w², ∂₁ϵ²w², ∂₂ϵ¹w², ∂₂ϵ²w², ϵ¹²w², ∂₁₂ϵ¹, ∂₁₂ϵ², ∂₁ϵ¹², ∂₂ϵ¹², v₁₂w¹², ∂₁v₁₂w¹, ∂₂v₁₂w¹, ϵ¹v₁₂w¹, ϵ²v₁₂w¹, ∂₁v₁₂w², ∂₂v₁₂w², ϵ¹v₁₂w², ϵ²v₁₂w², ∂₁₂v₁₂, ∂₁ϵ¹v₁₂, ∂₁ϵ²v₁₂, ∂₂ϵ¹v₁₂, ∂₂ϵ²v₁₂, ϵ¹²v₁₂, ∂₁v₁w¹², ∂₂v₁w¹², ϵ¹v₁w¹², ϵ²v₁w¹², ∂₁₂v₁w¹, ∂₁ϵ¹v₁w¹, ∂₁ϵ²v₁w¹, ∂₂ϵ¹v₁w¹, ∂₂ϵ²v₁w¹, ϵ¹²v₁w¹, ∂₁₂v₁w², ∂₁ϵ¹v₁w², ∂₁ϵ²v₁w², ∂₂ϵ¹v₁w², ∂₂ϵ²v₁w², ϵ¹²v₁w², ∂₁₂ϵ¹v₁, ∂₁₂ϵ²v₁, ∂₁ϵ¹²v₁, ∂₂ϵ¹²v₁, ∂₁v₂w¹², ∂₂v₂w¹², ϵ¹v₂w¹², ϵ²v₂w¹², ∂₁₂v₂w¹, ∂₁ϵ¹v₂w¹, ∂₁ϵ²v₂w¹, ∂₂ϵ¹v₂w¹, ∂₂ϵ²v₂w¹, ϵ¹²v₂w¹, ∂₁₂v₂w², ∂₁ϵ¹v₂w², ∂₁ϵ²v₂w², ∂₂ϵ¹v₂w², ∂₂ϵ²v₂w², ϵ¹²v₂w², ∂₁₂ϵ¹v₂, ∂₁₂ϵ²v₂, ∂₁ϵ¹²v₂, ∂₂ϵ¹²v₂, ∂₁₂w¹², ∂₁ϵ¹w¹², ∂₁ϵ²w¹², ∂₂ϵ¹w¹², ∂₂ϵ²w¹², ϵ¹²w¹², ∂₁₂ϵ¹w¹, ∂₁₂ϵ²w¹, ∂₁ϵ¹²w¹, ∂₂ϵ¹²w¹, ∂₁₂ϵ¹w², ∂₁₂ϵ²w², ∂₁ϵ¹²w², ∂₂ϵ¹²w², ∂₁₂ϵ¹², ∂₁v₁₂w¹², ∂₂v₁₂w¹², ϵ¹v₁₂w¹², ϵ²v₁₂w¹², ∂₁₂v₁₂w¹, ∂₁ϵ¹v₁₂w¹, ∂₁ϵ²v₁₂w¹, ∂₂ϵ¹v₁₂w¹, ∂₂ϵ²v₁₂w¹, ϵ¹²v₁₂w¹, ∂₁₂v₁₂w², ∂₁ϵ¹v₁₂w², ∂₁ϵ²v₁₂w², ∂₂ϵ¹v₁₂w², ∂₂ϵ²v₁₂w², ϵ¹²v₁₂w², ∂₁₂ϵ¹v₁₂, ∂₁₂ϵ²v₁₂, ∂₁ϵ¹²v₁₂, ∂₂ϵ¹²v₁₂, ∂₁₂v₁w¹², ∂₁ϵ¹v₁w¹², ∂₁ϵ²v₁w¹², ∂₂ϵ¹v₁w¹², ∂₂ϵ²v₁w¹², ϵ¹²v₁w¹², ∂₁₂ϵ¹v₁w¹, ∂₁₂ϵ²v₁w¹, ∂₁ϵ¹²v₁w¹, ∂₂ϵ¹²v₁w¹, ∂₁₂ϵ¹v₁w², ∂₁₂ϵ²v₁w², ∂₁ϵ¹²v₁w², ∂₂ϵ¹²v₁w², ∂₁₂ϵ¹²v₁, ∂₁₂v₂w¹², ∂₁ϵ¹v₂w¹², ∂₁ϵ²v₂w¹², ∂₂ϵ¹v₂w¹², ∂₂ϵ²v₂w¹², ϵ¹²v₂w¹², ∂₁₂ϵ¹v₂w¹, ∂₁₂ϵ²v₂w¹, ∂₁ϵ¹²v₂w¹, ∂₂ϵ¹²v₂w¹, ∂₁₂ϵ¹v₂w², ∂₁₂ϵ²v₂w², ∂₁ϵ¹²v₂w², ∂₂ϵ¹²v₂w², ∂₁₂ϵ¹²v₂, ∂₁₂ϵ¹w¹², ∂₁₂ϵ²w¹², ∂₁ϵ¹²w¹², ∂₂ϵ¹²w¹², ∂₁₂ϵ¹²w¹, ∂₁₂ϵ¹²w², ∂₁₂v₁₂w¹², ∂₁ϵ¹v₁₂w¹², ∂₁ϵ²v₁₂w¹², ∂₂ϵ¹v₁₂w¹², ∂₂ϵ²v₁₂w¹², ϵ¹²v₁₂w¹², ∂₁₂ϵ¹v₁₂w¹, ∂₁₂ϵ²v₁₂w¹, ∂₁ϵ¹²v₁₂w¹, ∂₂ϵ¹²v₁₂w¹, ∂₁₂ϵ¹v₁₂w², ∂₁₂ϵ²v₁₂w², ∂₁ϵ¹²v₁₂w², ∂₂ϵ¹²v₁₂w², ∂₁₂ϵ¹²v₁₂, ∂₁₂ϵ¹v₁w¹², ∂₁₂ϵ²v₁w¹², ∂₁ϵ¹²v₁w¹², ∂₂ϵ¹²v₁w¹², ∂₁₂ϵ¹²v₁w¹, ∂₁₂ϵ¹²v₁w², ∂₁₂ϵ¹v₂w¹², ∂₁₂ϵ²v₂w¹², ∂₁ϵ¹²v₂w¹², ∂₂ϵ¹²v₂w¹², ∂₁₂ϵ¹²v₂w¹, ∂₁₂ϵ¹²v₂w², ∂₁₂ϵ¹²w¹², ∂₁₂ϵ¹v₁₂w¹², ∂₁₂ϵ²v₁₂w¹², ∂₁ϵ¹²v₁₂w¹², ∂₂ϵ¹²v₁₂w¹², ∂₁₂ϵ¹²v₁₂w¹, ∂₁₂ϵ¹²v₁₂w², ∂₁₂ϵ¹²v₁w¹², ∂₁₂ϵ¹²v₂w¹², ∂₁₂ϵ¹²v₁₂w¹²)

julia> ∂1 * (:(x1^2+x2^2)*ϵ2)
ERROR: MethodError: *(::Basis{⟨++₁₂⟩,1,0x0000000000000004}, ::SBlade{⟨--¹²⟩',1,ϵ²,Expr}) is ambiguous. Candidates:
  *(a::F, b::SBlade{V,G,B,T} where B) where {F, V, G, T} in Grassmann at /Users/hli/.julia/packages/Grassmann/73ED5/src/algebra.jl:540
  *(a::A, b::B) where {A<:TensorAlgebra, B<:TensorAlgebra} in AbstractTensors at /Users/hli/.julia/packages/AbstractTensors/mOOch/src/AbstractTensors.jl:49
Possible fix, define
  *(::A<:TensorAlgebra, ::B<:(SBlade{V,G,B,T} where B where T where G where V))
Stacktrace:
 [1] top-level scope at REPL[3]:1

julia> 

I just download Julia 1.2, and installed the packages from GitHub.
Not sure whether the error is caused by the existence of both Julia 1.1 and Julia 1.2.
I will double check.

(And to be honest, I'm still trying to figure out the meaning of the output of @mixedbasis tangent(ℝ^2,3,2) :( )

[chakravala]

Are you using the most recent commit from the master branch? with v0.4.1 of Grassmann?

[goballooning]

It is Grassmann 0.3.
(I noticed that you called register() after the last commit of DirectSum, similar command occurs in the page of your recent Reduce.jl fix), but there is no such "at" JuliaRegistrator command after your last commit of Grassmann. )

Here are the details of my installation. I'm not sure whether the way of installation matters.

• Fresh install of Julia 1.2 from MacPorts, packages added from GitHub links.

julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 5 entries:
  "Grassmann"          => v"0.3.0"
  "DirectSum"          => v"0.4.1"
  "Reduce"             => v"1.2.3"
  "ComputedFieldTypes" => v"0.1.0"
  "AbstractTensors"    => v"0.3.0"

• Clean up and reinstall the package Grassmann.jl (Julia 1.1 installed using dmg file provided by julialang.org)

julia> Pkg.resolve()
 Resolving package versions...
  Updating `~/.julia/environments/v1.1/Project.toml`
 [no changes]
  Updating `~/.julia/environments/v1.1/Manifest.toml`
 [no changes]

julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 0 entries

julia> Pkg.add("Grassmann")
 Resolving package versions...
  Updating `~/.julia/environments/v1.1/Project.toml`
  [4df31cd9] + Grassmann v0.3.0
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [398f06c4] + AbstractLattices v0.1.2
  [a8e43f4a] + AbstractTensors v0.3.0
  [861a8166] + Combinatorics v0.7.0
  [459fdd68] + ComputedFieldTypes v0.1.0
  [22fd7b30] + DirectSum v0.4.1
  [9dda63f9] + ForceImport v0.0.3
  [4df31cd9] + Grassmann v0.3.0
  [edad4870] + Leibniz v0.0.3
  [f27b6e38] + Polynomials v0.5.2
  [93e0c654] + Reduce v1.2.2
  [ae029012] + Requires v0.5.2
  [90137ffa] + StaticArrays v0.11.0
  [a4af3ec5] + SyntaxTree v1.0.1
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [8dfed614] + Test 

julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 1 entry:
  "Grassmann" => v"0.3.0"

[chakravala]

My bad, I am still on v0.3.0 of Grassmann, got the version numbers mixed up in my head.

Try using pkg> dev Grassmann this is not registered yet, you are not using the dev version.

[goballooning]

After (v1.1) pkg>dev Grassmann, I git checkout 1f1f5b1, and (v1.1) build Grassmann
(about this tag: 1f1f5b1 -- [HEAD] generalized to higher order derivations)

But I met another error in trying the same example. I'm not quite confident in Julia, so please let me know if I can test other cases and provide more useful information.

As far as I remember, I've never met this error in using Grassmann days until recent changes.

julia> @mixedbasis tangent(ℝ^2,3,2);

julia> ∂1 * (:(x1^2+x2^2)*ϵ2)
ERROR: MethodError: no method matching derive(::Expr, ::Basis{⟨++--₁₂¹²⟩*,1,0x0000000000000010})
Closest candidates are:
  derive(::Any, ::Any, ::Any, ::Any) at /Users/hli/.julia/dev/Grassmann/src/algebra.jl:30
  derive(::N<:Number, ::Any) where N<:Number at /Users/hli/.julia/dev/Grassmann/src/algebra.jl:29
Stacktrace:
 [1] derive_mul at /Users/hli/.julia/dev/Grassmann/src/algebra.jl:46 [inlined]
 [2] *(::Basis{⟨++--₁₂¹²⟩*,1,0x0000000000000010}, ::SBlade{⟨++--₁₂¹²⟩*,1,ϵ²,Expr}) at /Users/hli/.julia/dev/Grassmann/src/algebra.jl:275
 [3] interop at /Users/hli/.julia/packages/AbstractTensors/mOOch/src/AbstractTensors.jl:29 [inlined]
 [4] *(::Basis{⟨++₁₂⟩,1,0x0000000000000004}, ::SBlade{⟨--¹²⟩',1,ϵ²,Expr}) at /Users/hli/.julia/dev/Grassmann/src/algebra.jl:1358
 [5] top-level scope at none:0

[chakravala]

Are you sure you have using Reduce, Grassmann ?

If you don't have derive(::Expr, ...) it is because you don't have Reduce loaded.

PS, I have registered and tagged v0.3.1 but it has not been merged into the registry yet.

[goballooning]

Really sorry. I do miss out using Reduce.
Your examples work now. (It seems faster under Julia 1.2)

julia> using Reduce
[ Info: Recompiling stale cache file /Users/hli/.julia/compiled/v1.1/Reduce/wEGBP.ji for Reduce [93e0c654-6965-5f22-aba9-9c1ae6b3c259]
Reduce (Free CSL version, revision 5122), 15-Sep-19 ...

julia> using Grassmann

julia> @mixedbasis tangent(ℝ^2,3,2);

julia> ∂1 * (:(x1^2+x2^2)*ϵ2)
(2x1)∂₁ϵ²

[chakravala]

Higher order derivatives are also usable, although there is still a small bug with nested ones.

julia> ∂12 * (:(x1^2*x2^2)*ϵ2)
(4 * x1 * x2)∂₁₂ϵ²

You can also use SymEngine (faster) or SymPy (slower) instead of Reduce if you want.

[goballooning]

What's "ϵ2" in the above examples?

[chakravala]

To clarify what the epsilons are, my differential geometric algebra is more complicated than just geometric algebra. The epsilon notation is a new notation I am using to help unify geometric algebra with differential geometry. Basically, the epsilon makes an element a differential form so that differential operators can act on it. By building the meaning of operator and tensor fields into the basis algebra, the overall usage should become more mathematically intuitive:

julia> using Reduce,Grassmann
Reduce (Free CSL version, revision 5151), 02-Oct-19 ...

julia> @mixedbasis tangent(ℝ^2,3,2);

julia> (∂1+∂12) * (:(x1^2*x2^2)*ϵ1 + :(sin(x1))*ϵ2)
0.0 + (2 * x1 * x2 ^ 2)∂₁ϵ¹ + (cos(x1))∂₁ϵ² + (4 * x1 * x2)∂₁₂ϵ¹

Currently only linear operators are supported... nonlinear operators will be supported soon.