Add ReverseSequence

[NeroBlackstone]

I don't know if this is the correct implementation

[avik-pal]

Let's redesign the interface a bit. I would default the dims to nothing. The behavior for that would be:

1. If the input is AbstractVector{T} and !isbitstype(T) then we simply call Iterators.reverse (note this is lazy)
2. For bitstype AbstractVector, call reverse(x).
3. For other arrays, it calls reverse(x; dims=ndims(x) - 1).

This essentially prevents unnecessary copies. For eg, for Bidirectional RNN we can simply construct an iterator once and make a lazy reverse iterator using this API without copying the entire array.

If dims is specified,

1. AbstractVector{T} and !isbitstype(T) throws an error
2. All other array inputs call reverse(x; dims=r.dims)

[NeroBlackstone]

If dims is specified,

1. AbstractVector{T} and !isbitstype(T) throws an error

Hi, I'm curious why isbitstype() is needed here. For any 1D array, we can't specify dim right? So why we need isbitstype(), it would be nice if you could give an example with specific code. Thank you very much.

[NeroBlackstone]

I have updated the implementation according to your guidance, except for the doubtful points above.

[NeroBlackstone]

test failed on gpu, I can't reproduce on local since I don't have AMDGPU or CUDA devices...

[ludvigk]

I think it fails because Base.Iterators.Reverse{CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}} is not isbitstype. CUDA doesn't like that. Even a minimal example like this fails for that reason:

g = CUDA.zeros(1)
gr = Iterators.reverse(g)
g .* gr

[ludvigk]

Using reverse instead of Iterators.reverse on arrays of type GPUArraysCore.AbstractGPUArray should fix this.

[NeroBlackstone]

Using reverse instead of Iterators.reverse on arrays of type GPUArraysCore.AbstractGPUArray should fix this.

I got a PC with GPU and test locally.

I found Float64 works well but Int throws errors. Weird...

[avik-pal]

Add this layer to the docs, also rebase with main. That will fix the doc build

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.13%. Comparing base (acd924f) to head (dbeb848).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #698      +/-   ##
==========================================
- Coverage   87.34%   81.13%   -6.21%     
==========================================
  Files          50       50              
  Lines        2505     2513       +8     
==========================================
- Hits         2188     2039     -149     
- Misses        317      474     +157     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[NeroBlackstone]

I noticed some test failures, should I follow up on this PR? Or leave all the rest to you？

[avik-pal]

Github won't let me edit files directly. So if you add the docs to https://github.com/LuxDL/Lux.jl/blob/main/docs/src/api/Lux/layers.md#misc-helper-layers the build will pass