Anisotropic Filtered Shadows

[mrjustaguy]

I've been looking at ways to reduce Shadow Aliasing at oblique angles, and currently there are only 2 approaches - Supersampling and TAA.
TAA suffers from regular TAA issues + sudden pixelization when sudden changes in movement occur (Godot specific issue that'll likely be gone with the planned FSR2 TAA)
SSAA is very expensive

Now while playing around with faked Decal Shadows I've compared linear anisotropic and simple linear results, and simple linear results in identical issues as normal shadows, while anisotropic results are great, which made me ask the question:

Could Shadows implement some form of in-shader (quasi-)Anisotropic Filtering for shadows to improve Shadow stability at oblique angles?

The Advantages are obvious, it'd remove the need for using TAA/SSAA to resolve the issue, allowing just MSAA and/or FXAA to be used with great results, and in cases where the only reason for TAA is to fix this issue it'd likely massively improve performance while eliminating ghosting

[Calinou]

Shadows don't have mipmaps generated in the first place – it would be too expensive to do so. You need to avoid aliasing by ensuring their texel density (relative to the screen resolution) isn't too high. This is actually where making shadows lower-resolution or blurrier improves quality 🙂

In general, if you have the GPU horsepower to render shadows that are so high-resolution they become aliased, you probably have the GPU horsepower to render the entire viewport at a higher resolution.

[mrjustaguy]

1. From what I've found (https://paroj.github.io/gltut/Texturing/Tut15%20Anisotropy.html#:~:text=While%20you%20could%20theoretically%20use,left%20up%20to%20the%20hardware.) technically Mipmaps aren't a hard requirement for making Anisotropic Filtering Algorithms
2. This has 0 to do with the Resolution of the shadow map, but rather the angle it's being viewed at that results in aliasing. Here's a video to show what I mean using Decals to showcase the issue and solution (switching between linear and linear anisotropic x16):

2023-06-30.11-42-13.mp4

3. There are Shadow Map techniques that do work with Anisotropic Filtering (https://developer.nvidia.com/gpugems/gpugems3/part-ii-light-and-shadows/chapter-8-summed-area-variance-shadow-maps - see figure 8.4)

The issue of Shadow Aliasing because of too high texel density vs screen resolution I've never experienced so...

On a side note, larger blur values can also get a similar result, so another solution would be to dynamically adjust blur based on the ndotv, so that as the angle gets sharper, you exponentially increase the shadow blur, resulting in higher temporal stability and less aliasing at those oblique angles, and that would at least be fairly easy to implement and toy around with

[Calinou]

On a side note, larger blur values can also get a similar result, so another solution would be to dynamically adjust blur based on the ndotv, so that as the angle gets sharper, you exponentially increase the shadow blur, resulting in higher temporal stability and less aliasing at those oblique angles, and that would at least be fairly easy to implement and toy around with

This sounds worth trying, but it may have a non-negligible performance cost.

[mrjustaguy]

Ok so update on this

I've toyed around with it a little, and increasing shadow blur based on view angle (using ndotv) works well at combating Aliasing, however has one tiny issue... It tends to result in aggressive shadow curving, as seen in this picture:

I've tried using reflect as that looks like something that should do the trick, but for whatever reason that doesn't want to compile still haven't figured out why tho..

[mrjustaguy]

after toying around with it some more, I've figured out that dynamically adjusting blur based on angle from the camera will always produce an effects like the above, so I went to resolving some other issues for my project aand...

I've accidentally found the culprit of excessive Shadow Aliasing - godotengine/godot#48776

once I reverted this, only a little bit of shadow aliasing remains present (though depends on how you set up your splits, too tightly packed splits alias a bit more), as shadows are naturally much blurrier in the distance, where aliasing gets really strong with insufficient blur..
Upon further investigation it seems that that PR is only good for use with low distance shadows, where both of the main issues with it are minimal (the pixelization of distant splits and now this aliasing) though TAA does reduce the aliasing problem if you are using it, though this is not applicable to mobile which would benefit most from low distance shadows...

[mrjustaguy]

Questions for anyone with GLSL experience - How do you set a sampler to use Anisotropic Filtering?

[mrjustaguy]

I've gotten an Idea what needs to be done to fix the issue above, and the NdotV method of increasing the blur is indeed the way to go, just have to do a non-uniform blur that only applies to the y axis of the pixels.. How to do that I'm not entirely sure yet, but I'm experimenting.

The hurdles to overcome are:

1. Figure out how to Transform Shadowmap XY coordinates into Viewport Y coordinate
2. Figure out a function to get the effect to be applied just enough for any given angle