Vulkan

[JSandusky]

Preparing to tackle Vulkan, these are my notes. This issue is an RFC for things I might be missing before I dive in.

Occlusion Query
disabled for first version

Vertex/Index Buffer changes
addition of a dynamic/static specification
changes expectations on data update, in Vulkan dynamic will be host-visible and not use the buffer copy operations

all setXXXX functions pertaining to GL-state go away (blend / etc)
Presumably these move into shader contexts
Use possibility space for configurations
blend = ADD|SUBTRACT|MULTIPLY
to enumerate the permutations that need to be determined using derivative pipelines
Contexts also specify targets

setMaterialRec
    is the most problematic function that must be moved into backend responsibility
    it now needs to know about the stage/substage of the renderpath so an appropriate Vulkan pipeline can be chosen
lighting / shadow functions present concerns
    Horde3D uses a standard lighting model, one has to modify the code to do anime shadowing already (where world/character interactions are significant and different), I'm assuming that anime style shadowing is the extreme
        to clarify, I mean the use of RG16F shadow maps where R = shadows everything, G = shadows only the static world, anime hair/brow shadows are not resolvable on current hardware (to the point they would match production, N-dimensional peeling is required for hair-shadowing and most other anime shadowing [to match production that is])

setShaderConst has to evolve for uniform-buffers
not terribly difficult to leave the same API and also support blob transfer, provided UBO members are uniquely named
adjustments portable to GL4 and GLES3 backends

Hardest on extensions like Terrain / overlays / advanced debug-drawing
    overlays tries to just use a shader combination GL-state rather than properly following
    the material->pipeline flow
    
    Conditional pipeline command registration?

Pipeline
introduce substage
stage is now just a container, Vulkan-wise the targets used by substages in stage are aggregated to the renderpass
If no substages are contained in a stage element then a root substage will be inferred
Targets can only be changed ONCE during a substage
Targets can only be bound ONCE during a substage

A pipeline must be loaded before any shaders, this changes the sample order
    if a pipeline is loaded later ALL shaders will have Vulkan pipelines generated for them
    Pipelines now become something the rendering backend needs to know about
    
Could divine sub-passes from how the <stage /> is written, this would likely be painful and predefining them offers an opportunity to raise meaningful errors regarding mismatches

SPIR-V
SPIR-V is offline compiled, though the compiler could be bundled that's an unecessary dependency
Add RIFF support to shaders because RIFF is trivial enough that tooling should never be a problem.
Include reference "compiler" that delegates to glslValidator and SPIR-V_Cross binaries to generate this format

RIFF form-type: HSHD (Horde Shader)
LIST
    ListType GL4_
    ListType GL2_
    ListType ES3_
    ListType SPIR
    
Each list begins with a reflection chunk, followed by a data chunk (source in the case of raw GLSL) identified as META and CODE, example:

"RIFF" (file-id)
    __size__
    "HSHD" (form-type)
    "H_FX" (chunk)
        __size__
        DATA
    "LIST" (riff-list)
        __size__
        "SPIR" (list type)
        "META" (chunk)
            __size__
            __data__
        "CODE" (chunk)
            __size__
            __data__
    "LIST" (riff-list)
        __size__
        "GL4_" (list type)
        "META" (riff-chunk)
            __size__
            __data__
        "CODE"
            __size__
            __data__

<CommandQueue>
	<Stage id="Main">
        <SubStage id="Attribpass">
            <SwitchTarget target="GBUFFER" />
            <ClearTarget depthBuf="true" colBuf0="true" />
            <DrawGeometry context="ATTRIBPASS" class="~Translucent" />
        </SubStage>
	
        <SubStage id="LightingPt1" link="pipelines/globalSettings.material.xml">
            <SwitchTarget target="" />
            <ClearTarget colBuf0="true" />
            
            <!-- Copy depth buffer to allow occlusion culling of lights -->
            <BindBuffer sampler="depthBuf" sourceRT="GBUFFER" bufIndex="32" />
            <DrawQuad material="materials/light.material.xml" context="COPY_DEPTH" />
        </SubStage>
        
        <SubStage id="LightingPt2" link="pipelines/globalSettings.material.xml">                
            <BindBuffer sampler="gbuf0" sourceRT="GBUFFER" bufIndex="0" />
            <BindBuffer sampler="gbuf1" sourceRT="GBUFFER" bufIndex="1" />
            <BindBuffer sampler="gbuf2" sourceRT="GBUFFER" bufIndex="2" />
            <BindBuffer sampler="gbuf3" sourceRT="GBUFFER" bufIndex="3" />
            
            <DrawQuad material="materials/light.material.xml" context="AMBIENT" class="Translucent"  />
            <DoDeferredLightLoop />
            
            <!-- particles ideally should be drawn without back to front - if you don't want translucent models just remove the order -->
            <DrawGeometry context="TRANSLUCENT" class="Translucent" order="BACK_TO_FRONT" />			
        </Substage>
    </Stage>
	
	<Stage id="Translucent">
		<ClearTarget depthBuf="true" colBuf0="false" />
	</Stage>
	
	<Stage id="Overlays">
		<DrawOverlays context="OVERLAY" />
	</Stage>
</CommandQueue>

[algts]

Hello. Adding vulkan to horde is great, but I think that several things should be noted:

• vulkan works best when multi threading is there. Currently horde is single threaded, so maybe the first step should be the generalization of the render device interface so it can receive command list. Command list then can be formed from multiple threads.
• engine should be able to use gl4 and gles 3 when vulkan is there - not every device supports vulkan yet. GL2 backend will be deprecated in the future, so it is irrelevant.

As for your proposes, there are several things that I don't fully understand:

• how are you going to specify whether the vertex/index buffer should be static or dynamic? Add parameter to model/mesh nodes?
• it seems that vulkan still supports passing uniforms (without UBO) to GPU pretty much like GL, and it is rather fast, if not the fastest method: http://kylehalladay.com/blog/tutorial/vulkan/2017/08/13/Vulkan-Uniform-Buffers.html
• are substages in pipelines really needed? Maybe we can impose restrictions on switching targets once per stage?
• Contexts also specify targets. How will this work for other render backends?
• binary shaders. If you introduce new RIFF format, will it be the only format for shaders? How will you edit the shaders then? How will the dependency code linkage (from utilityLib folder) work?
  Maybe it is better to introduce a binary representation (with the original shader files preserved for gl backends) of the horde's shader, that has a compiled FX section (maybe something like messagepack format) and spir-v shaders in it. Another benefit of this approach is that devices that support only gl (es) would be able to make a shader cache on the first launch of the application.
• lighting. If you are working on a game with anime looks, you should probably check the papers of the developers of XIII and Guilty Gear Xrd on making cel-shaded/anime look.

[JSandusky]

Contexts also specify targets. How will this work for other render backends?

It doesn't, I missed that I had left that there - the shader contexts knowing about render-target state is not necessary, just other state (blend/stencil/etc).

it seems that vulkan still supports passing uniforms (without UBO) to GPU pretty much like GL

I don't believe so, even push constants have to be specified as a UBO with the binding locatoin as push_constant, I'm pretty sure the glsl compiler isn't even supposed to allow non UBOs on targets greater than 4.0 core, the reference glslValidator compiler certainly doesn't (unless I failed to find a hidden switch) compile anything attempting to use loose uniforms. It was a lot of pain when I first started using Vulkan.

are substages in pipelines really needed?

Possibly not, it might be viable to identify where sub-passes logically are just based on the sequence of commands. Substages would just make it explicit. Definitely worth it to go for no substages if possible.

binary shaders. ...

Thinking about it, it seems completely reasonable to allow the SPIR-V inside the shaders as base64 in a block like all of the other shaders. That's also not completely atrocious even without tooling, most text-editors can paste a file as base64.

lighting. ...

I was just listing off a random very complicated lighting situation as a things are nowhere near crazy in Horde, you already have to change code to do anything significantly different (such as subtractive lights).

[JSandusky]

Most everything so far has been much less problematic than I had expected. Once descriptor-sets and UBOs are dealt with I should be up and rendering fairly quickly.

The odder problem has been getting all of the initialization for vulkan without forcing a dependency on another library into the core Horde3D library (VkInstance, VkSurfaceKHR, etc). For that I had to add an additional h3dSetExternalInterface(H3DExternalInterface::List, void*) function to the public API for feeding the necessary object handles.

So the instance, device, surface, graphics/present queues, and physical device are the user's responsibility to provide. Otherwise there's no dodging tying into some specific library like glfw or sdl.

I'm not making any attempts to support headless Vulkan usage.

---

Only 3rd party dependencies are VMA and SPRIV-Reflect

Texture and pipeline handling will take some iterations, but something that can be fiddled with shouldn't be more than a few days out.

Edit: I made the distinction between buffers being host_visible or GPU-only (staging required to update) based on whether data was given to create_______Buffer(...), if null when the buffer is created then it felt safe to assume that updates would be frequent. Textures in Vulkan are hell, they always go through the painful route right now.

[algts]

Do you really need h3dSetExternalInterface function? As far as I know, you can just work with the correctly initialized vulkan device (through glfw or other means). Can't you just pass it to init function? I thought of extending H3DRenderDevice to a more useful structure that could pass a device pointer to horde.

[JSandusky]

At the minimum I require VkInstance, VkPhysicalDevice, VkSurfaceKHR, and VkDevice to all be fed because they're products of the windowing system (ie. querying platform functions) - without committing to a this is the windowing library all shall use I have to either get the constructed objects from outside or take some function pointers to get them.

Sure a struct H3DInitParams would work that multiplexes to whatever anything needs/could-use, or a CivetWeb style convention of void** in pairs of ID, value would work as well in h3dInit.

That's a fairly minor thing though.

---

The final bits of shader handling are my main issue. The shader loading in H3D is objectively academic spaghetti so tying precompiled bytecode into that has turned out to be non-trivial.

The main issue there is that shader construction has thus far been about constructing the appropriate shader (prefixes, flags) whereas SPIR-V needs the permutations precompiled, although ShaderResource itself has the necessary information to fetch the relevant SPIR-V when it attempts to compile a shader - all of the code that occurs before that is about string-manipulation. Precompiled SPIR-V also introduces multiple CodeResource's for what in GLSL was one CodeResource (identified by the flags).

If I don't get anywhere in a few more days I'll toss everything into git, maybe other eyes will spot something I don't. Realistically it looks like I'll have to make ShaderResource and CodeResource either virtual or a lot less anal about things like storing unused code-sections.

[midn1]

What, GL2 is going to be deprecated? Oh hell naw, that's the whole reason I'm using Horde.

[algts]

Version 2 and several versions till version 3 of horde3d will still have gl2 backend (so, you are safe for several years :) ). Also, gl4 backend can work with 3.3 opengl version, that works on video cards from 2007. So it should not be a very large problem if gl2 backend is removed in the following years.