Fix remaining sync issues

[Ralith]

Fixes #390.

[patowen]

On this branch, I get the following errors:
Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x1e06f482a10, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x376bc9df | vkQueueSubmit(): Hazard WRITE_AFTER_READ for entry 0, VkCommandBuffer 0x1e06fd4edf0[post-frame], Submitted access info (submitted_usage: SYNC_CLEAR_TRANSFER_WRITE, command: vkCmdUpdateBuffer, seq_no: 1, reset_no: 1). Access info (prior_usage: SYNC_COMPUTE_SHADER_UNIFORM_READ, read_barriers: VK_PIPELINE_STAGE_2_DRAW_INDIRECT_BIT|VK_PIPELINE_STAGE_2_VERTEX_SHADER_BIT|VK_PIPELINE_STAGE_2_TESSELLATION_CONTROL_SHADER_BIT|VK_PIPELINE_STAGE_2_TESSELLATION_EVALUATION_SHADER_BIT|VK_PIPELINE_STAGE_2_GEOMETRY_SHADER_BIT|VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT|VK_PIPELINE_STAGE_2_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT|VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT|VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT|VK_PIPELINE_STAGE_2_TASK_SHADER_BIT_EXT|VK_PIPELINE_STAGE_2_MESH_SHADER_BIT_EXT|VK_PIPELINE_STAGE_2_RAY_TRACING_SHADER_BIT_KHR|VK_PIPELINE_STAGE_2_FRAGMENT_SHADING_RATE_ATTACHMENT_BIT_KHR|VK_PIPELINE_STAGE_2_TRANSFORM_FEEDBACK_BIT_EXT|VK_PIPELINE_STAGE_2_INDEX_INPUT_BIT|VK_PIPELINE_STAGE_2_VERTEX_ATTRIBUTE_INPUT_BIT, queue: VkQueue 0x1e06f482a10[], submit: 1, batch: 1, batch_tag: 20, command: vkCmdDispatch, command_buffer: VkCommandBuffer 0x1e06fd4a2d0[post-frame], seq_no: 10, reset_no: 1). id=SYNC-HAZARD-WRITE-AFTER-READ number=929810911 queue_labels= cmd_labels= objects=QUEUE 1e06f482a10

Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] Object 0: handle = 0x1e06f482a10, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x5c0ec5d6 | vkQueueSubmit(): Hazard WRITE_AFTER_WRITE for entry 0, VkCommandBuffer 0x1e06fd4a2d0[post-frame], Submitted access info (submitted_usage: SYNC_CLEAR_TRANSFER_WRITE, command: vkCmdUpdateBuffer, seq_no: 1, reset_no: 19). Access info (prior_usage: SYNC_CLEAR_TRANSFER_WRITE, write_barriers: SYNC_DRAW_INDIRECT_INDIRECT_COMMAND_READ|SYNC_VERTEX_SHADER_SHADER_BINDING_TABLE_READ|SYNC_VERTEX_SHADER_SHADER_SAMPLED_READ|SYNC_VERTEX_SHADER_SHADER_STORAGE_READ|SYNC_COMPUTE_SHADER_SHADER_BINDING_TABLE_READ|SYNC_COMPUTE_SHADER_SHADER_SAMPLED_READ|SYNC_COMPUTE_SHADER_SHADER_STORAGE_READ|SYNC_COMPUTE_SHADER_SHADER_STORAGE_WRITE|SYNC_COMPUTE_SHADER_UNIFORM_READ, queue: VkQueue 0x1e06f482a10[], submit: 35, batch: 1, batch_tag: 7770, command: vkCmdUpdateBuffer, command_buffer: VkCommandBuffer 0x1e06fd4edf0[post-frame], seq_no: 1, reset_no: 17). id=SYNC-HAZARD-WRITE-AFTER-WRITE number=1544472022 queue_labels= cmd_labels= objects=QUEUE 1e06f482a10

I only get the SYNC-HAZARD-WRITE-AFTER-READ error while generating terrain, but the SYNC-HAZARD-WRITE-AFTER-WRITE error occurs every frame.

If I break on validation error, I can see that both types of error SYNC-HAZARD-WRITE-AFTER-WRITE occur on the final queue_submit call with self.gfx.queue in client::graphics::draw::draw.

[patowen]

The latest changes seem to have fixed the SYNC-HAZARD-WRITE-AFTER-WRITE errors. All that's left is a SYNC-HAZARD-WRITE-AFTER-READ error:

Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x1d14a1e0240, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x376bc9df | vkQueueSubmit(): Hazard WRITE_AFTER_READ for entry 0, VkCommandBuffer 0x1d14a6ab7a0[post-frame], Submitted access info (submitted_usage: SYNC_COMPUTE_SHADER_SHADER_STORAGE_WRITE, command: vkCmdDispatch, seq_no: 29, reset_no: 367). Access info (prior_usage: SYNC_VERTEX_SHADER_SHADER_STORAGE_READ, read_barriers: VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT, queue: VkQueue 0x1d14a1e0240[], submit: 735, batch: 0, batch_tag: 173167, command: vkCmdDrawIndirect, command_buffer: VkCommandBuffer 0x1d14a6a94b0[frame], seq_no: 556, reset_no: 367). id=SYNC-HAZARD-WRITE-AFTER-READ number=929810911 queue_labels= cmd_labels= objects=QUEUE 1d14a1e0240

This does seem different from the SYNC-HAZARD-WRITE-AFTER-READ error from before. The breakpoint is the same.

[Ralith]

I'm not sure I believe the validation layers, but the behavior isn't entirely consistent with a false positive either, so maybe we do have a bug. Added a hacky barrier to mitigate the error for now, with details in a comment.

[patowen]

Strange. One thing to note is that on my machine, the validation error doesn't only trigger a handful of times at startup. It also triggers whenever I move so that new chunks generate.

EDIT: I can confirm though that the latest commit does resolve all the remaining synchronization validation errors.

EDIT 2: Either way, thanks for looking into this! Getting synchronization right seems quite annoying and tricky.

[Ralith]

I've never gotten it to recur after the first handful of frames, no matter how I wander, even through deep valleys. Using validation layers 1.3.280. If it's a false positive just from the surface extraction compute shader having write access to a buffer that the chunk vertex shader reads from, it should reproduce every single time surface extraction runs, because that's always the case.

[patowen]

Even more strange. I wonder why our setups are different, barring anything silly like forgetting that "Limit Duplicated Messages" is turned on.

I'm also on 1.3.280.0, so it could be OS/graphics card specific, although I would have expected validation layers to be more consistent (although I guess there could be a race condition that only triggers a validation error on certain machines based on how fast various components are).