not really. You're not really supposed to use fences for that fine grained control anyways

Perksey•6/4/22, 1:56 PM

i think blocking is fine as long as you figure out how it works wrt synchronization contexts

KaiOP•6/4/22, 1:57 PM

I have been looking at making a mixed CPU/GPU workflow engine, but there I just used Timeline semaphores to indicate the steps and had a dedicated CPU thread that just waited on the next step of CPU work. CPU threads are pretty cheap and the AMD closed source driver did a good job at indicating to the OS that the thread is just waiting on I/O

Perksey•6/4/22, 1:58 PM

possibly relevant https://github.com/john-h-k/Voltium/blob/a7d95d57c889708881b2078b0dce6ef0271b3082/sources/Voltium.Core/Devices/GpuTask.cs (but note the license, so use it for reference only)

PPerksey i think blocking is fine as long as you figure out how it works wrt synchronizat...

KaiOP•6/4/22, 1:58 PM

I don't think blocking a task thread is ever a good idea? Not sure, but I think that's worse then just creating a thread since it first "steals" a full thread from the thread pool, eventually that thread will be replaced, but when you eventually are done with your thread it'll just be discarded anyways. So just creating a new one seems reasonable?

Perksey•6/4/22, 1:58 PM

i mean, what do you think FlushAsync and friends do

Perksey•6/4/22, 1:58 PM

blocking is what ConfigureAwait is designed for

Perksey•6/4/22, 1:59 PM

but yeah this sort of scenario is why you're able to have such fine grained control over synchronization contexts if need be

KKai not really. You're not really supposed to use fences for that fine grained contr...

TechPizza•6/4/22, 2:01 PM

hold on what. what is the correct solution then?

PPerksey possibly relevant <https://github.com/john-h-k/Voltium/blob/a7d95d57c889708881b2...

KaiOP•6/4/22, 2:02 PM

I don't know Win32 well, but from what I can tell that just uses to make windows create a waiting thread from it's pool and do the waiting on there?

TTechPizza hold on what. what is the correct solution then?

KaiOP•6/4/22, 2:02 PM

depends on the use case, what are you doing?

TechPizza•6/4/22, 2:02 PM

waiting for a memory transfer to finish

KaiOP•6/4/22, 2:02 PM

depending on the size of the transfer a fence should be fine

KKai I don't know Win32 well, but from what I can tell that just uses `RegisterWaitFo...

Perksey•6/4/22, 2:03 PM

it's basically telling the OS "hey i'm waiting for this thing, when the driver tells you that thing has finished run my code please"

Perksey•6/4/22, 2:03 PM

unfortunately vulkan has no such mechanism

TechPizza•6/4/22, 2:03 PM

hundreds of MB per second :)

Perksey•6/4/22, 2:03 PM

i'm not saying that link is how you can do it, i'm just sending that purely because it's an example of how it's done with other apis

KaiOP•6/4/22, 2:04 PM

in most engines you have a bunch of small tranfers, which you would typically batch & connect with semapores (or timeline sempahores if you support that) and then wait on the last one in the chain with a semaphore.
You can use events in the middle as a lightweight way of detecting that a specific transfer is done already (if you don't have timeline semaphores)

PPerksey unfortunately vulkan has no such mechanism

KaiOP•6/4/22, 2:04 PM

well you could export a vulkan fence as win32 object and then do unholy things with it, but from the spec I don't think you're guaranteed that the returned win32 handle supports usage as a wait handle (and specifically nvidia doesn't support this apparently)

TechPizza•6/4/22, 2:08 PM

oh no, waiting for timeline semaphores requires an extension/vk1.2

KaiOP•6/4/22, 2:09 PM

Also regarding transfers, that's exactly why timeline semaphores are such an amazing feature. You just make a central semaphore, and whenever you transfer something you give it an ID (by incrementing a counter) and "enqueue" the transfer task by wrapping your GPU work in a starting "wait for semaphore to be at step ID -1" and at the end you just advance the semaphore to ID.
Then you a) get free syncronization with little to no work, and you can check whether a specific transfer has completed via just "is semaphore count > ID"
AND you even get pretty realistic loading bars by just saying "start of level load was ID = 185, end of level load is ID 1835" then just poll the semaphore count and show the progress bar as %

TTechPizza oh no, waiting for timeline semaphores requires an extension/vk1.2

KaiOP•6/4/22, 2:10 PM

timelines semaphores are a feature of the KHR sync2 extension... But if you don't support oldish devices almost all drivers have backported them as they require nearly no hardware support

KaiOP•6/4/22, 2:11 PM

like 40% windows support... Not great I guess, but it's a perfect opportunity for progressive enhancement imo

KaiOP•6/4/22, 2:12 PM

memory transfer is pretty easy to abstract away and you can always replace the timeline semaphore with either a fence and one submission / transfer or a pair of semaphore + event / transfer (but you can batch the transfers into bigger submissions)

TechPizza•6/4/22, 2:13 PM

since you seem quite knowledgeable about transfers, do you know the "correct" way to upload a lot of data quickly?

TechPizza•6/4/22, 2:14 PM

my code currently maps a buffer, writes a lot to it, unmaps it, and does some copy commands

TechPizza•6/4/22, 2:14 PM

i am looking into persistently mapped buffers though

TechPizza•6/4/22, 2:16 PM

and i also abuse the graphics queue instead of using the transfer queue

KaiOP•6/4/22, 2:17 PM

it depends as with anything. Most important thing is a) keep your GPU busy as always and b) use a transfer queue if you really need perf.
Nvidia drivers (used to? still do? a year ago it was this way) only use half the available PCIe bandwith with non-transfer queues.

Regarding buffers, I always went the route to have as many 1mb host-mapped buffers that as I could have or one buffer that was however big as the hardware supported (yes some platforms don't have 1mb of host-mapped space) and then used that as my hole to stuff data through. Designed everything around unblocking that buffer as soon as possible as it's the main blocker.
I think I also saw some gains on modern hardware by using larger blocks & manually invalidating data (which wastes some space but may improve throughput because the driver knows exactly what to transfer). I think I eventually moved to always manually invalidating the data range as host-coherent was often a subset of host-available and there's not much reason to make it coherent in the first place

Perksey•6/4/22, 2:19 PM

fwiw persistent host-mapped buffers is how directstorage works

Perksey•6/4/22, 2:19 PM

it maps the memory, and then talks directly to the nvme driver and says "write directly into this memory"

KaiOP•6/4/22, 2:20 PM

pretty sure that is just the best way to do it these days. Unfortunately not many improvements in sub-allocation performance, which I think could be better as it hands off that coherence problem to the driver without any of the downsides that a fixed block size has

PPerksey it maps the memory, and then talks directly to the nvme driver and says "write d...

KaiOP•6/4/22, 2:21 PM

iirc they also have some auto uncompression stuff that can align compressed data to specific block sizes so the unpacked data fits exactly into one of their GPU transfer blocks?

Perksey•6/4/22, 2:21 PM

yeah there's some opt-in stuff like that

PPerksey possibly relevant <https://github.com/john-h-k/Voltium/blob/a7d95d57c889708881b2...