Silk.NET•4y ago

(i meant mapped staging buffers)

TechPizza•6/4/22, 2:35 PM

multiple threads were uploading constantly

SSupine (i meant mapped staging buffers)

Kai•6/4/22, 2:35 PM

yeah, those are often pretty limited in space

SupineOP•6/4/22, 2:35 PM

Unless Unified memory?

TTechPizza multiple threads were uploading constantly

Kai•6/4/22, 2:35 PM

I don't think I really understand what they are doing tbh, this should be 100% IO bound work

TechPizza•6/4/22, 2:35 PM

... aren't staging buffers just in RAM?

TTechPizza ... aren't staging buffers just in RAM?

SupineOP•6/4/22, 2:36 PM

Exactly

Kai•6/4/22, 2:36 PM

well it depends* they need to be in a space that the driver has access to

Kai•6/4/22, 2:36 PM

and on integrated GPUs that can get pretty funky

KKai 🤔 I don't think I really understand what they are doing tbh, this should be 100...

TechPizza•6/4/22, 2:37 PM

most of the time on the profiler was being spent in copying to mapped memory

Kai•6/4/22, 2:38 PM

I mean many of the modern AMD cards have a memory type that looks like

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT
DEVICE_COHERENT_BIT_AMD
DEVICE_UNCACHED_BIT_AMD

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT
DEVICE_COHERENT_BIT_AMD
DEVICE_UNCACHED_BIT_AMD

with like 8GB
but on some iGPUs the best you get is

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT

at very limited space

TTechPizza most of the time on the profiler was being spent in copying to mapped memory

Kai•6/4/22, 2:38 PM

hmm. Could be related to it being coherent if the implementation is weird

how are you copying?

TechPizza•6/4/22, 2:38 PM

Unsafe.CopyBlockUnaligned :)

TechPizza•6/4/22, 2:39 PM

i tried a normal for-loop but it was slightly slower

Kai•6/4/22, 2:39 PM

I'm not sure, but can you try using Span.CopyTo? pretty sure that is the fastest you can get

Kai•6/4/22, 2:39 PM

cause that uses fancy vector instructions and stuffs

TechPizza•6/4/22, 2:40 PM

was the same speed as Unsafe.CopyBlockUnaligned

Kai•6/4/22, 2:40 PM

ah ok :(

SupineOP•6/4/22, 2:40 PM

I think they call same underlying method with

ref byte

ref byte

Kai•6/4/22, 2:40 PM

maybe they unified those two at some point, I think I heard about that

Kai•6/4/22, 2:40 PM

hmmm

Kai•6/4/22, 2:40 PM

Fuck it, let me create a project

Kai•6/4/22, 2:41 PM

I wanted to rewrite type references today but I guess I'm torturing my GPU today

Kai•6/4/22, 4:00 PM

Haven't really build stuff with the new(ish) chain extensions, and man that opens up some awesome uses that I didn't even think of when this was designed:

var queueProps = new QueueFamilyProperties2[(int)queuePropCount];
var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount];
for (var i = 0; i < queueProps.Length; i++)
{
    QueueFamilyProperties2.Chain(out queueProps[i])
        .AddNext(out queueGlobalPrios[i]);
}

var queueProps = new QueueFamilyProperties2[(int)queuePropCount];
var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount];
for (var i = 0; i < queueProps.Length; i++)
{
    QueueFamilyProperties2.Chain(out queueProps[i])
        .AddNext(out queueGlobalPrios[i]);
}

Perksey•6/4/22, 4:19 PM

honestly @thargy is a fucking genius

Kai•6/5/22, 7:01 AM

Is it good when my audio driver starts to crash when I run my memory test

Lyris the Kitori•6/5/22, 7:05 AM

yes

Kai•6/5/22, 7:06 AM

apparently if you leak GPU local memory the driver has a hard time tracking which it was and just... clears all of it, bringing the audio memory down with it xD

Kai•6/5/22, 7:06 AM

AMD drivers on point as always

Lyris the Kitori•6/5/22, 7:06 AM

still better than nvidia

Kai•6/5/22, 7:06 AM

true

Kai•6/5/22, 8:16 AM

So getting 5% throughput right now (about 1.6 GB/s)

Lyris the Kitori•6/5/22, 8:18 AM

nice

Kai•6/5/22, 8:19 AM

Not really, kinda hoped for an immediate 60+% with this code

Kai•6/5/22, 10:41 AM

I can get almost 19 GB/s now, given everything aligns perfectly

Kai•6/5/22, 10:42 AM

With Generic block sizes a bit less, up to 16 GB/s

Kai•6/5/22, 10:43 AM

(This is doing 6 consecutive uploads of 1 GB each, then it waits for those to complete)

Kai•6/5/22, 10:45 AM

If I don't wait for any transfers to complete (up to 1TB of "in flight" data) I can get 90% peak throughput. so That's ~28 GB/s

Kai•6/5/22, 10:46 AM

Note that this is on PCIe 4.0 x16 and I'm just going off the Task Manager to figure out the throughput, but seems reasonable

Kai•6/5/22, 10:47 AM

Looks like the main thing slowing this down is the CPU waiting for things to complete. If there is stuff to load at all times 90% throughput is possible

Kai•6/5/22, 10:49 AM

Code: https://gist.github.com/HurricanKai/0125b844666b26b531bb204f099d513d, would be cool if someone can test this on some other devices (maybe even iGPU?) to see what can be achieved there

Kai•6/5/22, 10:49 AM

note that there are a bunch of hardcoded values in there and the program will just die when something goes wrong

Kai•6/5/22, 10:50 AM

Most important are the config values at line 92, that decides how much data is shoved down the lanes and in how large blocks

Kai•6/5/22, 10:51 AM

(by passing

localBlockSize

localBlockSize

instead of

ulong.MaxValue

ulong.MaxValue

as the last parameter on line 96 an ideal scenario can be created, where there is no wasted memory. This seems to have little to no influence on speed)

Kai•6/5/22, 10:52 AM

Also note that my device has 2 Transfer queues (I'm using both in round-robin here) & I'm not using Graphics or Compute queue here, so in theory there is zero interference with other parts, other then memory usage

Kai•6/5/22, 10:58 AM

I can also get up to 80% throughput on smaller (100mb) blocks by tuning the block count.

Kai•6/5/22, 10:59 AM

@techpizza iirc you were interested in this. If you want I'm also happy to explain how this works

Kai•6/5/22, 11:01 AM

quite happy with the memory footprint here. That big buffer on the right is the target

Kai•6/5/22, 11:17 AM

I'll measure the precise throughput using write timestamp later, but I don't expect there to be any surprises there

Perksey•6/5/22, 12:07 PM

i love how Qualcomm are basically the only ones keeping OpenGL alive now

Perksey•6/5/22, 12:07 PM

they just added variable rate shading to GLES

var queueProps = new QueueFamilyProperties2[(int)queuePropCount]; var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount]; for (var i = 0; i < queueProps.Length; i++) { QueueFamilyProperties2.Chain(out queueProps[i]) .AddNext(out queueGlobalPrios[i]); }