Silk.NET•4y ago

cause your resources are very likely much larger than you can fit in host-mapped memory

KKai can you share the transfer code? Would love to have a look at it

TechPizza•6/4/22, 2:31 PM

there's nothing more to it than map, write, unmap, copycommand, submit, wait for fence

TechPizza•6/4/22, 2:32 PM

and the old code was so unreadable that i'm rewriting the hole thing

KaiOP•6/4/22, 2:32 PM

well how big are the blocks you're copying here?

TechPizza•6/4/22, 2:34 PM

i was roughly doing 8MB uploads

KaiOP•6/4/22, 2:34 PM

and then in sequence or in parallel?

KKai cause your resources are very likely much larger than you can fit in host-mapped...

Supine•6/4/22, 2:35 PM

(i meant mapped staging buffers)

TechPizza•6/4/22, 2:35 PM

multiple threads were uploading constantly

SSupine (i meant mapped staging buffers)

KaiOP•6/4/22, 2:35 PM

yeah, those are often pretty limited in space

Supine•6/4/22, 2:35 PM

Unless Unified memory?

TTechPizza multiple threads were uploading constantly

KaiOP•6/4/22, 2:35 PM

I don't think I really understand what they are doing tbh, this should be 100% IO bound work

TechPizza•6/4/22, 2:35 PM

... aren't staging buffers just in RAM?

TTechPizza ... aren't staging buffers just in RAM?

Supine•6/4/22, 2:36 PM

Exactly

KaiOP•6/4/22, 2:36 PM

well it depends* they need to be in a space that the driver has access to

KaiOP•6/4/22, 2:36 PM

and on integrated GPUs that can get pretty funky

KKai 🤔 I don't think I really understand what they are doing tbh, this should be 100...

TechPizza•6/4/22, 2:37 PM

most of the time on the profiler was being spent in copying to mapped memory

KaiOP•6/4/22, 2:38 PM

I mean many of the modern AMD cards have a memory type that looks like

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT
DEVICE_COHERENT_BIT_AMD
DEVICE_UNCACHED_BIT_AMD

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT
DEVICE_COHERENT_BIT_AMD
DEVICE_UNCACHED_BIT_AMD

with like 8GB
but on some iGPUs the best you get is

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT

at very limited space

TTechPizza most of the time on the profiler was being spent in copying to mapped memory

KaiOP•6/4/22, 2:38 PM

hmm. Could be related to it being coherent if the implementation is weird

how are you copying?

TechPizza•6/4/22, 2:38 PM

Unsafe.CopyBlockUnaligned :)

TechPizza•6/4/22, 2:39 PM

i tried a normal for-loop but it was slightly slower

KaiOP•6/4/22, 2:39 PM

I'm not sure, but can you try using Span.CopyTo? pretty sure that is the fastest you can get

KaiOP•6/4/22, 2:39 PM

cause that uses fancy vector instructions and stuffs

TechPizza•6/4/22, 2:40 PM

was the same speed as Unsafe.CopyBlockUnaligned

KaiOP•6/4/22, 2:40 PM

ah ok :(

Supine•6/4/22, 2:40 PM

I think they call same underlying method with

ref byte

ref byte

KaiOP•6/4/22, 2:40 PM

maybe they unified those two at some point, I think I heard about that

KaiOP•6/4/22, 2:40 PM

hmmm

KaiOP•6/4/22, 2:40 PM

Fuck it, let me create a project

KaiOP•6/4/22, 2:41 PM

I wanted to rewrite type references today but I guess I'm torturing my GPU today

KaiOP•6/4/22, 4:00 PM

Haven't really build stuff with the new(ish) chain extensions, and man that opens up some awesome uses that I didn't even think of when this was designed:

var queueProps = new QueueFamilyProperties2[(int)queuePropCount];
var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount];
for (var i = 0; i < queueProps.Length; i++)
{
    QueueFamilyProperties2.Chain(out queueProps[i])
        .AddNext(out queueGlobalPrios[i]);
}

var queueProps = new QueueFamilyProperties2[(int)queuePropCount];
var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount];
for (var i = 0; i < queueProps.Length; i++)
{
    QueueFamilyProperties2.Chain(out queueProps[i])
        .AddNext(out queueGlobalPrios[i]);
}

Perksey•6/4/22, 4:19 PM

honestly @thargy is a fucking genius

KaiOP•6/5/22, 7:01 AM

Is it good when my audio driver starts to crash when I run my memory test

Lyris the Kitori•6/5/22, 7:05 AM

yes

KaiOP•6/5/22, 7:06 AM

apparently if you leak GPU local memory the driver has a hard time tracking which it was and just... clears all of it, bringing the audio memory down with it xD

KaiOP•6/5/22, 7:06 AM

AMD drivers on point as always

Lyris the Kitori•6/5/22, 7:06 AM

still better than nvidia

KaiOP•6/5/22, 7:06 AM

true

KaiOP•6/5/22, 8:16 AM

So getting 5% throughput right now (about 1.6 GB/s)

Lyris the Kitori•6/5/22, 8:18 AM

nice

KaiOP•6/5/22, 8:19 AM

Not really, kinda hoped for an immediate 60+% with this code

KaiOP•6/5/22, 10:41 AM

I can get almost 19 GB/s now, given everything aligns perfectly

KaiOP•6/5/22, 10:42 AM

With Generic block sizes a bit less, up to 16 GB/s

KaiOP•6/5/22, 10:43 AM

(This is doing 6 consecutive uploads of 1 GB each, then it waits for those to complete)

KaiOP•6/5/22, 10:45 AM

If I don't wait for any transfers to complete (up to 1TB of "in flight" data) I can get 90% peak throughput. so That's ~28 GB/s

KaiOP•6/5/22, 10:46 AM

Note that this is on PCIe 4.0 x16 and I'm just going off the Task Manager to figure out the throughput, but seems reasonable

KaiOP•6/5/22, 10:47 AM

Looks like the main thing slowing this down is the CPU waiting for things to complete. If there is stuff to load at all times 90% throughput is possible

KaiOP•6/5/22, 10:49 AM

Code: https://gist.github.com/HurricanKai/0125b844666b26b531bb204f099d513d, would be cool if someone can test this on some other devices (maybe even iGPU?) to see what can be achieved there

KaiOP•6/5/22, 10:49 AM

note that there are a bunch of hardcoded values in there and the program will just die when something goes wrong

KaiOP•6/5/22, 10:50 AM

Most important are the config values at line 92, that decides how much data is shoved down the lanes and in how large blocks

KaiOP•6/5/22, 10:51 AM

(by passing

localBlockSize

localBlockSize

instead of

ulong.MaxValue

ulong.MaxValue

as the last parameter on line 96 an ideal scenario can be created, where there is no wasted memory. This seems to have little to no influence on speed)

KaiOP•6/5/22, 10:52 AM

Also note that my device has 2 Transfer queues (I'm using both in round-robin here) & I'm not using Graphics or Compute queue here, so in theory there is zero interference with other parts, other then memory usage

var queueProps = new QueueFamilyProperties2[(int)queuePropCount]; var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount]; for (var i = 0; i < queueProps.Length; i++) { QueueFamilyProperties2.Chain(out queueProps[i]) .AddNext(out queueGlobalPrios[i]); }