Silk.NET•4y ago

can you share the transfer code? Would love to have a look at it

SSupine `File.Open("pog.bin").CopyTo(new UnmanagedMemoryStream(mapped, len, len));` 😁

KaiOP•6/4/22, 2:29 PM

something like that is actually possible

KKai I'm just guessing it's using x16, so that'd put it at a max of 15.754 GB/s accor...

TechPizza•6/4/22, 2:29 PM

that validates the readout from task manager; 800 / 0.05 = 16000

KKai something like that is actually possible 😛

Supine•6/4/22, 2:30 PM

(it's bad to distribute uncompressed resources)

KaiOP•6/4/22, 2:30 PM

could throw a deflate stream in the middle don't think that'd be a problem

KaiOP•6/4/22, 2:31 PM

problem is more that you'd need to build some super smart logic to automagically advance the underlying unmanaged memory stream to the right location

KaiOP•6/4/22, 2:31 PM

cause your resources are very likely much larger than you can fit in host-mapped memory

KKai can you share the transfer code? Would love to have a look at it

TechPizza•6/4/22, 2:31 PM

there's nothing more to it than map, write, unmap, copycommand, submit, wait for fence

TechPizza•6/4/22, 2:32 PM

and the old code was so unreadable that i'm rewriting the hole thing

KaiOP•6/4/22, 2:32 PM

well how big are the blocks you're copying here?

TechPizza•6/4/22, 2:34 PM

i was roughly doing 8MB uploads

KaiOP•6/4/22, 2:34 PM

and then in sequence or in parallel?

KKai cause your resources are very likely much larger than you can fit in host-mapped...

Supine•6/4/22, 2:35 PM

(i meant mapped staging buffers)

TechPizza•6/4/22, 2:35 PM

multiple threads were uploading constantly

SSupine (i meant mapped staging buffers)

KaiOP•6/4/22, 2:35 PM

yeah, those are often pretty limited in space

Supine•6/4/22, 2:35 PM

Unless Unified memory?

TTechPizza multiple threads were uploading constantly

KaiOP•6/4/22, 2:35 PM

I don't think I really understand what they are doing tbh, this should be 100% IO bound work

TechPizza•6/4/22, 2:35 PM

... aren't staging buffers just in RAM?

TTechPizza ... aren't staging buffers just in RAM?

Supine•6/4/22, 2:36 PM

Exactly

KaiOP•6/4/22, 2:36 PM

well it depends* they need to be in a space that the driver has access to

KaiOP•6/4/22, 2:36 PM

and on integrated GPUs that can get pretty funky

KKai 🤔 I don't think I really understand what they are doing tbh, this should be 100...

TechPizza•6/4/22, 2:37 PM

most of the time on the profiler was being spent in copying to mapped memory

KaiOP•6/4/22, 2:38 PM

I mean many of the modern AMD cards have a memory type that looks like

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT
DEVICE_COHERENT_BIT_AMD
DEVICE_UNCACHED_BIT_AMD

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT
DEVICE_COHERENT_BIT_AMD
DEVICE_UNCACHED_BIT_AMD

with like 8GB
but on some iGPUs the best you get is

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT

DEVICE_LOCAL_BIT
HOST_VISIBLE_BIT
HOST_COHERENT_BIT

at very limited space

TTechPizza most of the time on the profiler was being spent in copying to mapped memory

KaiOP•6/4/22, 2:38 PM

hmm. Could be related to it being coherent if the implementation is weird

how are you copying?

TechPizza•6/4/22, 2:38 PM

Unsafe.CopyBlockUnaligned :)

TechPizza•6/4/22, 2:39 PM

i tried a normal for-loop but it was slightly slower

KaiOP•6/4/22, 2:39 PM

I'm not sure, but can you try using Span.CopyTo? pretty sure that is the fastest you can get

KaiOP•6/4/22, 2:39 PM

cause that uses fancy vector instructions and stuffs

TechPizza•6/4/22, 2:40 PM

was the same speed as Unsafe.CopyBlockUnaligned

KaiOP•6/4/22, 2:40 PM

ah ok :(

Supine•6/4/22, 2:40 PM

I think they call same underlying method with

ref byte

ref byte

KaiOP•6/4/22, 2:40 PM

maybe they unified those two at some point, I think I heard about that

KaiOP•6/4/22, 2:40 PM

hmmm

KaiOP•6/4/22, 2:40 PM

Fuck it, let me create a project

KaiOP•6/4/22, 2:41 PM

I wanted to rewrite type references today but I guess I'm torturing my GPU today

KaiOP•6/4/22, 4:00 PM

Haven't really build stuff with the new(ish) chain extensions, and man that opens up some awesome uses that I didn't even think of when this was designed:

var queueProps = new QueueFamilyProperties2[(int)queuePropCount];
var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount];
for (var i = 0; i < queueProps.Length; i++)
{
    QueueFamilyProperties2.Chain(out queueProps[i])
        .AddNext(out queueGlobalPrios[i]);
}

var queueProps = new QueueFamilyProperties2[(int)queuePropCount];
var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount];
for (var i = 0; i < queueProps.Length; i++)
{
    QueueFamilyProperties2.Chain(out queueProps[i])
        .AddNext(out queueGlobalPrios[i]);
}

Perksey•6/4/22, 4:19 PM

honestly @thargy is a fucking genius

KaiOP•6/5/22, 7:01 AM

Is it good when my audio driver starts to crash when I run my memory test

Lyris the Kitori•6/5/22, 7:05 AM

yes

KaiOP•6/5/22, 7:06 AM

apparently if you leak GPU local memory the driver has a hard time tracking which it was and just... clears all of it, bringing the audio memory down with it xD

KaiOP•6/5/22, 7:06 AM

AMD drivers on point as always

Lyris the Kitori•6/5/22, 7:06 AM

still better than nvidia

KaiOP•6/5/22, 7:06 AM

true

KaiOP•6/5/22, 8:16 AM

So getting 5% throughput right now (about 1.6 GB/s)

Lyris the Kitori•6/5/22, 8:18 AM

nice

KaiOP•6/5/22, 8:19 AM

Not really, kinda hoped for an immediate 60+% with this code

KaiOP•6/5/22, 10:41 AM

I can get almost 19 GB/s now, given everything aligns perfectly

KaiOP•6/5/22, 10:42 AM

With Generic block sizes a bit less, up to 16 GB/s

KaiOP•6/5/22, 10:43 AM

(This is doing 6 consecutive uploads of 1 GB each, then it waits for those to complete)

KaiOP•6/5/22, 10:45 AM

If I don't wait for any transfers to complete (up to 1TB of "in flight" data) I can get 90% peak throughput. so That's ~28 GB/s

KaiOP•6/5/22, 10:46 AM

Note that this is on PCIe 4.0 x16 and I'm just going off the Task Manager to figure out the throughput, but seems reasonable

var queueProps = new QueueFamilyProperties2[(int)queuePropCount]; var queueGlobalPrios = new QueueFamilyGlobalPriorityPropertiesKHR[(int)queuePropCount]; for (var i = 0; i < queueProps.Length; i++) { QueueFamilyProperties2.Chain(out queueProps[i]) .AddNext(out queueGlobalPrios[i]); }