C#•3y ago

❔ reduce CPU/GPU transfer ComputeSharp

I'm working on a project with ComputeSharp, and I'm struggling with the problem where I need to run a lot of compute shaders one after another. Is there a way to control this process from the GPU, so that my code isn't significantly slowed down? Or how should I go about this?
I can't just move it all into one shader because some of them depend on the results of the previous one and there is obviously no way to synchronize.

DKMK100OP•4/6/23, 9:53 PM

I think a ComputeContext from ComputeSharp 2.0 is what I want

Buddy•4/7/23, 4:36 AM

@Sergio Maybe something for you LETSFUCKINGdood

DDKMK100 I think a ComputeContext from ComputeSharp 2.0 is what I want

Sergio•4/7/23, 11:45 AM

Correct, you should use that to create a single pipeline with all your shaders and execute them in a single batch

Sergio•4/7/23, 11:45 AM

Remember to also insert the appropriate barriers where needed

DKMK100OP•4/7/23, 8:05 PM

I wasn't able to find any documentation on those pipelines, how do they work?
Also, what do I do if I need a list of texture2ds? I can't have an array on the CPU obv but there's no arrays on the GPU and the dimensions of a texture3D are too limiting

DKMK100OP•4/7/23, 8:06 PM

Unless you think I should flatten weights into a 1d buffer?

DKMK100OP•4/7/23, 8:27 PM

@Sergio I legit have no idea how I'm meant to store my neural net layers on the GPU...

DDKMK100 I wasn't able to find any documentation on those pipelines, how do they work? A...

Sergio•4/7/23, 8:54 PM

Have you read the wiki in the repo?

SSergio Have you read the wiki in the repo?

DKMK100OP•4/7/23, 10:41 PM

the regular github wiki? it doesn't seem to have much info beyond the getting started section, and none of the 5 pages even mention piplines or ComputeContext

or is there another wiki I'm missing?

Sergio•4/7/23, 10:45 PM

Oh, I thought I had added wikis for that too

Sergio•4/7/23, 10:46 PM

Anyway it's relatively easy, you can see an example here: https://github.com/Sergio0694/ComputeSharp/blob/78c5ae0e6ce8acfa6c5760ade8aee997fe7089f8/samples/ComputeSharp.ImageProcessing/Processors/HlslGaussianBlurProcessor%7BTPixel%7D.cs#L107

Sergio•4/7/23, 10:47 PM

Just create a compute context in a using, and use methods on it. There's lots of XML docs on all the APIs with more info

Sergio•4/7/23, 10:47 PM

You can also find lots of examples in the unit tests

Sergio•4/7/23, 10:47 PM

Also if you have a specific question on something I can help maybe, this otherwise is pretty general

DKMK100OP•4/7/23, 11:09 PM

right now I have a list of texture2D references on the CPU that I loop through and do stuff on one at a time, and the calculation depends on the previous one

what I want is a way to do this without having to go back to a list on the CPU each time

I don't know how I can save the data on the GPU though since the structure doesn't play nicely, do I just flatten it?

as for the compute context, can I just put a loop there and run the shader in the context that many times? Or how do I set up a pipeline with a dynamic amount of steps like that?

DKMK100OP•4/7/23, 11:33 PM

@Sergio
HLSL supports a Texture2DArray, is there a way to use that from ComputeSharp? I don't see one...

Sergio•4/8/23, 1:02 AM

I don't support texture arrays, no

Sergio•4/8/23, 1:02 AM

But why are you using textures at all

Sergio•4/8/23, 1:02 AM

If you're working with ND tensors in a neural network, you'll want to just use structured buffers

Sergio•4/8/23, 1:02 AM

Textures are for when you need to do sampling, which is not the case here

Sergio•4/8/23, 1:03 AM

Use a buffer and manually index things

Sergio•4/8/23, 1:03 AM

I have a sample for this too

Sergio•4/8/23, 1:03 AM

Eg. here's a fully connected neural network layer (with no activation): https://github.com/Sergio0694/ComputeSharp/blob/78c5ae0e6ce8acfa6c5760ade8aee997fe7089f8/samples/ComputeSharp.Benchmark/Blas/BlasHelpers.cs#L84

Sergio•4/8/23, 1:05 AM

Maybe I could build a full neural network sample with ComputeSharp at some point, even just something simple like a network to classify images from the MNIST dataset

SSergio But why are you using textures at all

DKMK100OP•4/8/23, 2:05 AM

what about convolutional layers? lol
textures would be pretty useful there...

SSergio I don't support texture arrays, no

DKMK100OP•4/8/23, 2:07 AM

is there a particular reason for that? or just never found the need to?
they seem pretty useful in some contexts

DDKMK100 what about convolutional layers? lol textures would be pretty useful there...

Sergio•4/8/23, 10:55 AM

Not really, no

Sergio•4/8/23, 10:55 AM

You'd still not need to sample, so you still don't need a texture