C#•3y ago

✅ ComputeSharp and activation functions

I'm working on making a neural net that I'm running on the GPU with the help of a library called ComputeSharp, and I want to use only pre-compiled shaders, but I also wanna have multiple activation function options so each layer can use a different activation function.

The thing is, I have no idea how I can do this without manually writting a bunch of different versions of each shader, for use with each activation function. I can't pass the activation function as a parameter since that would mean having to compile the shaders at runtime, and I'm hoping to avoid that. Is there an alternative I can use, possibly something akin to c++ templates?

Anyone here familiar with the library or know a solution I can use?

Aaron•3/19/23, 7:24 PM

@Sergio

Sergio•3/19/23, 7:25 PM

Ooh cool ahah

Sergio•3/19/23, 7:25 PM

"Anyone here familiar with the library"

I wrote the library, you could say I'm somewhat familiar when

Sergio•3/19/23, 7:27 PM

"I can't pass the activation function as a parameter since that would mean having to compile the shaders at runtime, and I'm hoping to avoid that."

@DKMK100 No, there's no way to do that. I've added the metaprogramming support (capturing delegates in a shader) pretty much to support exactly this scenario (activation functions in neural networks), but that has the tradeoff that you can no longer precompile shaders. That said, are you sure this is actually an issue? Ie. precompiling shaders is still relatively fast, and especially if you're training a neural network, which takes hours and hours, that's pretty much negligible

Sergio•3/19/23, 7:28 PM

I could in theory add some way to precompile combinations, but it's not implemented currently

DKMK100OP•3/19/23, 7:29 PM

hmm, that's a shame, I was hoping there'd be an equivalent to c++ templates where I could have it precompile a version for each activation function

Sergio•3/19/23, 7:29 PM

Is there a particular reason you wanted to precompile everything?

Sergio•3/19/23, 7:29 PM

I mean yes the extra startup speed is nice, but is it impactful in this context?

Sergio•3/19/23, 7:30 PM

And have you benchmarked how long it takes to compile the shader anyway?

DKMK100OP•3/19/23, 7:30 PM

I'm mostly worried if it's runtime compiled it might not be as high quality, this line is concerning:
"Additionally, precompiling shaders makes the code less error prone" but maybe I'm interpreting that wrong

DKMK100OP•3/19/23, 7:31 PM

the way the wiki on github is written sort of implies precompiled shaders will be better when running on the GPU and obviously if I'm running it that many times I want it to have any compile-time optimizations possible

DKMK100OP•3/19/23, 7:33 PM

is that actually a concern? or are they compiled the same way in both cases?

Sergio•3/19/23, 7:34 PM

No, the code quality is identical. It's exactly the same compiler being used

DKMK100OP•3/19/23, 7:35 PM

alright, guess I'll use runtime compilation then

DKMK100OP•3/19/23, 7:40 PM

do I need to pass the shader method delegate directly into the constructor of the compute shader, or can I save that somewhere else (eg. the neural net layer) and pass that into the constructor of the compute shader? Not sure how you keep track of valid shader methods

Sergio•3/19/23, 7:49 PM

You can set the delegate to the field whenever you want, the only thing that matters is that it's there when you try to run the shader. So you can either set it on construction, or assign the field later on

Sergio•3/19/23, 7:50 PM

Btw have you seen the sample in the benchmark project?

Sergio•3/19/23, 7:50 PM

There's a fully connected neural network minimal example there

Sergio•3/19/23, 7:50 PM

(with no activation, but easy enough to add)

SSergio Btw have you seen the sample in the benchmark project?

DKMK100OP•3/19/23, 7:52 PM

I have not, no

Sergio•3/19/23, 7:55 PM

See: https://github.com/Sergio0694/ComputeSharp/tree/main/samples/ComputeSharp.Benchmark/Blas

Sergio•3/19/23, 7:56 PM

Also @DKMK100 in case it's useful for reference: https://github.com/Sergio0694/NeuralNetwork.NET

Sergio•3/19/23, 7:56 PM

I've implemented fully connected and convolutional neural networks from scratch in C# there

Sergio•3/19/23, 7:56 PM

Including a lot of activation functions and cost functions, all with gradients as well

Sergio•3/19/23, 7:57 PM

Could be useful to eg. port them to shaders, if you're not already 100% familiar with the math

Sergio•3/19/23, 7:57 PM

Because deriving all the formulas from scratch is a fair bit of work

Sergio•3/19/23, 7:57 PM

Anyway, hope that helps!

DKMK100OP•3/19/23, 8:01 PM

that helps a lot, thank you so much!

DKMK100OP•3/19/23, 10:35 PM

erm, how does allocating a new texture2D work?
indexing is backwards on a texture2D on GPU vs the array its copied from, so when saying i want one of a certain size, which size goes first?

Sergio•3/19/23, 11:24 PM

You can read the parameter names

Sergio•3/19/23, 11:24 PM

Textures are in row major order, so the order matches that

Sergio•3/19/23, 11:24 PM

It's width, height

DKMK100OP•3/20/23, 3:27 AM

how can I make a list of a bunch of 2D textures that are larger than 2048?

DKMK100OP•3/20/23, 3:27 AM

do I flatten one of the dimensions and use a very cursed single 2D texture

DDKMK100 how can I make a list of a bunch of 2D textures that are larger than 2048?

Sergio•3/20/23, 12:09 PM

What do you mean? The maximum size per axis in a 2D texture is 16.384

Sergio•3/20/23, 12:09 PM

You should be perfectly able to just have a 2D texture "larger than 2048"

DKMK100OP•3/20/23, 3:32 PM

yea but how do I have a lot of them that I access in sequence?

DKMK100OP•3/20/23, 3:32 PM

a 3d texture can only be 2048 by 2048 by 2048 so I can't use on of those

DKMK100OP•3/20/23, 3:32 PM

but nothing else approximates a list...

DKMK100OP•3/20/23, 3:34 PM

if I just have an array of them outside the GPU, won't that slow me down significantly?

DKMK100OP•3/20/23, 3:34 PM

I guess I can test later today, but sounds pretty slow

Mifury•3/20/23, 4:14 PM

It sounds like you're looking for a way to use multiple activation functions with pre-compiled shaders, without having to manually write a separate version of each shader for each activation function. One approach you could consider is using template metaprogramming techniques.
Template metaprogramming is a technique used in C++ to generate code at compile-time based on template parameters. The idea is to write code that is generic enough to work with multiple types, and then instantiate the template with the specific type parameters at compile-time. This can result in more efficient code since it avoids the need for runtime polymorphism, and it can also simplify the code by eliminating the need for explicit type checks.

Sergio•3/20/23, 5:18 PM

ComputeSharp already supports this, it's just that there's no support to also precompile the combinations at compile time to a shader, they just get transpiled to HLSL. I'd need to add more annotations specifically for this, but it hasn't been a priority

DDKMK100 a 3d texture can only be 2048 by 2048 by 2048 so I can't use on of those

Sergio•3/20/23, 5:19 PM

Ah I get it, is this for like a 3D tensor, say, a 4k frame with 3 channels for RGB?

Sergio•3/20/23, 5:19 PM

You might want to just use a structured buffer for this and manually index

Sergio•3/20/23, 5:19 PM

Textures are mostly used when you need to interpolate, but you don't need that here

Sergio•3/20/23, 5:19 PM

So there's not much benefit anyway from using them

DKMK100OP•3/20/23, 7:05 PM

Update: I'm stupid and was doing something dumb

DKMK100OP•3/20/23, 7:05 PM

I need 3 2d operations to replace my one 3d operation

DKMK100OP•3/20/23, 7:06 PM

Which will solve my problem

✅ ComputeSharp and activation functions

Similar Threads

Similar Threads

Similar Threads