does the JIT not vectorize float math

MMaxine does the JIT not vectorize float math

L

Lyris the Kitori•8/12/23, 7:31 AM

it does, but System.Numerics is still faster, always, since it has special JIT codepaths

M

MaxineOP•8/12/23, 7:31 AM

fair

L

Lyris the Kitori•8/12/23, 7:32 AM

i believe System.Numerics has hand-written ASM for x86 and arm

M

MaxineOP•8/12/23, 7:32 AM

if you need extra functions can't you just change them to be extension methods on Matrix4x4

L

Lyris the Kitori•8/12/23, 7:32 AM

but theres also the need for it to be a generic anyway

L

Lyris the Kitori•8/12/23, 7:32 AM

so might as well just have a separate type

M

MaxineOP•8/12/23, 7:32 AM

i see

L

Lyris the Kitori•8/12/23, 7:33 AM

there are genuine cases for a double precision backed matrix

M

MaxineOP•8/12/23, 7:33 AM

you could have the best of both worlds by calling into system.numerics methods if T is a float

L

Lyris the Kitori•8/12/23, 7:34 AM

then you have function call overheads, and branching overhead

M

MaxineOP•8/12/23, 7:34 AM

JIT will get rid of it

L

Lyris the Kitori•8/12/23, 7:34 AM

will it?

L

Lyris the Kitori•8/12/23, 7:34 AM

can you be 100% certain

L

Lyris the Kitori•8/12/23, 7:34 AM

what about NAOT scenarios?

M

MaxineOP•8/12/23, 7:34 AM

yeah,

typeof(T) == typeof(other)

typeof(T) == typeof(other)

typeof(T) == typeof(other)

typeof(T) == typeof(other) gets elided

L

Lyris the Kitori•8/12/23, 7:34 AM

what about Mono?

L

Lyris the Kitori•8/12/23, 7:34 AM

MonoAOT?

M

MaxineOP•8/12/23, 7:40 AM

M

MaxineOP•8/12/23, 7:40 AM

i'll get back to you on that

K

Kai•8/12/23, 10:08 AM

no calling into System.Numerics is possible

K

Kai•8/12/23, 10:08 AM

it just makes everything so much harder

K

Kai•8/12/23, 10:09 AM

typeof(T) == typeof(float)

typeof(T) == typeof(float)

typeof(T) == typeof(float)

typeof(T) == typeof(float) is how our generic Scalar type works everywhere.

K

Kai•8/12/23, 10:10 AM

The problem is more that what you'd get is something like

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

// Scalar based implementation

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

// Scalar based implementation

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

// Scalar based implementation

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

// Scalar based implementation

K

Kai•8/12/23, 10:11 AM

now how is the JIT supposed to treat this? Obviously the branch will be elided with instantiation, so it seems likely that if T==float, the Numerics call is inlined and no further code is generated

K

Kai•8/12/23, 10:11 AM

but what to do from the outside? You get a call to F<T> somewhere - what to do? There's quite a lot of IL in that function, so heuristics indicate not to inline - but if T==float this is a big performance hit.

K

Kai•8/12/23, 10:12 AM

We can't blindly put AggressiveInlining here, the Scalar implementation should often not be inlined!

K

Kai•8/12/23, 10:13 AM

the solution looks something like this

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

static TOrRes Inner(param1 T, ...)
{
    // Scalar based implementation
}
return Inner(param1, ...);

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

static TOrRes Inner(param1 T, ...)
{
    // Scalar based implementation
}
return Inner(param1, ...);

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

static TOrRes Inner(param1 T, ...)
{
    // Scalar based implementation
}
return Inner(param1, ...);

if (typeof(T) == typeof(float)) return System.Numerics.<T>.F(Scalar.As(param1), ...);

static TOrRes Inner(param1 T, ...)
{
    // Scalar based implementation
}
return Inner(param1, ...);

K

Kai•8/12/23, 10:14 AM

now we can safely mark F<T> as AggressiveInlining, and JIT is happy

K

Kai•8/12/23, 10:14 AM

Well, at least hopefully. Even an extra function indirection with AggressiveInlining isn't free. It's almost free, but not quite. Heuristics will still be confused

K

Kai•8/12/23, 10:15 AM

now that ends up being a lot of code to add everywhere, so a huge effort. and it may even introduce some problems...

L

Lyris the Kitori•8/12/23, 10:16 AM

this would be a lot simpler if we had some sort of compile time metaprogramming, allowing us to bypass the jit problems and properly specialize the function for the type param :^)

K

Kai•8/12/23, 10:16 AM

(It's also worth mentioning that usually System.Numerics doesn't have a static function, but instead you're forced to construct an instance, but we could probably just reinterpret that and JIT might end up being happy (or not, I mean the Matrix4x4 is still a special type, and reinterpreting should just cause a vector load, but sometimes JIT likes to die))

K

Kai•8/12/23, 10:17 AM

All of these inner core functions in Silk are heavily optimized, working around quirks and issues in .NET JIT past and present, and adding virtually anything to it is bound to be complicated and a lot of work

K

Kai•8/12/23, 10:18 AM

Like, you can find a solid 4 weeks of discord discussion in #team-talk where all I was doing is essentially hand optimizing the C# code we generate when calling into native libs... any change to that and you can pop up the JIT ASM inspector and spent a lot of time

K

Kai•8/12/23, 10:19 AM

(and similar for Scalar, although often not quite as extensive, as it's a bit simpler overall)

LLyris the Kitori this would be a lot simpler if we had some sort of compile time metaprogramming,...

A

Aqua•8/12/23, 10:20 AM

.net 7 generic math?

K

Kai•8/12/23, 10:23 AM

that is not even getting close to what you'd need to solve this effectively

K

Kai•8/12/23, 10:23 AM

but there will likely never be a good solution for this in C#/.NET

L

Lyris the Kitori•8/12/23, 10:23 AM

unfortanately i doubt so too

L

Lyris the Kitori•8/12/23, 10:24 AM

unless we get some really cursed shit C# just isnt built to allow this kind of thing cleanly

K

Kai•8/12/23, 10:24 AM

best we can do is moving all this fuckery into BCL and performance tests covering it there, so JIT (and other) PRs that affect this get flagged automagically