test output from the generator

C

CurinOP•10/17/23, 7:41 PM

I just took over the pointergen project

C

CurinOP•10/17/23, 7:44 PM

it now generates all 3 levels

CCurin it now generates all 3 levels

P

Perksey•10/17/23, 7:50 PM

ah good, I was hoping you would.

Non-Generic

where T : unmanaged
where T : unmanaged
where T : unmanaged
where T : unmanaged

T doesn't seem to exist, this seems to be the non-generic variant

public readonly ref T Ref
public readonly ref T Ref
public readonly ref T Ref
public readonly ref T Ref

This needs to be

readonly ref MutPtr

readonly ref MutPtr

readonly ref MutPtr

readonly ref MutPtr (basically it's equivalent to

this[0]

this[0]

this[0]

this[0]. you could even implement it like that if you want)

public ref PTRMUT* GetPinnableReference()
public ref PTRMUT* GetPinnableReference()
public ref PTRMUT* GetPinnableReference()
public ref PTRMUT* GetPinnableReference()

This should be a reference to the raw pointer form of its inner dimension. to for

PtrMutPtr

PtrMutPtr

PtrMutPtr

PtrMutPtr this would be

ref readonly byte**

ref readonly byte**

ref readonly byte**

ref readonly byte** (so that when used in a

fixed

fixed

fixed

fixed statement it becomes

byte***

byte***

byte***

byte***)

public static bool operator ==(object? lh, PTRMUTPTRrh) => lh == null ? (void*)rh == null : false;
public static bool operator ==(object? lh, PTRMUTPTRrh) => lh == null ? (void*)rh == null : false;
public static bool operator ==(object? lh, PTRMUTPTRrh) => lh == null ? (void*)rh == null : false;
public static bool operator ==(object? lh, PTRMUTPTRrh) => lh == null ? (void*)rh == null : false;

Missed a space.

public static implicit operator byte******(PTRMUTPTR ptr) => (byte******)Unsafe.AsPointer(ref Unsafe.AsRef(in ptr.Ref));
public static implicit operator byte******(PTRMUTPTR ptr) => (byte******)Unsafe.AsPointer(ref Unsafe.AsRef(in ptr.Ref));
public static implicit operator byte******(PTRMUTPTR ptr) => (byte******)Unsafe.AsPointer(ref Unsafe.AsRef(in ptr.Ref));
public static implicit operator byte******(PTRMUTPTR ptr) => (byte******)Unsafe.AsPointer(ref Unsafe.AsRef(in ptr.Ref));

My my that's a little over 3 dimensions!

Also the to-pointer casts should be explicit instead of implicit - uses

Unsafe.AsPointer

Unsafe.AsPointer

Unsafe.AsPointer

Unsafe.AsPointer does not fix the

ref

ref

ref

ref and therefore should only be used for known-stack pointers (i.e. the user has to be explicit about this to ensure they don't accidentally do anything dangerous)

Generic

Same comments apply pretty much by the looks of things.

P

Perksey•10/17/23, 7:51 PM

thank you so much for doing this! this is one of the most integral parts of SilkX, and super glad to have community contributions here as a result.

P

Perksey•10/17/23, 7:52 PM

(I presume the type names are caps to prevent conflicts when trying it out)

M

Maxine•10/17/23, 7:52 PM

does all of this really need inline IL?

P

Perksey•10/17/23, 7:53 PM

only the single-dimension ones can be native C#

P

Perksey•10/17/23, 7:54 PM

you can't have ref fields to ref structs

M

Maxine•10/17/23, 7:54 PM

even with Unsafe.xxx?

M

Maxine•10/17/23, 7:54 PM

right

P

Perksey•10/17/23, 7:54 PM

can't use ref structs as generics

P

Perksey•10/17/23, 7:54 PM

or pointers as generics

M

Maxine•10/17/23, 7:54 PM

also why is it a ref byte

M

Maxine•10/17/23, 7:55 PM

from the looks of it the type of the ref doesn't matter, it's just odd to see byte there

P

Perksey•10/17/23, 7:55 PM

can't have a

ref void

ref void

ref void

ref void so

ref byte

ref byte

ref byte

ref byte was the next best thing

M

Maxine•10/17/23, 7:56 PM

also: should

GetPinnableReference()

GetPinnableReference()

GetPinnableReference()

GetPinnableReference() not return the fully unsafe pointer?

P

Perksey•10/17/23, 7:56 PM

yes this is one of my comments

M

Maxine•10/17/23, 7:56 PM

oh

M

Maxine•10/17/23, 7:56 PM

my bad

M

Maxine•10/17/23, 7:58 PM

does the conversion from

T[][][]

T[][][]

T[][][]

T[][][] actually work?

M

Maxine•10/17/23, 7:59 PM

like, does it return a T*** and not just a T[][]*?

P

Perksey•10/17/23, 7:59 PM

it doesn't look like it's correct right now, great spot, there's a little more nuance to it as expressed here: https://github.com/Perksey/SilkX/blob/main/sources/Core/SilkMarshal.cs#L219

M

Maxine•10/17/23, 8:00 PM

that's gonna be fun to free

P

Perksey•10/17/23, 8:01 PM

that's why SilkMarshal has the GCHandleSentinel

P

Perksey•10/17/23, 8:02 PM

we pin the arrays within, and then we make the lifetime of the pins dependent on the lifetime of the array we retur

P

Perksey•10/17/23, 8:02 PM

when the Ptr referencing that array goes out of scope, the sentinel gets collected, which frees the GCHandle, which allows the GC to collect the inner arrays

M

Maxine•10/17/23, 8:05 PM

i seeeee

C

CurinOP•10/17/23, 8:06 PM

Thanks guys, I'm down to just the final JaggedArray issue

P

Perksey•10/17/23, 8:06 PM

haha yeah sorry about that, i should've warned you about the nuances there

C

CurinOP•10/17/23, 8:06 PM

nbd, honestly it was at least good that most of the issues were typos on my part or simple changes.

C

CurinOP•10/17/23, 8:08 PM

would just using the SilkMarshal function work in this regard?

C

CurinOP•10/17/23, 8:08 PM

or will I need something to handle the higher dimension arrays

P

Perksey•10/17/23, 8:13 PM

I think the existing function is only written for 2 dimensions

C

CurinOP•10/17/23, 8:13 PM

yeah, just did some checking

P

Perksey•10/17/23, 8:14 PM

but adding a 3 dimension overload should be trivial, it's basically "do the same thing but call into the 2 dimension overload for each"

C

CurinOP•10/17/23, 8:15 PM

yeah, might even try for a 4d overload so we are prepared if someone needs to generate a 4 dimension type, since everything will break if the function doesn't exist

C

CurinOP•10/17/23, 8:20 PM

though if I just use the 2d to create the 3d we will end up with T*[][], will that still work?

CCurin though if I just use the 2d to create the 3d we will end up with T*[][], will th...

P

Perksey•10/17/23, 8:25 PM

I'm confused, isn't it

-    public static T*[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[]> array)
+    public static T**[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[][]> array)
        where T : unmanaged
    {
-        var ret = new T*[array.Length];
+        var ret = new T**[array.Length];
        for (var i = 0; i < array.Length; i++)
        {
-           var handle = GCHandle.Alloc(array[i], GCHandleType.Pinned);
+           var handle = GCHandle.Alloc(JaggedArrayToPointerArray(array[i]), GCHandleType.Pinned);
            var sentinel = new GCHandleSentinel(handle);
            var dep = new DependentHandle(ret, sentinel);
            if (!dep.IsAllocated)
            {
                throw new InvalidOperationException(
                    "Failed to allocate dependent handle to keep string alive"
                );
            }

            sentinel.DependentHandle = dep;
-           ret[i] = (T*)handle.AddrOfPinnedObject();
+           ret[i] = (T**)handle.AddrOfPinnedObject();
        }

        return ret;
    }

-    public static T*[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[]> array)
+    public static T**[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[][]> array)
        where T : unmanaged
    {
-        var ret = new T*[array.Length];
+        var ret = new T**[array.Length];
        for (var i = 0; i < array.Length; i++)
        {
-           var handle = GCHandle.Alloc(array[i], GCHandleType.Pinned);
+           var handle = GCHandle.Alloc(JaggedArrayToPointerArray(array[i]), GCHandleType.Pinned);
            var sentinel = new GCHandleSentinel(handle);
            var dep = new DependentHandle(ret, sentinel);
            if (!dep.IsAllocated)
            {
                throw new InvalidOperationException(
                    "Failed to allocate dependent handle to keep string alive"
                );
            }

            sentinel.DependentHandle = dep;
-           ret[i] = (T*)handle.AddrOfPinnedObject();
+           ret[i] = (T**)handle.AddrOfPinnedObject();
        }

        return ret;
    }

-    public static T*[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[]> array)
+    public static T**[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[][]> array)
        where T : unmanaged
    {
-        var ret = new T*[array.Length];
+        var ret = new T**[array.Length];
        for (var i = 0; i < array.Length; i++)
        {
-           var handle = GCHandle.Alloc(array[i], GCHandleType.Pinned);
+           var handle = GCHandle.Alloc(JaggedArrayToPointerArray(array[i]), GCHandleType.Pinned);
            var sentinel = new GCHandleSentinel(handle);
            var dep = new DependentHandle(ret, sentinel);
            if (!dep.IsAllocated)
            {
                throw new InvalidOperationException(
                    "Failed to allocate dependent handle to keep string alive"
                );
            }

            sentinel.DependentHandle = dep;
-           ret[i] = (T*)handle.AddrOfPinnedObject();
+           ret[i] = (T**)handle.AddrOfPinnedObject();
        }

        return ret;
    }

-    public static T*[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[]> array)
+    public static T**[] JaggedArrayToPointerArray<T>(ReadOnlySpan<T[][]> array)
        where T : unmanaged
    {
-        var ret = new T*[array.Length];
+        var ret = new T**[array.Length];
        for (var i = 0; i < array.Length; i++)
        {
-           var handle = GCHandle.Alloc(array[i], GCHandleType.Pinned);
+           var handle = GCHandle.Alloc(JaggedArrayToPointerArray(array[i]), GCHandleType.Pinned);
            var sentinel = new GCHandleSentinel(handle);
            var dep = new DependentHandle(ret, sentinel);
            if (!dep.IsAllocated)
            {
                throw new InvalidOperationException(
                    "Failed to allocate dependent handle to keep string alive"
                );
            }

            sentinel.DependentHandle = dep;
-           ret[i] = (T*)handle.AddrOfPinnedObject();
+           ret[i] = (T**)handle.AddrOfPinnedObject();
        }

        return ret;
    }

C

CurinOP•10/17/23, 8:26 PM

okay that is how you are handling it, I started going down this route after I sent that message

C

CurinOP•10/17/23, 8:26 PM

I was going too simple for the overload

P

Perksey•10/17/23, 8:27 PM

ah yeah, i think given we're really restricted on what we can have in generics i don't think we can be more generalised

C

CurinOP•10/17/23, 8:28 PM

 IL.Emit.Ldarg_0();
IL.Emit.Ldc_I4_0();
IL.Emit.Ldelema(TypeRef.Type(typeof(T).MakePointerType()));
IL.Emit.Newobj(
    MethodRef.Constructor(
        TypeRef.Type(typeof(ConstPtr2D<>).MakeGenericType(typeof(T))),
        TypeRef.Type(typeof(ConstPtr<>).MakeGenericType(typeof(T)).MakeByRefType())
    )
);
IL.Emit.Ret();
throw IL.Unreachable();

 IL.Emit.Ldarg_0();
IL.Emit.Ldc_I4_0();
IL.Emit.Ldelema(TypeRef.Type(typeof(T).MakePointerType()));
IL.Emit.Newobj(
    MethodRef.Constructor(
        TypeRef.Type(typeof(ConstPtr2D<>).MakeGenericType(typeof(T))),
        TypeRef.Type(typeof(ConstPtr<>).MakeGenericType(typeof(T)).MakeByRefType())
    )
);
IL.Emit.Ret();
throw IL.Unreachable();

 IL.Emit.Ldarg_0();
IL.Emit.Ldc_I4_0();
IL.Emit.Ldelema(TypeRef.Type(typeof(T).MakePointerType()));
IL.Emit.Newobj(
    MethodRef.Constructor(
        TypeRef.Type(typeof(ConstPtr2D<>).MakeGenericType(typeof(T))),
        TypeRef.Type(typeof(ConstPtr<>).MakeGenericType(typeof(T)).MakeByRefType())
    )
);
IL.Emit.Ret();
throw IL.Unreachable();

 IL.Emit.Ldarg_0();
IL.Emit.Ldc_I4_0();
IL.Emit.Ldelema(TypeRef.Type(typeof(T).MakePointerType()));
IL.Emit.Newobj(
    MethodRef.Constructor(
        TypeRef.Type(typeof(ConstPtr2D<>).MakeGenericType(typeof(T))),
        TypeRef.Type(typeof(ConstPtr<>).MakeGenericType(typeof(T)).MakeByRefType())
    )
);
IL.Emit.Ret();
throw IL.Unreachable();

C

CurinOP•10/17/23, 8:29 PM

and that is what actually goes in the function to convert our type to T* * * from T* *[]

C

CurinOP•10/17/23, 8:29 PM

or something like that atleast I just pulled that from ConstPtr2D

P

Perksey•10/17/23, 8:34 PM

ah yeah, that's an easy case though because we already know that we have a raw (untracked) pointer within the array, so the GC can just keep that array alive and not care about the pointers contained therein

P

Perksey•10/17/23, 8:35 PM

the jagged arrays are more complicated because ref fields aren't allowed for a reason - the GC currently can't see a ref within a ref so we have to manually keep the inner arrays alive

P

Perksey•10/17/23, 8:36 PM

this may immediately ring alarm bells like "but wait, that means a PtrPtr being created from a ref Ptr is super dangerous" and you'd be right if it weren't for the fact that we know that ref structs are on the stack anyway, so we don't need to worry about this

C

CurinOP•10/17/23, 8:37 PM

yep, ref structs are nice like that, its why you can really do some fun stuff with spans

M

Maxine•10/17/23, 8:39 PM

i can't wait until we have spans of spans/const generics/ref struct fields and all of this goes to waste

CCurin test output from the generator

P

Perksey•10/17/23, 8:39 PM

oh also don't forget being implicitly convertible from

NullPtr

NullPtr

. basically just go over the existing manual types and see if there's anything missing from the generated ones (as this is basically what i'm doing now)

test output from the generator

Non-Generic

Generic

Similar Threads

Non-Generic

Generic