table serialization adventures MessagePack
asp.netlinqadvanced
I'm serializing arbitrary tables of data from an API into a blazor wasm application - since DataTable cannot be serialized anymore, i need to do it myself.
For a table with 6000 rows and 75 columns, using newtonsoft for our (given, kinda naive) table implementation of basically List<List<object>> took 8 seconds (with the debugger running in VS, i need to reduce perf for us poor devs, for now)
Changing to System.Text.Json, writing custom serialization and deserialization brought this to about 3.5 seconds
Prefilling the target lists and accessing the "cells" using
I then switched to MessagePack, keeping those optimizations, which brought the deserialization to about 1.5 seconds.
i'm considering maybe getting rid of the per-column-type decision on which deserialization to use in favor of a pre-filled array of method invocations for that, so a
that i fill only once and then iterate
and i was wondering if that could be improved, maybe by compiling it using
this is the code that loops over the rows:
For a table with 6000 rows and 75 columns, using newtonsoft for our (given, kinda naive) table implementation of basically List<List<object>> took 8 seconds (with the debugger running in VS, i need to reduce perf for us poor devs, for now)
Changing to System.Text.Json, writing custom serialization and deserialization brought this to about 3.5 seconds
Prefilling the target lists and accessing the "cells" using
CollectionsMarshal.AsSpan<object?>I then switched to MessagePack, keeping those optimizations, which brought the deserialization to about 1.5 seconds.
i'm considering maybe getting rid of the per-column-type decision on which deserialization to use in favor of a pre-filled array of method invocations for that, so a
that i fill only once and then iterate
and i was wondering if that could be improved, maybe by compiling it using
System.Linq.Expressionrefthis is the code that loops over the rows:
