C#C
C#•2y ago
dreadfullydistinct

✅ Writing a source generator for this use case

I have some JSON data I want to load at startup, but image size has become a concern since in the published image these files are taking up 80MB. The general pattern I am using at the minute is

public static class MasterAsset
{
    public static MasterAssetData<Charas, CharaData> CharaData { get; private set; } = null!;

    public static async Task LoadAsync()
    {
        ValueTask<MasterAssetData<Charas, CharaData>> charaData = MasterAssetData.LoadAsync<
            Charas,
            CharaData
        >("CharaData.json", x => x.Id);

        ValuteTask<MasterAssetData<Other, Other>> other = ...
        
        CharaData = await charaData;
        Other = await other;
    }
}

Note that there are about 100 such properties and I am interleaving the async calls to load them in parallel. Here the MasterAssetData.LoadAsync factory takes the JSON file which is an array and processes it as a dict:
public static class MasterAssetData
{
    public static async ValueTask<MasterAssetData<TKey, TItem>> LoadAsync<TKey, TItem>(
        string path,
        Func<TItem, TKey> keySelector
    )
        where TItem : class
        where TKey : notnull
    {
        await using FileStream fs = File.OpenRead(path);

        List<TItem> items =
            await JsonSerializer.DeserializeAsync<List<TItem>>(fs);

        FrozenDictionary<TKey, TItem> frozenDict = items
            .ToDictionary(keySelector, x => x)
            .ToFrozenDictionary();

        return new MasterAssetData<TKey, TItem>(frozenDict);
    }
}

I wanted to write a console app to run at build time, to convert all the json files to MessagePack with LZ4 compression. I initially used Roslyn to attempt to parse the contents of the MasterAsset class as a standalone cs file (so no symbol info) to build an association between json filepath -> type name, e.g. CharaData.json -> CharaData, so that I could first load up the JSON to deserialize it before re-serializing as binary, but I ran into a number of stumbling blocks trying to go from that type name to an actual Type loaded via reflection, in the presence of things like generic types e.g. MasterAssetData<int, EventItem<BuildEventItem>>.

I think it would be easier if I rethought this approach and wrote a source generator so that I had

[GenerateMasterAsset<Charas, CharaData>("CharaData.json", nameof(CharaData.Id))]
public static partial class MasterAsset
{
}


which could then generate a property and LoadAsync method for you, like in my second code block. This would then allow me in my console app to get the json path via reflection of the metadata of MasterAsset rather than trying to parse the syntax tree for the function call, and it would save some typing when new entries are to be added. Downside is it feels a bit overkill to write an SG for just one class. I'd appreciate any thoughts, maybe the whole approach is horrific or maybe I'm on to something 🙂
Was this page helpful?