Application doesnt free up ram properly for some reason
im making a custom archive packager and i have a problem with it running out of ram in debug mode(when it reaches 1.4GB) it uses way too much ram and its even more with every file
it has 3 modes: misc, tex, sound
in misc it compresses with zstd at max compression level
in tex it goes through imageGrabber to give it a custom smaller header and decompresses the pixels and encodes them with zstd at max compression level
and in sound, it doesnt compress it as its already compressed good enough
and the only problem is with tex - it doesnt free the ram unlike misc does(and uses .5GB max in my case then after compression goes to 15MB, still a bit higher than it should be), you cant delete the last part file + dirs for some reason
the .sep is overall inspired by VALVe's vpk archive
https://github.com/wick3nd/SEP-packager/tree/main/src/SEP%20packager
i will be very thankful for help as im struggling with this problem for a week and i cant really get to image decompression before fixing this problem
GitHub
SEP-packager/src/SEP packager at main · wick3nd/SEP-packager
SteelEngine Package packager is a file compressing program for a in progress game engine - wick3nd/SEP-packager
6 Replies
15 MB does not sound too bad and could be fine
The GC does not free up every piece of memory as soon as it’s technically possible
Not being able to delete some files and directories sounds like an issue tho. That’s usually the case when there are file handles remaining. I assume that your app is the only one holding handles which means that the cleanup process is missing or insufficient (missing .Close() or explicit/implicit .Dispose call)
notes...
*
Decompression.cs should not write to the Console.Write()
* every field in Decompression is static, which means that you cannot reuse Decompression from multiple threads simultaneously
* same is true for Compression
* use $spectre for better console UI
* don't call Program.TUI() from Decompress()... reeks of a bad design, and could eventually overflow the stack
* in WriteArchiveData(), you are not using using FileStream and using BinaryWriter, which means those objects are held open for longer than they should be. that's why you're leaking memory and you can't touch the files. update Compression.cs:L70 and L71 to use using and you'll hav better results.Spectre.Console is a .NET library that allows for easy creation of console UIs, text formatting in the console, and command-line argument parsing.
https://spectreconsole.net/
Spectre.Console - Welcome!
Spectre.Console is a .NET library that makes it easier to create beautiful console applications.
Besides all the goodies posted earlier, learn how to use memory profiler so that you can walk through the objects in memory yourself. That's another useful way to tell what's up (even when you have no source code access).
i made the decompression.cs directly after program.cs so its poorly scripted(i should definitely redo it) and for the gui im redoing it in Terminal.GUI
that would be a very helpful thing to learn honestly but i dont kniw any good memory profilers
If you are using VS for Windows, then a memory profiler is built in https://devblogs.microsoft.com/visualstudio/choosing-a-net-memory-profiler-in-visual-studio-part-1/. If you, however, use other editors, you might want to use a search engine to study what options are there.