C#•2w ago

✅BinaryReader reads garbage when reading from compressed stream

This is a weird one - I think. So I have a rather complex data type that I write to and load from disk. I'm using a ZipArchive to get some data compression. Writing basically looks like this:

using var archive = new ZipArchive(someFileStream, ZipArchiveMode.Create, leaveOpen: true);
using var stream = archive.CreateEntry("myentry.txt", CompressionLevel.NoCompression).Open();
using var writer = new BinaryWriter(stream);
// serialize all data using writer

using var archive = new ZipArchive(someFileStream, ZipArchiveMode.Create, leaveOpen: true);
using var stream = archive.CreateEntry("myentry.txt", CompressionLevel.NoCompression).Open();
using var writer = new BinaryWriter(stream);
// serialize all data using writer

and reading like this:

using var archive = new ZipArchive(someFileStream, ZipArchiveMode.Read, leaveOpen: true);
using var stream = archive.GetEntry("myentry.txt").Open();
using var reader = new BinaryReader(stream);
// deserialize all data using reader

using var archive = new ZipArchive(someFileStream, ZipArchiveMode.Read, leaveOpen: true);
using var stream = archive.GetEntry("myentry.txt").Open();
using var reader = new BinaryReader(stream);
// deserialize all data using reader

The code above works fine as long as I have NoCompression specified. I can successfully write and then read the file back. However, as soon as I change the compression level to anything else (like Optimal), I get a random error in the middle of reading with reader where it tries to read the length of an array and it reads it as a billion or something (where it should be like 100). I've gone through it with the debugger and all data up to that point is read correctly. I also don't think this is a new code path it runs into the first time. The same code has been called multiple times already before running into the error. (more details in thread)

1 Reply

swagridOP•2w ago

I've also gone ahead and picked the same object and serialized it to disk twice. The first time I used NoCompression, the second time I used Optimal compression. Then I took both zip files and extracted them with 7zip and compared their content. They are both identical down to the byte. Another interesting part is if I buffer the stream on deserialization to a MemoryStream it suddenly works even with compression. If I change the reader code from above to

using var archive = new ZipArchive(someFileStream, ZipArchiveMode.Read, leaveOpen: true);
using var stream = archive.GetEntry("myentry.txt").Open();
using var mem = new MemoryStream();
stream.CopyTo(mem);
mem.Position = 0;
using var reader = new BinaryReader(mem);
// deserialize all data using reader

using var archive = new ZipArchive(someFileStream, ZipArchiveMode.Read, leaveOpen: true);
using var stream = archive.GetEntry("myentry.txt").Open();
using var mem = new MemoryStream();
stream.CopyTo(mem);
mem.Position = 0;
using var reader = new BinaryReader(mem);
// deserialize all data using reader

it works and I can load the data successfully. Any idea what's going on here? I'll try to create a minimal example, but it might take a while since not even all of my objects produce this error... Alright I found the issue and it was of course not a problem with either BinaryReader nor ZipArchive. At one point I read the data from the reader into a span and I didn't check that the amount read is actually the whole span. I changed

var result = new T[count];
if (count > 0)
{
    var span = MemoryMarshal.AsBytes(result.AsSpan());
    _ = reader.Read(span);
}

var result = new T[count];
if (count > 0)
{
    var span = MemoryMarshal.AsBytes(result.AsSpan());
    _ = reader.Read(span);
}

var result = new T[count];
if (count > 0)
{
    var span = MemoryMarshal.AsBytes(result.AsSpan());
    while (span.Length > 0)
    {
        var read = reader.Read(span);
        span = span[read..];
    }
}

var result = new T[count];
if (count > 0)
{
    var span = MemoryMarshal.AsBytes(result.AsSpan());
    while (span.Length > 0)
    {
        var read = reader.Read(span);
        span = span[read..];
    }
}

and now it works...

Gaming

Programming

✅BinaryReader reads garbage when reading from compressed stream

Did you find this page helpful?