C#•2y ago

Read stream twice without missing data

I want to grab the first byte from a file and then read the entire file (max. 128 bytes) starting from the beginning, including that first byte. However, the problem is that the first byte I initially read is missing. Here's the code:

c#
var bytesRead = await file.ReadAsync(new byte[1]);

// Check bytesRead
// ...

file.Seek(0, SeekOrigin.Begin);

byte[] buffer = new byte[128];
await file.ReadAsync(buffer, 0, buffer.Length);

c#
var bytesRead = await file.ReadAsync(new byte[1]);

// Check bytesRead
// ...

file.Seek(0, SeekOrigin.Begin);

byte[] buffer = new byte[128];
await file.ReadAsync(buffer, 0, buffer.Length);

Any ideas on how to fix this?

canton7•1/10/24, 11:41 AM

Note that

bytesRead

bytesRead

is the number of bytes which were read, not the value of the first byte

Ccanton7 Note that `bytesRead` is the number of bytes which were read, not the value of t...

LumeOP•1/10/24, 11:41 AM

Yes, I know.

LumeOP•1/10/24, 11:41 AM

That's intended

canton7•1/10/24, 11:42 AM

The first byte shouldn't be missing in your subsequent read though. I've done similar things plenty of times

Ccanton7 The first byte shouldn't be missing in your subsequent read though. I've done si...

LumeOP•1/10/24, 11:43 AM

It does. Another weird case is that sometimes the second buffer only contains the first byte and the rest of the data is lost.

canton7•1/10/24, 11:43 AM

Note that

Read

Read

reads at most the specified number of bytes. It's expected that it can read less

canton7•1/10/24, 11:44 AM

(but if it reads 0, that means you've reached the end)

canton7•1/10/24, 11:44 AM

There's

ReadExactly

ReadExactly

if you want to read exactly a specified number of bytes

LumeOP•1/10/24, 11:44 AM

That's the output with the code from above.

canton7•1/10/24, 11:45 AM

And what values does the second

file.ReadAsync

file.ReadAsync

return?

LumeOP•1/10/24, 11:45 AM

This is the second one

canton7•1/10/24, 11:45 AM

What value does it return?

canton7•1/10/24, 11:45 AM

I.e. the number of bytes read

canton7•1/10/24, 11:45 AM

int bytesRead = await file.ReadAsync(buffer, 0, buffer.Length);

int bytesRead = await file.ReadAsync(buffer, 0, buffer.Length);

Ccanton7 What value does it return?

LumeOP•1/10/24, 11:46 AM

canton7•1/10/24, 11:46 AM

Right, so all as expected

canton7•1/10/24, 11:46 AM

Call

ReadExactly

ReadExactly

, or call

Read

Read

multiple times until either it returns 0, or you've got the total number of bytes you want

canton7•1/10/24, 11:48 AM

(afk, back in half an hour)

Ccanton7 Right, so all as expected

LumeOP•1/10/24, 11:48 AM

Not really

. If I remove the

file.Seek()

file.Seek()

call. I get everything I want except the first byte. Like so:

Bailey•1/10/24, 12:00 PM

what about just setting the position file.position

Petris•1/10/24, 12:02 PM

Does the location you're reading from support seeking?

PPetris Does the location you're reading from support seeking?

LumeOP•1/10/24, 12:13 PM

Yes

BBailey what about just setting the position file.position

LumeOP•1/10/24, 12:13 PM

The behaviour is the same.

PPetris Does the location you're reading from support seeking?

LumeOP•1/10/24, 12:14 PM

LLume Not really 😅 . If I remove the `file.Seek()` call. I get everything I want exce...

canton7•1/10/24, 12:18 PM

No, Stream is document to work this way. There's no guarantee that it returns exactly the number of bytes you asked for

Petris•1/10/24, 12:18 PM

Steam?

Ccanton7 No, Stream is document to work this way. There's no guarantee that it returns ex...

Petris•1/10/24, 12:18 PM

Well is is if you use ReadExactly

canton7•1/10/24, 12:19 PM

Please, fix this bug. If you still see an issue, then let's investigate. But there's no point digging into one thing when we know you're doing something wrong

PPetris Well is is if you use ReadExactly

MODiX•1/10/24, 12:19 PM

canton7

There's

ReadExactly

ReadExactly

if you want to read exactly a specified number of bytes

Quoted by

@660066004059029524 from #Read stream twice without missing data (click here)

React with ❌ to remove this embed.

•

1/10/24, 11:44 AM

canton7•1/10/24, 12:20 PM

I've told him to use ReadExactly twice now, explicitly

canton7•1/10/24, 12:21 PM

I think it's entirely expected the first 1-byte read will read 1 byte into an internal cache, and then when you rewind / re-read it will first read the entire cache (all 1 byte of it), then the next read will actually read new data from file

canton7•1/10/24, 12:21 PM

(I'd have to re-read the code again to be sure)

Ccanton7 I've told him to use ReadExactly twice now, explicitly

LumeOP•1/10/24, 12:21 PM

But I don't understand the why. And I also don't have access to ReadExactly, I am on .NET 6.

canton7•1/10/24, 12:22 PM

Why is because that's how Stream.Read is documented to work. And various Stream implementations take advantage of this for various efficiency reasons. See for example my previous message

canton7•1/10/24, 12:23 PM

If you don't have

ReadExactly

ReadExactly

, then you can do something like:

int totalRead = 0;
while (totalRead < buffer.Length)
{
    int read = await stream.Read(buffer, totalRead, buffer.Length - totalRead);
    if (read == 0)
    {
        // End of stream
        break;
    }
    totalRead += read;
}

int totalRead = 0;
while (totalRead < buffer.Length)
{
    int read = await stream.Read(buffer, totalRead, buffer.Length - totalRead);
    if (read == 0)
    {
        // End of stream
        break;
    }
    totalRead += read;
}

canton7•1/10/24, 12:24 PM

I'll remember how to do it eventually

Ccanton7 If you don't have `ReadExactly`, then you can do something like: ```cs int total...

LumeOP•1/10/24, 12:25 PM

Thank you.

canton7•1/10/24, 12:31 PM

https://source.dot.net/#System.Private.CoreLib/src/libraries/System.Private.CoreLib/src/System/IO/Strategies/BufferedFileStreamStrategy.cs,214

Ccanton7 *Why* is because that's how Stream.Read is documented to work. And various Strea...

LumeOP•1/10/24, 12:39 PM

I believe there might be a misunderstanding between us. My intention is not to remove the trailing zeros from the byte array; that is perfectly acceptable. Instead, my goal is to read one byte, perform certain checks on it, and then read from the start of the original stream until it reaches a maximum of 128 bytes.

Example:

Stream: 1, 2, 3, 4, 5, 6, 7, 8, 9
var tmpBuffer = new byte[1];
Stream.Read(tmpBuffer) // tmpBuffer: 1

var buffer = new byte[128];
Stream.Read(buffer, 0, buffer.Length) // buffer: 2, 3, 4, 5, 6, 7, 8, 9 , 0, 0 , 0, ...

This is expected because the "cursor" moves one position to the right side (right?).

canton7•1/10/24, 12:40 PM

Stream.Read(buffer, 0, 1)

Stream.Read(buffer, 0, 1)

will only read 1 byte. I'm not sure why your

buffer

buffer

isn't

2, 0, 0, 0, ...

2, 0, 0, 0, ...

Ccanton7 `Stream.Read(buffer, 0, 1)` will only read 1 byte. I'm not sure why your `buffer...

LumeOP•1/10/24, 12:40 PM

Sorry I made a typo

LumeOP•1/10/24, 12:41 PM

Now I thought why not just reset the "cursor" using Stream.Position = 0 to the starting position. And then if I read it using Stream.Read. The output should be the following:
Stream.Read(buffer, 0, buffer.Length) // buffer: 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 0 , 0, ...

canton7•1/10/24, 12:41 PM

Note that the second

Stream.Read

Stream.Read

can return anywhere from 1 byte to 128 bytes. So your buffer might be

2, 0, 0, 0, ...

2, 0, 0, 0, ...

or it might be

2, 3, 0, 0, ...

2, 3, 0, 0, ...

or... You need to look at the return value to see how many bytes it actually read

Ccanton7 Note that the second `Stream.Read` can return anywhere from 1 byte to 128 bytes....

LumeOP•1/10/24, 12:44 PM

Yes, it depends on how many bytes I have in the byte array I i would like to read. The rest of the remaining bytes will be "filled up" with zeros. Correct?

canton7•1/10/24, 12:46 PM

No. It will read between 1 byte and the maximum number of bytes you requested. It will write to the section of the array that you tell it to (the second parameter). If it doesn't write to an array element, it just leaves it alone: it doesn't fill anything with zeros

canton7•1/10/24, 12:46 PM

From the docs:

Implementations of this method read a maximum of buffer.Length bytes from the current stream and store them in buffer. The current position within the stream is advanced by the number of bytes read; however, if an exception occurs, the current position within the stream remains unchanged. Implementations return the number of bytes read. If more than zero bytes are requested, the implementation will not complete the operation until at least one byte of data can be read (if zero bytes were requested, some implementations may similarly not complete until at least one byte is available, but no data will be consumed from the stream in such a case). Read returns 0 only if zero bytes were requested or when there is no more data in the stream and no more is expected (such as a closed socket or end of file). An implementation is free to return fewer bytes than requested even if the end of the stream has not been reached.

Ccanton7 No. It will read between 1 byte and the maximum number of bytes you requested. I...

LumeOP•1/10/24, 12:47 PM

I mean this line fills the whole array with zeros: byte[] buffer = new byte[128];

LumeOP•1/10/24, 12:47 PM

So yeah makes sense

Ccanton7 No. It will read between 1 byte and the maximum number of bytes you requested. I...

LumeOP•1/10/24, 12:47 PM

How can I reset the position back to the start once I read 1 byte?

canton7•1/10/24, 12:48 PM

stream.Position = 0

stream.Position = 0

, or

stream.Seek(0, SeekOrigin.Begin)

stream.Seek(0, SeekOrigin.Begin)

, as you have been doing