Read from memory stream at position

I am trying to read from a memory stream at the position its currently in. I have tried to use memoryStream.Read() but since I will not know how big the file is I can't use byte[] cipher = new byte[128] for example. It needs to be as big as what is left in the memory stream. I was trying to use memoryStream.ToArray() but that ignores the position the memory stream is at.