RecyclableMemoryStream

MemoryStream implementation that deals with pooling and managing memory streams which use potentially large buffers.

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

Assembly: ServiceStack.Text.dll

View Source

Declaration

public sealed class RecyclableMemoryStream : MemoryStream, IDisposable

Properties

Capacity

Gets or sets the capacity

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override int Capacity { get; set; }

Length

Gets the number of bytes written to this stream.

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override long Length { get; }

Position

Gets the current position in the stream

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override long Position { get; set; }

CanRead

Whether the stream can currently read

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override bool CanRead { get; }

CanSeek

Whether the stream can currently seek

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override bool CanSeek { get; }

CanTimeout

Always false

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override bool CanTimeout { get; }

CanWrite

Whether the stream can currently write

This class works in tandem with the RecyclableMemoryStreamManager to supply MemoryStream objects to callers, while avoiding these specific problems:

1. LOH allocations - since all large buffers are pooled, they will never incur a Gen2 GC
2. Memory waste - A standard memory stream doubles its size when it runs out of room. This leads to continual memory growth as each stream approaches the maximum allowed size.
3. Memory copying - Each time a MemoryStream grows, all the bytes are copied into new buffers. This implementation only copies the bytes when GetBuffer is called.
4. Memory fragmentation - By using homogeneous buffer sizes, it ensures that blocks of memory can be easily reused.

The stream is implemented on top of a series of uniformly-sized blocks. As the stream's length grows, additional blocks are retrieved from the memory manager. It is these blocks that are pooled, not the stream object itself.

The biggest wrinkle in this implementation is when GetBuffer() is called. This requires a single contiguous buffer. If only a single block is in use, then that block is returned. If multiple blocks are in use, we retrieve a larger buffer from the memory manager. These large buffers are also pooled, split by size--they are multiples/exponentials of a chunk size (1 MB by default).

Once a large buffer is assigned to the stream the small blocks are NEVER again used for this stream. All operations take place on the large buffer. The large buffer can be replaced by a larger buffer from the pool as needed. All blocks and large buffers are maintained in the stream until the stream is disposed (unless AggressiveBufferReturn is enabled in the stream manager).

View Source

Declaration

public override bool CanWrite { get; }

Methods

Finalize()

The finalizer will be called when a stream is not disposed properly.

Failing to dispose indicates a bug in the code using streams. Care should be taken to properly account for stream lifetime.

View Source

Declaration

protected void Finalize()

Dispose(Boolean)

Returns the memory used by this stream back to the pool.

View Source

Declaration

protected override void Dispose(bool disposing)

Parameters

Type	Name	Description
System.Boolean	disposing	Whether we're disposing (true), or being called by the finalizer (false)

|

Close()

Equivalent to Dispose

View Source

Declaration

public override void Close()

GetBuffer()

Returns a single buffer containing the contents of the stream. The buffer may be longer than the stream length.

IMPORTANT: Doing a Write() after calling GetBuffer() invalidates the buffer. The old buffer is held onto until Dispose is called, but the next time GetBuffer() is called, a new buffer from the pool will be required.

View Source

Declaration

public override byte[] GetBuffer()

Returns

System.Byte[]: A byte[] buffer IMPORTANT: Doing a Write() after calling GetBuffer() invalidates the buffer. The old buffer is held onto until Dispose is called, but the next time GetBuffer() is called, a new buffer from the pool will be required.

TryGetBuffer(out ArraySegment<Byte>)

Returns an ArraySegment that wraps a single buffer containing the contents of the stream.

GetBuffer has no failure modes (it always returns something, even if it's an empty buffer), therefore this method always returns a valid ArraySegment to the same buffer returned by GetBuffer.

View Source

Declaration

public override bool TryGetBuffer(out ArraySegment<byte> buffer)

Returns

System.Boolean: Always returns true. GetBuffer has no failure modes (it always returns something, even if it's an empty buffer), therefore this method always returns a valid ArraySegment to the same buffer returned by GetBuffer.

Parameters

Type	Name	Description
System.ArraySegment<System.Byte>	buffer	An ArraySegment containing a reference to the underlying bytes.

GetBuffer has no failure modes (it always returns something, even if it's an empty buffer), therefore this method always returns a valid ArraySegment to the same buffer returned by GetBuffer. |

ToArray()

Returns a new array with a copy of the buffer's contents. You should almost certainly be using GetBuffer combined with the Length to access the bytes in this stream. Calling ToArray will destroy the benefits of pooled buffers, but it is included for the sake of completeness.

View Source

Declaration

[Obsolete("This method has degraded performance vs. GetBuffer and should be avoided.")]
public override byte[] ToArray()

Returns

System.Byte[]

Read(Byte[], Int32, Int32)

Reads from the current position into the provided buffer

View Source

Declaration

public override int Read(byte[] buffer, int offset, int count)

Returns

System.Int32: The number of bytes read

Parameters

Type	Name	Description
System.Byte[]	buffer	Destination buffer

| | System.Int32 | offset | Offset into buffer at which to start placing the read bytes.

| | System.Int32 | count | Number of bytes to read.

|

SafeRead(Byte[], Int32, Int32, ref Int32)

Reads from the specified position into the provided buffer

View Source

Declaration

public int SafeRead(byte[] buffer, int offset, int count, ref int streamPosition)

Returns

System.Int32: The number of bytes read

Parameters

Type	Name	Description
System.Byte[]	buffer	Destination buffer

| | System.Int32 | offset | Offset into buffer at which to start placing the read bytes.

| | System.Int32 | count | Number of bytes to read.

| | System.Int32 | streamPosition | Position in the stream to start reading from

|

Write(Byte[], Int32, Int32)

Writes the buffer to the stream

View Source

Declaration

public override void Write(byte[] buffer, int offset, int count)

Parameters

Type	Name	Description
System.Byte[]	buffer	Source buffer

| | System.Int32 | offset | Start position

| | System.Int32 | count | Number of bytes to write

|

ToString()

Returns a useful string for debugging. This should not normally be called in actual production code.

View Source

Declaration

public override string ToString()

Returns

System.String

WriteByte(Byte)

Writes a single byte to the current position in the stream.

View Source

Declaration

public override void WriteByte(byte value)

Parameters

Type	Name	Description
System.Byte	value	byte value to write

|

ReadByte()

Reads a single byte from the current position in the stream.

View Source

Declaration

public override int ReadByte()

Returns

System.Int32: The byte at the current position, or -1 if the position is at the end of the stream.

SafeReadByte(ref Int32)

Reads a single byte from the specified position in the stream.

View Source

Declaration

public int SafeReadByte(ref int streamPosition)

Returns

System.Int32: The byte at the current position, or -1 if the position is at the end of the stream.

Parameters

Type	Name	Description
System.Int32	streamPosition	The position in the stream to read from

|

SetLength(Int64)

Sets the length of the stream

View Source

Declaration

public override void SetLength(long value)

Parameters

Type	Name
System.Int64	value

Seek(Int64, SeekOrigin)

Sets the position to the offset from the seek location

View Source

Declaration

public override long Seek(long offset, SeekOrigin loc)

Returns

System.Int64: The new position

Parameters

Type	Name	Description
System.Int64	offset	How many bytes to move

| | System.IO.SeekOrigin | loc | From where

|

WriteTo(Stream)

Synchronously writes this stream's bytes to the argument stream.

Important: This does a synchronous write, which may not be desired in some situations

View Source

Declaration

public override void WriteTo(Stream stream)

Parameters

Type	Name	Description
System.IO.Stream	stream	Destination stream

Important: This does a synchronous write, which may not be desired in some situations |

WriteTo(Stream, Int32, Int32)

Synchronously writes this stream's bytes, starting at offset, for count bytes, to the argument stream.

View Source

Declaration

public void WriteTo(Stream stream, int offset, int count)

Parameters

Type	Name	Description
System.IO.Stream	stream	Destination stream

| | System.Int32 | offset | Offset in source

| | System.Int32 | count | Number of bytes to write

|

Implements

• System.IDisposable