CryptoHives.Foundation.Memory Package

Buffer management utilities for .NET, built on ArrayPool<T> and the modern .NET memory APIs to keep allocations and GC pressure out of high-throughput code.

Overview

Renting from ArrayPool<T>.Shared instead of allocating avoids resize-copy churn and keeps large buffers off the Large Object Heap, which matters once throughput is high enough that allocations start showing up in GC pauses. ReadOnlySequence<T> then lets a producer hand that pooled data to a consumer without copying it at all. This package packages those patterns as drop-in MemoryStream and IBufferWriter<T> types, with explicit ownership contracts for the pooled memory and an opt-in clearArray flag on every type that owns it, for callers holding key material. It complements the modern .NET memory APIs rather than replacing them.

Target frameworks: net462, netstandard2.0, netstandard2.1, net8.0, net10.0. No dependency on any other CryptoHives package.

Installation

dotnet add package CryptoHives.Foundation.Memory

Namespace

using CryptoHives.Foundation.Memory.Buffers;
using CryptoHives.Foundation.Memory.Pools;

Classes

Buffer Management

Class	Description	Documentation
ArrayPoolMemoryStream	Memory stream using pooled buffers	Details
ArrayPoolBufferWriter<T>	IBufferWriter implementation with pooled chunks	Details
ReadOnlySequenceMemoryStream	Stream wrapper for ReadOnlySequence	Details

Segment Array Ownership

Class	Description	Documentation
ISegmentOwner<T>	Ownership interface for an `ArraySegment<T>`	Details
PooledSegment<T>	Rents from `ArrayPool<T>.Shared`; returns on dispose	Details
AllocatedSegment<T>	Wraps a GC-managed `T[]`; no pool return	Details
EmptySegment<T>	Zero-allocation null-object sentinel	Details

Sequence Ownership

Class	Description	Documentation
ISequenceOwner<T>	Ownership interface for a `ReadOnlySequence<T>`	Details
SequenceLease<T>	Zero-allocation payload handle carrying its producer	Details
SegmentSequence<T>	Adapts any `ISegmentOwner<T>` into a one-node sequence	Details
EmptySequence<T>	Zero-allocation null-object sentinel	Details

Object Pool Utilities

Class	Description	Documentation
ObjectOwner<T>	RAII wrapper for pooled objects	Details
ObjectPools	Ready-made rent helpers for common types	Details
PoolFactory	Builds pools, including for types this package does not reference	Details
ArrayPoolBufferWriterProvider<T>	Immutable writer settings that rent from a shared pool	Details

Internal Support Classes

Class	Description
ArrayPoolBufferSegment<T>	Internal buffer segment for ReadOnlySequence
ArrayPoolBufferSequence<T>	Internal `IDisposable` over a chain of pooled segments. Gated on `EXPERIMENTAL`, so it is not in shipped packages; it has never had a caller, and `SequenceLease<T>` fills the role.

Quick Examples

ArrayPoolMemoryStream

using var stream = new ArrayPoolMemoryStream();

// Write data
await stream.WriteAsync(data, cancellationToken);

// Get a zero-copy ReadOnlySequence
ReadOnlySequence<byte> sequence = stream.GetReadOnlySequence();

// Process without copying
ProcessSequence(sequence);

// The sequence's memory is returned to the pool once the stream is disposed

ArrayPoolBufferWriter

using var writer = new ArrayPoolBufferWriter<byte>();

// Get a span and write into it
Span<byte> span = writer.GetSpan(1024);
int written = encoder.GetBytes(text, span);
writer.Advance(written);

// Get the complete sequence
ReadOnlySequence<byte> result = writer.GetReadOnlySequence();

// Pooled chunks are returned once the writer is disposed

Handing a payload on, past the scope that built it

static SequenceLease<byte> BuildPayload()
{
    var writer = ObjectPools.RentBufferWriter<byte>();   // deliberately not `using`
    Serialize(writer);
    return writer.LeaseSequence();                       // 0 bytes; the writer rides along
}

using SequenceLease<byte> payload = BuildPayload();
var reader = new Utf8JsonReader(payload.Sequence);       // read in place, no copy
// disposing the lease returns the writer to its pool, and its buffers to ArrayPool

Pooled buffer writers

// One profile per use case; every profile draws from the same pool
static readonly ArrayPoolBufferWriterProvider<byte> JsonWriters = new(maxChunkBytes: 1 << 20);

using var writer = JsonWriters.Rent();
// ... write ...
// Disposing returns the writer itself to the pool, not just its buffers

ObjectOwner

// Pool your own type with a factory and a reset delegate
ObjectPool<MyClass> pool = PoolFactory.CreatePool(
    create: () => new MyClass(),
    reset: obj => { obj.Clear(); return true; });

using var owner = new ObjectOwner<MyClass>(pool);
MyClass obj = owner.PooledObject;

// Use obj...

// Automatically returned to the pool when owner is disposed

Segment Ownership

using CryptoHives.Foundation.Memory.Buffers;

// Pool-backed buffer — returned to ArrayPool on dispose
using ISegmentOwner<byte> pooled = PooledSegment<byte>.Rent(256);
Span<byte> span = pooled.Segment.AsSpan();
// fill span ...

// Wrap an existing array with no pool lifecycle
byte[] existing = new byte[256];
using ISegmentOwner<byte> alloc = AllocatedSegment<byte>.Create(existing);

// Empty sentinel — avoids null checks
ISegmentOwner<byte> none = EmptySegment<byte>.Instance;
if (none.Segment.Count == 0) { /* nothing to process */ }

Why Pooled Buffers

Renting from ArrayPool<T>.Shared instead of allocating avoids resize-copy churn and keeps large buffers off the Large Object Heap, which matters once you're pushing enough throughput that allocations start showing up in GC pauses. ReadOnlySequence<T> then lets you hand that pooled data to a consumer without copying it at all.

ArrayPoolMemoryStream: O(1) segment append, no copy-on-grow
ArrayPoolBufferWriter: exponential chunk growth with configurable limits
ReadOnlySequenceMemoryStream: zero-copy wrapper with O(n) seeking

Clearing Sensitive Buffers

A buffer arrives from ArrayPool<T> holding whatever the previous tenant left in it and, by default, goes back the same way. Any type here that owns pooled memory takes a clearArray flag that zeroes each buffer on its way back, so the next renter cannot read what you wrote:

using var stream  = new ArrayPoolMemoryStream(clearArray: true);
using var segment = PooledSegment<byte>.Rent(256, clearArray: true);

// For writers the flag belongs to the profile, not the call site
static readonly ArrayPoolBufferWriterProvider<byte> Secrets = new(clearArray: true);
using var writer = Secrets.Rent();

Type	How to ask
ArrayPoolMemoryStream	`clearArray` on any owning constructor
PooledSegment<T>	`Rent(minimumLength, clearArray)`
ArrayPoolBufferWriter<T>	`clearArray` on the constructor, or on the provider

It is opt-in everywhere. Zeroing costs a pass over each buffer, and most callers carry nothing worth hiding. Types that do not own their buffers — the read-only ArrayPoolMemoryStream constructor, AllocatedSegment<T> — have no such flag, because they never return anything to a pool.

For a pooled writer the flag is applied when the writer is rented and read when it is released, so the outgoing renter's setting is always the one in force. That ordering is what makes the zeroing trustworthy, and it is why the settings are not publicly writable.

Pools and Memory Pressure

The two layers behave differently, which is worth knowing before you size anything:

Rented arrays are released automatically. ArrayPool<T>.Shared registers a gen-2 GC callback and trims itself according to memory pressure.
Pooled objects are not. DefaultObjectPool<T> never trims — it drops an instance only when a policy rejects it or the pool is already full, and otherwise holds what it has for the life of the process. Bound it with maximumRetained if that matters. See PoolFactory.

A Few Things to Watch For

Always wrap streams and writers in a using so pooled buffers actually get returned.
A ReadOnlySequence<byte> from GetReadOnlySequence() is borrowed — valid only until the next write or dispose. When the payload has to leave that scope, use LeaseSequence() — it costs nothing and carries the producer along. See SequenceLease<T>.
ArrayPoolMemoryStream cannot hand out a single contiguous array, so GetBuffer() throws and TryGetBuffer() returns false. Reach for the sequence instead.
If you know roughly how much you'll write, pass a size hint; it cuts down on reallocations.
Keep writer/stream lifetimes short and scoped to the operation that needs them.
Carrying secrets? Pass clearArray: true, or the buffer goes back to the pool exactly as you wrote it.
Never touch a pooled writer after disposing it. A returned instance does not throw ObjectDisposedException; it silently becomes whatever the next renter is doing — the same contract as ArrayPool<T> itself.

Table of Contents