[API Proposal]: ZLibPipeReader

[wegylexy]

Background and motivation

Existing ZLibStream often consumes more bytes than it needs and discards the residue. A hack into its private fields is required to scavenge the unprocessed bytes.

See #61405

Proposed ZLibPipeReader only advances the underlying pipe reader to the end of the zlib stream.

API Proposal

namespace System.IO.Compression
{
    public class ZLibPipeReader : System.IO.Pipelines.PipeReader
    {
        public static ZLibPipeReader Create(System.IO.Pipelines.PipeReader basePipeReader);

        private ZLibPipeReader(System.IO.Pipelines.PipeReader basePipeReader);
    }
}

API Usage

var baseReader = PipeReader.Create(myGitPackStream, new(bufferSize: 16384));
var entries = await MyGitPackGetEntryCount(baseReader);
while (entries-- > 0)
{
    var (type, uncompressedSize) = await MyGitPackGetHeaderAsync(baseReader, cancellationToken);
    ZLibPipeReader inflateReader = new(baseReader);
    var result = inflateReader.ReadAtLeastAsync(uncompressedSize, cancellationToken);
}

Alternative Designs

namespace System.IO.Compression
{
    public class ZLibStream : Stream
    {
+        public ReadOnlyMemory<byte> Residue { get; }
    }
}

namespace System.IO
{
+    public class StackStream : Stream
+    {
+        public StackStream(Stream baseStream, bool leaveOpen = false);
+        public void Push(ReadOnlyMemory<byte> prolog);
+    }
}

MyStackStream stackStream = new(myGitPackStream);
var entries = await MyGitPackGetEntryCount(stackStream);
while (entries-- > 0)
{
    var (type, uncompressedSize) = await MyGitPackGetHeaderAsync(stackStream, cancellationToken);
    var buffer = GC.AllocateUninitializedArray<byte>((int)uncompressedSize).AsMemory();
    using ZLibStream inflateStream = new(stackStream, CompressionMode.Decompress, true);
    for (var read = 0; read < uncompressedSize; read += await inflateStream.ReadAsync(buffer[read..], cancellationToken) { }
    stackStream.Push(inflateStream.Residue);
}

Risks

No response

[stephentoub]

We don't have any plans to duplicate this functionality as pipereaders/writers.

We are open to the idea of creating lower-level encoder/decoder APIs, as was done with BrotliEncoder/Decoder, which can then be used by a reader/writer on top, just as BrotliStream is just a thin wrapper around those.
e.g. #39327

[wegylexy]

Ya, ZLipEncoder/Decoder will do.

[adityamandaleeka]

Looks like this has been answered. Please reopen if needed.