Fix infinite loop in DeflateStream/GZipStream BeginWrite with .NET Standard 2.0 derived classes#124143
Fix infinite loop in DeflateStream/GZipStream BeginWrite with .NET Standard 2.0 derived classes#124143
Conversation
- Added tests to reproduce infinite loop with derived streams - Fixed DeflateStream.BeginWrite to call WriteAsyncMemory directly - Fixed GZipStream.BeginWrite to call WriteAsyncMemory directly - Tests verify that derived streams with only byte[] WriteAsync override don't hang Co-authored-by: jeffhandley <1031940+jeffhandley@users.noreply.github.com>
Co-authored-by: jeffhandley <1031940+jeffhandley@users.noreply.github.com>
Co-authored-by: jeffhandley <1031940+jeffhandley@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes a compatibility hang where BeginWrite in DeflateStream / GZipStream could recurse indefinitely when used from derived classes compiled against older reference assemblies (notably .NET Standard 2.0), by avoiding WriteAsync virtual dispatch from BeginWrite.
Changes:
- Update
DeflateStream.BeginWriteto callWriteAsyncMemory(...)directly (instead ofWriteAsync(...)) to break the recursion cycle. - Update
GZipStream.BeginWriteto delegate to_deflateStream.WriteAsyncMemory(...)directly for the same reason. - Add regression tests intended to validate “does not hang” behavior for derived streams.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/DeflateStream.cs | Breaks the BeginWrite → WriteAsync recursion by calling internal async implementation directly. |
| src/libraries/System.IO.Compression/src/System/IO/Compression/GZipStream.cs | Mirrors the BeginWrite fix for GZipStream by calling into DeflateStream’s internal async write path. |
| src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.Deflate.cs | Adds a regression test for derived DeflateStream async write behavior. |
| src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.Gzip.cs | Adds a regression test for derived GZipStream async write behavior. |
| public async Task DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | ||
| { | ||
| // This test simulates a .NET Standard 2.0 derived GZipStream that only overrides | ||
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | ||
| // calling WriteAsync should not enter an infinite loop. | ||
| using (var ms = new MemoryStream()) | ||
| using (var compressor = new DerivedGZipStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | ||
| { | ||
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | ||
|
|
||
| // This should complete without hanging | ||
| var writeTask = compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask(); | ||
|
|
||
| // Set a timeout to detect infinite loop | ||
| var completedTask = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromSeconds(5))); | ||
|
|
||
| Assert.Same(writeTask, completedTask); | ||
| await writeTask; // Ensure no exceptions | ||
| Assert.True(compressor.WriteAsyncCalled); | ||
| } |
There was a problem hiding this comment.
Task.WhenAny(writeTask, Task.Delay(...)) detects a hang but doesn’t terminate writeTask. If the recursion bug regresses, the test may fail but leave a runaway operation that can affect subsequent tests. Prefer isolating this with RemoteExecutor and a short timeout, or another approach that guarantees termination on failure.
| public async Task DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | |
| { | |
| // This test simulates a .NET Standard 2.0 derived GZipStream that only overrides | |
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | |
| // calling WriteAsync should not enter an infinite loop. | |
| using (var ms = new MemoryStream()) | |
| using (var compressor = new DerivedGZipStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | |
| { | |
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | |
| // This should complete without hanging | |
| var writeTask = compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask(); | |
| // Set a timeout to detect infinite loop | |
| var completedTask = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromSeconds(5))); | |
| Assert.Same(writeTask, completedTask); | |
| await writeTask; // Ensure no exceptions | |
| Assert.True(compressor.WriteAsyncCalled); | |
| } | |
| public void DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | |
| { | |
| // This test simulates a .NET Standard 2.0 derived GZipStream that only overrides | |
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | |
| // calling WriteAsync should not enter an infinite loop. | |
| using RemoteInvokeHandle handle = RemoteExecutor.Invoke(() => | |
| { | |
| using (var ms = new MemoryStream()) | |
| using (var compressor = new DerivedGZipStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | |
| { | |
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | |
| // This should complete without hanging | |
| compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).GetAwaiter().GetResult(); | |
| Assert.True(compressor.WriteAsyncCalled); | |
| } | |
| }, | |
| new RemoteInvokeOptions | |
| { | |
| TimeOut = 5000 | |
| }); |
| // If a derived class (compiled against .NET Standard 2.0 where DeflateStream doesn't override | ||
| // WriteAsync(byte[], ...)) calls base.WriteAsync, it goes to Stream.WriteAsync, which calls | ||
| // BeginWrite. If BeginWrite then calls WriteAsync, it creates a loop. |
There was a problem hiding this comment.
The comment says “where DeflateStream doesn’t override WriteAsync(byte[], ...)”, but this type does override WriteAsync(byte[],...) in this assembly. To avoid confusion, please reword to clarify that the issue is with derived types compiled against older reference assemblies (e.g., netstandard2.0) where the base call binds to Stream.WriteAsync and thus routes through Begin/EndWrite.
| // If a derived class (compiled against .NET Standard 2.0 where DeflateStream doesn't override | |
| // WriteAsync(byte[], ...)) calls base.WriteAsync, it goes to Stream.WriteAsync, which calls | |
| // BeginWrite. If BeginWrite then calls WriteAsync, it creates a loop. | |
| // For derived types compiled against older reference assemblies (for example, .NET Standard 2.0) | |
| // where a base.WriteAsync(byte[], ...) call binds to Stream.WriteAsync, the call flows into | |
| // BeginWrite. If this BeginWrite implementation then called WriteAsync(byte[], ...), it would | |
| // re-enter that base implementation and create a loop. |
| // If a derived class (compiled against .NET Standard 2.0 where GZipStream doesn't override | ||
| // WriteAsync(byte[], ...)) calls base.WriteAsync, it goes to Stream.WriteAsync, which calls | ||
| // BeginWrite. If BeginWrite then calls WriteAsync, it creates a loop. |
There was a problem hiding this comment.
The comment says “where GZipStream doesn’t override WriteAsync(byte[], ...)”, but GZipStream does override WriteAsync(byte[],...) in this assembly. Please reword to make it explicit this is about code compiled against older reference assemblies (netstandard2.0) where the base call binds to Stream.WriteAsync and can re-enter BeginWrite.
| // If a derived class (compiled against .NET Standard 2.0 where GZipStream doesn't override | |
| // WriteAsync(byte[], ...)) calls base.WriteAsync, it goes to Stream.WriteAsync, which calls | |
| // BeginWrite. If BeginWrite then calls WriteAsync, it creates a loop. | |
| // If a derived class (compiled against older reference assemblies, e.g. netstandard2.0, | |
| // where GZipStream was not declared to override WriteAsync(byte[], ...)) calls base.WriteAsync, | |
| // the call binds to Stream.WriteAsync, which calls BeginWrite. If BeginWrite then calls WriteAsync, | |
| // it creates a loop. |
| [Fact] | ||
| public async Task DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | ||
| { | ||
| // This test simulates a .NET Standard 2.0 derived DeflateStream that only overrides | ||
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | ||
| // calling WriteAsync should not enter an infinite loop. | ||
| using (var ms = new MemoryStream()) | ||
| using (var compressor = new DerivedDeflateStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | ||
| { | ||
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | ||
|
|
||
| // This should complete without hanging | ||
| var writeTask = compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask(); | ||
|
|
||
| // Set a timeout to detect infinite loop | ||
| var completedTask = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromSeconds(5))); | ||
|
|
||
| Assert.Same(writeTask, completedTask); | ||
| await writeTask; // Ensure no exceptions | ||
| Assert.True(compressor.WriteAsyncCalled); | ||
| } |
There was a problem hiding this comment.
The new test doesn’t actually simulate the reported .NET Standard 2.0 scenario. DerivedDeflateStreamWithByteArrayWriteAsync.WriteAsync(byte[],...) calls base.WriteAsync(...), but in this build DeflateStream does override WriteAsync(byte[],...), so the base call binds to DeflateStream.WriteAsync rather than Stream.WriteAsync, and the BeginWrite/WriteAsync recursion can’t occur. Consider generating the derived type via Reflection.Emit (or a small netstandard2.0 test asset) so the override does a non-virtual call to Stream.WriteAsync(byte[],...), which will exercise BeginWrite on DeflateStream and reproduce the original hang.
| public async Task DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | ||
| { | ||
| // This test simulates a .NET Standard 2.0 derived DeflateStream that only overrides | ||
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | ||
| // calling WriteAsync should not enter an infinite loop. | ||
| using (var ms = new MemoryStream()) | ||
| using (var compressor = new DerivedDeflateStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | ||
| { | ||
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | ||
|
|
||
| // This should complete without hanging | ||
| var writeTask = compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask(); | ||
|
|
||
| // Set a timeout to detect infinite loop | ||
| var completedTask = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromSeconds(5))); | ||
|
|
||
| Assert.Same(writeTask, completedTask); | ||
| await writeTask; // Ensure no exceptions | ||
| Assert.True(compressor.WriteAsyncCalled); | ||
| } |
There was a problem hiding this comment.
Task.WhenAny(writeTask, Task.Delay(...)) will throw quickly on a hang, but it doesn’t stop writeTask. If the regression reappears, the runaway recursion may continue consuming thread pool/CPU and destabilize the rest of the test run. Prefer running the repro in RemoteExecutor with a short RemoteInvokeOptions.TimeOut, or otherwise structure the repro so the hung work is isolated/terminated.
| public async Task DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | |
| { | |
| // This test simulates a .NET Standard 2.0 derived DeflateStream that only overrides | |
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | |
| // calling WriteAsync should not enter an infinite loop. | |
| using (var ms = new MemoryStream()) | |
| using (var compressor = new DerivedDeflateStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | |
| { | |
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | |
| // This should complete without hanging | |
| var writeTask = compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask(); | |
| // Set a timeout to detect infinite loop | |
| var completedTask = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromSeconds(5))); | |
| Assert.Same(writeTask, completedTask); | |
| await writeTask; // Ensure no exceptions | |
| Assert.True(compressor.WriteAsyncCalled); | |
| } | |
| public void DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | |
| { | |
| var options = new RemoteInvokeOptions | |
| { | |
| TimeOut = 5000 | |
| }; | |
| RemoteExecutor.Invoke(static () => | |
| { | |
| // This test simulates a .NET Standard 2.0 derived DeflateStream that only overrides | |
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | |
| // calling WriteAsync should not enter an infinite loop. | |
| using (var ms = new MemoryStream()) | |
| using (var compressor = new DerivedDeflateStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | |
| { | |
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | |
| // This should complete without hanging; if it hangs, the RemoteExecutor timeout will terminate the process. | |
| compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask().GetAwaiter().GetResult(); | |
| Assert.True(compressor.WriteAsyncCalled); | |
| } | |
| return RemoteExecutor.SuccessExitCode; | |
| }, options).Dispose(); |
| [Fact] | ||
| public async Task DerivedStream_WithByteArrayWriteAsync_DoesNotHang() | ||
| { | ||
| // This test simulates a .NET Standard 2.0 derived GZipStream that only overrides | ||
| // WriteAsync(byte[], int, int, CancellationToken). When used in .NET Core 3.1+, | ||
| // calling WriteAsync should not enter an infinite loop. | ||
| using (var ms = new MemoryStream()) | ||
| using (var compressor = new DerivedGZipStreamWithByteArrayWriteAsync(ms, CompressionMode.Compress)) | ||
| { | ||
| byte[] data = new byte[] { 1, 2, 3, 4, 5 }; | ||
|
|
||
| // This should complete without hanging | ||
| var writeTask = compressor.WriteAsync(new ReadOnlyMemory<byte>(data)).AsTask(); | ||
|
|
||
| // Set a timeout to detect infinite loop | ||
| var completedTask = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromSeconds(5))); | ||
|
|
||
| Assert.Same(writeTask, completedTask); | ||
| await writeTask; // Ensure no exceptions | ||
| Assert.True(compressor.WriteAsyncCalled); | ||
| } |
There was a problem hiding this comment.
The new test doesn’t actually simulate the reported .NET Standard 2.0 scenario. DerivedGZipStreamWithByteArrayWriteAsync.WriteAsync(byte[],...) calls base.WriteAsync(...), but in this build GZipStream overrides WriteAsync(byte[],...), so the base call won’t bind to Stream.WriteAsync and won’t exercise the BeginWrite recursion this PR fixes. Consider using Reflection.Emit (or a netstandard2.0-compiled test asset) so the override can do a non-virtual call to Stream.WriteAsync(byte[],...) and reproduce the original hang.
|
@jeffhandley, this is fixing one issue at the expense of introducing another. If BeginWrite calls to the private non-overidable method, then a derived type that was overriding WriteAsync will no longer have its behaviors respected as part of BeginWrite calls, e.g. if WriteAsync was overridden to throttle or keep track of the number of calls or whatever, using BeginWrite will now bypass that logic. That's a fairly substantial breaking change. |
|
Also, if we did want to fix it this way, wouldn't the same issue exist for read? And do the tests actually protect against this? |
|
Thanks, @stephentoub. Closing for now as it was low-pri. Good catches on those points. |
Description
BeginWriteinDeflateStreamandGZipStreamcauses infinite recursion when invoked on derived classes compiled against .NET Standard 2.0 and used in .NET Core 3.1+.Root cause: In .NET Standard 2.0, these classes don't override
WriteAsync(byte[], int, int, CancellationToken). When a derived class callsbase.WriteAsync, it hitsStream.WriteAsyncwhich uses the Begin/End pattern, callingBeginWrite. The .NET Core 3.1+ implementation ofBeginWritethen callsWriteAsync, creating the loop:Fix: Changed
BeginWriteto callWriteAsyncMemorydirectly, bypassing theWriteAsyncindirection that creates the cycle.Changes
DeflateStream.cs:
GZipStream.cs: Same pattern, delegates to
_deflateStream.WriteAsyncMemory.Tests: Added regression tests simulating .NET Standard 2.0 derived streams with byte[]
WriteAsyncoverride. Tests verify completion within 5 seconds (previously hung indefinitely).Type of change
Original prompt
This section details on the original issue you should resolve
<issue_title>DeflateStream as a base class causes infinite loop in async scenario.</issue_title>
<issue_description>DeflateStream in .net standard 2.0 does not have the method WriteAsync(...). If a class derives from DeflateStream(.net standard 2.0) and used in .net core 3.1, then the WriteAsync(..) goes to infinite loop because:
-Since DeflateStream(.net standard 2.0) does not have WriteAsync(..), the Stream.WriteAsync is used.
-Stream.WriteAsync(..) internally calls BeginWrite(..)
-BeginWrite(..) method of DeflateStream(.net core 3.1) internally calls WriteAsync(..)
area-System.IO,untriagedWhen a class derives from
DeflateStream(compiled against .NET Standard 2.0, which lacks theWriteAsync(ReadOnlyMemory<byte>, CancellationToken)override) and is used on .NET Core 3.1+, callingWriteAsyncenters an infinite loop. The call chain is:Stream.WriteAsync→BeginWrite→DeflateStream.BeginWrite→WriteAsync→ back toStream.WriteAsync. Confirmed by @adamsitnik on .NET Core 3.1, 5.0, and 6.0.Expected Fix
Break the circular call chain in
DeflateStream.BeginWrite. WhenBeginWritedetects it's being called from the baseStream.WriteAsync(e.g., by checking if the concrete type overridesWriteAsyncwith the old byte[] signature but not the newReadOnlyMemory<byte>signature), it should call the underlying compression implementation directly instead of dispatching back throughWriteAsync. A similar pattern exists in other Stream subclasses that needed to handle the .NET Standard 2.0 → .NET Core migration.Test Coverage
DeflateStreamthat only overridesWriteAsync(byte[], int, int, CancellationToken).WriteAsync(ReadOnlyMemory<byte>, CancellationToken)and verify it completes without hanging.DeflateStreamandGZipStream(which usesDeflateStreaminternally).Repro