core: Fix inFlightSubStreams counting on retry commit #12649
Merged
+6
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Under certain retry and deadline timings, RetriableStream could leave the response future incomplete because inFlightSubStreams was not decremented consistently during commit. The change ensures inFlightSubStreams is decremented whenever a scheduled retry is committed, restoring correct close signaling without altering retry semantics.
becomeStar
commented
Feb 7, 2026
| final Future<?> retryFuture; | ||
| final boolean retryWasScheduled = scheduledRetry != null; | ||
| if (scheduledRetry != null) { | ||
| retryFuture = scheduledRetry.markCancelled(); |
Contributor
Author
There was a problem hiding this comment.
markCancelled() may return null if the scheduled retry was cancelled before setFuture() was called.
Contributor
There was a problem hiding this comment.
Yes, when the hang occurs I see markCancelled() had returned null causing the code in CommitTask to skip closing the master listener.
Contributor
|
/gcbrun |
kannanjgithub
approved these changes
Feb 9, 2026
kannanjgithub
added
the
kokoro:run
grpc-kokoro
removed
the
kokoro:run
kannanjgithub
added
kokoro:run
jdcormie
pushed a commit
to jdcormie/grpc-java
that referenced
this pull request
Feb 11, 2026
This PR fixes a race condition in RetriableStream where, under certain retry and deadline timings, the response future may never be completed. When a deadline cancellation occurs concurrently with retry commit, inFlightSubStreams may not be decremented, causing `ClientCallImpl.ClientStreamListenerImpl.closed` to never be invoked. As a result, blockingUnaryCall can hang indefinitely. After this change, the inFlightSubStreams counting is consistent whenever a scheduled retry is committed, ensuring the close signal is always delivered. I verified this using the issue reproduction code from the issue reporter, which previously caused blockingUnaryCall to hang and eventually hit a TimeoutException because a while loop never progressed. After this change, running the same reproduction code no longer hangs and continues as expected without timing out. Fixes grpc#12620
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes a race condition in RetriableStream where, under certain retry and deadline timings, the response future may never be completed.
When a deadline cancellation occurs concurrently with retry commit, inFlightSubStreams may not be decremented, causing
ClientCallImpl.ClientStreamListenerImpl.closedto never be invoked. As a result, blockingUnaryCall can hang indefinitely.After this change, the inFlightSubStreams counting is consistent whenever a scheduled retry is committed, ensuring the close signal is always delivered.
I verified this using the issue reproduction code from the issue reporter, which previously caused blockingUnaryCall to hang and eventually hit a TimeoutException because a while loop never progressed. After this change, running the same reproduction code no longer hangs and continues as expected without timing out.
Fixes #12620