Conversation
Add server-side connection rotation for the Triple protocol, aligned with the gRPC standard capability (see gRFC A9). Long-lived HTTP/2 connections pin clients to the same provider instance, preventing load balancers from redistributing traffic after scale-up or deployment. - TripleConfig: add maxConnectionAge (ms, -1 = disabled by default) and maxConnectionAgeGrace (ms, default 10s) with validation, plus the matching TripleBuilder methods. - TripleServerConnectionHandler: on channelActive schedule a one-shot task when maxConnectionAge > 0, with +/-10% jitter (same as grpc-java) to avoid mass simultaneous reconnections. On expiry send an advisory GOAWAY (NO_ERROR, last-stream-id = MAX_INT) so in-flight requests keep running while clients migrate to a new connection; after the grace period initiate the existing graceful shutdown sequence (final GOAWAY + PING + close). Tasks are cancelled on channelInactive. - TripleHttp2Protocol: pass the config into the handler on both the h2 direct and h1-upgrade server pipeline paths; raise the codec gracefulShutdownTimeoutMillis so it never fires before the age grace period. Backward compatible: the feature is disabled by default and no existing behavior changes unless maxConnectionAge is configured. Note: for zero request failures during rotation this pairs with the consumer-side graceful GOAWAY migration in apache#16345. Signed-off-by: Jamie Wilson <a2v1h4.76g1nr7j@gmail.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 3.3 #16454 +/- ##
============================================
+ Coverage 60.91% 60.94% +0.02%
- Complexity 15 11768 +11753
============================================
Files 1953 1953
Lines 89271 89331 +60
Branches 13473 13479 +6
============================================
+ Hits 54383 54440 +57
+ Misses 29309 29306 -3
- Partials 5579 5585 +6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- TripleServerConnectionHandlerGuardTest: cover the race-guard branches (channel dies between scheduling and task execution) via mocked context, unreachable through EmbeddedChannel lifecycle alone since channelInactive cancels the tasks first. - TripleHttp2ProtocolPipelineTest: verify the server pipeline wires maxConnectionAge/maxConnectionAgeGrace from TripleConfig into TripleServerConnectionHandler (both enabled and disabled defaults). - TripleConfigTest: cover the null-setter branch keeping defaults. - Enable debug logging for TripleServerConnectionHandler in log4j2-test.xml so the grace-period debug branch is exercised. Signed-off-by: Jamie Wilson <a2v1h4.76g1nr7j@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Add server-side
max-connection-agesupport for the Triple protocol, aligned with the standard gRPC capability (gRFC A9: Server-Side Connection Management).HTTP/2 multiplexes all requests over a single long-lived connection, so a Triple consumer stays pinned to the same provider instance forever — even after new instances are scaled up or traffic should be rebalanced. gRPC solves this with
maxConnectionAge/maxConnectionAgeGrace: when a connection exceeds the configured age, the server sends an advisory GOAWAY (NO_ERROR,last-stream-id = MAX_INT) so in-flight requests keep running while clients migrate to a new connection (and get redistributed by the load balancer), then closes the connection after the grace period.Triple currently has no equivalent: there is no way for a Triple server to proactively rotate long-lived connections. This PR adds it, disabled by default.
Brief changelog
TripleConfig: addmaxConnectionAge(ms, default-1= disabled) andmaxConnectionAgeGrace(ms, default10000) with validation, following the existing field conventions (getXxxOrDefault()); matchingTripleBuildermethods in dubbo-config-api.TripleServerConnectionHandler: onchannelActive, ifmaxConnectionAge > 0, schedule a one-shot task on the channel's EventLoop with ±10% jitter (same as grpc-java) to avoid mass simultaneous reconnections when many connections are established at the same time. On expiry:GracefulShutdown.sendGoAwayFrame(reuses the production code path — in-flight streams are unaffected);ctx.channel().close()so the request traverses this handler'sclose()override and runs the existing graceful-shutdown sequence (final GOAWAY + PING) instead of an abrupt disconnect;channelInactive(no leaks when the client disconnects first).TripleHttp2Protocol: pass the config into the handler on both server pipeline paths (direct h2 and h1-upgrade); raise the codec'sgracefulShutdownTimeoutMillisto at least the configured grace so Netty's built-in backstop never fires before it.Usage:
Verifying this change
New tests (all passing):
TripleServerConnectionHandlerTest— EmbeddedChannel with virtual clock: advisory GOAWAY (NO_ERROR+extraStreamIds=MAX_INT) fires within the jitter window and never before it; disabled by default (24h, no frames); tasks cancelled onchannelInactive; graceful-shutdown sequence (final GOAWAY + PING) initiated after the grace period.TripleServerConnectionHandlerConcurrencyTest— real NIO server/client, 100 connections sharing 2 EventLoop threads: concurrent rotation of all connections; client disconnects racing age expiry (random delays within ±20% of the window); server-sideclose()racing pending age tasks; 5 rounds of connection churn underResourceLeakDetector.PARANOIDwith zero leak reports.TripleConfigTest/TripleBuilderTest— defaults, validation and builder wiring.Regression: full test suites of
dubbo-rpc-triple(492),dubbo-common(1169) anddubbo-config-api(695) pass locally.Note for reviewers: with the current consumer-side GOAWAY handling (#16344), each rotation costs one request failure window on the consumer, so this feature is best paired with the graceful consumer migration in #16345. This PR is independent and safe to merge on its own (default: disabled).
Does this pull request potentially affect one of the following parts
TripleConfigfields; default values keep behavior unchanged)Documentation
TripleConfigfields (website docs can follow after merge)