Fix create_partitions sending empty assignments array instead of null (#3146) - #3147
Closed
ebrard wants to merge 1 commit into
Closed
Fix create_partitions sending empty assignments array instead of null (#3146)#3147ebrard wants to merge 1 commit into
ebrard wants to merge 1 commit into
Conversation
_process_create_partitions_input always built the per-topic assignments field as a list. For the int total-count form (and a bare NewPartitions with no manual assignments) this produced an empty list [], which serialises to a present-but-empty array. The broker reads that as a manual replica assignment and rejects the request with InvalidReplicationAssignmentError. Pass assignments=None when the caller supplies no manual assignments so it serialises to null and the broker auto-assigns replicas. Add unit coverage for the auto-assign and manual-assignment paths. Fixes dpkp#3146
Owner
|
Thanks; I think the fix is much simpler than this -- we can just update the default to 'null'. However, the tests are much appreciated! |
Owner
|
Updated in #3148 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3146.
Problem
create_partitions({topic: N})with a plain int total count — the form the docstring recommends — fails against a normal broker withInvalidReplicationAssignmentError(error 39):create_partitions({topic: NewPartitions(N)})(auto-assign, no manual assignments) is also broken — it raisesTypeError: 'NoneType' object is not iterable, because the branch iteratescount.new_assignmentsunconditionally even though it defaults toNone.Root cause
_process_create_partitions_inputalways built the per-topicassignmentsfield as a list. For the int form (and a bareNewPartitions) that yields an empty list[], which serialises to a present-but-empty array rather thannull. The broker reads a present array as "manual replica assignment" and rejects it because the assignment count doesn't match the number of new partitions.Verified at the byte level (assignments field only, after
count=00000006):nullmeans "auto-assign replicas"; a present array means "manual assignment". The empty list is what triggers error 39.Fix
Pass
assignments=Nonewhen the caller supplies no manual assignments, so it serialises tonulland the broker auto-assigns. Manual assignments (dict form and the deprecatedNewPartitionsform) are preserved unchanged.Minor related change: the dict branch now uses
count.get('assignments')instead ofcount['assignments'], so{'count': N}with no assignments means auto-assign rather than raisingKeyError. This is backward compatible (any dict that previously worked still works).Tests
Added unit coverage in
test/admin/test_admin_topics.pyasserting_process_create_partitions_inputproducesassignments is Nonefor the int and bare-NewPartitionsinputs, and preserves explicit manual assignments. The auto-assign test fails on the current code and passes with this change.Verified:
test/admin/test_admin_topics.py— 20 passed (incl. the 2 new tests)test/protocol/admin/test_protocol_admin.py— 121 passedconfluentinc/cp-kafka:latestbroker:create_partitions({topic: N})(int) now increases the partition count instead of raising error 39, confirmed withrpk topic describe.