Release Notes - Kafka - Version 4.1.0

Below is a summary of the JIRA issues addressed in the 4.1.0 release of Kafka. For full documentation of the release, a guide to get started, and information about the project, see the Kafka project site.

Note about upgrades: Please carefully review the upgrade documentation for this release thoroughly before upgrading your cluster. The upgrade notes discuss any critical information about incompatibilities and breaking changes, performance changes, and any other changes that might impact your production deployment of Kafka.

The documentation for the most recent release can be found at https://kafka.apache.org/documentation.html.

New Feature

  • [KAFKA-8830] - KIP-512: make Record Headers available in onAcknowledgement
  • [KAFKA-14121] - AlterPartitionReassignments API should allow callers to specify the option of preserving the replication factor
  • [KAFKA-16758] - Extend Consumer#close with option to leave the group or not
  • [KAFKA-17894] - Additional metrics for cooperative consumption
  • [KAFKA-18120] - KIP-891: Support for multiple versions of connect plugins.
  • [KAFKA-18182] - Add VersionRange to Plugins and DelegatingClassLoader APIs
  • [KAFKA-18215] - Accept plugin.version configurations for connector and converters
  • [KAFKA-18419] - Accept plugin.version configurations for transforms and predicates
  • [KAFKA-18666] - Controller-side monitoring for broker shutdown and startup
  • [KAFKA-18863] - Runtime additions for connector multiversioning.
  • [KAFKA-18988] - Updates to connector status and connector metrics
  • [KAFKA-19073] - Add Transactional ID pattern filter to ListTransactions API
  • [KAFKA-19223] - Explicit HWM replication

    • Improvement

  • [KAFKA-806] - Index may not always observe log.index.interval.bytes
  • [KAFKA-6629] - SegmentedCacheFunctionTest does not cover session window serdes
  • [KAFKA-10790] - Detect/Prevent Deadlock on Producer Network Thread
  • [KAFKA-10864] - Convert End Transaction Marker to use auto-generated protocal
  • [KAFKA-13810] - Document behavior of KafkaProducer.flush() w.r.t callbacks
  • [KAFKA-14402] - Transactions Server Side Defense
  • [KAFKA-15767] - Refactor TransactionManager to avoid use of ThreadLocal
  • [KAFKA-16260] - Deprecate window.size.ms and window.inner.serde.class in StreamsConfig
  • [KAFKA-16906] - Add consistent error handling for Transactions
  • [KAFKA-17091] - Add @FunctionalInterface to Streams interfaces
  • [KAFKA-17607] - Add CI step to verify LICENSE-binary
  • [KAFKA-17747] - Trigger rebalance on rack topology changes
  • [KAFKA-17937] - Use ValueSource to refactor AbstractFetcherThreadTest to eliminate unnecessary skip tests
  • [KAFKA-18023] - Enforcing Explicit Naming for Kafka Streams Internal Topics (KIP-1111)
  • [KAFKA-18168] - GlobalKTable does not checkpoint restored offsets until next 10K events
  • [KAFKA-18173] - Remove duplicate assertFutureError
  • [KAFKA-18275] - Restarting broker in testing should use the same port
  • [KAFKA-18276] - Migrate ProducerRebootstrapTest to new test infra
  • [KAFKA-18278] - the name and description of `run-gradle` is incorrect
  • [KAFKA-18300] - Improve coordinator records
  • [KAFKA-18329] - Delete old group coordinator (KIP-848)
  • [KAFKA-18332] - Fix KafkaRaftClient complexity
  • [KAFKA-18358] - Replace Deprecated $buildDir variable in build.gradle
  • [KAFKA-18408] - tweak the 'tag' field for BrokerHeartbeatRequest.json, BrokerRegistrationChangeRecord.json and RegisterBrokerRecord.json
  • [KAFKA-18409] - ShareGroupStateMessageFormatter should use ApiMessageFormatter
  • [KAFKA-18434] - enrich the authorization error message of connecting to controller
  • [KAFKA-18467] - Add create topic doc
  • [KAFKA-18479] - remove keepPartitionMetadataFile in UnifiedLog and LogManager
  • [KAFKA-18555] - Avoid casting MetadataCache to KRaftMetadataCache
  • [KAFKA-18570] - Update documemtation for log loading metrics during Kafka broker startup
  • [KAFKA-18573] - Add support for OAuth jwt-bearer grant type
  • [KAFKA-18576] - Convert ConfigType to Enum
  • [KAFKA-18617] - Support ClusterInstance in BeforeEach/AfterEach fixtures
  • [KAFKA-18639] - Add @Flaky to testAdminClientApisAuthenticationFailure, testOutdatedCoordinatorAssignment, and testThrottledProducerConsumer
  • [KAFKA-18658] - add import control for examples module
  • [KAFKA-18693] - Remove PasswordEncoder
  • [KAFKA-18695] - Remove quorum=kraft and kip932 from all integration tests
  • [KAFKA-18703] - Remove unused class PayloadKeyType
  • [KAFKA-18729] - Improve consumer check for assigned topics from regex
  • [KAFKA-18748] - Run new integration tests separately in PRs
  • [KAFKA-18751] - Complete delayed FETCH request when HWM is increased.
  • [KAFKA-18760] - Deprecate Optional and return String from public EndPoint#Listener
  • [KAFKA-18795] - Remove Records#downConvert
  • [KAFKA-18798] - The replica placement policy used by ReassignPartitionsCommand is not aligned with kraft controller
  • [KAFKA-18801] - Remove ClusterGenerator and revise docs of ClusterTemplate
  • [KAFKA-18804] - Remove slf4j warning when using tool script.
  • [KAFKA-18847] - Refactor OAuth layer to improve reusability
  • [KAFKA-18858] - Refactor FeatureControlManager to avoid using uninitialized MV
  • [KAFKA-18867] - add tests to describe topic configs with empty name
  • [KAFKA-18880] - Remove kafka.cluster.Broker and BrokerEndPointNotAvailableException
  • [KAFKA-18881] - Document the ConsumerRecord as non-thread safe
  • [KAFKA-18904] - Listing of configs for dynamically created resources is mysterious
  • [KAFKA-18907] - add suitable error message when the appended value is too larger
  • [KAFKA-18927] - Remove LATEST_0_11, LATEST_1_0, LATEST_1_1, LATEST_2_0
  • [KAFKA-18929] - Log a warning when a time retention based segment has a future timestamp
  • [KAFKA-18933] - Add integration tests for :clients outside of :core
  • [KAFKA-18935] - Ensure brokers do not return null records in FetchResponse
  • [KAFKA-18944] - Remove unused setters from ClusterConfig
  • [KAFKA-18945] - Enahnce the docs of Admin#describeCluster and Admin#describeConfigs for bootstrap-controller
  • [KAFKA-18947] - refactor MetadataShell
  • [KAFKA-18949] - fix testDeleteRecordsAfterCorruptRecords for consumer protocol
  • [KAFKA-18959] - increse the num_workers from 9 to 14
  • [KAFKA-18961] - Consider time-based refresh for server-side RE2J regex
  • [KAFKA-18969] - Rewrite ShareConsumerTest#setup by beforeEach and move ShareConsumerTest to clients-integration-tests module
  • [KAFKA-18980] - OffsetMetadataManager#cleanupExpiredOffsets should record the number of records rather than topic partitions
  • [KAFKA-18983] - Ensure all README.md(s) are mentioned by the root README.md
  • [KAFKA-18989] - Optimize FileRecord#searchForOffsetWithSize
  • [KAFKA-18990] - Optimize BaseQuotaTest#produceUntilThrottled to avoid redundant MetricName creation
  • [KAFKA-18999] - Remove BrokerMetadata
  • [KAFKA-19002] - Rewrite ListOffsetsIntegrationTest and move it to clients-integration-test
  • [KAFKA-19005] - improve the docuemtnation of DescribeTopicsOptions#partitionSizeLimitPerResponse
  • [KAFKA-19027] - Replace ConsumerGroupCommandTestUtils#generator by ClusterTestDefaults
  • [KAFKA-19030] - Remove metricNamePrefix from RequestChannel
  • [KAFKA-19049] - Remove the `@ExtendWith(ClusterTestExtensions.class)` from code base
  • [KAFKA-19053] - Remove FetchResponse#of which is not used in production anymore
  • [KAFKA-19055] - Cleanup the 0.10.x information from clients module
  • [KAFKA-19056] - Move EndToEndClusterIdTest to server module
  • [KAFKA-19060] - Documented null edge cases in the Clients API JavaDoc
  • [KAFKA-19064] - More exception handling for group coordinator events
  • [KAFKA-19068] - Eliminate the duplicate type check in creating ControlRecord
  • [KAFKA-19069] - DumpLogSegments does not dump the LEADER_CHANGE record
  • [KAFKA-19076] - UnifiedLog#maybeHandleIOException should replace `String` by `Supplier`
  • [KAFKA-19093] - Change the "Handler on Broker" to "Handler on Controller" for controller server
  • [KAFKA-19098] - Remove lastOffset from PartitionResponse
  • [KAFKA-19099] - Remove GroupSyncKey, GroupJoinKey, and MemberKey
  • [KAFKA-19101] - Remove ControllerMutationQuotaManager#throttleTimeMs unused parameter
  • [KAFKA-19102] - Enhance the docs of group.coordinator.append.linger.ms
  • [KAFKA-19103] - Remove OffsetConfig
  • [KAFKA-19121] - Move AddPartitionsToTxnConfig, TransactionStateManagerConfig, and QuorumConfig out of KafkaConfig
  • [KAFKA-19136] - Move metadata-related configs from KRaftConfigs to MetadataLogConfig
  • [KAFKA-19137] - Use `StandardCharsets.UTF_8` instead of `StandardCharsets.UTF_8.name()`
  • [KAFKA-19146] - Merge org.apache.kafka.server.common.OffsetAndEpoch and org.apache.kafka.raft.OffsetAndEpoch
  • [KAFKA-19151] - Enhance the docs of flush.ms by reminding users the flush is triggered by log.flush.scheduler.interval.ms
  • [KAFKA-19152] - Add top-level documentation for OAuth flows
  • [KAFKA-19170] - Migrate MetricsDuringTopicCreationDeletionTest to clients-integration-tests module
  • [KAFKA-19178] - Replace Vector by ArrayList for PluginClassLoader#getResources
  • [KAFKA-19180] - Fix the hanging testPendingTaskSize
  • [KAFKA-19209] - "index.interval.bytes" config doc should also mention time index
  • [KAFKA-19214] - Clean up use of Optionals in RequestManagers.entries()
  • [KAFKA-19248] - Plugins Test for Multiversioning in Kafka Connect
  • [KAFKA-19263] - The docs of delete.topic.enable used by Admin#deletetopics is out-of-date
  • [KAFKA-19267] - the min version used by ListOffsetsRequest should be 1 rather than 0
  • [KAFKA-19269] - "Unexpected error .." should not happen when the delete.topic.enable is false
  • [KAFKA-19273] - Ensure the delete policy is configured when the tiered storage is enabled
  • [KAFKA-19284] - add documentation to clarify the behavior of null values for all partitionsToOffsetAndMetadata methods.
  • [KAFKA-19287] - docuemnt all group coordinator metrics
  • [KAFKA-19295] - Remove AsyncKafkaConsumer event ID generation
  • [KAFKA-19297] - Refactor AsyncKafkaConsumer's use of Java Streams APIs in critical sections
  • [KAFKA-19311] - Document the different behaviors of commitAsync between classic consumer and async consumer
  • [KAFKA-19313] - Replace LogOffsetMetadata#UNIFIED_LOG_UNKNOWN_OFFSET by UnifiedLog.UNKNOWN_OFFSET
  • [KAFKA-19314] - Remove unnecessary code of closing snapshotWriter
  • [KAFKA-19322] - Remove the DelayedOperation constructor that accepts an external lock
  • [KAFKA-19326] - flaky test GroupMetadataManagerTest.testShareGroupInitializeSuccess()
  • [KAFKA-19330] - MockSerializer should extend Serializer
  • [KAFKA-19336] - Upgrade jackson libs since v2.16 is not maintained anymore
  • [KAFKA-19349] - Move CreateTopicsRequestWithPolicyTest to clients-integration-tests
  • [KAFKA-19355] - Remove interBrokerListenerName from ClusterControlManager

    • Bug

  • [KAFKA-12469] - The topic names in the metrics do not retain their format when extracting through JMX.
  • [KAFKA-15371] - MetadataShell is stuck when bootstrapping
  • [KAFKA-15900] - Flaky test: testOutdatedCoordinatorAssignment() – org.apache.kafka.clients.consumer.internals.EagerConsumerCoordinatorTest
  • [KAFKA-15931] - Cached transaction index gets closed if tiered storage read is interrupted
  • [KAFKA-16407] - ForeignKey INNER join ignores FK change when its previous value is null
  • [KAFKA-16434] - ForeignKey INNER join does not unset join result when FK becomes null
  • [KAFKA-17162] - DefaultTaskManagerTest may leak AwaitingRunnable thread
  • [KAFKA-17182] - Consumer fetch sessions are evicted too quickly with AsyncKafkaConsumer
  • [KAFKA-17184] - Remote index cache noisy logging
  • [KAFKA-17431] - KRaft servers require valid static socketserver configuration to start
  • [KAFKA-17516] - Client metrics configuration resources default configs not shown by kafka-configs.sh
  • [KAFKA-17792] - header parsing ends up timing out and using large quantities of memory if the string looks like a number
  • [KAFKA-17808] - InstanceAlreadyExistsException: kafka.admin.client:type=app-info,id=connector-dlq-adminclient- when add connector with tasks
  • [KAFKA-17981] - add Integration test for ConfigCommand to add config `key=[val1,val2]`
  • [KAFKA-17986] - Fix ConsumerRebootstrapTest and ProducerRebootstrapTest
  • [KAFKA-18064] - SASL mechanisms that do support neither integrity nor confidentality should throw exception on wrap/unwrap
  • [KAFKA-18067] - Kafka Streams can leak Producer client under EOS
  • [KAFKA-18089] - RemoteIndexCacheTest fails with caffeine > 3.1.1
  • [KAFKA-18115] - Issue loading big files for performance testing
  • [KAFKA-18117] - Support topic IDs in consumer SubscriptionState
  • [KAFKA-18135] - ShareConsumer HB UnsupportedVersion msg mixed with Consumer HB
  • [KAFKA-18274] - Failed to restart controller in testing due to closed socket channel
  • [KAFKA-18298] - Fix flaky PlaintextAdminIntegrationTest#testConsumerGroupsDeprecatedConsumerGroupState
  • [KAFKA-18345] - Investigate if binaryExponentialElectionBackoffMs can be removed or improved
  • [KAFKA-18371] - TopicBasedRemoteLogMetadataManagerConfig exposes sensitive configuration data in logs
  • [KAFKA-18400] - Don't use YYYY when formatting/parsing dates in Java client
  • [KAFKA-18418] - Flaky test in KafkaStreamsTest::shouldThrowOnCleanupWhileShuttingDownStreamClosedWithCloseOptionLeaveGroupFalse
  • [KAFKA-18440] - Admin does not convert the AuthorizationException to fatal error in using bootstrap controllers
  • [KAFKA-18441] - Fix flaky KafkaAdminClientTest#testAdminClientApisAuthenticationFailure
  • [KAFKA-18483] - Protect `Loggers` and `Log4jController` against `LinkageError`s
  • [KAFKA-18493] - Fix configure :streams:integration-tests project error
  • [KAFKA-18498] - Kafka Streams main thread does not release lock
  • [KAFKA-18537] - Fix flaky RemoteIndexCacheTest#testCleanerThreadShutdown
  • [KAFKA-18541] - Fix flaky KafkaStreamsTelemetryIntegrationTest#shouldPassCorrectMetricsDynamicInstances
  • [KAFKA-18547] - Failing test ClientUtilsTest#testParseAndValidateAddressesWithReverseLookup in trunk
  • [KAFKA-18557] - ShareCoordinatorServiceTest can leverage ShareCoordinatorTestConfig#testConfig to streamline the codebase
  • [KAFKA-18597] - max-buffer-utilization-percent is always 0
  • [KAFKA-18687] - New group coordinator triggers assignment recalculation on consumer group upgrade due to static member replacement
  • [KAFKA-18688] - Uniform homogeneous assignment is not always stable
  • [KAFKA-18713] - Kafka Streams Left-Join not always emitting the last value
  • [KAFKA-18723] - KRaft must handle corrupted records in the fetch response
  • [KAFKA-18758] - NullPointerException in shutdown following InvalidConfigurationException
  • [KAFKA-18770] - Fix flaky initializationError in ReplicationQuotasTest/RequestQuotaTest (thread leak)
  • [KAFKA-18805] - Consumer Heartbeat closed change should be locked
  • [KAFKA-18809] - share.coordinator.state.topic.min.isr is not applied
  • [KAFKA-18818] - Significant delay in Metadata Update After Admin Client Operations in KRaft Mode
  • [KAFKA-18837] - Validate controller.quorum.fetch.timeout.ms is a positive value
  • [KAFKA-18843] - MirrorMaker2 workerId is not unique, but use the same for all the workers
  • [KAFKA-18845] - Fail test QuorumControllerTest#testUncleanShutdownBrokerElrEnabled
  • [KAFKA-18859] - honor the error message of UnregisterBrokerResponse
  • [KAFKA-18923] - resource leak in RSM fetchIndex inputStream
  • [KAFKA-18962] - StateRestoreListener onBatchRestored method is called with the totalRestored on GlobalStateStore reprocess
  • [KAFKA-18991] - AbstractFetcherThread should verify the current leader epoch in the fetch request matches the leader epoch in the fetch state
  • [KAFKA-18992] - kafka-features.sh wronly output `upgraded kraft.version to 1`
  • [KAFKA-19047] - Broker registrations are slow if previously fenced or shutdown
  • [KAFKA-19054] - StreamThread exception handling with SHUTDOWN_APPLICATION may trigger a tight loop with MANY logs
  • [KAFKA-19071] - Incorrect documentation for remote.storage.enable
  • [KAFKA-19074] - Remove the cached responseData from ShareFetchResponse
  • [KAFKA-19080] - The constraint on segment.bytes is not enforced at topic level
  • [KAFKA-19128] - Kafka Streams should not get offsets when close dirty
  • [KAFKA-19130] - Do not add fenced brokers to BrokerRegistrationTracker on startup
  • [KAFKA-19131] - Exception thrown while updating the RemoteLogReader threads
  • [KAFKA-19147] - ConsumerGroupHeartbeat API leaks topic id and partition count of unauthorized topics
  • [KAFKA-19160] - Fetching stable offsets is very slow with a large number of partition and open transactions
  • [KAFKA-19163] - New group coordinator can incorrectly delete groups with pending transactional offsets
  • [KAFKA-19164] - New group coordinator won't delete groups with previously pending transactional offsets that were deleted
  • [KAFKA-19166] - Release tags are orphaned
  • [KAFKA-19171] - Kafka Streams crashes with UnsupportedOperationException
  • [KAFKA-19192] - Old bootstrap.metadata files cause problems with post-KAFKA-18601 servers
  • [KAFKA-19195] - In group describe operations all group IDs are forwarded to all group coordinator shards.
  • [KAFKA-19206] - ConsumerNetworkThread.cleanup() throws NullPointerException if initializeResources() previously failed
  • [KAFKA-19208] - KStream-GlobalKTable join should not drop left-null-key record
  • [KAFKA-19212] - Fix ElectionWasClean bug when partition reassignment completes
  • [KAFKA-19221] - IOException on log segment close shouldn't be ignored
  • [KAFKA-19234] - broker should return UNAUTHORIZATION error for non-existing topic in produce request
  • [KAFKA-19242] - Fix commit bugs caused by race condition during rebalancing.
  • [KAFKA-19246] - OffsetFetch API does not return group level errors correctly with version 1
  • [KAFKA-19274] - Group Coordinator Shards are not unloaded when __consumer_offsets topic is deleted
  • [KAFKA-19275] - client-state and thread-state metrics are always ""Unavailable"
  • [KAFKA-19288] - Ensure new consumer joining attempt not overwritten on delayed HB response to previous leave
  • [KAFKA-19294] - Fix BrokerLifecycleManager RPC timeouts
  • [KAFKA-19300] - AsyncConsumer#unsubscribe always timeout due to GroupAuthorizationException
  • [KAFKA-19334] - MetadataShell bypasses file lock unexpectedly due to lock file deletion
  • [KAFKA-19351] - AsyncConsumer#commitAsync should copy the input offsets
  • [KAFKA-19356] - AsyncConsumer should ensure consistency of assigned partitions and subscription
  • [KAFKA-19359] - [8.8] [CVE-2025-48734] [commons-beanutils] [1.9.4]
  • [KAFKA-19367] - InitProducerId with TV2 double-increments epoch if ongoing transaction is aborted
  • [KAFKA-19373] - Fix consumer integration tests checks on protocol name
  • [KAFKA-19383] - A duplicate topic record can be created when applying ClearElrRecord.
  • [KAFKA-19386] - Incorrect ExpirationReaper thread names from DelayedOperationPurgatory
  • [KAFKA-19392] - `metadata.log.segment.ms` is not applied
  • [KAFKA-19397] - TransactionManager.handleCompletedBatch throws NPE
  • [KAFKA-19398] - Error getting oldest-iterator-open-since-ms causes NPE in micrometer
  • [KAFKA-19407] - OffsetsOutOfOrderException on followers due to the race condition in the leader
  • [KAFKA-19411] - Delete ACLs records can exceed max records per user op
  • [KAFKA-19423] - Deflake streams_broker_bounce_test
  • [KAFKA-19427] - The __consumer_offsets topic applies the broker configuration message.max.bytes, which may cause the coordinator broker to allocate too much memory and cause OOM
  • [KAFKA-19444] - SASL GSSAPI not working with librdkafka and AK 4.x
  • [KAFKA-19445] - Group coordinator and share coordinator runtime metrics are aliased
  • [KAFKA-19495] - DefaultJwtRetriever could not be found with Docker native image
  • [KAFKA-19501] - System tests should use 17-bullseye instead of 17-buster
  • [KAFKA-19504] - AdminClient creates and adds second metrics reporter
  • [KAFKA-19520] - Bump Commons-Lang for CVE-2025-48924
  • [KAFKA-19522] - LastKnownLeader should only be elected if it is unfenced
  • [KAFKA-19576] - Fix typo in state-change log filename after rotate
  • [KAFKA-19642] - KRaft servers don't handle the cluser-level configs in starting

    • Task

  • [KAFKA-17671] - Create better documentation for transactions
  • [KAFKA-17830] - Cover unit tests for TBRLMM init failure scenarios
  • [KAFKA-18572] - Update Kafka Streams metric documenation
  • [KAFKA-18683] - File records slice errors when sliced for moved start position
  • [KAFKA-18787] - RemoteIndexCache fails to remove invalid files on init
  • [KAFKA-18826] - Add Global Thread Metrics
  • [KAFKA-18924] - Running the storage module tests produces a storage/storage.log file
  • [KAFKA-18971] - Update AK system tests for AK 4.0
  • [KAFKA-19058] - Running the streams/streams-scala module tests produces a streams-scala.log
  • [KAFKA-19100] - Use ProcessRole.toString instead of String in AclApis
  • [KAFKA-19264] - Remove fallback for thread pool sizes in RemoteLogManagerConfig

    • Test

  • [KAFKA-13747] - refactor TopologyTest to test different store type with parametrized
  • [KAFKA-15474] - AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky
  • [KAFKA-16918] - TestUtils#assertFutureThrows should use future.get with timeout
  • [KAFKA-17114] - DefaultStateUpdater::handleRuntimeException should update isRunning before calling `addToExceptionsAndFailedTasksThenClearUpdatingAndPausedTasks`
  • [KAFKA-17171] - Add test cases for `STATIC_BROKER_CONFIG`in kraft mode
  • [KAFKA-17421] - Add IT for ConsumerRecord#leaderEpoch
  • [KAFKA-18025] - Flaky ShareConsumerTest testAcquisitionLockTimeoutOnConsumer
  • [KAFKA-18033] - ShareConsumerTest is flaky
  • [KAFKA-18035] - TransactionsTest testBumpTransactionalEpochWithTV2Disabled failed on trunk
  • [KAFKA-18267] - Add test for CloseOption
  • [KAFKA-18310] - Flaky AbstractCoordinatorTest
  • [KAFKA-18546] - Change ClientUtilsTest#testParseAndValidateAddressesWithReverseLookup to mock test
  • [KAFKA-18568] - Flaky test ClientIdQuotaTest
  • [KAFKA-18606] - Flaky test DeleteSegmentsByRetentionTimeTest#executeTieredStorageTest
  • [KAFKA-18657] - Flaky SharePartitionManagerTest
  • [KAFKA-18777] - add `PartitionsWithLateTransactionsCount` to BrokerMetricNamesTest
  • [KAFKA-18914] - Migrate ConsumerRebootstrapTest to use new test infra
  • [KAFKA-18915] - Migrate AdminClientRebootstrapTest to use new test infra
  • [KAFKA-18975] - Move clients-integration-test out of core module
  • [KAFKA-19032] - Remove TestInfoUtils.TestWithParameterizedQuorumAndGroupProtocolNames
  • [KAFKA-19036] - Rewrite LogAppendTimeTest and move it to storage module
  • [KAFKA-19091] - Flaky test DelayedFutureTest#testDelayedFuture
  • [KAFKA-19153] - Add OAuth integration tests
  • [KAFKA-19220] - add tests to ensure the internal configs don't return by public APIs by default
  • [KAFKA-19229] - fix flaky testShareGroupDescribeWithGroupDescribeAndTopicDescribeAcl and testShareGroupDescribeWithoutGroupDescribeAcl
  • [KAFKA-19240] - Move MetadataVersionIntegrationTest to clients-integration-tests module
  • [KAFKA-19352] - PlaintextConsumerCommitTest > testCommitAsyncCompletedBeforeConsumerCloses [1] Type=Raft-Isolated

    • Sub-task

  • [KAFKA-4650] - Improve test coverage org.apache.kafka.streams.kstream.internals
  • [KAFKA-10551] - Support topic IDs in Produce request
  • [KAFKA-14145] - Faster propagation of high-watermark in KRaft topic partitions
  • [KAFKA-14484] - Move UnifiedLog to storage module
  • [KAFKA-14485] - Move LogCleaner to storage module
  • [KAFKA-14486] - Move LogCleanerManager to storage module
  • [KAFKA-14523] - Move RemoteLogManager to the storage module
  • [KAFKA-14690] - Add TopicId to OffsetCommit API
  • [KAFKA-14691] - Add TopicId to OffsetFetch API
  • [KAFKA-16524] - Metrics for KIP-853
  • [KAFKA-16538] - Support UpdateFeatures for kraft.version so we can go from static quorums to dynamic
  • [KAFKA-16718] - Add AdminClient.deleteShareGroupOffsets
  • [KAFKA-16720] - Add AdminClient.listShareGroupOffsets
  • [KAFKA-16729] - Support for read-committed isolation level
  • [KAFKA-16894] - Define switch to enable share groups for preview
  • [KAFKA-17039] - KIP-919 supports for `unregisterBroker`
  • [KAFKA-17351] - Validate compacted topics start offset handling in Share Partition
  • [KAFKA-17539] - Implement registerMetricsForSubscription
  • [KAFKA-17565] - Move MetadataCache interface to metadata module
  • [KAFKA-17809] - Fix flaky test: testExplicitAcknowledgementCommitAsync
  • [KAFKA-17836] - Move RackAwareTest to server module
  • [KAFKA-17856] - Move ConfigCommandTest and ConfigCommandIntegrationTest to tool module
  • [KAFKA-17897] - Deprecate Admin.listConsumerGroups
  • [KAFKA-17968] - Create VerifiableShareConsumer
  • [KAFKA-18042] - Reject the produce request with lower producer epoch early.
  • [KAFKA-18170] - Periodic snapshots for cold share partitions
  • [KAFKA-18172] - Move RemoteIndexCacheTest to storage module
  • [KAFKA-18198] - Disallow piggyback acknowledgments on initial ShareFetchRequest.
  • [KAFKA-18202] - Reject non-zero sequences when there is no producer ID state on the partition for transactions v2 idempotent producers
  • [KAFKA-18207] - Create TransactionCoordinatorRecordSerde for handling transaction records
  • [KAFKA-18232] - Prune share state topic metrics
  • [KAFKA-18246] - Fix ConnectRestApiTest.test_rest_api by adding multiversioning configs
  • [KAFKA-18260] - Add integration test to ShareConsumerTest for share.auto.offset.reset=by_duration
  • [KAFKA-18273] - Implement kafka-share-groups.sh --describe --verbose
  • [KAFKA-18301] - Make coordinator records first class citizen
  • [KAFKA-18302] - Update CoordinatorRecord
  • [KAFKA-18303] - Update share coordinator
  • [KAFKA-18304] - Introduce json converter generator
  • [KAFKA-18308] - Update CoordinatorSerde
  • [KAFKA-18370] - Fix to Kraft or remove tests associate with Zk Broker config in ReplicaFetcherThreadTest
  • [KAFKA-18392] - Require client-generated member IDs for ShareGroupHeartbeat
  • [KAFKA-18404] - Remove partitionMaxBytes from DelayedShareFetch
  • [KAFKA-18428] - Measure share consumers performance
  • [KAFKA-18433] - Adjust ShareFetchRequest RPC to optimise fetching and acknowledgement
  • [KAFKA-18449] - Add share-group state configs to reconfigure-server.properties
  • [KAFKA-18452] - Integrate share fetch batch size in share fetch
  • [KAFKA-18457] - Update DumpLogSegments to use coordinator record json converters
  • [KAFKA-18488] - Additional protocol tests for share consumption
  • [KAFKA-18494] - Initialize gaps in share partition cachedState to new available batches during share partition initialization
  • [KAFKA-18508] - Remove ControllerContext
  • [KAFKA-18513] - Add record count validation for share group state topic to ShareConsumerTest
  • [KAFKA-18522] - Limit fetch records batches as per acquired records
  • [KAFKA-18529] - ConsumerRebootstrapTest should run for async consumer
  • [KAFKA-18552] - Remove unnecessary version check from KafkaApisTest
  • [KAFKA-18558] - Include partitions from previous subscription in ShareFetchRequest only for non-empty acknowledgements.
  • [KAFKA-18559] - Cleanup FinalizedFeatures
  • [KAFKA-18590] - Cleanup DelegationTokenManager
  • [KAFKA-18593] - Remove ZkCachedControllerId In MetadataCache
  • [KAFKA-18594] - Cleanup BrokerLifecycleManager
  • [KAFKA-18604] - Update transaction coordinator
  • [KAFKA-18612] - Update ApiMessageFormatter
  • [KAFKA-18616] - Refactor DumpLogSegments's MessageParsers
  • [KAFKA-18618] - Reduce stickiness of share session handlers
  • [KAFKA-18620] - Remove UnifiedLog#legacyFetchOffsetsBefore
  • [KAFKA-18629] - DeleteShareGroups impl
  • [KAFKA-18630] - Clean ReplicaManagerBuilder
  • [KAFKA-18632] - ShareConsumerTest multibroker tests
  • [KAFKA-18651] - Core streams-specific broker configurations
  • [KAFKA-18653] - Fix mocks and potential thread leak issues causing silent RejectedExecutionException
  • [KAFKA-18694] - Migrate suitable classes to recorods in coordinator-common module
  • [KAFKA-18700] - Migrate SnapshotPath, Entry, OffsetAndEpoch, LogFetchInfo, and LogAppendInfo to record classes
  • [KAFKA-18706] - Move AclPublisher to metadata module
  • [KAFKA-18712] - Move Endpoint to server module
  • [KAFKA-18728] - Move ListOffsetsPartitionStatus to server module
  • [KAFKA-18732] - Implement broker topic metrics for share fetch
  • [KAFKA-18733] - Implement share group metrics for share fetch
  • [KAFKA-18734] - Implement share partition metrics for share fetch
  • [KAFKA-18738] - Revisit the DescribeShareGroupOffsets API to support multiple GroupIds in Request
  • [KAFKA-18755] - Align timeout in kafka-share-groups.sh with other group-related tools
  • [KAFKA-18756] - Add share group configs to system tests
  • [KAFKA-18757] - Create full-function SimpleAssignor to match KIP-932 description
  • [KAFKA-18761] - kafka-share-groups.sh --describe only shows offset information when members exist
  • [KAFKA-18763] - Resolve the flakiness in test_share_single_topic_partition system test
  • [KAFKA-18764] - Cluster-authorized share-state RPCs should throttle when not authorized
  • [KAFKA-18767] - Add client side config check for shareConsumer
  • [KAFKA-18772] - Define share group config defaults for Docker
  • [KAFKA-18779] - Validate responses from broker in client for ShareFetch and ShareAcknowledge APIs.
  • [KAFKA-18791] - Set default commit message to PR title + body
  • [KAFKA-18796] - Fix flaky test testAcquisitionLockOnOffsetWithWriteShareGroupStateFailure
  • [KAFKA-18799] - Remove AdminUtils
  • [KAFKA-18808] - add test to ensure the name= is not equal to default quota
  • [KAFKA-18811] - Allow passing the Consumer Configs to the AdminClientProperties for VerifiableShareConsumers
  • [KAFKA-18814] - Add support for client-side plugins
  • [KAFKA-18815] - Add support to Connect
  • [KAFKA-18817] - ShareGroupHeartbeat and ShareGroupDescribe API must check topic describe
  • [KAFKA-18828] - Update share group metrics code in group coordinator
  • [KAFKA-18829] - Can't use explicit ack when poll returns an empty batch
  • [KAFKA-18832] - ShareFetch behaviour seems incorrect when MaxBytes is less than record size
  • [KAFKA-18855] - MemoryRecords should have slice API
  • [KAFKA-18878] - Implement ShareSessionCache and DelayedShareFetchMetrics metrics for share fetch
  • [KAFKA-18882] - Remove BaseKey, TxnKey, and UnknownKey
  • [KAFKA-18888] - Add support for Authorizer
  • [KAFKA-18889] - Make records in ShareFetchResponse non-nullable
  • [KAFKA-18891] - Add support for RemoteLogMetadataManager and RemoteStorageManager
  • [KAFKA-18892] - Add support for ClientQuotaCallback
  • [KAFKA-18893] - Add support for ReplicaSelector
  • [KAFKA-18899] - Limit retry time for ShareConsumer.commitAsync
  • [KAFKA-18900] - Create a consumer config to select between implicit and explicit acknowledgement
  • [KAFKA-18901] - Improve redistribution of share group SimpleAssignor
  • [KAFKA-18902] - Implement ShareConsumer option to throw on poll if there are unacked records
  • [KAFKA-18909] - Move DynamicThreadPool to server module
  • [KAFKA-18910] - Remove kafka.utils.json
  • [KAFKA-18918] - Share partition lock held for onComplete exception
  • [KAFKA-18931] - Debug the case where kafka-share-groups.sh --describe --members does not match with the reality when share consumer is closed during broker outage
  • [KAFKA-18932] - Remove partition max bytes usage from share fetch requests
  • [KAFKA-18936] - Share fetch stuck when records are larger than fetch max bytes
  • [KAFKA-18942] - Add reviewers to PR body with committer-tools
  • [KAFKA-18946] - Move BrokerReconfigurable and DynamicProducerStateManagerConfig to server module
  • [KAFKA-18954] - Add a metric to track the election based on ELR
  • [KAFKA-18984] - failed to reset interval.ms by using kafka-client-metrics.sh
  • [KAFKA-19004] - Move DelayedDeleteRecords to server-common module
  • [KAFKA-19015] - Remove share session from cache on connection drop
  • [KAFKA-19017] - Change consumer-config to command-config in verifiable_share_consumer.py
  • [KAFKA-19018] - Send acquisition time in Share Fetch Response
  • [KAFKA-19019] - Verify Share Fetch with Tiered Storage
  • [KAFKA-19043] - Move DelayedRemoteListOffsets to the storage module
  • [KAFKA-19057] - Stabilize KIP-932 Kafka protocol RPC definitions in AK 4.1
  • [KAFKA-19062] - ShareConsumer should align handling of close timeout with async consumer.
  • [KAFKA-19063] - Send MaxRecords in share fetch request
  • [KAFKA-19075] - Include all share group dynamic configs in extractShareGroupConfigMap method in ShareGroupConfig
  • [KAFKA-19085] - SharePartitionManagerTest testMultipleConcurrentShareFetches throws silent exception and works incorrectly
  • [KAFKA-19087] - Move TransactionState to transaction-coordinator module
  • [KAFKA-19090] - Move DelayedFuture and DelayedFuturePurgatory to server module
  • [KAFKA-19109] - Don't print null in kafka-metadata-quorum describe status
  • [KAFKA-19110] - Add missing unit test for Streams-consumer integration
  • [KAFKA-19113] - Migrate DelegationTokenManager to server module
  • [KAFKA-19116] - Deletion of share group offsets does not reset the position as expected
  • [KAFKA-19119] - Move ApiVersionManager/SimpleApiVersionManager
  • [KAFKA-19133] - Support fetching for multiple remote fetch topic partitions in a single share fetch request
  • [KAFKA-19135] - Add IQ Support - move from feature branch
  • [KAFKA-19141] - Persist topic id in OffsetCommit record
  • [KAFKA-19145] - Move LeaderEndPoint to server module
  • [KAFKA-19154] - Return INVALID_OFFSET if requested topic id does not match persisted one
  • [KAFKA-19156] - Remove group.share.max.groups config
  • [KAFKA-19157] - Add group.share.max.share.sessions config
  • [KAFKA-19158] - Add a new error code for “share session limit reached”
  • [KAFKA-19159] - Remove time based eviction of share sessions from the cache
  • [KAFKA-19162] - Topology description sent to broker contains non-determinstically ordered topic configs
  • [KAFKA-19169] - Enhance AuthorizerIntegrationTest for share group APIs
  • [KAFKA-19173] - Add `Feature` for streams groups
  • [KAFKA-19182] - Move SchedulerTest to server module
  • [KAFKA-19183] - Replace Pool with ConcurrentHashMap
  • [KAFKA-19184] - Document kraft version upgrade
  • [KAFKA-19189] - Handle deletion of __share_group_state topic in share coordinator
  • [KAFKA-19198] - Deletion of topic assigned to a share group can result in NPE
  • [KAFKA-19201] - Handle deletion of a topic which has initialised share-group state
  • [KAFKA-19204] - Timeouts in coordinator runtime operations can break share group initialization and deletion
  • [KAFKA-19207] - Move ForwardingManagerMetrics and ForwardingManagerMetricsTest to server module
  • [KAFKA-19215] - Handle share partition fetch lock cleanly in DelayedShareFetch to avoid a member incorrectly releasing a share partition's fetch lock
  • [KAFKA-19216] - Eliminate flakiness in kafka.server.share.SharePartitionTest
  • [KAFKA-19217] - ShareConsumerTest.testComplexShareConsumer is unreliable
  • [KAFKA-19218] - Add missing leader epoch to read share group state summary response
  • [KAFKA-19226] - Add test_console_share_consumer system test
  • [KAFKA-19227] - Fix performance for share fetch with piggyback acknowledgements
  • [KAFKA-19231] - Handle fetch request when share session cache is full
  • [KAFKA-19232] - Handle share session limit reached error in clients
  • [KAFKA-19241] - Reuse the same socket on subsequent share requests in ShareFetchAcknowledgeRequestTest
  • [KAFKA-19245] - Update defaults for preview of queues configurations
  • [KAFKA-19251] - Resolve the flaky test ShareConsumerTest.testMultipleConsumersInGroupConcurrentConsumption
  • [KAFKA-19253] - Improve metadata handling for share version using feature listeners
  • [KAFKA-19258] - Release acquired records for members whose share session is evicted due to connection drop
  • [KAFKA-19260] - Move LoggingController to server module
  • [KAFKA-19268] - Missing mocks for SharePartitionManagerTest tests
  • [KAFKA-19271] - Private interface for injecting test wrappers for KIP-1071 enabled consumers
  • [KAFKA-19281] - Add metadata version check to enable share coordinator timer tasks.
  • [KAFKA-19285] - Improve test coverage for SharePartitionManager
  • [KAFKA-19290] - Exploit mapKey optimisation in protocol requests and responses
  • [KAFKA-19291] - Increase the timeout of remote storage share fetch requests in purgatory
  • [KAFKA-19302] - Move ReplicaState and Replica to server module
  • [KAFKA-19310] - Missing mocks for DelayedShareFetchTest tests related to Memory Records slicing
  • [KAFKA-19312] - Share partition fetch locks concurrent thread release
  • [KAFKA-19316] - Add share_group_command_test.py system test
  • [KAFKA-19317] - Fix flaky test : ShareConsumerTest.testAlterReadUncommittedToReadCommittedIsolationLevel()
  • [KAFKA-19320] - Add share_consume_bench_test.py system tests
  • [KAFKA-19321] - Add share_consumer_performance.py and related system tests
  • [KAFKA-19328] - SharePartitionManagerTest testMultipleConcurrentShareFetches doAnswer chaining needs verification
  • [KAFKA-19337] - Bug in ShareCoordinatorShard.generateShareStateRecord causes stateEpoch update misses.
  • [KAFKA-19338] - Share coordinator read state and write state should return error, if the share partition is not initialized.
  • [KAFKA-19342] - Add AuthorizerIntegrationTest for alter share group offsets
  • [KAFKA-19344] - Move assignable share partition method from describer to group spec.
  • [KAFKA-19346] - Move LogReadResult to server module
  • [KAFKA-19353] - Preview documentation
  • [KAFKA-19358] - Do not update share group offset reset strategy from VerifiableShareConsumer in share_consumer_test.py tests
  • [KAFKA-19369] - Add group.share.assignors config
  • [KAFKA-19370] - Create JMH benchmark for assessing performance of share group assignor
  • [KAFKA-19389] - High memory consumption for completed share fetch requests
  • [KAFKA-19414] - Remove Public APIs from 4.1
  • [KAFKA-19453] - Handle Share group records replay in group coordinator for non-existent share groups.