5.4.3

What's Changed

• Fixing Deadlock Bug Between ZK Callback & Event Thread On Acquiring Coordinator Object by @shrinandthakkar in (#964)
• Fixing the CPU Intensive RemoveAll with Lists in Sticky & Load Based Partition Assignment Strategies by @shrinandthakkar in (#965)
• Optimizing By Using Precomputed Set in Partition Assignment Routine by @shrinandthakkar in (#960)
• Standardize log messages for datastream operations by @amybubu in (#961)
• Upgrade Pre Assignment Clean Up To Remove Dependency Tasks Across Hosts by @shrinandthakkar in (#952)
• Quick Optimization For Partition Assignors by @shrinandthakkar in (#956)

Full Changelog: 5.4.0...5.4.3

5.4.0

What's Changed

• BMM Restart Improvements Part 2. Followers claim their tokens after handling stop assignments by @jzakaryan in #921
• BMM Restart Improvements Part 3. Leader polls for unclaimed tokens before declaring a datastream STOPPED by @jzakaryan in #922
• BMM Restart Improvements Part 4. Edge cases, cleanup and general improvements by @jzakaryan in #924
• Moved the datastream stop check poll to a separate thread to avoid blocking the event queue by @jzakaryan in #925
• BMM Restart Improvements: Bug fixes and logging improvements by @jzakaryan in #926
• Bump Kafka version by @vmaheshw in #929
• Handling Metrics and SLA Reporting for Throughput Violating Topics via Datastream Update API Part 1 by @shrinandthakkar in #928
• Added Support and Handling For Generic Headers by @shrinandthakkar in #931
• Minor improvements w.r.t #928 & Releasing a newer version by @shrinandthakkar in #932
• Exposed the Kafka Connector's task interrupt timeout as config by @jzakaryan in #933
• Separated Metrics Handling for Throughput Violating Topics by @shrinandthakkar in #930
• Released New Version 5.3.2 by @shrinandthakkar in #935
• Increased the size of token claims thread pool by @jzakaryan in #938
• Audit START/END log events for CoordinatorEvent operations by @ehoner in #936
• Add metrics for CoordinatorEvent queue by @ehoner in #937
• Minor improvements for assignment tokens feature by @jzakaryan in #941
• Fix Metric Registration Issue for Throughput Violations Reporting by @shrinandthakkar in #942
• Fix tests and reduce the overall test execution time by @vmaheshw in #939
• Fixes for NoOpTransportProvider by @jzakaryan in #940
• Add stop method to TransportProviderAdmin by @vmaheshw in #944
• Log existing datastream for update and delete call by @vmaheshw in #948
• EventQueue metrics are not exported by @ehoner in #945
• Added log messages to debug assignPartitions by @jzakaryan in #951
• Replacing Coordinator Queue With Deque & Fixing Usage Of toMap Util by @shrinandthakkar in #950
• Fixing Flaky Test testLeaderDoAssignmentForNewlyElectedLeaderFailurePathVariation by @shrinandthakkar in #953
• Minor version release - 5.4.0 by @ehoner in #954

Full Changelog: 5.1.0...5.4.0

5.1.0

The zookeeper dependency update in version 5 includes includes breaking changes. Some of the features below were included in untagged point releases.

What's Changed

• Add support for post datastream create,update,delete and state change action by @hshukla in #915
• Update zookeeper dependency to import from LinkedIn published version by @surajkn in #917
• Set datastream status to deleting for delete call by @hshukla in #918
• Bump version after release by @hshukla in #920
• BMM Restart Improvements Part 1. Leader Coordinator Issuing Assignment Tokens by @jzakaryan in #919

New Contributors

• @hshukla made their first contribution in #915

Full Changelog: 4.1.0...5.1.0

4.1.0

What's Changed

• Refactored StickyPartitionAssignmentStrategy and implemented task estimation logic in LoadBasedPartitionAssignmentStrategy by @jzakaryan in #835
• Fix flaky test testConsumeFromMultipleTopics by @vmaheshw in #838
• Refactor Optional parameters in the constructor of strategy by @vmaheshw in #836
• Implemented logic to prevent tasks from having more than specified number of partitions by @jzakaryan in #837
• Publishing artifacts to JFrog by @shrinandthakkar in #839
• Make the Throughput based assignment and task estimation based on partition assignment configurable. by @vmaheshw in #841
• Implemented metrics for LoadBasedPartitionAssignmentStrategy by @jzakaryan in #840
• Clear the CallbackStatus entry from the map in FlushlessEventProducerHandler by @vmaheshw in #843
• Handle the new session after session expiry by @vmaheshw in #770
• Fixed issue with config key by @jzakaryan in #847
• Print count of orphaned tasks and orphaned task locks in the log message by @surajkn in #844
• Datastream stop transition redesign by @shrinandthakkar in #842
• Fix flaky stopping simultaneous datastream test by @shrinandthakkar in #849
• Update the DatabaseChunkedReader to take the Connection as input rather than the DataSource by @somandal in #850
• Handle leaking TP exception in handleAssignmentChange by @surajkn in #845
• Migrating from AdminUtils with AdminClient by @surajkn in #848
• Fix running task data structure logic in AbstractKafkaConnector for failing to stop task by @surajkn in #851
• Removing partial Helix ZkClient dependency by @surajkn in #852
• Stop the tasks in parallel in AbstractKafkaConnector by @vmaheshw in #853
• Add additional log in LoadBasedPartitionAssignmentStrategy by @vmaheshw in #856
• Fixing flaky test testCreateDatastreamHappyPathDefaultRetention by @surajkn in #854
• Added toString() override in PartitionThroughputInfo by @jzakaryan in #858
• Add Stats to DatastreamTaskImpl by @vmaheshw in #855
• Make PartitionThroughputProvider metricsAware by @vmaheshw in #859
• Added base strategy metric info in LoadBasedPartitionAssignmentStrategy by @jzakaryan in #857
• Add alert metrics to identify that elastic task configurations require adjustment by @vmaheshw in #860
• Fix restartDeadTask logic when the task thread has died by @surajkn in #861
• Fix metric infos in PartitionThroughputProvider by @vmaheshw in #862
• Fix the metrics deregistration in AbstractKafkaConnector when multiple stop are called by @vmaheshw in #865
• Fix logging in LoadBasedPartitionAssignmentStrategy by @vmaheshw in #866
• Make Default byte in rate and Msg in rate configurable by @vmaheshw in #864
• Metrics are getting emitted in LoadBasedPartitionAssignmentStrategy only when it needs adjustment by @vmaheshw in #867
• Use topic level throughput information when partition level information is unavailable. by @vmaheshw in #871
• Fix compilation error by @vmaheshw in #874
• Loadbased Partition assigner not using topic level metrics to recognize partitions by @vmaheshw in #876
• Flushless producer supporting both comparable and non comparable offsets by @shrinandthakkar in #873
• LiveInstanceProvider subscription should be done only by the leader coordinator by @vmaheshw in #879
• Fixed issue with missing exception message during task initialization by @jzakaryan in #882
• Kafka upgrade by @srinagaraj in #881
• skipping onPartitionsRevoked during consumer.close() call by @srinagaraj in #886
• Scala 2.12 upgrade by @srinagaraj in #895
• Upgrade avro and move jackson from codehaus to fasterxml by @vmaheshw in #894
• Fix topic deletion when multiple duplicate streams expire at the same time by @surajkn in #897
• Use 2.4.1.57 kafka version by @shrinandthakkar in #901
• tests for min/max partitions per task metrics and minor code quality improvements by @ryannedolan in #887
• fix rebalancing-tasks bug and added tests by @shrinandthakkar in #900
• Refactor Stopping Tasks On Assignment Change of Tasks by @shrinandthakkar in #868
• Change python dependency in commit-msg git hook by @surajkn in #904
• Remove Scala Dependencies by @shrinandthakkar in #905
• Introduce broadcast API to TransportProvider by @surajkn in #903
• Dedupe tasks on LeaderDoAssignment by @jogrogan in #906
• Fix Stopping Logic and Maintain Stopping Latch Counter by @shrinandthakkar in #877
• Fixing test OnAssignmentChangeMultipleReassignments by @shrinandthakkar in #908
• Update kafka version by @jogrogan in #910
• Replace 101tec ZkClient with Helix ZkClient by @surajkn in #909
• Add retry to query retention time for destination topic by @surajkn in #863
• Upgrade Zookeeper version to 3.6.3 by @jogrogan in #913

New Contributors

• @surajkn made their first contribution in #844
• @srinagaraj made their first contribution in #881
• @ryannedolan made their first contribution in #887
• @jogrogan made their first contribution in #906

Full Changelog: 1.1.0...4.1.0

1.1.0

Added configs for fetching throughput info (#834)

1.0.2

Release Notes

1.0.1

Release Notes

1.0.0

First open-source release