github.com/harlow/kinesis-consumer

Highlights

• Avoid re-emitting already-consumed parent shards after an AllGroup restart (#200).
• Resume processing leases already owned by a restarted consumer-group worker (#201).
• Prevent Scan from deadlocking when shard processing and shard discovery fail concurrently (#203).
• Update the AWS eventstream dependency from 1.7.6 to 1.7.8 (#198).

This is a backward-compatible patch release with no public API changes.

Full changelog: v0.5.0...v0.5.1

Highlights

• Implement cooperative shard handoff for consumer-group rebalancing.
• Preserve parent/child shard ordering so child shards wait for parent completion.
• Release consumer-group shards cleanly on graceful shutdown.
• Improve throttling retry handling and buffered checkpoint flushing.

Breaking Change

• Remove the lease-stealing API introduced during the consumer-group work. This release is cut as v0.5.0 instead of a patch release because that public API surface was removed.

Included Changes

• Fix buffered checkpoint flushing on scan exit (#189)
• Flush checkpoints before lease release (#190)
• Document aggregation checkpoint limits (#191)
• Remove lease stealing API (#192)
• Improve throttling retry handling (#194)
• Gate child shards on parent completion (#193)
• Implement cooperative shard handoff for consumer-group rebalancing (#196)
• Document consumer-group handoff changes
• Release group shards on graceful shutdown
• Add shutdown and handoff end-to-end coverage
• Fix root package shutdown test coverage

Highlights

• Reworked consumer-group balancing to use release-based handoff instead of lease stealing.
• Added shard revocation handling so graceful rebalances stop local scans before leases are released.
• Fixed shutdown behavior so canceled GetRecords calls do not surface as false scan errors.

Testing

• Added opt-in Go integration harnesses covering:
  • startup before stream creation
  • late worker join and rebalance
  • failover after lease expiry
  • Redis checkpoint resume on restart
• Added GitHub Actions CI for:
  • go test ./...
  • example integration suite
• Made Redis unit tests hermetic with miniredis.

Other changes

• Hardened the consumer-group-ddb example for startup ordering and concurrent DynamoDB table creation.
• Removed Travis CI in favor of GitHub Actions.
• Improved README visibility for consumer-group usage and removed stale Travis references.

Release v0.4.0: experimental DynamoDB consumer groups

Most notable changes are:
switch to AWS SDK2
adding support for split shards
bugfixes

Final cut before moving to AWS SDK v2

Major changes:

• Remove concept of Client it was confusing as it wasn't a direct standin for a Kinesis client.
• Rename ScanError to ScanStatus as it's not always an error.

Minor changes:

• Update tests to use Kinesis mock

This is the last stable release from which there is a separate Client. It has caused confusion and will be removed going forward.

https://github.com/harlow/kinesis-consumer/releases/tag/v0.2.0

This is the last stable release of the consumer which aggregated records in batch before calling the callback func.

https://github.com/harlow/kinesis-consumer/releases/tag/v0.1.0