• 2.16.0 RC1

• Syntax and style fixes

Fixes

• DeserializingConsumer and DeserializingShareConsumer now deserialize the
  message key before the value, matching SerializingProducer and the Java
  client. When both key and value fail to deserialize, the key error is now
  surfaced instead of the value error.
• Minor fix for subjectPrefix parameter in subjects API (#2311)
• Fix race conditions (#2315)
• Fix segmentation fault after calling AdminClient.delete_records() followed
  by another Admin API call (e.g. list_topics()) on Python 3.14.
• Use asyncio.get_running_loop() instead of asyncio.get_event_loop() to avoid creating a new event loop and raise an error in case a loop isn't available (AlexCai26, #2339).

confluent-kafka-python 2.15.1 is based on librdkafka 2.15.1, see the
librdkafka release notes
for a complete list of changes, enhancements, fixes and upgrade considerations.

2.15.1rc2: make validation pass on setup_all_versions.py (#2354)

KIP-932 Queues for Kafka – Now in Preview

confluent-kafka-python 2.15.0 adds a Preview implementation of the KIP-932 share consumer (Queues for Kafka). Members of a share group cooperatively consume from the same partitions with per-record acquire/acknowledge semantics and redelivery, providing queue-like consumption on top of Kafka.

• New ShareConsumer and DeserializingShareConsumer clients: subscribe, batch poll (poll() returns a Messages batch), and close (with context-manager support).
• Implicit (default) and explicit acknowledgement modes (share.acknowledgement.mode), with AcknowledgeType ACCEPT / RELEASE / REJECT and Message.delivery_count().
• Synchronous and asynchronous commit (commit_sync / commit_async) and an acknowledgement-commit callback.
• Runtime SASL credential updates (set_sasl_credentials) and new IllegalStateException / ConcurrentModificationException exceptions.
• (#2217, #2239, #2241, #2252, #2253, #2265, #2273, #2277, #2279)

See the Share Consumer guide and the Share consumers section of librdkafka's INTRODUCTION.md.

Note: The share consumer is currently in Preview and should not be used in production environments. Its public interfaces may change before General Availability, and known limitations apply (see the guide). The share consumer is single-threaded and not thread-safe. It requires a broker with share groups enabled (generally available in Apache Kafka 4.2.0).

Features

• New optional install confluent-kafka[oauthbearer-aws] provides AWS IAM-based
  OAUTHBEARER authentication via AWS STS GetWebIdentityToken. Activate by setting sasl.oauthbearer.method=oidc,
  sasl.oauthbearer.metadata.authentication.type=aws_iam, and
  sasl.oauthbearer.config="region=...,audience=...". See the
  AWS IAM OAUTHBEARER guide for full configuration, and
  examples/oauth_oidc_ccloud_aws_iam.py
  for a worked example.

Enhancements

• Add support for union-of-pools and auto pool mapping for Schema Registry (#2182)

Fixes

• Fix Encryption fails when expanded union types have two references to the same record (ChristophMajcenAtXxxlutz, #2262)
• Make orjson optional in JSON serdes with stdlib json fallback (#2281)
• Handle non-http errors during retries (#2292)
• Use TLS certification verification with Hashicorp Vault (#2294)

confluent-kafka-python v2.15.0 is based on librdkafka v2.15.0, see the
librdkafka release notes
for a complete list of changes, enhancements, fixes and upgrade considerations.

v2.14.2 is a maintenance release with the following fixes and enhancements:

Fixes

• Fix URL joining in Python client (#2228)
• Handle anyOf/allOf in JSON transforms (#2237)
• Fix redefining a named Avro type in a diamond dependency pattern (#2238)
• Fix typing_extensions import errors on Python 3.12 (#2232)
• Fix Encryption fails when referencing self defined types (#2254)

confluent-kafka-python v2.14.2 is based on librdkafka v2.14.2, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

v2.14.0 is a feature release with the following features, fixes and enhancements:

Enhancements

• Implement async context manager protocol for AIOProducer and AIOConsumer (#2180)
• Add AssociatedNameStrategy (#2194)
• Add enableAt to RuleSet (#2218)

Fixes

• Ensure normalize.schemas config is passed during Protobuf ref lookup #2214
• Fix type annotations for context manager hooks so that they are correct for subclasses (#2181)
• Fix OAuth callback handling for Async IO clients to prevent initialization failures (#2219)

confluent-kafka-python v2.14.0 is based on librdkafka v2.14.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

v2.13.2 is a maintenance release with the following fixes and enhancements:

Enhancements

• Add Confluent-Client-Version header to requests to SR, Python client (#2148)
• Add UAMI OAuth changes (#2189)

Fixes

• Fixed memory leak in Producer.produce() when called with headers and raises BufferError (queue full) or RuntimeError (producer closed). The allocated rd_headers memory is now properly freed in error paths before returning. Fixes Issue https://github.com/confluentinc/confluent-kafka-python/issues/2167.
• Fixed type hinting of KafkaError class, Consumer.init(), Producer.init(), Producer.produce() and Consumer.commit() and introduced a script in tools directory to keep error codes up to date. Fixes Issue https://github.com/confluentinc/confluent-kafka-python/issues/2168.
• Fix the token expiration logic in SR Oauth (#2177)
• Ensure use of cachetools is thread-safe (#2178)
• Remove passing resolver in json validate (#2186)
• Fix JSON schema resolver bug (#2188)

confluent-kafka-python v2.13.2 is based on librdkafka v2.13.2, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Installation

pip install confluent-kafka==2.13.2

v2.13.0 is a feature release with the following features, fixes and enhancements:

Enhancements

• Enforced type hinting for all interfaces
• Handle OAuth Token Refreshes using background thread for Admin, Producer and Consumer clients (#2130)
• Added black and isort linting rules and enforcement to codebase (#2133, #2137)
• Enabled direct creation of Message objects (#2128)
• Added close() method to producer (#2039)
• Added context manager for librdkafka classes to enable easy scope cleanup (#2114)
• Expose deterministic partitioner functions (#2116)
• Add Accept-Version header for schemas (#2117)
• Enhanced the BufferTimeoutManager to flush the librdkafka queue
• Remove experimental module designation for Async classes (#2143)
• Add __len__ function to AIOProducer (#2140)
• Enhance Message class to include serialisation support and rich comparison (#2153)
• Support Strict Validation Flags in Avro Serializers (#2147)

Fixes

• Type hint __enter__ to return the same object type that called it (#2157)
• Fixed Consumer.poll(), Consumer.consume(), Producer.poll(), and Producer.flush() blocking indefinitely and not responding to Ctrl+C (KeyboardInterrupt) signals. The implementation now uses a "wakeable poll" pattern that breaks long blocking calls into smaller chunks (200ms) and periodically re-acquires the Python GIL to check for pending signals. This allows Ctrl+C to properly interrupt blocking operations. Fixes Issues #209 and #807 (#2126).
• Fix support for wrapped Avro unions (#2134)
• Fixed segfault exceptions on calls against objects that had closed internal objects (#2122)
• Handle evolution during field transformation of schemas (#2121)
• Handle null group name to prevent segfault in Admin list_consumer_group_offsets() (#2118)
• Ensure schemaId initialization is thread-safe (#2120)
• Fix error propagation rule for Python's C API
• Fix SR delete behavior with client-side caching
• Don't leave NewTopic partially-initialized on error (#2151)
• Fix formatting issue of TopicPartition that causes SystemError on Windows (#2141)
• Add Py_None check for msg and offset params in commit() and store_offsets() (#2145)
• Fix experimental module references (#2144)
• Update outdated example in README.md (#2150)

confluent-kafka-python v2.13.0 is based on librdkafka v2.13.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Installation

pip install confluent-kafka==2.13.0

v2.13.0b1 - 2025-11-25

This is a beta release intended for --pre installers to get early access to upcoming changes. Exact changes included are:

• Enforced type hinting for all interfaces
• Handle OAuth Token Refreshes
• Added black and isort linting rules and enforcement to codebase
• Fix support for wrapped Avro unions
• Enabled direct creation of Message objects
• Added close() method to producer
• Added context manager for librdkafka classes to enable easy scope cleanup
• Fixed segfault exceptions on calls against objects that had closed internal objects
• Handle evolution during field transformation of schemas
• Expose deterministic partitioner functions
• Handle null group name to prevent segfault in Admin list_consumer_group_offsets()
• Add Accept-Version header for schemas
• Ensure schemaId initialization is thread-safe
• Fix error propagation rule for Python's C API
• Fix SR delete behavior with client-side caching
• Enhanced the BufferTimeoutManager to flush the librdkafka queue

v2.12.2 - 2025-11-06

v2.12.2 is a maintenance release with the following change:

Schema Registry Client

• Handled evolution during field transformation
• Changed Accept-Version to Confluent-Accept-Unknown-Properties in request header

v2.12.1

v2.12.1 is a maintenance release with the following fixes:

• Restored macOS binaries compatibility with macOS 13
• libversion() now returns the string/integer tuple for varients on version -- use version() for string only response
• Added Python 3.14 support and dropped 3.7 support -- Free-threaded capabilities not fully supported yet
• Fixed use.schema.id in sr.lookup_schema()
• Removed tomli dependency from standard (non-documentation) requirements
• Fixed experimental asyncio example files to correctly use new capabilities
• Fixed invalid argument error on schema lookups on repeat requests
• Fixed documentation generation and added error checks for builds to prevent future breaks

confluent-kafka-python v2.12.0

v2.12.0 is a feature release with the following enhancements:

KIP-848 – General Availability

Starting with confluent-kafka-python 2.12.0, the next generation consumer group rebalance protocol defined in KIP-848 is production-ready. Please refer to the following migration guide for moving from classic to consumer protocol.

Note: The new consumer group protocol defined in KIP-848 is not enabled by default. There are few contract change associated with the new protocol and might cause breaking changes. group.protocol configuration property dictates whether to use the new consumer protocol or older classic protocol. It defaults to classic if not provided.

AsyncIO Producer (experimental)

Introduces beta class AIOProducer for asynchronous message production in asyncio applications.

Added

• AsyncIO Producer (experimental): Introduces beta class AIOProducer for asynchronous message production in asyncio applications. This API offloads blocking librdkafka calls to a thread pool and schedules common callbacks (error_cb, throttle_cb, stats_cb, oauth_cb, logger) onto the event loop for safe usage inside async frameworks.

Features

• Batched async produce: await AIOProducer(...).produce(topic, value=...) buffers messages and flushes when the buffer threshold or timeout is reached.
• Async lifecycle: await producer.flush(), await producer.purge(), and transactional operations (init_transactions, begin_transaction, commit_transaction, abort_transaction).

Limitations

• Per-message headers are not supported in the current batched async produce path. If headers are required, use the synchronous Producer.produce(...) or offload a sync produce call to a thread executor within your async app.

Guidance

• Use the AsyncIO Producer inside async apps/servers (FastAPI/Starlette, aiohttp, asyncio tasks) to avoid blocking the event loop.
• For batch jobs, scripts, or highest-throughput pipelines without an event loop, the synchronous Producer remains recommended.

Enhancement and Fixes

• Kafka OAuth/OIDC metadata based authentication examples with Azure IMDS (#2083).

confluent-kafka-python v2.12.0 is based on librdkafka v2.12.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.11.1

v2.11.1 is a maintenance release with the following enhancements:

confluent-kafka-python v2.11.1 is based on librdkafka v2.11.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.11.0

v2.11.0 is a feature release with the following enhancements:

confluent-kafka-python v2.11.0 is based on librdkafka v2.11.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.10.1

v2.10.1 is a maintenance release with the following fixes

• Handled None value for optional ctx parameter in ProtobufDeserializer (#1939)
• Handled None value for optional ctx parameter in AvroDeserializer (#1973)

confluent-kafka-python v2.10.1 is based on librdkafka v2.10.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.10.0

v2.10.0 is a feature release with the following fixes and enhancements:

• [KIP-848] Group Config is now supported in AlterConfigs, IncrementalAlterConfigs and DescribeConfigs. (#1856)
• [KIP-848] describe_consumer_groups() now supports KIP-848 introduced consumer groups. Two new fields for consumer group type and target assignment have also been added. Type defines whether this group is a classic or consumer group. Target assignment is only valid for the consumer protocol and its defaults to NULL. (#1873).

confluent-kafka-python v2.10.0 is based on librdkafka v2.10.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.9.0

v2.9.0 is a feature release with the following fixes and enhancements:

Add Client Credentials OAuth support for Schema Registry (#1919) Add custom OAuth support for Schema Registry (#1925) confluent-kafka-python v2.9.0 is based on librdkafka v2.8.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.8.2

v2.8.2 is a maintenance release with the following fixes and enhancements:

• Fixed caching to ensure cached schema matches input. (#1922)
• Fix handling of named Avro schemas (#1928)

confluent-kafka-python v2.8.2 is based on librdkafka v2.8.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Note: Versioning is skipped due to breaking change in v2.8.1. Do not run software with v2.8.1 installed.

confluent-kafka-python v2.8.0

v2.8.0 is a feature release with the features, fixes and enhancements:

• Ensure algorithm query param is passed for CSFLE (#1889)
• DGS-19492 Handle records nested in arrays/maps when searching for tags (#1890)

confluent-kafka-python v2.8.0 is based on librdkafka v2.8.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.7.0

Note: As part of this release, we are deprecating v2.6.2 release and yanking it from PyPI. Please refrain from using v2.6.2. Use v2.7.0 instead.

Note: This release modifies the dependencies of the Schema Registry client. If you are using the Schema Registry client, please ensure that you install the extra dependencies using the following syntax:

pip install confluent-kafka[schemaregistry]

or

pip install confluent-kafka[avro,schemaregistry]

Please see the README.md for more information related to installing protobuf, jsonschema or rules dependencies.

v2.7.0 is a feature release with the following features, fixes and enhancements:

• Support for Data Contracts with Schema Registry, including
  • Data Quality rules
  • Data Transformation rules
  • Client-Side Field Level Encryption (CSFLE)
  • Schema Migration rules (requires Python 3.9+)
• Migrated the Schema Registry client from requests to httpx
• Add support for multiple URLs (#409)
• Allow configuring timeout (#622)
• Fix deletion semantics (#1127)
• Python deserializer can take SR client (#1174)
• Fix handling of Avro unions (#1562)
• Remove deprecated RefResolver for JSON (#1840)
• Support delete of subject version (#1851)
• Added missing dependency on googleapis-common-protos when using protobufs. (#1881, @tenzer)

confluent-kafka-python v2.7.0 is based on librdkafka v2.6.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.6.2

[!WARNING] Due to an error in which we included dependency changes to a recent patch release, Confluent recommends users to refrain from upgrading to 2.6.2 of Confluent Kafka. Confluent will release a new minor version, 2.7.0, where the dependency changes will be appropriately included. Users who have already upgraded to 2.6.2 and made the required dependency changes are free to remain on that version and are recommended to upgrade to 2.7.0 when that version is available. Upon the release of 2.7.0, the 2.6.2 version will be marked deprecated. We apologize for the inconvenience and appreciate the feedback that we have gotten from the community.

Note: This version is yanked from PyPI. Use 2.7.0 instead.

Note: This release modifies the dependencies of the Schema Registry client. If you are using the Schema Registry client, please ensure that you install the extra dependencies using the following syntax:

pip install confluent-kafka[schemaregistry]

or

pip install confluent-kafka[avro,schemaregistry]

Please see the README.md for more information related to installing protobuf, jsonschema or rules dependencies.

v2.6.2 is a feature release with the following features, fixes and enhancements:

• Support for Data Contracts with Schema Registry, including
  • Data Quality rules
  • Data Transformation rules
  • Client-Side Field Level Encryption (CSFLE)
  • Schema Migration rules (requires Python 3.9+)
• Migrated the Schema Registry client from requests to httpx
• Add support for multiple URLs (#409)
• Allow configuring timeout (#622)
• Fix deletion semantics (#1127)
• Python deserializer can take SR client (#1174)
• Fix handling of Avro unions (#1562)
• Remove deprecated RefResolver for JSON (#1840)
• Support delete of subject version (#1851)

confluent-kafka-python is based on librdkafka v2.6.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.6.1

v2.6.1 is a maintenance release with the following fixes and enhancements:

• Removed python 3.6 support. (#1592)
• Migrated build system from setup.py to pyproject.toml in accordance with PEP 517 and PEP 518, improving project configuration, build system requirements management, and compatibility with modern Python packaging tools like pip and build. (#1592)
• Added an example for OAUTH OIDC producer with support for confluent cloud (#1769, @sarwarbhuiyan)

confluent-kafka-python is based on librdkafka v2.6.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.6.0

v2.6.0 is a feature release with the following features, fixes and enhancements:

• Added Python 3.13 wheels (#1828).
• KIP-848 EA: Admin API for listing consumer groups now has an optional filter to return only groups of given types (#1830).
• KIP-460 Admin Leader Election RPC (#1818)
• Added Transactional id resource type for ACL operations (@JohnPreston, #1834).

confluent-kafka-python is based on librdkafka v2.6.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.5.3

v2.5.3 is a maintenance release with the following fixes and enhancements:

Fixes

• Fix an assert being triggered during push telemetry call when no metrics matched on the client side.
• Minor enhancement to JSONDeserializer to retrieve schema from Schema Registry if not provided

confluent-kafka-python is based on librdkafka v2.5.3, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

[!WARNING] This version has introduced a regression in which an assert is triggered during PushTelemetry call. This happens when no metric is matched on the client side among those requested by broker subscription.

You won't face any problem if:

• Broker doesn't support KIP-714.
• KIP-714 feature is disabled on the broker side.
• KIP-714 feature is disabled on the client side. This is enabled by default. Set configuration enable.metrics.push to false.
• If KIP-714 is enabled on the broker side and there is no subscription configured there.
• If KIP-714 is enabled on the broker side with subscriptions that match the KIP-714 metrics defined on the client.

Having said this, we strongly recommend using v2.5.3 and above to not face this regression at all.

confluent-kafka-python v2.5.0

v2.5.0 is a feature release with the following features, fixes and enhancements:

• KIP-107 Added delete_records API. (#1710)
• Added an example to show the usage of the custom logger with AdminClient. (#1758)
• Improve caching on Schema Registry client. (#1744)
• Removed usage of strcpy to enhance security of the client. (#1745)
• Removed support for centos6 and centos7. (#1776)
• Fixed invalid write in OAUTHBEARER/OIDC extensions copy. (#1745)
• Fixed documentation for default value of operation_timeout and request_timeout in various Admin apis. (#1710)
• Fixed an issue related to import error of TopicCollection and TopicPartitionInfo classes when importing through other module like mypy. (#1764)
• Fixed a segfault when commit or store_offsets consumer method is called incorrectly with errored Message object. (#1754)
• Fixed logger not working when provided as an argument to AdminClient instead of a configuration property. (#1758)
• Fixed some memory leaks related to PyDict_SetItem. (#1710)

confluent-kafka-python is based on librdkafka v2.5.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.4.0

v2.4.0 is a feature release with the following features, fixes and enhancements:

• KIP-848: Added KIP-848 based new consumer group rebalance protocol. The feature is an Early Access: not production ready yet. Please refer detailed doc for more information.
• Fix segfault with describe_topics and flaky connection (@lpsinger, #1692)

confluent-kafka-python is based on librdkafka v2.4.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.3.0

v2.3.0 is a feature release with the following features, fixes and enhancements:

• Add Python 3.12 wheels
• KIP-117: Add support for AdminAPI describe_cluster() and describe_topics(). (@jainruchir, #1635)
• KIP-430: Return authorized operations in Describe Responses. (@jainruchir, #1635)
• KIP-516: Partial support of topic identifiers. Topic identifiers in metadata response are available through the new describe_topics function (#1645).
• KIP-396: completed the implementation with the addition of list_offsets (#1576).
• Add Rack to the Node type, so AdminAPI calls can expose racks for brokers (currently, all Describe Responses) (#1635, @jainruchir).
• Fix the Describe User Scram Credentials for Describe all users or empty users list. Please refer to issue(https://github.com/confluentinc/confluent-kafka-python/issues/1616) for more details (#1630).

confluent-kafka-python is based on librdkafka v2.3.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.2.0

v2.2.0 is a feature release with the following features, fixes and enhancements:

• KIP-339 IncrementalAlterConfigs API (#1517).
• KIP-554: User SASL/SCRAM credentials alteration and description (#1575).
• Added documentation with an example of FIPS compliant communication with Kafka cluster.
• Fixed wrong error code parameter name in KafkaError.

confluent-kafka-python is based on librdkafka v2.2.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.1.1

v2.1.1 is a maintenance release with the following fixes and enhancements:

Fixes

• Added a new ConsumerGroupState UNKNOWN. The typo state UNKOWN is deprecated and will be removed in the next major version.
• Fix some Admin API documentation stating -1 for infinite timeout incorrectly. Request timeout can't be infinite.

confluent-kafka-python is based on librdkafka v2.1.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.1.0

v2.1.0 is a feature release with the following features, fixes and enhancements:

• Added set_sasl_credentials. This new method (on the Producer, Consumer, and AdminClient) allows modifying the stored SASL PLAIN/SCRAM credentials that will be used for subsequent (new) connections to a broker (#1511).
• Wheels for Linux / arm64 (#1496).
• Added support for Default num_partitions in CreateTopics Admin API.
• Added support for password protected private key in CachedSchemaRegistryClient.
• Add reference support in Schema Registry client. (@RickTalken, #1304)
• Migrated travis jobs to Semaphore CI (#1503)
• Added support for schema references. (#1514 and @slominskir #1088)
• KIP-320: add offset leader epoch methods to the TopicPartition and Message classes (#1540).

confluent-kafka-python is based on librdkafka v2.1.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v2.0.2

v2.0.2 is a feature release with the following features, fixes and enhancements:

• Added Python 3.11 wheels.
• KIP-222 Add Consumer Group operations to Admin API.
• KIP-518 Allow listing consumer groups per state.
• KIP-396 Partially implemented: support for AlterConsumerGroupOffsets.
• As result of the above KIPs, added (#1449)
  • list_consumer_groups Admin operation. Supports listing by state.
  • describe_consumer_groups Admin operation. Supports multiple groups.
  • delete_consumer_groups Admin operation. Supports multiple groups.
  • list_consumer_group_offsets Admin operation. Currently, only supports 1 group with multiple partitions. Supports require_stable option.
  • alter_consumer_group_offsets Admin operation. Currently, only supports 1 group with multiple offsets.
• Added normalize.schemas configuration property to Schema Registry client (@rayokota, #1406)
• Added metadata to TopicPartition type and commit() (#1410).
• Added consumer.memberid() for getting member id assigned to the consumer in a consumer group (#1154).
• Implemented nb_bool method for the Producer, so that the default (which uses len) will not be used. This avoids situations where producers with no enqueued items would evaluate to False (@vladz-sternum, #1445).
• Deprecated AvroProducer and AvroConsumer. Use AvroSerializer and AvroDeserializer instead.
• Deprecated list_groups. Use list_consumer_groups and describe_consumer_groups instead.
• Improved Consumer Example to show atleast once semantics.
• Improved Serialization and Deserialization Examples.
• Documentation Improvements.

Upgrade considerations

OpenSSL 3.0.x upgrade in librdkafka requires a major version bump, as some legacy ciphers need to be explicitly configured to continue working, but it is highly recommended NOT to use them. The rest of the API remains backward compatible.

confluent-kafka-python is based on librdkafka v2.0.2, see the librdkafka v2.0.0 release notes and later ones for a complete list of changes, enhancements, fixes and upgrade considerations.

Note: There were no v2.0.0 and v2.0.1 releases.

confluent-kafka-python v1.9.2

v1.9.2 is a maintenance release with the following fixes and enhancements:

• Support for setting principal and SASL extensions in oauth_cb and handle failures (@Manicben, #1402)
• Wheel for macOS M1/arm64
• KIP-140 Admin API ACL fix: When requesting multiple create_acls or delete_acls operations, if the provided ACL bindings or ACL binding filters are not unique, an exception will be thrown immediately rather than later when the responses are read. (#1370).
• KIP-140 Admin API ACL fix: Better documentation of the describe and delete ACLs behavior when using the MATCH resource patter type in a filter. (#1373).
• Avro serialization examples: added a parameter for using a generic or specific Avro schema. (#1381).

confluent-kafka-python is based on librdkafka v1.9.2, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v1.9.0

v1.9.0 is a feature release:

• OAUTHBEARER OIDC support
• KIP-140 Admin API ACL support

Fixes

• The warnings for use.deprecated.format (introduced in v1.8.2) had its logic reversed, which result in warning logs to be emitted when the property was correctly configured, and the log message itself also contained text that had it backwards. The warning is now only emitted when use.deprecated.format is set to the old legacy encoding (True). #1265
• Use str(Schema) rather than Schema.to_json to prevent fastavro from raising exception TypeError: unhashable type: 'mappingproxy'. (@ffissore, #1156, #1197)
• Fix the argument order in the constructor signature for AvroDeserializer/Serializer: the argument order in the constructor signature for AvroDeserializer/Serializer was altered in v1.6.1, but the example is not changed yet. (@DLT1412, #1263)
• Fix the json deserialization errors from _schema_loads for valid primitive declarations. (@dylrich, #989)

confluent-kafka-python is based on librdkafka v1.9.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

confluent-kafka-python v1.8.2

v1.8.2 is a maintenance release with the following fixes and enhancements:

• IMPORTANT: Added mandatory use.deprecated.format to ProtobufSerializer and ProtobufDeserializer. See Upgrade considerations below for more information.
• Python 2.7 binary wheels are no longer provided. Users still on Python 2.7 will need to build confluent-kafka from source and install librdkafka separately, see README.md for build instructions.
• Added use.latest.version and skip.known.types (Protobuf) to the Serializer classes. (Robert Yokota, #1133).
• list_topics() and list_groups() added to AdminClient.
• Added support for headers in the SerializationContext (Laurent Domenech-Cabaud)
• Fix crash in header parsing (Armin Ronacher, #1165)
• Added long package description in setuptools (Bowrna, #1172).
• Documentation fixes by Aviram Hassan and Ryan Slominski.
• Don't raise AttributeError exception when CachedSchemaRegistryClient constructor raises a valid exception.

confluent-kafka-python is based on librdkafka v1.8.2, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Note: There were no v1.8.0 and v1.8.1 releases.

Upgrade considerations

Protobuf serialization format changes

Prior to this version the confluent-kafka-python client had a bug where nested protobuf schemas indexes were incorrectly serialized, causing incompatibility with other Schema-Registry protobuf consumers and producers.

This has now been fixed, but since the old defect serialization and the new correct serialization are mutually incompatible the user of confluent-kafka-python will need to make an explicit choice which serialization format to use during a transitory phase while old producers and consumers are upgraded.

The ProtobufSerializer and ProtobufDeserializer constructors now both take a (for the time being) configuration dictionary that requires the use.deprecated.format configuration property to be explicitly set.

Producers should be upgraded first and as long as there are old (<=v1.7.0) Python consumers reading from topics being produced to, the new (>=v1.8.2) Python producer must be configured with use.deprecated.format set to True.

When all existing messages in the topic have been consumed by older consumers the consumers should be upgraded and both new producers and the new consumers must set use.deprecated.format to False.

The requirement to explicitly set use.deprecated.format will be removed in a future version and the setting will then default to False (new format).

v1.7.0 is a maintenance release with the following fixes and enhancements:

• Add error_cb to confluent_cloud.py example (#1096).
• Clarify that doc output varies based on method (@slominskir, #1098).
• Docs say Schema when they mean SchemaReference (@slominskir, #1092).
• Add documentation for NewTopic and NewPartitions (#1101).

confluent-kafka-python is based on librdkafka v1.7.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

v1.6.1

v1.6.1 is a feature release:

• KIP-429 - Incremental consumer rebalancing support.
• OAUTHBEARER support.

Fixes

• Add return_record_name=True to AvroDeserializer (@slominskir, #1028)
• Fix deprecated schema.Parse call (@casperlehmann, #1006).
• Make reader schema optional in AvroDeserializer (@97nitt, #1000).
• Add **kwargs to legacy AvroProducer and AvroConsumer constructors to support all Consumer and Producer base class constructor arguments, such as logger (@venthur, #699).
• Add bool for permanent schema delete (@slominskir, #1029).
• The avro package is no longer required for Schema-Registry support (@jaysonsantos, #950).
• Only write to schema cache once, improving performance (@fimmtiu, #724).
• Improve Schema-Registry error reporting (@jacopofar, #673).
• producer.flush() could return a non-zero value without hitting the specified timeout.

confluent-kafka-python is based on librdkafka v1.6.1, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

v1.6.0

v1.6.0 is a feature release with the following features, fixes and enhancements:

• Bundles librdkafka v1.6.0 which adds support for Incremental rebalancing, Sticky producer partitioning, Transactional producer scalabilty improvements, and much much more. See link to release notes below.
• Rename asyncio.py example to avoid circular import (#945)
• The Linux wheels are now built with manylinux2010 (rather than manylinux1) since OpenSSL v1.1.1 no longer builds on CentOS 5. Older Linux distros may thus no longer be supported, such as CentOS 5.
• The in-wheel OpenSSL version has been updated to 1.1.1i.
• Added Message.latency() to retrieve the per-message produce latency.
• Added trove classifiers.
• Consumer destructor will no longer trigger consumer_close(), consumer.close() must now be explicitly called if the application wants to leave the consumer group properly and commit final offsets.
• Fix PY_SSIZE_T_CLEAN warning
• Move confluent_kafka/ to src/ to avoid pytest/tox picking up the local dir
• Added producer.purge() to purge messages in-queue/flight (@peteryin21, #548)
• Added AdminClient.list_groups() API (@messense, #948)
• Rename asyncio.py example to avoid circular import (#945)

confluent-kafka-python is based on librdkafka v1.6.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Confluent's Python client for Apache Kafka

v1.5.0

v1.5.0 is a maintenance release with the following fixes and enhancements:

• Bundles librdkafka v1.5.0 - see release notes for all enhancements and fixes.
• Documentation fixes
• Dockerfile examples
• List offsets example

confluent-kafka-python is based on librdkafka v1.5.0, see the librdkafka release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Confluent's Python client for Apache Kafka

v1.4.2 is a maintenance release with the following fixes and enhancements:

• Fix produce/consume hang after partition goes away and comes back, such as when a topic is deleted and re-created (regression in v1.3.0).
• Consumer: Reset the stored offset when partitions are un-assign()ed. This fixes the case where a manual offset-less commit() or the auto-committer would commit a stored offset from a previous assignment before a new message was consumed by the application
• Seed random number generator used for random broker selection.
• Multiple calls to Consumer.close will not raise RunTimeError (@mkmoisen , #678)
• Module lookup failures raise ImportError (@overstre #786)
• Fix SchemaRegistryClient basic auth configuration (@blown302 #853)
• Don't send empty credentials to SR in Authorization Header (#863)
• miscellaneous test cleanup and enhancements (@nicht #843 ,#863)

confluent-kafka-python is based on librdkafka v1.4.2, see the librdkafka v1.4.2 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Confluent's Python client for Apache Kafka

v1.4.0 is a feature release:

• KIP-98: Transactional Producer API
• KIP-345: Static consumer group membership (by @rnpridgeon)
• KIP-511: Report client software name and version to broker
• Generic Serde API (experimental)
• New AvroSerializer and AvroDeserializer implementations including configurable subject name strategies.
• JSON Schema support (For Schema Registry)
• Protobuf support (For Schema Registry)

confluent-kafka-python is based on librdkafka v1.4.0, see the librdkafka v1.4.0 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

Transactional Producer API

Release v1.4.0 for confluent-kafka-python adds complete Exactly-Once-Semantics (EOS) functionality, supporting the idempotent producer (since v1.0.0), a transaction-aware consumer (since v1.2.0) and full producer transaction support (v1.4.0).

This enables developers to create Exactly-Once applications with Apache Kafka.

See the Transactions in Apache Kafka page for an introduction and check the transactions example.

Generic Serializer API

Release v1.4.0 introduces a new, experimental, API which adds serialization capabilities to Kafka Producer and Consumer. This feature provides the ability to configure Producer/Consumer key and value serializers/deserializers independently. Previously all serialization must be handled prior to calling Producer.produce and after Consumer.poll.

This release ships with 3 built-in, Java compatible, standard serializer and deserializer classes:

* The StringSerializer codec is configurable and supports any one of Python's standard encodings. If left unspecified 'UTF-8' will be used.

Additional serialization implementations are possible through the extension of the Serializer and Deserializer base classes.

See avro_producer.py and avro_consumer.py for example usage.

Avro, Protobuf and JSON Schema Serializers

Release v1.4.0 for confluent-kafka-python adds support for two new Schema Registry serialization formats with its Generic Serialization API; JSON and Protobuf. A new set of Avro Serialization classes have also been added to conform to the new API.

Security fixes

Two security issues have been identified in the SASL SCRAM protocol handler:

• The client nonce, which is expected to be a random string, was a static string.
• If sasl.username and sasl.password contained characters that needed escaping, a buffer overflow and heap corruption would occur. This was protected, but too late, by an assertion.

Both of these issues are fixed in this release.

Enhancements

• Bump OpenSSL to v1.0.2u
• Bump monitoring-interceptors to v0.11.3

Fixes

General:

• Remove unused variable from README example (@qzyse2017, #691)
• Add delivery report to Avro example (#742)
• Update asyncio example companion blog URL (@filippovitale, #760)

Schema Registry/Avro:

• Trim trailing / from Schema Registry base URL (@IvanProdaiko94 , #749)
• Make automatic Schema registration optional (@dpfeif, #718)
• Bump Apache Avro to 1.9.2[.1] (@RyanSkraba, #779)
• Correct the SchemaRegistry authentication for SASL_INHERIT (@abij, #733)

Also see the librdkafka v1.4.0 release notes for fixes to the underlying client implementation.

Confluent's Python client for Apache Kafka

confluent-kafka-python is based on librdkafka v1.3.0, see the librdkafka v1.3.0 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

This is a feature release adding support for KIP-392 Fetch from follower, allowing a consumer to fetch messages from the closest replica to increase throughput and reduce cost.

Features

• KIP-392 - Fetch messages from closest replica / follower (by @mhowlett)
• Python 3.8 binary wheel support for OSX and Linux. Windows Python 3.8 binary wheels are not currently available.

Enhancements

• New example using python3 and asyncio (by @mhowlett)
• Add warnings for inconsistent security configuration.
• Optimizations to hdr histogram (stats) rollover.
• Print compression type per message-set when debug=msg
• Various doc fixes, updates and enhancements (@edenhill , @mhowlett)

Fixes

• Fix crash when new topic is not created. (@Mostafa Razavi,#725)
• Fix stringer/repr for SerializerError class(@ferozed, #675)
• Fix consumer_lag in stats when consuming from broker versions <0.11.0.0 (regression in librdkafka v1.2.0).
• Properly handle new Kafka-framed SASL GSSAPI frame semantics on Windows (#2542). 
This bug was introduced in v1.2.0 and broke GSSAPI authentication on Windows.
• Fix msgq (re)insertion code to avoid O(N^2) insert sort operations on retry (#2508).
 The msgq insert code now properly handles interleaved and overlapping message range inserts, which may occur during Producer retries for
 high-throughput applications.
• Fix producer insert msgq regression in v1.2.1 (#2450).
• Upgrade builtin lz4 to 1.9.2 (CVE-2019-17543, #2598).
• Don't trigger error when broker hostname changes (#2591).
• Less strict message.max.bytes check for individual messages (#993).
• Don't call timespec_get() on OSX (since it was removed in recent XCode) by @maparent .
• LZ4 is available from ProduceRequest 0, not 3 (fixes assert in #2480).
• Address 12 code issues identified by Coverity static code analysis.

Confluent's Python client for Apache Kafka

confluent-kafka-python is based on librdkafka v1.2.0, see the librdkafka v1.2.0 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

• Transaction aware consumer (isolation.level=read_committed) implemented by @mhowlett.
• Sub-millisecond buffering (linger.ms) on the producer.
• Improved authentication errors (KIP-152)

Consumer-side transaction support

This release adds consumer-side support for transactions. In previous releases, the consumer always delivered all messages to the application, even those in aborted or not yet committed transactions. In this release, the consumer will by default skip messages in aborted transactions. This is controlled through the new isolation.level configuration property which defaults to read_committed (only read committed messages, filter out aborted and not-yet committed transactions), to consume all messages, including for aborted transactions, you may set this property to read_uncommitted to get the behaviour of previous releases. For consumers in read_committed mode, the end of a partition is now defined to be the offset of the last message of a successfully committed transaction (referred to as the 'Last Stable Offset'). For non-transactional messages there is no change from previous releases, they will always be read, but a consumer will not advance into a not yet committed transaction on the partition.

Upgrade considerations

• linger.ms default was changed from 0 to 0.5 ms to promote some level of batching even with default settings.

New configuration properties

• Consumer property isolation.level=read_committed ensures the consumer will only read messages from successfully committed producer transactions. Default is read_committed. To get the previous behaviour, set the property to read_uncommitted, which will read all messages produced to a topic, regardless if the message was part of an aborted or not yet committed transaction.

Enhancements

• Cache FastAvro schema for improved Avro Serialization/Deserialization (@BBM89, #627)
• Protocol decoding optimization, increasing consume performance.
• Add CachedSchemaRegistry docs (@lowercase24 , #495)

Fixes

General:

• Rate limit IO-based queue wakeups to linger.ms, this reduces CPU load and lock contention for high throughput producer applications. (#2509)
• SSL: Use only hostname (not port) when valid broker hostname (by Hunter Jacksson)
• SSL: Ignore OpenSSL cert verification results if enable.ssl.certificate.verification=false (@salisbury-espinosa)
• SASL Kerberos/GSSAPI: don't treat kinit ECHILD errors as errors (@hannip)
• Refresh broker list metadata even if no topics to refresh (#2476)
• Correct AdminClient doc (@lowercase24, #653)
• Update Avro example to be compliant with csh (@andreyferriyan , #668)
• Correct Avro example typo (@AkhilGNair, #598)

Consumer:

• Make pause|resume() synchronous, ensuring that a subsequent poll() will not return messages for the paused partitions.
• Consumer doc fixes (@hrchu , #646, #648)

Producer:

• Fix message timeout handling for leader-less partitions.
• message.timeout.ms=0 is now accepted even if linger.ms > 0 (by Jeff Snyder)

Confluent's Python client for Apache Kafka

confluent-kafka-python is based on librdkafka v1.1.0, see the librdkafka v1.1.0 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

• In-memory SSL certificates (PEM, DER, PKCS#12) support (by @noahdav at Microsoft)
• Use Windows Root/CA SSL Certificate Store (by @noahdav at Microsoft)
• ssl.endpoint.identification.algorithm=https (off by default) to validate the broker hostname matches the certificate. Requires OpenSSL >= 1.0.2(included with Wheel installations))
• Improved GSSAPI/Kerberos ticket refresh
• Confluent monitoring interceptor package bumped to v0.11.1 (#634)

Upgrade considerations

• Windows SSL users will no longer need to specify a CA certificate file/directory (ssl.ca.location), librdkafka will load the CA certs by default from the Windows Root Certificate Store.
• SSL peer (broker) certificate verification is now enabled by default (disable with enable.ssl.certificate.verification=false)
• %{broker.name} is no longer supported in sasl.kerberos.kinit.cmd since kinit refresh is no longer executed per broker, but per client instance.

SSL

New configuration properties:

• ssl.key.pem - client's private key as a string in PEM format
• ssl.certificate.pem - client's public key as a string in PEM format
• enable.ssl.certificate.verification - enable(default)/disable OpenSSL's builtin broker certificate verification.
• enable.ssl.endpoint.identification.algorithm - to verify the broker's hostname with its certificate (disabled by default).
• Add new rd_kafka_conf_set_ssl_cert() to pass PKCS#12, DER or PEM certs in (binary) memory form to the configuration object.
• The private key data is now securely cleared from memory after last use.

Enhancements

• Bump message.timeout.ms max value from 15 minutes to 24 days (@sarkanyi, workaround for #2015)

Fixes

• SASL GSSAPI/Kerberos: Don't run kinit refresh for each broker, just per client instance.
• SASL GSSAPI/Kerberos: Changed sasl.kerberos.kinit.cmd to first attempt ticket refresh, then acquire.
• SASL: Proper locking on broker name acquisition.
• Consumer: max.poll.interval.ms now correctly handles blocking poll calls, allowing a longer poll timeout than the max poll interval.
• configure: Fix libzstd static lib detection
• PyTest pinned to latest version supporting python 2 (#634)

Confluent's Python client for Apache Kafka

confluent-kafka-python is based on librdkafka v1.0.1, see the librdkafka v1.0.1 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

v1.0.1 is a maintenance release with the following fixes:

• Fix consumer stall when broker connection goes down (issue #2266 introduced in v1.0.0)
• Fix AdminAPI memory leak when broker does not support request (@souradeep100, #2314)
• SR client: Don't disable cert verification if no ssl.ca.location set (#578)
• Treat ECONNRESET as standard Disconnects (#2291)
• OpenSSL version bump to 1.0.2s
• Update/fix protocol error response codes (@benesch)
• Update Consumer get_watermark_offsets docstring (@hrchu, #572)
• Update Consumer subscribe docstring to include on_assign and on_revoke args (@hrchu, #571)
• Update delivery report string formatting (@hrchu, #575)
• Update logging configuration code example document (@soxofaan , #579)
• Implement environment markers to fix poetry (@fishman, #583)

Confluent's Python client for Apache Kafka v1.0.0

confluent-kafka-python is based on librdkafka v1.0.0, see the librdkafka v1.0.0 release notes for a complete list of changes, enhancements and fixes and upgrade considerations.

v1.0.0 is a major feature release:

• Idempotent producer - guaranteed ordering, exactly-once producing) support.
• Sparse/on-demand connections - connections are no longer maintained to all brokers in the cluster.
• KIP-62 - max.poll.interval.ms support in the Consumer.

This release also changes configuration defaults and deprecates a set of configuration properties, make sure to read the Upgrade considerations section below.

Upgrade considerations (IMPORTANT)

Configuration default changes

The following configuration properties have changed default values, which may require application changes:

• acks(alias request.required.acks) now defaults to all; wait for all in-sync replica brokers to ack. The previous default, 1 , only waited for an ack from the partition leader. This change places a greater emphasis on durability at a slight cost to latency. It is not recommended that you lower this value unless latency takes a higher precedence than data durability in your application.

• broker.version.fallback now to defaults to 0.10, previously 0.9. broker.version.fallback.ms now defaults to 0. Users on Apache Kafka <0.10 must set api.version.request=false and broker.version.fallback=.. to their broker version. For users >=0.10 there is no longer any need to specify any of these properties.

• enable.partition.eof now defaults to false. KafkaError._PARTITION_EOF was previously emitted by default to signify the consumer has reached the end of a partition. Applications which rely on this behavior must now explicitly set enable.partition.eof=true if this behavior is required. This change simplifies the more common case where consumer applications consume in an endless loop.

group.id is now required for Python consumers.

Deprecated configuration properties

The following configuration properties have been deprecated. Use of any deprecated configuration property will result in a warning when the client instance is created. The deprecated configuration properties will be removed in a future release.

librdkafka:

• offset.store.method=file is deprecated.
• offset.store.path is deprecated.
• offset.store.sync.interval.ms is deprecated.
• produce.offset.report is no longer used. Offsets are always reported.
• queuing.strategy was an experimental property that is now deprecated.
• reconnect.backoff.jitter.ms is no longer used, see reconnect.backoff.ms and reconnect.backoff.max.ms.
• socket.blocking.max.ms is no longer used.
• topic.metadata.refresh.fast.cnt is no longer used.

confluent_kafka:

• default.topic.config is deprecated.
• `CachedSchemaRegistryClient: url: was str, now conf dict with all application config properties

Idempotent Producer

This release adds support for Idempotent Producer, providing exactly-once producing and guaranteed ordering of messages.

Enabling idempotence is as simple as setting the enable.idempotence configuration property to true.

There are no required application changes, but it is recommended to add support for the newly introduced fatal errors that will be triggered when the idempotent producer encounters an unrecoverable error that would break the ordering or duplication guarantees.

See Idempotent Producer in the manual and the Exactly once semantics blog post for more information.

Sparse connections

In previous releases librdkafka would maintain open connections to all brokers in the cluster and the bootstrap servers.

With this release librdkafka now connects to a single bootstrap server to retrieve the full broker list, and then connects to the brokers it needs to communicate with: partition leaders, group coordinators, etc.

For large scale deployments this greatly reduces the number of connections between clients and brokers, and avoids the repeated idle connection closes for unused connections.

Sparse connections is on by default (recommended setting), the old behavior of connecting to all brokers in the cluster can be re-enabled by setting enable.sparse.connections=false.

See Sparse connections in the manual for more information.

Original issue librdkafka #825.

KIP-62 - max.poll.interval.ms is enforced

This release adds support for max.poll.interval.ms (KIP-62), which requires the application to call consumer.poll() at least every max.poll.interval.ms. Failure to do so will make the consumer automatically leave the group, causing a group rebalance, and not rejoin the group until the application has called ..poll() again, triggering yet another group rebalance. max.poll.interval.ms is set to 5 minutes by default.

Enhancements

• OpenSSL version bumped to 1.0.2r
• AvroProducer now supports encoding with fastavro (#492)
• Simplify CachedSchemaRegistryClient configuration with configuration dict for application configs
• Add Delete Schema support to CachedSchemaRegistryClient
• CachedSchemaRegistryClient now supports HTTP Basic Auth (#440)
• MessageSerializer now supports specifying reader schema (#470)

Fixes

• Fix crash when calling Consumer.consume without setting group.id(now required)
• CachedSchemaRegistryClient handles get_compatibility properly

Build/installation/tooling

• Integration tests moved to docker-compose to aid in cluster set-up/tear-down
• Runner script ./tests/run.sh added to simplify unit and integration test execution

See librdkafka v0.11.6 release notes for enhancements and fixes in librdkafka.

New Features

• Confluent Monitoring Interceptors are now included with Linux and OSX binary wheel distributions. (#464)
• Experimental binary wheel distributions for Windows environments. (#451)

Enhancements

• OpenSSL version bump to 1.0.2p. (#437)
• Topic configurations have been moved into the global configuration dictionary to simplify configuration. The property default.topic.configuration has been deprecated and will be removed in 1.0, but still has precedence to topic configuration specified in the global configuration dictionary. (#446)

Fixes

• Handle debug configuration property prior to plugin.library.paths for enhanced debugging. (#464)
• Fix memory leak in message headers. (#458)
• Safely release handler resources. (#434, @coldeasy)

Admin Client support

v0.11.5 is a feature release that adds support for the Kafka Admin API (KIP-4).

Admin API

This release adds support for the Admin API, enabling applications and users to perform administrative Kafka tasks programmatically:

• Create topics - specifying partition count, replication factor and topic configuration.
• Delete topics - delete topics in cluster.
• Create partitions - extend a topic with additional partitions.
• Alter configuration - set, modify or delete configuration for any Kafka resource (topic, broker, ..).
• Describe configuration - view configuration for any Kafka resource.

The API closely follows the Java Admin API:

def example_create_topics(a, topics):
    new_topics = [NewTopic(topic, num_partitions=3, replication_factor=1) for topic in topics]
    # Call create_topics to asynchronously create topics
    fs = a.create_topics(new_topics)

    # Wait for operation to finish.
    for topic, f in fs.items():
        try:
            f.result()  # The result itself is None
            print("Topic {} created".format(topic))
        except Exception as e:
            print("Failed to create topic {}: {}".format(topic, e))

Additional examples can be found in examples/adminapi

Enhancements

• Schema Registry HTTPS support with TLS client auth added (#90)
• Metadata API list_topics() added (#161, @tbsaunde, @stephan-hof)
• Expose librdkafka built-in partitioner options directly (#396)
• Callback based throttle event handling; throttle_cb (#237) (#377)
• Added Unicode support for header values (#382)
• OpenSSL version bump to 1.0.2o (#410)
• Avro documentation added to the docs (#382)
• Python 3.7 support (#382)
• Allow passing headers as both list(tuples) and dict() (#355)
• Support for legacy setuptool's install_requires (#399)

Fixes

• Release GIL before making blocking calls (#412)
• Prevent application config dict mutation (#412)
• Intercept plugin configurations to ensure proper ordering (#404)
• test_compatibility() should return False not None would return None when unable to check compatibility (#372, @Enether)
• Schema Registry client returns false when unable to check compatibility(#372, @Enether)
• Fix invocation of SchemaParseException (#376)
• Fix call ordering to avoid callback crash on implicit close (#265)
• Fix memory leaks in generic client setters (#382)
• Fix AvroProducer/AvroConsumer key/value identity check (#342)
• Correct Producer.produce documentation to use correct time unit of seconds (#384) (#385)
• Fix KafkaError refcounting which could lead to memory leaks (#382)