NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
crates.io
crates.io, #4269 most downloaded on crates.io
Encode/decode Avro, Protobuf and JSON Schema data from/to Kafka using the Confluent Schema Registry
Latest stable
6.0.0
Last release 1 months ago
22 Oct 2026
Ships fairly regularly
a new release about every 2 months
Most releases are documented
notes for 23 of 28 stable releases
Nothing withdrawn
no release was ever pulled
8 years old
28 releases · first in 2018
Drag across the ruler to compare, or tap a release to open it.
…decodes with prost-reflect. See #206. This is a breaking change for anyone using the proto_decoder feature:
The details are below, but in short:
AvroDecoder<'a> is now
AvroDecoder, likewise for the other async types.SuppliedSchema or SuppliedReference with new and the with_ methods instead of a
struct expression, and add a wildcard arm when matching on SchemaType or
SubjectNameStrategy, as most public types are now #[non_exhaustive].proto_decoder: values are prost_reflect types instead of protofish ones, use
schema_registry_converter::prost_reflect to get the matching version. Bytes that aren't a
valid message are now an error.json: the blocking JsonDecoder and JsonEncoder take &self, a decoded value has an owned
Arc<JsonSchema>, validate it with decoder.validate(&result.schema, &result.value). The
blocking JsonEncoder::validate returns Result<(), SRCError>. Validation error messages
changed.proto_raw: a schema that isn't valid protobuf is now an error, and proto_resolver is no
longer public.SRCError::retryable_with_cause and non_retryable_with_cause take an error, or a &str or
String, instead of anything that's Display.easy feature no longer pulls in tokio, add it yourself if you relied on that. Optional
dependencies are no longer features of their own, so e.g. features = ["url"] or ["tokio"]
doesn't work anymore; futures still is a feature, enabling the async implementations.The crate now declares its minimum supported Rust version, rust-version = "1.88", the highest
version its dependencies require (apache-avro 0.22 needs 1.88). With Cargo's Rust-version aware
resolver (the default for edition 2024, or resolver = "3"), a project gets dependency versions
that still support its toolchain, or a clear error when the toolchain is older than 1.88.
The generic ProtoDecoder (both async and blocking, and EasyProtoDecoder) no longer uses
protofish, but compiles the schemas with protox and decodes with prost-reflect. See #206.
This is a breaking change for anyone using the proto_decoder feature:
decode and decode_with_header_id return a prost_reflect::Value, with the decoded
message as Value::Message(DynamicMessage) (no longer boxed). As before, a null payload gives
empty Value::Bytes, and bytes without the schema registry prefix are returned as-is as
Value::Bytes.DecodeResultWithContext.value is a prost_reflect::DynamicMessage, and its full_name is
a String.DecodeContext holds the compiled root schema as file_descriptor, a
prost_reflect::FileDescriptor (with everything it imports in its parent_pool()), instead of
the protofish context and the resolver.DecodeResultWithContext.context is now an Arc<DecodeContext>, as it already was
for the blocking decoder.DecodeContext.registered_schema is renamed to schema, matching the async
decoder and the proto_raw contexts.prost_reflect is re-exported as schema_registry_converter::prost_reflect, so the decoded
values can be used without adding a dependency on a matching version.Behaviour changes:
SRCError. Previously protofish returned a message containing whatever it could make of it.map fields are supported, enums can be looked up by name,
repeated values of a non-repeated field resolve to the last one, and unknown fields are kept.
With prost-reflect's serde feature a DynamicMessage serializes to the canonical protobuf
JSON mapping.map and group fields are skipped, as confluent does.remove_errors_from_cache. See #175.google/protobuf/* types now come with protox, the google/type/* common
types are still bundled. Imports are resolved by the name a reference is registered with.The logos-based scan of the schema text, which mapped message indexes to message names and
back, is replaced by parsing the schema with protox-parse, for proto_raw as well as for the
generic decoder. The scan miscounted messages after a brace in a comment (e.g. // a {b}); both
now share one implementation of confluent's index rules, skipping the messages generated for
map and group fields. For proto_raw this means:
proto_resolver module is no longer public, as MessageResolver and IndexResolver were
only meant for the crate's own encoders and decoders.SRCError ("Could not parse protobuf schema") when it's first used, instead of
being scanned leniently. Imports still aren't resolved, so proto_raw keeps not fetching
references.logos is no longer a direct dependency; proto_raw now depends on protox-parse and
prost-types.Features and dependencies:
easy feature no longer enables tokio, which none of the Easy* wrappers used.dep:, so they're
no longer features themselves, e.g. jsonschema, url, prost-reflect or tokio. futures
remains a feature, as it enables the async implementations.BytesResult, get_bytes_result and invalid_bytes_error in schema_registry_common are
hidden from the documentation, as they're only used by the decoders. They're still public.Other breaking changes in this release, so later additions don't need another major version:
#[non_exhaustive]:
SchemaType, SubjectNameStrategy, SuppliedSchema, SuppliedReference, RegisteredSchema,
RegisteredReference, RawRegisteredSchema, Metadata, RawError, SchemaIdHeader,
AvroSchema, the DecodeResult*, DecodeContext and RawDecodeResult structs, JsonSchema
and SRCError. Matching on the enums needs a wildcard arm, and the structs can't be created
with a struct expression outside this crate. Create a SuppliedSchema or SuppliedReference
with the new new function, and the with_references, with_properties and with_tags
methods; their fields stay public.AvroDecoder<'a> is
now AvroDecoder. They already owned everything they cache, and are Send + Sync + 'static.SRCError keeps the error that caused it, available via std::error::Error::source, so it can
be downcast, e.g. to a reqwest::Error, or to a RawError (which now implements Error) to
read the schema registry's error_code. cause still holds its message. The
retryable_with_cause and non_retryable_with_cause functions now take the cause as
Into<Box<dyn Error + Send + Sync>> instead of Display; any error type, or a &str or
String, can still be passed. PartialEq ignores the source.DecodeResult is now the same as the async one: the schema is an owned
Arc<JsonSchema> instead of a ScopedSchema borrowing from the decoder, so the blocking
JsonDecoder methods take &self instead of &mut self, like the other decoders. The async
DecodeResult also holds an Arc<JsonSchema>, instead of a clone of the schema for every
decoded message. Validate a decoded value with decoder.validate(&result.schema, &result.value), on the blocking and async JsonDecoder (and EasyJsonDecoder), which keeps
the compiled schemas, so validating against the same schema again doesn't compile it again.
There's also a free validate function in both blocking::json and async_impl::json, which
now takes the schema by reference; it compiles the schema on every call. The blocking decoder
also gains get_schema, and the blocking module get_json_schema, matching the async ones.JsonEncoder now takes &self instead of &mut self, and its
validate takes the value and the subject name strategy, returning Result<(), SRCError>
instead of valico's ValidationState.jsonschema instead of valico, which is no longer maintained.
It supports drafts 4, 6, 7, 2019-09 and 2020-12, using the $schema of a schema, and draft 7
when a schema doesn't declare one, like before. Compiled schemas are shared between threads,
so validations no longer wait on each other. It's also faster: in the benchmark, validating a
decoded value with the cached schema went from about 1.9µs to 0.3µs, and encoding, which
validates, from about 2.3µs to 0.35µs. Behaviour changes:
"Foo" is not of type "object" instead of
WrongType { path: "", detail: "The value must be object" }, with the location of the error
in the value when it isn't the root.$ref that can't be resolved from the schema's references now fails to compile, with
"error compiling schema" and the missing url as the cause, instead of the value being "not
valid because of missing references". Nothing is retrieved over the network or from files.$id is now the base for its own relative references in
the async encoder and decoder too, as the blocking ones already did.New in this release:
ProtoEncoder (async, blocking and EasyProtoEncoder), behind the new proto_encoder
feature, for protobuf messages only known at runtime. It encodes a DynamicMessage with
encode, or JSON in the canonical protobuf JSON mapping with encode_json, each also with a
_with_header_id variant. get_message_descriptor gives the descriptor of a message in the
registered schema, to create a DynamicMessage with. Like the decoder, it compiles the schema
together with the schemas it references, finds the message index by the full name of the
message, and fails when the registered schema isn't a protobuf schema.AvroDecoder gains
decode_with_schema_with_header_id and the blocking ProtoDecoder
decode_with_context_and_header_id, like the async ones, and the Easy* wrappers gain all the
_with_header_id methods of the codecs they wrap.Your coding agent can read these notes before it upgrades. Set up the MCP server →
feat: fetch a typed schema by ID on decoders by @michaelbeaumont in https://github.com/gklijs/schema_registry_converter/pull/199
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v5.0.0...v5.1.0
New async APIs for getting at the schemas themselves, not just the decoded values:
Expose a decoder's own schema lookup: AvroDecoder::get_schema(id) and
JsonDecoder::get_schema(id) (async) are now public, returning the fully self-contained,
reference-resolved AvroSchema/JsonSchema for an id, using (and populating) that decoder's
cache.
Add fetching a schema by an explicit subject and version, independent of any
SubjectNameStrategy: async_impl::schema_registry::get_schema_by_subject_and_version(sr_settings, subject, version) (None for "latest") returns the RegisteredSchema, which can be resolved
into a self-contained schema with the new async_impl::avro::get_avro_schema or
async_impl::json::get_json_schema.
Add AvroDecoder::decode_with_schema_with_header_id (async), the header-aware counterpart of
decode_with_schema, returning the full DecodeResultWithSchema for a message that may carry
its schema id/guid in a Kafka header. See #139.
ProtoDecoder (async) now returns the root RegisteredSchema (id, version, properties, tags,
...) as a new schema field on DecodeContext, and gains decode_with_context_and_header_id,
the header-aware counterpart of decode_with_context. Note: DecodeContext gaining a public
field is breaking if you construct one via struct-literal syntax, or destructure one exhaustively
without ..; normal usage, which only reads a DecodeContext returned by the decoder, is
unaffected.
Add allium spec to the project. Issues found were created in gh issues. by @gklijs in https://github.com/gklijs/schema_registry_converter/pull/179
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v4.10.0...v5.0.0
Two breaking changes bump this to a major version:
BytesResult (in schema_registry_common) now borrows the payload instead of owning it --
Invalid(Vec<u8>) / Valid(u32, Vec<u8>) are now Invalid(&'a [u8]) / Valid(u32, &'a [u8]),
and get_bytes_result returns BytesResult<'_> borrowing from its input, instead of copying the
whole payload into a fresh Vec on every call. This is on the hot path of every decode, for
every format (Avro, JSON, protobuf), so it matters more the larger your messages are -- see
#190 for measurements.
This only affects code that calls get_bytes_result/matches on BytesResult directly. The
built-in AvroDecoder, JsonDecoder, ProtoDecoder and ProtoRawDecoder (blocking and async)
are not affected -- their decode/decode_with_schema/etc. signatures haven't changed, and
existing code calling those is unaffected. If you do call get_bytes_result directly and need to
keep the bytes beyond the immediate match (store them, return them, carry them across an
.await), copy them out explicitly where you used to get an owned Vec for free:
match get_bytes_result(bytes) {
BytesResult::Valid(id, bytes) => {
let owned: Vec<u8> = bytes.to_vec();
// ... use `owned` instead of `bytes` wherever it needs to outlive this match
}
BytesResult::Invalid(bytes) => {
let owned: Vec<u8> = bytes.to_vec();
// ...
}
BytesResult::Null => {}
}
Second, apache-avro moved from ^0.21 to ^0.22, which turned Name::name/Name::namespace
(apache_avro::schema::Name, returned in DecodeResult.name/DecodeResultWithSchema.name) from
public fields into methods returning borrows (name() -> &str, namespace() -> Option<&str>)
instead of owned Strings. If you access those on a Name obtained through this crate, it's not
just adding parentheses -- .unwrap() on the Option<Name> now needs .as_ref() first, or the
borrow from .name()/.namespace() outlives the temporary it came from:
// before (apache-avro 0.21, schema_registry_converter <5.0.0)
let name: String = decode_result.name.unwrap().name;
let namespace: Option<String> = decode_result.name.unwrap().namespace;
// after (apache-avro 0.22, schema_registry_converter >=5.0.0)
let name: &str = decode_result.name.as_ref().unwrap().name();
let namespace: Option<&str> = decode_result.name.as_ref().unwrap().namespace();
Not breaking, but worth knowing about: Avro encode/decode now reuse a cached, already-resolved schema instead of re-resolving it from scratch on every single call -- internal, not observable in the API, but measurably faster, more so the more named types (records/enums/fixed) your schema has. See #190.
Add support for plugging in a reqwest_middleware::Middleware (async only, behind the new
middleware feature) via SrSettingsBuilder, for short-lived tokens that need periodic
refreshing (e.g. GCP workload identity federation) without recreating SrSettings -- and losing
its schema cache -- every time a token expires. See #144 and the "Authentication middleware /
token refresh" section of the README.
Fix the blocking protobuf decoder (ProtoDecoder) missing well-known/common-type imports (e.g.
google/protobuf/timestamp.proto) when they're imported by a referenced schema rather than
the top-level one -- the async decoder already handled this correctly. See #178.
Fix the blocking JSON decoder (JsonDecoder) skipping a reference that comes after one already
present in its (persistent, cross-call) scope, even when the later reference is a different
schema in the same reference list -- it was short-circuiting on the first "already resolved"
match instead of checking each reference independently. See #177.
Fix a retriable error (e.g. a transient 503 from the registry) getting stuck in a decoder's or
encoder's cache -- every subsequent call for that schema id/guid kept replaying the same stale
failure instead of retrying, until remove_errors_from_cache() was called by hand -- and fix
SRCError.cached being applied inconsistently between the blocking and async clients along the
way. Retriable errors are no longer cached by either client. See #171 and #175.
Fix a panic when decoding a 5-byte (or otherwise malformed/truncated) protobuf-framed payload
with ProtoDecoder/ProtoRawDecoder (blocking and async); such payloads now yield an SRCError
instead. As part of this, proto_resolver::to_index_and_data is now fallible: it returns
Result<(Vec<i32>, Vec<u8>), SRCError> instead of (Vec<i32>, Vec<u8>), a breaking change for
any caller using that function directly. Fixes #176.
Fix a follow-up issue in the same area: a brace character inside a string literal in a proto
schema (e.g. a field/option default value) could desync MessageResolver/IndexResolver's
index bookkeeping from the real message nesting, since its lexer didn't previously recognize
string literals and treated every {/} as message nesting. String literals are now skipped as
a whole when scanning for braces.
Fix the blocking client silently dropping properties/tags metadata when registering a schema
or reference (post_schema, post_reference) or checking compatibility --
blocking::schema_registry::get_body never received the metadata support
async_impl::schema_registry::get_body already had. Both clients now build the same metadata
block. See #174.
Add support for carrying the schema id/guid in a Kafka header instead of prefixing the payload,
matching Confluent Schema Registry 8.0's HeaderSchemaIdSerializer/DualSchemaIdDeserializer.
Every encoder/decoder (Avro, JSON, protobuf; blocking and async) gains _with_header_id
variants, plain-data types SchemaIdHeader/KEY_SCHEMA_ID_HEADER/VALUE_SCHEMA_ID_HEADER are
added to schema_registry_common, and schema lookup by guid (GET /schemas/guids/{guid}) is
now supported alongside lookup by id. No existing function/method signature changes, so this is
additive for all normal usage -- but RegisteredSchema, RawRegisteredSchema, AvroSchema, and
JsonSchema all gain a new public guid field, which is a breaking change if you construct
one of those via struct-literal syntax, or destructure one exhaustively without ... See #139
and the "Schema id in Kafka headers" section of the README.
AvroEncoder::encode_struct/encode_struct_with_header_id (blocking and async) no longer go
through an untyped intermediate apache_avro::types::Value plus a separate Value::resolve()
pass to make it match the schema -- they now serialize directly against the (still cached)
resolved schema via apache_avro's schema-aware serde serializer. For a schema with several
fields sharing the same named type, .resolve() alone used to dominate encode time; benchmarked
about 7x faster on a schema with 8 such fields, and about 2.7x faster even on a schema with none
(benches/avro_bench.rs's avro_encode_struct_cached_named_refs/avro_encode_struct_cached).
Not breaking for normal usage, but the exact wording of the SRCError returned for a value that
doesn't match its schema has changed (e.g. "Failed to resolve" is now "Could not get Avro bytes", with a more specific cause) -- don't match on that text. See #117.
Add avro_common::get_supplied_schema_for::<T>(), building a SuppliedSchema straight from a type
implementing apache_avro's own AvroSchema trait (most commonly via
#[derive(apache_avro::AvroSchema)]) instead of hand-writing/maintaining the schema as a
separate JSON string. Combine it with any *WithSchema SubjectNameStrategy variant, which
already registers the schema if the registry doesn't have it yet. Purely additive -- a thin
wrapper around the existing avro_common::get_supplied_schema. See #111 and the "Deriving an
Avro schema from a struct" section of the README.
feat: propagate proterties and tags by @jschmid1 in https://github.com/gklijs/schema_registry_converter/pull/172
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v4.9.0...v4.10.0
Propagate properties and tags. Minor code improvements.
feat: Add clone/debug to all easy encoders/decoders by @PookieBuns in https://github.com/gklijs/schema_registry_converter/pull/167
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v4.8.0...v4.9.0
Support schema compatibility checking. Expose subject and version in schema's. Add clone/debug to all easy encoders/decoders.
Some dependency updates. Better handling for comments in proto massages. Improved error response from schema registry, using the response from schema
Some dependency updates. Better handling for comments in proto massages. Improved error response from schema registry, using the response from schema registry. Adding clone/debug to encoders and decoders.
Full Changelog: v4.7.0...v4.8.0
Some dependency updates. Better handling for comments in proto massages. Improved error response from schema registry, using the response from schema registry. Adding clone/debug to encoders and decoders.
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v4.7.0...v4.8.0
Update avro dependency and add properties inside the metadata of the schema registry response.
Update avro dependency and add properties inside the metadata of the schema registry response.
Just adding support for serialization of the supplied schema.
Just adding support for serialization of the supplied schema.
Add support for properties and tags in metadata, fix a bug where the default values where missing in the SuppliedSchema.
Add support for properties and tags in metadata, fix a bug where the default values where missing in the SuppliedSchema.
Add support for properties and tags metadata. Update versions, most noteworthy apache avro.
Add option to not use a proxy, dependecy updates.
Add option to not use a proxy, dependecy updates.
Updated versions and small code improvements.
Updated versions and small code improvements.
Bugfixes and updates. You can check the pr's merged in between or the commits for details.
Bugfixes and updates. You can check the pr's merged in between or the commits for details.
Update dashmap requirement from ^5.5 to ^6.0 by @dependabot in https://github.com/gklijs/schema_registry_converter/pull/108
In loving memory of my cousin, Prica Koolen, who passed away way to soon.
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v4.1.0...v4.2.0
Updated versions. Add possibility to directly encode avro values. Make it possible to have slashes in subject names. By updating to the latest avro-rs being able to have custom name validators in Avro.
Update rdkafka requirement from ^0.29.0 to ^0.34.0 by @dependabot in https://github.com/gklijs/schema_registry_converter/pull/99
Full Changelog: https://github.com/gklijs/schema_registry_converter/compare/v4.0.0...v4.1.0
Closing the milestone 4.0.0., https://github.com/gklijs/schema_registry_converter/milestone/4 see also https://crates.io/crates/schema_registry_conver
Closing the milestone 4.0.0., https://github.com/gklijs/schema_registry_converter/milestone/4 see also https://crates.io/crates/schema_registry_converter/4.0.0.
Opened up/added some functionality. Updated versions. Simplified the SubjectNameStrategy enum.
Fix a problem with missing checks on some proto common types, like timestamp. Added functions to get the context when using protobuf. Added functions
Fix a problem with missing checks on some proto common types, like timestamp. Added functions to get the context when using protobuf. Added functions to get the Avro schema when using avro.
Several breaking changes in the API, making it easier to use as in most places we don't need a mutable reference anymore. Move to apache-avro for avro…
Several breaking changes in the API, making it easier to use as in most places we don't need a mutable reference anymore. Move to apache-avro for avro, which contains several fixes. Made some additional things public for other use cases. Added some methods to the API in case the used schema is required. Protobuf common types are 'supported' as long as the import is in the main schema, the schema's will be added to the list giving to protofish, so it can be deserialized.
This release will focus on making the library easier to use. At least the following should be done:
This release will focus on making the library easier to use. At least the following should be done:
full_name.rdkafka through.Dependencies updated and ci is now run in Github Actions also some improvements where made making it easier to use, and open up some additional use cases.
reqwest client, and use that to create the SrSettings. Mainly for custom security requirements.rustls_tls feature to let reqwest use rustls-tls.easy variant was added. This makes it easier to use the library, as internally an arc is used, making it easier to use.encode_single_message method was eded to encode when the schema contains only one message. The full name of the proto message is not needed for this.Just updating the dependencies.
Just updating the dependencies.
Maintenance release with mainly updated dependencies, making the blocking sr settings cloneable and no longer needs kafka_test feature to use both blo
Maintenance release with mainly updated dependencies, making the blocking sr settings cloneable and no longer needs kafka_test feature to use both blocking and async in the same project.
Support references in schema registry.
This release has a breaking change in the SubjectNameStrategy where the supplied schema now is in a Box, to keep the size of the Enum smaller. Another breaking change is that the protocol (http or https) needs to be included in the schema registry url. Since besides avro also protobuf and json schema, and to some degree custom formats are supported, avro is no behind a feature flag, and in its own module. Also add support for authentication and the use of a proxy in this version. Another major change is by default support for async.
To use the new version of the library, and continue to use it in a blocking way like it was before, you need to use the library like:
schema_registry_converter = { version = "2.0.2", default-features = false, features = ["avro", "blocking"]}
Also the Converters are moved to the blocking module, and to create the converters you need a SrSettings object, which can be created with just the schema registry url.
let sr_settings = SrSettings::new(String::from("http://localhost:8081"));
This release makes it easier to work with structs, instead of the raw Value type in a vector. To use structs with avro you need to add #[derive(Debug,
This release makes it easier to work with structs, instead of the raw Value type in a vector.
To use structs with avro you need to add #[derive(Debug, Deserialize, Serialize)] above your
struct and also have a dependency on serde with the derive feature enabled. like:
[dependencies.serde]
version = "1.0"
features = ["derive"]
encode_struct
instead of the encode function on the encoder.Made it easier to use the crate by changing some values to owed strings.
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
Nothing published for this version
One column per quarter.