Kafka Avro Schema Diff

Two Avro schemas compared as schemas rather than as text, with the direction each change breaks worked out from Avro's resolution rules.

Nothing is uploaded: the comparison runs in this tab.

Common mistakes

These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.

  1. Assuming a compatible change is compatible in both directions

    Adding a field with a default is backward compatible and not forward compatible. Which one you need depends on whether readers or writers upgrade first.

    Instead:Decide the deployment order first, then pick the compatibility mode that matches it.

  2. Renaming a field

    Avro matches by name, so a rename is a delete plus an add. Readers on the old schema lose the value entirely.

    Instead:Add an alias instead, which schema resolution understands.

  3. Changing a field's type to a union

    Promoting string to [null, string] is not compatible in either direction unless defaults line up exactly, because the encoding changes.

    Instead:Add a new field and deprecate the old one.

Every change, and the direction it breaks

A text diff of two schemas tells you which lines moved. This one pairs fields by name and states, from Avro's resolution rules, which direction each change stops working in.

Field order is free and looks dangerous

Avro writes a record's fields in schema order and resolves them by name, so a reader with a different order still finds each value. The encoded bytes change completely and the meaning does not. This is the one structural change that looks alarming in a diff and costs nothing, which is why it is called out rather than left in a list of changes that mostly do matter.

A union branch added breaks forward, and removed breaks backward

Adding a branch is safe for a consumer on the new schema and fatal for one still on the old, because that consumer has no branch for the value it just received. Removing one is the mirror image: data already written can carry the removed branch and a new reader has nowhere to put it. The field still exists and still has a plausible type in both cases, which is what makes this easy to miss in review.

A namespace move breaks nothing the registry checks

Avro compares the short name of a type, then the reader's aliases. So moving a record between namespaces resolves, the registry accepts it, and this page reports no incompatibility either, because that would be disagreeing with the thing it predicts. It is still a breaking change everywhere else: generated classes move package, the fingerprint changes, and any consumer pinning the fullname stops matching. That is why it appears as a change with its own severity.

Documentation and logical types are outside the fingerprint

The parsing canonical form keeps type, name, fields, symbols, items, values and size, and strips everything else. So a change to a doc string, an alias, a default or a logical type leaves the fingerprint identical while changing what generated code does. When the panel reports the same fingerprint on both sides and a list of changes above it, that is the reason.

This says what changed, not whether it is allowed

Whether a breaking change matters depends on the compatibility level set on the subject, and on which of your consumers have actually been redeployed. The compatibility checker on this site gives the registry's own verdict for a chosen level, including the transitive ones. Use this page to review a change and that one to predict the registry.

More kafka tools

Kafka Confluent Wire Format Decoder The five junk bytes in front of your payload Kafka Key to Partition Mapper Which partition does this key land on? Kafka Topic Name Validator Legal, risky, or 249 characters too long? Kafka Replication Safety Checker How many brokers can you lose Kafka Producer Config Linter Will it start, and will it lose a record? Kafka Message Payload Decoder The first five bytes are usually not data Kafka Connect Source Connector Generator tasks.max is a ceiling, not a count Kafka Connect Sink Connector Generator A dead letter queue with no context headers is a pile of records Kafka Connect SMT Chain Builder The order is the transforms list Kafka MirrorMaker 2 Config Generator It renames every topic by default Kafka Partition Reassignment Generator The throttle is not optional Strimzi Kafka Resource Generator Without the cluster label, nothing happens Kafka mTLS Config Generator The certificate is the identity Kafka Schema Registry Config Generator The compatibility direction is your deployment order Kafka Exactly-Once Config Generator Half of it is worse than none Kafka Broker and KRaft Config Generator The internal topics that break a one-broker cluster Kafka Quota Generator Byte rates are per broker, not per cluster Kafka Streams Config Generator application.id is four things at once Kafka Connect Worker Config Generator Security three times, or the tasks fail Kafka Retention and Unit Converter log.retention.hours does not take milliseconds Kafka Timestamp Converter Two sentinels and two meanings Kafka .properties to YAML Converter Dotted keys stay flat Kafka Streams Internal Topic Predictor Create them before Streams does Kafka ACL Generator The grant you forgot is on another resource type Kafka Topic Config Generator min.insync.replicas is the one that matters Kafka client.properties Generator The file every CLI tool asks for Kafka Producer Config Generator No password field, on purpose Kafka Consumer Config Generator The commit mode decides the semantics Kafka Disk and Retention Calculator retention.bytes is per partition Kafka Partition Count Calculator The number you can never reduce Kafka Cluster Sizing Calculator The traffic no client metric shows Kafka Consumer Lag Catch-Up Calculator Whether it ever clears, not just when Kafka Producer Batching Calculator linger.ms=0 still batches Kafka Segment and Index Sizing Why retention.ms is a lower bound Kafka Rebalance Duration Estimator What a rolling restart really costs Kafka Cost Estimator Your rates, so nothing goes stale Kafka Config Explorer by Version The answer depends on the release Kafka Default Config Reference What moved under a config you never edited Kafka OAuth Bearer Token Decoder Will Kafka accept it, and can it refresh Kafka Record Header Viewer Headers are a list, not a map Kafka Topic Regex Subscription Tester Kafka matches the whole name Kafka ACL Permission Matrix Viewer DENY beats every ALLOW Kafka Connect Config Validator The mistakes that raise no error Kafka Consumer Group Id Validator Which broker coordinates the group Kafka Partition Assignment Visualizer Leadership is the load, not replicas Kafka Consumer Assignment Visualizer The three assignors disagree Kafka ZooKeeper to KRaft Config Converter The authorizer class nobody changes Kafka Config to Strimzi Half of it belongs elsewhere Kafka Docker Compose Generator (KRaft) Reachable from inside and outside Kafka JAAS Config Decoder The line that stops SASL working Kafka CRC32C Calculator Which CRC, over which bytes Kafka Config Upgrade Checker What breaks when you upgrade Kafka Kafka Config Diff Which change actually changed something Kafka Consumer Config Linter Why the group rebalances, and where the records went Kafka Avro Schema Validator The defaults Avro accepts and rejects Kafka Schema Compatibility Checker What the registry will say, before you ask it Kafka Compression Comparison Measured on your bytes Kafka Delivery Semantics Exactly-once has a consumer half Kafka ksqlDB Query Builder It looks like SQL and the rules are not Kafka Connect SMT Predicate Tester negate reads backwards Kafka Streams Topology Viewer Count the repartitions Kafka Connect Pipeline Visualizer The order things really run in Kafka Protobuf Binary Decoder Works without the .proto Kafka Protobuf JSON Converter Why your JSON does not round-trip Kafka Protobuf to Avro Schema What does not survive the conversion Kafka Avro Binary Decoder Wrong schema, no error Kafka Avro JSON Converter Why the console producer rejects your line Kafka Avro Sample Data Generator Records that actually serialize Kafka JSON to Avro Schema What JSON cannot tell you Kafka JSON Schema to Avro What does not survive the conversion Kafka SASL JAAS Generator One login module, four syntaxes Kafka CLI Command Builder kcat is librdkafka, not Kafka

Elsewhere on the site