Inferred from samples
A number with no decimal part is inferred as int, which overflows above two billion
{"id":1,"name":"a"}
{"id":2,"name":null}Kafka JSON to Avro Schema
Infer an Avro schema from sample records. Every guess is listed with its path, because JSON cannot tell an int from a long, an absent key from a null one, or an empty array's element type from nothing at all.
Paste one record, a JSON array of them, or one JSON document per line. More samples give a better schema, because an optional field is only visible when it is missing from one of them.
Paste below, or drop a file anywhere on this panel
Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.
The answer appears here
Paste on the left and press Infer schema. Nothing leaves this tab.
Formatted
Results
Nothing else to flag.
No formatting problems, and nothing the rules object to. Worth remembering what that covers: this reads the file you pasted, not the account or cluster it will be applied to.
No finding matches that filter.
Examples
Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.
A field missing from one sample
Fields absent in any sample become nullable, because the inference cannot assume
{"id":1,"email":"a@b.com"}
{"id":2}Common mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Inferring from one sample
A field absent in the sample is absent from the schema, and a null is inferred as null-only.
Instead:Give several samples covering the real variation.
Accepting int for an id
A number with no decimal part infers as int, which overflows at 2,147,483,647. Ids reach that.
Instead:Widen to long for anything that counts up.
Trusting inferred nullability
A field present in every sample is inferred required, which breaks the first time it is absent.
Instead:Mark optional fields explicitly rather than relying on the samples.
What JSON cannot tell you, and what this does about it
Inferring a schema from data is guessing, and the guesses are the interesting part. Every one of them is listed here with its path, and the ones that cannot be guessed are marked rather than filled in.
One sample cannot show which fields are optional
A field is optional when it is absent from some records and present in others. From a single sample every field looks present and required, which produces a schema that cannot be evolved and rejects half the topic. Paste several records, one JSON document per line: a dump of a few hundred is better than a hand-written example, because it contains the cases nobody thought of.
kcat -C -t orders -e -c 500 \
-f '%s\n' > samples.jsonlJSON has one number type and Avro has four
A value of 3 could be an int, a long, a float or a fixed-point amount, and nothing in the document says which. This picks the narrowest type that holds every sample and tells you it did. Widening turns every integer into a long instead, which is worth doing for anything that will grow: an int that overflows fails at serialization rather than truncating, and widening a field from int to long later is a backward compatible change while the reverse is not. For money, neither is right: use bytes with a decimal logical type and a fixed scale.
An empty array carries no element type
This is the one thing here that more data of the same kind cannot fix. An array that is empty in every sample says nothing about what goes in it, so a placeholder is emitted and the finding is loud. Nothing else in this tool invents a type it has no evidence for.
Avro cannot carry a key like user-id at all
Names are letters, digits and underscores, and must not start with a digit. There is no escaping and no alias that helps, because an alias has to be a legal name too. So a JSON key with a hyphen, a dot or a space is renamed here and reported as critical: the schema you get does not describe the JSON you pasted, and the key has to change in whatever produces the records. A Connect ReplaceField transformation does it in a pipeline.
Nothing inferred has a default, and that matters at version two
There is nothing in the data to infer a default from, so every field comes out without one. That is fine for a first version and it means none of these fields can be added to a schema that is already registered. Adding defaults before you register is a minute's work and saves the argument later. The schema validator on this site says the same thing about any schema you paste.
What this cannot see
Semantics. It cannot tell a timestamp from a number, a currency amount from a float, or an id from any other string, so no logical type is ever guessed. Everything runs in your browser, which matters more than usual here: sample records are real records.
More kafka tools
Kafka Confluent Wire Format Decoder The five junk bytes in front of your payload Kafka Key to Partition Mapper Which partition does this key land on? Kafka Topic Name Validator Legal, risky, or 249 characters too long? Kafka Replication Safety Checker How many brokers can you lose Kafka Producer Config Linter Will it start, and will it lose a record? Kafka Message Payload Decoder The first five bytes are usually not data Kafka Connect Source Connector Generator tasks.max is a ceiling, not a count Kafka Connect Sink Connector Generator A dead letter queue with no context headers is a pile of records Kafka Connect SMT Chain Builder The order is the transforms list Kafka MirrorMaker 2 Config Generator It renames every topic by default Kafka Partition Reassignment Generator The throttle is not optional Strimzi Kafka Resource Generator Without the cluster label, nothing happens Kafka mTLS Config Generator The certificate is the identity Kafka Schema Registry Config Generator The compatibility direction is your deployment order Kafka Exactly-Once Config Generator Half of it is worse than none Kafka Broker and KRaft Config Generator The internal topics that break a one-broker cluster Kafka Quota Generator Byte rates are per broker, not per cluster Kafka Streams Config Generator application.id is four things at once Kafka Connect Worker Config Generator Security three times, or the tasks fail Kafka Retention and Unit Converter log.retention.hours does not take milliseconds Kafka Timestamp Converter Two sentinels and two meanings Kafka .properties to YAML Converter Dotted keys stay flat Kafka Streams Internal Topic Predictor Create them before Streams does Kafka ACL Generator The grant you forgot is on another resource type Kafka Topic Config Generator min.insync.replicas is the one that matters Kafka client.properties Generator The file every CLI tool asks for Kafka Producer Config Generator No password field, on purpose Kafka Consumer Config Generator The commit mode decides the semantics Kafka Disk and Retention Calculator retention.bytes is per partition Kafka Partition Count Calculator The number you can never reduce Kafka Cluster Sizing Calculator The traffic no client metric shows Kafka Consumer Lag Catch-Up Calculator Whether it ever clears, not just when Kafka Producer Batching Calculator linger.ms=0 still batches Kafka Segment and Index Sizing Why retention.ms is a lower bound Kafka Rebalance Duration Estimator What a rolling restart really costs Kafka Cost Estimator Your rates, so nothing goes stale Kafka Config Explorer by Version The answer depends on the release Kafka Default Config Reference What moved under a config you never edited Kafka OAuth Bearer Token Decoder Will Kafka accept it, and can it refresh Kafka Record Header Viewer Headers are a list, not a map Kafka Topic Regex Subscription Tester Kafka matches the whole name Kafka ACL Permission Matrix Viewer DENY beats every ALLOW Kafka Connect Config Validator The mistakes that raise no error Kafka Consumer Group Id Validator Which broker coordinates the group Kafka Partition Assignment Visualizer Leadership is the load, not replicas Kafka Consumer Assignment Visualizer The three assignors disagree Kafka ZooKeeper to KRaft Config Converter The authorizer class nobody changes Kafka Config to Strimzi Half of it belongs elsewhere Kafka Docker Compose Generator (KRaft) Reachable from inside and outside Kafka JAAS Config Decoder The line that stops SASL working Kafka CRC32C Calculator Which CRC, over which bytes Kafka Config Upgrade Checker What breaks when you upgrade Kafka Kafka Config Diff Which change actually changed something Kafka Consumer Config Linter Why the group rebalances, and where the records went Kafka Avro Schema Validator The defaults Avro accepts and rejects Kafka Schema Compatibility Checker What the registry will say, before you ask it Kafka Avro Schema Diff Which direction each change breaks Kafka Compression Comparison Measured on your bytes Kafka Delivery Semantics Exactly-once has a consumer half Kafka ksqlDB Query Builder It looks like SQL and the rules are not Kafka Connect SMT Predicate Tester negate reads backwards Kafka Streams Topology Viewer Count the repartitions Kafka Connect Pipeline Visualizer The order things really run in Kafka Protobuf Binary Decoder Works without the .proto Kafka Protobuf JSON Converter Why your JSON does not round-trip Kafka Protobuf to Avro Schema What does not survive the conversion Kafka Avro Binary Decoder Wrong schema, no error Kafka Avro JSON Converter Why the console producer rejects your line Kafka Avro Sample Data Generator Records that actually serialize Kafka JSON Schema to Avro What does not survive the conversion Kafka SASL JAAS Generator One login module, four syntaxes Kafka CLI Command Builder kcat is librdkafka, not Kafka