Kafka Avro Sample Data Generator

Generate sample records from an Avro schema, one JSON document per line, in the form a producer actually takes. Every record is serialized against the schema before it is shown.

Optional fields are null by default, which is what most records look like. Filling them exercises the union branches instead, which is what you want when testing a consumer.

Paste below, or drop a file anywhere on this panel

Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.

The answer appears here

Paste on the left and press Generate. Nothing leaves this tab.

Examples

Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.

Sample records

Data shaped by the schema, for a producer test that does not need real data

{"type":"record","name":"User","fields":[{"name":"id","type":"int"},{"name":"email","type":["null","string"],"default":null}]}

A schema with a union

How an optional field appears in Avro JSON, wrapped in its type

{"type":"record","name":"E","fields":[{"name":"ts","type":"long"},{"name":"tag","type":["null","string"],"default":null}]}

Common mistakes

These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.

  1. Using generated data as a fixture

    It is shaped by the schema and means nothing. Assertions written against it test the generator.

    Instead:Use it to exercise a pipeline, not to verify behaviour.

  2. Forgetting the union wrapping

    Sample data in Avro JSON wraps union values in their type. Feeding it to a plain JSON consumer fails.

    Instead:Pick the output format that matches what will read it.

  3. Assuming defaults fill optional fields

    Avro JSON requires every field present. A default applies to schema resolution, not encoding.

    Instead:Fill optional fields when the consumer expects them.

Records that serialize, which is the only claim worth making

Sample data is easy to generate and easy to generate wrongly. Every record here is serialized against the schema it came from before it is shown, so anything the schema cannot represent is reported instead of printed.

The Avro JSON encoding is what a producer takes

kafka-avro-console-producer, the REST proxy and every Avro JSON parser expect the wrapped form, where each non-null union value is a one-key object named for the branch's type and a record, enum or fixed branch uses its fullname. The plain form is easier to read and will be rejected. Both are available here and the plain one carries a warning, because that mismatch is the usual reason a first attempt fails.

kafka-avro-console-producer \
  --bootstrap-server broker:9093 \
  --topic orders \
  --property value.schema="$(cat schema.avsc)" \
  < samples.jsonl

It is deterministic on purpose

Values come from the schema and from a hash of each field's path, never from a random source. The same schema always produces the same records, so a change in the output means a change in the schema and nothing else. That makes it usable in a fixture, and it means two people looking at this page see the same thing.

Recursion terminates by taking the null branch

A recursive schema has no natural end, so a generator that follows one produces an infinite record. Nested records stop at a fixed depth and a nullable union takes null once it gets deep enough, which is why a linked-list schema gives a short chain rather than hanging. It is also why filling optional fields does not fill every one of them.

Logical types get plausible values

A timestamp-millis is a real epoch millisecond, a date is a day count, and a uuid is a well-formed v4-shaped string. They are not realistic data and they are the right type, which is what a serializer, a sink connector or a schema migration needs from them.

What this is not for

Anything that depends on the shape of real data. Sizes, cardinalities, distributions and compression ratios from these records mean nothing. Use it to exercise a consumer, a topic, a converter configuration or a Connect pipeline, and take the numbers from production.

More kafka tools

Kafka Confluent Wire Format Decoder The five junk bytes in front of your payload Kafka Key to Partition Mapper Which partition does this key land on? Kafka Topic Name Validator Legal, risky, or 249 characters too long? Kafka Replication Safety Checker How many brokers can you lose Kafka Producer Config Linter Will it start, and will it lose a record? Kafka Message Payload Decoder The first five bytes are usually not data Kafka Connect Source Connector Generator tasks.max is a ceiling, not a count Kafka Connect Sink Connector Generator A dead letter queue with no context headers is a pile of records Kafka Connect SMT Chain Builder The order is the transforms list Kafka MirrorMaker 2 Config Generator It renames every topic by default Kafka Partition Reassignment Generator The throttle is not optional Strimzi Kafka Resource Generator Without the cluster label, nothing happens Kafka mTLS Config Generator The certificate is the identity Kafka Schema Registry Config Generator The compatibility direction is your deployment order Kafka Exactly-Once Config Generator Half of it is worse than none Kafka Broker and KRaft Config Generator The internal topics that break a one-broker cluster Kafka Quota Generator Byte rates are per broker, not per cluster Kafka Streams Config Generator application.id is four things at once Kafka Connect Worker Config Generator Security three times, or the tasks fail Kafka Retention and Unit Converter log.retention.hours does not take milliseconds Kafka Timestamp Converter Two sentinels and two meanings Kafka .properties to YAML Converter Dotted keys stay flat Kafka Streams Internal Topic Predictor Create them before Streams does Kafka ACL Generator The grant you forgot is on another resource type Kafka Topic Config Generator min.insync.replicas is the one that matters Kafka client.properties Generator The file every CLI tool asks for Kafka Producer Config Generator No password field, on purpose Kafka Consumer Config Generator The commit mode decides the semantics Kafka Disk and Retention Calculator retention.bytes is per partition Kafka Partition Count Calculator The number you can never reduce Kafka Cluster Sizing Calculator The traffic no client metric shows Kafka Consumer Lag Catch-Up Calculator Whether it ever clears, not just when Kafka Producer Batching Calculator linger.ms=0 still batches Kafka Segment and Index Sizing Why retention.ms is a lower bound Kafka Rebalance Duration Estimator What a rolling restart really costs Kafka Cost Estimator Your rates, so nothing goes stale Kafka Config Explorer by Version The answer depends on the release Kafka Default Config Reference What moved under a config you never edited Kafka OAuth Bearer Token Decoder Will Kafka accept it, and can it refresh Kafka Record Header Viewer Headers are a list, not a map Kafka Topic Regex Subscription Tester Kafka matches the whole name Kafka ACL Permission Matrix Viewer DENY beats every ALLOW Kafka Connect Config Validator The mistakes that raise no error Kafka Consumer Group Id Validator Which broker coordinates the group Kafka Partition Assignment Visualizer Leadership is the load, not replicas Kafka Consumer Assignment Visualizer The three assignors disagree Kafka ZooKeeper to KRaft Config Converter The authorizer class nobody changes Kafka Config to Strimzi Half of it belongs elsewhere Kafka Docker Compose Generator (KRaft) Reachable from inside and outside Kafka JAAS Config Decoder The line that stops SASL working Kafka CRC32C Calculator Which CRC, over which bytes Kafka Config Upgrade Checker What breaks when you upgrade Kafka Kafka Config Diff Which change actually changed something Kafka Consumer Config Linter Why the group rebalances, and where the records went Kafka Avro Schema Validator The defaults Avro accepts and rejects Kafka Schema Compatibility Checker What the registry will say, before you ask it Kafka Avro Schema Diff Which direction each change breaks Kafka Compression Comparison Measured on your bytes Kafka Delivery Semantics Exactly-once has a consumer half Kafka ksqlDB Query Builder It looks like SQL and the rules are not Kafka Connect SMT Predicate Tester negate reads backwards Kafka Streams Topology Viewer Count the repartitions Kafka Connect Pipeline Visualizer The order things really run in Kafka Protobuf Binary Decoder Works without the .proto Kafka Protobuf JSON Converter Why your JSON does not round-trip Kafka Protobuf to Avro Schema What does not survive the conversion Kafka Avro Binary Decoder Wrong schema, no error Kafka Avro JSON Converter Why the console producer rejects your line Kafka JSON to Avro Schema What JSON cannot tell you Kafka JSON Schema to Avro What does not survive the conversion Kafka SASL JAAS Generator One login module, four syntaxes Kafka CLI Command Builder kcat is librdkafka, not Kafka

Elsewhere on the site

Terraform Validator & Formatter Kubernetes YAML Generator Kubernetes YAML Validator AWS IAM Policy Validator & Analyzer