Kafka Topic Config Generator
A kafka-topics.sh create command or a topic
properties file, with the replication factor and
min.insync.replicas checked against each other
by the replication safety analyzer on this site.
topic-config.sh
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Setting replication factor without min.insync.replicas
RF alone decides how many copies exist, not how many must acknowledge. With min.isr at 1 an acks=all write can be acknowledged by the leader alone.
Instead:min.insync.replicas is the setting that provides the durability.
Setting min.insync.replicas equal to the replication factor
One broker down then stops all writes to that topic.
Instead:min.isr = RF - 1.
Choosing cleanup.policy=compact for an event stream
Compaction keeps the latest value per key and discards the history, which for an event log is silent data loss.
Instead:compact for state, delete for events, and compact,delete only when you mean both.
The two numbers that decide whether the topic stays writable
replication.factor and min.insync.replicas are usually set at the same time and are almost never thought about together, which is where the outage comes from.
min.insync.replicas is what makes acks=all mean anything
acks=all does not wait for every replica, it waits for the in-sync ones, and min.insync.replicas is how many that has to be. With it at 1, acks=all waits for the leader alone and provides no more durability than acks=1, which makes a carefully configured producer pointless. It is a topic setting rather than a producer setting, so the durability of a write is decided partly by someone who may not have written the producer.
Setting it equal to the replication factor is the classic outage
replication.factor=3 with min.insync.replicas=3 requires every replica to be in sync for a write to succeed, so the topic stops accepting acks=all writes the moment one broker goes down. That includes a routine rolling restart, which means the config looks safe in review and takes the topic offline on the next upgrade. The pairing that works is 3 and 2: it survives one broker being unavailable, and it still refuses a write when two are.
Compaction is a different shape of retention
cleanup.policy=compact keeps the most recent record per key indefinitely, so the size depends on the number of distinct keys rather than on time, and time-based disk estimates do not apply. Every record needs a key, since a null key on a compacted topic is rejected. A delete is a tombstone, a record with a null value, and it survives for delete.retention.ms so consumers get a chance to observe it before it disappears. This generator omits retention.ms entirely on a compact-only topic, because emitting it implies a time bound that does not exist.
max.message.bytes is three settings, not one
Raising the topic limit is necessary and not sufficient. The producer's max.request.size has to be at least as large or it rejects the record before sending it, and the consumer's max.partition.fetch.bytes has to be too or it cannot fetch what was written. Changing one produces an error that points at a different component from the one that needs changing, which is why this is worth stating rather than leaving to be discovered.
What this cannot see
It does not know your broker defaults, so it cannot tell you which of these settings is already the value you want cluster-wide and therefore does not need a topic override. It also does not know your throughput, which is what decides retention.bytes: the disk and retention calculator on this site produces that number, and remember that the setting is per partition. Partitions and replication factor are arguments to topic creation rather than topic configs, which is why they appear in the command form and not in the properties output.