A week of retention
The millisecond values Kafka config actually takes, read back as durations
retention.ms=604800000 segment.bytes=1073741824
Kafka Retention and Unit Converter
Paste Kafka settings and see each duration and size converted, with the unit that setting actually takes. The values that are almost certainly in the wrong unit are flagged, because a broker accepts them without comment.
Paste below, or drop a file anywhere on this panel
Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.
The answer appears here
Paste on the left and press Convert. Nothing leaves this tab.
Formatted
Results
Nothing else to flag.
No formatting problems, and nothing the rules object to. Worth remembering what that covers: this reads the file you pasted, not the account or cluster it will be applied to.
No finding matches that filter.
Examples
Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.
A suspiciously small retention
100 ms of retention, which is a units mistake rather than a policy
retention.ms=100 max.message.bytes=1000000000
Common mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Assuming a Kafka duration is in seconds
Almost every duration in Kafka config is MILLISECONDS. retention.ms=3600 is one hour of retention in seconds and 3.6 seconds in reality.
Instead:Read the suffix. .ms is milliseconds, .seconds is seconds, and there are few of the latter.
Setting retention.bytes expecting a topic-wide limit
It is PER PARTITION. A topic with 12 partitions and retention.bytes=1GB can hold 12GB.
Instead:Divide by the partition count when sizing.
Setting max.message.bytes on the broker only
The topic-level override wins, and the producer's max.request.size and the consumer's fetch.max.bytes must both allow it too. Any one of them rejects the message.
Instead:Change all three together, and test with a message at the new size.
Kafka expresses the same idea in three different units
The unit is part of the setting name, it is not consistent, and a value in the wrong one is a legal number the broker accepts without comment.
log.retention.hours, log.retention.minutes and log.retention.ms all exist
They are three settings for one concept, and they take three different units. The precedence runs ms, then minutes, then hours, so setting the hours form while the ms form is also set does nothing at all. Most example configs use the hours form, most documentation quotes milliseconds, and copying a value from one into the other is wrong by a factor of 3,600,000 while remaining a perfectly valid number.
The failure is silent, which is why this is worth a page
log.retention.hours=604800000 is 69,000 years. The broker starts, reports healthy, and retains everything forever, and there is no error anywhere because the value is a legal number of hours. The reverse is worse: retention.ms=7 is seven milliseconds, so every record is deleted as soon as its segment rolls, and the symptom is a topic that appears to lose data at random. Both are flagged here.
Sizes are always bytes, with no suffix accepted
Every Kafka size setting is a plain byte count. There is no 1g, no 512MB and no suffix of any kind, and a value with one is rejected. They are also binary rather than decimal, so a gibibyte is 1,073,741,824 rather than 1,000,000,000, and treating them as decimal understates disk by 7% at the gigabyte scale and more above it.
The sentinels are separate meanings, not extreme values
-1 means unlimited on a retention setting and disabled on most timeouts, so reading it as a duration produces a topic that looks expired. 0 is a real value and usually means immediately rather than off. -2 means inherit the cluster default and appears on the tiered storage local.retention settings. None of the three is a small or large amount of time.
What this cannot see
It recognises the settings in its own table by name, and reads the unit from the suffix for anything else, which is a convention rather than a guarantee. It cannot know your intent, so a value that is legal and unusual is reported as unusual rather than as wrong. For whether a whole config is coherent, the producer and consumer linters on this site read the settings against each other.
Related kafka tools
Kafka Segment and Index Sizing Why retention.ms is a lower bound Kafka Timestamp Converter Two sentinels and two meanings Kafka Topic Config Generator min.insync.replicas is the one that matters Kafka Disk and Retention Calculator retention.bytes is per partition Kafka Connect Worker Config Generator Security three times, or the tasks fail Kafka .properties to YAML Converter Dotted keys stay flat
All 73 kafka tools