Kafka Consumer Config Generator
A consumer .properties file, with the heartbeat
kept in proportion to the session timeout and the commit mode presented as the
delivery semantics it actually is.
consumer.properties
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Leaving enable.auto.commit at its default
It commits on a timer, so offsets advance for records still being processed. A crash then skips them silently.
Instead:Commit after processing, or accept at-most-once and write it down.
Setting auto.offset.reset without deciding what it means
It fires only when there is no committed offset, so it lands on a new group or after retention removed the position. latest skips a backlog; earliest reprocesses everything.
Instead:Choose deliberately and alert when it triggers, because something else has already gone wrong.
Raising max.poll.records without max.poll.interval.ms
More records per poll means longer between polls. Exceeding the interval makes the broker declare the consumer dead, mid-batch, repeatedly.
Instead:Size them together against worst-case per-record time.
A consumer config is mostly about two clocks and one commit
The commit mode decides your delivery semantics, and two separate timeouts decide whether the group thinks you are alive.
The commit mode is the delivery semantics
enable.auto.commit=true, which is the default, commits offsets on a timer whether or not processing finished. A crash between the commit and the end of processing loses those records permanently, which is at-most-once. Committing manually after processing gives at-least-once, where a crash reprocesses instead of losing. Neither is exactly-once on its own: that needs transactions on both the producer and the consumer. This form asks for the semantics rather than for a boolean, because almost nobody who leaves auto-commit on has chosen at-most-once.
Two timeouts, two different failures
session.timeout.ms detects a dead process: heartbeats stop and the group rebalances. max.poll.interval.ms detects a stuck one: the process is alive and heartbeating but has not called poll(), so it is removed anyway. They are separate because a consumer that takes ten minutes on a batch is not dead. The heartbeat interval is emitted as a third of the session timeout, which is the documented relationship, so changing the timeout does not leave the two out of proportion.
max.poll.records times your processing time is a budget
500 records at 100 ms each is 50 seconds, which fits comfortably inside the 5 minute default. The same 500 records at 2 seconds each is 16 minutes and the group rebalances mid-batch, which presents as a consumer that restarts for no reason and reprocesses. Reducing max.poll.records is usually the right fix rather than raising the interval, because a long interval also delays detection of a genuinely stuck consumer.
The cooperative assignor is emitted with a fallback, deliberately
CooperativeStickyAssignor avoids every consumer pausing on every rebalance, and a group already running RangeAssignor cannot switch to it in one rolling restart: mid-restart the old and new members share no common protocol and the group does not form. So the output lists both, cooperative first, which is step one of the documented two-step rollout. Deploy it, wait for every member, then deploy again with the fallback removed. Our own consumer linter flagged the single-assignor form on this generator's output, which is how the fallback got added.
What this cannot see
It does not know how long your processing takes per record, which is the number the poll budget depends on, and it cannot tell whether a rebalance is caused by slow processing, a deploy loop or a network problem. It also cannot verify the secret file it references or that the group has permission to read the topic. For a backlog that already exists, the consumer lag calculator on this site answers whether it will ever clear.