Kafka Consumer Group Id Validator

Work out which __consumer_offsets partition a consumer group hashes to, and therefore which broker coordinates it, using Kafka's own hash. Also flags the ids that break JMX metric names, which the broker accepts happily.

The partition count is fixed when __consumer_offsets is created and cannot be altered afterwards, so if your cluster is not on the default 50 this has to match or every result is wrong.

Paste below, or drop a file anywhere on this panel

Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.

The answer appears here

Paste on the left and press Check group ids. Nothing leaves this tab.

Examples

Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.

An empty group id

A blank line, which is a real and distinct mistake rather than nothing

my-consumer-group
My Group

Which partition holds the offsets

The __consumer_offsets partition a group hashes to, which decides its coordinator

orders-consumer
payments-consumer
inventory-consumer

Common mistakes

These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.

  1. Using a group id with spaces or unusual characters

    It becomes part of metrics and file paths, and the same dot-versus-underscore collision as topic names applies.

    Instead:Stick to lowercase, hyphens and dots, and be consistent about which.

  2. Sharing one group id across unrelated consumers

    They will rebalance against each other and split partitions, so each sees only some of the messages.

    Instead:One group id per logical consumer application.

  3. Changing the group id to reset offsets

    A new group starts from auto.offset.reset, which usually means reprocessing everything or skipping the backlog.

    Instead:Use kafka-consumer-groups --reset-offsets on the existing group.

Which broker coordinates your group, and why the id matters elsewhere

Kafka puts almost no restriction on a group id, so this is not really a validator in the sense the topic name validator is. The useful answers are where the group's state lands and what the id collides with outside Kafka.

The group id decides which broker coordinates the group

Kafka hashes the group id with Java's String.hashCode, masks the sign off with 0x7fffffff, and takes it modulo the number of __consumer_offsets partitions. The leader of that partition is the group coordinator, and every join, heartbeat and offset commit for the group goes to that one broker. That placement is fixed by the hash: it cannot be moved without changing the group id, and changing the group id abandons the committed offsets. This page computes it exactly rather than approximately, which is why the number matches what the broker does.

partition = (hashCode(groupId) & 0x7fffffff)
            % offsets.topic.num.partitions

# masked, not Math.abs: in Java, Math.abs of
# Integer.MIN_VALUE is still negative, and a negative
# partition number would be a real bug

Two groups on the same partition fail together

Groups are placed by hash rather than spread deliberately, so two unrelated services can share a coordinator. That is normal and harmless until the broker holding that partition restarts, at which point both groups rebalance at the same moment and both recover at the same moment. When two services appear to have a correlated incident with no shared dependency, a shared coordinator is worth checking.

Kafka does not restrict the characters, and JMX does

The broker accepts any non-empty string. The consumer's own metrics, though, are registered under a JMX ObjectName that contains the group id, and a comma, equals sign, colon, quote, asterisk or question mark is either a separator or a wildcard there. The result is a group that works perfectly and whose metrics fail to register or cannot be queried, so the consumer is healthy and invisible. Since a group cannot be renamed without losing its offsets, this is worth getting right at the start.

offsets.topic.num.partitions cannot be changed later

It is 50 by default and fixed when __consumer_offsets is created. If your cluster was created with a different value, set it above or every partition on this page is wrong. It is also why a development cluster that created the topic with one partition keeps every group on one coordinator forever.

Console consumer groups accumulate

kafka-console-consumer invents a random group id per run unless you pass --group. Each becomes a real group with real state, and on a cluster people debug on they collect in the thousands, which slows group listing and lengthens coordinator loading after a broker restart. They do expire, after offsets.retention.minutes, which is 7 days by default.

More kafka tools

Kafka Confluent Wire Format Decoder The five junk bytes in front of your payload Kafka Key to Partition Mapper Which partition does this key land on? Kafka Topic Name Validator Legal, risky, or 249 characters too long? Kafka Replication Safety Checker How many brokers can you lose Kafka Producer Config Linter Will it start, and will it lose a record? Kafka Message Payload Decoder The first five bytes are usually not data Kafka Connect Source Connector Generator tasks.max is a ceiling, not a count Kafka Connect Sink Connector Generator A dead letter queue with no context headers is a pile of records Kafka Connect SMT Chain Builder The order is the transforms list Kafka MirrorMaker 2 Config Generator It renames every topic by default Kafka Partition Reassignment Generator The throttle is not optional Strimzi Kafka Resource Generator Without the cluster label, nothing happens Kafka mTLS Config Generator The certificate is the identity Kafka Schema Registry Config Generator The compatibility direction is your deployment order Kafka Exactly-Once Config Generator Half of it is worse than none Kafka Broker and KRaft Config Generator The internal topics that break a one-broker cluster Kafka Quota Generator Byte rates are per broker, not per cluster Kafka Streams Config Generator application.id is four things at once Kafka Connect Worker Config Generator Security three times, or the tasks fail Kafka Retention and Unit Converter log.retention.hours does not take milliseconds Kafka Timestamp Converter Two sentinels and two meanings Kafka .properties to YAML Converter Dotted keys stay flat Kafka Streams Internal Topic Predictor Create them before Streams does Kafka ACL Generator The grant you forgot is on another resource type Kafka Topic Config Generator min.insync.replicas is the one that matters Kafka client.properties Generator The file every CLI tool asks for Kafka Producer Config Generator No password field, on purpose Kafka Consumer Config Generator The commit mode decides the semantics Kafka Disk and Retention Calculator retention.bytes is per partition Kafka Partition Count Calculator The number you can never reduce Kafka Cluster Sizing Calculator The traffic no client metric shows Kafka Consumer Lag Catch-Up Calculator Whether it ever clears, not just when Kafka Producer Batching Calculator linger.ms=0 still batches Kafka Segment and Index Sizing Why retention.ms is a lower bound Kafka Rebalance Duration Estimator What a rolling restart really costs Kafka Cost Estimator Your rates, so nothing goes stale Kafka Config Explorer by Version The answer depends on the release Kafka Default Config Reference What moved under a config you never edited Kafka OAuth Bearer Token Decoder Will Kafka accept it, and can it refresh Kafka Record Header Viewer Headers are a list, not a map Kafka Topic Regex Subscription Tester Kafka matches the whole name Kafka ACL Permission Matrix Viewer DENY beats every ALLOW Kafka Connect Config Validator The mistakes that raise no error Kafka Partition Assignment Visualizer Leadership is the load, not replicas Kafka Consumer Assignment Visualizer The three assignors disagree Kafka ZooKeeper to KRaft Config Converter The authorizer class nobody changes Kafka Config to Strimzi Half of it belongs elsewhere Kafka Docker Compose Generator (KRaft) Reachable from inside and outside Kafka JAAS Config Decoder The line that stops SASL working Kafka CRC32C Calculator Which CRC, over which bytes Kafka Config Upgrade Checker What breaks when you upgrade Kafka Kafka Config Diff Which change actually changed something Kafka Consumer Config Linter Why the group rebalances, and where the records went Kafka Avro Schema Validator The defaults Avro accepts and rejects Kafka Schema Compatibility Checker What the registry will say, before you ask it Kafka Avro Schema Diff Which direction each change breaks Kafka Compression Comparison Measured on your bytes Kafka Delivery Semantics Exactly-once has a consumer half Kafka ksqlDB Query Builder It looks like SQL and the rules are not Kafka Connect SMT Predicate Tester negate reads backwards Kafka Streams Topology Viewer Count the repartitions Kafka Connect Pipeline Visualizer The order things really run in Kafka Protobuf Binary Decoder Works without the .proto Kafka Protobuf JSON Converter Why your JSON does not round-trip Kafka Protobuf to Avro Schema What does not survive the conversion Kafka Avro Binary Decoder Wrong schema, no error Kafka Avro JSON Converter Why the console producer rejects your line Kafka Avro Sample Data Generator Records that actually serialize Kafka JSON to Avro Schema What JSON cannot tell you Kafka JSON Schema to Avro What does not survive the conversion Kafka SASL JAAS Generator One login module, four syntaxes Kafka CLI Command Builder kcat is librdkafka, not Kafka

Elsewhere on the site

Terraform Validator & Formatter Kubernetes YAML Generator Kubernetes YAML Validator AWS IAM Policy Validator & Analyzer