A subscription pattern
Which topics a regex subscription matches, and when the consumer notices a new one
orders.created orders.shipped orders-legacy inventory.updated
Kafka Topic Regex Subscription Tester
Test a Kafka pattern subscription against a list of topic names. Anchored at both ends, because Kafka calls Matcher.matches(), and every construct where Java's regex differs from JavaScript's is reported rather than glossed over.
Do not add ^ and $: Kafka matches the whole
name already, and this page anchors the pattern for you so what you see is
what the consumer would subscribe to.
Paste below, or drop a file anywhere on this panel
Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.
The answer appears here
Paste on the left and press Test pattern. Nothing leaves this tab.
Formatted
Results
Nothing else to flag.
No formatting problems, and nothing the rules object to. Worth remembering what that covers: this reads the file you pasted, not the account or cluster it will be applied to.
No finding matches that filter.
Examples
Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.
An unescaped dot
A dot matches any character, so orders.created also matches ordersXcreated
orders.created ordersXcreated orders-created
Common mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Leaving a dot unescaped
In a regex a dot matches any character, so orders.created also matches ordersXcreated.
Instead:Escape it as orders\\.created.
Expecting a new topic to be picked up immediately
A pattern subscription re-evaluates on a metadata refresh, controlled by metadata.max.age.ms, which defaults to five minutes.
Instead:Lower it if faster discovery matters, and accept the extra metadata traffic.
Using a pattern that matches internal topics
A broad pattern can match __consumer_offsets or changelog topics, which a consumer should never read.
Instead:Anchor the pattern, and exclude names starting with two underscores.
Kafka matches the whole topic name, and it uses Java's regex
Two facts, and each one causes a different silent failure. A pattern subscription that matches nothing is not an error in Kafka.
It is a full match, not a search
Kafka calls Matcher.matches(), which requires the pattern to match the entire topic name. So a pattern of orders does not subscribe to orders-v2, orders.eu or orders_dlq. Nothing warns you: the consumer joins the group, is assigned no partitions for those topics, polls successfully and returns no records, which looks exactly like an upstream that has gone quiet. This page anchors the pattern at both ends before testing, which is why the answer here can be narrower than you expect.
Java and JavaScript disagree, sometimes silently
Kafka uses java.util.regex and this page runs in a browser, so the engines are not the same. Some Java constructs simply fail to compile here, which is obvious. The dangerous ones compile in both and mean different things: \v is a class of vertical whitespace in Java and exactly one character in JavaScript, [a-z&&[^m]] is a class intersection in Java and two literal ampersands in JavaScript, and \Q ... \E quotes literally in Java and does nothing in JavaScript. Every one of those is reported as a finding rather than quietly assumed equivalent, the same way the JSONPath tester on this site handles dialect differences.
A new topic is picked up on the next metadata refresh
The consumer re-evaluates the pattern when it refreshes metadata, which is metadata.max.age.ms and defaults to five minutes. A topic created now can therefore take that long to be subscribed, and while you are waiting it looks like the pattern is wrong. Lowering it costs more metadata requests against the brokers.
A pattern of .* is rarely what you want
It covers every topic in the cluster including every topic created in future, so an unrelated new topic rebalances your group, and on a large cluster the group can be assigned far more partitions than it can keep up with. Internal topics are excluded separately by exclude.internal.topics, which defaults to true, so __consumer_offsets is not visible to a pattern subscription unless you turn that off.
What this cannot see
It does not know your cluster, so the topic list is the one you type. With ACLs in force a pattern subscription only sees topics the principal is authorized to describe, which means a correct pattern can still match nothing in production and everything in a test cluster. It also cannot run Java, so where the two engines differ this page tells you rather than pretending.
Related kafka tools
Kafka Connect SMT Predicate Tester negate reads backwards Kafka Topic Name Validator Legal, risky, or 249 characters too long? Kafka Streams Internal Topic Predictor Create them before Streams does Kafka Topic Config Generator min.insync.replicas is the one that matters Kafka Confluent Wire Format Decoder The five junk bytes in front of your payload Kafka Key to Partition Mapper Which partition does this key land on?
All 73 kafka tools