Describe a record, paste your predicate and transform config, and get told
which transforms actually run. negate inverts the gate rather
than the predicate, and the two settings read backwards together, which is
what this exists to settle.
One name: value per line. Write value: null
literally to describe a tombstone, which is the one thing a JSON
description could not express unambiguously. Headers are
header.<name>:.
Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.
The answer appears here
Paste on the left and press Evaluate. Nothing leaves this tab.
Nothing else to flag.
No formatting problems, and nothing the rules object to. Worth remembering what that covers: this reads the file you pasted, not the account or cluster it will be applied to.
No finding matches that filter.
Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.
A predicate wired to a transform, and what happens to records that do not match
predicates=isTombstone predicates.isTombstone.type=org.apache.kafka.connect.transforms.predicates.RecordIsTombstone transforms=drop transforms.drop.predicate=isTombstone
negate inverts the match, which is the usual source of an inverted transform chain
predicates=isOrders predicates.isOrders.type=org.apache.kafka.connect.transforms.predicates.TopicNameMatches predicates.isOrders.pattern=orders.* transforms=mask transforms.mask.predicate=isOrders transforms.mask.negate=true
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
It does not. A predicate decides whether the TRANSFORM applies; records that do not match pass through unchanged.
Instead:Use the Filter transform with a predicate to actually drop records.
negate: true inverts the predicate, and it is easy to read a chain as doing the opposite of what it does.
Instead:Check every predicate for negate before reasoning about the chain.
They apply in the order named in the transforms list, and a predicate evaluates against the record as it is at that point, not the original.
Instead:Name them in execution order and test with a real record.
This is the sentence the whole page turns on. When a predicate does not match, the one transform it is attached to is skipped and the record carries on through the rest of the chain unchanged. Nothing is dropped unless the transform you gated was Filter.
transforms.drop.predicate=isTombstone with negate=true on a Filter transform drops everything that is NOT a tombstone. Read the two settings together and it sounds like it drops tombstones, which is the opposite. The rule is: negate=true means run this transform unless the predicate matches. This page states the outcome for the record you describe rather than leaving you to reason it out.
transforms=drop
transforms.drop.type=org.apache.kafka.connect.transforms.Filter
transforms.drop.predicate=isTombstone
transforms.drop.negate=trueThe pattern is a Java regex and Java anchors the whole string, so orders does not match the topic orders-eu. Almost every other tool in this space does a substring search, which is why a pattern that works everywhere else silently matches nothing here. Write orders.* if a prefix is what you meant.
RecordIsTombstone tests whether the record's value is null. An empty value is not a tombstone: it is a zero-length value and the predicate returns false. A null key has nothing to do with it at all. Compaction removes a key when it sees a null value, so getting this wrong means either keeping records you meant to delete or deleting ones you meant to keep.
It is true when a header with that name exists, whatever it contains, including nothing. There is no way to test a header's value with the predicates Kafka ships: that needs a custom predicate or a transform that can inspect it. Header names are case-sensitive, which catches people who assume HTTP conventions.
Defining one in predicates and never naming it from a transforms.<alias>.predicate is silently inert. Connect raises no error and logs no warning, so a chain that looks conditional runs unconditionally on every record. This page reports it, because it is invisible everywhere else.
Custom or third-party predicate classes, because it implements the three Apache Kafka ships and cannot evaluate one it does not have. What the gated transform actually does to the record, which is the transform's business. And whether your record description matches real traffic, since it is a description rather than a sample off the topic.