Watermarking is a provenance signal, not a safety control - it does not moderate, refuse, or prevent anything
[minor] security
Fields
id
gotchas/sec-watermark-provenance-is-not-a-compliance-controlClass
security
Severity
minor
Concepts
watermarkprovenancedetectioncompliancesafetycontent-moderation
Symptom
A deployment adds a watermark and reports the content-safety problem as solved. The watermark changes which tokens get sampled; it has no ability to refuse, to filter, or to detect harm at generation time. Harmful content is watermarked exactly as faithfully as benign content.
Root cause
Watermarking operates on the sampling distribution to plant a statistical signal for later verification. It is orthogonal to harm: the mechanism that promotes green tokens has no notion of what the tokens mean. Its output is an attribution claim ('this came from model X') usable only after the fact and only by someone holding the key - which makes it a compliance ARTEFACT rather than a runtime control. The distinction the field itself draws is between safety training (cannot adapt to new attack types, drops performance) and safeguards (external models or filters, of limited help); watermarking is neither.
Workaround
If the requirement is 'users must be able to tell AI text from human text', watermarking is a reasonable mechanism and EU AI Act-style marking obligations are what it is for. If the requirement is 'do not serve harmful content', watermarking contributes nothing and you still need in-decode filtering, a guard model, and over-refusal measurement.
notes
The compliance framing is a trap worth naming explicitly: a watermark gives you a defensible answer to 'how do you mark machine-generated content', and it is easy to record that as having addressed content safety. It has not. SCHEME RECLASSIFIED (2026-10-04): was 'config'. Filed as security because the subject is the watermark mechanism itself; the record argues the watermark is NOT a compliance control, which is a claim about watermarking, not about an audit or retention obligation.
Notes
The compliance framing is a trap worth naming explicitly: a watermark gives you a defensible answer to 'how do you mark machine-generated content', and it is easy to record that as having addressed content safety. It has not. SCHEME RECLASSIFIED (2026-10-04): was 'config'. Filed as security because the subject is the watermark mechanism itself; the record argues the watermark is NOT a compliance control, which is a claim about watermarking, not about an audit or retention obligation.
Cross-references
This record references no other records — it is a leaf. It is cited by 0.