In the coming weeks, according to OpenAI, AI-generated texts produced by ChatGPT and Codex within the European Union are set to include invisible watermarks. This is a response to the transparency requirements of the EU AI Act, which calls for machine-readable labelling of AI-generated content.
Watermarks are statistical patterns. The technology, called textGrain, embeds statistical patterns into the AI’s choice of words. Special detection systems can use these patterns to identify whether a text was generated with an OpenAI model. The technology therefore sits alongside existing watermarking approaches such as Google’s SynthID.
However, OpenAI explicitly acknowledges the limitations of the technology. In tests involving texts of 400 tokens, around 95% of watermarks were detected. For shorter texts of 200 tokens, the detection rate for textGrain fell to 80%. Even relatively minor changes can significantly reduce reliability: in a separate test, replacing 25% of words with synonyms reduced the detection rate from 92% to 17%.
Does adding statistical patterns leave the quality of the content unchanged? OpenAI says it found no significant differences across a range of benchmarks. Performance was up to 0.6% worse, while in some cases it was even up to 3% better.
But who can currently verify this? While tools for verifying images and audio files are openly available, access to the text detector is initially restricted to selected research institutions and specialist organisations. Watermark detectors – or tools designed to remove watermarks – could, however, create a new market in the near future and introduce new ways of assessing online content.
So what does this mean for professional research and information assessment?
Watermarks could provide an additional indication of where a text comes from, but for shorter passages they are not reliable proof either for or against the use of AI. Even if the watermarking method itself is disclosed, it is currently not possible to say precisely how these statistical adjustments affect the meaning of a particular text. What can be said is that such small changes may alter the density and precision of the information conveyed – potentially even diluting it. This means that not only critical source evaluation, but also careful scrutiny of the AI-generated output itself, remains essential.
By Marcel Ohrenschall