Even a small number of ‘poisoned documents’ can manipulate language models

—

© [2025] [𝘒𝘐-𝘎𝘦𝘯.] 𝘔𝘢𝘳𝘤𝘦𝘭 𝘖𝘩𝘳𝘦𝘯𝘴𝘤𝘩𝘢𝘭𝘭. 𝘉𝘢𝘴𝘪𝘦𝘳𝘦𝘯𝘥 𝘢𝘶𝘧 𝘞𝘦𝘳𝘬𝘦𝘯 𝘷𝘰𝘯 𝘈𝘯𝘥𝘳𝘦𝘢𝘴 𝘖𝘩𝘳𝘦𝘯𝘴𝘤𝘩𝘢𝘭𝘭.

New research into the security of large AI language models shows that even a very small number of “poisoned documents” can be enough to inject harmful instructions into language models. What matters is not the percentage of poisoned documents within the training data, but their absolute number. As a result, increasing model size does not necessarily provide better protection. On the contrary, language models with high learning efficiency may be particularly vulnerable because they are also good at identifying rare but consistent patterns in training data.

This means that online disinformation campaigns could become a problem when information researched using language models is reused. A relatively small number of sources containing consistently false information could already be enough to cause a language model to generate incorrect answers.

By Markus Hoffmann

Source: heise.de, 15 October 2025, by Dr Volker Zota