Until now, the guiding principle for LLMs has largely been: the more data, the better the output. But is that really a sustainable long-term solution, or is a different approach needed? Two limitations suggest that the principle of “more data = better output” cannot continue indefinitely:
- By 2026, prominent AI models are expected to have captured virtually the entire publicly accessible internet as training data. (EpochAI)
- The technical limits of LLMs in terms of quality, consistency, logic and reproducibility are becoming increasingly apparent. (Cambridge University)
These limitations stem from the probabilistic nature of the models. In other words, more data may bring a model closer to the correct answer, but never guarantee it with complete certainty. Yet industry-specific applications often require precision and deterministic reliability. Two trends are therefore emerging as possible ways of addressing these limitations:
- Not more data, but more specific data for better outputs. According to the CEO of Mistral, companies will increasingly need to rely on their own industry-specific data and combine it with customised open-source models to gain a qualitative advantage over models from Anthropic, Google or OpenAI. The implication for companies is clear: to maintain long-term sovereignty and reduce dependence on external providers, building internal AI capabilities and developing company-specific AI solutions will become increasingly important.
- Using data more effectively through alternative model architectures. Could it be possible to move beyond probabilistic logic and develop models that shift from statistical pattern recognition towards more explicit forms of knowledge representation? Research into this area is already under way. One focus is on AI systems that combine the probabilistic learning capabilities of neural networks with the deterministic logic of rule-based systems: so-called neuro-symbolic AI. Companies such as Amazon, IBM and Microsoft are already applying this approach in selected areas, with Amazon, for example, using it in parts of its logistics operations. The challenges involved are outlined clearly in this article by the European Data Protection Supervisor (EDPS).
The future is therefore likely to lie in combining specialised data with specialised models. The general-purpose “universal AI” approach may increasingly give way to tailored solutions.
By Marcel Ohrenschall
![LLMs are reaching their limits: do we need more data or a new approach? 1 © [2025] [𝘒𝘐-𝘎𝘦𝘯.] 𝘔𝘢𝘳𝘤𝘦𝘭 𝘖𝘩𝘳𝘦𝘯𝘴𝘤𝘩𝘢𝘭𝘭. 𝘉𝘢𝘴𝘪𝘦𝘳𝘦𝘯𝘥 𝘢𝘶𝘧 𝘞𝘦𝘳𝘬𝘦𝘯 𝘷𝘰𝘯 𝘈𝘯𝘥𝘳𝘦𝘢𝘴 𝘖𝘩𝘳𝘦𝘯𝘴𝘤𝘩𝘢𝘭𝘭.](https://research-ki.de/wp-content/uploads/2026/02/isometric_view_on_a_brain_ink_drawing_detailed_realist_29f52128-01a9-4dec-9819-13706b5d4c55_3.png)