It is not particularly helpful to think of an AI chatbot as an “essentially stupid” system. This assumption often leads people to write excessively detailed prompts, simply to cover or rule out every conceivable possibility just in case.
Working with AI becomes more effective if you instead think of it as the classic “absent-minded expert”: it knows exactly what would be required to produce the best possible result, but may fail to apply that knowledge unless explicitly prompted to do so. A precise instruction that activates this implicit knowledge is therefore often more effective than a lengthy prompt.
For example, if you ask an AI model to create a table using information drawn from various public sources, hallucinations cannot be ruled out. But if you ask how the instruction itself should be phrased to minimise hallucinations as far as possible, the model can provide very specific guidance on how to structure the task. In other words, the knowledge needed to compile information as accurately as possible is already there, but it remains implicit and may not be applied unless the instruction explicitly calls for it.
In practice, this means that a multi-step approach will generally produce better results than a single, highly detailed prompt. For example, if you want to compile and compare key figures for individual companies using publicly available annual reports and other documents, the process could look like this:
- In the first step, define the working instructions, for example how the output should be structured, that all claims must be supported by sources, or that any estimates generated by the model should be treated as errors.
- Only in the second step should you provide the subject context and/or the specific sources to be used. We have found it useful at this stage to focus initially on compiling facts or summarising content.
- More detailed questions involving analysis and interpretation should then be addressed in a third step.
The instructions for each stage can be developed iteratively in advance by asking the model how they should best be formulated or by specifying the intended outcome. Frequently used instructions should also be documented outside the AI system in order to build up a reusable prompt library.
By Markus Hoffmann