The biggest weakness of AI search agents appears to lie not in the research itself, but in recognising ambiguous requests early enough and asking users targeted follow-up questions. This is the conclusion of a new benchmark developed by researchers from Tencent Hunyuan and Tsinghua University.
As AI agents increasingly take on complex information tasks involving research, knowledge retrieval and multi-step reasoning, it is important to assess how robustly they perform in realistic search scenarios.
This is exactly what DiscoBench is designed to test. While previous benchmarks have generally assumed clear and fully specified queries, DiscoBench reflects real-world conditions more closely: requests may be vague, incomplete or even factually incorrect. The benchmark therefore examines whether language models can identify ambiguities on their own during longer search processes, ask appropriate follow-up questions and adapt their research accordingly. In complex tasks, an initial ambiguity can otherwise cause an agent to drift further and further in the wrong direction.
DiscoBench consists of 211 tasks containing a total of 463 points of ambiguity across 11 knowledge domains. Each task is divided into several checkpoints at which the agent must decide whether to continue researching, ask the user for clarification or provide an answer.
Eleven models were tested, including Claude Opus 4.7, GPT-5.4, Gemini 3.1 Pro Preview, Doubao Seed 2.0 Pro and DeepSeek V4 Pro. Overall, all models achieved accuracy rates below 50%. Doubao Seed 2.0 Pro recorded the highest end-to-end accuracy at just over 43%, followed by Gemini 3.1 Pro and Claude Opus 4.7 at around 40% each. These results were achieved without explicitly warning the models that queries might be ambiguous. When the system prompt instructed them to watch for ambiguity and ask follow-up questions where necessary, average accuracy improved from 28.6% to 33.7%. The effect remained limited, however: although the models identified ambiguities more often, this did not automatically lead to more successful research outcomes.
Two findings were particularly notable:
- The gap between performance at individual checkpoints and the final result showed how vulnerable multi-step search processes are. Claude Opus 4.7, for example, completed 57% of intermediate steps correctly but achieved an overall accuracy of only just under 40%. Even when many individual steps are successful, a single unresolved ambiguity can distort the final result.
- Some models repeatedly ran additional searches without involving the user. According to the benchmark, this suggests that the uncertainty may have been recognised internally but was not translated into a follow-up question. More search iterations therefore did not necessarily produce better results.
What does this mean for the use of AI in professional research?
What matters is not only whether an agent notices ambiguity, but also whether it can formulate useful follow-up questions that genuinely move the research forward. Overall, DiscoBench suggests that future AI search agents will need not only strong retrieval and reasoning capabilities, but also mechanisms that can reliably turn uncertainty into useful interactions with users.
By Ida Kandler