University of Bristol: Alice in Wonderland – Simple Tasks Showing Complete Reasoning Breakdown in State-of-the-Art LLMs

Summary of https://arxiv.org/pdf/2406.02061

Introduces the "Alice in Wonderland" (AIW) problem, a seemingly simple common-sense reasoning task, to evaluate the capabilities of state-of-the-art Large Language Models (LLMs). The authors demonstrate that even advanced models like GPT-4 and Claude 3 Opus exhibit a dramatic breakdown in generalization and basic reasoning when faced with minor variations of the AIW problem that do not alter its core structure or difficulty.

This breakdown is characterized by low average performance and significant fluctuations in accuracy across these variations, alongside overconfident, yet incorrect, explanations. The study further reveals that standardized benchmarks fail to detect these limitations, suggesting a potential overestimation of current LLM reasoning abilities, possibly due to data contamination or insufficient challenge diversity.

Ultimately, the AIW problem is presented as a valuable tool for uncovering fundamental weaknesses in LLMs' generalization and reasoning skills that are not apparent in current evaluation methods.

• Despite achieving high scores on various standardized benchmarks, many state-of-the-art Large Language Models (LLMs) exhibit surprisingly low correct response rates on the seemingly simple "Alice has brothers and sisters" (AIW) problem and its variations. Only a few large-scale closed models like GPT-4o and Claude 3 Opus show relatively better performance, while many others, including models claiming strong function, struggle significantly, sometimes even collapsing to a zero correct response rate.

• The document highlights a significant discrepancy between the performance of LLMs on standardized reasoning benchmarks and on the AIW problem, suggesting that current benchmarks may not accurately reflect true generalization and basic reasoning skills. Models that score highly on benchmarks like MMLU, MATH, ARC-c, GSM8K, and HellaSwag often perform poorly on AIW, indicating a potential issue with the benchmarks' ability to detect fundamental deficits in model function. This suggests that these benchmarks might suffer from issues like test data leakage.

• A key observation is the lack of robustness in SOTA LLMs, evidenced by strong performance fluctuations across structure and difficulty-preserving variations of the same AIW problem. Even slight changes in the numerical values within the problem statement can lead to drastically different correct response rates for many models. This sensitivity to minor variations points to underlying generalization deficits.

• The study reveals that LLMs often exhibit overconfidence and provide persuasive, explanation-like confabulations even when their answers to AIW problems are incorrect. This can mislead users into trusting wrong responses, especially in situations where verification is difficult. Furthermore, many models struggle to properly detect mistakes and revise their incorrect solutions, even when encouraged to do so.

• The AIW problem and its variations are presented as valuable tools for evaluating the robustness and generalization capabilities of LLMs, offering a method to reveal weaknesses that are not captured by standard benchmarks. The ability to create numerous diverse problem instances through variations addresses potential test set leakage issues. The introduction of a unified robustness score (R) is proposed to provide a more accurate model ranking by considering both average correct response rate and the degree of performance fluctuations across problem variations.