ChatGPT's Reasoning Limitations Revealed in WSU Study
Analysis based on 7 articles · First reported Mar 16, 2026 · Last updated Mar 18, 2026
The study's findings on ChatGPT's limitations in reasoning and consistency could temper investor enthusiasm for AI companies, particularly those focused on large language models. It suggests that while AI is powerful, its current capabilities may not meet the high expectations for advanced conceptual understanding, potentially leading to more cautious investment in the short term.
A study led by Mesut Cicek of Washington State University evaluated ChatGPT's ability to determine the truthfulness of scientific hypotheses. The AI, tested on over 700 hypotheses, achieved 76.5% accuracy in 2024 and 80% in 2025. However, after adjusting for random guessing, its performance was only about 60% better than chance. ChatGPT struggled significantly with identifying false statements (16.4% accuracy) and showed notable inconsistency, providing different answers to identical prompts 27% of the time. The researchers concluded that current AI tools lack true conceptual understanding and merely memorize, suggesting that artificial general intelligence is still distant. The study, published in the Rutgers Business Review, recommends caution for business leaders relying on AI for complex decisions.
Set up alerts, explore entity relationships, search across thousands of events, and build custom intelligence feeds.
Open Dashboard