OpenAI explains why language models ‘hallucinate’; evaluation incentives reward guessing over uncertainty

OpenAI finds a key problem in how large language models work. These models often give wrong information confidently. The issue is in how these models are trained and checked. Current methods reward guessing, even if uncertain. OpenAI suggests new ways to test models. These methods should value uncertainty. The goal is to make AI more reliable.