A New Lens on LLM Judgment
In a new paper by Loru et al. (2025), researchers explore a provocative question: Do large language models (LLMs) actually make judgments, or are they simply simulating them? Their findings suggest the latter. The study demonstrates that LLMs do not perform deliberative reasoning but instead lean heavily on probabilistic associations. They essentially mimic what they’ve been trained on, i. e. patterns from massive datasets, and produce outputs that sound like judgments, but lack the underlying reasoning structure that characterizes human decision-making.
This distinction between simulation and real judgment has profound implications in a world increasingly shaped by generative AI. For instance, when LLMs help draft legal arguments, medical advice, or policy proposals, their “conclusions” can carry persuasive force, even though they haven’t engaged in genuine reasoning. This raises questions about trust, accountability, and responsibility: Who is ultimately responsible for a model’s advice or “decision”? And can we allow systems whose reasoning is fundamentally different from human reasoning to play influential societal roles?
Why This Matters for Digital Sustainability
At the heart of the Digital Sustainability Hub’s mission lie three dimensions: Understanding, Awareness, and Action, applied across the pillars of the Sustainable Web Manifesto. Loru et al.’s study touches each of these:
Understanding: By revealing that LLMs simulate rather than reason, the research deepens our comprehension of how these systems function. It demystifies a commonly held belief that advanced models are approaching human-level cognition.
Awareness: Stakeholders, from policymakers to end-users, must be conscious of the limitations of AI-generated judgments. Presenting LLM outputs as decisions rather than simulations risks over-trusting these systems.
Action: This insight calls for stronger governance frameworks. If LLMs are merely simulating judgment, we need policies that reflect their nature, such as transparency about how outputs are generated, liability mechanisms, and clear boundaries for their use in domains where misjudgment could be harmful.
Ethical and Social Risks
Because LLM judgments are not based on understanding, they may amplify biases present in training data. The simulation of consensus might obscure the absence of true reasoned debate. Additionally, if people treat these simulated judgments as authoritative, there is a risk of overreliance, especially in sensitive contexts. Loru et al. argue that without proper guardrails, we might inadvertently grant too much epistemic authority to systems that lack genuine moral or rational grounding.
Building Responsible AI with Simulation Awareness
How can we act on these findings? A few directions emerge:
Design for transparency: Systems should clearly communicate when they are simulating judgment rather than reasoning. User interfaces might indicate the probabilistic nature of a recommendation or decision.
Limit deployment in high-stakes domains: For areas like legal advice, medical diagnostics, or public policy, reliance on LLM “judgments” should be limited or mediated by human experts.
Regulatory frameworks: Policymakers should develop governance models that reflect the gap between simulation and reasoning. This could include certification, auditing, or monitoring mechanisms to track how LLMs are used in practice.
A Call for Reflection in the Sustainability Community
For the Digital Sustainability Hub, Loru et al.’s work invites reflection on how we conceptualize “intelligent” systems in our vision of a sustainable digital future. Sustainability is not only about minimizing environmental impact but also about maintaining ethical integrity, social trust, and equitable governance in digital infrastructures. If powerful AI systems are widely accepted as decision-makers without scrutiny, we risk building unsustainable trust, i. e. trust not anchored in real understanding.
This study is a timely reminder: the road to digital sustainability must include epistemic sustainability, ensuring that the ways we rely on AI are grounded in honest representations of what these machines can and cannot do.
Full Paper:
pnas.org