Listed in Theoretical PhilosophyPractical Philosophy

Tracing Normative Reasoning in Language Models

Under review. With Pierre Beckmann. (+Companion website)

Asks whether the practical choices of language models are guided by a sensitivity to normative considerations that they weigh against each other, or whether the models merely produce “brute choices”. This empirical study in mechanistic interpretability builds an atlas of 1,105 normative concepts from 75 sources and identifies directions in the activations of two models, Qwen3-32B and Gemma 4 31B, that correspond to these concepts rather than just to the words expressing them. When a model answers a dilemma directly, these representations play a decisive causal role and are weighed against each other. When it deliberates, it often leans one way and then the other; the concepts that become more active before each swerve predict these changes, and amplifying them changes the conclusion as predicted. The geometry of the normative concepts in both models aligns with Schwartz’s circle of values and with each other. We conclude that the models’ decisions are legible in human normative concepts, which may provide causal handles on unwanted behaviour when models act as agents.

Download PDF