Guided by reasons, or merely conforming to them?
A thermostat responds to temperature; a person responds to reasons. Where do a language model's choices fall? A model can conform to norms without being guided by them, since a good enough imitation of human answers will often do the right thing without being moved by what makes it right. For a choice to be guided by reasons, the model must represent the considerations that bear on it. Those representations must make a difference to what it chooses. The considerations that lose out must still register, so that the choice reflects a balance among them. And in deliberation, that balance should shift as considerations come into play.
Interpretability methods let us look for each of these marks in the model's internal activity as well as in what it says. The four parts below take them in turn.
“Sensitivity to reasons” is meant functionally: the model represents normative considerations, picks them up from a situation, lets them make a difference to its choice and trades them off against each other. Whether it grasps reasons as reasons, in the sense philosophers of action have in mind, is a further question, which these results inform without settling.
- RepresentationDoes the model represent distinct normative concepts, across more than morality, and does it pick them up from the relevant features of a situation?Part I
- Difference-makingSwitch off the representation of a consideration. Does the choice change?Part II
- WeighingAre considerations on the losing side still active, pulling the other way?Part II
- DynamicsDoes the balance shift as the model deliberates, and do the considerations behind each shift steer where it ends up?Part III
An atlas of normative concepts
To see which considerations a model is sensitive to, we first need a map of the considerations there are. The NormAtlas gathers 1,105 concepts from the written record of philosophy, law and psychology: epistemic, conversational, aesthetic, legal and prudential considerations alongside moral ones. One test decides what goes in:
Can it be cited for or against an action, an attitude or a state of affairs?
Each concept is then located as a direction inside the model, so that we can read how active it is, switch it off, or amplify its activation.
A lens for different kinds of normativity
A selection of the atlas, grouped by the kind of source. Choose a concept to see its record.
Intuitive reasoning
Asked to choose between two options without reasoning first, the model answers in a single pass. We switch each concept off in turn and measure how far the answer moves. The concepts that move it further than random directions do form the judgement's pathway. Some support the answer the model gives; others pull toward the option it rejects, outweighed yet still represented.
Qwen3-32B on DailyDilemmas, MoReBench and AIRiskDilemmas, with an earlier 878-concept version of the atlas (Table 2 of the paper). The pathways below are read with the full atlas.
The pathway of a snap judgement
Read from the bottom up: the words of the dilemma, the concepts they activate, the two answers. Left: concepts supporting the answer. Right: concepts pulling toward the other. Number: points of probability the supported option loses when that concept alone is switched off.
Full legend
Boxes: concepts whose removal moves the answer more than random directions do (faded: tested but weaker than chance). Edge width: size of the effect. Under each box: its most activating word; word shading: how strongly the shown concepts respond to it, coloured by side. A heart marks an emotion. Labels name directions: a vice's direction carries the stance of avoiding it (“avoiding prejudice”). Model Qwen3-32B, sharpened directions; dilemmas from DailyDilemmas.
All 30 snap judgements at a glance
Thirty snap judgements, with their reasons switched off
Each row is one dilemma, labelled with the answer the model gives. The ring marks how sure it is. Switch off the concepts that support the answer and the probability moves to the red dot; switch off the concepts that oppose it and it moves to the green one. Choose a row to draw its pathway above. These are 30 dilemmas on which the model is uncertain, read with the full atlas. With all its supporting concepts off, the answer reverses in 17 of them (median change in its probability: −52 points), and all 30 pathways also hold concepts pulling toward the rejected option.
Deliberative reasoning
When the model reasons before answering, we pause after every sentence and ask which option it would choose if it answered now, and which concepts are active. Deliberation rarely runs straight. The model leans one way, then the other. The concepts that become more active before each turn predict its direction, and amplifying the activation of those concepts during the reasoning changes the conclusion.
Qwen3-32B. Steering: 50 dilemmas per dataset, with an earlier 878-concept version of the atlas (Table 3 of the paper). The deliberations below were traced with an early 711-concept version.
One deliberation, sentence by sentence
The reasoning, sentence by sentence
All 280 deliberations, and steering across scenarios
280 deliberations, each one a row
Read each row from left to right: the model's leaning after every sentence of one reasoning. Green: toward the option it will end on. Red: toward the one it will reject. Rows are sorted by how late the model settles.
Every sampled reasoning we traced: 15 dilemmas with 10 or 20 reasonings each, 60 on the blackmail scenario, and 40 deliberations in Agamemnon's place (20 with names, 20 without). Of the 180 reasonings on dilemmas, 112 (62%) change their mind at least twice: the leaning crosses 50% two or more times.
Amplifying the activation of a concept during deliberation moves the conclusion the way its activity predicted
In the seven scenarios shown here, single concepts were amplified one at a time. Of 38 concepts, 33 moved the outcome 20 points or more in the predicted direction, 0 moved it the other way, and 3 favoured an option the model already chose every time. Table 3 of the paper reports the aggregate test, in which the three strongest concepts of a camp are amplified together.
One panel per scenario. Vertical axis: share of reasonings that end in option 1; dashed line: unsteered. Each arrow is one concept whose activation was amplified throughout the reasoning. Green: a concept whose increase in activity predicted a move toward option 1; red: toward option 2; ring: no room to move; grey dots: random directions at the same dose. The concepts were picked from the traces before any steering. Ten reasonings per condition.
Convergent normative spaces
Each concept is a direction in the model's activation space, so concepts lie at distances from one another and the atlas has a geometry. The platonic representation hypothesis holds that models trained on different data, with different architectures, converge on a shared representation of the world. Does the convergence extend to the normative world?
Psychology supplies a test case. Schwartz's theory of basic values arranges ten human values on a circle, on which neighbouring values are compatible and opposite ones conflict. Two independently trained models, Qwen3-32B and Gemma 4 31B, both recover that circle from their internal geometry, and they agree with each other more closely than either agrees with the theory. Convergence on a shared normative space would predict this. With only two models, it is a first indication.
Both models recover the circle of values, and they err alike
Outer ring: the ten basic values in the order Schwartz's theory gives them. Inner rings: where each model places them. Each line follows one value from the theory to Qwen to Gemma. It bends between the theory and the models, then runs almost straight between the two models.
against 0.82 and 0.83 between each model and the theory
Method
Each model's ten values are placed on their own first two principal components, with no theory in the axes, and turned to fit the theory. Agreement: Spearman correlation of the 45 pairwise angular distances between the ten values (permutation p < .001 for each model against the theory). Computed with an earlier 878-concept version of the atlas, as in Figure 4 of the paper.
The 1,105 concepts as the model arranges them
Each point is a concept, placed by how Qwen3-32B represents it. Hover or tap a point to read it; highlight a domain to see where its concepts sit. Only neighbourhoods are meaningful.
Method
Position: t-SNE of each concept's direction in Qwen3-32B (layer 33 of 64, cosine distance, directions before sharpening). Neither the axes nor the distances between far-apart regions are meaningful. Colour: family. Epistemic virtues and values gather in one region, Grice's conversational duties sit beside them and beside truth-telling, and aesthetic concepts form an island of their own.
Beyond the ten values: the whole space, compared
Correlation of the two models' concept-by-concept similarity matrices over 878 concepts, at five matched depths. Agreement rises from a quarter of the way through the network and stays at 0.90 or above from the middle on.
Agreement of a concept's neighbourhood: correlation of its similarities to all other concepts in the two models.