A Polyphonic Conception of AI Understanding

With Pierre Beckmann.

Argues that whether large language models understand what they talk about is a question both unavoidable and ill-framed: unavoidable because users must tell outputs that merit epistemic trust from those that do not, and no purely mathematical or statistical vocabulary can draw that line without reintroducing the concept of understanding in all but name; ill-framed because the inherited concept presupposes a single locus of understanding. Mechanistic interpretability instead reveals LLMs to be deeply polyphonic systems whose outputs emerge from parallel mechanisms of uneven reliability. Drawing on evidence of medical, mathematical, logical, and spatial understanding in LLMs, the paper engineers a conception fit for such systems: a model understands a domain insofar as it contains sound circuitry that tracks relevant structure and reliably recruits that circuitry to control its outputs—which turns seemingly idle metaphysical disputes into tractable empirical questions that can guide the allocation of trust.

AI, LLM, understanding, mechanistic interpretability, conceptual engineering, conceptual adaptation, explainable AI