Explainable AI
4 writings
A Polyphonic Conception of AI Understanding
With Pierre Beckmann.
Argues that whether large language models understand what they talk about is a question both unavoidable and ill-framed: unavoidable because users must tell outputs that merit epistemic trust from those that do not, and no purely mathematical or statistical vocabulary can draw that line without reintroducing the concept of understanding in all but name; ill-framed because the inherited concept presupposes a single locus of understanding. Mechanistic interpretability instead reveals LLMs to be deeply polyphonic systems whose outputs emerge from parallel mechanisms of uneven reliability. Drawing on evidence of medical, mathematical, logical, and spatial understanding in LLMs, the paper engineers a conception fit for such systems: a model understands a domain insofar as it contains sound circuitry that tracks relevant structure and reliably recruits that circuitry to control its outputs—which turns seemingly idle metaphysical disputes into tractable empirical questions that can guide the allocation of trust.
Download PDF
Why We Care about Understanding: Competence through Predictive Compression
With Pierre Beckmann.
Offers a unifying account of understanding by reverse-engineering the function of both the state and the concept. Argues that we care about understanding because it grounds robust competence. Our concept of understanding evolved as an efficient proxy to track this elusive property, allowing us to identify who to trust and learn from. This highlights the sociality of understanding and how it shapes the character of human understanding. Understanding is the result of convergent pressures to predict the world using cognitive models that are not only accurate, but also compressed enough to be stored, demonstrated, and transmitted.
Download PDF
Mechanistic Indicators of Understanding in Large Language Models
Philosophical Studies. 2026. With Pierre Beckmann. doi:10.1007/s11098-026-02513-1
Draws on detailed technical evidence from research on mechanistic interpretability (MI) to argue that while LLMs differ profoundly from human cognition, they do more than tally up word co-occurrences: they form internal structures that are fruitfully compared to different forms of human understanding, such as conceptual, factual, and principled understanding. We synthesize MI’s most relevant findings to date while embedding them within an integrative theoretical framework for thinking about understanding in LLMs. As the phenomenon of “parallel mechanisms” shows, however, the differences between LLMs and human cognition are as philosophically fruitful to consider as the similarities.
Download PDF
Explainability through Systematicity: The Hard Systematicity Challenge for Artificial Intelligence
Minds and Machines 35 (35): 1–39. 2025. doi:10.1007/s11023-025-09738-9
Offers a framework for thinking about “the systematicity of thought” that distinguishes four senses of the phrase, defuses the alleged tension between systematicity and connectionism that Fodor and Pylyshyn influentially diagnosed, and identifies a “hard” form of the systematicity challenge that continues to defy connectionist models.
Download PDF