Decoding Large Language Models: How SPEX and ProxySPEX Uncover Critical Interactions at Scale
Large Language Models (LLMs) are powerful but notoriously opaque. Understanding why they produce specific outputs is essential for safety, trust, and improvement. Interpretability research tackles this through various lenses—feature attribution, data attribution, and mechanistic interpretability. However, all these approaches face a common enemy: complexity. Model behavior emerges from countless interactions between features, training examples, and internal components. Exhaustively analyzing these interactions is computationally impossible. This article introduces SPEX and ProxySPEX, two algorithms designed to identify the most influential interactions efficiently, using a clever ablation-based strategy.
What makes LLM interpretability so challenging?
LLMs achieve state-of-the-art performance by synthesizing complex relationships between input features, drawing on patterns from millions of training examples, and processing information through deeply interconnected internal components. Their behavior is rarely the result of any single element; instead, it emerges from intricate dependencies and interactions. For instance, a prediction might depend on the combined presence of certain words, data points, and neural circuits. To truly understand a model, we need to capture these influential interactions—but the number of potential interactions grows exponentially as the model scales. That makes exhaustive analysis computationally infeasible. This is the central challenge: how to identify the most important interactions without testing every possibility.
What are the three interpretability lenses used to study LLMs?
Interpretability researchers analyze LLMs from three complementary perspectives. First, feature attribution aims to pinpoint which input features (e.g., tokens, phrases) most drive a prediction. Methods like SHAP and LIME belong here. Second, data attribution connects model outputs to the training examples that most influenced them—essentially asking, “Which data points made the model behave this way?” Third, mechanistic interpretability dives into the model’s internal components (neurons, attention heads, layers) to understand the role each plays. While each lens offers unique insights, they all face the same scaling hurdle: as the number of features, data points, or components increases, the interactions among them explode. Any practical method must identify the most critical interactions without brute-force enumeration.
How does ablation help uncover model behavior?
Ablation is a core technique in interpretability. The idea is simple: remove or deactivate a component (e.g., a token, a training example, or a neuron) and measure how the model’s output changes. The larger the change, the more influential that component is. For feature attribution, we mask parts of the input prompt and observe prediction shifts. For data attribution, we train models on subsets of the data, omitting certain examples to see their impact on test outputs. For mechanistic interpretability, we intervene directly in the forward pass—zeroing out a specific internal unit. In all cases, the goal is to isolate drivers of a decision by systematically perturbing the system. The catch: each ablation costs time and compute (e.g., an extra inference or a full retraining). So we want to identify influential interactions with as few ablations as possible.
Why is scaling a problem for interaction analysis?
When we talk about “interactions,” we mean dependencies between multiple components—for example, two tokens together influencing a prediction more than either alone. As the number of features grows, the number of possible pairs grows quadratically, triplets cubically, and so on. For a typical LLM with a vocabulary of tens of thousands of tokens or millions of data points, enumerating all interactions is astronomically expensive. Even if each interaction check required only one ablation (it often requires more), the total cost would be prohibitive. That’s why we need methods that can prune the search space—focusing only on those interactions most likely to be influential. This is the problem that SPEX and ProxySPEX were designed to solve.
What are SPEX and ProxySPEX, and how do they work?
SPEX (Sparse Interaction Extraction) and ProxySPEX are algorithms that efficiently identify the most influential interactions in LLMs, regardless of the interpretability lens (feature, data, or mechanism). They are grounded in the idea that not all interactions matter. By using a sparse modeling approach, they approximate the complex interplay of components with a limited set of key pairwise (or higher-order) terms. ProxySPEX goes a step further: it learns a cheap proxy model of the original LLM’s behavior and then applies SPEX on that proxy. This dramatically reduces the number of expensive ablations needed. The algorithms output a shortlist of the interactions that most affect model outputs, allowing researchers to focus their analysis on what truly matters.
How does SPEX handle the exponential growth of possible interactions?
SPEX avoids enumerating all interactions by formulating the problem as a sparse recovery task. It assumes that only a small fraction of all possible interactions significantly affect model behavior. Under this assumption, SPEX can recover these influential interactions using a limited number of ablation experiments—much like compressed sensing reconstructs a signal from few measurements. The algorithm borrows ideas from statistics and signal processing (e.g., Lasso regression) to select a parsimonious set of interaction terms. ProxySPEX extends this by first training a lightweight surrogate model—like a linear model with pairwise features—that mimics the LLM’s outputs. Because the proxy is cheap to evaluate, SPEX can run many iterations on it to zero in on the critical interactions. Both methods are designed to scale gracefully as the number of components grows.
Can these methods be applied to different interpretability perspectives?
Yes—and that’s a key strength of SPEX and ProxySPEX. The ablation framework is lens-agnostic. For feature attribution, the “components” are input tokens; for data attribution, they are training examples; for mechanistic interpretability, they are internal units. In each case, we define a way to ablate a component (mask token, exclude example, silence neuron) and measure the resulting shift. SPEX then works on the resulting influence measurements. The only requirement is that we can perform ablations and obtain a scalar output difference. This versatility allows researchers to combine insights from all three lenses, painting a fuller picture of model behavior. For example, one might discover that a specific training example interacts with a particular attention head to produce a biased prediction—a finding no single lens would reveal on its own.