SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models
1 New Jersey Institute of Technology, 2 Rutgers University, 3 The Chinese University of Hong Kong, Shenzhen

Overview
SAGE turns sparse autoencoder feature interpretation into an iterative process. Explainer agents propose possible descriptions, design targeted tests, inspect activation feedback, and refine their hypotheses. A final synthesis combines the supported explanations and evidence into a description of the feature.
Results
Table 2 of the paper compares SAGE with Neuronpedia on three language models. The selected layer-3 results below show higher generative and predictive accuracy for SAGE in each model.
| Model | Method | Generative accuracy ↑ | Predictive accuracy ↑ |
|---|---|---|---|
| GPT-OSS-20B | Neuronpedia | 0.26 | 0.62 |
| GPT-OSS-20B | SAGE | 0.59 | 0.80 |
| Qwen3-4B | Neuronpedia | 0.22 | 0.68 |
| Qwen3-4B | SAGE | 0.54 | 0.72 |
| Gemma-2-2B | Neuronpedia | 0.75 | 0.68 |
| Gemma-2-2B | SAGE | 0.97 | 0.83 |
Qualitative Explanation Comparison
Table 3 shows how activation-guided tests narrow broad feature descriptions into more specific, testable explanations.
| Model / feature | Neuronpedia description | SAGE explanation |
|---|---|---|
| Gemma-2-2B · 11/5125 | Multithreading synchronization and thread safety. | Code locking constructs in Python, C++, and Java; ordinary uses of “lock” do not trigger the feature. |
| Qwen3-4B · 23/24625 | English negative contractions involving “n’t”. | A narrower detector for the “can’t” contraction suffix; tests with “won’t” and “don’t” refute the broader description. |
| GPT-OSS-20B · 3/121075 | Land-related contexts such as habitats and planets. | Lexical or morpheme-level response to “terrestrial” and related fragments. |
| Gemma-2-9B-IT · 20/1 | Java exception and error-handling references. | A lexical detector focused on “IllegalArgumentException” and related surface forms. |
BibTeX
@misc{han2026sage,
title={SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models},
author={Han, Jiaojiao and Xu, Wujiang and Jin, Mingyu and Du, Mengnan},
year={2026},
eprint={2511.20820},
archivePrefix={arXiv}
}