Extracting Concepts from GPT-4
Using new techniques for scaling sparse autoencoders, we automatically identified 16 million patterns in GPT-4's computations.
Unlike with most human creations, we don’t really understand the inner workings of neural networks. For example, engineers can directly design, assess, and fix cars based on the specifications of their components, ensuring safety and performance. However, neural networks are not designed directly; we instead design the algorithms that train them. The resulting networks are not well understood and cannot be easily decomposed into identifiable parts. This means we cannot reason about AI safety the same way we reason about something like car safety.
In order to understand and interpret neural networks, we first need to find useful building blocks for neural computations. Unfortunately, the neural activations inside a language model activate with unpredictable patterns, seemingly representing many concepts simultaneously. They also activate densely, meaning each activation is always firing on each input. But real world concepts are very sparse—in any given context, only a small fraction of all concepts are relevant. This motivates the use of sparse autoencoders, a method for identifying a handful of "features" in the neural network that are important to producing any given output, akin to the small set of concepts a person might have in mind when reasoning about a situation. Their features display sparse activation patterns that naturally align with concepts easy for humans to understand, even without direct incentives for interpretability.
Authors
Jeffrey Wu, Leo Gao, Tom Dupré la Tour, Henk Tillman
Acknowledgments
Taya Christianson, Elizabeth Proehl, Yo Shavit, Niko Felix, Cathy Yeh, Gabriel Goh, Rajan Troll, Alec Radford, Jan Leike, Ilya Sutskever, David Robinson, Greg Brockman