Sparse Autoencoders Reveal the Belly-Rub Feature

1. Introduction

Individual neurons in large models are polysemantic: a single unit may respond to unrelated inputs. Sparse autoencoders can separate these into cleaner features. We apply the method to Snort-1 and look for a feature that corresponds to the belly rub.

2. Method

We train a sparse autoencoder with a dictionary of 16,384 features on Snort-1 activations collected during 40 hours of ordinary sofa-based interaction. We label features by inspecting their strongest activating contexts. Most features were labeled “dinner.”

3. Results

Feature 4471 activates sharply on belly-rub contexts and on very little else. Ablating it reduces rolling over by 94% and increases looking at us in a disappointed way by 100%. Clamping the feature to a high value produces rolling over in situations the model was not trained for, including a board meeting.

We also report several other features of interest:

• The doorbell feature, which fires before the doorbell has been rung.
• A polysemantic vacuum feature, which responds to fear and to rage.
• A feature we have been unable to label, which activates only when we are not looking at it.

4. Limitations

Feature 4471 is monosemantic with respect to our dataset. We cannot rule out that it also responds to ear scratches, which were not sampled.

5. Conclusion

Belly rubs are represented in Snort-1 by a single, interpretable feature. The model appears to know exactly what it wants. We note that this was already suspected.