How LLMs Work: I Changed One Number and It Said It Was an Alien
Nothing is loaded from YouTube until you press play, and then without tracking cookies.
About this video
People describe LLMs like brains. This is a look at the calculator underneath, using one small model and one very suggestible unit.
In two minutes: what weights are, what layers and units are, and what happens when you turn a single unit up or down without retraining anything. The question stays the same each time ("who are you?"). At +0.8 the model says it's an extra intelligent alien life form from planet X. At 0 it says it's Qwen3.5. At -0.8 you get "I are a virtual assistant."
Try it yourself: github.com/joe-rlo/neuron-explorer
Chapters 0:00 A brain, or a very large calculator? 0:18 Weights: the frozen numbers 0:34 Layers and units 0:49 Steering a single unit 1:03 Weights vs units 1:27 The experiment: layer 22, unit N2742
What's covered Weights: the numbers a model learns in training (about 4 billion in this one), fixed afterward Units: where the math happens, over 9,000 per layer here Steering: changing one unit's output changes the response and leaves the weights alone Why editing the weights directly is harder: long rambling answers and broken models
More writing AI writing at: 0x4a6f65.com Build with AI: near.ai
#LLM #AIexplained #interpretability
Transcript
439 wordsCaptions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.
- 0:00
Yeah, I just said it was an alien from planet X. I didn't retrain it. I just tweaked the number inside it. Describe AI like a brain, but it isn't one. It's a really large calculator. Calculator is there for predicting the next word. You see all the insides and how this all works. It's a lot less confusing and spooky. These are sort of numbers that the model got from the training process.
- 0:21
This model that I'm showing you now only has about 4 billion of them. A very small model considering the current frontier models. After training, those numbers don't change. So, every chat is the same set of fixed math. Math happens through fixed layers. Each layer has smaller units. This one has about over 9,000 different units per layer. Each unit takes your text, does a bit of multiplication, and then outputs a number. Bigger numbers nudge one way to
- 0:47
the next word, smaller ones another. You can actually take it into these different layers, separate into the smaller units, and then go through and either turn up or down each individual unit. Changing that actually changes the response and what you get out of it. It does not actually change the weights in the original learning. The weights are the stored numbers. The units are where the math happen. Editing the weights themselves is different because those are the stored numbers. It becomes a much more complicated process because
- 1:11
you can run into really long rambling answers and essentially break the model. uh you can technically improve it as well, but it's not something that as straightforward as if I'm actually going to just manipulate the output of a particular unit inside a layer like this example here of going from uh layer 22 and from unit N2742.
- 1:35
By changing that, we can see by asking the question, who are you? If you turn it up, you get the I am the extra intelligent alien life form from planet X. Uh, if you leave it as is, you get the basic answer, which is what you would expect. And if I turn it down, it starts to, you can see it kind of gives an answer kind of. It's much more generic than what it should be. Uh, it doesn't actually tell you what you want to know. You can do this with any one of the different units on any one of these
- 1:58
different layers and test with it. If you want to try this out yourself, check out the repo that I've put in the notes and you can get started right