Welcome back
Last time we knelt beside a termite mound and met the neuron: a tiny, simple unit that listens to a few signals, adds them up, and passes one on. We ended with a question. Should every neighbour count equally?
Today we answer it, and to do that we’ll walk out onto a soccer pitch.
The midfielder’s ears
Picture a midfielder with the ball at her feet. In the space of a second, voices come at her from every direction.
The veteran captain shouts “Left!” The young winger, who is a little too eager, yells “Me! Me!” The goalkeeper, sixty metres away, calls something she can barely hear. Someone in the crowd is shouting too.
She doesn’t treat these voices equally, and she’d be a poor player if she did. Over many matches she has learned whom to trust. The captain’s call counts for a great deal. The eager winger’s counts for a little. The crowd counts for nothing at all, and if the opposing defender is shouting to trick her, she may even lean the other way.
Without thinking about it, she gives each voice a weight.
What a weight is
A neuron does exactly this. Each signal that arrives has its own weight attached to it: a number that says how much this particular input should matter.
So instead of simply adding up its inputs, as we said last time, the neuron does something slightly richer. For each input, it works out input × weight. Then it adds all of those together.
Let’s try it gently with the midfielder:
- The captain’s call is strong (say 1), and her trust in him is high (weight 3). That gives 3.
- The winger’s call is strong too (1), but her trust is modest (weight 1). That gives 1.
- The crowd is loud (1), but its weight is 0. That gives 0.
Her total leaning towards “go left” is 3 + 1 + 0, which comes to 4. The captain’s voice carried most of the decision, just as it should.
Weights can be large, small, or even negative
A few things are worth noticing about weights:
- A large weight means “this input matters a lot.” The captain.
- A weight near zero means “barely listen to this.” The crowd.
- A negative weight means “when this signal is strong, lean the opposite way.” The opposing defender trying to fool her.
That last kind is surprisingly useful. In a network that recognises cats, for example, a neuron might give a negative weight to “the picture has wheels.” The more it sees wheels, the less cat-like the picture seems.
Where knowledge actually lives
Here is the heart of today’s lesson, so let’s slow down for it.
Remember from Lesson 1 that a neural network learns from examples. What it changes when it learns is its weights. Nothing else is quite so important. The neurons themselves stay just as simple as before. What changes is how much each connection is trusted.
When the midfielder joined the team, she trusted everyone about equally. Season by season, match by match, she adjusted. That gradual adjustment is exactly what we’ll later call training. A brand-new network starts with weights that are more or less random guesses. A trained network has weights that have been tuned, very patiently, by thousands of examples.
So when people say a network “knows” something, what they really mean is that the knowledge is stored in its weights.
What to carry with you
Every connection carries a weight that says how much an input matters. A neuron multiplies each input by its weight and adds them up, and learning means adjusting those weights.