To Whom Shall the Host Lend Her Ear?

Welcome back

Last time, in the quiet library, the keeper showed us the LSTM: a network with learned gates that decide what to remember and what to forget as it reads a sequence one step at a time.

That was a real step forward. But notice that it still reads in a line, word after word, carrying memory along like a bucket of water. Today we leave the quiet library for somewhere much noisier, and discover a different way of remembering altogether.

The host at the table

Imagine you’re hosting a dinner party. Eight guests, candles, clinking glasses, and several conversations happening at once.

Someone across the table says, “Did you hear what happened to her?”

You don’t panic. Without thinking about it, you glance around the table. Who is “her”? You recall that two minutes ago your friend Clara was talking about her sister moving to Lisbon. Your mind quietly turns its attention there. The rest of the noise, the wine, the weather, the football, fades into the background.

You haven’t replayed the whole evening in order. You’ve simply looked back and picked out what matters for the question in front of you.

Attention in a network

Attention gives a network that same ability. When it’s working on one word, it can look across all the other words in the sentence at once and decide how much each one matters right now.

Take the sentence: “The trophy didn’t fit in the suitcase because it was too big.” When the network reaches the word “it,” attention lets it look back over every other word and give each a score. “Trophy” gets a high score. “Suitcase” gets some. “The” and “because” get very little.

How the host decides

Here is the idea in plain words. For each word, the network creates three small bundles of numbers:

1. A query: what this word is looking for. “Who is ‘her’?”
2. A key: what each word has to offer, like a little name tag. “I’m about Clara’s sister.”
3. A value: the actual information that word will share if chosen.

The network compares the query with every key. The better they match, the higher the score. Then, with the softmax from the post office lesson, those scores become percentages that add up to 100. Perhaps 70% Clara’s sister, 20% Lisbon, 10% everything else.

Finally, it blends the values in those proportions. The word “her” comes away enriched with exactly the context it needed.

And, as always, nobody writes these rules by hand. The queries, keys and values all come from learned weights, shaped by the same patient training you already know.

Why this matters so much

With attention, distance stops being a problem. The word that matters might be two words back or two hundred. Either way, the network can reach it directly in one step, instead of hoping a fading memory carries it all that way. That’s the fix for the whisper game we’ve been chasing for three lessons.

What to carry with you