Chapter 1 · Prototype

A network you can touch

A hands-on first look at how a neural network learns. You will watch a real one learn, and you can poke every number inside it.

What this is

In under a minute, this film shows a small neural network teaching itself to tell blue dots from orange ones. The rest of the page explains every part of it, then hands you the controls.

This page is the first chapter of an interactive guide to neural networks. You don't need to install anything, write code or own a graphics card. Everything runs in your browser.

The network on this page is tiny, but it learns the same way as the large ones behind image recognition and chatbots. It makes a guess, measures how wrong the guess was, and adjusts itself a little. After a few hundred rounds of that, it gets good. You need no math beyond multiplying and adding. Where formulas appear, plain words come with them.

The puzzle

Here are 160 dots. Each dot has a position, given by two numbers: x₁, how far right it is, and x₂, how far up. Each dot also has a color, blue or orange.

The puzzle: build a machine that looks only at a dot's position and tells you its color. It should also work for new dots it has never seen.

The obvious first attempt is a straight line, with blue on one side and orange on the other. Try it.

Your line gets
–

Drag either round handle to move the line. The two sides are colored whichever way gets more dots right. Hollow dots are on the wrong side.

The best any single straight line can do on these dots is –.

One line is not enough, because the blue dots sit in two opposite corners. We need a machine that can draw a more complicated boundary, and we want it to find that boundary by itself, from the examples.

This is the simplest kind of classification problem. Replace the two coordinates with the thousands of pixel values in a photo, and blue and orange with “cat” and “dog”, and you have image recognition. The machine is the same kind of thing, only much larger.

The tool: a neural network

A neural network is a calculation with adjustable knobs. Numbers go in, a number comes out, and the knobs decide what happens in between. The knobs are called parameters, and they come in two kinds: weights and biases.

The building block is the neuron. It does three small things:

  1. Multiply each input by its own weight.
  2. Add the results together, plus a bias.
  3. Pass the sum through an activation function, a fixed curve. Here it is tanh, which squashes any number into the range −1 to 1.

The first two steps describe a straight line across the plane: the places where the sum equals zero. The line you dragged in the puzzle is exactly one neuron. Right now it has w₁ = –, w₂ = – and b = –. Its sum is positive on the blue side, zero on the line and negative on the orange side. The activation turns that hard line into a soft edge.

One neuron can only draw one line, which is why it cannot solve the puzzle. So we use several. Our network has a hidden layer of four neurons, each drawing its own line. One output neuron combines their answers into a single number called ŷ (“y-hat”), between 0 and 1. ŷ is the network's estimate of the probability that the dot is blue.

With four hidden neurons the network has 17 parameters. Each hidden neuron has 2 weights and 1 bias, which makes 12. The output neuron has 4 weights and 1 bias, which makes 5 more. The weights start as random numbers, so at first the network's answers are random too. Data and training fix that.

The same network in five lines of math
zj = wj1·x₁ + wj2·x₂ + bjeach hidden neuron weighs the two inputs
hj = tanh(zj)and bends the result
zout = Σj vj·hj + cthe output weighs the hidden neurons
ŷ = σ(zout)and squashes the sum into a probability
L = −[y·log ŷ + (1−y)·log(1−ŷ)]the loss (log is the natural logarithm), small when ŷ agrees with the color (y = 1 for blue)

Data: learning from examples

The 160 dots are the training data: examples where we already know the right answer. Each example is a pair. The input is the position (x₁, x₂), and the label is the color, written y = 1 for blue and y = 0 for orange.

The network is never told a rule such as “blue when both numbers have the same sign”. It only sees the examples, and everything it learns comes from them. That is why the quality of the data matters so much in machine learning.

The playground has four sets of dots, from easy to hard: Clusters, XOR (the puzzle above), Circle and Spiral.

Training: guess, measure, adjust

Training means adjusting the 17 parameters until the network's answers match the labels. It is a loop of four stages:

  1. Forward pass. Feed every dot through the network and collect its answers, ŷ.
  2. Loss. Score how wrong each answer is. The loss is near 0 for a confident right answer and grows quickly for a confident wrong one. Averaged over all dots, it becomes one number that says how badly the network is doing.
  3. Backward pass. For every parameter, work out how the loss would change if you nudged that parameter up a little. That number is the parameter's gradient: its sign says which way is uphill, its size how steep. Backpropagation computes all of them in one sweep, from the output back toward the inputs, using the chain rule from calculus.
  4. Update. Move every parameter a small step against its gradient, which means downhill. Each step is the gradient times a small number called the learning rate, written η (“eta”).

Then repeat. Each round is one training step. The film at the top of the page shows this loop on the puzzle's dots, following the one dot the untrained network gets most wrong. Watch it again ↑

In the playground, Step walks through the four stages one click at a time, and Train runs them continuously.

The playground

The playground shows the data and the network, then the training progress and a step-by-step story of what the network is doing. At the bottom, the inspector opens up whatever you tap.

Pick a set of dots and press Train. Tap any dot to make it the probe (the ringed dot): the numbers in the network then describe that one dot. Tap any wire or neuron to look inside it.

Training
Data
Hidden neurons
4
Activation
Speed
The data
blue dot (y = 1) orange dot (y = 0) wrong right now boundary, 50% a hidden neuron's line background is the network's guess: orangeblue

The network 17 parameters
Step0training rounds so far
Loss–average wrongness, lower is better
Accuracy–dots on the correct side
Loss over traininghover the chart for values

How to read the figure

Color
Blue means positive and orange negative; for the dots, blue is y = 1 and orange y = 0. In the backward view, a wire's color shows which way the probe dot wants that weight to move: blue for up, orange for down. That is the opposite sign of its gradient, because training steps against the gradient.
Background
The shading behind the dots is the network's guess for every point of the plane. Deeper blue means more sure the point is blue. The dark curve is the boundary, where the guess is exactly 50%.
Hollow dots
Dots the network currently gets wrong. Watch them fill in as it trains.
Squares
Each square in the network is one neuron's output across the whole plane, on the same axes as the data plot. A hidden neuron's square is split by its line. The output square is the data plot's background in miniature.
Wires
In the forward view, a wire's thickness is the size of its weight. In the backward view, it shows how hard the probe dot pushes on that weight, and in which direction.
Probe
The ringed dot. Its values appear beside each neuron, as a small dot inside every square, and in the card under the plot.
Inspector
The bottom panel. The first sentence says in plain words what the selected part is doing. The panel beside it shows the arithmetic, and the slider lets you change the number yourself.

Things to try

What runs where

Everything runs in this page's JavaScript on your CPU, with no GPU, no server and no machine-learning library. Numbers are 64-bit floats and the random seed is fixed, so every visitor starts from the same weights and sees the same values, apart from the last digit or so, which can differ between browsers. One training step over all 160 dots takes well under a millisecond on a laptop.

Gradients come from the small autograd below, which is the exact code this page runs. Every arithmetic operation writes its inputs and its local derivative onto a tape. The backward pass walks the tape in reverse and applies the chain rule at each entry. The inspector computes the same gradients by hand from the forward values and checks them against autograd.

Show the autograd source

Where this is going

  1. 1A network you can touchThis page
  2. 2Loss and gradients: walking downhill
  3. 3Backprop, one operation at a time
  4. 4Going deeper: why layers help
  5. 5Seeing: convolutional networks
  6. 5′Listening: spectrograms and soundBranch
  7. 6Words as vectors: tokens and embeddings
  8. 7Attention: looking back over the whole story
  9. 8A tiny GPT, trained in your browser

Working prototype, October 2026. Every figure on this page is a live computation.
Source on GitHub · code under MIT, text and figures under CC BY 4.0.