Optimisation and AI

Gradient descent is not magic. It is a repeated mathematical choice.

This prototype shows how a point moves across a loss landscape by repeatedly stepping in the opposite direction to the gradient. It is a simple way into optimisation, calculus and the mathematics behind many machine learning systems.

The goal is to help learners experiment: change the learning rate, change the starting point, watch what happens, and explain why.

Interactive prototype

Watch gradient descent move across a loss landscape

The dot starts at a chosen point and repeatedly moves in the opposite direction to the gradient. Change the landscape, learning rate and start point to see what helps or harms convergence.

Iteration

0 / 24

Point

(-2.4, 2.3)

Loss

22.45

Gradient size

9.476

Controls

Landscape

Learning rate: 0.18

Iterations: 24

Start point

Step 0 of 24

Choose a learning rate, predict the first move, then step forward.

Next update: subtract 0.18 times the gradient from the current point.

What this shows

A lower loss means a better position according to the model. The algorithm is not thinking; it is repeatedly following a local rule.

A small learning rate can be safe but slow. A large learning rate can overshoot and sometimes make the loss worse.

This is one of the mathematical ideas behind machine learning: improving a model by reducing a loss function.

Current model

Simple bowl

L(x, y) = (x - 1)^2 + (y + 1)^2

A smooth convex landscape with one clear minimum. This is the cleanest place to understand the basic idea.

Good for introducing the rule: new position = old position - learning rate times gradient.

Joy in the process

The point is not just to finish. It is to notice, test and return.

These tools are invitations to explore. A good mistake, a surprising pattern or a question you cannot yet answer is part of the work, not a failure of it.

The challenge is deliberate: the site should support thinking, not remove the need for it.

Before changing a setting, pause and predict what you think will happen.

Change one thing at a time. What stayed the same, and what changed?

Try to create a surprising case, a broken case, or a beautiful pattern.

Ask what this connects to outside the page: maps, movement, nature, systems or decisions.

Reset, then try again with a new question in mind.

Guided exploration

Use the prototype to create predictions, not just observations.

Predict whether the next step will reduce the loss before pressing Next step.

Find a learning rate that converges slowly but reliably.

Find a learning rate that overshoots and makes the loss worse.

Compare the same learning rate on the simple bowl and narrow valley.

Future extensions

This prototype can grow into a richer classroom and exploration tool.

Add a 3D surface view alongside the contour view.

Compare gradient descent with momentum.

Add a linear regression example where the loss comes from real data points.

Add a teacher mode with prepared questions and discussion prompts.