Training a Model with Gradient Descent

A practical Python example: predicting coffee roasting quality using a linear model trained with gradient descent.

The Problem

Find the best roasting temperature and time to maximize coffee quality score (0–100).

The Model

quality = w1 × temperature + w2 × time + bias

We start with random weights and use gradient descent to learn the correct values.

Training Data

# (temperature °C, time minutes, quality score 0-100)
(180,  8,  55)   # too low temp → underdeveloped
(200, 10,  72)   # getting there
(210, 12,  88)   # great! sweet spot
(215, 12,  92)   # excellent
(220, 13,  90)   # still great
(230, 15,  70)   # too dark, bitter
(240, 16,  50)   # burnt

How Gradient Descent Works

┌─────────────────────────────────────────────────────┐
│           GRADIENT DESCENT FLOW                      │
└─────────────────────────────────────────────────────┘

  ┌──────────────────┐
  │ 1. FORWARD PASS  │  Input → Model → Prediction
  └────────┬─────────┘
           │
           ▼
  ┌──────────────────┐
  │ 2. COMPUTE LOSS  │  (prediction - actual)²
  └────────┬─────────┘
           │
           ▼
  ┌──────────────────┐
  │ 3. GRADIENT      │  Which direction reduces loss?
  │    (slope)       │  dw1 = 2/n × error × temp
  └────────┬─────────┘  dw2 = 2/n × error × time
           │
           ▼
  ┌──────────────────┐
  │ 4. UPDATE WEIGHTS│  w1 = w1 - learning_rate × dw1
  │    (descend!)    │  w2 = w2 - learning_rate × dw2
  └────────┬─────────┘
           │
           ▼
      Repeat 1000×
           │
           ▼
  ┌──────────────────┐
  │ 5. CONVERGED!    │  Loss minimized, weights learned
  └──────────────────┘

Full Python Code

import random

# === Training Data ===
training_data = [
    (180, 8,  55), (190, 9,  62), (200, 10, 72),
    (205, 11, 80), (210, 12, 88), (215, 12, 92),
    (220, 13, 90), (225, 14, 85), (230, 15, 70),
    (240, 16, 50), (200, 14, 68), (210, 8,  65),
    (215, 11, 89), (205, 12, 85), (220, 11, 87),
]

# === Normalize Data ===
def normalize(data):
    temps = [d[0] for d in data]
    times = [d[1] for d in data]
    temp_mean, temp_std = sum(temps)/len(temps), (sum((t-sum(temps)/len(temps))**2 for t in temps)/len(temps))**0.5
    time_mean, time_std = sum(times)/len(times), (sum((t-sum(times)/len(times))**2 for t in times)/len(times))**0.5
    normalized = [((t-temp_mean)/temp_std, (m-time_mean)/time_std, q) for t, m, q in data]
    return normalized, (temp_mean, temp_std), (time_mean, time_std)

norm_data, temp_stats, time_stats = normalize(training_data)

# === Initialize Random Weights ===
random.seed(42)
w1 = random.uniform(-0.1, 0.1)   # weight for temperature
w2 = random.uniform(-0.1, 0.1)   # weight for time
bias = random.uniform(-0.1, 0.1)  # bias

# === Gradient Descent Training ===
learning_rate = 0.1
epochs = 1000

for epoch in range(epochs):
    n = len(norm_data)
    dw1, dw2, db = 0, 0, 0
    
    for temp, time, quality in norm_data:
        pred = w1 * temp + w2 * time + bias
        error = pred - quality
        
        # Compute gradients
        dw1 += (2/n) * error * temp
        dw2 += (2/n) * error * time
        db  += (2/n) * error
    
    # Update weights (DESCEND the gradient!)
    w1   -= learning_rate * dw1
    w2   -= learning_rate * dw2
    bias -= learning_rate * db

print(f"Learned: w1={w1:.4f}, w2={w2:.4f}, bias={bias:.4f}")

# === Predict ===
def predict(temp_c, time_min):
    nt = (temp_c - temp_stats[0]) / temp_stats[1]
    nm = (time_min - time_stats[0]) / time_stats[1]
    return w1 * nt + w2 * nm + bias

print(f"215°C, 12min → Quality: {predict(215, 12):.1f}/100")
print(f"240°C, 16min → Quality: {predict(240, 16):.1f}/100")

Training Result

Loss over epochs:

Loss ▲
5936 │█
     │█
     │ █
     │  ██
     │    ████████████████  ← converged!
 165 │
     └──────────────────────► Epochs
      0        500       1000

Random guess:        16.7  (75.3 off!)
After training:      76.8  (15.2 off)
Actual:              92.0

Why 15.2 Error Remains (Linear Model Limitation)

Quality
  ▲
  │        ╭──╮
  │      ╭╯    ╰╮         ← Actual (non-linear curve)
  │    ╭╯        ╰╮
  │  ╭╯            ╰╮
  │╭╯     ────────── ╰╮   ← Linear model (straight line)
  │╯                    ╰  ← can't fit the curve!
  └────────────────────────► Temperature
  180    200    220    240

Linear model = straight line (can't capture "sweet spot")
Neural network = can learn curves (more layers + activations)

Key Takeaways

1. Gradient descent iteratively adjusts weights to minimize prediction error
2. Learning rate controls step size — too big = overshoot, too small = slow
3. Linear models can only learn straight-line relationships
4. Non-linear data (like coffee's sweet spot) needs neural networks or polynomial features
5. Same algorithm (gradient descent) trains everything from linear models to GPT — just more weights and layers