2  Day 1 — Math in code

▶  Open in Google Colab

This notebook recomputes every worked example and practice problem from the Day 1 page, in code, so you can confirm code and by-hand math agree. No new ideas here — Day 2 is where the code side of this course actually starts.

import torch
import warnings
warnings.filterwarnings('ignore')
import numpy as np
torch.manual_seed(0)

<torch._C.Generator at 0x7515f86177b0>

2.1 Dot product

w = torch.tensor([2., -1.])
x = torch.tensor([1., 4.])
print("w . x =", torch.dot(w, x).item())

# Practice problem 1
a = torch.tensor([2., -1., 3.])
b = torch.tensor([1., 4., 0.])
print("practice problem 1:", torch.dot(a, b).item())

w . x = -2.0
practice problem 1: -2.0

2.2 Gradient — checked two ways: by hand, and via automatic differentiation

# f(x0, x1) = 3*x0**2 + 2*x0*x1, at the point (1, 2)
x0 = torch.tensor(1., requires_grad=True)
x1 = torch.tensor(2., requires_grad=True)
f = 3*x0**2 + 2*x0*x1
f.backward()
print("autograd gradient:", (x0.grad.item(), x1.grad.item()))
print("by-hand answer was: (10, 2) -- matches")

autograd gradient: (10.0, 2.0)
by-hand answer was: (10, 2) -- matches

2.3 Gradient descent, iterated

def f(x):
    return (x - 3)**2

def fprime(x):
    return 2*(x - 3)

x = 0.0
eta = 0.3
for n in range(4):
    print(f"n={n}  x_n={x:.3f}  f(x_n)={f(x):.3f}  f'(x_n)={fprime(x):.3f}")
    x = x - eta * fprime(x)
print(f"n=4  x_n={x:.3f}  (converging toward the true minimum at x=3)")

n=0  x_n=0.000  f(x_n)=9.000  f'(x_n)=-6.000
n=1  x_n=1.800  f(x_n)=1.440  f'(x_n)=-2.400
n=2  x_n=2.520  f(x_n)=0.230  f'(x_n)=-0.960
n=3  x_n=2.808  f(x_n)=0.037  f'(x_n)=-0.384
n=4  x_n=2.923  (converging toward the true minimum at x=3)

2.4 Probability — exact calculation vs. simulation

# Exact: E[X] for a fair die, and P(even | X > 3)
outcomes = np.arange(1, 7)
print("E[X] exact:", outcomes.mean())

conditioned = outcomes[outcomes > 3]
p_even_given_gt3 = np.mean(conditioned % 2 == 0)
print("P(even | X > 3) exact:", p_even_given_gt3, "=", 2, "/", 3)

# Simulation: roll a lot of dice and check both empirically
rng = np.random.default_rng(0)
rolls = rng.integers(1, 7, size=500_000)
print("\nsimulated E[X]:            ", rolls.mean())

gt3 = rolls[rolls > 3]
print("simulated P(even | X > 3): ", np.mean(gt3 % 2 == 0))

E[X] exact: 3.5
P(even | X > 3) exact: 0.6666666666666666 = 2 / 3

simulated E[X]:             3.501258
simulated P(even | X > 3):  0.6657600979094258

Both the gradient-descent table and the probability calculations match the by-hand answers from the Day 1 page — the simulation’s numbers won’t be exactly 3.5 or exactly 2/3 (it’s random), but with half a million rolls they should be very close.