Episode 01 — The Knowledge

PyTorch

The framework that powers modern machine learning

01 / What It Is

What is PyTorch?

PyTorch is a framework — a collection of pre-built tools and functions — for building and training neural networks: programs that learn patterns from data.

01
Used by most researchers and increasingly in production apps
02
The main alternative was TensorFlow (Google) — PyTorch won the ecosystem war
03
Python-based, so it fits naturally into data science workflows

▲ ▲ ▲ Core Concept 1

Tensors

02 / Tensors

Numbers in boxes

A tensor is just a container of numbers arranged in a grid. More dimensions = more axes in the grid. That's it.

Scalar

5

0 dimensions

Vector

3

7

1

4

1 dimension — shape: [4]

Matrix

2

8

1

5

3

9

4

6

7

2 dimensions — shape: [3, 3]

3D Tensor

1

5

3

7

2

8

4

9

6

3 dimensions — shape: [3, 3, 3]

Think of it like a spreadsheet: a single cell is a scalar, a row is a vector, the whole sheet is a matrix, and a workbook with multiple sheets is a 3D tensor.

02 / Tensors

Every tensor has three properties

Shape

The size along each dimension — how many rows, columns, layers, etc.

          # A table with 3 rows and 4 columns

          x.shape → [3, 4]

Dtype

The type of number stored — whole numbers, decimals, or decimals with fewer digits (which use less memory).

          float32 # standard decimal

          float16 # half-precision

          int64   # whole number

Device

Where the tensor lives in the computer — on the CPU (main processor) or on the GPU (graphics card).

          x.to("cpu")

          x.to("cuda") # GPU

02 / Tensors

Why GPUs are faster

A CPU has a few very powerful cores that do tasks one-by-one.
A GPU has thousands of small cores that do many simple tasks at the same time.

CPU

8 powerful cores

Great at complicated, sequential work

GPU

Thousands of small cores

Great at simple math on huge arrays of numbers

ML training is mostly multiplying enormous grids of numbers together — exactly the kind of simple, repetitive math that thousands of GPU cores can split up and do all at once.

▲ ▲ ▲ Core Concept 2

Autograd

03 / Autograd

How models learn

A neural network is full of numbers called weights. Training means adjusting those weights so the network's predictions get more accurate.

But which direction should each weight change? And by how much?

The answer is gradients — a gradient is a number that tells you: "if you increase this weight slightly, how much does the error change?"

Autograd (short for "automatic gradients") is PyTorch's system that computes all of these gradients for you, automatically, no matter how complex your network is.

What PyTorch Does

Records every math operation you do on tensors
Builds a behind-the-scenes map of those operations
When you call .backward(), it walks backwards through that map and computes every gradient

Imagine hiking blindfolded on a hilly landscape. The gradient is like feeling the slope under your feet — it tells you which direction is downhill so you can walk toward the lowest valley (the smallest error).

03 / Autograd

Gradients point uphill

The gradient tells you the direction of steepest increase in error. To reduce error, you go the opposite direction.

Learning Rate

Controls how big a step you take each time you adjust the weights.

Too large — you overshoot the sweet spot and bounce around
Too small — you inch forward so slowly training takes forever
Just right — you smoothly converge toward the best weights

▲ ▲ ▲ Core Concept 3

The Training Loop

04 / Training Loop

Four steps, repeated

Every neural network learns through the same cycle. Each pass through this loop makes the model a little more accurate.

1

Forward Pass

Feed your data into the model. It makes a prediction based on its current weights.

→

2

Loss Calculation

Compare the prediction to the correct answer. The loss is a single number measuring how wrong the model was.

→

3

Backward Pass

Autograd computes the gradient of every weight — how each one contributed to the error.

→

4

Optimizer Step

Adjust every weight in the opposite direction of its gradient, by a small amount (the learning rate).

04 / Training Loop

In code

        for batch in data_loader:
        
    # 1. Forward pass
        
    prediction = model(batch)
        
    # 2. How wrong were we?
        
    loss = loss_fn(prediction, answer)
        
    # 3. Compute gradients
        
    loss.backward()
        
    # 4. Update weights
        
    optimizer.step()
        
    # Reset gradients for next round
        
    optimizer.zero_grad()

What Each Line Does

model(batch) — runs the forward pass, producing predictions
loss_fn(...) — measures the error as a single number
.backward() — tells autograd to compute all gradients
.step() — adjusts weights using those gradients
.zero_grad() — clears old gradients so they don't pile up from the previous round

▲ ▲ ▲ Core Concept 4

nn.Module

05 / nn.Module

The building block for every model

nn.Module is the base template that every neural network component in PyTorch is built from. When you create a model, you're defining a class that inherits from it.

It gives you two things:

Parameters

The weights that live inside the model — the numbers that get adjusted during training. nn.Module keeps track of all of them automatically.

forward()

A method you write that defines what happens when data passes through this component — which math operations to run, in what order.

        class PricePredictor(nn.Module):
        
  def __init__(self):
        
    super().__init__()
        
    # Define layers (contain weights)
        
    self.layer1 = nn.Linear(10, 64)
        
    self.layer2 = nn.Linear(64, 1)
        
  def forward(self, x):
        
    # Define the data flow
        
    x = self.layer1(x)
        
    x = F.relu(x) # activation
        
    x = self.layer2(x)
        
    return x

nn.Linear(10, 64) is a layer that takes 10 input numbers and produces 64 output numbers. F.relu is an activation function — it zeroes out any negative values, which helps the network learn non-obvious patterns. Without it, stacking layers would be no better than a single layer.

▲ ▲ ▲ Core Concept 5

DataLoader

06 / DataLoader

Feeding data in batches

You rarely send your entire dataset through the model at once — you'd run out of memory. Instead, you split it into smaller groups called batches.

The DataLoader handles this automatically: it divides your data into batches, shuffles the order each time (so the model doesn't memorize the sequence), and loads data in parallel to keep the GPU busy.

          loader = DataLoader(

            dataset,

            batch_size=32,   # 32 samples at a time

            shuffle=True,    # randomize order

            num_workers=4,  # load in parallel

          )

Why Batch Size Matters

Memory — larger batches need more GPU memory
Gradient quality — larger batches give a more stable estimate of the right direction to adjust weights
Generalization — smaller batches add useful randomness ("noise") that can help the model learn patterns that work on new data, not just the training data
Speed — larger batches use the GPU more efficiently, up to a point

Common batch sizes: 16, 32, 64, 128. You usually pick the largest one that fits in your GPU's memory.

▲ ▲ ▲ Watch Out

Common Gotchas

07 / Gotchas

Three mistakes everyone makes

Forgetting .zero_grad()

PyTorch adds new gradients to old ones by default — it doesn't replace them. If you forget to clear them, gradients pile up from previous rounds and your weights get nonsensical updates.
Fix: always call optimizer.zero_grad() before loss.backward()

Device mismatch

If your model lives on the GPU but your data is still on the CPU (or vice versa), PyTorch will throw an error. Every tensor involved in a computation must be on the same device.
Fix: move both to the same place with .to(device)

Forgetting .train() vs .eval()

Some layers behave differently during training vs. when you're just making predictions. For example, dropout (which randomly disables parts of the network to prevent memorization) is active during training but should be off during evaluation.
Fix: call model.train() before training and model.eval() before predicting

Recap

The five things to remember

01
Tensors are containers of numbers arranged in grids — they live on a device (CPU or GPU) and have a shape and number type.
02
Autograd automatically computes gradients — numbers that tell you how to adjust each weight to reduce error.
03
The training loop is four steps on repeat: predict → measure error → compute gradients → update weights.
04
nn.Module is the template for every model component — it holds weights and defines what happens when data flows through.
05
DataLoader splits data into batches, shuffles them, and feeds them to the model efficiently.