![](\"https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/pytorch-logo-dark.svg\"){kind=logo}
![](\"https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/docs/source/_static/images/logo.svg\"){kind=logo}
\n",
"\n",
"# Model Zoo -- LeNet-5 Trained on MNIST"
]
},
{
"cell_type": "markdown",
"id": "864369a2",
"metadata": {},
"source": [
"This notebook implements the classic LeNet-5 convolutional network [1] and applies it to MNIST digit classification. The basic architecture is shown in the figure below:\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "a919782c",
"metadata": {},
"source": [
"\n",
"\n",
"LeNet-5 is commonly regarded as the pioneer of convolutional neural networks, consisting of a very simple architecture (by modern standards). In total, LeNet-5 consists of only 7 layers. 3 out of these 7 layers are convolutional layers (C1, C3, C5), which are connected by two average pooling layers (S2 & S4). The penultimate layer is a fully connexted layer (F6), which is followed by the final output layer. The additional details are summarized below:\n",
"\n",
"- All convolutional layers use 5x5 kernels with stride 1.\n",
"- The two average pooling (subsampling) layers are 2x2 pixels wide with stride 1.\n",
"- Throughrout the network, tanh sigmoid activation functions are used. (**In this notebook, we replace these with ReLU activations**)\n",
"- The output layer uses 10 custom Euclidean Radial Basis Function neurons for the output layer. (**In this notebook, we replace these with softmax activations**)\n",
"- The input size is 32x32; here, we rescale the MNIST images from 28x28 to 32x32 to match this input dimension. Alternatively, we would have to change the \n",
"achieve error rate below 1% on the MNIST data set, which was very close to the state of the art at the time (produced by a boosted ensemble of three LeNet-4 networks).\n",
"\n",
"\n",
"### References\n",
"\n",
"- [1] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. [Gradient-based learning applied to document recognition](https://ieeexplore.ieee.org/document/726791). Proceedings of the IEEE, november 1998."
]
},
{
"cell_type": "markdown",
"id": "6f250573",
"metadata": {},
"source": [
"## General settings and hyperparameters"
]
},
{
"cell_type": "markdown",
"id": "542f0b67",
"metadata": {},
"source": [
"- Here, we specify some general hyperparameter values and general settings\n",
"- Note that for small datatsets, it is not necessary and better not to use multiple workers as it can sometimes cause issues with too many open files in PyTorch. So, if you have problems with the data loader later, try setting `NUM_WORKERS = 0` instead."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "c9c9f45f",
"metadata": {},
"outputs": [],
"source": [
"BATCH_SIZE = 256\n",
"NUM_EPOCHS = 20\n",
"LEARNING_RATE = 0.005\n",
"NUM_WORKERS = 4"
]
},
{
"cell_type": "markdown",
"id": "bf4c7a28",
"metadata": {},
"source": [
"## Implementing a Neural Network using PyTorch Lightning's `LightningModule`"
]
},
{
"cell_type": "markdown",
"id": "017ca5a5",
"metadata": {},
"source": [
"- In this section, we set up the main model architecture using the `LightningModule` from PyTorch Lightning.\n",
"- We start with defining our neural network model in pure PyTorch, and then we use it in the `LightningModule` to get all the extra benefits that PyTorch Lightning provides."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "5bf908bd",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"\n",
"\n",
"class PyTorchLeNet5(torch.nn.Module):\n",
"\n",
" def __init__(self, num_classes, grayscale=False):\n",
" super().__init__()\n",
" \n",
" self.grayscale = grayscale\n",
" self.num_classes = num_classes\n",
"\n",
" if self.grayscale:\n",
" in_channels = 1\n",
" else:\n",
" in_channels = 3\n",
"\n",
" self.features = torch.nn.Sequential(\n",
" torch.nn.Conv2d(in_channels, 6, kernel_size=5),\n",
" torch.nn.Tanh(),\n",
" torch.nn.MaxPool2d(kernel_size=2),\n",
" torch.nn.Conv2d(6, 16, kernel_size=5),\n",
" torch.nn.Tanh(),\n",
" torch.nn.MaxPool2d(kernel_size=2)\n",
" )\n",
"\n",
" self.classifier = torch.nn.Sequential(\n",
" torch.nn.Linear(16*5*5, 120),\n",
" torch.nn.Tanh(),\n",
" torch.nn.Linear(120, 84),\n",
" torch.nn.Tanh(),\n",
" torch.nn.Linear(84, num_classes),\n",
" )\n",
"\n",
" def forward(self, x):\n",
" x = self.features(x)\n",
" x = torch.flatten(x, start_dim=1)\n",
" logits = self.classifier(x)\n",
" return logits"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "b5a989d0",
"metadata": {},
"outputs": [],
"source": [
"import pytorch_lightning as pl\n",
"import torchmetrics\n",
"\n",
"\n",
"# LightningModule that receives a PyTorch model as input\n",
"class LightningModel(pl.LightningModule):\n",
" def __init__(self, model, learning_rate):\n",
" super().__init__()\n",
"\n",
" self.learning_rate = learning_rate\n",
" # The inherited PyTorch module\n",
" self.model = model\n",
"\n",
" # Save settings and hyperparameters to the log directory\n",
" # but skip the model parameters\n",
" self.save_hyperparameters(ignore=['model'])\n",
"\n",
" # Set up attributes for computing the accuracy\n",
" self.train_acc = torchmetrics.Accuracy()\n",
" self.valid_acc = torchmetrics.Accuracy()\n",
" self.test_acc = torchmetrics.Accuracy()\n",
" \n",
" # Defining the forward method is only necessary \n",
" # if you want to use a Trainer's .predict() method (optional)\n",
" def forward(self, x):\n",
" return self.model(x)\n",
" \n",
" # A common forward step to compute the loss and labels\n",
" # this is used for training, validation, and testing below\n",
" def _shared_step(self, batch):\n",
" features, true_labels = batch\n",
" logits = self(features)\n",
" loss = torch.nn.functional.cross_entropy(logits, true_labels)\n",
" predicted_labels = torch.argmax(logits, dim=1)\n",
"\n",
" return loss, true_labels, predicted_labels\n",
"\n",
" def training_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.log(\"train_loss\", loss)\n",
" \n",
" # To account for Dropout behavior during evaluation\n",
" self.model.eval()\n",
" with torch.no_grad():\n",
" _, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.train_acc.update(predicted_labels, true_labels)\n",
" self.log(\"train_acc\", self.train_acc, on_epoch=True, on_step=False)\n",
" self.model.train()\n",
" return loss # this is passed to the optimzer for training\n",
"\n",
" def validation_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.log(\"valid_loss\", loss)\n",
" self.valid_acc(predicted_labels, true_labels)\n",
" self.log(\"valid_acc\", self.valid_acc,\n",
" on_epoch=True, on_step=False, prog_bar=True)\n",
"\n",
" def test_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.test_acc(predicted_labels, true_labels)\n",
" self.log(\"test_acc\", self.test_acc, on_epoch=True, on_step=False)\n",
"\n",
" def configure_optimizers(self):\n",
" optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n",
" return optimizer"
]
},
{
"cell_type": "markdown",
"id": "9a4f8ac0",
"metadata": {},
"source": [
"## Setting up the dataset"
]
},
{
"cell_type": "markdown",
"id": "f99658ec",
"metadata": {},
"source": [
"- In this section, we are going to set up our dataset."
]
},
{
"cell_type": "markdown",
"id": "c0193d9e",
"metadata": {},
"source": [
"### Inspecting the dataset"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "a85aeca4",
"metadata": {},
"outputs": [],
"source": [
"from torchvision import datasets\n",
"from torchvision import transforms\n",
"from torch.utils.data import DataLoader\n",
"\n",
"\n",
"train_dataset = datasets.MNIST(root='./data', \n",
" train=True, \n",
" transform=transforms.ToTensor(),\n",
" download=True)\n",
"\n",
"train_loader = DataLoader(dataset=train_dataset, \n",
" batch_size=BATCH_SIZE, \n",
" num_workers=NUM_WORKERS,\n",
" drop_last=True,\n",
" shuffle=True)\n",
"\n",
"test_dataset = datasets.MNIST(root='./data', \n",
" train=False,\n",
" transform=transforms.ToTensor())\n",
"\n",
"test_loader = DataLoader(dataset=test_dataset, \n",
" batch_size=BATCH_SIZE,\n",
" num_workers=NUM_WORKERS,\n",
" drop_last=False,\n",
" shuffle=False)\n",
"\n",
"# Checking the dataset\n",
"all_train_labels = []\n",
"all_test_labels = []\n",
"\n",
"for images, labels in train_loader: \n",
" all_train_labels.append(labels)\n",
"all_train_labels = torch.cat(all_train_labels)\n",
" \n",
"for images, labels in test_loader: \n",
" all_test_labels.append(labels)\n",
"all_test_labels = torch.cat(all_test_labels)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "7e7d853a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training labels: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
"Training label distribution: tensor([5915, 6732, 5945, 6121, 5832, 5412, 5909, 6256, 5839, 5943])\n",
"\n",
"Test labels: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
"Test label distribution: tensor([ 980, 1135, 1032, 1010, 982, 892, 958, 1028, 974, 1009])\n"
]
}
],
"source": [
"print('Training labels:', torch.unique(all_train_labels))\n",
"print('Training label distribution:', torch.bincount(all_train_labels))\n",
"\n",
"print('\\nTest labels:', torch.unique(all_test_labels))\n",
"print('Test label distribution:', torch.bincount(all_test_labels))"
]
},
{
"cell_type": "markdown",
"id": "0fd094dc",
"metadata": {},
"source": [
"### Performance baseline"
]
},
{
"cell_type": "markdown",
"id": "26126a41",
"metadata": {},
"source": [
"- Especially for imbalanced datasets, it's quite useful to compute a performance baseline.\n",
"- In classification contexts, a useful baseline is to compute the accuracy for a scenario where the model always predicts the majority class -- you want your model to be better than that!"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "8a23adbe",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Baseline ACC: 11.35%\n"
]
}
],
"source": [
"majority_prediction = torch.argmax(torch.bincount(all_test_labels))\n",
"baseline_acc = torch.mean((all_test_labels == majority_prediction).float())\n",
"print(f'Baseline ACC: {baseline_acc*100:.2f}%')"
]
},
{
"cell_type": "markdown",
"id": "0db99229",
"metadata": {},
"source": [
"### Setting up a `DataModule`"
]
},
{
"cell_type": "markdown",
"id": "ac1c0141",
"metadata": {},
"source": [
"- There are three main ways we can prepare the dataset for Lightning. We can\n",
" 1. make the dataset part of the model;\n",
" 2. set up the data loaders as usual and feed them to the fit method of a Lightning Trainer -- the Trainer is introduced in the next subsection;\n",
" 3. create a LightningDataModule.\n",
"- Here, we are going to use approach 3, which is the most organized approach. The `LightningDataModule` consists of several self-explanatory methods as we can see below:\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "9bcf3be3",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from torch.utils.data.dataset import random_split\n",
"from torch.utils.data import DataLoader\n",
"\n",
"\n",
"class DataModule(pl.LightningDataModule):\n",
" def __init__(self, data_path='./'):\n",
" super().__init__()\n",
" self.data_path = data_path\n",
" \n",
" def prepare_data(self):\n",
" datasets.MNIST(root=self.data_path,\n",
" download=True)\n",
" self.resize_transform = transforms.Compose(\n",
" [transforms.Resize((32, 32)),\n",
" transforms.ToTensor()])\n",
" \n",
" return\n",
"\n",
" def setup(self, stage=None):\n",
" # Note transforms.ToTensor() scales input images\n",
" # to 0-1 range\n",
" train = datasets.MNIST(root=self.data_path, \n",
" train=True, \n",
" transform=self.resize_transform,\n",
" download=False)\n",
"\n",
" self.test = datasets.MNIST(root=self.data_path, \n",
" train=False, \n",
" transform=self.resize_transform,\n",
" download=False)\n",
"\n",
" self.train, self.valid = random_split(train, lengths=[55000, 5000])\n",
"\n",
" def train_dataloader(self):\n",
" train_loader = DataLoader(dataset=self.train, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=True,\n",
" shuffle=True,\n",
" num_workers=NUM_WORKERS)\n",
" return train_loader\n",
"\n",
" def val_dataloader(self):\n",
" valid_loader = DataLoader(dataset=self.valid, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=False,\n",
" shuffle=False,\n",
" num_workers=NUM_WORKERS)\n",
" return valid_loader\n",
"\n",
" def test_dataloader(self):\n",
" test_loader = DataLoader(dataset=self.test, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=False,\n",
" shuffle=False,\n",
" num_workers=NUM_WORKERS)\n",
" return test_loader"
]
},
{
"cell_type": "markdown",
"id": "d68d27e2",
"metadata": {},
"source": [
"- Note that the `prepare_data` method is usually used for steps that only need to be executed once, for example, downloading the dataset; the `setup` method defines the the dataset loading -- if you run your code in a distributed setting, this will be called on each node / GPU. \n",
"- Next, lets initialize the `DataModule`; we use a random seed for reproducibility (so that the data set is shuffled the same way when we re-execute this code):"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "92f2d713",
"metadata": {},
"outputs": [],
"source": [
"torch.manual_seed(1) \n",
"data_module = DataModule(data_path='./data')"
]
},
{
"cell_type": "markdown",
"id": "e70c23c9",
"metadata": {},
"source": [
"## Training the model using the PyTorch Lightning Trainer class"
]
},
{
"cell_type": "markdown",
"id": "5dec03e7",
"metadata": {},
"source": [
"- Next, we initialize our model.\n",
"- Also, we define a call back so that we can obtain the model with the best validation set performance after training.\n",
"- PyTorch Lightning offers [many advanced logging services](https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html) like Weights & Biases. Here, we will keep things simple and use the `CSVLogger`:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "cf93512a",
"metadata": {},
"outputs": [],
"source": [
"from pytorch_lightning.callbacks import ModelCheckpoint\n",
"from pytorch_lightning.loggers import CSVLogger\n",
"\n",
"\n",
"pytorch_model = PyTorchLeNet5(\n",
" num_classes=10, grayscale=True)\n",
"\n",
"lightning_model = LightningModel(\n",
" model=pytorch_model, learning_rate=LEARNING_RATE)\n",
"\n",
"callbacks = [ModelCheckpoint(\n",
" save_top_k=1, mode='max', monitor=\"valid_acc\")] # save top 1 model \n",
"logger = CSVLogger(save_dir=\"logs/\", name=\"my-model\")"
]
},
{
"cell_type": "markdown",
"id": "58fb64a9",
"metadata": {},
"source": [
"- Now it's time to train our model:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "0f5fef0d",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"GPU available: True, used: True\n",
"TPU available: False, using: 0 TPU cores\n",
"IPU available: False, using: 0 IPUs\n",
"LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
"\n",
" | Name | Type | Params\n",
"--------------------------------------------\n",
"0 | model | PyTorchLeNet5 | 61.7 K\n",
"1 | train_acc | Accuracy | 0 \n",
"2 | valid_acc | Accuracy | 0 \n",
"3 | test_acc | Accuracy | 0 \n",
"--------------------------------------------\n",
"61.7 K Trainable params\n",
"0 Non-trainable params\n",
"61.7 K Total params\n",
"0.247 Total estimated model params size (MB)\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validation sanity check: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "dfb9f7e0835941c0a4d7db2203a79f0e",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training took 1.67 min in total.\n"
]
}
],
"source": [
"import time\n",
"\n",
"\n",
"trainer = pl.Trainer(\n",
" max_epochs=NUM_EPOCHS,\n",
" callbacks=callbacks,\n",
" progress_bar_refresh_rate=50, # recommended for notebooks\n",
" accelerator=\"auto\", # Uses GPUs or TPUs if available\n",
" devices=\"auto\", # Uses all available GPUs/TPUs if applicable\n",
" logger=logger,\n",
" log_every_n_steps=100)\n",
"\n",
"start_time = time.time()\n",
"trainer.fit(model=lightning_model, datamodule=data_module)\n",
"\n",
"runtime = (time.time() - start_time)/60\n",
"print(f\"Training took {runtime:.2f} min in total.\")"
]
},
{
"cell_type": "markdown",
"id": "9fccc4e9",
"metadata": {},
"source": [
"## Evaluating the model"
]
},
{
"cell_type": "markdown",
"id": "914f982f",
"metadata": {},
"source": [
"- After training, let's plot our training ACC and validation ACC using pandas, which, in turn, uses matplotlib for plotting (you may want to consider a [more advanced logger](https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html) that does that for you):"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "0ed54af7",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
"┃ Test metric ┃ DataLoader 0 ┃\n",
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
"│ test_acc │ 0.9873999953269958 │\n",
"└───────────────────────────┴───────────────────────────┘\n",
"\n"
],
"text/plain": [
"┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
"┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n",
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
"│\u001b[36m \u001b[0m\u001b[36m test_acc \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 0.9873999953269958 \u001b[0m\u001b[35m \u001b[0m│\n",
"└───────────────────────────┴───────────────────────────┘\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"[{'test_acc': 0.9873999953269958}]"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"trainer.test(model=lightning_model, datamodule=data_module, ckpt_path='best')"
]
},
{
"cell_type": "markdown",
"id": "461b364a",
"metadata": {},
"source": [
"## Predicting labels of new data"
]
},
{
"cell_type": "markdown",
"id": "f32de2de",
"metadata": {},
"source": [
"- You can use the `trainer.predict` method on a new `DataLoader` or `DataModule` to apply the model to new data.\n",
"- Alternatively, you can also manually load the best model from a checkpoint as shown below:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "ba4570eb",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"logs/my-model/version_22/checkpoints/epoch=10-step=2353.ckpt\n"
]
}
],
"source": [
"path = trainer.checkpoint_callback.best_model_path\n",
"print(path)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "8c1f7796",
"metadata": {},
"outputs": [],
"source": [
"lightning_model = LightningModel.load_from_checkpoint(\n",
" path, model=pytorch_model)\n",
"lightning_model.eval();"
]
},
{
"cell_type": "markdown",
"id": "4389e871",
"metadata": {},
"source": [
"- Note that our PyTorch model, which is passed to the Lightning model requires input arguments. However, this is automatically being taken care of since we used `self.save_hyperparameters()` in our PyTorch model's `__init__` method.\n",
"- Now, below is an example applying the model manually. Here, pretend that the `test_dataloader` is a new data loader."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "3ff7cd66",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([7, 2, 1, 0, 4])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test_dataloader = data_module.test_dataloader()\n",
"\n",
"all_true_labels = []\n",
"all_predicted_labels = []\n",
"for batch in test_dataloader:\n",
" features, labels = batch\n",
" \n",
" with torch.no_grad(): # since we don't need to backprop\n",
" logits = lightning_model(features)\n",
" predicted_labels = torch.argmax(logits, dim=1)\n",
" all_predicted_labels.append(predicted_labels)\n",
" all_true_labels.append(labels)\n",
" \n",
"all_predicted_labels = torch.cat(all_predicted_labels)\n",
"all_true_labels = torch.cat(all_true_labels)\n",
"all_predicted_labels[:5]"
]
},
{
"cell_type": "markdown",
"id": "ff0a65db",
"metadata": {},
"source": [
"Just as an internal check, if the model was loaded correctly, the test accuracy below should be identical to the test accuracy we saw earlier in the previous section."
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "6c06ef45",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test accuracy: 0.9874 (98.74%)\n"
]
}
],
"source": [
"test_acc = torch.mean((all_predicted_labels == all_true_labels).float())\n",
"print(f'Test accuracy: {test_acc:.4f} ({test_acc*100:.2f}%)')"
]
},
{
"cell_type": "markdown",
"id": "4e80fe10",
"metadata": {},
"source": [
"## Single-image usage"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "335380c9",
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "45998e5f",
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"id": "ab0bdec5",
"metadata": {},
"source": [
"- Assume we have a single image as shown below:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "9fd24838",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAN8UlEQVR4nO3dUYxUdZbH8d+RHaKREUFa7Tgg7GiMZBNhrOAmbibquBPxQSRmzGAygunImEAcDA9L2BjQJ7JhZpiHzSSM4MBmFjIRjDzg7BiCUTQBCwMtLrqiYQdGAk0woSdRseHsQ182DXb9q6l7b93qPt9P0qmqe+rW/1jy61tV/9v1N3cXgLHvqqobANAehB0IgrADQRB2IAjCDgTxd+0cbMqUKT59+vR2DgmEcvToUZ0+fdqGq+UKu5k9JOk3ksZJesnd16TuP336dNXr9TxDAkio1WoNay2/jDezcZL+XdJcSTMlLTCzma0+HoBy5XnPPkfSEXf/zN3PSdoqaV4xbQEoWp6w3yLp2JDbx7NtlzCzxWZWN7N6X19fjuEA5JEn7MN9CPCtc2/dfb2719y91tXVlWM4AHnkCftxSVOH3P6epM/ztQOgLHnC/p6k281shpmNl/RTSTuKaQtA0VqeenP3ATNbKum/NDj1ttHdPyysMwCFyjXP7u47Je0sqBcAJeJ0WSAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQuZZsNrOjkvolnZc04O61IpoCULxcYc/c7+6nC3gcACXiZTwQRN6wu6Q/m9l+M1s83B3MbLGZ1c2s3tfXl3M4AK3KG/Z73f0HkuZKWmJmP7z8Du6+3t1r7l7r6urKORyAVuUKu7t/nl2ekvSqpDlFNAWgeC2H3cyuNbPvXrwu6ceSDhXVGIBi5fk0/iZJr5rZxcf5T3f/UyFdAShcy2F3988k3VVgLwBKxNQbEARhB4Ig7EAQhB0IgrADQRTxhzDI6d13303We3t7k/Wenp6Gtf7+/uS+b775ZrJ+6FB1p07cf//9yfqdd97Z8mPfcMMNyXo2pTymcGQHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSCYZ2+DtWvXJuurVq1K1r/66qtkffv27Q1rb7/9dnLfc+fOJetVeuGFF0p77McffzxZbzbP/uKLLybrt9122xX3VDaO7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQhLl72war1Wper9fbNl6RUs/Tnj17kvvOnTs3Wf/yyy9b6qkI48aNS9bvvvvuZP2jjz5K1pudI5AyMDCQrF+4cKHlx85r4sSJyfqZM2fa1MmlarWa6vX6sCcJcGQHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSD4e/bM119/nayvW7euYW3lypUFd3Opa665JllfsGBBw9r8+fOT+95zzz3JerPvVy/TkSNHkvV58+Yl683OAchj/PjxpT12WZoe2c1so5mdMrNDQ7ZNNrM3zOyT7HJSuW0CyGskL+N/L+mhy7atkLTL3W+XtCu7DaCDNQ27u78l6fJz/+ZJ2pRd3yTp0WLbAlC0Vj+gu8ndT0hSdnljozua2WIzq5tZva+vr8XhAORV+qfx7r7e3WvuXuvq6ip7OAANtBr2k2bWLUnZ5aniWgJQhlbDvkPSwuz6QkmvFdMOgLI0nWc3sy2S7pM0xcyOS1olaY2kP5pZj6S/SPpJmU0W4eOPP07WFy1alKzv27evwG4utXTp0mR9+fLlyfq0adOKbKdjHDx4MFkvcx69u7s7WX/nnXdKG7ssTcPu7o3O2PhRwb0AKBGnywJBEHYgCMIOBEHYgSAIOxBEmD9xfe6555L1PFNrzZb3ffbZZ5P1Zsv/Tpgw4Yp7Gg2OHTuWrD/zzDOljd3s/9m2bduS9VtvvbXIdtqCIzsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBBFmnv31119P1q+6Kv177+abb25YW716dXLfp59+Olkfy86dO9ew9sgjjyT3LXPZ47Vr1ybrzb5iezTiyA4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQYSZZ3/ppZeS9bNnzybrTz31VMPa9ddf30pLY8LAwECynvqa7N7e3qLbucTMmTMb1pp9x8BYxJEdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4IIM8/e09NTdQuj0vnz55P1/fv3J+sbNmwosp1L3HXXXcn67t27G9aafX/BWNT0v9jMNprZKTM7NGTbajP7q5kdyH4eLrdNAHmN5Nfb7yU9NMz2X7v7rOxnZ7FtASha07C7+1uSyvt+IABtkeeNy1Iz681e5k9qdCczW2xmdTOr9/X15RgOQB6thv23kr4vaZakE5J+2eiO7r7e3WvuXuvq6mpxOAB5tRR2dz/p7ufd/YKk30maU2xbAIrWUtjNrHvIzfmSDjW6L4DO0HSe3cy2SLpP0hQzOy5plaT7zGyWJJd0VNLPy2sRVdq6dWuy/uSTT5Y29owZM5L1LVu2JOsTJ04ssp1Rr2nY3X3BMJvLO1MCQCninUYEBEXYgSAIOxAEYQeCIOxAEGH+xBXD27t3b7K+ZMmSNnXybc2m/e644442dTI2cGQHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSCYZx/jvvnmm2R92bJlyXp/f3+B3Vzq+eefT9Znz55d2tgRcWQHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSCYZx/jXnnllWR93759pY7/2GOPNaytWLEiue+4ceOKbic0juxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EATz7GPc8uXLS338KVOmJOubN29uWLv66quLbgcJTY/sZjbVzHab2WEz+9DMfpFtn2xmb5jZJ9nlpPLbBdCqkbyMH5C03N3vlPSPkpaY2UxJKyTtcvfbJe3KbgPoUE3D7u4n3P397Hq/pMOSbpE0T9Km7G6bJD1aUo8ACnBFH9CZ2XRJsyXtlXSTu5+QBn8hSLqxwT6LzaxuZvW+vr6c7QJo1YjDbmYTJG2TtMzdz450P3df7+41d691dXW10iOAAowo7Gb2HQ0G/Q/uvj3bfNLMurN6t6RT5bQIoAhNp97MzCRtkHTY3X81pLRD0kJJa7LL10rpEE2tWbOmYe3kyZOljv3EE08k60yvdY6RzLPfK+lnkj4wswPZtpUaDPkfzaxH0l8k/aSUDgEUomnY3X2PJGtQ/lGx7QAoC6fLAkEQdiAIwg4EQdiBIAg7EAR/4joKfPHFF8n6unXrSht7/vz5yfratWtLGxvF4sgOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0Ewzz4KHDx4MFkv8+u+pk2blqyzrPLowZEdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Jgnn0U2LlzZ2VjP/jgg5WNjWJxZAeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIEayPvtUSZsl3SzpgqT17v4bM1st6WlJF/+YeqW7VzchPIYtWrQoWX/55Zcb1s6ePZvcd+nSpcn6Aw88kKxj9BjJSTUDkpa7+/tm9l1J+83sjaz2a3dnlQBgFBjJ+uwnJJ3Irveb2WFJt5TdGIBiXdF7djObLmm2pL3ZpqVm1mtmG81sUoN9FptZ3czqZX59EoC0EYfdzCZI2iZpmbuflfRbSd+XNEuDR/5fDrefu69395q717q6uvJ3DKAlIwq7mX1Hg0H/g7tvlyR3P+nu5939gqTfSZpTXpsA8moadjMzSRskHXb3Xw3Z3j3kbvMlHSq+PQBFGcmn8fdK+pmkD8zsQLZtpaQFZjZLkks6KunnJfQHSTNnzkzWP/3004a1gYGB5L6TJ09uqSeMPiP5NH6PJBumxJw6MIpwBh0QBGEHgiDsQBCEHQiCsANBEHYgCL5Kegy47rrrqm4BowBHdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0Iwty9fYOZ9Un63yGbpkg63bYGrkyn9tapfUn01qoie7vV3Yf9/re2hv1bg5vV3b1WWQMJndpbp/Yl0Vur2tUbL+OBIAg7EETVYV9f8fgpndpbp/Yl0Vur2tJbpe/ZAbRP1Ud2AG1C2IEgKgm7mT1kZh+b2REzW1FFD42Y2VEz+8DMDphZveJeNprZKTM7NGTbZDN7w8w+yS6HXWOvot5Wm9lfs+fugJk9XFFvU81st5kdNrMPzewX2fZKn7tEX2153tr+nt3Mxkn6H0n/LOm4pPckLXD3/25rIw2Y2VFJNXev/AQMM/uhpL9J2uzu/5Bt+zdJZ9x9TfaLcpK7/0uH9LZa0t+qXsY7W62oe+gy45IelbRIFT53ib4eVxuetyqO7HMkHXH3z9z9nKStkuZV0EfHc/e3JJ25bPM8SZuy65s0+I+l7Rr01hHc/YS7v59d75d0cZnxSp+7RF9tUUXYb5F0bMjt4+qs9d5d0p/NbL+ZLa66mWHc5O4npMF/PJJurLifyzVdxrudLltmvGOeu1aWP8+rirAPt5RUJ83/3evuP5A0V9KS7OUqRmZEy3i3yzDLjHeEVpc/z6uKsB+XNHXI7e9J+ryCPobl7p9nl6ckvarOW4r65MUVdLPLUxX38/86aRnv4ZYZVwc8d1Uuf15F2N+TdLuZzTCz8ZJ+KmlHBX18i5ldm31wIjO7VtKP1XlLUe+QtDC7vlDSaxX2colOWca70TLjqvi5q3z5c3dv+4+khzX4ifynkv61ih4a9PX3kg5mPx9W3ZukLRp8WfeNBl8R9Ui6QdIuSZ9kl5M7qLf/kPSBpF4NBqu7ot7+SYNvDXslHch+Hq76uUv01ZbnjdNlgSA4gw4IgrADQRB2IAjCDgRB2IEgCDsQBGEHgvg/av0i7OGhSbEAAAAASUVORK5CYII=\n",
"text/plain": [
"