chapter1/4_cifar10_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "训练一个分类器\n",
    "=====================\n",
    "\n",
    "上一讲中已经看到如何去定义一个神经网络，计算损失值和更新网络的权重。\n",
    "你现在可能在想下一步。\n",
    "\n",
    "\n",
    "关于数据？\n",
    "----------------\n",
    "\n",
    "一般情况下处理图像、文本、音频和视频数据时，可以使用标准的Python包来加载数据到一个numpy数组中。\n",
    "然后把这个数组转换成 ``torch.*Tensor``。\n",
    "\n",
    "-  图像可以使用 Pillow, OpenCV\n",
    "-  音频可以使用 scipy, librosa\n",
    "-  文本可以使用原始Python和Cython来加载，或者使用 NLTK或\n",
    "   SpaCy 处理\n",
    "\n",
    "特别的，对于图像任务，我们创建了一个包\n",
    "``torchvision``，它包含了处理一些基本图像数据集的方法。这些数据集包括\n",
    "Imagenet, CIFAR10, MNIST 等。除了数据加载以外，``torchvision`` 还包含了图像转换器，\n",
    "``torchvision.datasets`` 和 ``torch.utils.data.DataLoader``。\n",
    "\n",
    "``torchvision``包不仅提供了巨大的便利，也避免了代码的重复。\n",
    "\n",
    "在这个教程中，我们使用CIFAR10数据集，它有如下10个类别\n",
    "：‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,\n",
    "‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’。CIFAR-10的图像都是\n",
    "3x32x32大小的，即，3颜色通道，32x32像素。\n",
    "\n",
    "![](https://pytorch.org/tutorials/_images/cifar10.png)\n",
    "\n",
    "\n",
    "训练一个图像分类器\n",
    "----------------------------\n",
    "\n",
    "依次按照下列顺序进行：\n",
    "\n",
    "1. 使用``torchvision``加载和归一化CIFAR10训练集和测试集\n",
    "2. 定义一个卷积神经网络\n",
    "3. 定义损失函数\n",
    "4. 在训练集上训练网络\n",
    "5. 在测试集上测试网络\n",
    "\n",
    "\n",
    "1. 读取和归一化 CIFAR10\n",
    "------------------------------\n",
    "\n",
    "使用``torchvision``可以非常容易地加载CIFAR10。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torchvision的输出是[0,1]的PILImage图像，我们把它转换为归一化范围为[-1, 1]的张量。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data\\cifar-10-python.tar.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████████▉| 170M/170M [20:39<00:00, 155kB/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们展示一些训练图像。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "171MB [20:51, 155kB/s]                                                                                                 "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  cat   car   cat  ship\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 展示图像的函数\n",
    "\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "\n",
    "\n",
    "# 获取随机数据\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 展示图像\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# 显示图像标签\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "2. 定义一个卷积神经网络\n",
    "-------------------------------\n",
    "从之前的神经网络一节复制神经网络代码，并修改为输入3通道图像。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "3. 定义损失函数和优化器\n",
    "----------------------------------------\n",
    "\n",
    "我们使用交叉熵作为损失函数，使用带动量的随机梯度下降。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",   
    "4. 训练网路\n",
    "--------------------------------\n",
    "有趣的时刻开始了。\n",
    "我们只需在数据迭代器上循环，将数据输入给网络，并优化。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for epoch in range(2):  # 多批次循环\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # 获取输入\n",
    "        inputs, labels = data\n",
    "\n",
    "        # 梯度置0\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # 正向传播，反向传播，优化\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # 打印状态信息\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # 每2000批次打印一次\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "5. 在测试集上测试网络\n",
    "-------------------------------------\n",
    "\n",
    "我们在整个训练集上进行了2次训练，但是我们需要检查网络是否从数据集中学习到有用的东西。\n",
    "通过预测神经网络输出的类别标签与实际情况标签进行对比来进行检测。\n",
    "如果预测正确，我们把该样本添加到正确预测列表。\n",
    "第一步，显示测试集中的图片并熟悉图片内容。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:    cat  ship  ship plane\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztfWmMHdl13ner6u2vX+/d7ObOITm7NDMajSRblmXJTkayLRmJ7Mgx7EGiYIDAQuzAQCzHPxwB+WEjgR0HcBQMLFmyY1hWJNlSZMWRPFq9jDScVZrhcBmuTTa72Xv321/VzY9zbp3TG9lkU2x2+34A0cVb9aruvXWr6pzzncVYa+Hh4eHhsf0RbHUHPDw8PDxuDfwL3cPDw2OHwL/QPTw8PHYI/Avdw8PDY4fAv9A9PDw8dgj8C93Dw8Njh8C/0D08PDx2CDb1QjfGPG6MOWGMOW2M+cit6pSHh4eHx43D3GxgkTEmBHASwE8AGAPwLICft9a+euu65+Hh4eGxUUSb+O1jAE5ba88AgDHm0wDeD2DdF3qxWLQ9PT2buKSHh4fHPz6Mj49PWWsHr3fcZl7ouwFcVP8fA/CWa/2gp6cHTz755CYu6eHh4fGPDx/96EfPb+S4zdjQzRptq+w3xpgnjTHHjDHHarXaJi7n4eHh4XEtbOaFPgZgr/r/HgCXVx5krX3KWvuotfbRYrG4ict5eHh4eFwLm3mhPwvgiDHmoDEmC+CDAL54a7rl4eHh4XGjuGkburW2Y4z5MID/ByAE8Alr7Ss3ep79818AABibpG3ZDHXLBPK9abWaAIBO3KZjstl0X5zQb20iFh8TxACAIFR9bpdoH2hfJttI94Vw15RzxEkHANDuSN+ShC1NJuL+iOWpyfu0LSrhcRkjra0WjSGOo1VjD7hvrUTaqtQN1Fpx2la67wlofPjDH063O53OqmveCtzw+eyKv7op0G3UGrhGbbgzbv4SdbybZznJtby11uq3O/5jH/vYqn37f5TnNu6kbdNXrwAAmg1ZM4fuOgwA6OmuAAAyofQnm6GFl9VtvJ4jo9ZYpw4AKJcyfA7pa8TboVrEs7MzAICurq60LZPJ8HnpOBPIOTpJCwAQrCG6BUYaa1Uyh0YRrcl8Pp/ua7XoHB1+BgGgkC/wtaRvv/+7v7Ps/Hv2DqXb5YGj9LtQnttKVxkAsNiUdV1dmOb+0v1O1GKIeBCFKJe25UN+hannNn0AuSlO5PyuLVFt7hpu7HR9nss11o7h+2cC/V6I1ziOfpvLUX+zgfQblrZNVuavNn0cAPD1Z76/6lwbxWZIUVhrvwzgy5s5h4eHh4fHrcGmXui3Ai2WsqytSyNLpzmU0qYA9CWLIpa8tcTBX12TkcamkyoS+QJGLAGG3BSpc5iEpGZ0RApx0nKiztEyJLnEIX1hW3pfHPC55GttWMrPq75FLBkFEXU8brdVRzo8JDmHk0jDcH0LWRiG6+67VbhZiV/PRypHKSkycSKV5TFY2ec0JgORhuQsm5fQ10K5SPc2sPJ4NKvUlrSE2M9n6bylAh0Xqcu4tZNTi6yQ5fuuxtKM3XG0rrJqnbgpiiK5t07yD5SU7+Ymx1qrXibVWpuvKXDarYWcN+CLZVhKdVI/ALSbTR6fGgtLnbjGmkisSPmdsJfOlZFnOg5JQg8ySkKvL1Hf4ir3Q87XtHRcW0nGDZ5fJbSj1SYtKuBnol6Td4t7TvT4nMYcBPIcWqfZ8GRqi0CnE/Mxck1j3PtJ1kxvL405V+ji88s9S9y6zkk/4qUyNgsf+u/h4eGxQ+Bf6B4eHh47BFtucrFskoAVU4dlMsrEohImbVKBwgKbNZTa6qwNmpjIskrVsaLSJO1w2XFOdQIAY1cQcwAMEzg2FNWxHpNud2Wa1LNqS9SopSVqC62ctyvP5Jgi9SpFIpQKORpnErTSfUFqXpGxuxG0k/XNBNqE8IOqE7uR8y4zb7jjl+mmbpc2EdGcN9s0H5HWs2P6bWjWunayRtvGcK2xRGz2CpTZKxvStTKBtOUCNqe5fYrQbNbJNBOGisCL6L63m0KsBmATW4farJFHMmbTUjZTkOPdPKg15sjhmM2GOt5j+upVAMDwQK8cz+aVMCvXCvlabp6V5QcRH99UJLEjbNttaVuJwMq+mPsbq+cgNjTmfJf0o3//MP12fhYAUK4tpftaDXpHxGV5HpNuijzvysrcu+sGbJdtNeX5cg4U+bzcl3RK1Zpw69j9DZSNt8NjTvTy48tnI1m7hQITx3BmQzHpJM6cq2XqW+DE4CV0Dw8Pjx2CLZfQo5gl81C+jgFLGrlQff0d48RfykAzP/zTjpZgHcmTFelm14G7AQALc1MAgKlpkWQyEUnjAeTL3erQ9NStBEQdP08Sj831AwDaoZA8LZYcluZn0rZLEyxp5JXkNT4HANi3i67Z36WlOOfKKGN3wkdsV7tGOWjJ+Fa4K94SKT/tt9Ie2LWzo8SbNmtKp86cAQAM7xJ3t4TJ7cE+kTDzTCQlm+jjteYoy1J40hHJLmTpKqMIuQy3BTGto2xGSX0hu8Yq7SsT0L1NjNLIEnbHbTA5qtZTg8deLMoaDh1TqsVDnocqu1Q+99zz6a42awq9lTenbbkcOweoKUhdZ1l7DZS7oLHOOUDWpE0cMbi+hN6BuFYGoLWehIoQZi0tVNpaidnNSpHv8fPPpvtaUyStjzxwt/TtKj1zTSPzVuaBLdaJWM2rseRYYw/6hYAMmBTVr5Rmkc4btVlzactkLZbovuTm59O2aO99AIBaT3falrDWFfM9yydCrKYWgVjawnjz8rWX0D08PDx2CPwL3cPDw2OHYMtNLk4vN5Gk1XXqcEdHUDIB1WI1OKvIpjh26p8ySfA5tF/vW378JwAAz/39PwAALrPpBQCqHRf5KarY+bFJAMDZsUtpW653BACwZ/ggXTMnamWL1cVMWbJcdhqkJk5PSpqbYi+Za8aWKPqwodTn4S5SCYsZUUPjNqnNOhhuJR24Fil6OyJFr22aYfIto6J62ce8viQk+Nw8qcYTU2SqKnSJ+tzPEZE6qtGRgDp6dI3OrujFxpFl855V58i4yY+l3yEceU9tGeXX3XbqdiLnCCs0D8aquAP2d05cNHIs63ppgUxz5aKQgAHPt47ajDiyeo7J0JkFMSUW2E+7pSwjrTZdK8rqNUNtMUdid5S5yUVpZ5WPteU1m8TrmwH1zDsTYqDGHnd4rMrWYdgk0jB03zOJrAUzQKa42qL0rX32JPXXiFkq4emqOv929Xxl2xw/clGR8jwf2tGiwebTsMFzJZdEcxf1sX5FTKtdhp550z0g4+PrtgNHNKvYC57vUJHsUbB5M6eX0D08PDx2CLZcQm8G9CWer6kIMpZuessiVlSYZIpYQtGEVep2pAgaR5rWarNp29e+RHljJuZI4phYku/Z+Ut03PnLkuI9zJO0HoeVtK1UoS9xpkj7orxIBjmWIvOBjGWqRVFqI3v2pW0NJmvOnCEJfWZO5ZTZTec9MCiaQoZd94xyGxP5jMervv42uTGZNA3MXENA0FJ5sIaEHrMUlrA0oqNZXQTe1emFtG2hSmOt6/wdNRpNkCPyuVqXe1suskSq+ubk/Y0qIDeqqeSMc7GT+XZk6JouhwlHJiqXw4g1ykgxj6Gh+bCxvns8PnYEiJVr29IizdsFfc3IRVaLNLm3QvPmXBRfevnldN8b7r8fAJBol8qY5jevXXpZU6jXWAOO5Pwd1hDDSJwD2pwvqNlcPyV2rKT3hNew1TIkOzG0tHsjX7d7kedqcDjdVxjaT/2xQkaCXS/twK60qZ7h3CxXKC8MlAtwlZ9XO9yftmUS6lNDafgl1hJbizS+ps6xU+CI3Krcl6iftAeTUW6ZnK+li38aKg2gY2juTaBcdLH5aG8voXt4eHjsEPgXuoeHh8cOwZabXK7WSc2YaQsp+s2/+wYA4L6jYrr4sfuJbOhlf3VNxrgkPIFSX2ImXxSXhrPnyc95pk6qkC32pfvCMpNvfWIeKHD905ZKmdpiIq7SS32rlKWPk1fIhLIwq8gSVgnzBTHNXJglMjZTIXVyclyqS5WvLAIAdlXk+IJL1ZsoMm0FqjWd3IxVTqVqutTCoUr05LZdOlCVEwtBsvpb76JYta1jic0BjhwtKOKswRF148rkMjlL24kizNpsT6ktEoE8OSXzN3ZpHABw35FDadtdB/ZQ/5VffkrOukhfbWVx3dZhCtegSkM2+SVtMScEbOKrz8tYwOYGy0mdwoKMPcv3Kqvm27TJ1BZrMwVHQ5uUiBVzU7VKpoWJCTm+VCnzNVViMp7z1hIdl1f+8FfniFh9/vtihinl6JqHD8mcRmz6adZo/RUilUiqSWsrVmmkY/eoNdR8rISaYpfCNlkWK8L71LOcYXNX7vQpOv1z3073dd7MpiqVhtZyjEh2UZ6NBmgeyhzvEebk+KRE5zdWEfWcHK+rX95BmUtsrlmiNZkZFucHXKR9UUXMoo2rNL9hUdqSo+Sb3uDEXoEi8bMdmpxI2RLtNTj+jcJL6B4eHh47BNeV0I0xnwDwUwAmrbUPcFsfgD8HcADAOQA/Z62dXe8c1+xAN0kJtWn5trSzRDzO1FTy9xa5EVWy7OaliBQnkYahkDaNFkm4VxX/NLVIX+diDxEivYNCVFYTkjQGoKLymEBpZURqalRJgmks0fH7FblSY2l8siXSsmFpaX5GSWUsrdT56x9mpd8TCzSN4/OiFewfYA3kGl/wuboMtFwkrSFQeSVcsY5lgrcja1wQ7rK0tWt869dwh7wyTi6dfX2k7RTyIvk0GzTmYk7adg2SpmWV+Fat0VhLLMm0GirdKQ96qSnj66R5NpQbXeo+6fatGuYyifFa3pZ5V8BAHeQk9JzSCspMPnczmRWw+yUA5Pge57VAylpU0JC1kBY94EIprQVZa10l2tfbJ5rk2THSAs9cvJK2nTz9NABgdook0qWGnKPWppozEZQbIkv+D959NG17308+DgDYzeu5mZdxNqpV/p1cs8IF6E19EeshE8r6c+mvHTkKSArZSMmV5Vm6VmeM3HwrSttYvEzXb+UlGtOC3gvmymTaVhplQrPCmifkWSqwu2x2TvrdYCK6MzWetmV5DjsLNFe5GXGMaNdZmyqIhjN3lpwpsgWR0LtGiMR1qaCsclFsOjJcreFWsnkRfSMS+icBPL6i7SMAnrbWHgHwNP/fw8PDw2MLcV0J3Vr7LWPMgRXN7wfwTt7+FIBvAPj1m+nA3W94DAAw9syJtK3cTV//x972lrStGJKducUSspY+DWeji63k++gaovrVL758Ss7bQ9Lh7v3kymWVLS7DUnjSnE7bWq1k1bVC/qK+8tJLAICKSlBfLJFkUFJ2tMtXJgAszzMTstTRx+5mc7Ni/5udoe2z4+KaNTpMLllRVkU3rEBUEU0hZum6revvsW0y/Quxa7pgFS2R2jV8GJ0Arzwk0wAXl+8DynW0h12/2m11LpbaimWxSToJ3XCwmFEuYrmCc+9SZdWYGFlmc1zVN7lmZvkhvHt9Ef3iuXPcb5nvxQVad3FbNIVLl0g7meU1UF0Se/JQP0nV5ZIEBYVcnKWlMhRGnGso4FxCVSW9N9xgVKGNC5eJfzk7JjxDtUW/zXez61xJJsatxFJWZLfx8xSMc/nyRNr27W//HQDgXuYqBntEIq0vkeTvysMBQPteyqeyNL++Yp7Lytitk9YTpTKzhhMoN9slDgRcevSNAIBK9KZ0X22R7kFb5X0yOZ4bVZ4xU6DrVtk9U7vbtjlfSkY9G3WeG+00WGe7fm2JrlkqyFgafHyuLM95Xxe9e2L1rljitQt2oyy0VcZG7pP2MG7fgvxJN2tDH7bWjgMA/x26zvEeHh4eHj9g/MBJUWPMk8aYY8aYYzpPs4eHh4fHrcXNui1OGGNGrLXjxpgRAJPrHWitfQrAUwAwOjq6SqcodpOpYP8hIWjqbIHYd/Bw2jbAavvc2XMAgLaOLuuQ6eKxd/xM2rbv0KMAgIMPnkvbnnuBzCS9ZTJhXJ6UXC4RuzHldHEF7u1SVciuuRlSO/vKGX0I9YPNKgODksvFFW2YmhUTiuFoyi52eYxCRYywyv36xbG0bbCX1PIje5Tr1Ap84o//l5yf+5FR6l+5i1TGwweFCH7zG8itypW9tMos5EhGq+0rLseOMqs4wi6bo/NrsjObJRNKf69yn3S1YVWNxjRHSIbO0ejI+eeYJJ5TqUoX58kE0Naumkxk9rPr2ZHDQlhlXDShLgwfLDPALMO3//4ZHq4qsOKI7LqshXNXiLhLa38q8ai3m0wWJUUS5/i4jHJljNilLuCaojVFaEZ8DqvyFl2ZISK9rdjtYpdzt+N8R0vK3ZLvR6Mh/a500Xnf+qYH07Yqp3xusIvuhQtiSnn99ddp7MrF7vw0zX29JueNckLuA0CpJA4GHZ6HdqzvGReaUWSgYRNUYZiIz4WqjOXqPI3dKHfcFtdMzWpycY5+43JB5bLyHCzwGs9n1KvPpTVWkaJNjl4G1wyer8uadGl0iiqatmsPmXhDbQZM6+HyvdK1LNybQy3K5Bb4Ld6shP5FAE/w9hMAvrDpnnh4eHh4bAobcVv8MxABOmCMGQPwWwB+G8BnjDEfAnABwM/ebAfCHBELlyeOp20PvYmS8Ze65YsfLhIBFbOUEKnyWWcuEnHx9t6DcuIiBZ90lVSV9oiuVWA3wXxWlQrnr/Pu0ZG06VWWTLKK3FlgYubgXtIojt5zX7pvZoaLWVQkQOEyu1MZRcL09JJUO8/Sp85/UijSb+uL0u9TFzjYQxFbw5K6go6vqeCnOm1nVJDPIgu4RdUW33sPAKBhmTxSEnqOJSUt1bpCFToLYXcfaSMp8aTcHZ0bVqikcRfppWWRhKWVcxz4dWlSFL6ZadKI6nWR7OImS6Iq54vLKbJnLwVr7du7J91XSteKJn3Xl9BfPEX9KBZEI7KsETY7cl+6OWumI/9aSgq+ukT3IFRz1ZUnjawTCwlumAQM2bfNRBKolquSZNlqC9k6M+PIUF0ujf62OEfMYlXmqsXurHsHxfWxv5cWjwtcAoCZWcoD099D/Xj0jfen+8bYNXW+Lmv4tTG6L4Fa1wcl7QoAIFKZTgtd9MwtqZJyEas0scoyGHHwTcBrMlHuloYL3kTqmm6r3VIZJlnLjljy1hqRI0NjpQW60nYdtSozBSYt49VZW13ul0xHaQrsMaAzNuZjl6GTr6WWnAusW+5FvPnsqBvxcvn5dXa9e9NX9/Dw8PC4ZfCRoh4eHh47BFueyyWTJ4Km0dDqM9dvVBGUxZIjmcgUoOuNliNSmT751MfTtp/+Fx+mc6jotizXUnTFMg4e2p3um5whgquxJGrzriHyW9cFA5pc5/HQYSJs7zosZO78C1TLsbooaqUjdToqQq7OJpEerj8YW4la6+4ldbGjKhKEAY1v7LKYIobfgGX4uX/2z6WPTBaWVP4YR8IUlKnKpZZYWOD8Kh0xBWSYpIuU/61l1bWu/LNtQudzVdE1ERvx8ZmMjkBdbbZx/rcNzn9SUjkyejmfTtySvuVDGtfctJgMxi6dAwAcZiI9DJRpybqK9irF8DVcfhfYrGc18cixBYVQ5mPP3ruo/y5N8BVZa1NsKhoeFo/e3ACZgapz4s+dcCRsdy/ZK3I5iaVo8JBrHTG55Pk5iNuyxkImF13Rl0xWFdrI0/Zjj4gJ5ej+UTp/S9b62ddpXK+feBUA8LY3C2G6dy8df+FlyTnUjl1OpfVrimZVP7JcUzexYuYsMAneUWmKFzlSNmbiM98tpqLhEpvAFHno1rU2V4RwNVPpry7MsRYsP5va5BKzr7tLUxyoa2adoUclimryO0XnjorY5BiD88fooiv83Oi6rtr0erPwErqHh4fHDsGWS+iGI8hqSjJusISZ0XkcptmliPO1ZDCX7hvpoS/mqeMSFXp57DRt1KT02/mxcwCAh3dRdOru/cIsjk6ShFQ9LVJIX46kw64eKSv1+utn6ZqjJN3PLYj01OYv/cRVJYE5skS5JtZYQjec20FTISWXvTGRyM+sofloTV3BekjaIkGkEoraX87SeQt5mdM6Z8qrtakf586ck2syKbrv4P607exFmssv/fXTaVubM1zmOV9LUZ3fRdd1VyTqsKebpKyHHxYVY3CApNK79tCcBspd0ElZjrgChOyqD4n0NjpC92p0N5HaOoNfjV3blmks1xBlMkzUDw6Npm15JqSnpsSdtMpRyy7cr6EiQLsHaW3tVq63Xd00zsqASO3TTKTHLLG1VUU35yJZU0Riq+0IT9FYsi6jZ47uccaKBjXEcz/YK/cgzwTfYK+wmBV27Zu+cAEAcP71c+m+XX20/ucnnknbMkyGt8L1XyGRyl0SchbJvMrvMjdJBO/MkuRQuTpO89vbRev/gftEU8iwdt5UhHCbNQRN6Lv174q+BIqod1KyLp0Yp0SsZi2X5wbSmVyRnkOeuYiP12vX/SbjNCf9oPPpA+WCGV/DlXaj8BK6h4eHxw6Bf6F7eHh47BBsucklTX2r1JeRAVK3tPr+tZfJJ7yXk+wf6RMVKJ9jUigSX+yrk+fo9E2JeNt3F/mph3zeYkUIqIFhIqymZ0S9nWcyVBc2HxoidTlic1BDkZcu6VJdmQc6/OOOOkmjyak5O/Q97VcquOFag1kjY8kxaRTb5ZF4Gn/5f76SbiecsD9QPrxlJpi7lPnjwBEa82A/mRj6RySKtI/7lFfJpeaOkznqe8el7mrdumIa9P9IqcMV/u3hfWK2edtjj9C1SuLjXWK13Wm8LTWnHfatrs2Lia3NftyFovStp4fMDROcDG1KFckocMTi8C6Z52JRxSCsQC+b2EJlTmhyIQ+jZKCZaerTwgKnQVYmwpAjDM9fkgRYlQUyl3R3S5yC8z9vslOAUQRhzkUzluS+F6yLLNW5gOmZKBXYHGnFHLOnn+alqAjK6gL1u6NMOa74x0E2ER1/7Uy67+hRSsQFRYBevky+6fleMXsBens5CeiKrSTK/LHIMR1Xr4opcW6Wznvy5e8CAF576R/SfYcPU8zHgcP3pm29A2w2UuYKlyraFTvRhoww9WFXfUsLvUibq5ErhXQU6crHa149jaxeg21PSddlye/4rOp+63fJzcJL6B4eHh47BFsuobsoru6yEFY9XbRtVM6QBUuSxtQsfSkHuqTrJSZ04kAkk3OXzwEAhnslGf5+/sI7d7DvPifRqZfGSZLvKovUnmG3qldOX1A9dpGO9LepvqpLHKHXowoSdFjsHJ9QCfi7qE8Ru0YViyKBufwnaAuxGlepb8ND6+dyefaF76fbhQwRlM2mELZZJvXe8tY3p23nL5GkPc2c1AP3i2tblgnNWlOk/AxrNo88IoRmgyMRsyxNHjkk0br3c4rV0QGRSCtFureJclO9eIWiFCdnubjH1NV0X5XJ8rk5kdBbnMI2o1wwXS4ZF0ncVgRlsYfm7QHI+Lq7159LJ2nXVCRqaFwJP9EKYk7FGnEEcmJFPsrm6PwDAxJ5XOY1nleuoN3c74jvmXbntOwa2FHupN3s0hmo6MqE08RGLrqyKZJ3NyeQsR3RGmPWeloq0rHO96PIa/P8FVl/r75O2l+zKRGo7QbNrw019b4+nFSbz8vY77mbIpUP3yvuw7VFktZfeZ5cgF84JkTst79FGuLxV2WtH733IQDAkbtFau/ppfXmyOJwWR/d/K6Re1mTra5kXmd12UcXPRorEjVJ3SfXx7L01MaVzZQ1rFNs3yy8hO7h4eGxQ+Bf6B4eHh47BFtucnHRe7uGxCfc1RhMFLk4sodU+WNsSpkzkqLWhqSWdw8I8dhdYR/QvKjWB9jkUuaUvX/0iT9J99X4Wgt1IdNq7AesM23u4kjOxgypf9WcviaZhV47If7wExNkPlhQ0aM9PXTCSonU51CRWBmO3gtrl9K2wRLt786LQqeSkAIArl5U/vN9ZDbas0dIwPvecITOn5NzvPIiEU/DrAaXVTWjSa6vWKqIyaq/Qse97/F3pG0BO3R3d9NxA/3iPz/DqYbPnpf5mJ8jM9DCvETHLjL5PMdpimcWJAK0wwRvRqU1znKFoEBF1nVXaFw9HFnaq8xTOTZpZQti2lqqC+m8Ev3sQ659+8tcfSZR6V8zAc3HEPurGxUlm2WfaWcKAoA8R0uGKs+uM7GkVZqUycX54NeqsnZcxGJOLUrL5pfaPM33pXMy3zPs/NxTkOOHOcVwPq9r8LIJJSJzU1QU8vwq1/fcOyLPXBdX81pork/kJSotrkviZQPdRn0LlW96Tz+loX37O2ntHj4sJry//eY3AABnz8qzUX2Bn9sFMck9+AaqdrR3L51Lp6eOO7TGY9W3hE27y6p0pfVz3V/Z5ertaoLcWUu0z7sjSNNrLSNF+R2nzDbahHOz8BK6h4eHxw7BlkvojgSs9IqE3ompW7lI3MCOcmGGY8+R5LWQkQi8xJC0N7xbvvSvHid3px/60X+Vtv0DFy6oVklKbLekwMXkFeeKJ9+4Ja4BGKmovN6AJPjdBTrH/FWRhjohScbDQ0KsxuzqVVcSYaNOEmmVybdOIhJYu0GRckMZkQRHyyRJNTvStlJCv3TylXR7gYmzn/4n/zZte/xxSo75N18T98YhJguHihxFqlzh8hw9N9wtkloXb+eVu2CHpRonieqcNVdOkCR1YVJc91pcqCTKS5rYri4ikYdYYmy3VhNRGVWkwOW80LkvurpoLJVKF+9TdSo5n87EhNzvRmP96llFlk7birgtsAtmT0W0niRN5UyEZkHVSU1JLyUdJpbbtBzliou4v4qs6/D97sTS14VpGoN+cDMsoS/NkzY4flmio4f7aCw9JYl2rrF0nShNocNndETsbi7YAAB3c53Rh+6ToiEnz9Dz8sL3xLFgJXTK6IALUASRaN0ZdgqIVXSlSz8bMEl85KgQ8Am7+Y6Pfy5tm52isZ5qilY3cYnqE991hEjXe++XcwwNE0kdqXdLp83FN1RK3Zhr5Lr7uGZBlGU5ZVbvT1M08zzoU6TFZJTovywa9SbhJXQPDw+PHYKNFLjYC+CPAewC+fo8Za39fWNMH4A/B3AAwDkAP2egHt4UAAAgAElEQVStXb8E+DpwuUt6B0SC6PDXvBFIYYR8mSUNzlB44aIEI7z9zeSO1liSL2axi9wExy9J7o3TJ6naecdVA1feTFW223b1i5vZ/DxJRt1lkUjvPkq5JZ596TUAwPPHz0o/fuy9AJZniTxzmiT4OZWx0bk8Nuokme8fFsmuwEEkfX0iGduIJIdOa323poYqBfbgG6mP73r3u9K2/h6ybf/wW5T9myW7LtYUKmWRmkMu2uCq0gNiq9VFB+ZnyW5bYYknURlkDt39AABgaI9kpJyZJc2mq0dcGV3mPmNXV2R3dlhXGg0AltimbFXJMFc44eI42f6dFgQAbS7+ofO7FEvrBxZVWZvqUgUuXJDRpMrTs8DBTglnZTzsAnAA9HD+kzCjpU/a1lpMi+uZ1Zg7aTSl350WzZVRBTFsk44vKY2lp4c0nEKWbNyRkXXSw9pdd5esyRafo6aySbY4w2nAgS69SjMrcpbSMcXTsHCN++8+krZdVe6mdC7NB7C9XPUty7sT/SCy5OpszC2lre3ZewAAcODAgbTt2Qm63x1VHu/q5Bz3h6T348dfTve5wKm77pJ+Dw+T22RXl/BF4AC/Rott7urZy7BGpoOInNuijiuyRrtG0qjS06cFMQThLShwsREJvQPg16y19wJ4K4BfNsbcB+AjAJ621h4B8DT/38PDw8Nji3DdF7q1dtxa+zxvLwI4DmA3gPcD+BQf9ikAP7P2GTw8PDw8bgduiBQ1xhwA8DCA7wAYttaOA/TSN8YMXeOn6yLhGo3dfVLUoFonNacWi4riCDBXK/LkK8oVrkaqTbkkuUi49gDOnxQ18RKTRW97G6XP1WlJuzgdbt+ouEldmCGzSr2pktuXSL2tDBJp9HCX1K68yur4ufMvylhqZJ6Ym5drDQ2SatxtqT/7y+LqN1ThohBGTCguZWpJqbDi9Ec4dM9D6fYHf+nf0PhiUctPnCZiMjEqBw6Tp21W/2bmVNKaxOWxEfrVFVZPIMTW4gL1JJwg1fiyqgfqCpUkDSGbSkzAnjklprCznLLVuf31Dch8OPPA/LyQXtNTRAxaZUIJ2B3OBC6viYo8ZgI2r1MHL62klQU5dpGcnpKxvD5L13RRlgDQ00vk98gI5RNpqajCdovMNomVPi6wWayuzEExR3CGbM7StSudWSVfkrEU2F2xodZuwkRiqcxusGqdZDlKUhPIjmBuKBLQ8HGOlGyrIiZj02RJrakapI5U3DUi638lQmVySLfVNWF4vpa587nfmFX7XJRpV5eYg1KyclnxEmfCo2stzsp9fIFTUL/y0rNpW18/3cddu4QI3jVygK9JZph+ZYod5IK+RhHv7j53lBmww6Rp6raoXR/Z3GWV+c0mK000N44Nk6LGmDKAzwH4VWvtwvWOV7970hhzzBhzrFZb37PAw8PDw2Nz2JCEbigF4OcA/Km19vPcPGGMGWHpfATA5Fq/tdY+BeApABgdHV3F6i1yIpGCylSXZp5LVLk0JlMG+kh6OxlINrjJGZJ8pkP5wnWX6St6zwNCdJw5R5KgKyKgicojR4gkOXLwrrTt/DhJJK+88r20bXqKg1S4CEKvclUbe4Uk+vEp+d4ZJnZDFeA0spfcv/bzF3tfl0hgeS5l1WzowAeSqLRb1Up84Bf+Zbrdu4ukppe+L1KwI5daSgqImaRzpdY0KeNKe8VaguC2YJkYwLlTOAvm1LS4KDq3OxVLgp5KD/dHJN2ZadZGWEqcmhICtMnaSUe5fcZcBjBUuVyKeZrnnHNp1BXZXfIeiPRUUFkkV2KOid7Ll8T9r8Rk9T2q4ILLSFnk/DSNumhVs7Pk3tpuyzhrnGulqNw+uyu07ks5+ltQZGfEUmesSNFOp8XnVdk7XfmztBiDKprAWm5bPXlRyKReolxpOZvk9FXSRKamxcXTZUWcVfl0nKaV6xJtaiWM1RI6/dVEoWGpVuc4SSVt/usISACoL1E/rlyRghiXL9P2fFGOy/A6ciR/SeWPKUZ0nCbIL3FRjVPn5J1Sr1MRl05M5xoYlGInDz5IAYpHDotEPzhIa6HSLc4duQJpEhZ8ffXsddIkjoqYvh2kqKGckh8HcNxa+7tq1xcBPMHbTwD4wqZ74+Hh4eFx09iIhP7DAH4RwPeMMc44/B8B/DaAzxhjPgTgAoCf/cF00cPDw8NjI7juC91a+7dYPyvkuzfbgTOnSc3Zd0TSX+YDTgPaEuIqYrVJiBEhUctctOGee8QP+G++8mUAQG1e/NWL/URenR4j69DePUKiHrybCi/klBp/aB/tn5sR9/pXuW5pwoTL2KyQRwtM5jZiMR8tzJFZZ0gRLuenqa1vL5kfpnPKJzphElWZV2zEtRQTUd9XelG/8OKxdPvl79F310BMOS5fRqSLMKSpYDN8jKjqEafb1elOXT6VrOpvwH7qoaV9laxEyQZslmqHyjzAkbPKbRhZzrXSrrF/dFVMVi0mDU1bRY+yzaelSPOYo0Gri3R8Ud3HwW7qR6RMHc6ysRY12jdI66RXFR5xBRoiNR+LS0RMLi1Rf3M5MZc4UlGnXx0dJjI8lxfzgCNDLecTqTakRw0mnOdmJb/Q9Az5eteVeedeTlOcYd/+5QUduN6pWk9NroU6lkZHiw95i81Ztaqcf36OTI9ZFfXqxv70176Wtr3jLQ9jGVTxhsT5l3dUhCabZJQ7PExqDqJ9oYqcfen55wAAS7Pi797P/vUXx6Wtwj70WX5uEhVhXSmzP7yKD8hGXBgkp+IwAjbjzpKZ6dxZicSem6V5e/6Yyt3DcRt790o07SgXjBkZpWd/dFjeNyVO020Kqt5psH5sxEbhI0U9PDw8dgi2PJfLi6dJWt73wGNpWwL6OhpNAvIXfoEJmrk5IW36+8hl772P/1ja9tAbKY/DZz7/F2mb4bwM3Vx9ffeouFyVmawLOyKZ9O2i6Rk5KFLWPBcneP5FkoLHl5S7VIYI2O4RIYoGDlPbssII7CZ4got2nL4iEmyW2aO6ioys8jR0EpEq3rPCSfTb3/xqul3jzHPZjCpdVnSkrNzy0HL+DlclPaMldOpHPqcIW3b7y6osfVGJxprP0jhzKh+FSxViVJZIR263VeGMBhOeqVSrI+z4eF3aLg3xVRJxT4m2u0s0pnJBpOBchs6XMXIfjXI/XIk2k3TazTFil8p4GdHnyu/x/CnROM9SeL0q46xzhsm68jl1mlCQcW5ssuZPHH8VAHD+3Lm0zUU5W+UOOTpCDgB9nPGyrrzJ3PbcrBCa00z61pUG7HIOOU+0uQXRkgKe+2Ika8fli7lyRTTglRJ6WxXVcKS86cg5XFSqdtazoDZHoi4tyWS5Yip3HxVt/pGHHgUAPPeyFL145lnKIjrHxVHijtyDoREiN9/+9renbRHf53PnxcX5mWcoF9QD91EUeqVbnCsmeMwTE+IA4NburmFxbzx48ABdnx0Lqovi9ukcDDKRaAWNNXIY3Si8hO7h4eGxQ+Bf6B4eHh47BFtucjk5Tyr9VKxSj2ZIBQ9aSkVJXA0++js6IjaHH/khIjTzGVFDD+6nyM+f/MAH07bP/sVf0bWu0HnH50XZazROAwCyEJV3pk7bp8+LWglWi+wgmXR6h8X8kNYVVNGYCZsnEiMmAJeMap4jOfMZlYSMU9hWjUouxWSkTbRKtlw9Gx6U6LnxOhFEcSxqdoXrnEaqbwtTRPYuLlS5X6KaJk5dXit6TZlVMgW6DzZD13eJ1QAgYJtLUSUrc5Xp4/Zqcxo4CZTJiu0iz+RmQZk/+rpITd2rYgD2jJD/r+M9mw1R1QNL6ylSkX09FVp3Ncm1leLkSUoJe//996VtBTah6OkImH5MODpwQkXJumRvzboya7AJMVZmlUOHDwAABoeo/7rwQobNPD0qUZYjVHWZTOdD/toJShu7pApiuH06hiFhk1J1Ueaoxv2scTRrS5nEXDGNCxNCPLoar/E16mDaZRGg1m2kcFGeKogViSNS+VYVVL3dH3nnu3mX/MAVrzj6kJhsH3gT1c11ZVcDRRO7AiyHDkm8ScRzeuCIpNkd3UdEc4EjjruVycWNyxVwAcSsMjQoacBdsq+QTVWBYn9jdnBoKztdYtafy43CS+geHh4eOwRbLqGfmKNvyhf+VqIxH9pP0squrBAGRZYSRnbRF3BkQKSWuw4xuWlFqhjnvCqf+PRfpW3PvUgkk4tEXRZ4aR0pJeeIc3SNWBN97ArYYYK1EyjS0M2mKiXVaPF51Zc4YoI0ZGnMqlwnHaaIMupr7kqRtdrrR5LZtkj03SWSOBYVsdqOSWq7594H5DejJK1McnTgpIoOXOK8Ljpdg5MsbSznLUUkhdzzRkpLelmVlru6QBpAvSUSY50LS+io1By7UpZYE+lRuUsGuYL7yKhIPod3k1vhUE7E1CV2dZxht74wK/NXLBEJXlYRuf2cv+PyWSHCHNos3TeWRMMJHBmpRExXvCJm18RTp06m+xbnHTEtj5grAhIp8TrhkMGAI22hXDH7WavSZGuNUy7X6zKnFy+OLTtOBR/CsotnrSX3zEnX1SnRgDPcT1fyr6MiKavstthRrpISabm+VFlX2knILpiRVRG8/Lx2VARvh+fBnV+XsXMCf0dpOK4cXEvlUBndx/mYEk5Rm6giEvycn70grqD1lssDpAqmdB9cdv3ZeblmxBJ3qXJABuvyIc3LmC9PzPA5qOM5lQ7cBcCasqyPxuz6ZRE3Ci+he3h4eOwQ+Be6h4eHxw7BlptcllgN+ZvnRV09+TpFj77nTUJK3TVKqv3ZMxSp+Y43i+kgz6r6YkvUuc/8NaXHfP5VSbBUc1FqbPIIVKpSpxYFKrrNmUlipc412RTSZpXQKN/mJkdcajIoilbXvyxyIqEsXAXydBdiJhV1UqwOE4jZLqnyszIX2vRlScQVt0l1qyt1uHaREpP1qQrrg5xWNsNVcgoqi1Y9dBVYtF1qtZpdq5OZ5h1cNer+eyV51YULZM6YnpNI26Yj2xSZFjHRXWAWa0ARoD2lEl9Z7sGVKRrLiSlJ0mSY2KoMkRmpUBHCtMgkqk7LW1Yk10oU+J61lFnDkdXL6mQ6/3M2V1QqEr2cZ5/+cklIvZDHVVTRps7Eceo1Suw2PyOmgHmO6IyVz3kmyxGraj3lWH83PH81FW06ycRdrSnqfMhj6O2W9dRi81yNneQ7KvlXkppXdP5Xng+zvkz4rW99XcbSoapBpUjmI+Z111ZmFUfMu4Rk+llqs2lLP4+OcGw0pS1OK2BxKmpVP7Svh8y55bKumEVj0PyuScfnEp6piE4ec6BMKBEn/QrM6uPcEJaFVxh+fxTl+KDB5kJFeN8ovITu4eHhsUOw5RJ6/wDlt5iZlc/jOEe1/T3X7QSAuL2ft+hLOLhLojxNSF/g7x6TaLG/+hpFejUTkQjAX+ogWP0di1lytOoz7dzRtJTgojwzLBkY/TnlPBSa9HK1KHXumZCvH1qWOKzSFFjK12L7yC6SJrsqSqqsLZfQd430pdtjF8Z4TLqYAG2fPXkibZpnd0J39apyi6yyNJTEy5hjOl4VE2g1SaJ7/m+/AgB4Z0nG+QCPs94t0rIjAXUUcIMJu3mO3tTk7PnXKBpvqi6Ri40MXb8wJGPu3UUSV65CYwpVpGiR3f5yRSHZTbj+0neusXFH7oGLMk46SlvjsTtStKAiKQPWGusqJ0pzhrTFC7o4Bc+DSyHr8uUAQp5n8kor4Eu0WjJ/i7MkkTcaS/xXiGx3p/JqzbfrnIJX1X91BKb7q8lI517YUdqJZak2m1mfqM+rSOV2yPdFpcTOsdNBolxdndtmwNfUJHTC+W60VuAiZhOrooB51NbV7TSKhObbF6i6uFHIKaubEtmaEqQ8PF2ztM0as9a63Zox6tlY+Z5pqahXy+doqNdHLiRtanR0P24WXkL38PDw2CHYcgndSbMZlQWw0yDp6uyESGXNKgV7vOMRqiBf6JGcCfNcDOKb35GMg3W2/bZVtrscu4056WOtCkqhkhbSj62yreVYsjNOVArU8TmSQgqq/JlzcWqrQJpFltpcUEZTSYLdveyyOSKJ8svsD1lXgSArP8X7jkomtwV24auOTakjOOueckeb4etmecwtZS8Xu+1qt7RlBQkYp16m/BkXF0XyGQxoPpZpOCy1LCl7/RVLUuFptqmOqRwgtSJrOPukwMDwQZJg8j3iupreB5aaymXRFIpsTw/UGrPXsP0ucJ6g2qK4LU5epjXZaEjfXPk4l8dD32On6QUqmCnDgW+OVwEkw2XENnftothmO7LOB9Ns0tpZVO5x7raVKuwOqyRD26Z5bi7JWndFMuaVROokc2efNspentjVwWUut41J1i+6kqj7uFQlHqUY6ntAf2O1mF0AVIvdcDsd5crHhTysksYlq6U8hx22ocdOG1T32gVVaeHZWupns6Fz28TLjteau035nFi1uaBCXSRm+TXDlu43587p1YVvaHsUXkL38PDw+EcP/0L38PDw2CG4rsnFGJMH8C1QTYUIwGettb9ljDkI4NMA+gA8D+AXrVWhmhtESjJpYjAk1bGlSJuJJVKLnj9BxNJ7a6ICLVoyRVyaFZNEnlXuTk3O0WAV09WAjFQUn9u3zC3NOLcnOc4Gy1POZnLigrbErl4tlYLXmV+02cGZWKocsVruEfNKL+eCaKmUn6+xS1tGuWu9aYVWVukVgnBwmPKrjCuTS6r+qd802azi6k1q18D4GhGAy/bwidusslenJN9HkOOUxMpl7jJf40WIOn464vkokxpf2itFMgZHKSdPPxedAIAcuwK2VE8smwVyEVe5jzQx7doUaXkN37Ar58iFVldhdyq40RG/nL7XVX/X6naWzTs6j43brwnHDpsYlpa45mtT51xhlzmjXQhpXWRVMYbh3aN8DoroXJgVN9EOF6ywioR25pRaS5thnDnD+dhh1fEZNXZXeKJWU2bAFbh4UZwUTo1TP0qqRmjEtqJ4WUkOmlMXDZoooj7LuX50mzPRxDq1Ec+zIy2NypHiyFZt23L5YPR9ce61SeyiSBXZySbKZTmbXAEPuzqy1f2yrfJExX20LnY/KK7Z3e6WbiKly0Yk9CaAd1lr3wjgIQCPG2PeCuB3APyetfYIgFkAH7r5bnh4eHh4bBYbKUFnATg/qwz/swDeBcCVmv8UgP8E4GM33ANHNujCARz8kqi8Dy6fytlJkgg+8Zkvp/ve9U5Kcn/2skiHVRcsoL5ZGZepjqWEonI7ynLhivqiSNeOuLCKtMwwQekkQE2EOUkwUQRKnV3UdJs7roel6n6VFP/qNAWWzE1Jhse58xRMdfjQQayHQl4kthwHsGRUPpOYyTH98e+kkguPT++8hpSwjCJjaWiJx/eakvq6uTzdaw0pBPAKay/TFZFc+/fSuEYOkjTeo1wwc+wGGah8HG1eK2GkSrmxRBylQTZyfCpda5eya5CiYcKue8p1NHUv1OdlbS2wTmKTczTZBbPTlvXkJG5dcd7BkeeZrC4RyGUDNanMazGfU+5/BfrNzDRdU2dRzLDGGerq8qyNdrQ0uYLUWxZI4wp+KK1niYuo1KqSD2YlAqvKFzppNRap1mkDy4KTQnZbtM41UGlaLBmrOKt07q1yTXQ3woqPYgonhWvX4g5fv62cAhJ+B1lXIlA9D2leJtURg9VjsUx+dziAsaLyEe15kJw7IiP3e+4k57PaI9rojWJDNnRjTMgFoicBfBXA6wDmrIQRjgHYvc5vnzTGHDPGHFvLq8TDw8PD49ZgQy90a21srX0IwB4AjwG4d63D1vntU9baR621jxZVbmMPDw8Pj1uLG/JDt9bOGWO+AeCtAHqMMRFL6XsAXL7mj9dBP1cqb6iCBFWOZMuG4s/t0mo6X+JvfvfldN9Zrm84VxVmZGaJ1GbFLaLE6nuH1a6cql7vVPV8QeWJCJyPsKj2zme2wyYGo/1TWQWLVYX6FvvJFlT+Dpdkv2+ATC0tRQg3uaBDPSfXTDh6UFeEX4m2iuiscj6Orh65ZqNKarYuoBCzephmbFWpW81qq0AKq9IDWyaUquwj/G1VlOR8jdqmVb6KaJgqoI/sGUzbDg7Sdn83zUugok2rLCc0FLEVseqva37mOQo04urr+YIIDzmeex2FeS0ka+QRccqoVaYfy2xyatJR53CRhrE2GfA60uvOrTFH0i6zeiVuPQmpHDP53MrIva1zWltnakk0Acq5XxpKO3bjstoX2x3vzBWqHxGPxbaEyJ6dJjNau7X+muwoP/SYj2sFmhB2eX10URRu4mcpUPfApchNtGmEzWKJSjftCGln/dDHO5OZtvIkzj9cmdicmSk1zWj/cjYLQRO2zmyj3gdtTmPddzcV09h9YG+6r8H1SF9/TWJnCm22bEsQ/A3juhK6MWbQGNPD2wUAPw7gOICvA/gAH/YEgC/cfDc8PDw8PDaLjUjoIwA+ZSghQgDgM9baLxljXgXwaWPMfwbwAoCP30wHGix15tSnpckSUiYUKbXDH0qXsD8oiBR3jsnQQJE2HZaeOorQbHBGuSpHamrix0lNpaxIcQUmSgMlVTjCsVCk6+ucGlc5U16i3JMiJkR6K0Ja7uojrWTXLiL/5qoiySxwZsKleYlS7OFCB1NXdeTnADTaqop9mKWx9w7KNdtlmstOW2W2S9xfJkyVhO6GrCMGU+lNs3+OuONshG2VQ6XZTf2+q0dInt4+iu4sV2TplYt033JMODdUvpQWuzlaJV2Hzt1U94O3M6xpabdFV7xBE2z2Gqxvg139Iu2u6lzhtOsjj90VutDraaXkzR2grupITp575zYYq8jLNs9DqDSzNucDiZV7balJmo2TzHWunWadpfs1SsUla0T8un5Eer653zMTkj+ozRGr+hasgh4653wJsnLNjMt2Gi+ryME/5blSp7MuQ6HSEPOsgfRWhEh3JedcQRY9pyG7mOaUBuzytCyLjuX74iJnFxdUHhZenkkkczTPqRSjAenH/qNEfPZy9Pel106n+6ZOU0bZSPUtf428OBvFRrxcXgbw8BrtZ0D2dA8PDw+POwA+UtTDw8Njh2DLk3M5lTCnkhgVHTHSFlXTuZkm7AWtEwYlrJ51WorEil0KTU1s0XaSpuiU79nsDJk6ZtQ1K1wYoVtFYVbYdz0PMse46t0AELFKGKpal01O5uQKJOjjOjWu1VhTSYzmpnnswubmOSKxcY3oxlCpaz39ZA4ql5QfepNNUMrk0omdb7rzPVaJxvhbHyxLB8pmBJVcKmIVusgmjq4uFcHIRQTKOSG3S+ybns2JutrizSX2m68rgtcRt3ml3mZD57MtanOwwpyh73uLSa9sVpFYmfXn0kX/BsqskXGmPm0u4b65GVpWtD2NHFTJq+LVxLSLlHaFLlotue91NrXEdRXRyaRoSZmlCt2k0nd4nO2GnCNYwyaS+uNrgtyFg7ApqqRiNKpcG3ZhQcyAzmKl18xKhB01x1y3M1ERwhbU3xAqZTBvS1StIjSNXfYXABJOvleLJJGfRHu79Ndqvjmau9GWvrm1bpb5sqed5DOpUFS+via8K5zKefCoxIoE/K468ex36JqTYjIN+f7pQiVrmcBuFF5C9/Dw8NghMPYWfBU2itHRUfvkk0/etut5eHh47AR89KMffc5a++j1jvMSuoeHh8cOgX+he3h4eOwQ+Be6h4eHxw6Bf6F7eHh47BDcVlLUGHMVQBXA1PWOvcMxgO09hu3ef2D7j2G79x/Y/mPYTv3fb60dvN5Bt/WFDgDGmGMbYWvvZGz3MWz3/gPbfwzbvf/A9h/Ddu//WvAmFw8PD48dAv9C9/Dw8Ngh2IoX+lNbcM1bje0+hu3ef2D7j2G79x/Y/mPY7v1fhdtuQ/fw8PDw+MHAm1w8PDw8dghu6wvdGPO4MeaEMea0MeYjt/PaNwNjzF5jzNeNMceNMa8YY36F2/uMMV81xpziv71b3ddrgYt8v2CM+RL//6Ax5jvc/z83xmSvd46thDGmxxjzWWPMa3wv3rYN78G/5zX0fWPMnxlj8nfyfTDGfMIYM2mM+b5qW3PODeG/83P9sjHmka3ruWCdMfwXXkcvG2P+wlVj432/wWM4YYz5p1vT683htr3QueLRHwB4D4D7APy8Mea+23X9m0QHwK9Za+8F1VH9Ze7zRwA8ba09AuBp/v+djF8BlQ10+B0Av8f9nwXwoS3p1cbx+wD+2lp7D4A3gsaybe6BMWY3gH8H4FFr7QOgWj4fxJ19Hz4J4PEVbevN+XsAHOF/TwL42G3q4/XwSawew1cBPGCtfQOAkwB+AwD4uf4ggPv5N//DLMunuz1wOyX0xwCcttaesda2AHwawPtv4/VvGNbacWvt87y9CHqR7Ab1+1N82KcA/MzW9PD6MMbsAfCTAP6Q/28AvAvAZ/mQO73/FQDvAJc4tNa2rLVz2Eb3gBEBKBhjIgBFAOO4g++DtfZbAGZWNK835+8H8MeW8AyogPzI7enp+lhrDNbar1hJUv8MpCTz+wF82lrbtNaeBXAa27Ai2+18oe8GcFH9f4zbtgWMMQdApfi+A2DYWjsO0EsfwNDW9ey6+G8A/gMAl+W/H8CcWtR3+n04BOAqgD9is9EfGmNK2Eb3wFp7CcB/BXAB9CKfB/Acttd9ANaf8+36bP9rAP+Xt7frGJbhdr7Q16qAui1cbIwxZQCfA/Cr1tqF6x1/p8AY81MAJq21z+nmNQ69k+9DBOARAB+z1j4MSh1xx5pX1gLbmt8P4CCAUQAlkJliJe7k+3AtbLc1BWPMb4JMqn/qmtY47I4ew1q4nS/0MQB71f/3ALh8G69/UzDGZEAv8z+11n6emyecSsl/J9f7/RbjhwG8zxhzDmTiehdIYu9h1R+48+/DGIAxa+13+P+fBb3gt8s9AIAfB3DWWnvVWtsG8HkAP4TtdR+A9ed8Wz3bxpgnAPwUgF+w4re9raMrqJEAAAF9SURBVMawHm7nC/1ZAEeY2c+CCIgv3sbr3zDY3vxxAMettb+rdn0RwBO8/QSAL9zuvm0E1trfsNbusdYeAM3316y1vwDg6wA+wIfdsf0HAGvtFQAXjTF3c9O7AbyKbXIPGBcAvNUYU+Q15cawbe4DY705/yKAX2Jvl7cCmHemmTsNxpjHAfw6gPdZa2tq1xcBfNAYkzPGHAQRvN/dij5uCtba2/YPwHtBzPLrAH7zdl77Jvv7dpDa9TKAF/nfe0F26KcBnOK/fVvd1w2M5Z0AvsTbh0CL9TSA/w0gt9X9u07fHwJwjO/DXwLo3W73AMBHAbwG4PsA/gRA7k6+DwD+DGTvb4Ok1w+tN+cgc8Uf8HP9PZA3z506htMgW7l7nv+nOv43eQwnALxnq/t/M/98pKiHh4fHDoGPFPXw8PDYIfAvdA8PD48dAv9C9/Dw8Ngh8C90Dw8Pjx0C/0L38PDw2CHwL3QPDw+PHQL/Qvfw8PDYIfAvdA8PD48dgv8P8QITwTAXGKoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 显示图片\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们看看神经网络认为以上图片是什么。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出是10个标签的能量。\n",
    "一个类别的能量越大，神经网络越认为它是这个类别。所以让我们得到最高能量的标签。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:  plane plane plane plane\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果看来不错。\n",
    "\n",
    "接下来让看看网络在整个测试集上的结果如何。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 9 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果看起来不错，至少比随机选择要好，随机选择的正确率为10%。\n",
    "似乎网络学习到了一些东西。\n",
    "\n",
    "\n",
    "\n",
    "在识别哪一个类的时候好，哪一个不好呢？\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of plane : 99 %\n",
      "Accuracy of   car :  0 %\n",
      "Accuracy of  bird :  0 %\n",
      "Accuracy of   cat :  0 %\n",
      "Accuracy of  deer :  0 %\n",
      "Accuracy of   dog :  0 %\n",
      "Accuracy of  frog :  0 %\n",
      "Accuracy of horse :  0 %\n",
      "Accuracy of  ship :  0 %\n",
      "Accuracy of truck :  0 %\n"
     ]
    }
   ],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一步?\n",
    "\n",
    "我们如何在GPU上运行神经网络呢？\n",
    "\n",
    "在GPU上训练\n",
    "----------------\n",
    "把一个神经网络移动到GPU上训练就像把一个Tensor转换GPU上一样简单。并且这个操作会递归遍历有所模块，并将其参数和缓冲区转换为CUDA张量。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "# 确认我们的电脑支持CUDA，然后显示CUDA信息：\n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本节的其余部分假定`device`是CUDA设备。\n",
    "\n",
    "然后这些方法将递归遍历所有模块并将模块的参数和缓冲区\n",
    "转换成CUDA张量：\n",
    "\n",
    "\n",
    "```python\n",
    "\n",
    "    net.to(device)\n",
    "```\n",
    "\n",
    "记住：inputs, targets 和 images 也要转换。\n",
    "\n",
    "```python\n",
    "\n",
    "        inputs, labels = inputs.to(device), labels.to(device)\n",
    "```\n",
    "为什么我们没注意到GPU的速度提升很多？那是因为网络非常的小。\n",
    "\n",
    "**实践:** \n",
    "尝试增加你的网络的宽度（第一个``nn.Conv2d``的第2个参数，第二个``nn.Conv2d``的第一个参数，它们需要是相同的数字），看看你得到了什么样的加速。\n",
    "\n",
    "**实现的目标**:\n",
    "\n",
    "- 深入了解了PyTorch的张量库和神经网络\n",
    "- 训练了一个小网络来分类图片\n",
    "\n",
    "***译者注：后面我们教程会训练一个真正的网络，使识别率达到90%以上。***\n",
    "\n",
    "多GPU训练\n",
    "-------------------------\n",
    "如果你想使用所有的GPU得到更大的加速，\n",
    "请查看[数据并行处理](5_data_parallel_tutorial.ipynb)。\n",
    "\n",
    "下一步？\n",
    "-------------------\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "-  :doc:`训练神经网络玩电子游戏 </intermediate/reinforcement_q_learning>`\n",
    "-  `在ImageNet上训练最好的ResNet`\n",
    "-  `使用对抗生成网络来训练一个人脸生成器`\n",
    "-  `使用LSTM网络训练一个字符级的语言模型`\n",
    "-  `更多示例`\n",
    "-  `更多教程`\n",
    "-  `在论坛上讨论PyTorch`\n",
    "-  `Slack上与其他用户讨论`\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Pytorch for Deeplearning",
   "language": "python",
   "name": "pytorch"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}