chapter2/2.1.1.pytorch-basics-tensor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyTorch 基础 : 张量\n",
    "在第一章中我们已经通过官方的入门教程对PyTorch有了一定的了解，这一章会详细介绍PyTorch 里面的基础知识。\n",
    "全部掌握了这些基础知识，在后面的应用中才能更加快速进阶，如果你已经对PyTorch有一定的了解，可以跳过此章"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.0'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 首先要引入相关的包\n",
    "import torch\n",
    "import numpy as np\n",
    "#打印一下版本\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 张量(Tensor)\n",
    "张量的英文是Tensor，它是PyTorch里面基础的运算单位，与Numpy的ndarray相同都表示的是一个多维的矩阵。\n",
    "与ndarray的最大区别就是，PyTorch的Tensor可以在 GPU 上运行，而 numpy 的 ndarray 只能在 CPU 上运行，在GPU上运行大大加快了运算速度。\n",
    "\n",
    "下面我们生成一个简单的张量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.6904, 0.7419, 0.8010],\n",
       "        [0.1722, 0.2442, 0.8181]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.rand(2, 3)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上生成了一个，2行3列的的矩阵，我们看一下他的大小："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 3])\n",
      "torch.Size([2, 3])\n"
     ]
    }
   ],
   "source": [
    "# 可以使用与numpy相同的shape属性查看\n",
    "print(x.shape)\n",
    "# 也可以使用size()函数，返回的结果都是相同的\n",
    "print(x.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "张量（Tensor）是一个定义在一些向量空间和一些对偶空间的笛卡儿积上的多重线性映射，其坐标是|n|维空间内，有|n|个分量的一种量， 其中每个分量都是坐标的函数， 而在坐标变换时，这些分量也依照某些规则作线性变换。r称为该张量的秩或阶（与矩阵的秩和阶均无关系）。 (来自百度百科)\n",
    "\n",
    "下面我们来生成一些多维的张量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 3, 4, 5])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[[[0.9071, 0.0616, 0.0006, 0.6031, 0.0714],\n",
       "          [0.6592, 0.9700, 0.0253, 0.0726, 0.5360],\n",
       "          [0.5416, 0.1138, 0.9592, 0.6779, 0.6501],\n",
       "          [0.0546, 0.8287, 0.7748, 0.4352, 0.9232]],\n",
       "\n",
       "         [[0.0730, 0.4228, 0.7407, 0.4099, 0.1482],\n",
       "          [0.5408, 0.9156, 0.6554, 0.5787, 0.9775],\n",
       "          [0.4262, 0.3644, 0.1993, 0.4143, 0.5757],\n",
       "          [0.9307, 0.8839, 0.8462, 0.0933, 0.6688]],\n",
       "\n",
       "         [[0.4447, 0.0929, 0.9882, 0.5392, 0.1159],\n",
       "          [0.4790, 0.5115, 0.4005, 0.9486, 0.0054],\n",
       "          [0.8955, 0.8097, 0.1227, 0.2250, 0.5830],\n",
       "          [0.8483, 0.2070, 0.1067, 0.4727, 0.5095]]],\n",
       "\n",
       "\n",
       "        [[[0.9438, 0.2601, 0.2885, 0.5457, 0.7528],\n",
       "          [0.2971, 0.2171, 0.3910, 0.1924, 0.2570],\n",
       "          [0.7491, 0.9749, 0.2703, 0.2198, 0.9472],\n",
       "          [0.1216, 0.6647, 0.8809, 0.0125, 0.5513]],\n",
       "\n",
       "         [[0.0870, 0.6622, 0.7252, 0.4783, 0.0160],\n",
       "          [0.7832, 0.6050, 0.7469, 0.7947, 0.8052],\n",
       "          [0.1755, 0.4489, 0.0602, 0.8073, 0.3028],\n",
       "          [0.9937, 0.6780, 0.9425, 0.0059, 0.0451]],\n",
       "\n",
       "         [[0.3851, 0.8742, 0.5932, 0.4899, 0.8354],\n",
       "          [0.8577, 0.3705, 0.0229, 0.7097, 0.7557],\n",
       "          [0.1505, 0.3527, 0.0843, 0.0088, 0.8741],\n",
       "          [0.6041, 0.8797, 0.6189, 0.9495, 0.1479]]]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y=torch.rand(2,3,4,5)\n",
    "print(y.size())\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在同构的意义下，第零阶张量 （r = 0） 为标量 （Scalar），第一阶张量 （r = 1） 为向量 （Vector）， 第二阶张量 （r = 2） 则成为矩阵 （Matrix），第三阶以上的统称为多维张量。\n",
    "\n",
    "其中要特别注意的就是标量，我们先生成一个标量：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(3.1433)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "torch.Size([])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#我们直接使用现有数字生成\n",
    "scalar =torch.tensor(3.1433223)\n",
    "print(scalar)\n",
    "#打印标量的大小\n",
    "scalar.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于标量，我们可以直接使用 .item() 从中取出其对应的python对象的数值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.143322229385376"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scalar.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "特别的：如果张量中只有一个元素的tensor也可以调用`tensor.item`方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([3.1433])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "torch.Size([1])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor = torch.tensor([3.1433223]) \n",
    "print(tensor)\n",
    "tensor.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.143322229385376"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 基本类型\n",
    "Tensor的基本数据类型有五种：\n",
    "- 32位浮点型：torch.FloatTensor。 (默认)\n",
    "- 64位整型：torch.LongTensor。\n",
    "- 32位整型：torch.IntTensor。\n",
    "- 16位整型：torch.ShortTensor。\n",
    "- 64位浮点型：torch.DoubleTensor。\n",
    "\n",
    "除以上数字类型外，还有\n",
    "byte和chart型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "long=tensor.long()\n",
    "long"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3.1426], dtype=torch.float16)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "half=tensor.half()\n",
    "half"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.int32)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int_t=tensor.int()\n",
    "int_t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3.1433])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flo = tensor.float()\n",
    "flo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.int16)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "short = tensor.short()\n",
    "short"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.int8)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ch = tensor.char()\n",
    "ch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([3], dtype=torch.uint8)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bt = tensor.byte()\n",
    "bt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numpy转换\n",
    "使用numpy方法将Tensor转为ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.46819344  1.3774964 ]\n",
      " [ 0.9491934   1.4543315 ]\n",
      " [-0.42792308  0.99790514]]\n"
     ]
    }
   ],
   "source": [
    "a = torch.randn((3, 2))\n",
    "# tensor转化为numpy\n",
    "numpy_a = a.numpy()\n",
    "print(numpy_a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "numpy转化为Tensor\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.4682,  1.3775],\n",
       "        [ 0.9492,  1.4543],\n",
       "        [-0.4279,  0.9979]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch_a = torch.from_numpy(numpy_a)\n",
    "torch_a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Tensor和numpy对象共享内存，所以他们之间的转换很快，而且几乎不会消耗什么资源。但这也意味着，如果其中一个变了，另外一个也会随之改变。***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 设备间转换\n",
    "一般情况下可以使用.cuda方法将tensor移动到gpu，这步操作需要cuda设备支持"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'torch.FloatTensor'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cpu_a=torch.rand(4, 3)\n",
    "cpu_a.type()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'torch.cuda.FloatTensor'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gpu_a=cpu_a.cuda()\n",
    "gpu_a.type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用.cpu方法将tensor移动到cpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'torch.FloatTensor'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cpu_b=gpu_a.cpu()\n",
    "cpu_b.type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们有多GPU的情况，可以使用to方法来确定使用那个设备，这里只做个简单的实例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'torch.cuda.FloatTensor'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#使用torch.cuda.is_available()来确定是否有cuda设备\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)\n",
    "#将tensor传送到设备\n",
    "gpu_b=cpu_b.to(device)\n",
    "gpu_b.type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 初始化\n",
    "Pytorch中有许多默认的初始化方法可以使用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.3804, 0.0297, 0.5241],\n",
       "        [0.4111, 0.8887, 0.4642],\n",
       "        [0.7302, 0.5913, 0.7182],\n",
       "        [0.3048, 0.8055, 0.2176],\n",
       "        [0.6195, 0.1620, 0.7726]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用[0,1]均匀分布随机初始化二维数组\n",
    "rnd = torch.rand(5, 3)\n",
    "rnd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1.],\n",
       "        [1., 1.]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##初始化，使用1填充\n",
    "one = torch.ones(2, 2)\n",
    "one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0.],\n",
       "        [0., 0.]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##初始化，使用0填充\n",
    "zero=torch.zeros(2,2)\n",
    "zero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 0.],\n",
       "        [0., 1.]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#初始化一个单位矩阵，即对角线为1 其他为0\n",
    "eye=torch.eye(2,2)\n",
    "eye"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 常用方法\n",
    "PyTorch中对张量的操作api 和 NumPy 非常相似，如果熟悉 NumPy 中的操作，那么他们二者基本是一致的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.6922, -0.4824,  0.8594],\n",
      "        [ 0.4509, -0.8155, -0.0368],\n",
      "        [ 1.3533,  0.5545, -0.0509]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(3, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.8594, 0.4509, 1.3533]) tensor([2, 0, 0])\n"
     ]
    }
   ],
   "source": [
    "# 沿着行取最大值\n",
    "max_value, max_idx = torch.max(x, dim=1)\n",
    "print(max_value, max_idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 1.0692, -0.4014,  1.8568])\n"
     ]
    }
   ],
   "source": [
    "# 每行 x 求和\n",
    "sum_x = torch.sum(x, dim=1)\n",
    "print(sum_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3821, -2.6932, -1.3884],\n",
      "        [ 0.7468, -0.7697, -0.0883],\n",
      "        [ 0.7688, -1.3485,  0.7517]])\n"
     ]
    }
   ],
   "source": [
    "y=torch.randn(3, 3)\n",
    "z = x + y\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "正如官方60分钟教程中所说，以_为结尾的，均会改变调用值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.3821, -2.6932, -1.3884],\n",
      "        [ 0.7468, -0.7697, -0.0883],\n",
      "        [ 0.7688, -1.3485,  0.7517]])\n"
     ]
    }
   ],
   "source": [
    "# add 完成后x的值改变了\n",
    "x.add_(y)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "张量的基本操作都介绍的的差不多了，下一章介绍PyTorch的自动求导机制"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}