{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyTorch Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- This tutorial briefly describes some basic operations on Tensors in PyTorch. \n",
    "- This [documentation](http://pytorch.org/docs/master/torch.html) from pytorch contains information about all the possible operations on Tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting started with Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  2  3\n",
      " 4  5  6\n",
      "[torch.FloatTensor of size 2x3]\n",
      "\n",
      "shape:  torch.Size([2, 3])\n"
     ]
    }
   ],
   "source": [
    "#Create Torch Tensor from a list\n",
    "x = torch.Tensor([[1,2,3],[4,5,6]])\n",
    "print (x) # prints the tensor\n",
    "print ('shape: ',x.shape) # returns the shape of the tensor. You can also use x.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- you can use the `.numpy()` function to obtain the numpy array from tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numpy array: \n",
      "[[1. 2. 3.]\n",
      " [4. 5. 6.]]\n"
     ]
    }
   ],
   "source": [
    "print ('numpy array: ')\n",
    "print (x.numpy()) # obtains the numpy array from the pytorch Tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The default `torch.Tensor` is a float tensor as you can see above.\n",
    "- You can use `torch.LongTensor` for tensor of integer types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1\n",
      " 4\n",
      " 6\n",
      "[torch.LongTensor of size 3]\n",
      "\n",
      "shape:  torch.Size([3])\n",
      "numpy array:  [1 4 6]\n"
     ]
    }
   ],
   "source": [
    "x = torch.LongTensor([1,4,6])\n",
    "print (x) # prints the tensor\n",
    "print ('shape: ', x.shape) # returns the shape of the tensor. You can also use x.size()\n",
    "print ('numpy array: ',x.numpy()) #obtains the numpy array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- You can also create Tensors from numpy arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1. 1. 1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "one_arr = np.ones(5) # a numpy array\n",
    "print (one_arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      " 1\n",
      "[torch.DoubleTensor of size 5]\n",
      "\n",
      "torch.Size([5])\n"
     ]
    }
   ],
   "source": [
    "x = torch.from_numpy(one_arr)\n",
    "print (x) \n",
    "print (x.shape) #obtains the shape of the tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1. 1. 1. 1. 1.]\n"
     ]
    }
   ],
   "source": [
    "print (x.numpy()) #obtains the numpy array from the Tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indexing into a Tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "(0 ,.,.) = \n",
      "  0.2629  0.2525  0.1080  0.0792  0.8236\n",
      "  0.6245  0.3958  0.0872  0.7196  0.9855\n",
      "  0.9911  0.1854  0.5447  0.5830  0.5799\n",
      "  0.5995  0.2357  0.5548  0.2315  0.7305\n",
      "\n",
      "(1 ,.,.) = \n",
      "  0.7462  0.3834  0.6658  0.2997  0.2163\n",
      "  0.9693  0.3457  0.0226  0.3908  0.9446\n",
      "  0.2554  0.5765  0.0044  0.0785  0.2974\n",
      "  0.8040  0.7930  0.5340  0.5711  0.0133\n",
      "\n",
      "(2 ,.,.) = \n",
      "  0.9761  0.2568  0.7862  0.3860  0.4732\n",
      "  0.4621  0.7859  0.3441  0.6674  0.4566\n",
      "  0.6447  0.3531  0.1142  0.2946  0.7443\n",
      "  0.1688  0.2544  0.1592  0.0313  0.5451\n",
      "[torch.DoubleTensor of size 3x4x5]\n",
      "\n",
      "shape:  torch.Size([3, 4, 5])\n"
     ]
    }
   ],
   "source": [
    "a = np.random.rand(3,4,5)\n",
    "x = torch.from_numpy(a)\n",
    "print (x)\n",
    "print ('shape: ',x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.2629  0.2525  0.1080  0.0792  0.8236\n",
      " 0.6245  0.3958  0.0872  0.7196  0.9855\n",
      " 0.9911  0.1854  0.5447  0.5830  0.5799\n",
      " 0.5995  0.2357  0.5548  0.2315  0.7305\n",
      "[torch.DoubleTensor of size 4x5]\n",
      "\n",
      "torch.Size([4, 5])\n"
     ]
    }
   ],
   "source": [
    "# you can index into them like arrays\n",
    "print (x[0]) #gives you a matrix Tensor\n",
    "print (x[0].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.9911\n",
      " 0.1854\n",
      " 0.5447\n",
      " 0.5830\n",
      " 0.5799\n",
      "[torch.DoubleTensor of size 5]\n",
      "\n",
      "torch.Size([5])\n"
     ]
    }
   ],
   "source": [
    "# you can index into them like arrays\n",
    "print (x[0][2]) # gives you a vector Tensor\n",
    "print (x[0][2].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5829977920624786\n"
     ]
    }
   ],
   "source": [
    "print (x[0][2][3]) #this gives you a scalar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reshaping a Tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- To reshape a tensor to a different size, you can use the `.view()` function. \n",
    "- The `.view()` function returns a tensor with the same data as the self tensor but of a different size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.9032  0.8118  0.6585  0.2786  0.0186  0.1433\n",
      " 0.4052  0.0165  0.4320  0.7576  0.7524  0.6990\n",
      "[torch.DoubleTensor of size 2x6]\n",
      "\n",
      "shape:  torch.Size([2, 6])\n"
     ]
    }
   ],
   "source": [
    "a = np.random.rand(2,6)\n",
    "x = torch.from_numpy(a)\n",
    "print (x)\n",
    "print ('shape: ',x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.9032  0.8118  0.6585  0.2786\n",
      " 0.0186  0.1433  0.4052  0.0165\n",
      " 0.4320  0.7576  0.7524  0.6990\n",
      "[torch.DoubleTensor of size 3x4]\n",
      "\n",
      "torch.Size([3, 4])\n"
     ]
    }
   ],
   "source": [
    "y = x.view(3,4) #reshapes it into a tensor of size 3 x 4\n",
    "print (y)\n",
    "print (y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "(0 ,.,.) = \n",
      "  0.9032  0.8118\n",
      "  0.6585  0.2786\n",
      "  0.0186  0.1433\n",
      "\n",
      "(1 ,.,.) = \n",
      "  0.4052  0.0165\n",
      "  0.4320  0.7576\n",
      "  0.7524  0.6990\n",
      "[torch.DoubleTensor of size 2x3x2]\n",
      "\n",
      "torch.Size([2, 3, 2])\n"
     ]
    }
   ],
   "source": [
    "y = x.view(2,3,2) #reshapes the tensor into 2 x 3 x 2\n",
    "print (y)\n",
    "print (y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- if one of the dimensions is `-1`, as shown below, then its size can be inferred. So, you cannot have multiple -1's in view."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.9032  0.8118  0.6585\n",
      " 0.2786  0.0186  0.1433\n",
      " 0.4052  0.0165  0.4320\n",
      " 0.7576  0.7524  0.6990\n",
      "[torch.DoubleTensor of size 4x3]\n",
      "\n",
      "torch.Size([4, 3])\n"
     ]
    }
   ],
   "source": [
    "y = x.view(4,3) #reshapes the tensor into 4x3\n",
    "print (y)\n",
    "print (y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.9032  0.8118  0.6585\n",
      " 0.2786  0.0186  0.1433\n",
      " 0.4052  0.0165  0.4320\n",
      " 0.7576  0.7524  0.6990\n",
      "[torch.DoubleTensor of size 4x3]\n",
      "\n",
      "torch.Size([4, 3])\n"
     ]
    }
   ],
   "source": [
    "y = x.view(4,-1) #same as above, but the second dimension can be inferred.\n",
    "print (y)\n",
    "print (y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Operations on Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- You can do some basic operations on tensors like arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.7349  0.4462  0.7814  0.8546  0.0565\n",
      "[torch.DoubleTensor of size 1x5]\n",
      "\n",
      "torch.Size([1, 5])\n"
     ]
    }
   ],
   "source": [
    "a = np.random.rand(1,5)\n",
    "x = torch.from_numpy(a)\n",
    "print (x)\n",
    "print (x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1  1  1  1  1\n",
      "[torch.DoubleTensor of size 1x5]\n",
      "\n",
      "torch.Size([1, 5])\n"
     ]
    }
   ],
   "source": [
    "b = np.ones((1,5))\n",
    "y = torch.from_numpy(b)\n",
    "print (y)\n",
    "print (y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1.7349  1.4462  1.7814  1.8546  1.0565\n",
      "[torch.DoubleTensor of size 1x5]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print (x + y) #element wise addition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "-0.2651 -0.5538 -0.2186 -0.1454 -0.9435\n",
      "[torch.DoubleTensor of size 1x5]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print (x - y) #element wise subtraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 0.7349  0.4462  0.7814  0.8546  0.0565\n",
      "[torch.DoubleTensor of size 1x5]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print (x * y)  #element wise multiplication"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Another useful operation on tensors is concatenation.\n",
    "- You can use the `torch.cat()` function, it takes in a list/sequence of tensors and concatenates them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 0.7349  0.4462  0.7814  0.8546  0.0565\n",
       " 1.0000  1.0000  1.0000  1.0000  1.0000\n",
       "[torch.DoubleTensor of size 2x5]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.cat([x,y]) # concatenates them along the row (dim=0) by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 0.7349  0.4462  0.7814  0.8546  0.0565  1.0000  1.0000  1.0000  1.0000  1.0000\n",
       "[torch.DoubleTensor of size 1x10]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.cat([x,y], dim=1) # concatenates them along the column when dim=1 is mentioned."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- If you want to expand a Tensor along the singleton dimension, you can use the `.expand()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " 1\n",
      " 2\n",
      " 3\n",
      "[torch.FloatTensor of size 3x1]\n",
      "\n",
      "torch.Size([3, 1])\n"
     ]
    }
   ],
   "source": [
    "x = torch.Tensor([[1],[2],[3]])\n",
    "print (x)\n",
    "print (x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1  1  1  1\n",
       " 2  2  2  2\n",
       " 3  3  3  3\n",
       "[torch.FloatTensor of size 3x4]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.expand(3,4) #expands it along the second dimension from (3 x 1) to be (3 x 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 5  6\n",
       " 5  6\n",
       " 5  6\n",
       " 5  6\n",
       " 5  6\n",
       "[torch.FloatTensor of size 5x2]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.Tensor([5,6]).view(-1,2) #reshaping it to 1 x 2\n",
    "x.expand(5,2) #expands it along the first dimension from (1 x 2) to (5 x 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- These are the basic operations on the Tensors that will be very useful. You can check the [documentation here](http://pytorch.org/docs/master/torch.html) for all list of operations."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}