{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tutorial: NumPy" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "__author__ = \"Christopher Potts, Will Monroe, and Lucy Li\"\n", "__version__ = \"CS224u, Stanford, Spring 2022\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "\n", "1. [Motivation](#Motivation)\n", "1. [Vectors](#Vectors)\n", " 1. [Vector Initialization](#Vector-Initialization)\n", " 1. [Vector indexing](#Vector-indexing)\n", " 1. [Vector assignment](#Vector-assignment)\n", " 1. [Vectorized operations](#Vectorized-operations)\n", " 1. [Comparison with Python lists](#Comparison-with-Python-lists)\n", "1. [Matrices](#Matrices)\n", " 1. [Matrix initialization](#Matrix-initialization)\n", " 1. [Matrix indexing](#Matrix-indexing)\n", " 1. [Matrix assignment](#Matrix-assignment)\n", " 1. [Matrix reshaping](#Matrix-reshaping)\n", " 1. [Numeric operations](#Numeric-operations)\n", "1. [Practical example: a shallow neural network](#Practical-example:-a-shallow-neural-network)\n", "1. [Going beyond NumPy alone](#Going-beyond-NumPy-alone)\n", " 1. [Pandas](#Pandas)\n", " 1. [Scikit-learn](#Scikit-learn)\n", " 1. [SciPy](#SciPy)\n", " 1. [Matplotlib](#Matplotlib)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Motivation\n", "\n", "Why should we care about NumPy? \n", "\n", "- It allows you to perform tons of operations on vectors and matrices. \n", "- It makes things run faster than naive for-loop implementations (a.k.a. vectorization). \n", "- We use it in our class (see files prefixed with `np_` in your `cs224u` directory). \n", "- It's used in a ton in machine learning / AI. \n", "- Its arrays are often inputs into other important Python packages' functions. \n", "\n", "In Jupyter notebooks, NumPy documentation is two clicks away: Help -> NumPy reference. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Vectors" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vector Initialization" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0., 0., 0., 0., 0.])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.zeros(5)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1., 1., 1., 1., 1.])" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.ones(5)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 3, 4, 5])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# convert list to numpy array\n", "np.array([1,2,3,4,5])" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1.0, 1.0, 1.0, 1.0, 1.0]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# convert numpy array to list\n", "np.ones(5).tolist()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1., 2., 3., 4., 5.])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# one float => all floats\n", "np.array([1.0,2,3,4,5])" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1., 2., 3., 4., 5.])" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# same as above\n", "np.array([1,2,3,4,5], dtype='float')" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18])" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# spaced values in interval\n", "np.array([x for x in range(20) if x % 2 == 0])" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18])" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# same as above\n", "np.arange(0,20,2)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.73498855, 0.8747517 , 0.21096778, 0.62535706, 0.16202865,\n", " 0.59492923, 0.81269283, 0.51122571, 0.26356335, 0.93557229])" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# random floats in [0, 1)\n", "np.random.random(10)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([12, 14, 7, 7, 6, 8, 5, 9, 9, 8])" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# random integers\n", "np.random.randint(5, 15, size=10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vector indexing" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "x = np.array([10,20,30,40,50])" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "10" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x[0]" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 20])" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# slice\n", "x[0:2]" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 20, 30, 40, 50])" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x[0:1000]" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "50" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# last value\n", "x[-1]" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([50])" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# last value as array\n", "x[[-1]]" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([30, 40, 50])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# last 3 values\n", "x[-3:]" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 30, 50])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# pick indices\n", "x[[0,2,4]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vector assignment\n", "\n", "Be careful when assigning arrays to new variables! " ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "#x2 = x # try this line instead\n", "x2 = x.copy()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 20, 30, 40, 50])" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x2[0] = 10\n", "\n", "x2" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 10, 10, 40, 50])" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x2[[1,2]] = 10\n", "\n", "x2" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 10, 10, 0, 1])" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x2[[3,4]] = [0, 1]\n", "\n", "x2" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([10, 20, 30, 40, 50])" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# check if the original vector changed\n", "x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vectorized operations" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "150" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.sum()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "30.0" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.mean()" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "50" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.max()" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x.argmax()" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([2.30258509, 2.99573227, 3.40119738, 3.68887945, 3.91202301])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.log(x)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([2.20264658e+04, 4.85165195e+08, 1.06864746e+13, 2.35385267e+17,\n", " 5.18470553e+21])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.exp(x)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 20, 40, 60, 80, 100])" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x + x # Try also with *, -, /, etc." ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([11, 21, 31, 41, 51])" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x + 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Comparison with Python lists\n", "\n", "Vectorizing your mathematical expressions can lead to __huge__ performance gains. The following example is meant to give you a sense for this. It compares applying `np.log` to each element of a list with 10 million values with the same operation done on a vector." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "# log every value as list, one by one\n", "def listlog(vals):\n", " return [np.log(y) for y in vals]" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1.72118195, 1.94784745, 1.17111963, ..., 1.52879058, 1.31396677,\n", " 1.22061111])" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# get random vector\n", "samp = np.random.random_sample(int(1e7))+1\n", "samp" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 84.5 ms, sys: 14.9 ms, total: 99.4 ms\n", "Wall time: 98.3 ms\n" ] } ], "source": [ "%time _ = np.log(samp)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 8.97 s, sys: 288 ms, total: 9.26 s\n", "Wall time: 9.26 s\n" ] } ], "source": [ "%time _ = listlog(samp)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Matrices\n", "\n", "The matrix is the core object of machine learning implementations. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix initialization" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [4, 5, 6]])" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.array([[1,2,3], [4,5,6]])" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 2., 3.],\n", " [4., 5., 6.]])" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.array([[1,2,3], [4,5,6]], dtype='float')" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0., 0., 0., 0., 0.],\n", " [0., 0., 0., 0., 0.],\n", " [0., 0., 0., 0., 0.]])" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.zeros((3,5))" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 1., 1., 1., 1.],\n", " [1., 1., 1., 1., 1.],\n", " [1., 1., 1., 1., 1.]])" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.ones((3,5))" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 0., 0.],\n", " [0., 1., 0.],\n", " [0., 0., 1.]])" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.identity(3)" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 0, 0],\n", " [0, 2, 0],\n", " [0, 0, 3]])" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.diag([1,2,3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix indexing" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [4, 5, 6]])" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X = np.array([[1,2,3], [4,5,6]])\n", "X" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 3])" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X[0]" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1" ] }, "execution_count": 46, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X[0,0]" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 3])" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# get row\n", "X[0, : ]" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 4])" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# get column\n", "X[ : , 0]" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 3],\n", " [4, 6]])" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# get multiple columns\n", "X[ : , [0,2]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix assignment" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [4, 5, 6]])" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# X2 = X # try this line instead\n", "X2 = X.copy()\n", "\n", "X2" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[20, 2, 3],\n", " [ 4, 5, 6]])" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X2[0,0] = 20\n", "\n", "X2" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[3, 3, 3],\n", " [4, 5, 6]])" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X2[0] = 3\n", "\n", "X2" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[3, 3, 5],\n", " [4, 5, 6]])" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X2[: , -1] = [5, 6]\n", "\n", "X2" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [4, 5, 6]])" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# check if original matrix changed\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix reshaping" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 3, 4, 5, 6])" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "z = np.arange(1, 7)\n", "\n", "z" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(6,)" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "z.shape" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [4, 5, 6]])" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Z = z.reshape(2,3)\n", "\n", "Z" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(2, 3)" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Z.shape" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 3, 4, 5, 6])" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Z.reshape(6)" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 2, 3, 4, 5, 6])" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# same as above\n", "Z.flatten()" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 4],\n", " [2, 5],\n", " [3, 6]])" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# transpose\n", "Z.T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Numeric operations" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 2., 3.],\n", " [4., 5., 6.]])" ] }, "execution_count": 62, "metadata": {}, "output_type": "execute_result" } ], "source": [ "A = np.array(range(1,7), dtype='float').reshape(2,3)\n", "\n", "A" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [], "source": [ "B = np.array([1, 2, 3])" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 1., 4., 9.],\n", " [ 4., 10., 18.]])" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# not the same as A.dot(B)\n", "A * B" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[2., 4., 6.],\n", " [5., 7., 9.]])" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "A + B" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1. , 1. , 1. ],\n", " [4. , 2.5, 2. ]])" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "A / B" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([14., 32.])" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# matrix multiplication\n", "A.dot(B)" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([14., 32.])" ] }, "execution_count": 68, "metadata": {}, "output_type": "execute_result" } ], "source": [ "B.dot(A.T)" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[14., 32.],\n", " [32., 77.]])" ] }, "execution_count": 69, "metadata": {}, "output_type": "execute_result" } ], "source": [ "A.dot(A.T)" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1, 2, 3],\n", " [2, 4, 6],\n", " [3, 6, 9]])" ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# outer product\n", "# multiplying each element of first vector by each element of the second\n", "np.outer(B, B)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Practical example: a shallow neural network" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following is a practical example of numerical operations on NumPy matrices. \n", "\n", "In our class, we have a shallow neural network implemented in `np_shallow_neural_network.py`. See how the forward and backward passes use no for loops, and instead takes advantage of NumPy's ability to vectorize manipulations of data. \n", "\n", "```python\n", "def forward_propagation(self, x):\n", " h = self.hidden_activation(x.dot(self.W_xh) + self.b_xh)\n", " y = softmax(h.dot(self.W_hy) + self.b_hy)\n", " return h, y\n", "\n", "def backward_propagation(self, h, predictions, x, labels):\n", " y_err = predictions.copy()\n", " y_err[np.argmax(labels)] -= 1\n", " d_b_hy = y_err\n", " h_err = y_err.dot(self.W_hy.T) * self.d_hidden_activation(h)\n", " d_W_hy = np.outer(h, y_err)\n", " d_W_xh = np.outer(x, h_err)\n", " d_b_xh = h_err\n", " return d_W_hy, d_b_hy, d_W_xh, d_b_xh\n", "```\n", "\n", "The forward pass essentially computes the following: \n", " $$h = f(xW_{xh} + b_{xh})$$\n", " $$y = \\text{softmax}(hW_{hy} + b_{hy}),$$\n", "where $f$ is `self.hidden_activation`. \n", "\n", "The backward pass propagates error by computing local gradients and chaining them. Feel free to learn more about backprop [here](http://cs231n.github.io/optimization-2/), though it is not necessary for our class. Also look at this [neural networks case study](http://cs231n.github.io/neural-networks-case-study/) to see another example of how NumPy can be used to implement forward and backward passes of a simple neural network. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Going beyond NumPy alone\n", "\n", "These are examples of how NumPy can be used with other Python packages. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pandas\n", "We can convert numpy matrices to Pandas dataframes. In the following example, this is useful because it allows us to label each row. You may have noticed this being done in our first unit on distributed representations. " ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
012345
gnarly1.00.01.00.00.00.0
wicked0.01.00.01.00.00.0
awesome1.01.01.01.00.00.0
lame0.00.00.00.01.01.0
terrible0.00.00.00.00.01.0
\n", "
" ], "text/plain": [ " 0 1 2 3 4 5\n", "gnarly 1.0 0.0 1.0 0.0 0.0 0.0\n", "wicked 0.0 1.0 0.0 1.0 0.0 0.0\n", "awesome 1.0 1.0 1.0 1.0 0.0 0.0\n", "lame 0.0 0.0 0.0 0.0 1.0 1.0\n", "terrible 0.0 0.0 0.0 0.0 0.0 1.0" ] }, "execution_count": 72, "metadata": {}, "output_type": "execute_result" } ], "source": [ "count_df = pd.DataFrame(\n", " np.array([\n", " [1,0,1,0,0,0],\n", " [0,1,0,1,0,0],\n", " [1,1,1,1,0,0],\n", " [0,0,0,0,1,1],\n", " [0,0,0,0,0,1]], dtype='float64'),\n", " index=['gnarly', 'wicked', 'awesome', 'lame', 'terrible'])\n", "count_df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Scikit-learn\n", "\n", "In `sklearn`, NumPy matrices are the most common input and output and thus a key to how the library's numerous methods can work together. Many of the cs224u's model built by Chris operate just like `sklearn` ones, such as the classifiers we used for our sentiment analysis unit. " ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [], "source": [ "from sklearn.linear_model import LogisticRegression\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import classification_report\n", "from sklearn import datasets" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Dimensions of X: (150, 4)\n", "\n", "Dimensions of y: (150,)\n" ] } ], "source": [ "iris = datasets.load_iris()\n", "X = iris.data\n", "y = iris.target\n", "print(type(X))\n", "print(\"Dimensions of X:\", X.shape)\n", "print(type(y))\n", "print(\"Dimensions of y:\", y.shape)" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X_iris_train: \n", "y_iris_train: \n", "\n", " precision recall f1-score support\n", "\n", " setosa 1.00 1.00 1.00 11\n", " versicolor 0.93 0.93 0.93 15\n", " virginica 0.95 0.95 0.95 19\n", "\n", " accuracy 0.96 45\n", " macro avg 0.96 0.96 0.96 45\n", "weighted avg 0.96 0.96 0.96 45\n", "\n" ] } ], "source": [ "# split data into train/test\n", "X_iris_train, X_iris_test, y_iris_train, y_iris_test = train_test_split(\n", " X, y, train_size=0.7, test_size=0.3)\n", "print(\"X_iris_train:\", type(X_iris_train))\n", "print(\"y_iris_train:\", type(y_iris_train))\n", "print()\n", "\n", "# start up model\n", "maxent = LogisticRegression(\n", " fit_intercept=True,\n", " solver='liblinear',\n", " multi_class='auto')\n", "\n", "# train on train set\n", "maxent.fit(X_iris_train, y_iris_train)\n", "\n", "# predict on test set\n", "iris_predictions = maxent.predict(X_iris_test)\n", "fnames_iris = iris['feature_names']\n", "tnames_iris = iris['target_names']\n", "\n", "# how well did our model do?\n", "print(classification_report(y_iris_test, iris_predictions, target_names=tnames_iris))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### SciPy\n", "\n", "SciPy contains what may seem like an endless treasure trove of operations for linear algebra, optimization, and more. It is built so that everything can work with NumPy arrays. " ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [], "source": [ "from scipy.spatial.distance import cosine\n", "from scipy.stats import pearsonr\n", "from scipy import linalg" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.12344299765526168" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# cosine distance\n", "a = np.random.random(10)\n", "b = np.random.random(10)\n", "cosine(a, b)" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(0.4622531159089377, 0.17859601186804078)" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# pearson correlation (coeff, p-value)\n", "pearsonr(a, b)" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[-1.48, 0.36, 0.88],\n", " [ 0.56, 0.08, -0.36],\n", " [ 0.16, -0.12, 0.04]])" ] }, "execution_count": 79, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# inverse of matrix\n", "A = np.array([[1,3,5],[2,5,1],[2,3,8]])\n", "linalg.inv(A)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To learn more about how NumPy can be combined with SciPy and Scikit-learn for machine learning, check out this [notebook tutorial](https://github.com/cgpotts/csli-summer/blob/master/advanced_python/intro_to_python_ml.ipynb) by Chris Potts and Will Monroe. (You may notice that over half of this current notebook is modified from theirs.) Their tutorial also has some interesting exercises in it! " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matplotlib" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt" ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABMHUlEQVR4nO3deXxU1f3/8ddksq+QhEBCAmENkU12QRFQEEVBrSJWfyq1WLW1Wq0b1YpbS91qlbq0fi2UVm3dRUWLCwgqsgnKvgdIwpKF7GSSydzfHzcZCAmQhMzcmcn7+Xjcx9y5c2fuZy7IvD3n3HNthmEYiIiIiFggyOoCREREpO1SEBERERHLKIiIiIiIZRRERERExDIKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURAR8QErVqzg8ssvp0uXLoSFhdGxY0dGjhzJb3/7W6tLa5HCwkKuvvpqkpKSsNlsXHbZZZbW88c//pH333+/wfYlS5Zgs9lYsmSJ12sSEZNNU7yLWOvjjz9mypQpjB07lptuuonk5GT279/P6tWr+c9//kN2drbVJTbbnXfeyYsvvsg//vEPevToQXx8PL1797asnujoaK688krmzZtXb3tJSQmbNm3ijDPOIDY21priRNo4BRERi40ZM4acnBy2bNlCcHBwvddcLhdBQf7XcDlhwgRycnLYtGmT1aUAJw4iImI9//sXTiTAFBQUkJiY2CCEAA1CiMvl4sknn6RPnz6EhYWRlJTE9ddf36DVZOzYsfTr14/ly5czatQoIiIiSE9PZ+7cuYDZCjN48GAiIyPp378/n376aYNjb9++nWuuuYakpCTCwsLIzMzkhRdeOOl3ycrKwmaz8fnnn7N582ZsNpu76+NE3SB17zk2JEyfPp3o6Gh27NjBpEmTiI6OJi0tjd/+9rc4HI5673c4HDz66KNkZmYSHh5OQkIC48aN49tvvwXAZrNRXl7OP//5T3c9Y8eOBU7cNbNgwQJGjhxJZGQkMTExTJgwgeXLl9fb5+GHH8Zms7Fx40Z++tOfEhcXR8eOHbnxxhspLi6ut+9bb73FiBEjiIuLIzIyku7du3PjjTee9FyKtBUKIiIWGzlyJCtWrOD2229nxYoVVFdXn3DfW2+9lfvuu48JEyawYMECHnvsMT799FNGjRpFfn5+vX0PHDjAz372M2bMmMEHH3xA//79ufHGG3n00UeZOXMm9957L++88w7R0dFcdtll5Obmut+7adMmhg0bxoYNG3jmmWf46KOPuPjii7n99tt55JFHTlhfcnIyy5cvZ9CgQXTv3p3ly5ezfPlyBg8e3OzzUl1dzZQpUzj//PP54IMPuPHGG3n22Wd54okn3Ps4nU4uuugiHnvsMS655BLee+895s2bx6hRo9i7dy8Ay5cvJyIigkmTJrnrefHFF0943Ndff51LL72U2NhY3njjDV599VUOHz7M2LFj+frrrxvsf8UVV9C7d2/eeecd7r//fl5//XXuvPNO9+vLly9n2rRpdO/enf/85z98/PHHPPTQQzidzmafE5GAZIiIpfLz841zzjnHAAzACAkJMUaNGmXMnj3bKC0tde+3efNmAzB++ctf1nv/ihUrDMD43e9+5942ZswYAzBWr17t3lZQUGDY7XYjIiLCyMnJcW9ft26dARjPP/+8e9vEiRON1NRUo7i4uN6xbrvtNiM8PNwoLCw86XcaM2aM0bdv33rbFi9ebADG4sWL623fvXu3ARhz5851b7vhhhsMwHjzzTfr7Ttp0iQjIyPD/Xz+/PkGYLzyyisnrScqKsq44YYbGmw/vqaamhojJSXF6N+/v1FTU+Per7S01EhKSjJGjRrl3jZr1iwDMJ588sl6n/nLX/7SCA8PN1wul2EYhvH0008bgFFUVHTSGkXaKrWIiFgsISGBZcuWsWrVKv70pz9x6aWXsm3bNmbOnEn//v3dLR2LFy8GzG6LYw0fPpzMzEy++OKLetuTk5MZMmSI+3l8fDxJSUmceeaZpKSkuLdnZmYCsGfPHgAqKyv54osvuPzyy4mMjMTpdLqXSZMmUVlZyXfffdfq5+F4NpuNyZMn19s2YMAAd50An3zyCeHh4a3WzbF161Zyc3O57rrr6nWLRUdHc8UVV/Ddd99RUVFR7z1TpkxpUGNlZSWHDh0CYNiwYQBcddVVvPnmm+Tk5LRKrSKBQkFExEcMHTqU++67j7feeovc3FzuvPNOsrKyePLJJwFzLAmYAeN4KSkp7tfrxMfHN9gvNDS0wfbQ0FDADCB1x3E6ncyZM4eQkJB6y6RJkwAadAN5QmRkJOHh4fW2hYWFuesEyMvLIyUlpdUG9J7qHLtcLg4fPlxve0JCQoMaAY4cOQLAueeey/vvv4/T6eT6668nNTWVfv368cYbb7RKzSL+TkFExAeFhIQwa9YsADZs2AAc/cHbv39/g/1zc3NJTExslWO3b98eu93O9OnTWbVqVaNLXSBpjrpQcfxg09MJNR06dCA3NxeXy9XizzjWqc5xUFAQ7du3b/bnXnrppXzxxRcUFxezZMkSUlNTueaaaxoMgBVpixRERCzW2I8ewObNmwHc3SjnnXceAP/+97/r7bdq1So2b97M+eef3yr1REZGMm7cONauXcuAAQMYOnRog+X4VoCmSE9PB+DHH3+st33BggUtrvWiiy6isrLylJflhoWFuVsoTiYjI4POnTvz+uuvYxwzs0F5eTnvvPOO+0qalgoLC2PMmDHuAbdr165t8WeJBIqG1wuKiFdNnDiR1NRUJk+eTJ8+fXC5XKxbt45nnnmG6Oho7rjjDsD8kfzFL37BnDlzCAoK4qKLLiIrK4vf//73pKWl1btS43Q999xznHPOOYwePZpbb72V9PR0SktL2bFjBx9++CFffvllsz+zU6dOjB8/ntmzZ9O+fXu6du3KF198wbvvvtviOn/6058yd+5cbrnlFrZu3cq4ceNwuVysWLGCzMxMrr76agD69+/PkiVL+PDDD0lOTiYmJoaMjIwGnxcUFMSTTz7JtddeyyWXXMLNN9+Mw+HgqaeeoqioiD/96U/NrvGhhx4iOzub888/n9TUVIqKinjuuecICQlhzJgxLf7uIoFCQUTEYg8++CAffPABzz77LPv378fhcJCcnMz48eOZOXOmezApwEsvvUSPHj149dVXeeGFF4iLi+PCCy9k9uzZLWqlOJEzzjiD77//nscee4wHH3yQQ4cO0a5dO3r16tWibpk6//rXv/j1r3/NfffdR01NDZMnT+aNN95g6NChLfq84OBgFi5cyOzZs3njjTf4y1/+QkxMDAMHDuTCCy907/fcc8/xq1/9iquvvpqKigrGjBlzwmndr7nmGqKiopg9ezbTpk3Dbrdz1llnsXjxYkaNGtXsGkeMGMHq1au57777yMvLo127dgwdOpQvv/ySvn37tuh7iwQSzawqIiIiltEYEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLKIiIiIiIZXx6HhGXy0Vubi4xMTHYbDaryxEREZEmMAyD0tLSJt0LyqeDSG5uLmlpaVaXISIiIi2wb98+UlNTT7qPTweRmJgYwPwisbGxFlcjIiIiTVFSUkJaWpr7d/xkfDqI1HXHxMbGKoiIiIj4maYMq9BgVREREbGMgoiIiIhYRkFERERELOPTY0SawjAMnE4nNTU1VpcirSQkJAS73W51GSIi4gV+HUSqqqrYv38/FRUVVpcirchms5Gamkp0dLTVpYiIiIf5bRBxuVzs3r0bu91OSkoKoaGhmvQsABiGQV5eHtnZ2fTq1UstIyIiAc5vg0hVVRUul4u0tDQiIyOtLkdaUYcOHcjKyqK6ulpBREQkwPn9YNVTTR0r/kctWyIibYd+xUVERMQyHg0iS5cuZfLkyaSkpGCz2Xj//fc9eTgRERHxMx4NIuXl5QwcOJC//vWvnjyMtLL33nuP4OBgevfuzaFDh6wuR0REAphHg8hFF13E448/zk9+8pMm7e9wOCgpKam3iHctXryYa665hlmzZpGUlMSFF17Y4M8hKyuLn//853Tr1o2IiAh69OjBrFmzqKqqsqhqERHxVz41RmT27NnExcW5l7S0NKtLalPWrFnD5Zdfzp///Gd+//vf87///Y/4+HimTJlCZWWle78tW7bgcrn429/+xsaNG3n22Wd5+eWX+d3vfmdh9SIi0lRVThdfb8/n0Q838Z+Vey2txaeCyMyZMykuLnYv+/bta9b7DcOgosppyWIYRpNqnD9/PgkJCTgcjnrbr7jiCq6//vpmfd+mWrJkCaGhoSxbtsy97ZlnniExMZH9+/cDsHXrVi655BKef/55br31VgCioqL4+OOPiY2NZdq0aTidTgAuvPBC5s6dywUXXED37t2ZMmUKd999N++++65H6hcRkdN3qLSSN1ft4+Z/rWbQo4v4f6+u4B/f7OatNdmW1uVT84iEhYURFhbW4vcfqa7hjIf+14oVNd2mRycSGXrq0zl16lRuv/12FixYwNSpUwHIz8/no48+4tNPPz3h+/r27cuePXtO+HrXrl3ZuHFjo6+NHTuW3/zmN1x33XX88MMPZGVl8cADD/DGG2+QnJwMQEZGhjuUHCssLIwFCxac8nsVFxcTHx9/yv1ERMQ7XC6D9TnFfLnlEF9uOcT6nOJ6rydGh3Fenw6cn9nRogpNPhVE2oKIiAiuueYa5s6d6w4ir732GqmpqYwdO/aE71u4cCHV1dUnfD0kJOSkx3388cf5/PPP+cUvfsHGjRu57rrruPzyy1v0HY63c+dO5syZwzPPPNMqnyciIi1TWlnN19vz+WLLIZZszSO/rH7r+4DUOM7rk8R5fZLolxJHUJD18zYFVBCJCLGz6dGJlh27qW666SaGDRtGTk4OnTt3Zu7cuUyfPv2kE3l17dr1tOoLDQ3l3//+NwMGDKBr16785S9/Oa3Pq5Obm8uFF17I1KlTmTFjRqt8poiINN2uvDJ3q8eqrEKqa44OFYgKtTO6VwfOy0xibEYHkmLCLay0cR4NImVlZezYscP9fPfu3axbt474+Hi6dOnS6sez2WxN6h6x2qBBgxg4cCDz589n4sSJrF+/ng8//PCk7zmdrpk63377LQCFhYUUFhYSFRXV/OKPkZuby7hx4xg5ciR///vfT+uzRESkaaqcLlbuLqwNHwfJKqh/49duiVGMy0ji/MwkhqXHExrsU8NBG/Dor/bq1asZN26c+/ldd90FwA033MC8efM8eWifN2PGDJ599llycnIYP378Ka8QOt2umZ07d3LnnXfyyiuv8Oabb3L99dfzxRdftHiK/JycHMaNG8eQIUOYO3euptoXEfGgA8WVLNlqdrcs255HeVWN+7UQu43h3eIZl2F2uXTv4F93LrcZTb3cwwIlJSXExcVRXFxMbGxsvdcqKyvZvXs33bp1Izzc95qaTqWkpITk5GScTifz589n2rRpHjtWTU0No0ePJjk5mXfeeYcDBw7Qv39/7r33Xu65555mf15ubi5jxoyhS5cuzJ8/v96N6Tp16nTa9fr7n62IyOmqcrr4fu9hlmzNY8nWQ2w5UFrv9cToMMZldOC8Pkmc0yuRmPCT/8+ot53s9/t4vt+PEaBiY2O54oor+Pjjj7nssss8eqw//OEPZGVlubt/OnXqxP/93/9x1VVXMWHCBM4888xmfd6iRYvYsWMHO3bsIDU1td5rPpxrRUR82p6CcpZuy+Orbfks35lfr9XDZoOBqe0Ym9GBcRlJ9O/sGwNNW4NaRCw0YcIEMjMzef75560uxacEwp+tiMiplDmcLN9ZwNJteSzdnsee48Z6JESFMrpXImMzkji3dwfio0ItqrT51CLi4woLC1m0aBFffvml7sMjItJGuFwGm/aX8NW2PJZuy+P7vYfrXeESHGRjSNf2nNu7A2N6d+CM5NiAafU4GQURCwwePJjDhw/zxBNPkJGRYXU5IiLiIfllDpZtz2PptnyWbc8jv6z+Pbm6JkRybq8OnNu7AyN7JBAd1vZ+ltveN/YBWVlZVpcgIiIeUOV0sWbPYZZuN1s9NubWv2loVKidkT0SOLd3B87t1YH0xNObRiEQKIiIiIi0kGEY7CmocAeP5TsL6g0yBeibEusOHkO6tvf5eT28TUFERESkGQrKHHy7s4BvduSzbHs+OUVH6r2eGB3K6F4dOLd3Iuf07ECHmJbfQ60tUBARERE5icrqGlZlFfL19ny+3pHfoLslxH50kOm5vdrOINPWoiAiIiJyDIezhnV7i/huVyHLd+Xz/d4iqpyuevv06RTDOT0TOadXIsO7xfvF7UV8lc6ciIi0aVVOFz9mF7F8ZwHf7S5gzZ7DVFbXDx6dYsM5p1ci5/RMZFTPBJ+8eZy/UhAREZE2pbrGxY/ZxXy3q4DvdhWwOuswR6rrDzBNjA5lRPcEzuqewMjuCfToEHXSO6RLyymItBE2m4333nuv1aeTLygoIDMzk5UrV5Kenn7K/R0OB7169eK9995jyJAhrVqLiEhjnDUuNuSWmC0euwpYnVXY4MqW9pEhZujoYQaPnknRCh5eoiAip2X27NlMnjy5SSEEICwsjLvvvpv77ruPzz//3LPFiUibVOMy2Jhrtngs31nAqqzDlDmc9fZpFxnCiG7x7vDROylGA0wtoiAiLXbkyBFeffVVFi5c2Kz3XXvttdxzzz1s3ryZzMxMD1UnIm1F3dTpdV0tK3YXUlpZP3jEhAczopsZOs7qHk9mJ13Z4isCK4gYBlRXnHo/TwiJNG+PeArz58/nzjvvJDc3l7Cwo9eWX3HFFURFRTF//nxPVum2fv167rjjDpYvX05kZCRXXHEFf/7zn4mOjgbA6XRy1113MX/+fOx2OzNmzODAgQMUFxfz/vvvA/DJJ58QHBzMyJEj3Z/76KOP8vLLL7N+/XoSEhIAmDJlCkVFRSxZsoSgoCASEhIYNWoUb7zxBo8++qhXvq+IBA6Xy2DLgVKzxWNXASt3F1J8pLrePjFhwQw/psUjMzkWu4KHTwqsIFJdAX9MsebYv8uF0FNP1Tt16lRuv/12FixYwNSpUwHIz8/no48+4tNPPz3h+/r27cuePXtO+HrXrl3ZuHFjk0qtqKjgwgsv5KyzzmLVqlUcOnSIGTNmcNtttzFv3jwAnnjiCV577TXmzp1LZmYmzz33HO+//z7jxo1zf87SpUsZOnRovc9+4IEH+PTTT5kxYwbvvfceL7/8MkuXLuWHH34gKOjobILDhw9n2bJlTapXRNo2wzDYdrCM5Tvz+W5XISt2F3C4on7wiAq1M6xbPCNrB5j2TYkl2K4ZTP1BYAURPxAREcE111zD3Llz3UHktddeIzU1lbFjx57wfQsXLqS6uvqEr4eEhDS5htdee40jR44wf/58oqLM8PTXv/6VyZMn88QTT9CxY0fmzJnDzJkzufzyy92vH98Fk5WVRUpK/eBnt9v597//zZlnnsn999/PnDlz+Pvf/07Xrl3r7de5c2fdc0dEGmUYBjvzyli+02zxWLGrkILy+jeLiwy1MzQ9nrO6m+Gjf+c4BQ8/FVhBJCTSbJmw6thNdNNNNzFs2DBycnLo3Lkzc+fOZfr06ScdoX38D/np2Lx5MwMHDnSHEICzzz4bl8vF1q1bCQ8P5+DBgwwfPtz9ut1uZ8iQIbhcR6+tP3LkCOHhDa+l7969O08//TQ333wz06ZN49prr22wT0REBBUVFnWjiYhPcbkMth8qY1VWYe04j0Lyyxz19gkPCWJo13j3GI8Bqe0IUfAICIEVRGy2JnWPWG3QoEEMHDiQ+fPnM3HiRNavX8+HH3540ve0ZteMYRgnDD3Hbj9+H8Mw6j1PTEzk8OHDjX7O0qVLsdvtZGVl4XQ6CQ6u/1etsLCQDh06NKleEQksldU1/JhdzKqsQtbsOczqrEJKjhtcGhYcxJCu7d1jPAamttPN4gJUYAURPzJjxgyeffZZcnJyGD9+PGlpaSfdvzW7Zs444wz++c9/Ul5e7m4V+eabbwgKCqJ3797ExcXRsWNHVq5cyejRowGoqalh7dq1nHnmme7PGTRoEP/+978bfP5///tf3n33XZYsWcK0adN47LHHeOSRR+rts2HDBgYNGtTkmkXEfx0ur2L1nsOs3lPI6qzDrM8upqqm/sylESF2BnVpx7B0s9XjzLR2hIfYLapYvElBxCLXXnstd999N6+88kqTrpRpza6Za6+9llmzZnHDDTfw8MMPk5eXx69//Wuuu+46OnbsCMCvf/1rZs+eTc+ePenTpw9z5szh8OHD9VpJJk6cyMyZMzl8+DDt27cHIDs7m1tvvZUnnniCc845h3nz5nHxxRdz0UUXcdZZZ7nfu2zZMh577LFW+04i4hsMw2BvYQWrs8zgsSrrMDsOlTXYLzE6jGHp7RmaHs+w9PZkJseqq6WNUhCxSGxsLFdccQUff/xxq892eiqRkZH873//44477mDYsGH1Lt+tc99993HgwAGuv/567HY7v/jFL5g4cSJ2+9H/Q+nfvz9Dhw7lzTff5Oabb8YwDKZPn87w4cO57bbbAJgwYQK33XYb/+///T/WrVtHdHQ0y5cvp7i4mCuvvNKr31tEWp+zxsWm/SX1gkdeqaPBfj06RDEsPd4dPLrER2rmUgHAZhzf8e9DSkpKiIuLo7i4mNjY2HqvVVZWsnv3brp169bogEl/MGHCBDIzM3n++eetLuWUXC4XmZmZXHXVVfVaMhYuXMjdd9/Nhg0b6l2eezJTp05l0KBB/O53v2v09UD4sxUJVMUV1azdd5jv9xxmzd7DrN1bRMVx06WH2G307xznDh5DurYnPirUoorFCif7/T6eWkQsUFhYyKJFi/jyyy/561//anU5jdqzZw+LFi1izJgxOBwO/vrXv7J7926uueaaevtNmjSJ7du3k5OTc8pxLmDea2bgwIHceeednipdRFqJy2VeRvv93sN8v6eINXsb72aJCQ9maNe6bpZ4BqTGaXyHNJmCiAUGDx7M4cOHeeKJJ8jIyLC6nEYFBQUxb9487r77bgzDoF+/fnz++eeNTsl+xx13NPlzw8LCePDBB1uzVBFpJSWV1fywr4jv9xTx/d7DrN17uMHVLADpCZEM7tKeQV3bMzw9nl5J0ZouXVpMQcQC/jCRV1paGt98843VZYiIhzhrXGw7WMYP2UWsre1i2ZFXxvGd9eEhQQxIbceQru3N8NGlHYnRYY1/qEgLKIiIiAQ4wzDIKqjgx+wifthXzA/ZRWzMLaay2tVg364JkQxKa8eZae0Y0jWePskxuppFPMrvg4gPj7WVFtKfqcjpOVBcybp9RfyYXcSP2cX8mF3UaBdLTFgw/VPj3C0dA9PU2iHe57dBpG4Cr4qKCiIiIiyuRlpTVZV5T4ljLxUWkcYVVVTxQ3YxP+4rMh+zizjUyOWzocFB9E2JZWBqOwamxTEgtR3dEqI0tkMs57dBxG63065dOw4dOgSYc2PomnT/53K5yMvLIzIyssG08CJtXUWVkw05JWYXS3YxP+wrYm9hw3s22YNs9O4Yw8BUM3AMSI0jo5O6WMQ3+fW/9J06dQJwhxEJDEFBQXTp0kXBUtq0KqeLrQdKWZddxI/7zC6W7YdKcTXSc9ktMYoBtaFjYGocfVPiiAhVi6L4B78OIjabjeTkZJKSkk56HxbxL6GhoU2eHE0kEDhrXOzKL3eP5/ghu5jNuSUN7scC0Ck2nAGpcQxMa8fA1Hb07xxHXGTT7zUl4mv8OojUsdvtGk8gIn7B4axh+8EyNuQUsyG3mI25JWzeX9LoFSztIkPcrRx1j0mxmm1YAktABBEREV9U5nCyZX8Jm/aXsDGnhA25xWw7WEp1TcP+lahQO31T4swuljQzdOh+LNIWKIiIiJwmwzDIKTrCptwSNu8vZfP+EjYfKGFPQcOBpABxESH06xxLv5Q4+naOo19KLOm6gkXaKAUREZFmqKw2u1Y217Z0bNpfwpb9JY3O0wHmmI7M5BjOSImlf2dzIGlq+wi1dIjUUhARETmBvFKH2bpRGzg27y9hZ145NY1cuhJit9EzKcYMHcmxnJEcS5/kWN11VuQUFEREpM2rcrrYlV/G1gOltYGjlE25JeSXNZwYDKB9ZAiZtWEjs3bpmRRNaLCu9hJpLgUREWkzXC5zLMfWA6VsPVhqPh4oZVd+WaMDSG02c46OutBRFzw6xoapa0WklSiIiEjAMQyD/cWVbD9Uxo5DZWyrDR7bD5ZSXlXT6HtiwoLJ6BTjbuHITI4ho1MMkaH6Z1LEk/RfmIj4rRqXQc7hI2w/VMr2Q2VsP1jGjrwydh4qo8zR+ODRELuNHh2iyehkBo2MjuZj53YaQCpiBQUREfF51TUu9hRUsONQKTsOlblDx868MhzOhhOBAQQH2UhPjKJnh2h6dYx2h470xCjdc0XEhyiIiIjPcDhr2J1fzvaDZbXdKmbw2J1f3ugYDjDvKts9MYpeHWPolRRNr6RoeiZF0zUhSoNHRfyAgoiIeJXLZXCgpJKs/HJ2F5Sbj/kV7MwrY09BeaM3dQOIDLXTszZk9EqKqX2MJi0+ErsmAhPxWwoiItLqXC6Dg6WV7M4vZ09BRW3YKCerwHx+ou4UgNjwYHp1jHF3qfRMiqZXxxiSY8M186hIAFIQEZEWMQyDQ6UOM2DUtm7sya8gq8AMHI3dxK1OcJCNLvGRpCdG0TUhkm6JUfToYLZwdIjRpbEibYmCiIickGEY5JU6yKpr1XB3pZgtG0eqG78UFsAeZCOtfQTpiVGkJ0TR7ZjQ0bldBMEaMCoiKIiItHmGYZBfVkVWQV3AKCcrv8K9fqJ5NwCCbJDa3mzZ6JZgPtYFj9T2Ebo6RUROSUFEJMAZhkHJESc5RUfYX3yE3KIj5BZXsq+wthslv+KEc26AGTY6t48gPcEMGOmJUXRLjKwNG5G6MkVETouCiIifq6yuYX9xJfuLjtSGjUp32MgtMoNHxUlaNcCcyjwlLoJuiVGk14aMutCRFh9BWLDdS99GRNoaBRERH1bjMsdo5Na1ZBQdIbfIDBh1gaOgvKpJnxUfFUpKu3CS4yLo3M5czG6USNLiIwkPUdgQEe9TEBGxyIm6THKLjrC/qJKcoiMcLKnEeaKJNY4REWInpV04Ke0iSImLILl2vXO7CJLjzPAREaqgISK+R0FExENO1WWyv+jISQeC1rEH2egUG05ynBkuktuF0/mYwNG5XQRxESG65FVE/JKCiEgLHN9lUteCYbZsNL/LpC5kpNQ9totwt3B0iA7Tpa4iErAURESOYRgGRRXV5JU5yCs9ZilzcLCkskVdJnWtFsnHhoy4CPd4DXWZiEhbpiAiAc/lMig6Uk1BmYOC8ioKy6soKHOQV1ZFXmllg8BxopurHc8eZKNjTFhtd4kZLMzAUduaERdBu0h1mYiInIyCiPgdl8ug+Eg1BeUOCsrMYJFfXkVhWZW5rTZoFNaGjsLyqhPeSO1E4iJC6BATRofoMJJizccOMWEkt4ugc21LRlKMukxERE6XgohYprK6htJKJyWV1ebjkep666WVToqOVHG4oprC2sBRUF7F4YoqapqbLDBvppYYHUZ8VCjxUaFm0KhbosNIig2nQ0wYidGhmjdDRMRLFESkRVwug7Iq59EAURsc6gUJh7Pe9nrrlU6qTnIH1qaIOSZYJESFkhAdWrseRkK0+Rhfu719ZKhmABUR8UFeCSIvvvgiTz31FPv376dv37785S9/YfTo0d44tJyAw2m2RpyoJeL4VoqS414rczgxmt8o0YDNBtFhwcSGhxATHkxsRAix4SHEuteDaR9lBom6oJEYHaZgISISIDweRP773//ym9/8hhdffJGzzz6bv/3tb1x00UVs2rSJLl26ePrwfqfGZVBZXYPD6cLhrMFR7aKy9vFk2yqra19zujhSVcORqhrKq5wcqaqhoqqGiionpY6jwcNxmq0RdULtQcRG1A8SMeHm89iIEGLC6m9zh43abdGhwQQFaTCniEhbZTOM1vj/2hMbMWIEgwcP5qWXXnJvy8zM5LLLLmP27NknfW9JSQlxcXEUFxcTGxvbajWVVFazO68cl2HgMsxLNl2GGQJchkGNy6DGMKipMXC6DJwuFzUuA2eNue50mftU1xjU1D531u5b43LhrDGoqjFDgjtUHBMUHO6gYa5XHrOtKZeEtiazNaLxsFAvUJzgNU0LLiIix2vO77dHW0SqqqpYs2YN999/f73tF1xwAd9++22D/R0OBw6Hw/28pKTEI3Wt3VvEDf9Y6ZHPbk0hdhvhwXbCQoIIC7YTFhxEaHAQ4SHmeljto/t5sLlfeEgQkaF2IkODiQy1E1G7HhMefDRchIcQHR6MXa0RIiJth9MBhbsgf1vtsh2SMuGcOy0ryaNBJD8/n5qaGjp27Fhve8eOHTlw4ECD/WfPns0jjzziyZIAc5Kpzu0iCAqCIJuNIJsNGxAUZMNus5mPQRAcFERwkA17kI0QexD2IBvBQTaC7TaCg2qf2221+wQRYre59wmxHxcQQsyAUBcowmoDxtGgcTREhAXbCQ0OUkgQEZGWqSg8JmzUBo78bXA4C4zjuubTRwduEKlz/IROhmE0OsnTzJkzueuuu9zPS0pKSEtLa/V6hneL55v7z2v1zxUREfGaGicU7TkaMvK3QcEO87Gi4MTvC4uFxF6Q2Nt87DTQezU3wqNBJDExEbvd3qD149ChQw1aSQDCwsIICwvzZEkiIiL+par8aNjI23q0haNwJ9Sc5J5WcWn1A0dib3OJ7mhesugjPBpEQkNDGTJkCJ999hmXX365e/tnn33GpZde6slDi4iI+JfKktqwsQUObTZDR95WKN574vcEh0NCr9qgcUzoSOgJoVHeq/00eLxr5q677uK6665j6NChjBw5kr///e/s3buXW265xdOHFhER8T2VxbUhYwsc2mI+5m2FkuwTvycyARIzoENv87EucMSlQZB/z6nk8SAybdo0CgoKePTRR9m/fz/9+vVj4cKFdO3a1dOHFhERsc6Rw40HjtLcE78nuhMk9YEOfaBDhvmYmAFRCd6r28s8Po/I6fDUPCIiIiKtpqKwNmQcFzjKGl4d6haTclzgyDRbOyLae69uD/KZeUREREQChqPMDBiHNpljOOoeTxY4YlMbDxzhcd6r28cpiIiIiBzLVWNO+nVgPRzcAAc3maGjaM+J3xOXZoYNd+jINMdwhKs1/1QUREREpO06chgObjSXA+vNx0ObwXmk8f2jO5ozkSadcfSxQwaExXi37gCiICIiIoGvXivHxtqWjo1QvK/x/UMizaDRsV/tcobZyhHAg0atoiAiIiKB5UjRMWFjAxzYcPJWjrgu0KkfdOx7NHjEd4Mg3dTTGxRERETEP9W1ctSFjVO1cgRHmC0bdWGjUz+zayWinVfLlvoURERExPe1pJWjY99jWjr6q5XDRymIiIiI72jQylEbPtTKEbAURERExBpOh9mqsf+Ho8uhTVBd0fj+auUISAoiIiLiedVHzPk49q89GjoObgJXdcN93a0ctWGjY19zUStHQFIQERGR1uUoM7tT9v8AuevMx7wtYNQ03De8HaScCckDzaXTAIjvrlaONkRBREREWq6yGPb/WL97JX8b0MhtzCITjwkdtY/tuoDN5uWixZcoiIiISNM4Ss3Qkfs95K41l8Jdje8bk1w/cCQPhNgUhQ5pQEFEREQaqq40u1dy6kLH9+YN3xpr6YjrAskD6oeOmI7erlj8lIKIiEhb56oxx3DkrKldvjevXnE5G+4b2xlSBh1dks/UtOdyWhRERETamtIDsG8lZK8yg0fuOqgub7hfZCJ0Hgwpg48GD7V0SCtTEBERCWTOKjjwoxk66sJHY5ODhcaYA0nrgkfnweat7TWmQzxMQUREJJCU5B4NHNmrzNaOGkf9fWxB5uyjqcMgdSh0HgqJvXTJrFhCQURExF85Heblsse2dpTkNNwvIh7ShpuhI3W42doRFuP9ekUaoSAiIuIPDAOKs4+2dOxbaXa51FTV388WZM5Cmjq8NnwMMycIUxeL+CgFERERX1RdCfvX1e9mKd3fcL/IxPqtHSmDICza6+WKtJSCiIiILyjOhr3fHdPasb7hfVhsdujU32zlqAsf7buptUP8moKIiIi3uVyQtxn2LjfDx97vGr+SJSrpaPdK6jCztSM00vv1iniQgoiIiKdVV5qzk+5dbi77Vpj3aDmWzW7OTnrs2A7dh0XaAAUREZHWVn3E7F7Z8w1kfQ3ZqxteQhsSBWnDoMtI6HKWeQmtxnZIG6QgIiJyuqoqIHslZNUGj5zVDa9miUoyA0fXUeZjx/5g1z/BIvqvQESkuaoqzO6VrK/NVo/s1Q0HlsYkQ/o50PVs8zGhp7pZRBqhICIicipV5UeDR9Y35v1ZGgSPFDNw1C2au0OkSRRERESO5yg7Jnh8DbnfN7wTbWxqbeiobfHQZbQiLaIgIiJSVQH7voPdS2uDx9qGwSMurX6LR7uuCh4irUBBRETanhqnGTZ2LTGX7JUNB5e26wLpo4+O8Wjf1YpKRQKegoiIBD7DgPxttcHjK8haBo6S+vvEpkK3c80l/WwziIiIxymIiEhgqiiEXYthxxew88uG92mJaF8bPMZA97EaXCpiEQUREQkMrhqzu2XH5+aSswYM19HX7WHQdaQZOrqPhU4DIMhuVbUiUktBRET8V+lB2PmFGTx2fglHDtd/Pakv9DwfepxnTiIWEmFNnSJyQgoiIuI/aqrNy2rrWj0OrK//elgc9BgLPcdDj/MhrrMlZYpI0ymIiIhvK9prjvPY8bk50LSqtP7rKYPM4NFzvHm/Fk2bLuJX9F+siPiW6kpz2vS68JG/tf7rkYlmd0vP8dB9HER3sKZOEWkVCiIiYr2CnUe7W3YvA+eRo6/ZgiB1eG2rx/mQfCYEBVlWqoi0LgUREfE+w4ADP8LmD2HzR5C3uf7rMSnHtHqMMS+1FZGApCAiIt7hcpkzmG7+EDYvMMd+1AkKhq6jjo71SDpDc3qItBEKIiLiOTXV5iymmz+ELR9D2cGjrwVHQK/x0Gcy9J4IEe0sK1NErKMgIiKtq6rCnNNj84ew7ROoLD76WlgcZFwImZPNy2tDI62rU0R8goKIiJy+ymLYtsjsctnxOVRXHH0tqgP0udgMH+nnQnCodXWKiM9REBGRlinLg60fm4NNdy0BV/XR1+K6mMEjczKkDddU6iJyQgoiItJ0Rftgy0dmt8ve5fXv5ZKYcTR8JA/UYFMRaRIFERE5ubxtsOVDM3zkrq3/Wsog6HOJGT46ZFhTn4j4NQUREanPMGD/D7WX2X543MymNvMy28zJ5riPdl0sK1NEAoOCiIiAqwb2rTwaPoqPneMjxJxULHMyZEyC6CTr6hSRgKMgItJWOasga6k52HTLx1B+6OhrIZHmxGKZU6D3BRAeZ12dIhLQFERE2hLDgJzvYc1c2LQAHMfM8REeB70vqp3j4zzN8SEiXqEgItIWVJbA+jdhzTw4sP7o9qikY+b4GK05PkTE6xRERAKVYUDu97B6Lmx45+gkY/Yw6Hs5DL4OuozUHB8iYikFEZFAU1kC698yu1+Obf1IzIAh02Hg1RAZb1l5IiLHUhARCQSGYc7xsWYurH8HqsvN7fYw6HuZGUC6jNQkYyLicxRERPyZo9Rs/Vg9Fw78eHR7Ym8Y8jO1foiIz1MQEfFHOd+bA0/Xv12/9eOMS2Hoz9T6ISJ+Q0FExF/UtX6smWfOfFonoZcZPgb+VK0fIuJ3FEREfF3uWrPrpV7rR6jZ+jHkZ+aU62r9EBE/5dEg8oc//IGPP/6YdevWERoaSlFRkScPJxI4HKVm8Fgzt2Hrx5DpZutHVIJl5YmItBaPBpGqqiqmTp3KyJEjefXVVz15KJHAkLuu9sqXt6GqzNzmbv2YDl3PVuuHiAQUjwaRRx55BIB58+Z58jAi/s1Rak44tnou7F93dHtCz9orX9T6ISKBy6fGiDgcDhwOh/t5SUmJhdWIeFjuutorX96q3/qROcUcfKrWDxFpA3wqiMyePdvdiiISkBxlsOFtM4Dkrj26PaFn7diPa9T6ISJtSlBz3/Dwww9js9lOuqxevbpFxcycOZPi4mL3sm/fvhZ9jojP2f8DfHQnPNMHPrzDDCFBIdDvCrjhI7htNYz6tUKIiLQ5zW4Rue2227j66qtPuk96enqLigkLCyMsLKxF7xXxOY4yc+zHmnnmzefqxPcwWz/OvAaiEq2qTkTEJzQ7iCQmJpKYqH88RU5o/4/mlS8/vgVVpea2oBDInGyO/UgfrbEfIiK1PDpGZO/evRQWFrJ3715qampYt24dAD179iQ6OtqThxbxrqryo60fOWuObo/vXtv6ca1aP0REGuHRIPLQQw/xz3/+0/180KBBACxevJixY8d68tAi3nFgvXnZ7Y9vNmz9GDLdbP0IavZQLBGRNsNmGIZhdREnUlJSQlxcHMXFxcTGxlpdjoipqhw2vGt2vxzb+tG+29HWj+gOlpUnImK15vx++9TluyI+7cCG2rEfb4Kjdo6boBDIvKS29eNctX6IiDSTgojIyRgGZH0NS5+C3V8d3a7WDxGRVqEgItIYw4CdX8DSp2HvcnNbUDD0qW396DZGrR8iIq1AQUTkWIYBWz8xW0Dq5v6wh8Lg6+HsO6BdF2vrExEJMAoiIgAuF2z+AJY+AwfXm9uCI2DojeaMp7HJ1tYnIhKgFESkbatxmvN/LHsG8rea20KjYfhNcNavNP5DRMTDFESkbXJWwY//gWV/hsO7zW3hcTDiVhhxM0TGW1ufiEgboSAibYvLBevfhC//AMV7zW2RCTDyVzBshhlGRETEaxREpO3YuRg++705GypAVBKcfbs5DiQ0ytraRETaKAURCXwHN8JnD8GOz83nYbEw+i4YcQuERFhbm4hIG6cgIoGrJBcW/wHWvQ6Gy5wHZNgMOPdeiEqwujoREUFBRAJRZQl88xwsfwGcR8xtZ1wG5z8ECT0sLU1EROpTEJHAUVMNa+bBkj9BRb65Le0suOBxSBtmaWkiItI4BRHxf4YBWz6Czx+Ggh3mtoSeMP5hc0p2m83K6kRE5CQURMS/7VtlXglTdz+YyEQYe795Pxh7iKWliYjIqSmIiH8q2AlfPAKbPjCfB0eYc4GcfQeEx1pbm4iINJmCiPiX8gJY+iSsehVc1YANBl0L4x6A2BSrqxMRkWZSEBH/UH0EVrxsTsnuKDG39RwPEx6Fjn2trU1ERFpMQUR8m2GYN6X7bBaUZJvbOvWHCY9Bj3HW1iYiIqdNQUR8V95W+Pi3kLXMfB6bCuf/HvpfBUFB1tYmIiKtQkFEfNO61+Gju8wJyYIjYPRvYdRtmpJdRCTAKIiIb6muhE/vMycmA+hxHlzyF2jf1cqqRETEQxRExHcczoI3r4f9PwA2GPc7GH23umFERAKYgoj4hm3/g3dvgspiiIiHK181W0NERCSgKYiItVw15h1ylz1jPk8dBlPnQVyqpWWJiIh3KIiIdcry4J0bYfdS8/nwm80b1AWHWluXiIh4jYKIWGPvd/DWdCjdDyFRMOV56H+l1VWJiIiXKYiIdxkGfPeSeaM6lxMSM2Dav6BDhtWViYiIBRRExHsqS2DBbUdvVNfvSpj8HIRFW1uXiIhYRkFEvOPgJnjzOijYAUEhMPGPMPwmsNmsrkxERCykICKe98N/4aPfQHWFOU37Vf+E1KFWVyUiIj5AQUQ8x+mAT2fC6lfN5z3Og5/8H0QlWFuXiIj4DAUR8YyivfDmDZD7PWCDMffBmHshyG51ZSIi4kMURKT1bf/MnCX1yGGIaG+2gvQab3VVIiLigxREpPW4amDJn2DpU4ABKYPN8SDtulhdmYiI+CgFEWkd5fnwzgzYtdh8PmyGeWVMcJi1dYmIiE9TEJHTt28VvHUDlORASCRMfh4GTLW6KhER8QMKItJyhgEr/w7/ewBc1ZDQy5wlNSnT6spERMRPKIhIyzjKYMGvYeO75vO+l8OUORAWY21dIiLiVxREpPkObTFnSc3fBkHB5h1zR9yiWVJFRKTZFESkeda/DQtuh+pyiEmBqfOgywirqxIRET+lICJN46yCRQ+YY0IAuo2BK16F6A7W1iUiIn5NQUROrWgfvDUdclabz8+9B8bO1CypIiJy2hRE5OR2fGHOD3KkEMLbwU/+Dr0nWl2ViIgECAURaZzLZc6QumQ2YEDymXDVfGjf1erKREQkgCiISEPlBea9YnZ+YT4f8jO48E8QEm5tXSIiEnAURKS+7DXmLKnF+yA4Ai55Fs78qdVViYhIgFIQEZNhwKr/g09nmrOkxvcwZ0nt2NfqykREJIApiAhUlcOHd8D6t8znmZPh0hcgPM7aukREJOApiLR1edvMWVLztoDNDhMehZG/0iypIiLiFQoibdmGd837xVSVQXQnc5bUriOtrkpERNoQBZG2yFkFnz0EK14yn6ePhiv/AdFJ1tYlIiJtjoJIW1OcY86Smr3SfH7OnTDuQbDrr4KIiHiffn3akp2L4Z2fQ0UBhMXB5S9Dn0lWVyUiIm2YgkhbYBiw7Bn48nHAgE4DzFlS47tZXZmIiLRxCiJtwZePmUEEYNB1MOkpCImwtiYREREURALfV08dDSEXPgFn3WJtPSIiIscIsroA8aBv58Dix831Cx5XCBEREZ+jIBKoVr4Cix4018c9CKN+bW09IiIijVAQCUTf/wsW3m2un3MXnHu3tfWIiIicgMeCSFZWFj//+c/p1q0bERER9OjRg1mzZlFVVeWpQwrA+rfN2VIBzvolnP+QpmsXERGf5bHBqlu2bMHlcvG3v/2Nnj17smHDBm666SbKy8t5+umnPXXYti1vG7z/S8CAIT+DiX9UCBEREZ9mMwzD8NbBnnrqKV566SV27drVpP1LSkqIi4ujuLiY2NhYD1fn52qc8I8LIGcN9Dgfrn0bgtTzJiIi3tec32+vXr5bXFxMfHz8CV93OBw4HA7385KSEm+UFRi++YsZQsLj4NK/KoSIiIhf8Nqv1c6dO5kzZw633HLiS0hnz55NXFyce0lLS/NWef7twAZY8idz/aInITbF2npERESaqNlB5OGHH8Zms510Wb16db335ObmcuGFFzJ16lRmzJhxws+eOXMmxcXF7mXfvn3N/0ZtjbMK3r8FXNWQcTEMmGZ1RSIiIk3W7DEi+fn55Ofnn3Sf9PR0wsPDATOEjBs3jhEjRjBv3jyCmtFloDEiTbD4j/DVExARD79aAdFJVlckIiJtnEfHiCQmJpKYmNikfXNychg3bhxDhgxh7ty5zQoh0gS5a2Fp7RVIFz+jECIiIn7HY4NVc3NzGTt2LF26dOHpp58mLy/P/VqnTp08ddi2o7oS3rsFjBroezn0+4nVFYmIiDSbx4LIokWL2LFjBzt27CA1NbXea168YjhwLfkj5G2BqCSY9IzV1YiIiLSIx/pKpk+fjmEYjS5ymvatNG9oBzD5OYhKsLYeERGRFtKgDX9TVVHbJeOCgT+FPpOsrkhERKTFFET8zRePQuFOiEmBC/9kdTUiIiKnRUHEn+xeBiteMtenzIGIdpaWIyIicroURPyFoxQ++KW5PvgG6DXe2npERERagYKIv1j0eyjaC+26wMQ/WF2NiIhIq1AQ8Qc7voA1c831S1+AsBhr6xEREWklCiK+7kgRLPi1uT78Zuh2rqXliIiItCYFEV/3v99BSQ7Ed4fxs6yuRkREpFUpiPiyLQth3WuADS57CUKjrK5IRESkVSmI+KqKQvjwDnN91G3Q5Sxr6xEREfEABRFftfBuKD8EiRkw7kGrqxEREfEIBRFftPF92PAO2Oxw+csQEm51RSIiIh6hIOJryg7Bx3eZ66Pvgs6Dra1HRETEgxREfIlhwEd3QkUBdOwP595rdUUiIiIepSDiS358E7Z8BEEhcPlLEBxqdUUiIiIepSDiK0py4ZN7zPUx90Gn/tbWIyIi4gUKIr7AMGDB7VBZDCmD4Zw7ra5IRETEKxREfMHaf8GOz8AeZl4lYw+2uiIRERGvUBCxWtFe+PR35vp5D0KHDGvrERER8SIFESu5XPDBr6CqFNLOgpG/sroiERERr1IQsdLqV2H3UgiOgMtehCC71RWJiIh4lYKIVQp2wmcPmesTHoGEHtbWIyIiYgEFESu4aswumeoKSB8Nw26yuiIRERFLKIhY4buXYO9yCI2GS1+AIP0xiIhI26RfQG/L2wpfPGquT/wDtO9qbT0iIiIWUhDxphonvHcL1Dig53gYfIPVFYmIiFhKQcSbvvkL5H4PYXEw+Xmw2ayuSERExFIKIt5yYAMs+ZO5ftETENfZ2npERER8gIKINzir4P1bwFUNGZNg4NVWVyQiIuITFES8YelTcGA9RMTDJX9Rl4yIiEgtBRFPy/kelj1jrl/8DMR0tLYeERERH6Ig4knVlfD+rWDUQN/Lod9PrK5IRETEpyiIeNKSP0LeFojqAJOesboaERERn6Mg4il7V8C3c8z1yc9BVIK19YiIiPggBRFPqKqo7ZJxwYCroc/FVlckIiLikxREPOGLR6BwJ8SkwEV/sroaERERn6Ug0tqKs2HF38z1KXMgor219YiIiPgwBZHWtv5twICuZ0Ov8VZXIyIi4tMURFrb+rfMx/5Tra1DRETEDyiItKaDG+HgBggKgTMutboaERERn6cg0pp+fNN87D0RIuOtrUVERMQPKIi0FperdnwI6pYRERFpIgWR1rJ3OZRkQ1gs9L7Q6mpERET8goJIa1lf2y2TOQVCwq2tRURExE8oiLQGZxVsfN9cH6BuGRERkaZSEGkNOz6DyiKI7gTpo62uRkRExG8oiLSGuqtl+l8JQXZraxEREfEjCiKnq7IYtn1qrg+4ytpaRERE/IyCyOna/CE4KyExAzoNsLoaERERv6IgcrrqumUGTAWbzdpaRERE/IyCyOko2Q+7l5rrmsRMRESk2RRETseGdwAD0kZA+3SrqxEREfE7CiKno24SM7WGiIiItIiCSEvlbYP9P0BQMPT9idXViIiI+CUFkZaqaw3pOR6iEqytRURExE8piLSEYRwziZm6ZURERFpKQaQl9q2Eoj0QGg0Zk6yuRkRExG8piLREXbdMn0sgNNLaWkRERPyYgkhz1VTDxvfMdd1pV0RE5LR4NIhMmTKFLl26EB4eTnJyMtdddx25ubmePKTn7fwSKgogqgN0G2t1NSIiIn7No0Fk3LhxvPnmm2zdupV33nmHnTt3cuWVV3rykJ5XN0i13xVgD7a2FhERET9nMwzD8NbBFixYwGWXXYbD4SAkJOSU+5eUlBAXF0dxcTGxsbFeqPAUHKXwVC9wHoGbvoTOQ6yuSERExOc05/fba/9LX1hYyGuvvcaoUaNOGEIcDgcOh8P9vKSkxFvlNc2Wj80QEt8DUgZbXY2IiIjf8/hg1fvuu4+oqCgSEhLYu3cvH3zwwQn3nT17NnFxce4lLS3N0+U1j/tOu1fpTrsiIiKtoNlB5OGHH8Zms510Wb16tXv/e+65h7Vr17Jo0SLsdjvXX389J+oNmjlzJsXFxe5l3759Lf9mra3sEOxabK5rEjMREZFW0ewxIvn5+eTn5590n/T0dMLDwxtsz87OJi0tjW+//ZaRI0ee8lg+NUbku5fh0/vMcSE3fWltLSIiIj7Mo2NEEhMTSUxMbFFhdZnn2HEgfsN9p92rrK1DREQkgHhssOrKlStZuXIl55xzDu3bt2fXrl089NBD9OjRo0mtIT6lYCfkrAGbHfrpTrsiIiKtxWODVSMiInj33Xc5//zzycjI4MYbb6Rfv3589dVXhIWFeeqwnrH+LfOxxziITrK2FhERkQDisRaR/v378+WXATCWot6ddtUtIyIi0pp0r5lTyfkeCndCSCT0udjqakRERAKKgsip1A1SzZgEYdHW1iIiIhJgFEROpsYJG94x1weoW0ZERKS1KYiczO4lUJ4HkQnQ4zyrqxEREQk4CiIn82Pt1TJ9fwL2U9+kT0RERJpHQeREqipgy0fmurplREREPEJB5ES2LoSqMmifDqnDrK5GREQkICmInIh77pCputOuiIiIhyiINKa8AHZ+Ya5rEjMRERGPURBpzMZ3weWE5IHQobfV1YiIiAQsBZHGbHzPfFRriIiIiEcpiByvqgL2rTTX+0yythYREZEApyByvJzV4KqGmBRo383qakRERAKagsjx9nxrPnYdpatlREREPExB5HhZX5uPXUdZW4eIiEgboCByLGcVZK8y19PPsbYWERGRNkBB5Fi5a8FZad7kLlGX7YqIiHiagsix9nxjPmp8iIiIiFcoiByr7rLdLiOtrUNERKSNUBCpYxhHx4ekDre2FhERkTZCQaRO0R6oyIegEOjU3+pqRERE2gQFkTrZq83H5AEQEm5tLSIiIm2Egkgdd7fMMGvrEBERaUMUROrUtYgoiIiIiHiNggiA0wEHfjTXOw+xthYREZE2REEEYP+PUFMFkYnQPt3qakRERNoMBREwZ1QFszVEE5mJiIh4jYIIwMEN5qMu2xUREfEqBRGAQ5vMx45nWFuHiIhIG6Mg4nLBoc3melJfa2sRERFpYxREivdCVRnYQyGhh9XViIiItCkKIgdru2USM8AeYm0tIiIibYyCyKGN5qPGh4iIiHidgkhdi0iSgoiIiIi3KYi4r5jRQFURERFva9tBxOmA/O3mulpEREREvK5tB5H8bWDUQHgcxKZYXY2IiEib07aDiHt8SF9N7S4iImKBth1EdMWMiIiIpdp2ENEVMyIiIpZq20FEV8yIiIhYqu0GkSOHoSTHXE/KtLYWERGRNqrtBpG6bpm4NPOqGREREfG6thtEDml8iIiIiNXabhA5WHfFjMaHiIiIWKXtBhENVBUREbFc2wwihgGHNpvr6poRERGxTNsMIsX7wFECQSGQ2MvqakRERNqsthlE6q6YSewN9hBraxEREWnDgq0uwBIJPWDcA7psV0RExGJtM4gk9oIx91pdhYiISJvXNrtmRERExCcoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLeCWIOBwOzjzzTGw2G+vWrfPGIUVERMQPeCWI3HvvvaSkpHjjUCIiIuJHPB5EPvnkExYtWsTTTz/t6UOJiIiIn/HoTe8OHjzITTfdxPvvv09kZOQp93c4HDgcDvfzkpIST5YnIiIiFvNYEDEMg+nTp3PLLbcwdOhQsrKyTvme2bNn88gjjzTYrkAiIiLiP+p+tw3DOPXORjPNmjXLAE66rFq1ynjuueeMUaNGGU6n0zAMw9i9e7cBGGvXrj3hZ1dWVhrFxcXuZdOmTac8lhYtWrRo0aLFN5d9+/adMlfYjCbFlaPy8/PJz88/6T7p6elcffXVfPjhh9hsNvf2mpoa7HY71157Lf/85z9PeSyXy0Vubi4xMTH1PudESkpKSEtLY9++fcTGxp76y8hp0zn3Pp1z79M59z6dc+9rzXNuGAalpaWkpKQQFHTy4ajNDiJNtXfv3npdKrm5uUycOJG3336bESNGkJqa2urHLCkpIS4ujuLiYv3F9RKdc+/TOfc+nXPv0zn3PqvOucfGiHTp0qXe8+joaAB69OjhkRAiIiIi/kczq4qIiIhlPHr57rHS09ObNnr2NISFhTFr1izCwsI8ehw5Sufc+3TOvU/n3Pt0zr3PqnPusTEiIiIiIqeirhkRERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERy/hdEHnxxRfp1q0b4eHhDBkyhGXLlp10/6+++oohQ4YQHh5O9+7defnll71UaeBozjl/9913mTBhAh06dCA2NpaRI0fyv//9z4vVBobm/j2v88033xAcHMyZZ57p2QIDUHPPucPh4IEHHqBr166EhYXRo0cP/vGPf3ip2sDQ3HP+2muvMXDgQCIjI0lOTuZnP/sZBQUFXqrW/y1dupTJkyeTkpKCzWbj/fffP+V7vPIb2tyb3lnpP//5jxESEmK88sorxqZNm4w77rjDiIqKMvbs2dPo/rt27TIiIyONO+64w9i0aZPxyiuvGCEhIcbbb7/t5cr9V3PP+R133GE88cQTxsqVK41t27YZM2fONEJCQozvv//ey5X7r+ae8zpFRUVG9+7djQsuuMAYOHCgd4oNEC0551OmTDFGjBhhfPbZZ8bu3buNFStWGN98840Xq/ZvzT3ny5YtM4KCgoznnnvO2LVrl7Fs2TKjb9++xmWXXeblyv3XwoULjQceeMB45513DMB47733Trq/t35D/SqIDB8+3LjlllvqbevTp49x//33N7r/vffea/Tp06fetptvvtk466yzPFZjoGnuOW/MGWecYTzyyCOtXVrAauk5nzZtmvHggw8as2bNUhBppuae808++cSIi4szCgoKvFFeQGruOX/qqaeM7t2719v2/PPPG6mpqR6rMZA1JYh46zfUb7pmqqqqWLNmDRdccEG97RdccAHffvtto+9Zvnx5g/0nTpzI6tWrqa6u9litgaIl5/x4LpeL0tJS4uPjPVFiwGnpOZ87dy47d+5k1qxZni4x4LTknC9YsIChQ4fy5JNP0rlzZ3r37s3dd9/NkSNHvFGy32vJOR81ahTZ2dksXLgQwzA4ePAgb7/9NhdffLE3Sm6TvPUb6rUp3k9Xfn4+NTU1dOzYsd72jh07cuDAgUbfc+DAgUb3dzqd5Ofnk5yc7LF6A0FLzvnxnnnmGcrLy7nqqqs8UWLAack53759O/fffz/Lli0jONhv/pP2GS0557t27eLrr78mPDyc9957j/z8fH75y19SWFiocSJN0JJzPmrUKF577TWmTZtGZWUlTqeTKVOmMGfOHG+U3CZ56zfUb1pE6thstnrPDcNosO1U+ze2XU6suee8zhtvvMHDDz/Mf//7X5KSkjxVXkBq6jmvqanhmmuu4ZFHHqF3797eKi8gNefvucvlwmaz8dprrzF8+HAmTZrEn//8Z+bNm6dWkWZozjnftGkTt99+Ow899BBr1qzh008/Zffu3dxyyy3eKLXN8sZvqN/871NiYiJ2u71BWj506FCDxFanU6dOje4fHBxMQkKCx2oNFC0553X++9//8vOf/5y33nqL8ePHe7LMgNLcc15aWsrq1atZu3Ytt912G2D+SBqGQXBwMIsWLeK8887zSu3+qiV/z5OTk+ncuTNxcXHubZmZmRiGQXZ2Nr169fJozf6uJed89uzZnH322dxzzz0ADBgwgKioKEaPHs3jjz+uFm4P8NZvqN+0iISGhjJkyBA+++yzets/++wzRo0a1eh7Ro4c2WD/RYsWMXToUEJCQjxWa6BoyTkHsyVk+vTpvP766+q/babmnvPY2FjWr1/PunXr3Mstt9xCRkYG69atY8SIEd4q3W+15O/52WefTW5uLmVlZe5t27ZtIygoiNTUVI/WGwhacs4rKioICqr/k2W32wE8fmf3tsprv6GtOvTVw+ou93r11VeNTZs2Gb/5zW+MqKgoIysryzAMw7j//vuN6667zr1/3aVHd955p7Fp0ybj1Vdf1eW7zdTcc/76668bwcHBxgsvvGDs37/fvRQVFVn1FfxOc8/58XTVTPM195yXlpYaqampxpVXXmls3LjR+Oqrr4xevXoZM2bMsOor+J3mnvO5c+cawcHBxosvvmjs3LnT+Prrr42hQ4caw4cPt+or+J3S0lJj7dq1xtq1aw3A+POf/2ysXbvWfcm0Vb+hfhVEDMMwXnjhBaNr165GaGioMXjwYOOrr75yv3bDDTcYY8aMqbf/kiVLjEGDBhmhoaFGenq68dJLL3m5Yv/XnHM+ZswYA2iw3HDDDd4v3I819+/5sRREWqa553zz5s3G+PHjjYiICCM1NdW46667jIqKCi9X7d+ae86ff/5544wzzjAiIiKM5ORk49prrzWys7O9XLX/Wrx48Un/fbbqN9RmGGrTEhEREWv4zRgRERERCTwKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJiGQURERERscz/B+wEJt4XwlxtAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "a = np.sort(np.random.random(30))\n", "b = a**2\n", "c = np.log(a)\n", "plt.plot(a, b, label='y = x^2')\n", "plt.plot(a, c, label='y = log(x)')\n", "plt.legend()\n", "plt.title(\"Some functions\")\n", "plt.show()" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 4 }