{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 强大的pandas"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 1. pandas简介"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"#公认的pandas导包方式\n",
"import pandas as pd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2. pandas数据结构介绍"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"pandas有两个常用的数据结构：Series和DataFrame，他们为很多问题提供了解决模板"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2.1 Series"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Series是一种一维的数组型对象，它包含了一个值序列，并且包含了数据标签，称为索引，最简单的序列可以仅仅由一个数组形成"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 4\n",
"1 7\n",
"2 -5\n",
"3 3\n",
"dtype: int64\n",
"a 4\n",
"b 7\n",
"c -5\n",
"d 3\n",
"dtype: int64\n"
]
}
],
"source": [
"obj = pd.Series([4,7,-5,3])\n",
"print(obj)#输出的数据中左边为索引，右边为数据\n",
"obj1 = pd.Series([4,7,-5,3], index=['a', 'b', 'c', 'd'])#指定索引\n",
"print(obj1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"在Series中你可以使用索引进行取值"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"7\n",
"4\n",
"a 4\n",
"b 7\n",
"d 3\n",
"dtype: int64\n"
]
}
],
"source": [
"print(obj[1])\n",
"print(obj1['a'])\n",
"print(obj1[['a', 'b','d']])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"在Series中也可以进行数学操作"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"a 4\n",
"b 7\n",
"d 3\n",
"dtype: int64\n",
"a 8\n",
"b 14\n",
"c -10\n",
"d 6\n",
"dtype: int64\n",
"a 54.598150\n",
"b 1096.633158\n",
"c 0.006738\n",
"d 20.085537\n",
"dtype: float64\n"
]
}
],
"source": [
"import numpy as np\n",
"print(obj1[obj1>0])\n",
"print(obj1*2)\n",
"print(np.exp(obj1))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"如果你已经有数据包含在python的字典中，可以使用字典直接生成Series："
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Ohio 35000\n",
"Texas 71000\n",
"Oregon 16000\n",
"Utah 5000\n",
"dtype: int64\n"
]
}
],
"source": [
"sdata = {'Ohio':35000, 'Texas':71000, 'Oregon':16000, 'Utah':5000}\n",
"obj2 = pd.Series(sdata)\n",
"print(obj2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2.2 DataFrame"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"DataFrame表示的是矩阵的数组表，它包含以排序的列集合，每一列可以是不同的值类型(数组/字符串/布尔值等)，它可以被视为一个共享相同索引的Series的字典。\n",
"\n",
"有多种方式可以构建DataFrame，其中最常用的方式是利用包含等长度列表或Numpy数组的字典来形成DataFrame。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" state year pop\n",
"0 Ohio 2000 1.5\n",
"1 Ohio 2001 1.7\n",
"2 Ohio 2002 3.6\n",
"3 Nevada 2001 2.4\n",
"4 Nevada 2002 2.9\n",
"5 Nevada 2003 3.2\n"
]
}
],
"source": [
"data={'state':['Ohio', 'Ohio', 'Ohio', 'Nevada', 'Nevada', 'Nevada'],\n",
" 'year':[2000,2001,2002,2001,2002,2003],\n",
" 'pop':[1.5,1.7,3.6,2.4,2.9,3.2]}\n",
"frame = pd.DataFrame(data)\n",
"print(frame)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" state year pop\n",
"0 Ohio 2000 1.5\n",
"1 Ohio 2001 1.7\n",
"2 Ohio 2002 3.6\n",
"3 Nevada 2001 2.4\n",
"4 Nevada 2002 2.9\n"
]
}
],
"source": [
"print(frame.head())#展示数据的前五列"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" year state pop\n",
"0 2000 Ohio 1.5\n",
"1 2001 Ohio 1.7\n",
"2 2002 Ohio 3.6\n",
"3 2001 Nevada 2.4\n",
"4 2002 Nevada 2.9\n",
"5 2003 Nevada 3.2\n"
]
}
],
"source": [
"#指定列的顺序\n",
"print(pd.DataFrame(data, columns=['year', 'state', 'pop']))"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" year state pop debt\n",
"one 2000 Ohio 1.5 NaN\n",
"two 2001 Ohio 1.7 NaN\n",
"three 2002 Ohio 3.6 NaN\n",
"four 2001 Nevada 2.4 NaN\n",
"five 2002 Nevada 2.9 NaN\n",
"six 2003 Nevada 3.2 NaN\n"
]
}
],
"source": [
"#如果指定的类不包含在字典当中，那么会出现缺失值\n",
"frame1 = pd.DataFrame(data, columns=['year', 'state', 'pop', 'debt'],\n",
" index=['one', 'two', 'three', 'four', 'five', 'six'])\n",
"print(frame1)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Index(['year', 'state', 'pop', 'debt'], dtype='object')\n"
]
}
],
"source": [
"print(frame1.columns)#显示数据的列索引"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"one Ohio\n",
"two Ohio\n",
"three Ohio\n",
"four Nevada\n",
"five Nevada\n",
"six Nevada\n",
"Name: state, dtype: object\n"
]
}
],
"source": [
"#也可以像字典类型通过索引取某一列的值\n",
"print(frame1['state'])"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" year state pop debt\n",
"one 2000 Ohio 1.5 16.5\n",
"two 2001 Ohio 1.7 16.5\n",
"three 2002 Ohio 3.6 16.5\n",
"four 2001 Nevada 2.4 16.5\n",
"five 2002 Nevada 2.9 16.5\n",
"six 2003 Nevada 3.2 16.5\n"
]
}
],
"source": [
"#列的引用是可以修改的，可以将空的debt的值进行修改\n",
"frame1['debt']=16.5\n",
"print(frame1)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" year state pop debt\n",
"one 2000 Ohio 1.5 0.0\n",
"two 2001 Ohio 1.7 1.0\n",
"three 2002 Ohio 3.6 2.0\n",
"four 2001 Nevada 2.4 3.0\n",
"five 2002 Nevada 2.9 4.0\n",
"six 2003 Nevada 3.2 5.0\n"
]
}
],
"source": [
"#还可以用np中的arange进行赋值\n",
"frame1['debt']=np.arange(6.)\n",
"print(frame1)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" one two three four five six\n",
"year 2000 2001 2002 2001 2002 2003\n",
"state Ohio Ohio Ohio Nevada Nevada Nevada\n",
"pop 1.5 1.7 3.6 2.4 2.9 3.2\n",
"debt 0 1 2 3 4 5\n"
]
}
],
"source": [
"#还可以对DataFrame数据进行转置操作\n",
"print(frame1.T)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2.3 索引对象"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"pandas中的索引对象是用于存储轴标签和其他元数据的。在构造Series或DataFrame时，你所使用的任意数组或标签序列都可以在内部转换为索引对象。"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Index(['a', 'b', 'c'], dtype='object')\n"
]
}
],
"source": [
"obj = pd.Series(range(3), index=['a','b','c'])\n",
"index = obj.index\n",
"print(index)#索引对象是不可变的"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Int64Index([0, 1, 2], dtype='int64')\n",
"0 1.5\n",
"1 -2.5\n",
"2 0.0\n",
"dtype: float64\n"
]
}
],
"source": [
"labels = pd.Index(np.arange(3))\n",
"print(labels)\n",
"obj2 = pd.Series([1.5,-2.5,0], index=labels)\n",
"print(obj2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3. 基本功能"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3.1 重建索引"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"reindex是pandas对象的重要方法，该方法用于创建一个符合新索引的新对象。"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"d 4.5\n",
"b 7.2\n",
"a -5.3\n",
"c 3.6\n",
"dtype: float64\n"
]
}
],
"source": [
"obj = pd.Series([4.5,7.2,-5.3,3.6],index=['d','b','a','c'])\n",
"print(obj)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"a -5.3\n",
"b 7.2\n",
"c 3.6\n",
"d 4.5\n",
"e NaN\n",
"dtype: float64\n"
]
}
],
"source": [
"obj2 = obj.reindex(['a','b','c','d','e'])#如果原本索引值不存在会引入缺失值\n",
"print(obj2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"对于一些顺序数据，比如时间序列，在重建索引时可能需要进行插值或填值，我们可以使用`ffill`方法进行插值，该方法会将上面的值传递到下面"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 blue\n",
"2 purple\n",
"4 yellow\n",
"dtype: object\n"
]
}
],
"source": [
"obj3 = pd.Series(['blue', 'purple', 'yellow'], index=[0,2,4])\n",
"print(obj3)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 blue\n",
"1 blue\n",
"2 purple\n",
"3 purple\n",
"4 yellow\n",
"5 yellow\n",
"dtype: object\n"
]
}
],
"source": [
"obj4 = obj3.reindex(range(6), method='ffill')\n",
"print(obj4)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" A\n",
"1 1\n",
"2 2\n",
"3 3\n",
" A\n",
"3 1\n",
"1 2\n",
"2 3\n"
]
}
],
"source": [
"df1 = pd.DataFrame({'A':[1,2,3]}, index=[1,2,3])\n",
"df2 = pd.DataFrame({'A':[1,2,3]}, index=[3,1,2])\n",
"print(df1)\n",
"print(df2)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" A\n",
"1 -1\n",
"2 -1\n",
"3 2\n"
]
}
],
"source": [
"print(df1-df2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}