{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 14.2 MovieLens 1M Dataset（MovieLens 1M数据集）\n", "\n", "这个数据集是电影评分数据：包括电影评分，电影元数据（风格类型，年代）以及关于用户的人口统计学数据（年龄，邮编，性别，职业等）。\n", "\n", "MovieLens 1M数据集含有来自6000名用户对4000部电影的100万条评分数据。分为三个表：评分，用户信息，电影信息。这些数据都是dat文件格式，可以通过pandas.read_table将各个表分别读到一个pandas DataFrame对象中：" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Make display smaller\n", "pd.options.display.max_rows = 10" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/xu/anaconda/envs/py35/lib/python3.5/site-packages/ipykernel/__main__.py:3: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.\n", " app.launch_new_instance()\n" ] } ], "source": [ "unames = ['user_id', 'gender', 'age', 'occupation', 'zip']\n", "users = pd.read_table('../datasets/movielens/users.dat', sep='::', \n", " header=None, names=unames)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "因为sep='::'有点像是正则表达式，于是有了上面的错误。在这个[帖子](https://stackoverflow.com/questions/27301477/python-file-path-failing-in-pycharm-regex-confusion)找到了解决方法，设置engine为python即可。\n", "\n", "Looks like on Python 2.7 Pandas just doesn't handle separators that look regexish. The initial \"error\" can be worked around by adding engine='python' as a named parameter in the call, as suggested in the warning.\n", "\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "users = pd.read_table('../datasets/movielens/users.dat', sep='::', \n", " header=None, names=unames, engine='python')" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [], "source": [ "rnames = ['user_id', 'movie_id', 'rating', 'timestamp']\n", "ratings = pd.read_table('../datasets/movielens/ratings.dat', sep='::', header=None, names=rnames, engine='python')" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "mnames = ['movie_id', 'title', 'genres']\n", "movies = pd.read_table('../datasets/movielens/movies.dat', sep='::', header=None, names=mnames, engine='python')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "加载前几行验证一下数据加载工作是否顺利" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
user_idgenderageoccupationzip
01F11048067
12M561670072
23M251555117
34M45702460
45M252055455
\n", "
" ], "text/plain": [ " user_id gender age occupation zip\n", "0 1 F 1 10 48067\n", "1 2 M 56 16 70072\n", "2 3 M 25 15 55117\n", "3 4 M 45 7 02460\n", "4 5 M 25 20 55455" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "users[:5]" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
user_idmovie_idratingtimestamp
0111935978300760
116613978302109
219143978301968
3134084978300275
4123555978824291
\n", "
" ], "text/plain": [ " user_id movie_id rating timestamp\n", "0 1 1193 5 978300760\n", "1 1 661 3 978302109\n", "2 1 914 3 978301968\n", "3 1 3408 4 978300275\n", "4 1 2355 5 978824291" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ratings[:5]" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
movie_idtitlegenres
01Toy Story (1995)Animation|Children's|Comedy
12Jumanji (1995)Adventure|Children's|Fantasy
23Grumpier Old Men (1995)Comedy|Romance
34Waiting to Exhale (1995)Comedy|Drama
45Father of the Bride Part II (1995)Comedy
\n", "
" ], "text/plain": [ " movie_id title genres\n", "0 1 Toy Story (1995) Animation|Children's|Comedy\n", "1 2 Jumanji (1995) Adventure|Children's|Fantasy\n", "2 3 Grumpier Old Men (1995) Comedy|Romance\n", "3 4 Waiting to Exhale (1995) Comedy|Drama\n", "4 5 Father of the Bride Part II (1995) Comedy" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "movies[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "注意，年龄和职业是以编码形式给出的，它们的具体含义请参考改数据集的REAMDE文件。分析散布在三个表中的数据不是一件轻松的事情。假设我们想要根据性别和年龄来计算某部电影的平均得分，如果将所有的数据都合并到一个表中的话，问题就简单多了。我们先用pandas的merge函数将ratings和users合并到一起，然后再将movies也合并进去。pandas会根据列名的重叠情况推断出哪些列是合并（或连接）键：" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [], "source": [ "data = pd.merge(pd.merge(ratings, users), movies)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
user_idmovie_idratingtimestampgenderageoccupationziptitlegenres
0111935978300760F11048067One Flew Over the Cuckoo's Nest (1975)Drama
1211935978298413M561670072One Flew Over the Cuckoo's Nest (1975)Drama
21211934978220179M251232793One Flew Over the Cuckoo's Nest (1975)Drama
31511934978199279M25722903One Flew Over the Cuckoo's Nest (1975)Drama
41711935978158471M50195350One Flew Over the Cuckoo's Nest (1975)Drama
\n", "
" ], "text/plain": [ " user_id movie_id rating timestamp gender age occupation zip \\\n", "0 1 1193 5 978300760 F 1 10 48067 \n", "1 2 1193 5 978298413 M 56 16 70072 \n", "2 12 1193 4 978220179 M 25 12 32793 \n", "3 15 1193 4 978199279 M 25 7 22903 \n", "4 17 1193 5 978158471 M 50 1 95350 \n", "\n", " title genres \n", "0 One Flew Over the Cuckoo's Nest (1975) Drama \n", "1 One Flew Over the Cuckoo's Nest (1975) Drama \n", "2 One Flew Over the Cuckoo's Nest (1975) Drama \n", "3 One Flew Over the Cuckoo's Nest (1975) Drama \n", "4 One Flew Over the Cuckoo's Nest (1975) Drama " ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.head()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "user_id 1\n", "movie_id 1193\n", "rating 5\n", "timestamp 978300760\n", "gender F\n", "age 1\n", "occupation 10\n", "zip 48067\n", "title One Flew Over the Cuckoo's Nest (1975)\n", "genres Drama\n", "Name: 0, dtype: object" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.iloc[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "现在，只要稍微熟悉一下pandas，就能轻松地根据任意个用户或电影属性对评分数据进行聚合操作了。为了按性别计算每部电影的平均得分，我们可以使用pivot_table方法：" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [], "source": [ "mean_ratings = data.pivot_table('rating', index='title',\n", " columns='gender', aggfunc='mean')" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
genderFM
title
$1,000,000 Duck (1971)3.3750002.761905
'Night Mother (1986)3.3888893.352941
'Til There Was You (1997)2.6756762.733333
'burbs, The (1989)2.7934782.962085
...And Justice for All (1979)3.8285713.689024
\n", "
" ], "text/plain": [ "gender F M\n", "title \n", "$1,000,000 Duck (1971) 3.375000 2.761905\n", "'Night Mother (1986) 3.388889 3.352941\n", "'Til There Was You (1997) 2.675676 2.733333\n", "'burbs, The (1989) 2.793478 2.962085\n", "...And Justice for All (1979) 3.828571 3.689024" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean_ratings[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "该操作产生了另一个DataFrame，其内容为电影平均得分，行标为电影名称，列表为性别。现在，我们打算过滤掉评分数据不够250条的电影（这个数字可以自己设定）。为了达到这个目的，我们先对title进行分组，然后利用size()得到一个含有各电影分组大小的Series对象：" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ratings_by_title = data.groupby('title').size()" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "title\n", "$1,000,000 Duck (1971) 37\n", "'Night Mother (1986) 70\n", "'Til There Was You (1997) 52\n", "'burbs, The (1989) 303\n", "...And Justice for All (1979) 199\n", "1-900 (1994) 2\n", "10 Things I Hate About You (1999) 700\n", "101 Dalmatians (1961) 565\n", "101 Dalmatians (1996) 364\n", "12 Angry Men (1957) 616\n", "dtype: int64" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ratings_by_title[:10]" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": true }, "outputs": [], "source": [ "active_titles = ratings_by_title.index[ratings_by_title >= 250]" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Index([''burbs, The (1989)', '10 Things I Hate About You (1999)',\n", " '101 Dalmatians (1961)', '101 Dalmatians (1996)', '12 Angry Men (1957)',\n", " '13th Warrior, The (1999)', '2 Days in the Valley (1996)',\n", " '20,000 Leagues Under the Sea (1954)', '2001: A Space Odyssey (1968)',\n", " '2010 (1984)',\n", " ...\n", " 'X-Men (2000)', 'Year of Living Dangerously (1982)',\n", " 'Yellow Submarine (1968)', 'You've Got Mail (1998)',\n", " 'Young Frankenstein (1974)', 'Young Guns (1988)',\n", " 'Young Guns II (1990)', 'Young Sherlock Holmes (1985)',\n", " 'Zero Effect (1998)', 'eXistenZ (1999)'],\n", " dtype='object', name='title', length=1216)\n" ] } ], "source": [ "print(active_titles)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "上面的active_titles中的电影，都是评论是大于250条以上的。我们可以用这些标题作为索引，从mean_ratings中选出这些评论大于250条的电影：" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
genderFM
title
'burbs, The (1989)2.7934782.962085
10 Things I Hate About You (1999)3.6465523.311966
101 Dalmatians (1961)3.7914443.500000
101 Dalmatians (1996)3.2400002.911215
12 Angry Men (1957)4.1843974.328421
.........
Young Guns (1988)3.3717953.425620
Young Guns II (1990)2.9347832.904025
Young Sherlock Holmes (1985)3.5147063.363344
Zero Effect (1998)3.8644073.723140
eXistenZ (1999)3.0985923.289086
\n", "

1216 rows × 2 columns

\n", "
" ], "text/plain": [ "gender F M\n", "title \n", "'burbs, The (1989) 2.793478 2.962085\n", "10 Things I Hate About You (1999) 3.646552 3.311966\n", "101 Dalmatians (1961) 3.791444 3.500000\n", "101 Dalmatians (1996) 3.240000 2.911215\n", "12 Angry Men (1957) 4.184397 4.328421\n", "... ... ...\n", "Young Guns (1988) 3.371795 3.425620\n", "Young Guns II (1990) 2.934783 2.904025\n", "Young Sherlock Holmes (1985) 3.514706 3.363344\n", "Zero Effect (1998) 3.864407 3.723140\n", "eXistenZ (1999) 3.098592 3.289086\n", "\n", "[1216 rows x 2 columns]" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean_ratings = mean_ratings.loc[active_titles]\n", "mean_ratings" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "想要查看女性观众喜欢的电影，可以按F列进行降序操作：" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
genderFM
title
Close Shave, A (1995)4.6444444.473795
Wrong Trousers, The (1993)4.5882354.478261
Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)4.5726504.464589
Wallace & Gromit: The Best of Aardman Animation (1996)4.5631074.385075
Schindler's List (1993)4.5626024.491415
Shawshank Redemption, The (1994)4.5390754.560625
Grand Day Out, A (1992)4.5378794.293255
To Kill a Mockingbird (1962)4.5366674.372611
Creature Comforts (1990)4.5138894.272277
Usual Suspects, The (1995)4.5133174.518248
\n", "
" ], "text/plain": [ "gender F M\n", "title \n", "Close Shave, A (1995) 4.644444 4.473795\n", "Wrong Trousers, The (1993) 4.588235 4.478261\n", "Sunset Blvd. (a.k.a. Sunset Boulevard) (1950) 4.572650 4.464589\n", "Wallace & Gromit: The Best of Aardman Animation... 4.563107 4.385075\n", "Schindler's List (1993) 4.562602 4.491415\n", "Shawshank Redemption, The (1994) 4.539075 4.560625\n", "Grand Day Out, A (1992) 4.537879 4.293255\n", "To Kill a Mockingbird (1962) 4.536667 4.372611\n", "Creature Comforts (1990) 4.513889 4.272277\n", "Usual Suspects, The (1995) 4.513317 4.518248" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "top_female_ratings = mean_ratings.sort_values(by='F', ascending=False)\n", "top_female_ratings[:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 1 Measuring Rating Disagreement（计算评分分歧）\n", "\n", "假设我们想要找出男性和女性观众分歧最大的电影。一个办法是给mean_ratings加上一个用于存放平均得分之差的列，并对其进行排序：" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": true }, "outputs": [], "source": [ "mean_ratings['diff'] = mean_ratings['M'] - mean_ratings['F']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "按‘diff’排序即可得到分歧最大且女性观众更喜欢的电影：" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
genderFMdiff
title
Dirty Dancing (1987)3.7903782.959596-0.830782
Jumpin' Jack Flash (1986)3.2547172.578358-0.676359
Grease (1978)3.9752653.367041-0.608224
Little Women (1994)3.8705883.321739-0.548849
Steel Magnolias (1989)3.9017343.365957-0.535777
............
French Kiss (1995)3.5357143.056962-0.478752
Little Shop of Horrors, The (1960)3.6500003.179688-0.470312
Guys and Dolls (1955)4.0517243.583333-0.468391
Mary Poppins (1964)4.1977403.730594-0.467147
Patch Adams (1998)3.4732823.008746-0.464536
\n", "

15 rows × 3 columns

\n", "
" ], "text/plain": [ "gender F M diff\n", "title \n", "Dirty Dancing (1987) 3.790378 2.959596 -0.830782\n", "Jumpin' Jack Flash (1986) 3.254717 2.578358 -0.676359\n", "Grease (1978) 3.975265 3.367041 -0.608224\n", "Little Women (1994) 3.870588 3.321739 -0.548849\n", "Steel Magnolias (1989) 3.901734 3.365957 -0.535777\n", "... ... ... ...\n", "French Kiss (1995) 3.535714 3.056962 -0.478752\n", "Little Shop of Horrors, The (1960) 3.650000 3.179688 -0.470312\n", "Guys and Dolls (1955) 4.051724 3.583333 -0.468391\n", "Mary Poppins (1964) 4.197740 3.730594 -0.467147\n", "Patch Adams (1998) 3.473282 3.008746 -0.464536\n", "\n", "[15 rows x 3 columns]" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted_by_diff = mean_ratings.sort_values(by='diff')\n", "sorted_by_diff[:15]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "对行进行反序操作，并取出前15行，得到的则是男性更喜欢，而女性评价较低的电影：" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
genderFMdiff
title
Good, The Bad and The Ugly, The (1966)3.4949494.2213000.726351
Kentucky Fried Movie, The (1977)2.8787883.5551470.676359
Dumb & Dumber (1994)2.6979873.3365950.638608
Longest Day, The (1962)3.4117654.0314470.619682
Cable Guy, The (1996)2.2500002.8637870.613787
Evil Dead II (Dead By Dawn) (1987)3.2972973.9092830.611985
Hidden, The (1987)3.1379313.7450980.607167
Rocky III (1982)2.3617022.9435030.581801
Caddyshack (1980)3.3961353.9697370.573602
For a Few Dollars More (1965)3.4090913.9537950.544704
\n", "
" ], "text/plain": [ "gender F M diff\n", "title \n", "Good, The Bad and The Ugly, The (1966) 3.494949 4.221300 0.726351\n", "Kentucky Fried Movie, The (1977) 2.878788 3.555147 0.676359\n", "Dumb & Dumber (1994) 2.697987 3.336595 0.638608\n", "Longest Day, The (1962) 3.411765 4.031447 0.619682\n", "Cable Guy, The (1996) 2.250000 2.863787 0.613787\n", "Evil Dead II (Dead By Dawn) (1987) 3.297297 3.909283 0.611985\n", "Hidden, The (1987) 3.137931 3.745098 0.607167\n", "Rocky III (1982) 2.361702 2.943503 0.581801\n", "Caddyshack (1980) 3.396135 3.969737 0.573602\n", "For a Few Dollars More (1965) 3.409091 3.953795 0.544704" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Reverse order of rows, take first 10 rows\n", "sorted_by_diff[::-1][:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "如果只是想要找出分歧最大的电影（不考虑性别因素），则可以计算得分数据的方差或标准差：" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# 根据电影名称分组的得分数据的标准差\n", "rating_std_by_title = data.groupby('title')['rating'].std()" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 根据active_titles进行过滤\n", "rating_std_by_title = rating_std_by_title.loc[active_titles]" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "title\n", "Dumb & Dumber (1994) 1.321333\n", "Blair Witch Project, The (1999) 1.316368\n", "Natural Born Killers (1994) 1.307198\n", "Tank Girl (1995) 1.277695\n", "Rocky Horror Picture Show, The (1975) 1.260177\n", "Eyes Wide Shut (1999) 1.259624\n", "Evita (1996) 1.253631\n", "Billy Madison (1995) 1.249970\n", "Fear and Loathing in Las Vegas (1998) 1.246408\n", "Bicentennial Man (1999) 1.245533\n", "Name: rating, dtype: float64" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Order Series by value in descending order\n", "rating_std_by_title.sort_values(ascending=False)[:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "这里我们注意到，电影分类是以竖线`|`分割的字符串形式给出的。如果想对不同的电影分类进行分析的话，就需要先将其转换成更有用的形式才行。" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [py35]", "language": "python", "name": "Python [py35]" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 0 }