{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Pew Research data on religious affiliation\n",
"\n",
"Elements of Data Science\n",
"\n",
"by [Allen Downey](https://allendowney.com)\n",
"\n",
"[MIT License](https://opensource.org/licenses/MIT)\n",
"\n",
"Loading and cleaning data from Pew Research Center, Religion and Public Life.\n",
"\n",
"\"[In U.S., Decline of Christianity Continues at Rapid Pace](https://www.pewforum.org/2019/10/17/in-u-s-decline-of-christianity-continues-at-rapid-pace/): An update on America's changing religious landscape\", October 17, 2019.\n",
"\n",
"We can download the tables directly from Pew Research:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2019-10-24 15:21:35-- https://www.pewforum.org/wp-content/uploads/sites/7/2019/10/Detailed-Tables-v1-FOR-WEB.pdf\n",
"Resolving www.pewforum.org (www.pewforum.org)... 192.0.66.2\n",
"Connecting to www.pewforum.org (www.pewforum.org)|192.0.66.2|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 1069037 (1.0M) [application/pdf]\n",
"Saving to: ‘Detailed-Tables-v1-FOR-WEB.pdf’\n",
"\n",
"Detailed-Tables-v1- 100%[===================>] 1.02M --.-KB/s in 0.07s \n",
"\n",
"2019-10-24 15:21:36 (13.9 MB/s) - ‘Detailed-Tables-v1-FOR-WEB.pdf’ saved [1069037/1069037]\n",
"\n"
]
}
],
"source": [
"import os\n",
"\n",
"if not os.path.exists('Detailed-Tables-v1-FOR-WEB.pdf'):\n",
" !wget https://www.pewforum.org/wp-content/uploads/sites/7/2019/10/Detailed-Tables-v1-FOR-WEB.pdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The data are in a table in a PDF document, so I'll use [tabula-py](https://tabula-py.readthedocs.io/en/latest/) to read them."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"IN_COLAB = 'google.colab' in sys.modules\n",
"\n",
"if IN_COLAB:\n",
" !pip install tabula-py"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"from tabula import read_pdf\n",
"import pandas as pd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I'll read the table from page 1, specifying which column to use as the index and which columns to read. I'm using data from the Aggregated Political Surveys (columns 3-12) and skipping the Religious Landscape Studies (columns 1-2)."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" 2009 | \n",
" 2010 | \n",
" 2011 | \n",
" 2012 | \n",
" 2013 | \n",
" 2014.1 | \n",
" 2015 | \n",
" 2016 | \n",
" 2017 | \n",
" 2019 | \n",
"
\n",
" \n",
" \n",
" \n",
" Christian | \n",
" 77 | \n",
" 76 | \n",
" 75 | \n",
" 73 | \n",
" 73 | \n",
" 71.0 | \n",
" 69.0 | \n",
" 68.0 | \n",
" 67.0 | \n",
" 65 | \n",
"
\n",
" \n",
" Protestant | \n",
" 51 | \n",
" 51 | \n",
" 50 | \n",
" 49 | \n",
" 49 | \n",
" 48.0 | \n",
" 46.0 | \n",
" 44.0 | \n",
" 45.0 | \n",
" 43 | \n",
"
\n",
" \n",
" Catholic | \n",
" 23 | \n",
" 23 | \n",
" 23 | \n",
" 22 | \n",
" 22 | \n",
" 21.0 | \n",
" 21.0 | \n",
" 21.0 | \n",
" 20.0 | \n",
" 20 | \n",
"
\n",
" \n",
" Mormon | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2 | \n",
"
\n",
" \n",
" Orthodox Christian | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" <1 | \n",
"
\n",
" \n",
" Other Christian | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" Non-Christian faiths | \n",
" 5 | \n",
" 5 | \n",
" 5 | \n",
" 6 | \n",
" 5 | \n",
" 6.0 | \n",
" 6.0 | \n",
" 7.0 | \n",
" 6.0 | \n",
" 7 | \n",
"
\n",
" \n",
" Jewish | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2.0 | \n",
" 1.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2 | \n",
"
\n",
" \n",
" Muslim | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1 | \n",
"
\n",
" \n",
" Buddhist | \n",
" 1 | \n",
" <1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1 | \n",
"
\n",
" \n",
" Hindu | \n",
" <1 | \n",
" <1 | \n",
" <1 | \n",
" <1 | \n",
" <1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1 | \n",
"
\n",
" \n",
" Other | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 3.0 | \n",
" 2.0 | \n",
" 3 | \n",
"
\n",
" \n",
" Unaffiliated | \n",
" 17 | \n",
" 17 | \n",
" 19 | \n",
" 19 | \n",
" 20 | \n",
" 21.0 | \n",
" 24.0 | \n",
" 23.0 | \n",
" 25.0 | \n",
" 26 | \n",
"
\n",
" \n",
" Atheist | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 3 | \n",
" 3.0 | \n",
" 4.0 | \n",
" 4.0 | \n",
" 4.0 | \n",
" 4 | \n",
"
\n",
" \n",
" Agnostic | \n",
" 3 | \n",
" 2 | \n",
" 3 | \n",
" 3 | \n",
" 3 | \n",
" 3.0 | \n",
" 4.0 | \n",
" 4.0 | \n",
" 5.0 | \n",
" 5 | \n",
"
\n",
" \n",
" Nothing in particular | \n",
" 12 | \n",
" 12 | \n",
" 14 | \n",
" 14 | \n",
" 14 | \n",
" 14.0 | \n",
" 16.0 | \n",
" 16.0 | \n",
" 16.0 | \n",
" 17 | \n",
"
\n",
" \n",
" Don’t know/refused | \n",
" 2 | \n",
" 2 | \n",
" 1 | \n",
" 2 | \n",
" 2 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2 | \n",
"
\n",
" \n",
" NaN | \n",
" 100 | \n",
" 100 | \n",
" 100 | \n",
" 100 | \n",
" 100 | \n",
" 100.0 | \n",
" 100.0 | \n",
" 100.0 | \n",
" 100.0 | \n",
" 100 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" 2009 2010 2011 2012 2013 2014.1 2015 2016 2017 \\\n",
"Christian 77 76 75 73 73 71.0 69.0 68.0 67.0 \n",
"Protestant 51 51 50 49 49 48.0 46.0 44.0 45.0 \n",
"Catholic 23 23 23 22 22 21.0 21.0 21.0 20.0 \n",
"Mormon 2 2 2 2 2 2.0 2.0 2.0 2.0 \n",
"Orthodox Christian 1 1 1 1 1 1.0 1.0 1.0 1.0 \n",
"Other Christian NaN NaN NaN NaN NaN NaN NaN NaN NaN \n",
"Non-Christian faiths 5 5 5 6 5 6.0 6.0 7.0 6.0 \n",
"Jewish 2 2 2 2 2 2.0 1.0 2.0 2.0 \n",
"Muslim 1 1 1 1 1 1.0 1.0 1.0 1.0 \n",
"Buddhist 1 <1 1 1 1 1.0 1.0 1.0 1.0 \n",
"Hindu <1 <1 <1 <1 <1 1.0 1.0 1.0 1.0 \n",
"Other 2 2 2 2 2 2.0 2.0 3.0 2.0 \n",
"Unaffiliated 17 17 19 19 20 21.0 24.0 23.0 25.0 \n",
"Atheist 2 2 2 2 3 3.0 4.0 4.0 4.0 \n",
"Agnostic 3 2 3 3 3 3.0 4.0 4.0 5.0 \n",
"Nothing in particular 12 12 14 14 14 14.0 16.0 16.0 16.0 \n",
"Don’t know/refused 2 2 1 2 2 1.0 1.0 2.0 2.0 \n",
"NaN 100 100 100 100 100 100.0 100.0 100.0 100.0 \n",
"\n",
" 2019 \n",
"Christian 65 \n",
"Protestant 43 \n",
"Catholic 20 \n",
"Mormon 2 \n",
"Orthodox Christian <1 \n",
"Other Christian NaN \n",
"Non-Christian faiths 7 \n",
"Jewish 2 \n",
"Muslim 1 \n",
"Buddhist 1 \n",
"Hindu 1 \n",
"Other 3 \n",
"Unaffiliated 26 \n",
"Atheist 4 \n",
"Agnostic 5 \n",
"Nothing in particular 17 \n",
"Don’t know/refused 2 \n",
"NaN 100 "
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pages = 1\n",
"pandas_options = dict(index_col=0, usecols=[0,3,4,5,6,7,8,9,10,11,12])\n",
"df = read_pdf('Detailed-Tables-v1-FOR-WEB.pdf', pages=pages, pandas_options=pandas_options)\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Because there are two columns with the heading `2014`, `tabula-py` makes one of them `2014.1`. So I'll rename it."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" 2009 | \n",
" 2010 | \n",
" 2011 | \n",
" 2012 | \n",
" 2013 | \n",
" 2014 | \n",
" 2015 | \n",
" 2016 | \n",
" 2017 | \n",
" 2019 | \n",
"
\n",
" \n",
" \n",
" \n",
" Christian | \n",
" 77 | \n",
" 76 | \n",
" 75 | \n",
" 73 | \n",
" 73 | \n",
" 71.0 | \n",
" 69.0 | \n",
" 68.0 | \n",
" 67.0 | \n",
" 65 | \n",
"
\n",
" \n",
" Protestant | \n",
" 51 | \n",
" 51 | \n",
" 50 | \n",
" 49 | \n",
" 49 | \n",
" 48.0 | \n",
" 46.0 | \n",
" 44.0 | \n",
" 45.0 | \n",
" 43 | \n",
"
\n",
" \n",
" Catholic | \n",
" 23 | \n",
" 23 | \n",
" 23 | \n",
" 22 | \n",
" 22 | \n",
" 21.0 | \n",
" 21.0 | \n",
" 21.0 | \n",
" 20.0 | \n",
" 20 | \n",
"
\n",
" \n",
" Mormon | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2 | \n",
"
\n",
" \n",
" Orthodox Christian | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" <1 | \n",
"
\n",
" \n",
" Other Christian | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" Non-Christian faiths | \n",
" 5 | \n",
" 5 | \n",
" 5 | \n",
" 6 | \n",
" 5 | \n",
" 6.0 | \n",
" 6.0 | \n",
" 7.0 | \n",
" 6.0 | \n",
" 7 | \n",
"
\n",
" \n",
" Jewish | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2.0 | \n",
" 1.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2 | \n",
"
\n",
" \n",
" Muslim | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1 | \n",
"
\n",
" \n",
" Buddhist | \n",
" 1 | \n",
" <1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1 | \n",
"
\n",
" \n",
" Hindu | \n",
" <1 | \n",
" <1 | \n",
" <1 | \n",
" <1 | \n",
" <1 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 1 | \n",
"
\n",
" \n",
" Other | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 3.0 | \n",
" 2.0 | \n",
" 3 | \n",
"
\n",
" \n",
" Unaffiliated | \n",
" 17 | \n",
" 17 | \n",
" 19 | \n",
" 19 | \n",
" 20 | \n",
" 21.0 | \n",
" 24.0 | \n",
" 23.0 | \n",
" 25.0 | \n",
" 26 | \n",
"
\n",
" \n",
" Atheist | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 2 | \n",
" 3 | \n",
" 3.0 | \n",
" 4.0 | \n",
" 4.0 | \n",
" 4.0 | \n",
" 4 | \n",
"
\n",
" \n",
" Agnostic | \n",
" 3 | \n",
" 2 | \n",
" 3 | \n",
" 3 | \n",
" 3 | \n",
" 3.0 | \n",
" 4.0 | \n",
" 4.0 | \n",
" 5.0 | \n",
" 5 | \n",
"
\n",
" \n",
" Nothing in particular | \n",
" 12 | \n",
" 12 | \n",
" 14 | \n",
" 14 | \n",
" 14 | \n",
" 14.0 | \n",
" 16.0 | \n",
" 16.0 | \n",
" 16.0 | \n",
" 17 | \n",
"
\n",
" \n",
" Don’t know/refused | \n",
" 2 | \n",
" 2 | \n",
" 1 | \n",
" 2 | \n",
" 2 | \n",
" 1.0 | \n",
" 1.0 | \n",
" 2.0 | \n",
" 2.0 | \n",
" 2 | \n",
"
\n",
" \n",
" NaN | \n",
" 100 | \n",
" 100 | \n",
" 100 | \n",
" 100 | \n",
" 100 | \n",
" 100.0 | \n",
" 100.0 | \n",
" 100.0 | \n",
" 100.0 | \n",
" 100 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" 2009 2010 2011 2012 2013 2014 2015 2016 2017 \\\n",
"Christian 77 76 75 73 73 71.0 69.0 68.0 67.0 \n",
"Protestant 51 51 50 49 49 48.0 46.0 44.0 45.0 \n",
"Catholic 23 23 23 22 22 21.0 21.0 21.0 20.0 \n",
"Mormon 2 2 2 2 2 2.0 2.0 2.0 2.0 \n",
"Orthodox Christian 1 1 1 1 1 1.0 1.0 1.0 1.0 \n",
"Other Christian NaN NaN NaN NaN NaN NaN NaN NaN NaN \n",
"Non-Christian faiths 5 5 5 6 5 6.0 6.0 7.0 6.0 \n",
"Jewish 2 2 2 2 2 2.0 1.0 2.0 2.0 \n",
"Muslim 1 1 1 1 1 1.0 1.0 1.0 1.0 \n",
"Buddhist 1 <1 1 1 1 1.0 1.0 1.0 1.0 \n",
"Hindu <1 <1 <1 <1 <1 1.0 1.0 1.0 1.0 \n",
"Other 2 2 2 2 2 2.0 2.0 3.0 2.0 \n",
"Unaffiliated 17 17 19 19 20 21.0 24.0 23.0 25.0 \n",
"Atheist 2 2 2 2 3 3.0 4.0 4.0 4.0 \n",
"Agnostic 3 2 3 3 3 3.0 4.0 4.0 5.0 \n",
"Nothing in particular 12 12 14 14 14 14.0 16.0 16.0 16.0 \n",
"Don’t know/refused 2 2 1 2 2 1.0 1.0 2.0 2.0 \n",
"NaN 100 100 100 100 100 100.0 100.0 100.0 100.0 \n",
"\n",
" 2019 \n",
"Christian 65 \n",
"Protestant 43 \n",
"Catholic 20 \n",
"Mormon 2 \n",
"Orthodox Christian <1 \n",
"Other Christian NaN \n",
"Non-Christian faiths 7 \n",
"Jewish 2 \n",
"Muslim 1 \n",
"Buddhist 1 \n",
"Hindu 1 \n",
"Other 3 \n",
"Unaffiliated 26 \n",
"Atheist 4 \n",
"Agnostic 5 \n",
"Nothing in particular 17 \n",
"Don’t know/refused 2 \n",
"NaN 100 "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = df.rename(columns={'2014.1': '2014'})\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I transpose the DataFrame so the dates are the index."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Christian | \n",
" Protestant | \n",
" Catholic | \n",
" Mormon | \n",
" Orthodox Christian | \n",
" Other Christian | \n",
" Non-Christian faiths | \n",
" Jewish | \n",
" Muslim | \n",
" Buddhist | \n",
" Hindu | \n",
" Other | \n",
" Unaffiliated | \n",
" Atheist | \n",
" Agnostic | \n",
" Nothing in particular | \n",
" Don’t know/refused | \n",
" NaN | \n",
"
\n",
" \n",
" \n",
" \n",
" 2009 | \n",
" 77 | \n",
" 51 | \n",
" 23 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 17 | \n",
" 2 | \n",
" 3 | \n",
" 12 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2010 | \n",
" 76 | \n",
" 51 | \n",
" 23 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" <1 | \n",
" <1 | \n",
" 2 | \n",
" 17 | \n",
" 2 | \n",
" 2 | \n",
" 12 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2011 | \n",
" 75 | \n",
" 50 | \n",
" 23 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 19 | \n",
" 2 | \n",
" 3 | \n",
" 14 | \n",
" 1 | \n",
" 100 | \n",
"
\n",
" \n",
" 2012 | \n",
" 73 | \n",
" 49 | \n",
" 22 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 19 | \n",
" 2 | \n",
" 3 | \n",
" 14 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2013 | \n",
" 73 | \n",
" 49 | \n",
" 22 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 20 | \n",
" 3 | \n",
" 3 | \n",
" 14 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2014 | \n",
" 71 | \n",
" 48 | \n",
" 21 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 2 | \n",
" 21 | \n",
" 3 | \n",
" 3 | \n",
" 14 | \n",
" 1 | \n",
" 100 | \n",
"
\n",
" \n",
" 2015 | \n",
" 69 | \n",
" 46 | \n",
" 21 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 2 | \n",
" 24 | \n",
" 4 | \n",
" 4 | \n",
" 16 | \n",
" 1 | \n",
" 100 | \n",
"
\n",
" \n",
" 2016 | \n",
" 68 | \n",
" 44 | \n",
" 21 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 7 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 3 | \n",
" 23 | \n",
" 4 | \n",
" 4 | \n",
" 16 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2017 | \n",
" 67 | \n",
" 45 | \n",
" 20 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 2 | \n",
" 25 | \n",
" 4 | \n",
" 5 | \n",
" 16 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2019 | \n",
" 65 | \n",
" 43 | \n",
" 20 | \n",
" 2 | \n",
" <1 | \n",
" NaN | \n",
" 7 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 3 | \n",
" 26 | \n",
" 4 | \n",
" 5 | \n",
" 17 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" Christian Protestant Catholic Mormon Orthodox Christian Other Christian \\\n",
"2009 77 51 23 2 1 NaN \n",
"2010 76 51 23 2 1 NaN \n",
"2011 75 50 23 2 1 NaN \n",
"2012 73 49 22 2 1 NaN \n",
"2013 73 49 22 2 1 NaN \n",
"2014 71 48 21 2 1 NaN \n",
"2015 69 46 21 2 1 NaN \n",
"2016 68 44 21 2 1 NaN \n",
"2017 67 45 20 2 1 NaN \n",
"2019 65 43 20 2 <1 NaN \n",
"\n",
" Non-Christian faiths Jewish Muslim Buddhist Hindu Other Unaffiliated \\\n",
"2009 5 2 1 1 <1 2 17 \n",
"2010 5 2 1 <1 <1 2 17 \n",
"2011 5 2 1 1 <1 2 19 \n",
"2012 6 2 1 1 <1 2 19 \n",
"2013 5 2 1 1 <1 2 20 \n",
"2014 6 2 1 1 1 2 21 \n",
"2015 6 1 1 1 1 2 24 \n",
"2016 7 2 1 1 1 3 23 \n",
"2017 6 2 1 1 1 2 25 \n",
"2019 7 2 1 1 1 3 26 \n",
"\n",
" Atheist Agnostic Nothing in particular Don’t know/refused NaN \n",
"2009 2 3 12 2 100 \n",
"2010 2 2 12 2 100 \n",
"2011 2 3 14 1 100 \n",
"2012 2 3 14 2 100 \n",
"2013 3 3 14 2 100 \n",
"2014 3 3 14 1 100 \n",
"2015 4 4 16 1 100 \n",
"2016 4 4 16 2 100 \n",
"2017 4 5 16 2 100 \n",
"2019 4 5 17 2 100 "
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = df.transpose()\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now I can convert the index from string to int."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Christian | \n",
" Protestant | \n",
" Catholic | \n",
" Mormon | \n",
" Orthodox Christian | \n",
" Other Christian | \n",
" Non-Christian faiths | \n",
" Jewish | \n",
" Muslim | \n",
" Buddhist | \n",
" Hindu | \n",
" Other | \n",
" Unaffiliated | \n",
" Atheist | \n",
" Agnostic | \n",
" Nothing in particular | \n",
" Don’t know/refused | \n",
" NaN | \n",
"
\n",
" \n",
" \n",
" \n",
" 2009 | \n",
" 77 | \n",
" 51 | \n",
" 23 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 17 | \n",
" 2 | \n",
" 3 | \n",
" 12 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2010 | \n",
" 76 | \n",
" 51 | \n",
" 23 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" <1 | \n",
" <1 | \n",
" 2 | \n",
" 17 | \n",
" 2 | \n",
" 2 | \n",
" 12 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2011 | \n",
" 75 | \n",
" 50 | \n",
" 23 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 19 | \n",
" 2 | \n",
" 3 | \n",
" 14 | \n",
" 1 | \n",
" 100 | \n",
"
\n",
" \n",
" 2012 | \n",
" 73 | \n",
" 49 | \n",
" 22 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 19 | \n",
" 2 | \n",
" 3 | \n",
" 14 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2013 | \n",
" 73 | \n",
" 49 | \n",
" 22 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 5 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" <1 | \n",
" 2 | \n",
" 20 | \n",
" 3 | \n",
" 3 | \n",
" 14 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2014 | \n",
" 71 | \n",
" 48 | \n",
" 21 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 2 | \n",
" 21 | \n",
" 3 | \n",
" 3 | \n",
" 14 | \n",
" 1 | \n",
" 100 | \n",
"
\n",
" \n",
" 2015 | \n",
" 69 | \n",
" 46 | \n",
" 21 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 2 | \n",
" 24 | \n",
" 4 | \n",
" 4 | \n",
" 16 | \n",
" 1 | \n",
" 100 | \n",
"
\n",
" \n",
" 2016 | \n",
" 68 | \n",
" 44 | \n",
" 21 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 7 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 3 | \n",
" 23 | \n",
" 4 | \n",
" 4 | \n",
" 16 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2017 | \n",
" 67 | \n",
" 45 | \n",
" 20 | \n",
" 2 | \n",
" 1 | \n",
" NaN | \n",
" 6 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 2 | \n",
" 25 | \n",
" 4 | \n",
" 5 | \n",
" 16 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
" 2019 | \n",
" 65 | \n",
" 43 | \n",
" 20 | \n",
" 2 | \n",
" <1 | \n",
" NaN | \n",
" 7 | \n",
" 2 | \n",
" 1 | \n",
" 1 | \n",
" 1 | \n",
" 3 | \n",
" 26 | \n",
" 4 | \n",
" 5 | \n",
" 17 | \n",
" 2 | \n",
" 100 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" Christian Protestant Catholic Mormon Orthodox Christian Other Christian \\\n",
"2009 77 51 23 2 1 NaN \n",
"2010 76 51 23 2 1 NaN \n",
"2011 75 50 23 2 1 NaN \n",
"2012 73 49 22 2 1 NaN \n",
"2013 73 49 22 2 1 NaN \n",
"2014 71 48 21 2 1 NaN \n",
"2015 69 46 21 2 1 NaN \n",
"2016 68 44 21 2 1 NaN \n",
"2017 67 45 20 2 1 NaN \n",
"2019 65 43 20 2 <1 NaN \n",
"\n",
" Non-Christian faiths Jewish Muslim Buddhist Hindu Other Unaffiliated \\\n",
"2009 5 2 1 1 <1 2 17 \n",
"2010 5 2 1 <1 <1 2 17 \n",
"2011 5 2 1 1 <1 2 19 \n",
"2012 6 2 1 1 <1 2 19 \n",
"2013 5 2 1 1 <1 2 20 \n",
"2014 6 2 1 1 1 2 21 \n",
"2015 6 1 1 1 1 2 24 \n",
"2016 7 2 1 1 1 3 23 \n",
"2017 6 2 1 1 1 2 25 \n",
"2019 7 2 1 1 1 3 26 \n",
"\n",
" Atheist Agnostic Nothing in particular Don’t know/refused NaN \n",
"2009 2 3 12 2 100 \n",
"2010 2 2 12 2 100 \n",
"2011 2 3 14 1 100 \n",
"2012 2 3 14 2 100 \n",
"2013 3 3 14 2 100 \n",
"2014 3 3 14 1 100 \n",
"2015 4 4 16 1 100 \n",
"2016 4 4 16 2 100 \n",
"2017 4 5 16 2 100 \n",
"2019 4 5 17 2 100 "
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df.index = df.index.astype(int)\n",
"df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And write the result to a file."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"df.to_csv('pew_religion_table1.csv')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}