{ "cells": [ { "cell_type": "markdown", "id": "skilled-leather", "metadata": {}, "source": [ "## Step 1: A Framework for Unlocking and Linking WWII Japanese American Incarceration Biographical Data - Context Based Data Manipuation and Analysis Process- Part 1 \n", "\n", "Data manipuation is the process of fine tuning data to organize and prepare the data for research purposes. Data manipulation can include sorting, filtering, cleaning, normalizing, and joining separate datasets. Once the data has been prepared, data analysis can be perfomed. According to the Office of Research Integrity, \"Data Analysis is the process of systematically applying statistical and/or logical techniques to describe and illustrate, condense and recap, and evaluate data.\" Below are the steps performed for data manipulation and analysis process using Python programming language on the Japanese American World War II datasets. The modules are split into two parts - Part 1 and Part 2." ] }, { "cell_type": "markdown", "id": "duplicate-formula", "metadata": {}, "source": [ "### Acquiring or Accessing the Data\n", "\n", "The data for this project comes from three disparate collections housed at the National Archives and Records Administration (NARA) this includes the War Relocation Authority (WRA), Final Accountability Roster (FAR), and Internal Security Case Reports, also referred to as Index Card or Incident Cards. The data sources are included in this module for the WRA and FAR as comma separated values (csv) files. The link below will direct you to those data files: \n", "\n", "WRA [link]\n", "FAR [link]\n", "Incident Card upon request\n", "\n", "The WRA dataset has X rows of data, the FAR has X rows, and the Incident Card has X rows.\n", "\n", "To work with a csv file in Python, one of the first steps is to import a Python library called as 'pandas' which will help the program convert the csv file into a dataframe format or commonly called as a table format. We import the library into the program as shown below:" ] }, { "cell_type": "code", "execution_count": 1, "id": "spiritual-importance", "metadata": {}, "outputs": [], "source": [ "# Import libraries\n", "# Pandas is used for data science/data analysis and machine learning tasks \n", "import pandas as pd\n", "# Numpy provides support for multi-dimensional arrays\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "painted-auckland", "metadata": {}, "source": [ "Using the pandas library, new dataframes can be created from the datasets imported into the notebook. Once the dataframes have been created from the imported files, the print() function is used to display the top first 5 rows in the dataframe." ] }, { "cell_type": "code", "execution_count": 2, "id": "parliamentary-powell", "metadata": {}, "outputs": [ { "ename": "FileNotFoundError", "evalue": "[Errno 2] No such file or directory: 'far_cleanup-09-26-2021.csv'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn [2], line 5\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[39m# Creating a dataframe is essentially creating a table-like data structure that can read csv files, flat files, and other delimited data.\u001b[39;00m\n\u001b[1;32m 2\u001b[0m \u001b[39m# Converting input data into a dataframe is the starting point with Python programming language for big data analytics\u001b[39;00m\n\u001b[1;32m 3\u001b[0m \u001b[39m# Below command reads in the WRA, FAR, and Incident card datasets. The WRA and FAR should already be loaded in a folder called 'Datasets' wra_cleanup-09-26-2021.csv and Copy-far_cleanup-09-26-2021.csv\u001b[39;00m\n\u001b[1;32m 4\u001b[0m wradf \u001b[39m=\u001b[39m pd\u001b[39m.\u001b[39mread_csv(\u001b[39m'\u001b[39m\u001b[39mwra_cleanup-09-26-2021.csv\u001b[39m\u001b[39m'\u001b[39m,dtype\u001b[39m=\u001b[39m\u001b[39mobject\u001b[39m,na_values\u001b[39m=\u001b[39m[],keep_default_na\u001b[39m=\u001b[39m\u001b[39mFalse\u001b[39;00m)\n\u001b[0;32m----> 5\u001b[0m fardf \u001b[39m=\u001b[39m pd\u001b[39m.\u001b[39;49mread_csv(\u001b[39m'\u001b[39;49m\u001b[39mfar_cleanup-09-26-2021.csv\u001b[39;49m\u001b[39m'\u001b[39;49m,dtype\u001b[39m=\u001b[39;49m\u001b[39mobject\u001b[39;49m,na_values\u001b[39m=\u001b[39;49m[],keep_default_na\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m)\n\u001b[1;32m 6\u001b[0m incidentdf \u001b[39m=\u001b[39m pd\u001b[39m.\u001b[39mread_csv(\u001b[39m'\u001b[39m\u001b[39mincidentdfv1.csv\u001b[39m\u001b[39m'\u001b[39m,dtype\u001b[39m=\u001b[39m\u001b[39mobject\u001b[39m,na_values\u001b[39m=\u001b[39m[],keep_default_na\u001b[39m=\u001b[39m\u001b[39mFalse\u001b[39;00m)\n\u001b[1;32m 8\u001b[0m \u001b[39m# Below command prints the first 5 records of the Incident Cards after the data is copied from the csv file\u001b[39;00m\n", "File \u001b[0;32m~/.local/lib/python3.10/site-packages/pandas/io/parsers/readers.py:948\u001b[0m, in \u001b[0;36mread_csv\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, date_format, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options, dtype_backend)\u001b[0m\n\u001b[1;32m 935\u001b[0m kwds_defaults \u001b[39m=\u001b[39m _refine_defaults_read(\n\u001b[1;32m 936\u001b[0m dialect,\n\u001b[1;32m 937\u001b[0m delimiter,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 944\u001b[0m dtype_backend\u001b[39m=\u001b[39mdtype_backend,\n\u001b[1;32m 945\u001b[0m )\n\u001b[1;32m 946\u001b[0m kwds\u001b[39m.\u001b[39mupdate(kwds_defaults)\n\u001b[0;32m--> 948\u001b[0m \u001b[39mreturn\u001b[39;00m _read(filepath_or_buffer, kwds)\n", "File \u001b[0;32m~/.local/lib/python3.10/site-packages/pandas/io/parsers/readers.py:611\u001b[0m, in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m 608\u001b[0m _validate_names(kwds\u001b[39m.\u001b[39mget(\u001b[39m\"\u001b[39m\u001b[39mnames\u001b[39m\u001b[39m\"\u001b[39m, \u001b[39mNone\u001b[39;00m))\n\u001b[1;32m 610\u001b[0m \u001b[39m# Create the parser.\u001b[39;00m\n\u001b[0;32m--> 611\u001b[0m parser \u001b[39m=\u001b[39m TextFileReader(filepath_or_buffer, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwds)\n\u001b[1;32m 613\u001b[0m \u001b[39mif\u001b[39;00m chunksize \u001b[39mor\u001b[39;00m iterator:\n\u001b[1;32m 614\u001b[0m \u001b[39mreturn\u001b[39;00m parser\n", "File \u001b[0;32m~/.local/lib/python3.10/site-packages/pandas/io/parsers/readers.py:1448\u001b[0m, in \u001b[0;36mTextFileReader.__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m 1445\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39moptions[\u001b[39m\"\u001b[39m\u001b[39mhas_index_names\u001b[39m\u001b[39m\"\u001b[39m] \u001b[39m=\u001b[39m kwds[\u001b[39m\"\u001b[39m\u001b[39mhas_index_names\u001b[39m\u001b[39m\"\u001b[39m]\n\u001b[1;32m 1447\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mhandles: IOHandles \u001b[39m|\u001b[39m \u001b[39mNone\u001b[39;00m \u001b[39m=\u001b[39m \u001b[39mNone\u001b[39;00m\n\u001b[0;32m-> 1448\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_engine \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_make_engine(f, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mengine)\n", "File \u001b[0;32m~/.local/lib/python3.10/site-packages/pandas/io/parsers/readers.py:1705\u001b[0m, in \u001b[0;36mTextFileReader._make_engine\u001b[0;34m(self, f, engine)\u001b[0m\n\u001b[1;32m 1703\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mb\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mnot\u001b[39;00m \u001b[39min\u001b[39;00m mode:\n\u001b[1;32m 1704\u001b[0m mode \u001b[39m+\u001b[39m\u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39mb\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[0;32m-> 1705\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mhandles \u001b[39m=\u001b[39m get_handle(\n\u001b[1;32m 1706\u001b[0m f,\n\u001b[1;32m 1707\u001b[0m mode,\n\u001b[1;32m 1708\u001b[0m encoding\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49moptions\u001b[39m.\u001b[39;49mget(\u001b[39m\"\u001b[39;49m\u001b[39mencoding\u001b[39;49m\u001b[39m\"\u001b[39;49m, \u001b[39mNone\u001b[39;49;00m),\n\u001b[1;32m 1709\u001b[0m compression\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49moptions\u001b[39m.\u001b[39;49mget(\u001b[39m\"\u001b[39;49m\u001b[39mcompression\u001b[39;49m\u001b[39m\"\u001b[39;49m, \u001b[39mNone\u001b[39;49;00m),\n\u001b[1;32m 1710\u001b[0m memory_map\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49moptions\u001b[39m.\u001b[39;49mget(\u001b[39m\"\u001b[39;49m\u001b[39mmemory_map\u001b[39;49m\u001b[39m\"\u001b[39;49m, \u001b[39mFalse\u001b[39;49;00m),\n\u001b[1;32m 1711\u001b[0m is_text\u001b[39m=\u001b[39;49mis_text,\n\u001b[1;32m 1712\u001b[0m errors\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49moptions\u001b[39m.\u001b[39;49mget(\u001b[39m\"\u001b[39;49m\u001b[39mencoding_errors\u001b[39;49m\u001b[39m\"\u001b[39;49m, \u001b[39m\"\u001b[39;49m\u001b[39mstrict\u001b[39;49m\u001b[39m\"\u001b[39;49m),\n\u001b[1;32m 1713\u001b[0m storage_options\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49moptions\u001b[39m.\u001b[39;49mget(\u001b[39m\"\u001b[39;49m\u001b[39mstorage_options\u001b[39;49m\u001b[39m\"\u001b[39;49m, \u001b[39mNone\u001b[39;49;00m),\n\u001b[1;32m 1714\u001b[0m )\n\u001b[1;32m 1715\u001b[0m \u001b[39massert\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mhandles \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m\n\u001b[1;32m 1716\u001b[0m f \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mhandles\u001b[39m.\u001b[39mhandle\n", "File \u001b[0;32m~/.local/lib/python3.10/site-packages/pandas/io/common.py:863\u001b[0m, in \u001b[0;36mget_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001b[0m\n\u001b[1;32m 858\u001b[0m \u001b[39melif\u001b[39;00m \u001b[39misinstance\u001b[39m(handle, \u001b[39mstr\u001b[39m):\n\u001b[1;32m 859\u001b[0m \u001b[39m# Check whether the filename is to be opened in binary mode.\u001b[39;00m\n\u001b[1;32m 860\u001b[0m \u001b[39m# Binary mode does not support 'encoding' and 'newline'.\u001b[39;00m\n\u001b[1;32m 861\u001b[0m \u001b[39mif\u001b[39;00m ioargs\u001b[39m.\u001b[39mencoding \u001b[39mand\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mb\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mnot\u001b[39;00m \u001b[39min\u001b[39;00m ioargs\u001b[39m.\u001b[39mmode:\n\u001b[1;32m 862\u001b[0m \u001b[39m# Encoding\u001b[39;00m\n\u001b[0;32m--> 863\u001b[0m handle \u001b[39m=\u001b[39m \u001b[39mopen\u001b[39;49m(\n\u001b[1;32m 864\u001b[0m handle,\n\u001b[1;32m 865\u001b[0m ioargs\u001b[39m.\u001b[39;49mmode,\n\u001b[1;32m 866\u001b[0m encoding\u001b[39m=\u001b[39;49mioargs\u001b[39m.\u001b[39;49mencoding,\n\u001b[1;32m 867\u001b[0m errors\u001b[39m=\u001b[39;49merrors,\n\u001b[1;32m 868\u001b[0m newline\u001b[39m=\u001b[39;49m\u001b[39m\"\u001b[39;49m\u001b[39m\"\u001b[39;49m,\n\u001b[1;32m 869\u001b[0m )\n\u001b[1;32m 870\u001b[0m \u001b[39melse\u001b[39;00m:\n\u001b[1;32m 871\u001b[0m \u001b[39m# Binary mode\u001b[39;00m\n\u001b[1;32m 872\u001b[0m handle \u001b[39m=\u001b[39m \u001b[39mopen\u001b[39m(handle, ioargs\u001b[39m.\u001b[39mmode)\n", "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'far_cleanup-09-26-2021.csv'" ] } ], "source": [ "# Creating a dataframe is essentially creating a table-like data structure that can read csv files, flat files, and other delimited data.\n", "# Converting input data into a dataframe is the starting point with Python programming language for big data analytics\n", "# Below command reads in the WRA, FAR, and Incident card datasets. The WRA and FAR should already be loaded in a folder called 'Datasets' wra_cleanup-09-26-2021.csv and Copy-far_cleanup-09-26-2021.csv\n", "wradf = pd.read_csv('wra_cleanup-09-26-2021.csv',dtype=object,na_values=[],keep_default_na=False)\n", "fardf = pd.read_csv('far_cleanup-09-26-2021.csv',dtype=object,na_values=[],keep_default_na=False)\n", "incidentdf = pd.read_csv('incidentdfv1.csv',dtype=object,na_values=[],keep_default_na=False)\n", "\n", "# Below command prints the first 5 records of the Incident Cards after the data is copied from the csv file\n", "incidentdf.head(5)" ] }, { "cell_type": "markdown", "id": "metric-helen", "metadata": {}, "source": [ "Below are the variables in the Incident Cards that were chosen to be cleaned and manipulated for use in the following steps.\n", "\n", "Offense -- This indicates the offenses assigned to Japanese Americans by government officials\n", "Case # -- A case number was assigned according to the type of offense\n", "Date -- This indicates the date the offense took place \n", "Residence -- The location of where that specific individual resided \n", "Other -- This indicates additional notes that were recorded by officials on the Index, \"Incident Cards\"\n", "First Name -- The first name of the incarcerated Japanese American\n", "Last Name -- The last name of the incarcerated Japanese American\n", "\n", "\n", "Below are the variables in the WRA and FAR that were chosen to be cleaned and manipulated for use in the following steps.\n", "First Name -- The first name of the incarcerated Japanese American\n", "Last Name -- The last name of the incarcerated Japanese American\n", "Family Number -- A number assigned to families upon entry into the assembly centers\n" ] }, { "cell_type": "markdown", "id": "cubic-scope", "metadata": {}, "source": [ "We anticipated errors and misinterpretation of names, numbers, etc. in the data since these databases were either transcribed manually or physical copies digitized using OCR techniques. We approached this by individually exploring and cleaning the selected columns using the text and numerical operation functions in Python. To begin, we looked at the datasets holistically to identify variables that would allow for us to generate meaningful stories and visualizations. After a list of variables were confirmed, we analyzed each of them in detail to document bad data and eliminate them if possible, modify data types, exclude them from the final visualizations if found to be invalid, etc.\n", "\n", "This project involved team members with diverse backgrounds and knowledge in the technical, historical, and archival domains. The project presented opportunties for collaboration and individual work. We decided it was more productive to have a hybrid approach where we analyzed the data on our own and created sub-projects and reported our results and progress back to the group for discussion. With respect to the analysis performed on the dataset, decisions were data-driven or historical facts driven.\n", "\n", "Through researching the literature, conversations with community partner Geoff Froh, from Densho, and experts in the field, the team members were able to follow steps where specific variables were identified and shared among the entire group for their inputs before finalizing the results. " ] }, { "cell_type": "markdown", "id": "pediatric-retail", "metadata": {}, "source": [ "## Case Number (Issue of Incident Card)\n", "Through many conversations and pulling in our differing expertise we were able to develop ideas that let us integrate a range of methods and techniques to exhibit multifaceted perspectives of incarcerated Japanese Americans during World War II. Out of these discussions one of the many sub projects that developed was Spatial Analysis of incarcerated Japanese Americans recorded as being invovled in the uprising ('riot') at Tule Lake on November 4, 1943. \n", "\n", "Additionally, much of our discussion was also centered on how the data should be analyzed, presented, and shared without impacting the sensitivity of the people involved, especially since this set of collection is unique.\n", "\n", "One of the variables conisdered for this project was case number. This field indicates the case number assigned to offenses that occured at Tule Lake. Due to the case number being entered on the index cards by different government personnel the formatting was inconsistent. There were a number of issues with the case number field in the original dataset. Examples of the variations of case number formats includes 'A7-'; ‘A-7 ’; A-7- also have 'A-7-P' also have 'A-7-p'; 'A-7p' Also have 'A-7P'; ‘A-7,’; 'A7,'. " ] }, { "cell_type": "code", "execution_count": null, "id": "dated-acoustic", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "count 15748\n", "unique 4475\n", "top \n", "freq 680\n", "Name: casenumber, dtype: object" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The below command prints out the descriptive details of the column 'casenumber'\n", "incidentdf[\"casenumber\"].describe()" ] }, { "cell_type": "code", "execution_count": null, "id": "attached-columbia", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['A-607', 'A7-P138', 'A-7 Page 1465', ..., '#477', 'A-7 Page 1393',\n", " 'A-7-P180'], dtype=object)" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The below command displays an array of unique date values in the 'casenumber' column\n", "incidentdf[\"casenumber\"].unique()" ] }, { "cell_type": "markdown", "id": "human-arbitration", "metadata": {}, "source": [ "### Indentifying Cases of 'Riot'" ] }, { "cell_type": "markdown", "id": "adopted-mattress", "metadata": {}, "source": [ "As can be seen from above, there are varying formats for case number, some have dashes, others have a lowercase or a capital 'P', and some are missing the letter but have the word 'Page', etc. To identify and capture all of the variations of case number we can use regular expressions within the contains() function as shown below: " ] }, { "cell_type": "code", "execution_count": null, "id": "intimate-thesis", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/opt/conda/lib/python3.8/site-packages/pandas/core/strings/accessor.py:101: UserWarning: This pattern has match groups. To actually get the groups, use str.extract.\n", " return func(self, *args, **kwargs)\n" ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# Apply regex will help us find all those variations\n", "caseno = incidentdf[incidentdf['casenumber'].str.contains(\"A[-\\s]?7([p,?-p\\s-].*)?$\", na=False)]" ] }, { "cell_type": "code", "execution_count": null, "id": "confidential-bikini", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The below command prints out the number of times these variations appear within the dataframe. \n", "caseno['casenumber'].unique" ] }, { "cell_type": "markdown", "id": "authorized-noise", "metadata": {}, "source": [ "As can be seen above there are, give or take, about 3,850 cases associated with 'riot' that meet the regular expression criteria used to capture several variations. \n", "\n", "**Note, this number does not necesarily mean we captured all of the case numbers linked to 'riot',. Since the data was manually entered on the index cards by government personnel it is likely there are errors and some cards tied to 'riot' are not captured in this variation. \n", "\n", "Another alternative for looking for cards linked to 'riot' is to search through the offense column. As shown below. " ] }, { "cell_type": "code", "execution_count": null, "id": "final-french", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Image# 3872\n", "newdate 3872\n", "casenumber 3872\n", "offense 3872\n", "newoffense 3872\n", "DEATH 3872\n", "name 3872\n", "lastname 3872\n", "firstname 3872\n", "Other Names (known as) 3872\n", "Job/Role/Title 3872\n", "Residence 3872\n", "NEW-residence 3872\n", "X 3872\n", "Y 3872\n", "otherpeople 3872\n", "Age 3872\n", "Family / Alien No. 3872\n", "Other 3872\n", "Unnamed: 19 3872\n", "dtype: int64" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# Search through the 'newoffense' column using contains function\n", "noffriot = incidentdf[incidentdf['newoffense'].str.contains('riot')]\n", "noffriot.count()" ] }, { "cell_type": "markdown", "id": "collectible-search", "metadata": {}, "source": [ "After performing the filter, the output of cards linked to 'riot' is higher than that of the case number column. Let's apply the unique function to view what unique values the contains function captured when 'riot' was searched for in the new offense column. " ] }, { "cell_type": "code", "execution_count": null, "id": "aggregate-antarctica", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['riot'], dtype=object)" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# the unique function will return unique values in a series aka column. \n", "noffriot['newoffense'].unique()" ] }, { "cell_type": "markdown", "id": "owned-influence", "metadata": {}, "source": [ "As shown above other values that contain the word 'riot' like 'disorderly conduct false riot call' and 'assualt attempted riot' are also returned. Though these cases may be associated with the 'riot' that occured on 11/4/1944 we want to focus on cases that only contain 'riot'. \n", "\n", "Regular expressions can also be applied here to filter and only return values that contain 'riot'. Regular expressions are a sequence of characters, that can sometimes comprise of several characters, used to search through strings for patterns. Though regular expressions are helpful and a powerful tool, they can also be challenging as some characters have to be in a specific order. Here are a few helpful tools to introduce regular expressions: [Regular Expressions HOWTO](https://docs.python.org/3/howto/regex.html), [pythex.org](https://pythex.org), [How to use Regex in Pandas](https://kanoki.org/2019/11/12/how-to-use-regex-in-pandas/), [AutoRegex](https://www.autoregex.xyz). \n", "\n", "Below we use the ^ regular expression to return values where 'riot' is at the start of the string. By simply using this character we have reduced the number of values returned which more closely aligns with the number of cases linked to 'riot'. " ] }, { "cell_type": "code", "execution_count": null, "id": "appropriate-parish", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Image# 3860\n", "newdate 3860\n", "casenumber 3860\n", "offense 3860\n", "newoffense 3860\n", "DEATH 3860\n", "name 3860\n", "lastname 3860\n", "firstname 3860\n", "Other Names (known as) 3860\n", "Job/Role/Title 3860\n", "Residence 3860\n", "NEW-residence 3860\n", "X 3860\n", "Y 3860\n", "otherpeople 3860\n", "Age 3860\n", "Family / Alien No. 3860\n", "Other 3860\n", "Unnamed: 19 3860\n", "dtype: int64" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The ^ is used to search for values at the start of a string. \n", "noffriot = incidentdf[incidentdf['newoffense'].str.contains('^riot')]\n", "noffriot.count()" ] }, { "cell_type": "markdown", "id": "fifty-editor", "metadata": {}, "source": [ "To further explore the differences in the results of 'riot' in the new offense and the case number column we can use OpenRefine. OpenRefine will allow us to quickly view, filter, cluster and confirm the accuracy of case numbers associated with 'riot'. \n", "\n", "In the instance seen below, the difference is a result of the case number being inncorrectly entered or designated at the initial time of entry. **NOTE: It is unclear whether this error is an outcome of the initial data entry or a result of the data being manually entered from the index cards to a spreadsheet. \n", "\n", "
\"Locations\"
\n", "
\"Locations\"
\n", "\n", "More time could be spent refining the regular expression string used to search through the case numbers, but this is a good opportunity to practice using regular expressions on your own." ] }, { "cell_type": "markdown", "id": "circular-wireless", "metadata": {}, "source": [ "## Discovering Influential Figures in the 'Riot'\n", "\n", "Once the incident card data has been fine tuned and reflects cases only related to 'riot' we can then begin exploring the dataset to discover individuals who's names appears the most among the 'riot' cards. The decision was made to begin this project by focusing on the key figures invovled in the 'riot' with the possiblility of expanding and applying these methods to a larger dataset. \n", "\n", "The case number series that was filtered using the contains function earlier in the process will be used to identify the individuals recorded as being part of the 'riot'. As a reminder, when the data was filtered, caseno was assigned to that new output so we know that we are working specifically with data associated with case numbers linked to 'riot'. By applying values count to the caseno series we can identify these key or influential figures. \n", "\n", "The values count function will return objects, in descending order, that contain counts of unique values. So the first element will always be the most frequently occuring element, as shown below: " ] }, { "cell_type": "code", "execution_count": null, "id": "excellent-embassy", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "kuratomi 58\n", "kobayashi 40\n", "kai 36\n", "tom yoshio kobayashi 23\n", "\"b\" area 22\n", "tsuda 20\n", "kobayashi, tom yoshio 20\n", "rev. kai 19\n", "yamane, tokio 18\n", "tom kobayashi 17\n", "tokio yamane 16\n", "mori 16\n", "george kuratomi 15\n", "\"b\" area report 15\n", "tokunaga 15\n", "kazama 14\n", "kuratomi, george 14\n", "terada, singer 14\n", "oshita, tomoji 14\n", "kazama, tomio 14\n", "shigenobu murakimi 13\n", "yoshiyama, tom 13\n", "isamu uchida 13\n", "oki, kakuma 12\n", "yamauchi, dr. 11\n", "saito, shu 11\n", "oscar itani 11\n", "abe, tetsuo 11\n", "tetsuo abe 11\n", "nakao, masaru 11\n", "Name: name, dtype: int64" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The below command returns count values of the names in the caseno series\n", "riotcount = caseno['name'].value_counts()\n", "riotcount.head(30)" ] }, { "cell_type": "markdown", "id": "timely-agent", "metadata": {}, "source": [ "As shown above, the name Kuratomi appears the most frequently among the 'riot' incident cards. Following, Kuratomi is Kobayashi, Kai, Tom Yoshio Kobayashi, Tsuda, Reverend Kai, and Tokio Yamane, to name a few. \n", "\n", "Also recorded in this count is \"b\" area or \"b\" area report. Area B is a significant location in Tule Lake. After the 'riot' of November 4, suspected \"troublemakers\" were picked up--it is recorded around 209 in November and 107 in December--and placed in \"Area B\" also known as the \"Surveillance Area\" and later designated as the \"Stockade\". The barracks underwent 24 hour surveillance by armed soldiers from watchtowers, were searched almost daily for \"contraband\", visitors were not permitted except by permission from the Tule Lake Director, and curfew was set from 7:00 P.M. to 6:00 A.M. The stockade was situated northwest of the hospital for the entire period of its existence, November 5, 1943, to August 24, 1944, and enclosed by a high wire fence. \n", "\n", "
\"Locations\"
\n", "
*Note.* Photograph is courtesy from Densho Digital Archive, photograph, accessed from the National Park Service, Tule Lake website. [1]
\n", "\n" ] }, { "cell_type": "markdown", "id": "uniform-milan", "metadata": {}, "source": [ "From the output above, we'll take 25 of the individuals listed and explore their geographical spaces in more detail; one of those being, Kuratomi.\n", "\n", "Now that we've organized our data and identified key figures, we are prepared to begin gathering geographic information. All three datasets--the WRA, FAR, and Incident Cards-- will be used to identify different geographical spaces, specifically the WRA will be helpful for pinpointing places of origin before being forced to an assembly center as well as the location of the first assembly center, the FAR will be useful for identifying forced migration to internal and external camp locations, and the Incident Cards will be helpful for locating where individuals lived during incarceration. " ] }, { "cell_type": "markdown", "id": "dutch-collapse", "metadata": {}, "source": [ "The “Japanese-American Internee Data File, 1942 – 1946”, with camp intake records of evacuated Japanese-Americans, also known as WRA Form 26 is one of the major federal records associated with the War Relocation Authority (WRA), the agency established to handle the forced relocation and detention of Japanese-Americans during World War II. At the time, clerks recorded people’s information on the WRA Form 26 when brought into assembly centers or camps, afterwards, 34 fields were encoded onto punch cards. Later on the National Archives and Records Administration (NARA) digitized these punch cards and permenantly houses these records. The WRA data files are publically available. " ] }, { "cell_type": "markdown", "id": "aboriginal-jonathan", "metadata": {}, "source": [ "
\"Locations\"
\n", "\n", "
\"Locations\"
\n", "\n", "
\"Locations\"
" ] }, { "cell_type": "markdown", "id": "strange-origin", "metadata": {}, "source": [ "The “Final Accountability Rosters (FAR) of Evacuees at Relocation Centers, 1944-1946, records the camp outtake of evacuees at the time of their final release or transfer. During this time, clerks recorded outtake information on individuals as they exited camps. It's important to note that some individuals exited multiple camps due to forced transfers and have more than one FAR record. The FAR records were later transcribed by Densho, Ancestry.com, among others.\n" ] }, { "cell_type": "markdown", "id": "lonely-belize", "metadata": {}, "source": [ "
\"Locations\"
" ] }, { "cell_type": "markdown", "id": "suffering-scholarship", "metadata": {}, "source": [ "The Index Cards, also called, Incident Cards are Internal Security Case Reports” index cards, from the War Relocation Authority (Record Group 210 from 1941-47) records series. The Incident Cards reference narrative reports prepared by camp investigators, police officers, and directors of internal security, relating cases of alleged “disorderly conduct, rioting, seditious behavior,” etc. at each of the 10 camps, with detailed information on the names and addresses in the camps of the persons involved, the time and place where the alleged incident occurred, an account of what happened, and a statement of action taken by the investigating officer. There are 25,045 index cards, 63% of which (15,648) come from the Tule Lake concentration camp. \n", " " ] }, { "cell_type": "markdown", "id": "burning-bleeding", "metadata": {}, "source": [ "
\"Locations\"
" ] }, { "cell_type": "markdown", "id": "ceramic-particular", "metadata": {}, "source": [ "## Finding Names in the Incident Cards, WRA, & FAR" ] }, { "cell_type": "markdown", "id": "sufficient-short", "metadata": {}, "source": [ "Though this data is rich with information the WRA, FAR, and Incident Cards present unique challenges for us. For example, at times it was difficult to determine the order of a persons first and last name as well as many of the names are similiar to one another with one or two characters being different. Keeping this in mind, we have to be conscientious we are looking at the correct individual(s). A method for doing this is comparing information across the three separate data files like age, residence, family, individual or alien registration number, or using details found in the Other column. At other times we found that information was incomplete, for example, the WRA has truncated last names, leaving us to deduce that we were looking at the correct person after comparing the previously mentioned fields. \n", "\n", "As shown previously, we will follow a similiar process by using the contains function to locate data on Kuratomi. Because I am familiar with the data, I know Kuratomi's first name is George--a good practice to get in the habit of is to become familiar with the data you are working with before manipulating and analyzing it. \n", "\n", "By performing a simple contains search, as shown below, on the name column we are supplied with Kuratomi's residence, including other valueable information like his age, other individuals linked to Kuratomi, case numbers, as well as notes that are recorded on that specific index card which can be helpful for identifying this is correct person. \n", "\n", "By looking at the table below, under New-residence we find the the location of George's barrack, 6605-E. The address can be interpreted in the following manner, 66 as the block, 05 as the ward and E as the assigned barrack. " ] }, { "cell_type": "code", "execution_count": null, "id": "adequate-population", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Image#newdatecasenumberoffensenewoffenseDEATHnamelastnamefirstnameOther Names (known as)Job/Role/TitleResidenceNEW-residenceXYotherpeopleAgeFamily / Alien No.OtherUnnamed: 19
2338Box10-0217.jpg43-11-04A-7-P417riotriotgeorge kuratomiNote smuggled out on one of prisoners being re...
2366Box10-0245.jpg43-11-04A-7-P139riotriotgeorge kuratomi30Chairman, Age 30 years, Came from Jerome. List...
2368Box10-0247.jpg43-11-04A-7-P92riotriotgeorge kuratomi6605-E66-05635285.27724638923.446Youth leader from Jerome - given as possible l...
2369Box10-0248.jpg43-11-04A-7P89riotriotgeorge kuratomiOne of names listed on \"Approved Segregation C...
5358Box12-0840.jpg43-11-04A-7-P139riotriotgeorge kuratomi -1kuratomigeorgereverend kai -2, \"slim\" tsuda - 3, hayashi -4,...Kazama gave as his group of representatives, a...
6326Box13-0614.jpg43-11-04A-7-P404riotriotgeorge kuratomiYOSHITOMO HONDA (Alien) 3714-D 21896\\n\\nSent f...
6328Box13-0616.jpg43-11-04A-7-P402riotriotgeorge kuratomiChairman of meeting at Mess Hall #15 on 11/4/4...
6334Box13-0622.jpg43-11-04A-7-P373riotriotgeorge kuratomiTAKAHASHI\\n\\nMentioned during questioning of G...
6335Box13-0623.jpg43-11-04A-7-P377riotriotgeorge kuratomiUCHIDA\\n\\nName brought up during questioning o...
6336Box13-0624.jpg43-11-04A-7-P373riotriotgeorge kuratomiTOM KOBAYASHI\\n\\nMentioned during questioning ...
6337Box13-0625.jpg43-11-04riotriotgeorge kuratomiMITS\\n\\nName brought up during questioning of ...
6338Box13-0626.jpg43-11-04A-7-P375riotriotgeorge kuratomiSUGIMOTO\\n\\nName brought up during questioning...
6339Box13-0627.jpg43-11-04A-7-P390riotriotgeorge kuratomiMR. KOMIYA\\n\\nMr. Komiya is the Japanese secre...
6376Box13-0664.jpg43-11-04A-7-P89riotriotgeorge kuratomiOne of names listed on \"Approved Segregation C...
6408Box13-0696.jpg43-11-04A-7-P417riotriotgeorge kuratomiGEORGE\\n\\nNote smuggled out on one of prisoner...
6417Box13-0705.jpg43-11-04A-7-P139riotriotgeorge kuratomi30Listed as first in importance by KAZAMA in con...
6418Box13-0706.jpg43-11-04A-7-P92riotriotgeorge kuratomi6605-E66-05635285.27724638923.446Yout leader from Jerome - given as possible le...
6421Box13-0709.jpg43-11-04A-7-P286riotriotgeorge kuratomi, chairmanf.d. amat, consult of the spanish embassy / h....The above were present at meeting at Dining Ha...
\n", "
" ], "text/plain": [ " Image# newdate casenumber offense newoffense DEATH \\\n", "2338 Box10-0217.jpg 43-11-04 A-7-P417 riot riot \n", "2366 Box10-0245.jpg 43-11-04 A-7-P139 riot riot \n", "2368 Box10-0247.jpg 43-11-04 A-7-P92 riot riot \n", "2369 Box10-0248.jpg 43-11-04 A-7P89 riot riot \n", "5358 Box12-0840.jpg 43-11-04 A-7-P139 riot riot \n", "6326 Box13-0614.jpg 43-11-04 A-7-P404 riot riot \n", "6328 Box13-0616.jpg 43-11-04 A-7-P402 riot riot \n", "6334 Box13-0622.jpg 43-11-04 A-7-P373 riot riot \n", "6335 Box13-0623.jpg 43-11-04 A-7-P377 riot riot \n", "6336 Box13-0624.jpg 43-11-04 A-7-P373 riot riot \n", "6337 Box13-0625.jpg 43-11-04 riot riot \n", "6338 Box13-0626.jpg 43-11-04 A-7-P375 riot riot \n", "6339 Box13-0627.jpg 43-11-04 A-7-P390 riot riot \n", "6376 Box13-0664.jpg 43-11-04 A-7-P89 riot riot \n", "6408 Box13-0696.jpg 43-11-04 A-7-P417 riot riot \n", "6417 Box13-0705.jpg 43-11-04 A-7-P139 riot riot \n", "6418 Box13-0706.jpg 43-11-04 A-7-P92 riot riot \n", "6421 Box13-0709.jpg 43-11-04 A-7-P286 riot riot \n", "\n", " name lastname firstname Other Names (known as) \\\n", "2338 george kuratomi \n", "2366 george kuratomi \n", "2368 george kuratomi \n", "2369 george kuratomi \n", "5358 george kuratomi -1 kuratomi george \n", "6326 george kuratomi \n", "6328 george kuratomi \n", "6334 george kuratomi \n", "6335 george kuratomi \n", "6336 george kuratomi \n", "6337 george kuratomi \n", "6338 george kuratomi \n", "6339 george kuratomi \n", "6376 george kuratomi \n", "6408 george kuratomi \n", "6417 george kuratomi \n", "6418 george kuratomi \n", "6421 george kuratomi, chairman \n", "\n", " Job/Role/Title Residence NEW-residence X Y \\\n", "2338 \n", "2366 \n", "2368 6605-E 66-05 635285.2772 4638923.446 \n", "2369 \n", "5358 \n", "6326 \n", "6328 \n", "6334 \n", "6335 \n", "6336 \n", "6337 \n", "6338 \n", "6339 \n", "6376 \n", "6408 \n", "6417 \n", "6418 6605-E 66-05 635285.2772 4638923.446 \n", "6421 \n", "\n", " otherpeople Age \\\n", "2338 \n", "2366 30 \n", "2368 \n", "2369 \n", "5358 reverend kai -2, \"slim\" tsuda - 3, hayashi -4,... \n", "6326 \n", "6328 \n", "6334 \n", "6335 \n", "6336 \n", "6337 \n", "6338 \n", "6339 \n", "6376 \n", "6408 \n", "6417 30 \n", "6418 \n", "6421 f.d. amat, consult of the spanish embassy / h.... \n", "\n", " Family / Alien No. Other \\\n", "2338 Note smuggled out on one of prisoners being re... \n", "2366 Chairman, Age 30 years, Came from Jerome. List... \n", "2368 Youth leader from Jerome - given as possible l... \n", "2369 One of names listed on \"Approved Segregation C... \n", "5358 Kazama gave as his group of representatives, a... \n", "6326 YOSHITOMO HONDA (Alien) 3714-D 21896\\n\\nSent f... \n", "6328 Chairman of meeting at Mess Hall #15 on 11/4/4... \n", "6334 TAKAHASHI\\n\\nMentioned during questioning of G... \n", "6335 UCHIDA\\n\\nName brought up during questioning o... \n", "6336 TOM KOBAYASHI\\n\\nMentioned during questioning ... \n", "6337 MITS\\n\\nName brought up during questioning of ... \n", "6338 SUGIMOTO\\n\\nName brought up during questioning... \n", "6339 MR. KOMIYA\\n\\nMr. Komiya is the Japanese secre... \n", "6376 One of names listed on \"Approved Segregation C... \n", "6408 GEORGE\\n\\nNote smuggled out on one of prisoner... \n", "6417 Listed as first in importance by KAZAMA in con... \n", "6418 Yout leader from Jerome - given as possible le... \n", "6421 The above were present at meeting at Dining Ha... \n", "\n", " Unnamed: 19 \n", "2338 \n", "2366 \n", "2368 \n", "2369 \n", "5358 \n", "6326 \n", "6328 \n", "6334 \n", "6335 \n", "6336 \n", "6337 \n", "6338 \n", "6339 \n", "6376 \n", "6408 \n", "6417 \n", "6418 \n", "6421 " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The contains function will return values with the element 'George Kuratomi'\n", "incidentdf[incidentdf['name'].str.contains(\"george kuratomi\", na=False)]" ] }, { "cell_type": "markdown", "id": "conservative-approval", "metadata": {}, "source": [ "The .iloc is used to view the details of the notes in the Other column. The .iloc function is an integer position based function, meaning you can use indexing for selection of a particular position or cell and row within a table (dataset or also commonly called an array). " ] }, { "cell_type": "code", "execution_count": null, "id": "knowing-pledge", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"TAKAHASHI\\n\\nMentioned during questioning of Geroge Kuratomi on 12/3/43 in connection with Kashima's funeral.\"" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The below function returns information the cell with the index of 6334, 18\n", "incidentdf.iloc[5358,18]" ] }, { "cell_type": "markdown", "id": "auburn-leeds", "metadata": {}, "source": [ "As with the Incident Card data, performing the contains function allows us to extract values containing Kuratomi. \n", "\n", "Demonstrated below, George Kuratomi, can be found in the second row of the WRA datafile. Similiar to the Incident Cards, the WRA provides personal information that can be used to confirm we are looking at the correct person. To view additional details of a row, the .iloc function can be applied here as well. In the table below we can extract the following geographic information for Kuratomi: original state, assembly center, and first camp. This data will be helpful later to find the latitute and longitude coordinates for these separate locations. " ] }, { "cell_type": "code", "execution_count": null, "id": "moved-bumper", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
m_datasetm_pseudoidm_campm_lastnameNEW-m_lastnamem_firstnameNEW-m_firstnamem_birthyearNEW-m_birthyearm_gender...w_citizenshipstatusw_highestgradew_languagew_religionw_occupqual1w_occupqual2w_occupqual3w_occuppotn1w_occuppotn2w_filenumber
40132wra-master2-poston_kuratomi_1934_akiko2-postonKURATOMIKURATOMIAKIKOAKIKO19341934F...Did not attend Japanese Lang. School: Does not...No schooling or kindergarten [J]Not applicable (11 years old and under)Buddhist [1]200060
40133wra-master6-jerome_kuratomi_1915_george6-jeromeKURATOMIKURATOMIGEORGEGEORGE19151915M...Did not attend Japanese Lang. School: Has SS #...In U.S.: High School 4 [I]Japanese speak, read (& write), English speak,...Buddhist [1]71170276956857
40134wra-master6-jerome_kuratomi_1897_hide6-jeromeKURATOMIKURATOMIHIDEHIDE18971897F...Did not attend Japanese Lang. School: Has AR #...In Japan: High School 4 [D]Japanese speak, read (& write), English speakBuddhist [1]625614952839
40135wra-master6-jerome_kuratomi_1923_ikuko6-jeromeKURATOMIKURATOMIIKUKOIKUKO19231923F...Did not attend Japanese Lang. School: Has neit...In U.S.: College 1 [5]Japanese speak and English speak, read (& write)No religion, undecided, none, atheist, agnosti...1254952840
40136wra-master6-jerome_kuratomi_1884_iwami6-jeromeKURATOMIKURATOMIIWAMIIWAMI18841884M...Did not attend Japanese Lang. School: Has both...In Japan: High School 2 [B]Japanese speak, read (& write), English speak,...Baptist [A]7273303433953282
\n", "

5 rows × 46 columns

\n", "
" ], "text/plain": [ " m_dataset m_pseudoid m_camp m_lastname \\\n", "40132 wra-master 2-poston_kuratomi_1934_akiko 2-poston KURATOMI \n", "40133 wra-master 6-jerome_kuratomi_1915_george 6-jerome KURATOMI \n", "40134 wra-master 6-jerome_kuratomi_1897_hide 6-jerome KURATOMI \n", "40135 wra-master 6-jerome_kuratomi_1923_ikuko 6-jerome KURATOMI \n", "40136 wra-master 6-jerome_kuratomi_1884_iwami 6-jerome KURATOMI \n", "\n", " NEW-m_lastname m_firstname NEW-m_firstname m_birthyear NEW-m_birthyear \\\n", "40132 KURATOMI AKIKO AKIKO 1934 1934 \n", "40133 KURATOMI GEORGE GEORGE 1915 1915 \n", "40134 KURATOMI HIDE HIDE 1897 1897 \n", "40135 KURATOMI IKUKO IKUKO 1923 1923 \n", "40136 KURATOMI IWAMI IWAMI 1884 1884 \n", "\n", " m_gender ... w_citizenshipstatus \\\n", "40132 F ... Did not attend Japanese Lang. School: Does not... \n", "40133 M ... Did not attend Japanese Lang. School: Has SS #... \n", "40134 F ... Did not attend Japanese Lang. School: Has AR #... \n", "40135 F ... Did not attend Japanese Lang. School: Has neit... \n", "40136 M ... Did not attend Japanese Lang. School: Has both... \n", "\n", " w_highestgrade \\\n", "40132 No schooling or kindergarten [J] \n", "40133 In U.S.: High School 4 [I] \n", "40134 In Japan: High School 4 [D] \n", "40135 In U.S.: College 1 [5] \n", "40136 In Japan: High School 2 [B] \n", "\n", " w_language \\\n", "40132 Not applicable (11 years old and under) \n", "40133 Japanese speak, read (& write), English speak,... \n", "40134 Japanese speak, read (& write), English speak \n", "40135 Japanese speak and English speak, read (& write) \n", "40136 Japanese speak, read (& write), English speak,... \n", "\n", " w_religion w_occupqual1 \\\n", "40132 Buddhist [1] \n", "40133 Buddhist [1] 71 \n", "40134 Buddhist [1] \n", "40135 No religion, undecided, none, atheist, agnosti... \n", "40136 Baptist [A] 72 \n", "\n", " w_occupqual2 w_occupqual3 w_occuppotn1 w_occuppotn2 w_filenumber \n", "40132 200060 \n", "40133 170 27 6 956857 \n", "40134 625 614 952839 \n", "40135 125 4 952840 \n", "40136 73 303 433 953282 \n", "\n", "[5 rows x 46 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The contains function will return results with Kuratomi.\n", "wradf[wradf['NEW-m_lastname'].str.contains(\"KURATOMI\", na=False)].head(5)" ] }, { "cell_type": "code", "execution_count": null, "id": "fancy-juice", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "m_dataset wra-master\n", "m_pseudoid 6-jerome_kuratomi_1915_george\n", "m_camp 6-jerome\n", "m_lastname KURATOMI\n", "NEW-m_lastname KURATOMI\n", "m_firstname GEORGE\n", "NEW-m_firstname GEORGE\n", "m_birthyear 1915\n", "NEW-m_birthyear 1915\n", "m_gender M\n", "m_originalstate CA\n", "m_familyno 4119\n", "NEW-m_familyno 4119\n", "m_familyno_normal 4119\n", "NEW-m_familyno_normal 4119\n", "m_individualno 04119A\n", "m_altfamilyid \n", "m_altindividualid \n", "m_ddrreference \n", "m_notes \n", "w_assemblycenter Santa Anita\n", "w_originaladdress San Diego, California [13&14]\n", "w_birthcountry Father-Japan, Mother-Japan [1]\n", "w_fatheroccupus Unskilled laborers (except farm) [9]\n", "w_fatheroccupabr Unknown [&]\n", "w_yearsschooljapan 7\n", "w_gradejapan 1 - 8 yrs. of school only [1]\n", "w_schooldegree None [0]\n", "w_yearofusarrival \n", "w_timeinjapan 5 yrs. but less than 10 yrs. in Japan [4]\n", "w_notimesinjapan \n", "w_ageinjapan Between ages 0-9 and also 10-19 and also 20 an...\n", "w_militaryservice No Military or Naval Service, No Public Assist...\n", "w_maritalstatus Single\n", "w_ethnicity Japanese, No spouse [4]\n", "w_birthplace California\n", "w_citizenshipstatus Did not attend Japanese Lang. School: Has SS #...\n", "w_highestgrade In U.S.: High School 4 [I]\n", "w_language Japanese speak, read (& write), English speak,...\n", "w_religion Buddhist [1]\n", "w_occupqual1 71\n", "w_occupqual2 170\n", "w_occupqual3 \n", "w_occuppotn1 27\n", "w_occuppotn2 6\n", "w_filenumber 956857\n", "Name: 40133, dtype: object" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# Apply the .iloc to a row index to view all information within that given row\n", "wradf.iloc[40133]" ] }, { "cell_type": "markdown", "id": "likely-bronze", "metadata": {}, "source": [ "Lastly, we will look through the FAR to locate exit locations for George Kuratomi. Exit locations include forced movements from one camp to another throughout incarceration. \n", "\n", "A quick scan of the table below, we find that George Kuratomi doesn't appear among the listed names, this is because 'George' is an evangelical name used by Toshio Kuratomi. This was a phenomena discovered by simply becoming familiar with the data. In fact, George was not the only individual to use an evangelical name during incarceration. Now that we are equipped with this information, this will require us to do further investigation so we are careful not to rule out a person because their doesnt immediately appear in a dataset. It may end up being the case here where an alternative name is used." ] }, { "cell_type": "code", "execution_count": null, "id": "offshore-tackle", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
FAR Exit Datasetoriginal_orderfar_line_idfamily_numberNEW-family_numberlast_name_correctedNEW-last_name_correctedlast_name_originalfirst_name_correctedfirst_name_original...type_of_final_departureNEW-type_of_final_departuredate_of_final_departurefinal_departure_statecamp_address_originalcamp_address_blockcamp_address_barrackscamp_address_roomreferencenotes
16101granada13820371340914091KuratomiKuratomiKuratomiHideHide...Ind-InvitInd-Invit7/26/44 0:00CO
16102jerome13856381140914091KuratomiKuratomiKuratomiHideHide...T-TT6/19/44 0:00
16103jerome13857381240914091KuratomiKuratomiKuratomiIkukoIkuko...Ind-InvitInd-Invit6/1/44 0:00IL
16104granada13819371240914091KuratomiKuratomiKuratomiMasakoMasako...Ind-InvitInd-Invit7/26/44 0:00CO
16105jerome13855381040914091KuratomiKuratomiKuratomiMasakoMasako...T-TT6/19/44 0:00
16106granada13818371140914091KuratomiKuratomiKuratomiRintaroRintaro...Ind-InvitInd-Invit7/26/44 0:00CO
16107jerome13858381340914091KuratomiKuratomiKuratomiRintaroRintaro...T-TT6/19/44 0:00
16255jerome13859381441194119KuratomiKuratomiKuratomiToshioToshio...T-ST-S9/26/43 0:00
16256tulelake1107261025141194119KuratomiKuratomiKuratomiUtakoUtako...Term-with GrantTerm-With Grant1/10/46 0:00PA(Nee Terada - 1685)
16257tulelake1107271025241194119KuratomiKuratomiKuratomiYuriYuri...Term-with GrantTerm-With Grant1/10/46 0:00PA
38724poston17304411003310033KuratomiKuratomiKuratomiAkikoAkiko...Term W-GTerm-With Grant1/16/45 0:00CO2811A
38725poston17303401003310033KuratomiKuratomiKuratomiCharlotteCharlotte...Term W-GTerm-With Grant1/16/45 0:00CO2811A
38726poston17302391003310033KuratomiKuratomiKuratomiMasatoMasato...Ind-EmplInd-Empl12/6/44 0:00CO2811A
38727poston17305421003310033KuratomiKuratomiKuratomiTatsutoTatsuto...Term W-GTerm-With Grant1/16/45 0:00CO2811A
142361rohwer14581447711-17811-178KuratomiKuratomiKuratomiIwamiIwami...Term W-GTerm-With Grant8/31/45 0:00NY6-8-C68C
142362jerome13851380611-17811-178KuratomiKuratomiKuratomiIwamiIwami...T-TT6/30/44 0:00
142363rohwer14582447811-17811-178KuratomiKuratomiKuratomiRayRay...Ind-EmplInd-Empl9/5/44 0:00IL6-8-C68C
142364jerome13853380811-17811-178KuratomiKuratomiKuratomiRayRay...T-TT6/30/44 0:00
142365rohwer14583447911-17811-178KuratomiKuratomiKuratomiSadacoSadaco...Term W-GTerm-With Grant8/31/45 0:00NY6-8-C68C
142366jerome13854380911-17811-178KuratomiKuratomiKuratomiSadakoSadako...T-TT6/10/44 0:00
142367rohwer14584448011-17811-178KuratomiKuratomiKuratomiTsuyokaTsuyoka...Term W-GTerm-With Grant8/31/45 0:00NY6-8-C68C
142368jerome13852380711-17811-178KuratomiKuratomiKuratomiTsuyokaTsuyoka...T-TT6/10/44 0:00
\n", "

22 rows × 36 columns

\n", "
" ], "text/plain": [ " FAR Exit Dataset original_order far_line_id family_number \\\n", "16101 granada1 3820 3713 4091 \n", "16102 jerome1 3856 3811 4091 \n", "16103 jerome1 3857 3812 4091 \n", "16104 granada1 3819 3712 4091 \n", "16105 jerome1 3855 3810 4091 \n", "16106 granada1 3818 3711 4091 \n", "16107 jerome1 3858 3813 4091 \n", "16255 jerome1 3859 3814 4119 \n", "16256 tulelake1 10726 10251 4119 \n", "16257 tulelake1 10727 10252 4119 \n", "38724 poston1 7304 41 10033 \n", "38725 poston1 7303 40 10033 \n", "38726 poston1 7302 39 10033 \n", "38727 poston1 7305 42 10033 \n", "142361 rohwer1 4581 4477 11-178 \n", "142362 jerome1 3851 3806 11-178 \n", "142363 rohwer1 4582 4478 11-178 \n", "142364 jerome1 3853 3808 11-178 \n", "142365 rohwer1 4583 4479 11-178 \n", "142366 jerome1 3854 3809 11-178 \n", "142367 rohwer1 4584 4480 11-178 \n", "142368 jerome1 3852 3807 11-178 \n", "\n", " NEW-family_number last_name_corrected NEW-last_name_corrected \\\n", "16101 4091 Kuratomi Kuratomi \n", "16102 4091 Kuratomi Kuratomi \n", "16103 4091 Kuratomi Kuratomi \n", "16104 4091 Kuratomi Kuratomi \n", "16105 4091 Kuratomi Kuratomi \n", "16106 4091 Kuratomi Kuratomi \n", "16107 4091 Kuratomi Kuratomi \n", "16255 4119 Kuratomi Kuratomi \n", "16256 4119 Kuratomi Kuratomi \n", "16257 4119 Kuratomi Kuratomi \n", "38724 10033 Kuratomi Kuratomi \n", "38725 10033 Kuratomi Kuratomi \n", "38726 10033 Kuratomi Kuratomi \n", "38727 10033 Kuratomi Kuratomi \n", "142361 11-178 Kuratomi Kuratomi \n", "142362 11-178 Kuratomi Kuratomi \n", "142363 11-178 Kuratomi Kuratomi \n", "142364 11-178 Kuratomi Kuratomi \n", "142365 11-178 Kuratomi Kuratomi \n", "142366 11-178 Kuratomi Kuratomi \n", "142367 11-178 Kuratomi Kuratomi \n", "142368 11-178 Kuratomi Kuratomi \n", "\n", " last_name_original first_name_corrected first_name_original ... \\\n", "16101 Kuratomi Hide Hide ... \n", "16102 Kuratomi Hide Hide ... \n", "16103 Kuratomi Ikuko Ikuko ... \n", "16104 Kuratomi Masako Masako ... \n", "16105 Kuratomi Masako Masako ... \n", "16106 Kuratomi Rintaro Rintaro ... \n", "16107 Kuratomi Rintaro Rintaro ... \n", "16255 Kuratomi Toshio Toshio ... \n", "16256 Kuratomi Utako Utako ... \n", "16257 Kuratomi Yuri Yuri ... \n", "38724 Kuratomi Akiko Akiko ... \n", "38725 Kuratomi Charlotte Charlotte ... \n", "38726 Kuratomi Masato Masato ... \n", "38727 Kuratomi Tatsuto Tatsuto ... \n", "142361 Kuratomi Iwami Iwami ... \n", "142362 Kuratomi Iwami Iwami ... \n", "142363 Kuratomi Ray Ray ... \n", "142364 Kuratomi Ray Ray ... \n", "142365 Kuratomi Sadaco Sadaco ... \n", "142366 Kuratomi Sadako Sadako ... \n", "142367 Kuratomi Tsuyoka Tsuyoka ... \n", "142368 Kuratomi Tsuyoka Tsuyoka ... \n", "\n", " type_of_final_departure NEW-type_of_final_departure \\\n", "16101 Ind-Invit Ind-Invit \n", "16102 T-T T \n", "16103 Ind-Invit Ind-Invit \n", "16104 Ind-Invit Ind-Invit \n", "16105 T-T T \n", "16106 Ind-Invit Ind-Invit \n", "16107 T-T T \n", "16255 T-S T-S \n", "16256 Term-with Grant Term-With Grant \n", "16257 Term-with Grant Term-With Grant \n", "38724 Term W-G Term-With Grant \n", "38725 Term W-G Term-With Grant \n", "38726 Ind-Empl Ind-Empl \n", "38727 Term W-G Term-With Grant \n", "142361 Term W-G Term-With Grant \n", "142362 T-T T \n", "142363 Ind-Empl Ind-Empl \n", "142364 T-T T \n", "142365 Term W-G Term-With Grant \n", "142366 T-T T \n", "142367 Term W-G Term-With Grant \n", "142368 T-T T \n", "\n", " date_of_final_departure final_departure_state camp_address_original \\\n", "16101 7/26/44 0:00 CO \n", "16102 6/19/44 0:00 \n", "16103 6/1/44 0:00 IL \n", "16104 7/26/44 0:00 CO \n", "16105 6/19/44 0:00 \n", "16106 7/26/44 0:00 CO \n", "16107 6/19/44 0:00 \n", "16255 9/26/43 0:00 \n", "16256 1/10/46 0:00 PA \n", "16257 1/10/46 0:00 PA \n", "38724 1/16/45 0:00 CO \n", "38725 1/16/45 0:00 CO \n", "38726 12/6/44 0:00 CO \n", "38727 1/16/45 0:00 CO \n", "142361 8/31/45 0:00 NY 6-8-C \n", "142362 6/30/44 0:00 \n", "142363 9/5/44 0:00 IL 6-8-C \n", "142364 6/30/44 0:00 \n", "142365 8/31/45 0:00 NY 6-8-C \n", "142366 6/10/44 0:00 \n", "142367 8/31/45 0:00 NY 6-8-C \n", "142368 6/10/44 0:00 \n", "\n", " camp_address_block camp_address_barracks camp_address_room reference \\\n", "16101 \n", "16102 \n", "16103 \n", "16104 \n", "16105 \n", "16106 \n", "16107 \n", "16255 \n", "16256 \n", "16257 \n", "38724 28 11 A \n", "38725 28 11 A \n", "38726 28 11 A \n", "38727 28 11 A \n", "142361 6 8 C \n", "142362 \n", "142363 6 8 C \n", "142364 \n", "142365 6 8 C \n", "142366 \n", "142367 6 8 C \n", "142368 \n", "\n", " notes \n", "16101 \n", "16102 \n", "16103 \n", "16104 \n", "16105 \n", "16106 \n", "16107 \n", "16255 \n", "16256 (Nee Terada - 1685) \n", "16257 \n", "38724 \n", "38725 \n", "38726 \n", "38727 \n", "142361 \n", "142362 \n", "142363 \n", "142364 \n", "142365 \n", "142366 \n", "142367 \n", "142368 \n", "\n", "[22 rows x 36 columns]" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# Apply the contains function to return results for Kuratomi\n", "fardf[fardf['NEW-last_name_corrected'].str.contains(\"Kuratomi\", na=False)]" ] }, { "cell_type": "code", "execution_count": null, "id": "peripheral-encyclopedia", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "FAR Exit Dataset jerome1\n", "original_order 3859\n", "far_line_id 3814\n", "family_number 4119\n", "NEW-family_number 4119\n", "last_name_corrected Kuratomi\n", "NEW-last_name_corrected Kuratomi\n", "last_name_original Kuratomi\n", "first_name_corrected Toshio\n", "first_name_original Toshio\n", "NEW-first_name_corrected Toshio\n", "other_names George\n", "NEW-other_names George\n", "date_of_birth \n", "NEW-date_of_birth \n", "year_of_birth \n", "NEW-year_of_birth \n", "sex M\n", "NEW_marital_status S\n", "citizenship C\n", "alien_registration_no. \n", "type_of_original_entry SAAC\n", "NEW-type_of_original_entry SaAC\n", "NEW-pre-evacuation_address San Diego\n", "pre-evacuation_state CA\n", "date_of_original_entry 10/30/42 0:00\n", "type_of_final_departure T-S\n", "NEW-type_of_final_departure T-S\n", "date_of_final_departure 9/26/43 0:00\n", "final_departure_state \n", "camp_address_original \n", "camp_address_block \n", "camp_address_barracks \n", "camp_address_room \n", "reference \n", "notes \n", "Name: 16255, dtype: object" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# Apply the .iloc function to view additional information in index (row) below\n", "fardf.iloc[16255]" ] }, { "cell_type": "markdown", "id": "supreme-benefit", "metadata": {}, "source": [ "In this first module, I have shown how to use Pythons contains function to manipulate and analyze specific data within datasets. We harnessed the power of regular expressions to return different variations of values that were inconsisently entered, inlcuding limiting the contains function to return particular values. We identified key figures involved in the 'riot' at Tule Lake by using the counts value function. Futhermore, we located the place of origin, assembly center, first camp location, and place of residence at Tule Lake for one of the many persons invovled in the 'riot' that took place at Tule Lake on November 4, 1944. \n", "\n", "In the following module, we will focus on how to use the geographical data collected here to construct datasets that will allow us model and visualize the movements of 25 selected individuals. " ] }, { "cell_type": "code", "execution_count": null, "id": "different-laser", "metadata": {}, "outputs": [ { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [ "# The below command let's us save the modified dataframes into a new output csv file. \n", "# This can be useful when using these files for further steps of processing.\n", "caseno.to_csv('casenodf.csv', index=False)" ] }, { "cell_type": "markdown", "id": "sharing-blade", "metadata": {}, "source": [ "### References\n", "[1] Photograph; Tule Lake, California. Tule Lake Historic Jail Rehabilitation Project Public Review. . .; Densho Digital Archive; Found on the National Park Service, Tule Lake. [https://www.nps.gov/tule/getinvolved/tule-lake-historic-jail-rehabilitation-project-public-review.htm, May 31, 2018]" ] }, { "cell_type": "markdown", "id": "charged-contents", "metadata": {}, "source": [ "## Notebooks\n", "\n", "The below module is organized into a sequential set of Python Notebooks that allows us to interact with the collections related to the Framework for Unlocking and Linking WWII Japanese American Incarceration Biographical Data to explore, clean, prepare, visualize and analyze it from historical context perspective.\n", "\n", "1. A Framework for Unlocking and Linking WWII Japanese American Incarceration Biographical Data - Context Based Data Manipulation and Analysis - Part 2\n", "2. A Framework for Unlocking and Linking WWII Japanese American Incarceration Biographical Data - Data Visualization " ] }, { "cell_type": "code", "execution_count": null, "id": "dated-placement", "metadata": {}, "outputs": [ { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] }, { "ename": "", "evalue": "", "output_type": "error", "traceback": [ "\u001b[1;31mnotebook controller is DISPOSED. \n", "\u001b[1;31mView Jupyter log for further details." ] } ], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" }, "vscode": { "interpreter": { "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1" } } }, "nbformat": 4, "nbformat_minor": 5 }