{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Georeferencing with ESRI ArcMap Instructions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Author:** Rebecca Vandewalle rcv3@illinois.edu\n", "
**Created:** 5-21-21\n", "\n", "This notebook contains instructions for how to georeference an image. These instructions are intended for those who have access to ESRI ArcMap." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "- [Introduction](#intro)\n", "- [Gather Initial Data for Georeferencing](#gather)\n", "- [Open ArcMap](#open)\n", "- [Load the Major Streets Dataset](#load_streets)\n", "- [Load the Chicago Map](#load_map)\n", "- [Navigating](#navigating)\n", "- [Using the Identify Tool](#identify)\n", "- [Preparing to Georeference](#prep_georef)\n", "- [Starting Georeferencing: Adding Control Points](#add_control)\n", "- [Completing the Georeferencing Process: Rectifying](#rectifying)\n", "- [Loading the Rectified Image](#load_final_image)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook is intended to demonstrate how to transform geospatial data from an image to a digital file that can be used in maps. This demo will use ESRI ArcMap.\n", "\n", "Learning Goals:\n", "- Georeference an image using reference data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Gather Initial Data for Georeferencing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will need to start with two datasets:\n", "1. The map image that we wish to collect data from\n", "2. A reference data set to give spatial information to the map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The map image we will work with is Rand McNally & Co.'s new street number guide map of Chicago (1916) from the Harvard Map Collection. You can download the map from Wikipedia [here](https://commons.wikimedia.org/wiki/File:1916_Chicago_map_-_South.jpg) (make sure to download the full version) or use the map that has been uploaded to the root folder of this tutorial (`1916_Chicago_map_-_South.jpg`).\n", "\n", "\"Rand\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The reference data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We also will need data that has geospatial information included to use as a reference when we add geospatial information to the map. This dataset needs to include features that we can find the location of in both the data set and the map. It is slightly difficult because the map is from 1916 and the reference data is current, however many of the modern major roads and railroads are in the same place as they were in 1916 and can be used to reference the map.\n", "\n", "The City of Chicago has a nice data portal [here](https://data.cityofchicago.org/browse?tags=shapefiles) (use the `gis` tag and the `shapefiles` tag). From this portal we can download the [Major Streets dataset](https://data.cityofchicago.org/Transportation/Major-Streets/ueqs-5wr6) by going to this link and selecting `Download`. For convienience, this dataset is also available in the `Chicago_Major_Streets` folder." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prepare to work with the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that you have the data we will start working in ArcMap. If you are accessing ArcMap using a remote desktop, you will first need to transfer your files to the remote desktop." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Open ArcMap" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To open ArcMap, click on the `Start Menu`, click on the `ArcGIS` folder, and then select `ArcMap 10.8`. A new ArcMap `Getting Started` window will open. Select `Blank Map`.\n", "\n", "\n", " \n", "\n", "\n", "\n", " \n", "
\"open \"open
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A new ArcMap project window will open:\n", "\n", "\"initial" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Load the Major Streets Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First you need to add data. Select the `Add Data` button. This will open the `Add Data` window. In the `Add Data` window some folder options will be displayed, however if the folder you have stored your data in is not one of the drop down options, select the `Connect to Folder` button and then navigate to folder holding your data and select `OK`.\n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", " \n", "
\"add \"folder\"add\"folder
\n", " \n", "After connecting to the folder with your data you will see the data files you have uploaded. Select ONLY `Major_Streets.shp` and click `Add`.\n", " \n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You will now see `Major_Streets` loaded into your project. The initial display color is random. Now would be a good time to save your ArcMap file by selecting `File -> Save`.\n", "\n", "\"major" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now you can check the properties of `Major_Streets`. Right Click on `Major_Streets` in the `Table of Contents` pane and select `Properties...` at the bottom of the pop up menu. This will open the `Layer Properties` window to the `General` tab. Select the `Source` tab and look at the source information. Note that there is information under `Projected Coordinate System`. This indicates that this data file has been projected, which means that the coordinates indicate locations on a flat surface instead of raw latitude and longitude values. This information will be used to add spatial information to the Chicago map.\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", "
\"major\"major\"major
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Load the Chicago Map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Select the `Add Data` button again to load the Chicago Map. A dialog box will say that the spatial reference is unknown. Select `OK`. \"unknown" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we will need to display the `Georeferencing` toolbar. In the top menu bar select `Customize -> Toolbars -> Georeferencing`. You can drag the `Georeferencing` toolbar and dock it on the bottom of the menu bars. Now right click `Major_Streets` and select `Zoom to Layer`. Next, on the `Georeferencing` toolbar select `Fit to Display`.\n", "\n", "
\n", " \n", " \n", " \n", " \n", " \n", "
\"folder\"folder\"folder
\n", " \n", "Now your Chicago Map should appear under your `Major_Streets` data.\n", " \n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It can be hard to see the `Major_Streets` data if the color does not stand out from the map as in this example. You can change the color used for the `Major_Streets` data by right clicking on `Major_Streets` in the `Table of Contents` pane and selecting `Properties...` at the bottom of the pop up menu. This will open the `Layer Properties` window to the `General` tab. Select the `Symbology` tab. Click on the `Symbol` line in the `Symbol` box to open the `Symbol Selector` dialog box. Then you can change the color to a color that is easier to see.\n", "\n", "\"color" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Navigating" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to georeference you will need to shift the view of the data on the screen. The navigation controls are placed in the second toolbar row by default. Click on the magnifying glass with the plus sign to zoom in, Click on the magnifying glass with the minus sign to zoom out, click on the hand to pan, and click on the globe to return to the full extent (show all the data).\n", "\n", "\"zoom" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Using the Identify Tool" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A key action needed for georeferencing is figuring out what the features that you are trying to match are. For this you can use the `Identify` tool. Click on the `Identify` tool, which looks like a blue circle with a white `I` to select the tool. Then click on a feature (i.e. the road line) to see information about it. It will identify features from the top layer in the `Table of Contents` by default, however you can change that behavior in the `Identify` dropdown box.\n", "\n", "
\n", " \n", " \n", " \n", " \n", "
\"identify\"identified
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Preparing to Georeference" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before starting to georeference it is important to look at both the `Major_Streets` data set and the map itself to identify places where they match up. Ideally you will want to choose at least one point for every corner of the map image for the best results. It helps to look for irregular features that will be easy to spot on both the Chicago map and the `Major_Streets` data. I opened the full map in another window so that I could zoom in and read all the small street names. Then I made a sketch map with a smaller version of the map and made notes for myself for where some easily identifiable features were. Since the map is from 1916, some street names and locations have changed in the modern data set.\n", "\n", "\"annotated" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once you are more familiar with streets on the map and streets in the `Major_Streets` dataset, you will see that the map only covers the bottom portion of the `Major_Streets` dataset. You can zoom in to this area and then select `Fit to Display` on the `Georeferencing` toolbar again to start the georeferencing process with the map closer to its actual position.\n", "\n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Starting Georeferencing: Adding Control Points" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To start the Georeferencing process you will need to add `Control Points`. These are like anchors that connect locations on the map to locations on the `Major_Streets` dataset. You will always add `Control Points` by starting with a position on the map and ending with a position on the `Major_Streets` dataset.\n", "\n", "To start adding `Control Points` select the `Control Points` button on the `Georeferencing` toolbar. It looks like a green `X` and an arrow pointing to a red `X`.\n", "\n", "\"add" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding the First Control Point" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now you can start adding `Control Points`. I started with Archer Street because the bend that the street makes is easy to find. Zoom in so that you can easily see street names and streets on the map. In this example the locations are close together because I started from displaying the map close to its actual location. \n", "- Locate the bend of Archer Street on both the Chicago map and the `Major_Streets` dataset. Use the `Identify` tool to make sure you have found Archer Street on the `Major_Streets` dataset.\n", "- With the `Control Points` button selected, click on the reference point (the bend of Archer Street) **on the Chicago Map**. You will see a green cross with a number `1` at that point and as you move your mouse you will see a line connecting the green cross with your mouse position.\n", "- Next click on the bend of Archer Street **on the `Major_Streets` dataset**. Because this is the first control point, the Chicago map will shift so that the two points are in the same place so you will then see a red cross with a number `1` at that point and the streets will start to align with the Chicago map.\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\"control\"control
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding the Second Control Point" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next you will add the second `Control Point`. It is best if you do this on the oposite corner of the map from your first control point. In this example, I used the intersection of 134th street and Avenue O.\n", "- Locate the intersection of 134th street and Avenue O on both the Chicago map and the `Major_Streets` dataset. Use the `Identify` tool to help find the streets.\n", "- With the `Control Points` button selected, click on the reference point (the intersection of 134th street and Avenue O) **on the Chicago Map**. You will see a green cross with a number `2` at that point.\n", "- Next click on the intersection of 134th street and Avenue O **on the `Major_Streets` dataset**. The map will shift again so that the two points are approximately in the same place and the streets in the area should line up.\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\"control\"control
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Checking the Full Map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After you have added two `Control Points` in oposite corners of the Chicago map, zoom to the full map (right cick on the map and select `Zoom to Layer`) and see how well it is currently lined up with the `Major_Streets` dataset. It should match pretty well at this point.\n", "\n", "\"fit" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding the Third Control Point" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now it is good to add a third `Control Point` in another corner of the Chicago map. For this example, I choose the intersection of 71st and Yates. The `Major_Streets` road and the Chicago map were well lined up at this location, so I simply clicked twice to add a `Control Point` in the same spot in both. Even if the map lines up well already in this corner it is good to add a `Control Point` as ArcMap will continue making adjustments to the map as you add more `Control Points`.\n", "\n", "\"control" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding the Fourth Control Point" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally I added a fourth `Control Point` near the last corner of the map. As the map and `Major_Streets` were slightly off here I first clicked on the intersection of Cicero and 87th on the Chicago map then on the `Major_Streets` intersection.\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\"control\"control
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fixing a Bad Control Point" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, what happens if you add a bad `Control Point`? In this example I added a bad fifth control point where I clicked in the center of the block and matched it with a section of Cicero Avenue. This shifted the Chicago map again, and now you can see large blue lines between my two fourth `Control Points`, which had been in the correct location, and my two bad fifth `Control Points`. Large blue lines between `Control Points` is a sign that the map and the data layer is not well aligned.\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\"folder\"folder
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To fix this I first opened the `Link` table by pressing the `Link` table button in the `Georeferencing` toolbar.\n", "\n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are very large numbers in the `Residual` column in the `Link` table, which is a bad sign. The values for the bad fifth `Control Point` are the largest.\n", "\n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To fix this I simply clicked on the row for the fifth `Control Point` and selected the `Delete Control Point` button at the top of the toolbar in the `Link` dialog box. Afterwards you can see that the values in the `Residual` columns are much smaller. It will be impossible for each of the values to be zero as some error in this process is expected.\n", "\n", "\"folder\n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Completing the Georeferencing Process: Rectifying" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once you are satisfied with the control points, you will need to complete the georeferencing process and save the final image. In the `Georeferencing` toolbar select `Rectify...`. This will open the `Save As` dialogue box as you will be saving a copy of the image that includes spatial information. A common format is a `GeoTIFF` file, which is a `TIFF` image file that includes spatial information (it will just appear as `TIFF` in this box). Click on the folder icon to change the output location to the folder that you want to save the image in. You can also change the name of the final image if you would like.\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\"folder\"folder
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Loading the Rectified Image" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now you can open the final rectified image in ArcMap using the `Add Data` button. It will show up in the `Table of Contents` and will be displayed on the screen.\n", "\n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Right click on the final image in the `Table of Contents` pane and select `Properties...` and click on the `Source` tab. You can now see that the image has spatial reference information which is the same as the `Major_Streets` dataset.\n", "\n", "\"folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Large Image Size" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The rectified image will be larger that the starting image. It could easily be too big to transfer in and out of a remote desktop. If this is the case, using a cloud based data storage system, such as Illinois Box, could be helpful in transfering the file." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3-0.8.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }