{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# FIRST import all the necessary libraries and modules!\n", "import cv2 # import OpenCV\n", "import numpy as np # import NumPy\n", "\n", "# import functions \n", "import sys\n", "sys.path.insert(0, '../..')\n", "from utils import * " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Contours\n", "\n", "

\n", "In this lab, we will learn about contours and how to use contours to help us identify objects. Furthermore, we will be learning about the following functions: \n", "

\n", "

\n", " \n", "

\n", " We will learn how to create a function to help us detect drawings on a piece of paper! \n", "

\n", "\n", "### What is a contour? \n", "\n", "

\n", " A contour is a curve joining all continuous points along a boundary. The green below outlines the contours of a computer mouse:\n", "

\n", "\n", "\"mouse\n", "\n", "\n", "### Why contours?\n", "\n", "

\n", " Contours can help us analyze an object's shape and detect objects, so they will be helpful in the future when we want detect specific objects or obstacles in the car's field of view (ie. cones).\n", "

\n", "\n", "\"object" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "# Find and Draw Contours\n", "\n", "

\n", " In the following section, we will learn how to find and draw contours. \n", "
To find the contours of an image, a few steps need to be taken: \n", "

\n", "\n", "

\n", " STEP #1: Read the image.\n", "

\n", "\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #1: Read 'star.png' via cv2.imread. Save as 'img'.\n", "\n", "\n", "# Save the original image to display later.\n", "img_copy = img.copy()\n", "\n", "# Show image in popup window\n", "cv2.imshow('original', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " STEP #2: Convert the image into grayscale.\n", "

\n", "\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #2: Convert the BGR image to grayscale via cv2.COLOR_BGR2GRAY. \n", "# Save as 'img_gray' \n", "\n", "\n", "# Show image in popup window\n", "cv2.imshow('grayscale', img_gray)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " STEP #3: Convert the grayscale image into a binary image with cv2.threshold. \n", "

\n", "\n", "

\n", " cv2.threshold looks at each pixel to see if it is greater than or less than the threshold value, and then reassigns the input minVal/maxVal to the pixel.\n", "

\n", "\n", "

\n", " It has the following format: \n", "

\n", " \n", "```python\n", "thresh = cv2.threshold(, , , )[1]\n", "```\n", "\n", " \n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #3: Threshold the image. Save as 'thresh'.\n", "# Use threshold_value=240, maxVal=255, and minVal=0\n", "\n", "\n", "# Show image in popup window\n", "cv2.imshow('threshold', thresh)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " STEP #4: Invert the threshold mask\n", "

\n", "\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #4: Invert the threshold mask via cv2.bitwise_not. Save as 'mask'\n", "\n", "\n", "# Show image in popup window\n", "cv2.imshow('mask', mask)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " STEP #5: Use cv2.findContours to find the list of contours.\n", "

\n", "\n", "

\n", " It has the following format:\n", "

\n", " \n", "```python\n", "contours = cv2.findContours(, 3, 2)[0]\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #5: Find contours of 'mask' via cv2.findContours. Save as 'contours'.\n", "\n", "\n", "# Print contours found\n", "print(contours)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " STEP #6: Draw contours via cv2.drawContours.\n", "

\n", "\n", "

\n", " It has the following format:\n", "

\n", " \n", "```python\n", "cv2.drawContours(, , , , )\n", "```\n", "\n", "

\n", " Notes: \n", "
Use -1 for <contour_index> to draw all contours. \n", "
To draw a specific contour, index into the list via [contours[i]] for <contours>, and set 0 for <contour_index>\n", "

\n", "\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #6: Draw GREEN contours via cv2.drawContours. \n", "# Use contour_index=-1 and thickness=3.\n", "\n", "\n", "# Show image in popup window\n", "cv2.imshow('contours', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Exercise: \n", "
Draw blue contours around the opencv logo opencv_logo.png only. Experiment with the first parameter of cv2.threshold. Note that the opencv logo naturally has a white background.\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #1: Read the image 'opencv_logo.png'. Save as 'img'\n", "\n", "\n", "# TASK #2: Convert the image into grayscale via cv2.cvtcolor\n", "\n", "\n", "# TASK #3: Threshold the image via cv2.threshold.\n", "# Use threshold_value=240, maxVal=255, and minVal=0\n", "\n", "\n", "# TASK #4: Find the list of contours of 'thresh' via cv2.findContours\n", "\n", "\n", "# TASK #5: Draw BLUE contours via cv2.drawContours on 'img'\n", "\n", "\n", "\n", "# Show image in popup window\n", "cv2.imshow('opencv contours', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Contour Features and Properties\n", "

\n", " Now that we know a bit more about contours and how to use them, lets extract some data from them! \n", "

\n", "\n", "### Contour Area\n", "

\n", " cv2.contourArea(<single_contour>) calculates the total area that a contour encloses. \n", "

\n", "\n", "

\n", " It is necessary to select a single contour when looking for the area, even when contours only has one contour stored in it.\n", "

\n", " \n", "

\n", " Exercise: \n", "
Find the contour area of star.png\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Get contours of star.png\n", "img = cv2.imread('star.png')\n", "img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", "thresh = cv2.threshold(img_gray, 240, 255, 0)[1]\n", "inverted_thresh = cv2.bitwise_not(thresh)\n", "contours = cv2.findContours(inverted_thresh, 3, 2)[0]\n", "\n", "\n", "# TASK #1: Index into the first contour and save it to 'cnt'.\n", "\n", "\n", "# TASK #2: Get the contour area of the first contour\n", "\n", "\n", "# TASK #3: Print the area\n", "\n", "\n", "\n", "# Show the first contour \n", "cv2.drawContours(img, [cnt], 0, (255, 0, 0), 3)\n", "cv2.imshow('star contour', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Straight Bounding Rectangle\n", "\n", "

\n", " We can use cv2.boundingRect to draw a rectangle. \n", "
It has the following format: \n", "

\n", " \n", " ```python\n", "x, y, w, h = cv2.boundingRect()\n", " ```\n", " \n", " \n", "\n", "

\n", " Exercise: \n", "
Find the straight bounding rectangle of bolt.jpg\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Get contours of bolt.png\n", "img = cv2.imread('bolt.jpg')\n", "img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", "thresh = cv2.threshold(img_gray, 240, 255, 0)[1]\n", "contours = cv2.findContours(thresh, 3, 2)[0]\n", "\n", "\n", "# TASK #1: Get bounding rectangle of the first contour\n", "\n", "\n", "# TASK #2: Draw a GREEN bounding rectangle using cv2.rectangle\n", "\n", "\n", "\n", "# Show bounding rectangle in popup window\n", "cv2.imshow('bounding rectangle', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Minimum Enclosing Rotated Box\n", "\n", "

\n", " Another bounding rectangle is cv2.minAreaRect. This rectangle encloses the contour with the smallest area, and can be rotated.\n", "

\n", " \n", "

\n", " The function has the following format:\n", "

\n", " \n", "```python\n", "rect = cv2.minAreaRect()\n", "```\n", "\n", " \n", "\n", "

\n", " Exercise: \n", "
Find the minimum enclosing rectangle of bolt.jpg\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Get contours of bolt.png\n", "img = cv2.imread('bolt.jpg')\n", "img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", "thresh = cv2.threshold(img_gray, 240, 255, 0)[1]\n", "contours = cv2.findContours(thresh, 3, 2)[0]\n", "\n", "\n", "# TASK: Get min area bounding rectangle of the first contour. \n", "# Save as 'rect'.\n", "\n", "\n", "\n", "# Draw the rectangle\n", "box = np.int0(cv2.boxPoints(rect))\n", "res = cv2.drawContours(img, [box], 0, (0,0,255), 2)\n", "\n", "# Show min area bounded rectangle\n", "cv2.imshow('min bounded rectangle', res)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Optional Note on cv2.minAreaRect:\n", "

\n", "rect = cv2.minAreaRect(<contour>) will output rectangle data so that box_points = cv2.boxPoints(rect) can calculate box points. The box points will be floating data types at first, so we want to use np.int0(box_points) to convert the datatype from floats to integers. Then, we can use cv2.drawContours to draw the minimum enclosed (rotated) box around the image." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Minimum Enclosing Circle\n", "\n", "

\n", " cv2.minEnclosingCircle gives us the center and radius of the minimum enclosing circle of a contour.\n", "

\n", " \n", "

\n", " The function has the following format:\n", "

\n", " \n", "```python\n", "(x,y), radius = cv2.minEnclosingCircle()\n", "```\n", "\n", " \n", "\n", "

\n", " Exercise: \n", "
Find the minimum enclosing circle of bolt.jpg\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Get contours of bolt.png\n", "img = cv2.imread('bolt.jpg')\n", "img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", "thresh = cv2.threshold(img_gray, 240, 255, 0)[1]\n", "contours = cv2.findContours(thresh, 3, 2)[0]\n", "\n", "# TASK #1: Get the minimum enclosing circle of the first contour\n", "\n", "\n", "# Convert to values to int\n", "center = (int(x),int(y))\n", "radius = int(radius)\n", "\n", "# TASK #2: Draw a GREEN bounding circle using cv2.circle\n", "\n", "\n", "# Show image in popup\n", "cv2.imshow('enclosing circle', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "---\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Let's Draw it All!\n", "\n", "

\n", " Let's put everything we've learned altogether now! Draw a straight bounded rectangle, a minimum area rectangle, and a minimum enclosing circle around bolt.jpg. Make sure to use different colors for the different contours!\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #1: Read the image 'bolt.jpg'. Save as 'img'\n", "\n", "\n", "# TASK #2: Convert the image into grayscale \n", "\n", "\n", "# TASK #3: Threshold the image via cv2.threshold.\n", "# Use threshold_value=240, maxVal=255, and minVal=0\n", "\n", "\n", "# TASK #3: Find the contours of thresh\n", "\n", "\n", "# TASK #4: Use cv2.boundingRect to find x, y, w, h\n", "\n", "\n", "# TASK #5: Draw a BLUE straight rectangle \n", "\n", "\n", "# TASK #6: Use cv2.minAreaRect to find the minimum enclosing rectangle. \n", "# Save as 'rect'\n", "\n", "\n", "# Draw minimum enclosing rectangle\n", "box = np.int0(cv2.boxPoints(rect))\n", "res = cv2.drawContours(img, [box], 0, (0,255,0), 2)\n", "\n", "# TASK #7: Use cv2.minEnclosingCircle to find radius and (x,y)\n", "\n", "\n", "# Convert values to int\n", "(x,y) = int(x), int(y)\n", "radius = int(radius)\n", "\n", "# TASK #8: Draw the RED circle \n", "\n", "\n", "# Show the image\n", "cv2.imshow('all contours', img)\n", "close_windows()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "---\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Drawing Detector\n", "

\n", " Let's draw a contour around our very own drawings! \n", "
Here is what we need to do:\n", "

    \n", "
  • Get a piece of white printer paper AND colored marker from a TA.
    • \n", "
  • Draw an image using your colored marker on the printer paper.
    • \n", "
  • Use hsv_select_live to find the hsv threshold values of your drawn image.
    • \n", "
  • Fill in the hsv_lower and hsv_upper values below.
    • \n", "
  • Run the cell block. Your drawings should now be detected via contours.
    • \n", "
  • Try it again with different drawings!
    • \n", "
\n", "

\n", " \n", "

\n", " Note:\n", " Make sure your white piece of paper fills the entire video image, else you might be detecting background images rather than your drawing!\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #1: Run this cell block to find the HSV lower and upper bounds for your drawn image!\n", "hsv_select_live()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# TASK #2: Define the corresponding HSV bounds found from above!\n", "hsv_lower = (None, None, None) # replace None with your values!\n", "hsv_upper = (None, None, None) # replace None with your values!\n", "\n", "\n", "# Show video\n", "color_range = [hsv_lower, hsv_upper]\n", "def find_drawings(frame):\n", " res = detectDrawings(frame, color_range)\n", " cv2.imshow('detected drawings', res)\n", "video(find_drawings)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.1" } }, "nbformat": 4, "nbformat_minor": 4 }