{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# FIRST import all the necessary libraries and modules!\n", "import cv2 # import OpenCV\n", "import numpy as np # import NumPy\n", "\n", "# import instructor made functions \n", "import sys\n", "sys.path.insert(0, '../..')\n", "from utils import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Birdie Mask Lab\n", "\n", "

\n", " One cool thing we can do with OpenCV is object detection, in particular, by colors! \n", "

\n", "\n", "\"mask_blue\"\n", " \n", "

\n", " In this lab, we will learn how to:\n", "

\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Masks\n", "## What is a Mask?\n", "\n", "

\n", " Masks are images. They tell the computer what to cover and what to show.\n", "

\n", "\n", "

\n", " Consider the hacker mask below. \n", "
The only parts that are see-through are the eyes:\n", "

\n", " \n", "\"hacker_mask\"\n", "\n", "

\n", " So when we put that mask on a face... Only the eyes show!\n", "

\n", "\n", "\"hacker_mask_eyes\"\n", "\n", "

\n", " Similar to the face mask above, OpenCV masks: \n", "

\n", "

\n", " \n", "

\n", " Below is an example of a masked image of a hand: \n", "
Discuss! Which part is masked, and which part is unmasked?\n", "

\n", " \n", "\"mask_hand\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Masks in OpenCV\n", " \n", "

\n", " On a mask, there can only be 2 values in each pixel: 0 and 255. \n", "
When we apply a mask to an image, if the pixel value is: \n", "

\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# HSV Masking\n", "\n", "

\n", " HSV tends to be better for tracking an object by color than BGR.\n", "

\n", "

\n", " Why do you think this is?\n", "

\n", "

\n", " We can mask video frames using lower and upper bounds for hue (H), saturation (S), and value (V) to track objects by their color.\n", "

\n", "\n", "![HSV](hsv.png)\n", "\n", "

\n", " Exercise\n", "

\n", "

\n", " Run the code below, and move the HSV trackbar values around to get familiar with how masks work. Press ESC to close the pop-up.\n", "

\n", " \n", "

\n", " You should get a pop-up window that looks something like this:\n", "

\n", "\n", "\"hsv_range_selector\"" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": false }, "outputs": [], "source": [ "hsv_select('birdie.jpg')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Making a Mask" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Different HSV values affect the types of colors we mask!\n", "

\n", " \n", "

\n", " We can generate a mask based on the color specified using the function cv2.inRange. This function takes in the image and lower/upper bounds of the color we want to make a mask of. It has the following format: \n", "

\n", "\n", "```python\n", " mask = cv2.inRange(, , )\n", "```\n", "\n", "

\n", " Exercise\n", "

\n", "

\n", " Let's mask the birdie's feet!\n", "

\n", "

\n", " Hint: Use hsv_select to find the upper and lower HSV bounds!\n", "

\n", "

\n", " Your mask should look something like this:\n", "

\n", "\n", "\"mask_birdie_feet\"\n", "\n", "

\n", " We're going to write and use a function to find the mask so that we can reuse our code.\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Run this function to find the HSV upper and lower bounds!\n", "hsv_select('birdie.jpg')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def mask_birdie(hsv_lower, hsv_upper):\n", " # TASK 1: Read and save 'birdie.jpg'\n", " \n", " \n", " # TASK 2: Use cv2.cvtColor to convert the image to HSV\n", " \n", " \n", " # TASK 3: Create a mask for 'birdie.jpg' with cv2.inRange\n", " \n", " \n", " # TASK 4: Return the mask\n", " \n", " \n", "# TASK 5: Find the HSV lower and upper bounds for the birdie's FEET\n", "hsv_lower_feet = (None, None, None) # Replace None with integers!\n", "hsv_upper_feet = (None, None, None) # Replace None with integers!\n", "\n", "# TASK 6: Call the mask_birdie function (pass in the HSV bounds above!)\n", "\n", "\n", "# TASK 7: Show the mask in a popup window\n", "\n", "\n", "# TASK 8: Close the window\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Applying Masks via Bitwise Operations!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Bitwise functions help us create masks of two or more colors! \n", "

\n", " \n", "

\n", " cv2.bitwise_and show the real colors of a masked image. It has the following format:\n", "

\n", " \n", "```python\n", " color_mask = cv2.bitwise_and(, , mask=)\n", "```\n", " \n", "

\n", " cv2.bitwise_not flips each pixel in a mask (turning True -> False and False -> True). This helps us control the colors we want to show. It has the following format: \n", "

\n", " \n", "```python\n", " inv_mask = cv2.bitwise_not()\n", "```\n", " \n", "

\n", " Below are examples of different bitwise functions, and how they affect images. \n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"mask_bitwise_examples\" " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Applying the Mask: `cv2.bitwise_and`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " We can use cv2.bitwise_and to create a new image where True pixels on the mask will show real colors. \n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Exercise\n", "

\n", "

\n", " Use cv2.bitwise_and to mask birdie.jpg to get its real colors! \n", "

\n", "\n", "

\n", " Your mask should look something like this:\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"mask_bitwiseand_birdiefeet\" " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Useful Notes:\n", "

    \n", "
  • Inputs <image1> and <image2> should be the SAME image. Use the original img.
    • \n", "
  • Use our masked birdie feet from the previous exercise as the input mask for:\n", "
    mask=<input_mask>
    • \n", "
\n", "

" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def color_mask_birdie(mask):\n", " # TASK 1: Read 'birdie.jpg'\n", " img = cv2.imread('birdie.jpg')\n", " \n", " # TASK 2: Use cv2.bitwise_and to create a color mask\n", " \n", " \n", " # TASK 3: Return the color mask\n", " \n", " \n", "# TASK 4: Use the color_mask_birdie function to create a color mask\n", "# Hint: Use the same mask you created before from \"Making a Mask\"! (It should still be saved)\n", "\n", "\n", "# TASK 5: Show the color mask in a popup window\n", "\n", "\n", "# TASK 6: Close the window\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Inverting the Mask: `cv2.bitwise_not`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " Exercise\n", "

\n", "

\n", " Now let's reverse, or invert the mask using cv2.bitwise_not so we get the real color of the background instead of the birdie!\n", "

\n", " \n", "

\n", " Hint: Use the combined mask you made in the Combining Masks exercise!\n", "

\n", "\n", "

\n", " Your mask should look something like this:\n", "

\n", " \n", "\"mask_bitwisenot_birdiebody\" " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Make sure to run the previous cell blocks first!\n", "# TASK 1: Use cv2.bitwise_not to invert the mask. Use the mask you made from \"Making a Mask\".\n", "\n", "\n", "# TASK 2: Use color_mask_birdie to show the real colors\n", "\n", "\n", "# TASK 3: Show the masked image in a popup window\n", "\n", "\n", "# TASK 4: Close the window \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", " CONGRATS!!! You can now pick out colors from images!\n", "

\n", "

\n", " In the next lab, using the tools we've learned so far, we will make our very own green screens! \n", "

" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.1" } }, "nbformat": 4, "nbformat_minor": 4 }