{"cells":[{"cell_type":"markdown","metadata":{"id":"Go5NMXdNVRTm"},"source":["**Name:** \\_\\_\\_\\_\\_\n","\n","**EID:** \\_\\_\\_\\_\\_"]},{"cell_type":"markdown","metadata":{"id":"Zo0aXFwvVRTn"},"source":["# Tutorial 3: Face Detection in Images\n","\n","In this tutorial you will train support vector machine and kernel support vector machine classifiers to detect whether there is a face in a small image patch.\n","\n","First we need to initialize Python. Run the below cell."]},{"cell_type":"code","execution_count":1,"metadata":{"id":"SyLU36qOVRTn","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728476565861,"user_tz":-480,"elapsed":6089,"user":{"displayName":"mian zou","userId":"03736536006696926061"}},"outputId":"e11600af-8934-44a5-d8aa-87f172ac209d"},"outputs":[{"output_type":"stream","name":"stderr","text":[":4: DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`\n"," IPython.core.display.set_matplotlib_formats(\"svg\")\n"]}],"source":["%matplotlib inline\n","import IPython.core.display\n","# setup output image format (Chrome works best)\n","IPython.core.display.set_matplotlib_formats(\"svg\")\n","import matplotlib.pyplot as plt\n","import matplotlib\n","from numpy import *\n","from sklearn import *\n","import os\n","import zipfile\n","import fnmatch\n","random.seed(4487)\n","from scipy import ndimage\n","from scipy import signal\n","import skimage.color\n","import skimage.exposure\n","import skimage.io\n","import skimage.util"]},{"cell_type":"markdown","metadata":{"id":"TYbTjBMfVRTo"},"source":["## 1. Loading Data and Pre-processing\n","Next we need to load the images. Download `faces.zip`, and put it in the same direcotry as this ipynb file. **Do not unzip the file.** Then run the following cell to load the images."]},{"cell_type":"code","source":["from google.colab import drive\n","drive.mount('/content/drive')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lR6wQp6ePP2R","executionInfo":{"status":"ok","timestamp":1728476705423,"user_tz":-480,"elapsed":27868,"user":{"displayName":"mian zou","userId":"03736536006696926061"}},"outputId":"abe928e2-6b84-437a-ee0f-ff2ea7dd3c46"},"execution_count":2,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}]},{"cell_type":"code","execution_count":3,"metadata":{"id":"SnrImE6dVRTo","outputId":"8eef64a8-aa8b-40c7-fb4c-3367b938add0","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728476733192,"user_tz":-480,"elapsed":6209,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["1745\n","944\n"]}],"source":["imgdata = {'train':[], 'test':[]}\n","classes = {'train':[], 'test':[]}\n","\n","# the dataset is too big, so subsample the training and test sets...\n","# reduce training set by a factor of 4\n","train_subsample = 4\n","train_counter = [0, 0]\n","# maximum number of samples in each class for test set\n","test_maxsample = 472\n","test_counter = [0, 0]\n","\n","# load the zip file\n","filename = '/content/drive/MyDrive/tutorial/faces.zip' #'faces.zip'\n","zfile = zipfile.ZipFile(filename, 'r')\n","\n","for name in zfile.namelist():\n"," # check file name matches\n"," if fnmatch.fnmatch(name, \"faces/*/*/*.png\"):\n","\n"," # filename is : faces/train/face/fname.png\n"," (fdir1, fname) = os.path.split(name) # get file name\n"," (fdir2, fclass) = os.path.split(fdir1) # get class (face, nonface)\n"," (fdir3, fset) = os.path.split(fdir2) # get training/test set\n"," # class 1 = face; class 0 = non-face\n"," myclass = int(fclass == \"face\")\n","\n"," loadme = False\n"," if fset == 'train':\n"," if (train_counter[myclass] % train_subsample) == 0:\n"," loadme = True\n"," train_counter[myclass] += 1\n"," elif fset == 'test':\n"," if test_counter[myclass] < test_maxsample:\n"," loadme = True\n"," test_counter[myclass] += 1\n","\n"," if (loadme):\n"," # open file in memory, and parse as an image\n"," myfile = zfile.open(name)\n"," #img = matplotlib.image.imread(myfile)\n"," img = skimage.io.imread(myfile)\n"," # convert to grayscale\n"," img = skimage.color.rgb2gray(skimage.color.rgba2rgb(img))\n"," myfile.close()\n","\n"," # append data\n"," imgdata[fset].append(img)\n"," classes[fset].append(myclass)\n","\n","\n","zfile.close()\n","imgsize = img.shape\n","\n","print(len(imgdata['train']))\n","print(len(imgdata['test']))\n","trainclass2start = sum(classes['train'])"]},{"cell_type":"markdown","metadata":{"id":"PM4sQyPxVRTp"},"source":["Each image is a 19x19 array of pixel values. Run the below code to show an example:"]},{"cell_type":"code","execution_count":4,"metadata":{"id":"Z2yyn2eWVRTp","outputId":"72e78c08-0115-4fa7-80cb-8091fd7d22c7","colab":{"base_uri":"https://localhost:8080/","height":317},"executionInfo":{"status":"ok","timestamp":1728476738457,"user_tz":-480,"elapsed":736,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["(19, 19)\n"]},{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:25:38.031430\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}}],"source":["print(img.shape)\n","plt.subplot(1,2,1)\n","plt.imshow(imgdata['train'][0], cmap='gray', interpolation='nearest')\n","plt.title(\"face sample\")\n","plt.subplot(1,2,2)\n","plt.imshow(imgdata['train'][trainclass2start], cmap='gray', interpolation='nearest')\n","plt.title(\"non-face sample\")\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"n1n0Ln0rVRTp"},"source":["Run the below code to show more images!"]},{"cell_type":"code","execution_count":5,"metadata":{"id":"wlvGnyYYVRTp","outputId":"6ef0bd87-2f2b-4618-a9db-93bbc818ca90","colab":{"base_uri":"https://localhost:8080/","height":669},"executionInfo":{"status":"ok","timestamp":1728476743186,"user_tz":-480,"elapsed":1130,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:25:42.094684\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}}],"source":["# function to make an image montage\n","def image_montage(X, imsize=None, maxw=10):\n"," \"\"\"X can be a list of images, or a matrix of vectorized images.\n"," Specify imsize when X is a matrix.\"\"\"\n"," tmp = []\n"," numimgs = len(X)\n","\n"," # create a list of images (reshape if necessary)\n"," for i in range(0,numimgs):\n"," if imsize != None:\n"," tmp.append(X[i].reshape(imsize))\n"," else:\n"," tmp.append(X[i])\n","\n"," # add blanks\n"," if (numimgs > maxw) and (mod(numimgs, maxw) > 0):\n"," leftover = maxw - mod(numimgs, maxw)\n"," meanimg = 0.5*(X[0].max()+X[0].min())\n"," for i in range(0,leftover):\n"," tmp.append(ones(tmp[0].shape)*meanimg)\n","\n"," # make the montage\n"," tmp2 = []\n"," for i in range(0,len(tmp),maxw):\n"," tmp2.append( hstack(tmp[i:i+maxw]) )\n"," montimg = vstack(tmp2)\n"," return montimg\n","\n","# show a few images\n","plt.figure(figsize=(9,9))\n","plt.imshow(image_montage(imgdata['train'][::20]), cmap='gray', interpolation='nearest')\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"YrSNDjl-VRTp"},"source":["Each image is a 2d array, but the classifier algorithms work on 1d vectors. Run the following code to convert all the images into 1d vectors by flattening. The result should be a matrix where each row is a flattened image."]},{"cell_type":"code","execution_count":6,"metadata":{"id":"Kf3BetPIVRTp","outputId":"f11945ff-488c-4431-f00b-583d179d1fff","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477080802,"user_tz":-480,"elapsed":336,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["(1745, 361)\n","(1745,)\n","(944, 361)\n","(944,)\n"]}],"source":["trainX = empty((len(imgdata['train']), prod(imgsize)))\n","for i,img in enumerate(imgdata['train']):\n"," trainX[i,:] = ravel(img)\n","trainY = asarray(classes['train']) # convert list to numpy array\n","print(trainX.shape)\n","print(trainY.shape)\n","\n","testX = empty((len(imgdata['test']), prod(imgsize)))\n","for i,img in enumerate(imgdata['test']):\n"," testX[i,:] = ravel(img)\n","testY = asarray(classes['test']) # convert list to numpy array\n","print(testX.shape)\n","print(testY.shape)"]},{"cell_type":"markdown","metadata":{"id":"YrGk4pjFVRTq"},"source":["## 2. Detection Using Pixel Values\n","\n","Train kernel SVM using either RBF or polynomia kernel classifiers to classify an image patch as face or non-face. Evaluate all classifiers on the test set.\n","\n","Normalize the features and setup all the parameters and models."]},{"cell_type":"code","execution_count":7,"metadata":{"id":"-kV9tY6CVRTq","executionInfo":{"status":"ok","timestamp":1728477108751,"user_tz":-480,"elapsed":341,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["# Normalization: sklearn.preprocessing.MinMaxScaler()\n","scaler = preprocessing.MinMaxScaler(feature_range=(-1,1)) # make scaling object\n","trainXn = scaler.fit_transform(trainX) # use training data to fit scaling parameters\n","testXn = scaler.transform(testX) # apply scaling to test data\n","\n","clfs = {}"]},{"cell_type":"code","execution_count":8,"metadata":{"id":"NhqZVV9TVRTq","executionInfo":{"status":"ok","timestamp":1728477114960,"user_tz":-480,"elapsed":349,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["# setup all the parameters and models\n","exps = {\n"," 'svm-lin': {\n"," 'paramgrid': {'C': logspace(-2,3,10)},\n"," 'clf': svm.SVC(kernel='linear') },\n"," 'svm-rbf': {\n"," 'paramgrid': {'C': logspace(-2,3,10), 'gamma': logspace(-4,3,10) },\n"," 'clf': svm.SVC(kernel='rbf') },\n"," 'svm-poly': {\n"," 'paramgrid': {'C': logspace(-2,3,10), 'degree': [2, 3, 4] },\n"," 'clf': svm.SVC(kernel='poly') },\n"," }"]},{"cell_type":"markdown","metadata":{"id":"UnSvYw8SVRTq"},"source":["Try to train each classifier and show the parameters."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"-EvpuOUbVRTq"},"outputs":[],"source":["### INSERT YOUR CODE HERE\n","## HINT\n","## 1. Classfiers: sklearn.model_selection.GridSearchCV()"]},{"cell_type":"code","execution_count":9,"metadata":{"id":"Q_Ph0qd-VRTq","outputId":"ce7dae91-f4d2-488f-a235-d3f93dede134","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477407358,"user_tz":-480,"elapsed":250359,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["=== svm-lin ===\n","Fitting 5 folds for each of 10 candidates, totalling 50 fits\n","best params: {'C': 0.01}\n","=== svm-rbf ===\n","Fitting 5 folds for each of 100 candidates, totalling 500 fits\n","best params: {'C': 1.6681005372000592, 'gamma': 0.021544346900318846}\n","=== svm-poly ===\n","Fitting 5 folds for each of 30 candidates, totalling 150 fits\n","best params: {'C': 5.994842503189409, 'degree': 2}\n"]}],"source":["# run the experiment\n","for (name,ex) in exps.items():\n"," print(\"=== \" + name + \" ===\")\n"," if name in clfs:\n"," print(\"exists skipping\")\n"," else:\n"," myclf = model_selection.GridSearchCV(ex['clf'], ex['paramgrid'], cv=5, verbose=1, n_jobs=-1)\n"," myclf.fit(trainXn, trainY)\n","\n"," print(\"best params:\", myclf.best_params_)\n","\n"," clfs[name] = myclf"]},{"cell_type":"markdown","metadata":{"id":"gkOcog46VRTq"},"source":["Calculate the training and test accuracy for the each classifier."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"IJSxuAJxVRTq"},"outputs":[],"source":["### INSERT YOUR CODE HERE\n","## HINT\n","# 1. clf.predict()\n","# 2. metrics.accuracy_score()"]},{"cell_type":"code","execution_count":10,"metadata":{"id":"wiTf6c-AVRTq","outputId":"1b5da5b8-8a57-46e1-ba39-0bc6ab581caf","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477427045,"user_tz":-480,"elapsed":2080,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["svm-lin : train=0.983; test=0.643\n","svm-rbf : train=1.000; test=0.658\n","svm-poly : train=1.000; test=0.662\n"]}],"source":["### INSERT YOUR CODE HERE\n","predYtrain = {}\n","predYtest = {}\n","for (name,clf) in clfs.items():\n"," predYtrain[name] = clf.predict(trainXn)\n"," predYtest[name] = clf.predict(testXn)\n","\n"," # calculate accuracy\n"," trainacc = metrics.accuracy_score(trainY, predYtrain[name])\n"," testacc = metrics.accuracy_score(testY, predYtest[name])\n","\n"," print(\"{:10s}: train={:.3f}; test={:.3f}\".format(name, trainacc, testacc))"]},{"cell_type":"code","execution_count":11,"metadata":{"id":"X6hEC3vbVRTr","executionInfo":{"status":"ok","timestamp":1728477436760,"user_tz":-480,"elapsed":356,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["# set variables for later\n","predY = predYtest['svm-poly']\n","#adaclf = clfs['ada'].best_estimator_\n","svmclf_rbf = clfs['svm-rbf'].best_estimator_\n","svmclf_poly = clfs['svm-poly'].best_estimator_\n","#rfclf = clfs['rf'].best_estimator_"]},{"cell_type":"markdown","metadata":{"id":"mP1JTyNTVRTr"},"source":["_Which classifier was best?_\n","- **INSERT YOUR ANSWER HERE**"]},{"cell_type":"markdown","metadata":{"id":"9X-gaBnfVRTr"},"source":["- **INSERT YOUR ANSWER HERE**\n","- SVM polynomial"]},{"cell_type":"markdown","metadata":{"id":"KyOnKMd5VRTr"},"source":["### Error analysis\n","The accuracy only tells part of the classifier's performance. We can also look at the different types of errors that the classifier makes:\n","- _True Positive (TP)_: classifier correctly said face\n","- _True Negative (TN)_: classifier correctly said non-face\n","- _False Positive (FP)_: classifier said face, but not a face\n","- _False Negative (FN)_: classifier said non-face, but was a face\n","\n","This is summarized in the following table:\n","\n","\n","\n","\n","\n","\n","
Actual
FaceNon-face
PredictionFaceTrue Positive (TP)False Positive (FP)
Non-faceFalse Negative (FN)True Negative (TN)
\n","\n","We can then look at the _true positive rate_ and the _false positive rate_.\n","- _true positive rate (TPR)_: proportion of true faces that were correctly detected\n","- _false positive rate (FPR)_: proportion of non-faces that were mis-classified as faces.\n","\n","Use the below code to calculate the TPR and FPR of your classifiers."]},{"cell_type":"code","execution_count":12,"metadata":{"id":"KRF1tdxOVRTr","outputId":"0aa98a7a-3494-424b-c813-be33cde1f584","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477444332,"user_tz":-480,"elapsed":335,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["TP= 154\n","FP= 1\n","TN= 471\n","FN= 318\n","TPR= 0.326271186440678\n","FPR= 0.00211864406779661\n"]}],"source":["# predY is the prediction from the classifier\n","\n","Pind = where(testY==1) # indicies for face\n","Nind = where(testY==0) # indicies for non-face\n","\n","TP = count_nonzero(testY[Pind] == predY[Pind])\n","FN = count_nonzero(testY[Pind] != predY[Pind])\n","TN = count_nonzero(testY[Nind] == predY[Nind])\n","FP = count_nonzero(testY[Nind] != predY[Nind])\n","\n","TPR = TP / (TP+FN)\n","FPR = FP / (FP+TN)\n","\n","print(\"TP=\", TP)\n","print(\"FP=\", FP)\n","print(\"TN=\", TN)\n","print(\"FN=\", FN)\n","print(\"TPR=\", TPR)\n","print(\"FPR=\", FPR)"]},{"cell_type":"markdown","metadata":{"id":"mA-NsAe4VRTr"},"source":["_How does the classifier make errors?_\n","- **INSERT YOUR ANSWER HERE**"]},{"cell_type":"markdown","metadata":{"id":"xHp5aBUPVRTr"},"source":["- **INSERT YOUR ANSWER HERE**\n","- high TPR, means it gets all the faces."]},{"cell_type":"markdown","metadata":{"id":"trBQaLoiVRTr"},"source":["For kernel SVM, we can look at the support vectors to see what the classifier finds difficult."]},{"cell_type":"code","execution_count":13,"metadata":{"id":"DAQM2kAnVRTr","outputId":"fea45962-25bc-4113-f2a9-551f9bfba896","colab":{"base_uri":"https://localhost:8080/","height":637},"executionInfo":{"status":"ok","timestamp":1728477480605,"user_tz":-480,"elapsed":1716,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["num support vectors: 307\n"]},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":13},{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:37:59.158989\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}}],"source":["# svmclf is the trained SVM classifier\n","\n","print(\"num support vectors:\", len(svmclf_poly.support_vectors_))\n","si = svmclf_poly.support_ # get indicies of support vectors\n","\n","# get all the patches for each support vector\n","simg = [ imgdata['train'][i] for i in si ]\n","\n","# make montage\n","outimg = image_montage(simg, maxw=20)\n","\n","plt.figure(figsize=(9,9))\n","plt.imshow(outimg, cmap='gray', interpolation='nearest')"]},{"cell_type":"markdown","metadata":{"id":"vzoUCyf9VRTr"},"source":["Comment on anything you notice about what the SVM finds difficult (i.e., on the decision boundary or within the margin)\n","- **INSERT YOUR ANSWER HERE**"]},{"cell_type":"markdown","metadata":{"id":"S13vZwt4VRTr"},"source":["- **INSERT YOUR ANSWER HERE**\n","- glasses on faces. some non-faces look like faces."]},{"cell_type":"markdown","metadata":{"id":"C_B9VQQkVRTr"},"source":["## 3. Detection using Image Feature\n","\n","### Image Feature Extraction\n","The detection performance is not that good using pixel values. The problem is that we are using the raw pixel values as features, so it is difficult for the classifier to interpret larger structures of the face that might be important. To fix the problem, we will extract features from the image using a set of filters.\n","\n","Run the below code to look at the filter output. The filters are a sets of black and white boxes that respond to similar structures in the image. After applying the filters to the image, the filter response map is aggregated over a 4x4 window. Hence each filter produces a 5x5 feature response. Since there are 4 filters, then the feature vector is 100 dimensions."]},{"cell_type":"code","execution_count":14,"metadata":{"id":"HLiXAEyYVRTr","executionInfo":{"status":"ok","timestamp":1728477539648,"user_tz":-480,"elapsed":347,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["def extract_features(imgs, doplot=False):\n"," # the filter layout\n"," lay = [array([-1,1]), array([-1,1,-1]),\n"," array([[1],[-1]]), array([[-1],[1],[-1]])]\n"," sc=8 # size of each filter patch\n"," poolmode = 'i' # pooling mode (interpolate)\n"," cmode = 'same' # convolution mode\n"," brick = ones((sc,sc)) # filter patch\n"," ks = []\n"," for l in lay:\n"," tmp = [brick*i for i in l]\n"," if (l.ndim==1):\n"," k = hstack(tmp)\n"," else:\n"," k = vstack(tmp)\n"," ks.append(k)\n","\n"," # get the filter response size\n"," if (poolmode=='max') or (poolmode=='absmax'):\n"," tmpimg = maxpool(maxpool(imgs[0]))\n"," else:\n"," tmpimg = ndimage.interpolation.zoom(imgs[0], 0.25)\n"," fs = prod(tmpimg.shape)\n","\n"," # get the total feature length\n"," fst = fs*len(ks)\n","\n"," # filter the images\n"," X = empty((len(imgs), fst))\n"," for i,img in enumerate(imgs):\n"," x = empty(fst)\n","\n"," # for each filter\n"," for j,th in enumerate(ks):\n"," # filter the image\n"," imgk = signal.convolve(img, ks[j], mode=cmode)\n","\n"," # do pooling\n"," if poolmode == 'maxabs':\n"," mimg = maxpool(maxpool(abs(imgk)))\n"," elif poolmode == 'max':\n"," mimg = maxpool(maxpool(imgk))\n"," else:\n"," mimg = ndimage.interpolation.zoom(imgk, 0.25)\n","\n"," # put responses into feature vector\n"," x[(j*fs):(j+1)*fs] = ravel(mimg)\n","\n"," if (doplot):\n"," plt.subplot(3,len(ks),j+1)\n"," plt.imshow(ks[j], cmap='gray', interpolation='nearest')\n"," plt.title(\"filter \" + str(j))\n"," plt.subplot(3,len(ks),len(ks)+j+1)\n"," plt.imshow(imgk, cmap='gray', interpolation='nearest')\n"," plt.title(\"filtered image\")\n"," plt.subplot(3,len(ks),2*len(ks)+j+1)\n"," plt.imshow(mimg, cmap='gray', interpolation='nearest')\n"," plt.title(\"image features\")\n"," X[i,:] = x\n","\n"," return X"]},{"cell_type":"code","execution_count":15,"metadata":{"id":"kbtmmg_pVRTr","outputId":"d8ee0b2d-907b-4f63-efd0-0e330bbb71bb","colab":{"base_uri":"https://localhost:8080/","height":1000},"executionInfo":{"status":"ok","timestamp":1728477544562,"user_tz":-480,"elapsed":3500,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stderr","text":[":22: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," tmpimg = ndimage.interpolation.zoom(imgs[0], 0.25)\n",":44: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," mimg = ndimage.interpolation.zoom(imgk, 0.25)\n"]},{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:39:02.009378\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:39:03.400975\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}}],"source":["# new features\n","img = imgdata['train'][0]\n","plt.imshow(img, cmap='gray', interpolation='nearest')\n","plt.title(\"image\")\n","plt.figure(figsize=(9,9))\n","extract_features([img], doplot=True);"]},{"cell_type":"markdown","metadata":{"id":"rESFHlDhVRTs"},"source":["Now lets extract image features on the training and test sets. It may take a few seconds."]},{"cell_type":"code","execution_count":16,"metadata":{"scrolled":true,"id":"CWVryn92VRTs","outputId":"15e13087-c27f-4490-b7f6-dad92fbc6360","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477563125,"user_tz":-480,"elapsed":6252,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stderr","text":[":22: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," tmpimg = ndimage.interpolation.zoom(imgs[0], 0.25)\n",":44: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," mimg = ndimage.interpolation.zoom(imgk, 0.25)\n"]},{"output_type":"stream","name":"stdout","text":["(1745, 100)\n","(944, 100)\n"]}],"source":["trainXf = extract_features(imgdata['train'])\n","print(trainXf.shape)\n","testXf = extract_features(imgdata['test'])\n","print(testXf.shape)"]},{"cell_type":"markdown","metadata":{"id":"yKSvB5_RVRTs"},"source":["### Detection\n","Now train AdaBoost and SVM classifiers on the image feature data. Evaluate on the test set."]},{"cell_type":"code","execution_count":17,"metadata":{"id":"taPIvr5hVRTs","executionInfo":{"status":"ok","timestamp":1728477568213,"user_tz":-480,"elapsed":412,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["### Nomalization\n","scalerf = preprocessing.MinMaxScaler(feature_range=(-1,1)) # make scaling object\n","trainXfn = scalerf.fit_transform(trainXf) # use training data to fit scaling parameters\n","testXfn = scalerf.transform(testXf) # apply scaling to test data\n","\n","clfs2 = {}"]},{"cell_type":"code","execution_count":18,"metadata":{"id":"JH-ZlpQHVRTs","executionInfo":{"status":"ok","timestamp":1728477571418,"user_tz":-480,"elapsed":332,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["# setup all the parameters and models\n","exps = {\n"," 'svm-lin': {\n"," 'paramgrid': {'C': logspace(-2,3,10)},\n"," 'clf': svm.SVC(kernel='linear') },\n"," 'svm-rbf': {\n"," 'paramgrid': {'C': logspace(-2,3,10), 'gamma': logspace(-4,3,10) },\n"," 'clf': svm.SVC(kernel='rbf') },\n"," 'svm-poly': {\n"," 'paramgrid': {'C': logspace(-2,3,10), 'degree': [2, 3, 4] },\n"," 'clf': svm.SVC(kernel='poly') },\n"," }\n","clfs2 = {}"]},{"cell_type":"markdown","metadata":{"id":"EQVYSwRWVRTs"},"source":["Try to train each classifier and show the parameters."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"WLGbjVUzVRTs"},"outputs":[],"source":["### INSERT YOUR CODE HERE\n","## HINT\n","## Classfiers: sklearn.model_selection.GridSearchCV()"]},{"cell_type":"code","execution_count":19,"metadata":{"id":"kXqQ4D9kVRTs","outputId":"9a2dc6e8-9f1a-4fe0-b1fc-68e131d0be90","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477713776,"user_tz":-480,"elapsed":119133,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["=== svm-lin ===\n","Fitting 5 folds for each of 10 candidates, totalling 50 fits\n","best params: {'C': 5.994842503189409}\n","=== svm-rbf ===\n","Fitting 5 folds for each of 100 candidates, totalling 500 fits\n","best params: {'C': 278.2559402207126, 'gamma': 0.021544346900318846}\n","=== svm-poly ===\n","Fitting 5 folds for each of 30 candidates, totalling 150 fits\n","best params: {'C': 21.544346900318846, 'degree': 2}\n"]}],"source":["### INSERT YOUR CODE HERE\n","# run the experiment\n","for (name,ex) in exps.items():\n"," print(\"=== \" + name + \" ===\")\n"," if name in clfs2:\n"," print(\"exists skipping\")\n"," else:\n"," myclf = model_selection.GridSearchCV(ex['clf'], ex['paramgrid'], cv=5, verbose=1, n_jobs=-1)\n"," myclf.fit(trainXfn, trainY)\n","\n"," print(\"best params:\", myclf.best_params_)\n","\n"," clfs2[name] = myclf"]},{"cell_type":"markdown","metadata":{"id":"xhcvXFMjVRTs"},"source":["Calculate the training and test accuracy for the each classifier."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"L7IEkQCdVRTs"},"outputs":[],"source":["### INSERT YOUR CODE HERE"]},{"cell_type":"code","execution_count":20,"metadata":{"id":"VpBVfuw_VRTs","outputId":"4d1c0136-b42b-4927-c491-357d9b100c6f","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477724401,"user_tz":-480,"elapsed":342,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["svm-lin : train=0.962; test=0.713\n","svm-rbf : train=0.998; test=0.734\n","svm-poly : train=0.991; test=0.754\n"]}],"source":["### INSERT YOUR CODE HERE\n","# Hint: Refer to last tutoiral\n","predYtrain = {}\n","predYtest = {}\n","for (name,clf) in clfs2.items():\n"," predYtrain[name] = clf.predict(trainXfn)\n"," predYtest[name] = clf.predict(testXfn)\n","\n"," # calculate accuracy\n"," trainacc = metrics.accuracy_score(trainY, predYtrain[name])\n"," testacc = metrics.accuracy_score(testY, predYtest[name])\n","\n"," print(\"{:10s}: train={:.3f}; test={:.3f}\".format(name, trainacc, testacc))"]},{"cell_type":"code","execution_count":21,"metadata":{"id":"TdIOHqcbVRTt","executionInfo":{"status":"ok","timestamp":1728477730014,"user_tz":-480,"elapsed":1,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["testY = predYtest['svm-rbf']\n","bestclf = clfs2['svm-rbf']"]},{"cell_type":"markdown","metadata":{"id":"Il5IJ95KVRTt"},"source":["### Error Analysis\n","Similar to before, repeat the error analysis for the new classifiers."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"oZ80yqabVRTt"},"outputs":[],"source":["### INSERT YOUR CODE HERE\n"]},{"cell_type":"code","execution_count":22,"metadata":{"id":"bCn9ijFJVRTt","outputId":"d09ce66d-68bb-4e70-e031-cd3fd39b925e","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477733825,"user_tz":-480,"elapsed":341,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["TP= 137\n","FP= 18\n","TN= 679\n","FN= 110\n","TPR= 0.5546558704453441\n","FPR= 0.02582496413199426\n"]}],"source":["### INSERT YOUR CODE HERE\n","\n","Pind = where(testY==1) # indicies for face\n","Nind = where(testY==0) # indicies for non-face\n","TP = sum(testY[Pind] == predY[Pind])\n","FN = sum(testY[Pind] != predY[Pind])\n","TN = sum(testY[Nind] == predY[Nind])\n","FP = sum(testY[Nind] != predY[Nind])\n","\n","\n","TPR = TP / (TP+FN)\n","FPR = FP / (FP+TN)\n","\n","print(\"TP=\", TP)\n","print(\"FP=\", FP)\n","print(\"TN=\", TN)\n","print(\"FN=\", FN)\n","print(\"TPR=\", TPR)\n","print(\"FPR=\", FPR)"]},{"cell_type":"markdown","metadata":{"id":"9bh7ckg9VRTt"},"source":["How has the classifier using image features improved?\n","- **INSERT YOUR ANSWER HERE**"]},{"cell_type":"markdown","metadata":{"id":"jv9gfR90VRTt"},"source":["- **INSERT YOUR ANSWER HERE**\n","- higher TPR but lower FPR"]},{"cell_type":"markdown","metadata":{"id":"D7mkK18cVRTt"},"source":["# Test image\n","Now let us try your face detector on a real image. Download the \"nasa-small.png\" image and put it in the same directory as your ipynb file. The below code will load the image, crop out image patches and then extract features. (this may take a few minutes)"]},{"cell_type":"code","execution_count":27,"metadata":{"id":"RNYCRDVpVRTt","executionInfo":{"status":"ok","timestamp":1728477841319,"user_tz":-480,"elapsed":329,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["fname = '/content/drive/MyDrive/tutorial/nasa-small.png'"]},{"cell_type":"code","execution_count":28,"metadata":{"id":"qKLgmqJLVRTt","outputId":"8a919a04-b7bb-4f26-b433-cfbfa4301fe3","colab":{"base_uri":"https://localhost:8080/","height":310},"executionInfo":{"status":"ok","timestamp":1728477845936,"user_tz":-480,"elapsed":2558,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["(210, 480)\n"]},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":28},{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:44:04.405097\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}}],"source":["# load image\n","testimg3 = skimage.io.imread(fname)\n","\n","# convert to grayscale\n","#testimg = skimage.color.rgb2gray(testimg3)\n","testimg = skimage.color.rgb2gray(skimage.color.rgba2rgb(testimg3))\n","print(testimg.shape)\n","plt.imshow(testimg, cmap='gray')"]},{"cell_type":"code","execution_count":29,"metadata":{"id":"oacM7WIqVRTt","outputId":"d5ede042-0307-43bd-8cc2-082bdd2de48e","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1728477861641,"user_tz":-480,"elapsed":11931,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["(48, 116, 19, 19)\n","(5568, 19, 19)\n"]},{"output_type":"stream","name":"stderr","text":[":22: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," tmpimg = ndimage.interpolation.zoom(imgs[0], 0.25)\n",":44: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," mimg = ndimage.interpolation.zoom(imgk, 0.25)\n"]}],"source":["# step size for the sliding window\n","step = 4\n","\n","# extract window patches with step size of 4\n","patches = skimage.util.view_as_windows(testimg, (19,19), step=step)\n","psize = patches.shape\n","print(psize)\n","# collapse the first 2 dimensions\n","patches2 = patches.reshape((psize[0]*psize[1], psize[2], psize[3]))\n","print(patches2.shape )\n","\n","# histogram equalize patches (improves contrast)\n","patches3 = empty(patches2.shape)\n","for i in range(patches2.shape[0]):\n"," patches3[i,:,:] = skimage.exposure.equalize_hist(patches2[i,:,:])\n","\n","# extract features\n","newXf = extract_features(patches3)"]},{"cell_type":"markdown","metadata":{"id":"tmBPuI2iVRTt"},"source":["Now predict using your classifier. The extracted features are in `newXf`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"9-OES8tcVRTt"},"outputs":[],"source":["### INSERT YOUR CODE HERE\n","## HINT\n","## 1. Apply scaling to test data"]},{"cell_type":"code","execution_count":30,"metadata":{"id":"gB-TYSIiVRTt","executionInfo":{"status":"ok","timestamp":1728477864733,"user_tz":-480,"elapsed":348,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[],"source":["### INSERT YOUR CODE HERE\n","newXfn = scalerf.transform(newXf) # apply scaling to test data\n","prednewY = bestclf.predict(newXfn)"]},{"cell_type":"markdown","metadata":{"collapsed":true,"id":"zoa5fkAbVRTt"},"source":["Now we we will view the results on the image. Use the below code. `prednewY` is the vector of predictions."]},{"cell_type":"code","execution_count":31,"metadata":{"id":"z7Jr0A_mVRTu","outputId":"7cb3ad6d-3df1-4132-939e-8af17932b10d","colab":{"base_uri":"https://localhost:8080/","height":791},"executionInfo":{"status":"ok","timestamp":1728477868974,"user_tz":-480,"elapsed":1812,"user":{"displayName":"mian zou","userId":"03736536006696926061"}}},"outputs":[{"output_type":"stream","name":"stderr","text":[":5: DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.\n"," imgY2 = ndimage.interpolation.zoom(imgY, step, output=None, order=0)\n"]},{"output_type":"execute_result","data":{"text/plain":["(-0.5, 479.5, 209.5, -0.5)"]},"metadata":{},"execution_count":31},{"output_type":"display_data","data":{"text/plain":["
"],"image/svg+xml":"\n\n\n \n \n \n \n 2024-10-09T12:44:27.511470\n image/svg+xml\n \n \n Matplotlib v3.7.1, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n"},"metadata":{}}],"source":["# reshape prediction to an image\n","imgY = prednewY.reshape(psize[0], psize[1])\n","\n","# zoom back to image size\n","imgY2 = ndimage.interpolation.zoom(imgY, step, output=None, order=0)\n","# pad the top and left with half the window size\n","imgY2 = vstack((zeros((9, imgY2.shape[1])), imgY2))\n","imgY2 = hstack((zeros((imgY2.shape[0],9)), imgY2))\n","# pad right and bottom to same size as image\n","if (imgY2.shape[0] != testimg.shape[0]):\n"," imgY2 = vstack((imgY2, zeros((testimg.shape[0]-imgY2.shape[0], imgY2.shape[1]))))\n","if (imgY2.shape[1] != testimg.shape[1]):\n"," imgY2 = hstack((imgY2, zeros((imgY2.shape[0],testimg.shape[1]-imgY2.shape[1]))))\n","\n","# show detections with image\n","#detimg = dstack(((0.5*imgY2+0.5)*testimg, 0.5*testimg, 0.5*testimg))\n","nimgY2 = 1-imgY2\n","tmp = nimgY2*testimg\n","detimg = dstack((imgY2+tmp, tmp, tmp))\n","\n","# show it!\n","plt.figure(figsize=(9,9))\n","plt.subplot(2,1,1)\n","plt.imshow(imgY2, interpolation='nearest')\n","plt.title('detection map')\n","plt.subplot(2,1,2)\n","plt.imshow(detimg)\n","plt.title('image')\n","plt.axis('image')"]},{"cell_type":"markdown","metadata":{"id":"XQr-VC1aVRTu"},"source":["_How did your face detector do?_\n","- **INSERT YOUR ANSWER HERE**"]},{"cell_type":"markdown","metadata":{"id":"uPrO1qntVRTu"},"source":["- **INSERT YOUR ANSWER HERE**\n","- got some faces, but also had lots of FP!"]},{"cell_type":"markdown","metadata":{"id":"-ic3IwDRVRTu"},"source":["- You can try it on your own images. The faces should all be around 19x19 pixels though.\n","- We only used 1/4 of the training data. Try using more data to train it!"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"CO-rJ6AHVRTu"},"outputs":[],"source":[]},{"cell_type":"code","execution_count":null,"metadata":{"id":"UkUN-XdCVRTu"},"outputs":[],"source":[]},{"cell_type":"code","execution_count":null,"metadata":{"id":"916Ni_b_VRTu"},"outputs":[],"source":[]},{"cell_type":"code","execution_count":null,"metadata":{"id":"8enKqIksVRTu"},"outputs":[],"source":[]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Vv8paUx2VRTu"},"outputs":[],"source":[]}],"metadata":{"anaconda-cloud":{},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.12"},"colab":{"provenance":[]}},"nbformat":4,"nbformat_minor":0}