![](\"images/pixel_comparison.png\"){kind=tracking}
\n",
"![](\"images/distance_kiank.png\\\")
\n",
"By computing the distance between two encodings and thresholding, you can determine if the two pictures represent the same person
![](\"images/triplet_comparison.png\")
\n", "
In the next section, you'll call the pictures from left to right: Anchor (A), Positive (P), Negative (N)
![](\"images/f_x.png\")
\n",
"\n",
"Training will use triplets of images $(A, P, N)$:\n",
"\n",
"- A is an \"Anchor\" image--a picture of a person.\n",
"- P is a \"Positive\" image--a picture of the same person as the Anchor image.\n",
"- N is a \"Negative\" image--a picture of a different person than the Anchor image.\n",
"\n",
"These triplets are picked from the training dataset. $(A^{(i)}, P^{(i)}, N^{(i)})$ is used here to denote the $i$-th training example.\n",
"\n",
"You'd like to make sure that an image $A^{(i)}$ of an individual is closer to the Positive $P^{(i)}$ than to the Negative image $N^{(i)}$) by at least a margin $\\alpha$:\n",
"\n",
"$$\n",
"|| f\\left(A^{(i)}\\right)-f\\left(P^{(i)}\\right)||_{2}^{2}+\\alpha<|| f\\left(A^{(i)}\\right)-f\\left(N^{(i)}\\right)||_{2}^{2}\n",
"$$\n",
"\n",
"\n",
"You would thus like to minimize the following \"triplet cost\":\n",
"\n",
"$$\\mathcal{J} = \\sum^{m}_{i=1} \\large[ \\small \\underbrace{\\mid \\mid f(A^{(i)}) - f(P^{(i)}) \\mid \\mid_2^2}_\\text{(1)} - \\underbrace{\\mid \\mid f(A^{(i)}) - f(N^{(i)}) \\mid \\mid_2^2}_\\text{(2)} + \\alpha \\large ] \\small_+ \\tag{3}$$\n",
"Here, the notation \"$[z]_+$\" is used to denote $max(z,0)$.\n",
"\n",
"**Notes**:\n",
"\n",
"- The term (1) is the squared distance between the anchor \"A\" and the positive \"P\" for a given triplet; you want this to be small.\n",
"- The term (2) is the squared distance between the anchor \"A\" and the negative \"N\" for a given triplet, you want this to be relatively large. It has a minus sign preceding it because minimizing the negative of the term is the same as maximizing that term.\n",
"- $\\alpha$ is called the margin. It's a hyperparameter that you pick manually. You'll use $\\alpha = 0.2$.\n",
"\n",
"Most implementations also rescale the encoding vectors to haven L2 norm equal to one (i.e., $\\mid \\mid f(img)\\mid \\mid_2$=1); you won't have to worry about that in this assignment.\n",
"\n",
"\n",
"### Exercise 1 - triplet_loss\n",
"\n",
"Implement the triplet loss as defined by formula (3). These are the 4 steps:\n",
"\n",
"1. Compute the distance between the encodings of \"anchor\" and \"positive\": $\\mid \\mid f(A^{(i)}) - f(P^{(i)}) \\mid \\mid_2^2$\n",
"2. Compute the distance between the encodings of \"anchor\" and \"negative\": $\\mid \\mid f(A^{(i)}) - f(N^{(i)}) \\mid \\mid_2^2$\n",
"3. Compute the formula per training example: $ \\mid \\mid f(A^{(i)}) - f(P^{(i)}) \\mid \\mid_2^2 - \\mid \\mid f(A^{(i)}) - f(N^{(i)}) \\mid \\mid_2^2 + \\alpha$\n",
"4. Compute the full formula by taking the max with zero and summing over the training examples:$$\\mathcal{J} = \\sum^{m}_{i=1} \\large[ \\small \\mid \\mid f(A^{(i)}) - f(P^{(i)}) \\mid \\mid_2^2 - \\mid \\mid f(A^{(i)}) - f(N^{(i)}) \\mid \\mid_2^2+ \\alpha \\large ] \\small_+ \\tag{3}$$\n",
"\n",
"*Hints*:\n",
"\n",
"- Useful functions: `tf.reduce_sum()`, `tf.square()`, `tf.subtract()`, `tf.add()`, `tf.maximum()`.\n",
"\n",
"- For steps 1 and 2, sum over the entries of $\\mid \\mid f(A^{(i)}) - f(P^{(i)}) \\mid \\mid_2^2$ and $\\mid \\mid f(A^{(i)}) - f(N^{(i)}) \\mid \\mid_2^2$.\n",
"\n",
"- For step 4, you will sum over the training examples.\n",
"\n",
"*Additional Hints*:\n",
"\n",
"- Recall that the square of the L2 norm is the sum of the squared differences: $||x - y||_{2}^{2} = \\sum_{i=1}^{N}(x_{i} - y_{i})^{2}$\n",
"\n",
"- Note that the anchor, positive and negative encodings are of shape (*m*,128), where *m* is the number of training examples and 128 is the number of elements used to encode a single example.\n",
"\n",
"- For steps 1 and 2, maintain the number of *m* training examples and sum along the 128 values of each encoding. `tf.reduce_sum` has an axis parameter. This chooses along which axis the sums are applied.\n",
"\n",
"- Note that one way to choose the last axis in a tensor is to use negative indexing (axis=-1).\n",
"\n",
"- In step 4, when summing over training examples, the result will be a single scalar value.\n",
"\n",
"- For `tf.reduce_sum` to sum across all axes, keep the default value axis=None."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"deletable": false,
"nbgrader": {
"cell_type": "code",
"checksum": "29fad7814c380400d588375341b0dfc3",
"grade": false,
"grade_id": "cell-f05732f7068382cb",
"locked": false,
"schema_version": 3,
"solution": true,
"task": false
}
},
"outputs": [],
"source": [
"# GRADED FUNCTION: triplet_loss\n",
"\n",
"def triplet_loss(y_true, y_pred, alpha = 0.2):\n",
" \"\"\"\n",
" Implementation of the triplet loss as defined by formula (3)\n",
" \n",
" Arguments:\n",
" y_true -- true labels, required when you define a loss in Keras, you don't need it in this function.\n",
" y_pred -- python list containing three objects:\n",
" anchor -- the encodings for the anchor images, of shape (None, 128)\n",
" positive -- the encodings for the positive images, of shape (None, 128)\n",
" negative -- the encodings for the negative images, of shape (None, 128)\n",
" \n",
" Returns:\n",
" loss -- real number, value of the loss\n",
" \"\"\"\n",
" \n",
" anchor, positive, negative = y_pred[0], y_pred[1], y_pred[2]\n",
" # YOUR CODE STARTS HERE (≈ 4 lines)\n",
" # Step 1: Compute the (encoding) distance between the anchor and the positive\n",
" pos_dist = tf.reduce_sum(tf.square(tf.subtract(anchor,positive)),axis=-1)\n",
" # Step 2: Compute the (encoding) distance between the anchor and the negative\n",
" neg_dist = tf.reduce_sum(tf.square(tf.subtract(anchor,negative)),axis=-1)\n",
" # Step 3: subtract the two previous distances and add alpha.\n",
" basic_loss = tf.maximum(tf.add(tf.subtract(pos_dist,neg_dist),alpha),0)\n",
" # Step 4: Take the maximum of basic_loss and 0.0. Sum over the training examples.\n",
" loss = tf.reduce_sum(basic_loss)\n",
" \n",
" # YOUR CODE ENDS HERE\n",
" \n",
" return loss"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"deletable": false,
"editable": false,
"nbgrader": {
"cell_type": "code",
"checksum": "902d2a22882a6bbd15dc99e1bb5e6d09",
"grade": true,
"grade_id": "cell-440ff81e6bcda96a",
"locked": true,
"points": 1,
"schema_version": 3,
"solution": false,
"task": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"loss = tf.Tensor(527.2598, shape=(), dtype=float32)\n"
]
}
],
"source": [
"tf.random.set_seed(1)\n",
"y_true = (None, None, None) # It is not used\n",
"y_pred = (tf.keras.backend.random_normal([3, 128], mean=6, stddev=0.1, seed = 1),\n",
" tf.keras.backend.random_normal([3, 128], mean=1, stddev=1, seed = 1),\n",
" tf.keras.backend.random_normal([3, 128], mean=3, stddev=4, seed = 1))\n",
"loss = triplet_loss(y_true, y_pred)\n",
"\n",
"assert type(loss) == tf.python.framework.ops.EagerTensor, \"Use tensorflow functions\"\n",
"print(\"loss = \" + str(loss))\n",
"\n",
"y_pred_perfect = ([1., 1.], [1., 1.], [1., 1.,])\n",
"loss = triplet_loss(y_true, y_pred_perfect, 5)\n",
"assert loss == 5, \"Wrong value. Did you add the alpha to basic_loss?\"\n",
"y_pred_perfect = ([1., 1.],[1., 1.], [0., 0.,])\n",
"loss = triplet_loss(y_true, y_pred_perfect, 3)\n",
"assert loss == 1., \"Wrong value. Check that pos_dist = 0 and neg_dist = 2 in this example\"\n",
"y_pred_perfect = ([1., 1.],[0., 0.], [1., 1.,])\n",
"loss = triplet_loss(y_true, y_pred_perfect, 0)\n",
"assert loss == 2., \"Wrong value. Check that pos_dist = 2 and neg_dist = 0 in this example\"\n",
"y_pred_perfect = ([0., 0.],[0., 0.], [0., 0.,])\n",
"loss = triplet_loss(y_true, y_pred_perfect, -2)\n",
"assert loss == 0, \"Wrong value. Are you taking the maximum between basic_loss and 0?\"\n",
"y_pred_perfect = ([[1., 0.], [1., 0.]],[[1., 0.], [1., 0.]], [[0., 1.], [0., 1.]])\n",
"loss = triplet_loss(y_true, y_pred_perfect, 3)\n",
"assert loss == 2., \"Wrong value. Are you applying tf.reduce_sum to get the loss?\"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Expected Output**:\n",
"\n",
"| \n", " loss\n", " | \n", "\n", " 527.2598\n", " | \n", "
![](\"images/distance_matrix.png\")
\n", "
Example of distance outputs between three individuals' encodings
![](\"images/camera_0.jpg\\\")
"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"deletable": false,
"editable": false,
"nbgrader": {
"cell_type": "code",
"checksum": "cc561d44a84cd1bc2309e10140b370df",
"grade": true,
"grade_id": "cell-014d077254ad7d52",
"locked": true,
"points": 1,
"schema_version": 3,
"solution": false,
"task": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"It's bertrand, welcome in!\n",
"It's bertrand, welcome in!\n",
"It's not younes, please go away\n",
"It's not younes, please go away\n",
"It's younes, welcome in!\n"
]
},
{
"data": {
"text/plain": [
"(0.5992949, True)"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"assert(np.allclose(verify(\"images/camera_1.jpg\", \"bertrand\", database, FRmodel), (0.54364836, True)))\n",
"assert(np.allclose(verify(\"images/camera_3.jpg\", \"bertrand\", database, FRmodel), (0.38616243, True)))\n",
"assert(np.allclose(verify(\"images/camera_1.jpg\", \"younes\", database, FRmodel), (1.3963861, False)))\n",
"assert(np.allclose(verify(\"images/camera_3.jpg\", \"younes\", database, FRmodel), (1.3872949, False)))\n",
"verify(\"images/camera_0.jpg\", \"younes\", database, FRmodel)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Expected Output**:\n",
"\n",
"| \n", " It's Younes, welcome in!\n", " | \n", "\n", " (0.5992946, True)\n", " | \n", "
![](\"images/camera_2.jpg\")
\n",
"\n",
"Run the verification algorithm to check if Benoit can enter."
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"It's not kian, please go away\n"
]
},
{
"data": {
"text/plain": [
"(1.0259346, False)"
]
},
"execution_count": 60,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"verify(\"images/camera_2.jpg\", \"kian\", database, FRmodel)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Expected Output**:\n",
"\n",
"| \n", " It's not Kian, please go away\n", " | \n", "\n", " (1.0259346, False)\n", " | \n", "
| \n", " it's Younes, the distance is 0.5992946\n", " | \n", "\n", " (0.5992946, 'younes')\n", " | \n", "