{ "cells": [ { "cell_type": "markdown", "id": "db662146", "metadata": {}, "source": [ "[![image](https://raw.githubusercontent.com/visual-layer/visuallayer/main/imgs/vl_horizontal_logo.png)](https://www.visual-layer.com)" ] }, { "cell_type": "markdown", "id": "b74da379", "metadata": {}, "source": [ "# Clean Image Folder\n", "\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/visual-layer/fastdup/blob/main/examples/cleaning-image-dataset.ipynb)\n", "[![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/visual-layer/fastdup/blob/main/examples/cleaning-image-dataset.ipynb)\n", "\n", "This notebook shows how you can use fastdup analyze an image folder from potential issues and export a list of problematic files for further action.\n", "\n", "By the end of this notebook you will learn how to:\n", "\n", "+ Find various dataset issues with fastdup.\n", "+ Export a list of problematic images for further action." ] }, { "cell_type": "markdown", "id": "45bae965", "metadata": {}, "source": [ "## Installation\n", "\n", "If you're new, we encourage you to run the notebook in [Google Colab](https://colab.research.google.com/github/visual-layer/fastdup/blob/main/examples/cleaning-image-dataset.ipynb) or [Kaggle](https://kaggle.com/kernels/welcome?src=https://github.com/visual-layer/fastdup/blob/main/cleaning-image-dataset.ipynb) for the best experience. If you'd like to just view and skim through the notebook, we recommend viewing using [nbviewer](https://nbviewer.org/github/visual-layer/fastdup/blob/main/examples/cleaning-image-dataset.ipynb). \n", "\n", "Let's start with the installation:" ] }, { "cell_type": "code", "execution_count": 1, "id": "1e6a3577", "metadata": {}, "outputs": [], "source": [ "!pip install fastdup -Uq" ] }, { "cell_type": "markdown", "id": "fc9b6b89", "metadata": {}, "source": [ "Now, test the installation by printing out the version. If there's no error message, we are ready to go!" ] }, { "cell_type": "code", "execution_count": 2, "id": "6a49b5eb", "metadata": { "executionInfo": { "elapsed": 2034, "status": "ok", "timestamp": 1677668109538, "user": { "displayName": "Tom Shani", "userId": "00667426488827942961" }, "user_tz": -120 }, "id": "6a49b5eb" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/usr/bin/dpkg\n" ] }, { "data": { "text/plain": [ "'1.23'" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import fastdup\n", "fastdup.__version__" ] }, { "cell_type": "markdown", "id": "ff4dfa80-d1e4-46d1-ae10-e8715c16bb07", "metadata": { "id": "ff4dfa80-d1e4-46d1-ae10-e8715c16bb07" }, "source": [ "## Download Dataset\n", "\n", "In this notebook let's use a widely available and relatively well curated [Food-101](https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101/) dataset.\n", "\n", "The Food-101 dataset consists of 101 food classes with 1,000 images per class. That is a total of 101,000 images.\n", "\n", "Let's download only from the dataset and extract them into our local directory:" ] }, { "cell_type": "code", "execution_count": null, "id": "9a45d227", "metadata": {}, "outputs": [], "source": [ "!wget http://data.vision.ee.ethz.ch/cvl/food-101.tar.gz\n", "!tar -xf food-101.tar.gz" ] }, { "cell_type": "markdown", "id": "7e2e70a3", "metadata": { "id": "7e2e70a3", "tags": [] }, "source": [ "## Run fastdup" ] }, { "cell_type": "markdown", "id": "b04063f6", "metadata": {}, "source": [ "Once the extraction completes, we can run fastdup on the images.\n", "\n", "For that let's create a `fastdup` object and specify the input directory which points to the folder of images." ] }, { "cell_type": "code", "execution_count": 3, "id": "29858edc", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Warning: fastdup create() without work_dir argument, output is stored in a folder named work_dir in your current working path.\n" ] } ], "source": [ "fd = fastdup.create(input_dir=\"food-101/images/\")" ] }, { "cell_type": "markdown", "id": "13777969", "metadata": {}, "source": [ "> **NOTE**: If you're running this example on Google Colab, we recommend running with `num_images=40000` in the following cell. This limits fastdup to run on 40000 images instead of the entire dataset which takes shorter time to complete on Google Colab." ] }, { "cell_type": "code", "execution_count": 4, "id": "637c1650", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "FastDup Software, (C) copyright 2022 Dr. Amir Alush and Dr. Danny Bickson.\n", "2023-07-11 15:23:39 [INFO] Going to loop over dir food-101/images\n", "2023-07-11 15:23:40 [INFO] Found total 101000 images to run on, 101000 train, 0 test, name list 101000, counter 101000 \n", "2023-07-11 15:27:12 [INFO] Found total 101000 images to run onmated: 0 Minutes\n", "Finished histogram 29.784\n", "Finished bucket sort 29.980\n", "2023-07-11 15:27:37 [INFO] 24498) Finished write_index() NN model\n", "2023-07-11 15:27:37 [INFO] Stored nn model index file work_dir/nnf.index\n", "2023-07-11 15:27:49 [INFO] Total time took 249055 ms\n", "2023-07-11 15:27:49 [INFO] Found a total of 230 fully identical images (d>0.990), which are 0.11 %\n", "2023-07-11 15:27:49 [INFO] Found a total of 88 nearly identical images(d>0.980), which are 0.04 %\n", "2023-07-11 15:27:49 [INFO] Found a total of 5296 above threshold images (d>0.900), which are 2.62 %\n", "2023-07-11 15:27:49 [INFO] Found a total of 10103 outlier images (d<0.050), which are 5.00 %\n", "2023-07-11 15:27:49 [INFO] Min distance found 0.379 max distance 1.000\n", "2023-07-11 15:27:49 [INFO] Running connected components for ccthreshold 0.900000 \n", ".0\n", " ########################################################################################\n", "\n", "Dataset Analysis Summary: \n", "\n", " Dataset contains 101000 images\n", " Valid images are 100.00% (101,000) of the data, invalid are 0.00% (0) of the data\n", " Similarity: 1.93% (1,946) belong to 23 similarity clusters (components).\n", " 98.07% (99,054) images do not belong to any similarity cluster.\n", " Largest cluster has 2,572 (2.55%) images.\n", " For a detailed analysis, use `.connected_components()`\n", "(similarity threshold used is 0.9, connected component threshold used is 0.9).\n", "\n", " Outliers: 5.97% (6,031) of images are possible outliers, and fall in the bottom 5.00% of similarity values.\n", " For a detailed list of outliers, use `.outliers()`.\n" ] }, { "data": { "text/plain": [ "0" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# fd.run(num_images=40000, ccthreshold=0.9) # runs fastdup on a subset of 40000 images from the dataset\n", "fd.run(ccthreshold=0.9) # runs fastdup on the entire dataset" ] }, { "cell_type": "markdown", "id": "1a4ec356", "metadata": {}, "source": [ "> **Note**: `ccthreshold` is a parameter to in the connected components algorithm. Read more [here](https://visual-layer.readme.io/docs/dataset-cleanup#threshold-for-similarity-clusters) on how to set an appropriate value for your dataset." ] }, { "cell_type": "markdown", "id": "fa3b8707", "metadata": {}, "source": [ "Get a summary of the run showing potentially problematic files." ] }, { "cell_type": "code", "execution_count": 5, "id": "8ec0a195", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " ########################################################################################\n", "\n", "Dataset Analysis Summary: \n", "\n", " Dataset contains 101000 images\n", " Valid images are 100.00% (101,000) of the data, invalid are 0.00% (0) of the data\n", " Similarity: 1.93% (1,946) belong to 23 similarity clusters (components).\n", " 98.07% (99,054) images do not belong to any similarity cluster.\n", " Largest cluster has 2,572 (2.55%) images.\n", " For a detailed analysis, use `.connected_components()`\n", "(similarity threshold used is 0.9, connected component threshold used is 0.9).\n", "\n", " Outliers: 5.97% (6,031) of images are possible outliers, and fall in the bottom 5.00% of similarity values.\n", " For a detailed list of outliers, use `.outliers()`.\n" ] }, { "data": { "text/plain": [ "['Dataset contains 101000 images',\n", " 'Valid images are 100.00% (101,000) of the data, invalid are 0.00% (0) of the data',\n", " 'Similarity: 1.93% (1,946) belong to 23 similarity clusters (components).',\n", " '98.07% (99,054) images do not belong to any similarity cluster.',\n", " 'Largest cluster has 2,572 (2.55%) images.',\n", " 'For a detailed analysis, use `.connected_components()`\\n(similarity threshold used is 0.9, connected component threshold used is 0.9).\\n',\n", " 'Outliers: 5.97% (6,031) of images are possible outliers, and fall in the bottom 5.00% of similarity values.',\n", " 'For a detailed list of outliers, use `.outliers()`.']" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.summary()" ] }, { "cell_type": "markdown", "id": "be16bf24", "metadata": { "tags": [] }, "source": [ "## Broken Images\n", "\n", "The lowest hanging fruit is to find a list of broken images and remove them from your dataset. These images are most probably corrupted file and could not be loaded." ] }, { "cell_type": "markdown", "id": "9754a904", "metadata": {}, "source": [ "To get the broken images simply run" ] }, { "cell_type": "code", "execution_count": 6, "id": "71f062ba", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
filenameindexerror_codeis_validfd_index
\n", "
" ], "text/plain": [ "Empty DataFrame\n", "Columns: [filename, index, error_code, is_valid, fd_index]\n", "Index: []" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "broken_images = fd.invalid_instances()\n", "broken_images" ] }, { "cell_type": "markdown", "id": "1625710f", "metadata": {}, "source": [ "This dataset is a carefully curated, so we did not find any broken images. Which is great!" ] }, { "cell_type": "markdown", "id": "396e9f68", "metadata": {}, "source": [ "## List of Broken Images\n", "If there are broken images however, you can easily get a list of the images." ] }, { "cell_type": "code", "execution_count": 7, "id": "2827cbcb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list_of_broken_images = broken_images['filename'].to_list()\n", "list_of_broken_images" ] }, { "cell_type": "markdown", "id": "ba78da12", "metadata": { "tags": [] }, "source": [ "## Duplicate Image Pairs\n", "\n", "Show a gallery of duplicate image pairs. Distance of `1.0` indicate that the image pairs are exact copies." ] }, { "cell_type": "code", "execution_count": 8, "id": "516ee9fc", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 137.27it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Stored similarity visual view in work_dir/galleries/duplicates.html\n" ] }, { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Duplicates Report\n", " \n", " \n", "\n", "\n", "\n", "
\n", "
\n", "
\n", " \n", " \"logo\"\n", " \n", "
\n", " \n", "
\n", "
\n", "
\n", "

Duplicates Report

\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance1.0
From/fried_rice/2820757.jpg
To/fried_rice/2899815.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance1.0
From/chocolate_cake/55122.jpg
To/chocolate_cake/51717.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance1.0
From/grilled_salmon/606368.jpg
To/grilled_salmon/599021.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance1.0
From/paella/2199941.jpg
To/paella/2199939.jpg
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " \n", "
\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.vis.duplicates_gallery(num_images=5)" ] }, { "cell_type": "markdown", "id": "85cc68f7", "metadata": { "tags": [] }, "source": [ "## Image Clusters\n", "\n", "Visualize image clusters from the dataset.\n", "\n", "> **Note**: Setting `num_images=5` shows a gallery of with 5 rows. Change this value to view more/less." ] }, { "cell_type": "code", "execution_count": 9, "id": "c08827bf", "metadata": { "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00, 4.66it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Finished OK. Components are stored as image files work_dir/galleries/components_[index].jpg\n", "Stored components visual view in work_dir/galleries/components.html\n", "Execution time in seconds 7.7\n" ] }, { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Components Report\n", " \n", " \n", "\n", "\n", "\n", "
\n", "
\n", "
\n", " \n", " \"logo\"\n", " \n", "
\n", " \n", "
\n", "
\n", "
\n", "

Components Report

Showing groups of similar images

\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
component18214
num_images754
mean_distance0.9
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
component24712
num_images464
mean_distance0.9
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
component31543
num_images139
mean_distance0.9001
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
component5021
num_images98
mean_distance0.9002
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
component32388
num_images44
mean_distance0.9006
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " \n", "
\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.vis.component_gallery(num_images=5)" ] }, { "cell_type": "markdown", "id": "b3b1fdaf", "metadata": {}, "source": [ "## List of Duplicates\n", "Now let's single out all duplicates and near-duplicates by running using the connected components function:\n", "\n" ] }, { "cell_type": "code", "execution_count": 10, "id": "d0bdff93", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
indexcomponent_idsumcountmean_distancemin_distancemax_distancefilenameerror_codeis_validfd_index
01311314.00004.01.00001.00001.0000food-101/images/apple_pie/1461580.jpgVALIDTrue131
11341314.00004.01.00001.00001.0000food-101/images/apple_pie/1469191.jpgVALIDTrue134
21381373.86504.00.96620.96620.9662food-101/images/apple_pie/1486972.jpgVALIDTrue138
31391373.86504.00.96620.96620.9662food-101/images/apple_pie/1487150.jpgVALIDTrue139
42492473.60754.00.90190.90190.9019food-101/images/apple_pie/1946376.jpgVALIDTrue249
\n", "
" ], "text/plain": [ " index component_id sum count mean_distance min_distance max_distance filename error_code is_valid fd_index\n", "0 131 131 4.0000 4.0 1.0000 1.0000 1.0000 food-101/images/apple_pie/1461580.jpg VALID True 131\n", "1 134 131 4.0000 4.0 1.0000 1.0000 1.0000 food-101/images/apple_pie/1469191.jpg VALID True 134\n", "2 138 137 3.8650 4.0 0.9662 0.9662 0.9662 food-101/images/apple_pie/1486972.jpg VALID True 138\n", "3 139 137 3.8650 4.0 0.9662 0.9662 0.9662 food-101/images/apple_pie/1487150.jpg VALID True 139\n", "4 249 247 3.6075 4.0 0.9019 0.9019 0.9019 food-101/images/apple_pie/1946376.jpg VALID True 249" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "connected_components_df , _ = fd.connected_components()\n", "connected_components_df.head()" ] }, { "cell_type": "markdown", "id": "e9539649", "metadata": {}, "source": [ "Let's now write a utility function to get the clusters:" ] }, { "cell_type": "code", "execution_count": 11, "id": "7c79f527", "metadata": {}, "outputs": [], "source": [ "# a function to group connected components\n", "def get_clusters(df, sort_by='count', min_count=2, ascending=False):\n", " # columns to aggregate\n", " agg_dict = {'filename': list, 'mean_distance': max, 'count': len}\n", "\n", " if 'label' in df.columns:\n", " agg_dict['label'] = list\n", " \n", " # filter by count\n", " df = df[df['count'] >= min_count]\n", " \n", " # group and aggregate columns\n", " grouped_df = df.groupby('component_id').agg(agg_dict)\n", " \n", " # sort\n", " grouped_df = grouped_df.sort_values(by=[sort_by], ascending=ascending)\n", " return grouped_df" ] }, { "cell_type": "code", "execution_count": 12, "id": "1de9e526", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
filenamemean_distancecount
component_id
18214[food-101/images/chicken_curry/2394967.jpg, food-101/images/chicken_curry/2451199.jpg, food-101/images/chicken_curry/2701143.jpg, food-101/images/chicken_curry/3496679.jpg, food-101/images/chicken_curry/66223.jpg, food-101/images/chicken_curry/844686.jpg, food-101/images/chicken_curry/882723.jpg, food-101/images/chicken_curry/968094.jpg, food-101/images/clam_chowder/1023891.jpg, food-101/images/clam_chowder/1072684.jpg, food-101/images/clam_chowder/1113834.jpg, food-101/images/clam_chowder/1156854.jpg, food-101/images/clam_chowder/1223949.jpg, food-101/images/clam_chowder/1242818.jpg, food-101/images/clam_chowder/1322415.jpg, food-101/images/clam_chowder/1418012.jpg, food-101/images/clam_chowder/1437241.jpg, food-101/images/clam_chowder/1511884.jpg, food-101/images/clam_chowder/1563396.jpg, food-101/images/clam_chowder/1581820.jpg, food-101/images/clam_chowder/1598939.jpg, food-101/images/clam_chowder/1742147.jpg, food-101/images/clam_chowder/1871872.jpg, food-101/images/clam_chowder/1942294.jpg, food-101/images/clam_chowder/1945594.jpg, food-101/images/clam_chowder/1993404.jpg, food-101/images/clam_chowder/2018024.jpg, food-101/images/clam_chowder/2024817.jpg, food-101/images/clam_chowder/2027156.jpg, food-101/images/clam_chowder/2113399.jpg, food-101/images/clam_chowder/2140406.jpg, food-101/images/clam_chowder/2140703.jpg, food-101/images/clam_chowder/2148133.jpg, food-101/images/clam_chowder/2219398.jpg, food-101/images/clam_chowder/2248997.jpg, food-101/images/clam_chowder/2305126.jpg, food-101/images/clam_chowder/2340686.jpg, food-101/images/clam_chowder/2361787.jpg, food-101/images/clam_chowder/2396225.jpg, food-101/images/clam_chowder/2398168.jpg, food-101/images/clam_chowder/2426821.jpg, food-101/images/clam_chowder/2476027.jpg, food-101/images/clam_chowder/2499937.jpg, food-101/images/clam_chowder/2508514.jpg, food-101/images/clam_chowder/2509774.jpg, food-101/images/clam_chowder/2523293.jpg, food-101/images/clam_chowder/2542800.jpg, food-101/images/clam_chowder/2599110.jpg, food-101/images/clam_chowder/2602712.jpg, food-101/images/clam_chowder/2603953.jpg, food-101/images/clam_chowder/2673628.jpg, food-101/images/clam_chowder/2676197.jpg, food-101/images/clam_chowder/2685745.jpg, food-101/images/clam_chowder/2742139.jpg, food-101/images/clam_chowder/2770581.jpg, food-101/images/clam_chowder/2789238.jpg, food-101/images/clam_chowder/2862215.jpg, food-101/images/clam_chowder/2869301.jpg, food-101/images/clam_chowder/2871657.jpg, food-101/images/clam_chowder/2888266.jpg, food-101/images/clam_chowder/2894611.jpg, food-101/images/clam_chowder/2897057.jpg, food-101/images/clam_chowder/2926608.jpg, food-101/images/clam_chowder/2961270.jpg, food-101/images/clam_chowder/298228.jpg, food-101/images/clam_chowder/2992252.jpg, food-101/images/clam_chowder/3018696.jpg, food-101/images/clam_chowder/3045182.jpg, food-101/images/clam_chowder/3073323.jpg, food-101/images/clam_chowder/3081370.jpg, food-101/images/clam_chowder/3107862.jpg, food-101/images/clam_chowder/3109973.jpg, food-101/images/clam_chowder/3126055.jpg, food-101/images/clam_chowder/3137773.jpg, food-101/images/clam_chowder/3142771.jpg, food-101/images/clam_chowder/3228022.jpg, food-101/images/clam_chowder/3264840.jpg, food-101/images/clam_chowder/3291238.jpg, food-101/images/clam_chowder/3322877.jpg, food-101/images/clam_chowder/3395059.jpg, food-101/images/clam_chowder/3549975.jpg, food-101/images/clam_chowder/3588064.jpg, food-101/images/clam_chowder/3656593.jpg, food-101/images/clam_chowder/3689947.jpg, food-101/images/clam_chowder/3830343.jpg, food-101/images/clam_chowder/3853148.jpg, food-101/images/clam_chowder/390727.jpg, food-101/images/clam_chowder/3914755.jpg, food-101/images/clam_chowder/513498.jpg, food-101/images/clam_chowder/546975.jpg, food-101/images/clam_chowder/596255.jpg, food-101/images/clam_chowder/655847.jpg, food-101/images/clam_chowder/686716.jpg, food-101/images/clam_chowder/75800.jpg, food-101/images/clam_chowder/758162.jpg, food-101/images/clam_chowder/762499.jpg, food-101/images/clam_chowder/771369.jpg, food-101/images/clam_chowder/772778.jpg, food-101/images/clam_chowder/777422.jpg, food-101/images/clam_chowder/795839.jpg, ...]0.9139754
24712[food-101/images/club_sandwich/1158836.jpg, food-101/images/club_sandwich/122236.jpg, food-101/images/club_sandwich/1243806.jpg, food-101/images/club_sandwich/1297247.jpg, food-101/images/club_sandwich/1318118.jpg, food-101/images/club_sandwich/1413794.jpg, food-101/images/club_sandwich/1719340.jpg, food-101/images/club_sandwich/1775789.jpg, food-101/images/club_sandwich/1811271.jpg, food-101/images/club_sandwich/1818664.jpg, food-101/images/club_sandwich/1840706.jpg, food-101/images/club_sandwich/1886101.jpg, food-101/images/club_sandwich/191288.jpg, food-101/images/club_sandwich/2163422.jpg, food-101/images/club_sandwich/2214212.jpg, food-101/images/club_sandwich/2272423.jpg, food-101/images/club_sandwich/2778614.jpg, food-101/images/club_sandwich/3106065.jpg, food-101/images/club_sandwich/3132951.jpg, food-101/images/club_sandwich/3251770.jpg, food-101/images/club_sandwich/3310968.jpg, food-101/images/club_sandwich/3526250.jpg, food-101/images/club_sandwich/3543955.jpg, food-101/images/club_sandwich/3550782.jpg, food-101/images/club_sandwich/3646665.jpg, food-101/images/club_sandwich/3664710.jpg, food-101/images/club_sandwich/574816.jpg, food-101/images/club_sandwich/588478.jpg, food-101/images/crab_cakes/3467918.jpg, food-101/images/fish_and_chips/1195292.jpg, food-101/images/fish_and_chips/2484873.jpg, food-101/images/fish_and_chips/3616236.jpg, food-101/images/french_fries/1049122.jpg, food-101/images/french_fries/1071481.jpg, food-101/images/french_fries/1074446.jpg, food-101/images/french_fries/1080215.jpg, food-101/images/french_fries/109401.jpg, food-101/images/french_fries/1099260.jpg, food-101/images/french_fries/1150103.jpg, food-101/images/french_fries/1165966.jpg, food-101/images/french_fries/1221000.jpg, food-101/images/french_fries/1295274.jpg, food-101/images/french_fries/1327248.jpg, food-101/images/french_fries/1361604.jpg, food-101/images/french_fries/1369116.jpg, food-101/images/french_fries/1370900.jpg, food-101/images/french_fries/1384733.jpg, food-101/images/french_fries/1394977.jpg, food-101/images/french_fries/147628.jpg, food-101/images/french_fries/1504529.jpg, food-101/images/french_fries/1509620.jpg, food-101/images/french_fries/1538500.jpg, food-101/images/french_fries/1552191.jpg, food-101/images/french_fries/1610240.jpg, food-101/images/french_fries/1622796.jpg, food-101/images/french_fries/166821.jpg, food-101/images/french_fries/168924.jpg, food-101/images/french_fries/1692353.jpg, food-101/images/french_fries/1700344.jpg, food-101/images/french_fries/1712331.jpg, food-101/images/french_fries/1724805.jpg, food-101/images/french_fries/1740113.jpg, food-101/images/french_fries/1745333.jpg, food-101/images/french_fries/1763119.jpg, food-101/images/french_fries/1810352.jpg, food-101/images/french_fries/184954.jpg, food-101/images/french_fries/1850310.jpg, food-101/images/french_fries/1875469.jpg, food-101/images/french_fries/1897333.jpg, food-101/images/french_fries/1928660.jpg, food-101/images/french_fries/1936184.jpg, food-101/images/french_fries/1969264.jpg, food-101/images/french_fries/2001959.jpg, food-101/images/french_fries/2011974.jpg, food-101/images/french_fries/2041482.jpg, food-101/images/french_fries/2056007.jpg, food-101/images/french_fries/2073415.jpg, food-101/images/french_fries/2080959.jpg, food-101/images/french_fries/2127530.jpg, food-101/images/french_fries/2189231.jpg, food-101/images/french_fries/2212633.jpg, food-101/images/french_fries/2226284.jpg, food-101/images/french_fries/2239352.jpg, food-101/images/french_fries/2246387.jpg, food-101/images/french_fries/2248373.jpg, food-101/images/french_fries/2269039.jpg, food-101/images/french_fries/2278380.jpg, food-101/images/french_fries/2348229.jpg, food-101/images/french_fries/2369999.jpg, food-101/images/french_fries/2371960.jpg, food-101/images/french_fries/2452990.jpg, food-101/images/french_fries/2485920.jpg, food-101/images/french_fries/2493036.jpg, food-101/images/french_fries/2521012.jpg, food-101/images/french_fries/2521724.jpg, food-101/images/french_fries/2624567.jpg, food-101/images/french_fries/2649729.jpg, food-101/images/french_fries/2672237.jpg, food-101/images/french_fries/2700217.jpg, food-101/images/french_fries/2708944.jpg, ...]0.9144464
31543[food-101/images/dumplings/1016850.jpg, food-101/images/dumplings/1021310.jpg, food-101/images/dumplings/1045500.jpg, food-101/images/dumplings/1066326.jpg, food-101/images/dumplings/1086100.jpg, food-101/images/dumplings/1134035.jpg, food-101/images/dumplings/1228546.jpg, food-101/images/dumplings/1243140.jpg, food-101/images/dumplings/1245056.jpg, food-101/images/dumplings/1269986.jpg, food-101/images/dumplings/1270308.jpg, food-101/images/dumplings/1275130.jpg, food-101/images/dumplings/1276808.jpg, food-101/images/dumplings/1308246.jpg, food-101/images/dumplings/1376074.jpg, food-101/images/dumplings/140004.jpg, food-101/images/dumplings/1433645.jpg, food-101/images/dumplings/1450685.jpg, food-101/images/dumplings/1542637.jpg, food-101/images/dumplings/1545564.jpg, food-101/images/dumplings/1563646.jpg, food-101/images/dumplings/1564985.jpg, food-101/images/dumplings/1598923.jpg, food-101/images/dumplings/1630799.jpg, food-101/images/dumplings/1664893.jpg, food-101/images/dumplings/1693794.jpg, food-101/images/dumplings/1695231.jpg, food-101/images/dumplings/1789500.jpg, food-101/images/dumplings/1813271.jpg, food-101/images/dumplings/1848359.jpg, food-101/images/dumplings/1848509.jpg, food-101/images/dumplings/1872410.jpg, food-101/images/dumplings/1881086.jpg, food-101/images/dumplings/1897260.jpg, food-101/images/dumplings/1918394.jpg, food-101/images/dumplings/1925731.jpg, food-101/images/dumplings/1927353.jpg, food-101/images/dumplings/1948609.jpg, food-101/images/dumplings/1998135.jpg, food-101/images/dumplings/2000371.jpg, food-101/images/dumplings/2052342.jpg, food-101/images/dumplings/211325.jpg, food-101/images/dumplings/2154990.jpg, food-101/images/dumplings/2166764.jpg, food-101/images/dumplings/2191822.jpg, food-101/images/dumplings/2199788.jpg, food-101/images/dumplings/2229749.jpg, food-101/images/dumplings/231024.jpg, food-101/images/dumplings/231026.jpg, food-101/images/dumplings/231028.jpg, food-101/images/dumplings/2330342.jpg, food-101/images/dumplings/233293.jpg, food-101/images/dumplings/2362785.jpg, food-101/images/dumplings/2364879.jpg, food-101/images/dumplings/2373653.jpg, food-101/images/dumplings/2379541.jpg, food-101/images/dumplings/2380724.jpg, food-101/images/dumplings/2444294.jpg, food-101/images/dumplings/2450566.jpg, food-101/images/dumplings/2478984.jpg, food-101/images/dumplings/2492331.jpg, food-101/images/dumplings/2500721.jpg, food-101/images/dumplings/2524385.jpg, food-101/images/dumplings/2537158.jpg, food-101/images/dumplings/2538305.jpg, food-101/images/dumplings/2545565.jpg, food-101/images/dumplings/2546897.jpg, food-101/images/dumplings/2561548.jpg, food-101/images/dumplings/2571523.jpg, food-101/images/dumplings/2585099.jpg, food-101/images/dumplings/2600333.jpg, food-101/images/dumplings/2606645.jpg, food-101/images/dumplings/2630977.jpg, food-101/images/dumplings/263589.jpg, food-101/images/dumplings/263764.jpg, food-101/images/dumplings/2642265.jpg, food-101/images/dumplings/2675187.jpg, food-101/images/dumplings/267852.jpg, food-101/images/dumplings/2707946.jpg, food-101/images/dumplings/2736144.jpg, food-101/images/dumplings/2750447.jpg, food-101/images/dumplings/2770853.jpg, food-101/images/dumplings/2800182.jpg, food-101/images/dumplings/28220.jpg, food-101/images/dumplings/2834398.jpg, food-101/images/dumplings/2865709.jpg, food-101/images/dumplings/2904302.jpg, food-101/images/dumplings/2909040.jpg, food-101/images/dumplings/2950605.jpg, food-101/images/dumplings/2951551.jpg, food-101/images/dumplings/2975772.jpg, food-101/images/dumplings/3030550.jpg, food-101/images/dumplings/3030640.jpg, food-101/images/dumplings/3042784.jpg, food-101/images/dumplings/3101730.jpg, food-101/images/dumplings/3101737.jpg, food-101/images/dumplings/310672.jpg, food-101/images/dumplings/3182351.jpg, food-101/images/dumplings/3191742.jpg, food-101/images/dumplings/322034.jpg, ...]0.9142139
5021[food-101/images/beet_salad/1217698.jpg, food-101/images/ceviche/804338.jpg, food-101/images/fried_rice/1019981.jpg, food-101/images/fried_rice/1023430.jpg, food-101/images/fried_rice/1054627.jpg, food-101/images/fried_rice/1076217.jpg, food-101/images/fried_rice/1163008.jpg, food-101/images/fried_rice/1185768.jpg, food-101/images/fried_rice/1200965.jpg, food-101/images/fried_rice/1242614.jpg, food-101/images/fried_rice/1264045.jpg, food-101/images/fried_rice/1281471.jpg, food-101/images/fried_rice/1285028.jpg, food-101/images/fried_rice/1311992.jpg, food-101/images/fried_rice/1315188.jpg, food-101/images/fried_rice/1323164.jpg, food-101/images/fried_rice/1396018.jpg, food-101/images/fried_rice/1421149.jpg, food-101/images/fried_rice/1429619.jpg, food-101/images/fried_rice/1534541.jpg, food-101/images/fried_rice/1610064.jpg, food-101/images/fried_rice/1677058.jpg, food-101/images/fried_rice/1691031.jpg, food-101/images/fried_rice/1727509.jpg, food-101/images/fried_rice/1774109.jpg, food-101/images/fried_rice/1802914.jpg, food-101/images/fried_rice/1865368.jpg, food-101/images/fried_rice/1874287.jpg, food-101/images/fried_rice/1915049.jpg, food-101/images/fried_rice/2009382.jpg, food-101/images/fried_rice/2022629.jpg, food-101/images/fried_rice/2029077.jpg, food-101/images/fried_rice/2078444.jpg, food-101/images/fried_rice/2136311.jpg, food-101/images/fried_rice/2180004.jpg, food-101/images/fried_rice/2186269.jpg, food-101/images/fried_rice/2261572.jpg, food-101/images/fried_rice/2406661.jpg, food-101/images/fried_rice/2543815.jpg, food-101/images/fried_rice/2552407.jpg, food-101/images/fried_rice/2648223.jpg, food-101/images/fried_rice/2697544.jpg, food-101/images/fried_rice/2699922.jpg, food-101/images/fried_rice/2785231.jpg, food-101/images/fried_rice/2850904.jpg, food-101/images/fried_rice/3034238.jpg, food-101/images/fried_rice/3052175.jpg, food-101/images/fried_rice/3134947.jpg, food-101/images/fried_rice/3277448.jpg, food-101/images/fried_rice/3295993.jpg, food-101/images/fried_rice/3321202.jpg, food-101/images/fried_rice/3331345.jpg, food-101/images/fried_rice/3367571.jpg, food-101/images/fried_rice/3387843.jpg, food-101/images/fried_rice/3402627.jpg, food-101/images/fried_rice/3436140.jpg, food-101/images/fried_rice/3592807.jpg, food-101/images/fried_rice/3620105.jpg, food-101/images/fried_rice/3641662.jpg, food-101/images/fried_rice/3708254.jpg, food-101/images/fried_rice/3766596.jpg, food-101/images/fried_rice/3824406.jpg, food-101/images/fried_rice/3917282.jpg, food-101/images/fried_rice/421911.jpg, food-101/images/fried_rice/461757.jpg, food-101/images/fried_rice/568773.jpg, food-101/images/fried_rice/578945.jpg, food-101/images/fried_rice/583326.jpg, food-101/images/fried_rice/647756.jpg, food-101/images/fried_rice/709970.jpg, food-101/images/fried_rice/919785.jpg, food-101/images/fried_rice/931764.jpg, food-101/images/fried_rice/983544.jpg, food-101/images/gnocchi/2022753.jpg, food-101/images/greek_salad/1647381.jpg, food-101/images/nachos/3234783.jpg, food-101/images/pad_thai/1898870.jpg, food-101/images/pad_thai/2319689.jpg, food-101/images/risotto/129835.jpg, food-101/images/risotto/1615122.jpg, food-101/images/risotto/2311593.jpg, food-101/images/risotto/2512272.jpg, food-101/images/risotto/2600659.jpg, food-101/images/risotto/2976547.jpg, food-101/images/risotto/3786320.jpg, food-101/images/risotto/695270.jpg, food-101/images/risotto/713463.jpg, food-101/images/risotto/919251.jpg, food-101/images/risotto/960618.jpg, food-101/images/scallops/209391.jpg, food-101/images/scallops/2995961.jpg, food-101/images/shrimp_and_grits/1043600.jpg, food-101/images/shrimp_and_grits/1787413.jpg, food-101/images/shrimp_and_grits/1996765.jpg, food-101/images/shrimp_and_grits/2292751.jpg, food-101/images/shrimp_and_grits/2533991.jpg, food-101/images/shrimp_and_grits/3086347.jpg, food-101/images/shrimp_and_grits/3281299.jpg]0.910398
32388[food-101/images/edamame/1047955.jpg, food-101/images/edamame/1086703.jpg, food-101/images/edamame/113525.jpg, food-101/images/edamame/1191326.jpg, food-101/images/edamame/1474750.jpg, food-101/images/edamame/1541444.jpg, food-101/images/edamame/1620027.jpg, food-101/images/edamame/1677511.jpg, food-101/images/edamame/1714523.jpg, food-101/images/edamame/1753683.jpg, food-101/images/edamame/2035309.jpg, food-101/images/edamame/2040753.jpg, food-101/images/edamame/2204418.jpg, food-101/images/edamame/2241023.jpg, food-101/images/edamame/2275716.jpg, food-101/images/edamame/2390868.jpg, food-101/images/edamame/239734.jpg, food-101/images/edamame/2399411.jpg, food-101/images/edamame/2473555.jpg, food-101/images/edamame/2483789.jpg, food-101/images/edamame/2499082.jpg, food-101/images/edamame/2588718.jpg, food-101/images/edamame/2670224.jpg, food-101/images/edamame/2708664.jpg, food-101/images/edamame/2789682.jpg, food-101/images/edamame/2846777.jpg, food-101/images/edamame/2847124.jpg, food-101/images/edamame/2916151.jpg, food-101/images/edamame/3041151.jpg, food-101/images/edamame/3253578.jpg, food-101/images/edamame/3325153.jpg, food-101/images/edamame/3432193.jpg, food-101/images/edamame/3520889.jpg, food-101/images/edamame/3555531.jpg, food-101/images/edamame/3558096.jpg, food-101/images/edamame/3620419.jpg, food-101/images/edamame/3666348.jpg, food-101/images/edamame/3788141.jpg, food-101/images/edamame/3831507.jpg, food-101/images/edamame/601042.jpg, food-101/images/edamame/677508.jpg, food-101/images/edamame/684483.jpg, food-101/images/edamame/825581.jpg, food-101/images/edamame/952423.jpg]0.908944
\n", "
" ], "text/plain": [ " filename \\\n", "component_id \n", "18214 [food-101/images/chicken_curry/2394967.jpg, food-101/images/chicken_curry/2451199.jpg, food-101/images/chicken_curry/2701143.jpg, food-101/images/chicken_curry/3496679.jpg, food-101/images/chicken_curry/66223.jpg, food-101/images/chicken_curry/844686.jpg, food-101/images/chicken_curry/882723.jpg, food-101/images/chicken_curry/968094.jpg, food-101/images/clam_chowder/1023891.jpg, food-101/images/clam_chowder/1072684.jpg, food-101/images/clam_chowder/1113834.jpg, food-101/images/clam_chowder/1156854.jpg, food-101/images/clam_chowder/1223949.jpg, food-101/images/clam_chowder/1242818.jpg, food-101/images/clam_chowder/1322415.jpg, food-101/images/clam_chowder/1418012.jpg, food-101/images/clam_chowder/1437241.jpg, food-101/images/clam_chowder/1511884.jpg, food-101/images/clam_chowder/1563396.jpg, food-101/images/clam_chowder/1581820.jpg, food-101/images/clam_chowder/1598939.jpg, food-101/images/clam_chowder/1742147.jpg, food-101/images/clam_chowder/1871872.jpg, food-101/images/clam_chowder/1942294.jpg, food-101/images/clam_chowder/1945594.jpg, food-101/images/clam_chowder/1993404.jpg, food-101/images/clam_chowder/2018024.jpg, food-101/images/clam_chowder/2024817.jpg, food-101/images/clam_chowder/2027156.jpg, food-101/images/clam_chowder/2113399.jpg, food-101/images/clam_chowder/2140406.jpg, food-101/images/clam_chowder/2140703.jpg, food-101/images/clam_chowder/2148133.jpg, food-101/images/clam_chowder/2219398.jpg, food-101/images/clam_chowder/2248997.jpg, food-101/images/clam_chowder/2305126.jpg, food-101/images/clam_chowder/2340686.jpg, food-101/images/clam_chowder/2361787.jpg, food-101/images/clam_chowder/2396225.jpg, food-101/images/clam_chowder/2398168.jpg, food-101/images/clam_chowder/2426821.jpg, food-101/images/clam_chowder/2476027.jpg, food-101/images/clam_chowder/2499937.jpg, food-101/images/clam_chowder/2508514.jpg, food-101/images/clam_chowder/2509774.jpg, food-101/images/clam_chowder/2523293.jpg, food-101/images/clam_chowder/2542800.jpg, food-101/images/clam_chowder/2599110.jpg, food-101/images/clam_chowder/2602712.jpg, food-101/images/clam_chowder/2603953.jpg, food-101/images/clam_chowder/2673628.jpg, food-101/images/clam_chowder/2676197.jpg, food-101/images/clam_chowder/2685745.jpg, food-101/images/clam_chowder/2742139.jpg, food-101/images/clam_chowder/2770581.jpg, food-101/images/clam_chowder/2789238.jpg, food-101/images/clam_chowder/2862215.jpg, food-101/images/clam_chowder/2869301.jpg, food-101/images/clam_chowder/2871657.jpg, food-101/images/clam_chowder/2888266.jpg, food-101/images/clam_chowder/2894611.jpg, food-101/images/clam_chowder/2897057.jpg, food-101/images/clam_chowder/2926608.jpg, food-101/images/clam_chowder/2961270.jpg, food-101/images/clam_chowder/298228.jpg, food-101/images/clam_chowder/2992252.jpg, food-101/images/clam_chowder/3018696.jpg, food-101/images/clam_chowder/3045182.jpg, food-101/images/clam_chowder/3073323.jpg, food-101/images/clam_chowder/3081370.jpg, food-101/images/clam_chowder/3107862.jpg, food-101/images/clam_chowder/3109973.jpg, food-101/images/clam_chowder/3126055.jpg, food-101/images/clam_chowder/3137773.jpg, food-101/images/clam_chowder/3142771.jpg, food-101/images/clam_chowder/3228022.jpg, food-101/images/clam_chowder/3264840.jpg, food-101/images/clam_chowder/3291238.jpg, food-101/images/clam_chowder/3322877.jpg, food-101/images/clam_chowder/3395059.jpg, food-101/images/clam_chowder/3549975.jpg, food-101/images/clam_chowder/3588064.jpg, food-101/images/clam_chowder/3656593.jpg, food-101/images/clam_chowder/3689947.jpg, food-101/images/clam_chowder/3830343.jpg, food-101/images/clam_chowder/3853148.jpg, food-101/images/clam_chowder/390727.jpg, food-101/images/clam_chowder/3914755.jpg, food-101/images/clam_chowder/513498.jpg, food-101/images/clam_chowder/546975.jpg, food-101/images/clam_chowder/596255.jpg, food-101/images/clam_chowder/655847.jpg, food-101/images/clam_chowder/686716.jpg, food-101/images/clam_chowder/75800.jpg, food-101/images/clam_chowder/758162.jpg, food-101/images/clam_chowder/762499.jpg, food-101/images/clam_chowder/771369.jpg, food-101/images/clam_chowder/772778.jpg, food-101/images/clam_chowder/777422.jpg, food-101/images/clam_chowder/795839.jpg, ...] \n", "24712 [food-101/images/club_sandwich/1158836.jpg, food-101/images/club_sandwich/122236.jpg, food-101/images/club_sandwich/1243806.jpg, food-101/images/club_sandwich/1297247.jpg, food-101/images/club_sandwich/1318118.jpg, food-101/images/club_sandwich/1413794.jpg, food-101/images/club_sandwich/1719340.jpg, food-101/images/club_sandwich/1775789.jpg, food-101/images/club_sandwich/1811271.jpg, food-101/images/club_sandwich/1818664.jpg, food-101/images/club_sandwich/1840706.jpg, food-101/images/club_sandwich/1886101.jpg, food-101/images/club_sandwich/191288.jpg, food-101/images/club_sandwich/2163422.jpg, food-101/images/club_sandwich/2214212.jpg, food-101/images/club_sandwich/2272423.jpg, food-101/images/club_sandwich/2778614.jpg, food-101/images/club_sandwich/3106065.jpg, food-101/images/club_sandwich/3132951.jpg, food-101/images/club_sandwich/3251770.jpg, food-101/images/club_sandwich/3310968.jpg, food-101/images/club_sandwich/3526250.jpg, food-101/images/club_sandwich/3543955.jpg, food-101/images/club_sandwich/3550782.jpg, food-101/images/club_sandwich/3646665.jpg, food-101/images/club_sandwich/3664710.jpg, food-101/images/club_sandwich/574816.jpg, food-101/images/club_sandwich/588478.jpg, food-101/images/crab_cakes/3467918.jpg, food-101/images/fish_and_chips/1195292.jpg, food-101/images/fish_and_chips/2484873.jpg, food-101/images/fish_and_chips/3616236.jpg, food-101/images/french_fries/1049122.jpg, food-101/images/french_fries/1071481.jpg, food-101/images/french_fries/1074446.jpg, food-101/images/french_fries/1080215.jpg, food-101/images/french_fries/109401.jpg, food-101/images/french_fries/1099260.jpg, food-101/images/french_fries/1150103.jpg, food-101/images/french_fries/1165966.jpg, food-101/images/french_fries/1221000.jpg, food-101/images/french_fries/1295274.jpg, food-101/images/french_fries/1327248.jpg, food-101/images/french_fries/1361604.jpg, food-101/images/french_fries/1369116.jpg, food-101/images/french_fries/1370900.jpg, food-101/images/french_fries/1384733.jpg, food-101/images/french_fries/1394977.jpg, food-101/images/french_fries/147628.jpg, food-101/images/french_fries/1504529.jpg, food-101/images/french_fries/1509620.jpg, food-101/images/french_fries/1538500.jpg, food-101/images/french_fries/1552191.jpg, food-101/images/french_fries/1610240.jpg, food-101/images/french_fries/1622796.jpg, food-101/images/french_fries/166821.jpg, food-101/images/french_fries/168924.jpg, food-101/images/french_fries/1692353.jpg, food-101/images/french_fries/1700344.jpg, food-101/images/french_fries/1712331.jpg, food-101/images/french_fries/1724805.jpg, food-101/images/french_fries/1740113.jpg, food-101/images/french_fries/1745333.jpg, food-101/images/french_fries/1763119.jpg, food-101/images/french_fries/1810352.jpg, food-101/images/french_fries/184954.jpg, food-101/images/french_fries/1850310.jpg, food-101/images/french_fries/1875469.jpg, food-101/images/french_fries/1897333.jpg, food-101/images/french_fries/1928660.jpg, food-101/images/french_fries/1936184.jpg, food-101/images/french_fries/1969264.jpg, food-101/images/french_fries/2001959.jpg, food-101/images/french_fries/2011974.jpg, food-101/images/french_fries/2041482.jpg, food-101/images/french_fries/2056007.jpg, food-101/images/french_fries/2073415.jpg, food-101/images/french_fries/2080959.jpg, food-101/images/french_fries/2127530.jpg, food-101/images/french_fries/2189231.jpg, food-101/images/french_fries/2212633.jpg, food-101/images/french_fries/2226284.jpg, food-101/images/french_fries/2239352.jpg, food-101/images/french_fries/2246387.jpg, food-101/images/french_fries/2248373.jpg, food-101/images/french_fries/2269039.jpg, food-101/images/french_fries/2278380.jpg, food-101/images/french_fries/2348229.jpg, food-101/images/french_fries/2369999.jpg, food-101/images/french_fries/2371960.jpg, food-101/images/french_fries/2452990.jpg, food-101/images/french_fries/2485920.jpg, food-101/images/french_fries/2493036.jpg, food-101/images/french_fries/2521012.jpg, food-101/images/french_fries/2521724.jpg, food-101/images/french_fries/2624567.jpg, food-101/images/french_fries/2649729.jpg, food-101/images/french_fries/2672237.jpg, food-101/images/french_fries/2700217.jpg, food-101/images/french_fries/2708944.jpg, ...] \n", "31543 [food-101/images/dumplings/1016850.jpg, food-101/images/dumplings/1021310.jpg, food-101/images/dumplings/1045500.jpg, food-101/images/dumplings/1066326.jpg, food-101/images/dumplings/1086100.jpg, food-101/images/dumplings/1134035.jpg, food-101/images/dumplings/1228546.jpg, food-101/images/dumplings/1243140.jpg, food-101/images/dumplings/1245056.jpg, food-101/images/dumplings/1269986.jpg, food-101/images/dumplings/1270308.jpg, food-101/images/dumplings/1275130.jpg, food-101/images/dumplings/1276808.jpg, food-101/images/dumplings/1308246.jpg, food-101/images/dumplings/1376074.jpg, food-101/images/dumplings/140004.jpg, food-101/images/dumplings/1433645.jpg, food-101/images/dumplings/1450685.jpg, food-101/images/dumplings/1542637.jpg, food-101/images/dumplings/1545564.jpg, food-101/images/dumplings/1563646.jpg, food-101/images/dumplings/1564985.jpg, food-101/images/dumplings/1598923.jpg, food-101/images/dumplings/1630799.jpg, food-101/images/dumplings/1664893.jpg, food-101/images/dumplings/1693794.jpg, food-101/images/dumplings/1695231.jpg, food-101/images/dumplings/1789500.jpg, food-101/images/dumplings/1813271.jpg, food-101/images/dumplings/1848359.jpg, food-101/images/dumplings/1848509.jpg, food-101/images/dumplings/1872410.jpg, food-101/images/dumplings/1881086.jpg, food-101/images/dumplings/1897260.jpg, food-101/images/dumplings/1918394.jpg, food-101/images/dumplings/1925731.jpg, food-101/images/dumplings/1927353.jpg, food-101/images/dumplings/1948609.jpg, food-101/images/dumplings/1998135.jpg, food-101/images/dumplings/2000371.jpg, food-101/images/dumplings/2052342.jpg, food-101/images/dumplings/211325.jpg, food-101/images/dumplings/2154990.jpg, food-101/images/dumplings/2166764.jpg, food-101/images/dumplings/2191822.jpg, food-101/images/dumplings/2199788.jpg, food-101/images/dumplings/2229749.jpg, food-101/images/dumplings/231024.jpg, food-101/images/dumplings/231026.jpg, food-101/images/dumplings/231028.jpg, food-101/images/dumplings/2330342.jpg, food-101/images/dumplings/233293.jpg, food-101/images/dumplings/2362785.jpg, food-101/images/dumplings/2364879.jpg, food-101/images/dumplings/2373653.jpg, food-101/images/dumplings/2379541.jpg, food-101/images/dumplings/2380724.jpg, food-101/images/dumplings/2444294.jpg, food-101/images/dumplings/2450566.jpg, food-101/images/dumplings/2478984.jpg, food-101/images/dumplings/2492331.jpg, food-101/images/dumplings/2500721.jpg, food-101/images/dumplings/2524385.jpg, food-101/images/dumplings/2537158.jpg, food-101/images/dumplings/2538305.jpg, food-101/images/dumplings/2545565.jpg, food-101/images/dumplings/2546897.jpg, food-101/images/dumplings/2561548.jpg, food-101/images/dumplings/2571523.jpg, food-101/images/dumplings/2585099.jpg, food-101/images/dumplings/2600333.jpg, food-101/images/dumplings/2606645.jpg, food-101/images/dumplings/2630977.jpg, food-101/images/dumplings/263589.jpg, food-101/images/dumplings/263764.jpg, food-101/images/dumplings/2642265.jpg, food-101/images/dumplings/2675187.jpg, food-101/images/dumplings/267852.jpg, food-101/images/dumplings/2707946.jpg, food-101/images/dumplings/2736144.jpg, food-101/images/dumplings/2750447.jpg, food-101/images/dumplings/2770853.jpg, food-101/images/dumplings/2800182.jpg, food-101/images/dumplings/28220.jpg, food-101/images/dumplings/2834398.jpg, food-101/images/dumplings/2865709.jpg, food-101/images/dumplings/2904302.jpg, food-101/images/dumplings/2909040.jpg, food-101/images/dumplings/2950605.jpg, food-101/images/dumplings/2951551.jpg, food-101/images/dumplings/2975772.jpg, food-101/images/dumplings/3030550.jpg, food-101/images/dumplings/3030640.jpg, food-101/images/dumplings/3042784.jpg, food-101/images/dumplings/3101730.jpg, food-101/images/dumplings/3101737.jpg, food-101/images/dumplings/310672.jpg, food-101/images/dumplings/3182351.jpg, food-101/images/dumplings/3191742.jpg, food-101/images/dumplings/322034.jpg, ...] \n", "5021 [food-101/images/beet_salad/1217698.jpg, food-101/images/ceviche/804338.jpg, food-101/images/fried_rice/1019981.jpg, food-101/images/fried_rice/1023430.jpg, food-101/images/fried_rice/1054627.jpg, food-101/images/fried_rice/1076217.jpg, food-101/images/fried_rice/1163008.jpg, food-101/images/fried_rice/1185768.jpg, food-101/images/fried_rice/1200965.jpg, food-101/images/fried_rice/1242614.jpg, food-101/images/fried_rice/1264045.jpg, food-101/images/fried_rice/1281471.jpg, food-101/images/fried_rice/1285028.jpg, food-101/images/fried_rice/1311992.jpg, food-101/images/fried_rice/1315188.jpg, food-101/images/fried_rice/1323164.jpg, food-101/images/fried_rice/1396018.jpg, food-101/images/fried_rice/1421149.jpg, food-101/images/fried_rice/1429619.jpg, food-101/images/fried_rice/1534541.jpg, food-101/images/fried_rice/1610064.jpg, food-101/images/fried_rice/1677058.jpg, food-101/images/fried_rice/1691031.jpg, food-101/images/fried_rice/1727509.jpg, food-101/images/fried_rice/1774109.jpg, food-101/images/fried_rice/1802914.jpg, food-101/images/fried_rice/1865368.jpg, food-101/images/fried_rice/1874287.jpg, food-101/images/fried_rice/1915049.jpg, food-101/images/fried_rice/2009382.jpg, food-101/images/fried_rice/2022629.jpg, food-101/images/fried_rice/2029077.jpg, food-101/images/fried_rice/2078444.jpg, food-101/images/fried_rice/2136311.jpg, food-101/images/fried_rice/2180004.jpg, food-101/images/fried_rice/2186269.jpg, food-101/images/fried_rice/2261572.jpg, food-101/images/fried_rice/2406661.jpg, food-101/images/fried_rice/2543815.jpg, food-101/images/fried_rice/2552407.jpg, food-101/images/fried_rice/2648223.jpg, food-101/images/fried_rice/2697544.jpg, food-101/images/fried_rice/2699922.jpg, food-101/images/fried_rice/2785231.jpg, food-101/images/fried_rice/2850904.jpg, food-101/images/fried_rice/3034238.jpg, food-101/images/fried_rice/3052175.jpg, food-101/images/fried_rice/3134947.jpg, food-101/images/fried_rice/3277448.jpg, food-101/images/fried_rice/3295993.jpg, food-101/images/fried_rice/3321202.jpg, food-101/images/fried_rice/3331345.jpg, food-101/images/fried_rice/3367571.jpg, food-101/images/fried_rice/3387843.jpg, food-101/images/fried_rice/3402627.jpg, food-101/images/fried_rice/3436140.jpg, food-101/images/fried_rice/3592807.jpg, food-101/images/fried_rice/3620105.jpg, food-101/images/fried_rice/3641662.jpg, food-101/images/fried_rice/3708254.jpg, food-101/images/fried_rice/3766596.jpg, food-101/images/fried_rice/3824406.jpg, food-101/images/fried_rice/3917282.jpg, food-101/images/fried_rice/421911.jpg, food-101/images/fried_rice/461757.jpg, food-101/images/fried_rice/568773.jpg, food-101/images/fried_rice/578945.jpg, food-101/images/fried_rice/583326.jpg, food-101/images/fried_rice/647756.jpg, food-101/images/fried_rice/709970.jpg, food-101/images/fried_rice/919785.jpg, food-101/images/fried_rice/931764.jpg, food-101/images/fried_rice/983544.jpg, food-101/images/gnocchi/2022753.jpg, food-101/images/greek_salad/1647381.jpg, food-101/images/nachos/3234783.jpg, food-101/images/pad_thai/1898870.jpg, food-101/images/pad_thai/2319689.jpg, food-101/images/risotto/129835.jpg, food-101/images/risotto/1615122.jpg, food-101/images/risotto/2311593.jpg, food-101/images/risotto/2512272.jpg, food-101/images/risotto/2600659.jpg, food-101/images/risotto/2976547.jpg, food-101/images/risotto/3786320.jpg, food-101/images/risotto/695270.jpg, food-101/images/risotto/713463.jpg, food-101/images/risotto/919251.jpg, food-101/images/risotto/960618.jpg, food-101/images/scallops/209391.jpg, food-101/images/scallops/2995961.jpg, food-101/images/shrimp_and_grits/1043600.jpg, food-101/images/shrimp_and_grits/1787413.jpg, food-101/images/shrimp_and_grits/1996765.jpg, food-101/images/shrimp_and_grits/2292751.jpg, food-101/images/shrimp_and_grits/2533991.jpg, food-101/images/shrimp_and_grits/3086347.jpg, food-101/images/shrimp_and_grits/3281299.jpg] \n", "32388 [food-101/images/edamame/1047955.jpg, food-101/images/edamame/1086703.jpg, food-101/images/edamame/113525.jpg, food-101/images/edamame/1191326.jpg, food-101/images/edamame/1474750.jpg, food-101/images/edamame/1541444.jpg, food-101/images/edamame/1620027.jpg, food-101/images/edamame/1677511.jpg, food-101/images/edamame/1714523.jpg, food-101/images/edamame/1753683.jpg, food-101/images/edamame/2035309.jpg, food-101/images/edamame/2040753.jpg, food-101/images/edamame/2204418.jpg, food-101/images/edamame/2241023.jpg, food-101/images/edamame/2275716.jpg, food-101/images/edamame/2390868.jpg, food-101/images/edamame/239734.jpg, food-101/images/edamame/2399411.jpg, food-101/images/edamame/2473555.jpg, food-101/images/edamame/2483789.jpg, food-101/images/edamame/2499082.jpg, food-101/images/edamame/2588718.jpg, food-101/images/edamame/2670224.jpg, food-101/images/edamame/2708664.jpg, food-101/images/edamame/2789682.jpg, food-101/images/edamame/2846777.jpg, food-101/images/edamame/2847124.jpg, food-101/images/edamame/2916151.jpg, food-101/images/edamame/3041151.jpg, food-101/images/edamame/3253578.jpg, food-101/images/edamame/3325153.jpg, food-101/images/edamame/3432193.jpg, food-101/images/edamame/3520889.jpg, food-101/images/edamame/3555531.jpg, food-101/images/edamame/3558096.jpg, food-101/images/edamame/3620419.jpg, food-101/images/edamame/3666348.jpg, food-101/images/edamame/3788141.jpg, food-101/images/edamame/3831507.jpg, food-101/images/edamame/601042.jpg, food-101/images/edamame/677508.jpg, food-101/images/edamame/684483.jpg, food-101/images/edamame/825581.jpg, food-101/images/edamame/952423.jpg] \n", "\n", " mean_distance count \n", "component_id \n", "18214 0.9139 754 \n", "24712 0.9144 464 \n", "31543 0.9142 139 \n", "5021 0.9103 98 \n", "32388 0.9089 44 " ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "clusters_df = get_clusters(connected_components_df)\n", "clusters_df.head()" ] }, { "cell_type": "markdown", "id": "c2f5e919", "metadata": {}, "source": [ "The above shows the component (clusters) with the highest duplicates/near-duplicates." ] }, { "cell_type": "markdown", "id": "cc367db9", "metadata": {}, "source": [ "Now let's keep one image from each cluster and remove the rest:\n", "\n" ] }, { "cell_type": "code", "execution_count": 13, "id": "1eb7e07e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Found 2893 highly similar images to discard\n" ] } ], "source": [ "# First sample from each cluster that is kept\n", "cluster_images_to_keep = []\n", "list_of_duplicates = []\n", "\n", "for cluster_file_list in clusters_df.filename:\n", " # keep first file, discard rest\n", " keep = cluster_file_list[0]\n", " discard = cluster_file_list[1:]\n", " \n", " cluster_images_to_keep.append(keep)\n", " list_of_duplicates.extend(discard)\n", "\n", "print(f\"Found {len(set(list_of_duplicates))} highly similar images to discard\")" ] }, { "cell_type": "code", "execution_count": 14, "id": "911029fb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['food-101/images/chicken_curry/2451199.jpg',\n", " 'food-101/images/chicken_curry/2701143.jpg',\n", " 'food-101/images/chicken_curry/3496679.jpg',\n", " 'food-101/images/chicken_curry/66223.jpg',\n", " 'food-101/images/chicken_curry/844686.jpg',\n", " 'food-101/images/chicken_curry/882723.jpg',\n", " 'food-101/images/chicken_curry/968094.jpg',\n", " 'food-101/images/clam_chowder/1023891.jpg',\n", " 'food-101/images/clam_chowder/1072684.jpg',\n", " 'food-101/images/clam_chowder/1113834.jpg',\n", " 'food-101/images/clam_chowder/1156854.jpg',\n", " 'food-101/images/clam_chowder/1223949.jpg',\n", " 'food-101/images/clam_chowder/1242818.jpg',\n", " 'food-101/images/clam_chowder/1322415.jpg',\n", " 'food-101/images/clam_chowder/1418012.jpg',\n", " 'food-101/images/clam_chowder/1437241.jpg',\n", " 'food-101/images/clam_chowder/1511884.jpg',\n", " 'food-101/images/clam_chowder/1563396.jpg',\n", " 'food-101/images/clam_chowder/1581820.jpg',\n", " 'food-101/images/clam_chowder/1598939.jpg',\n", " 'food-101/images/clam_chowder/1742147.jpg',\n", " 'food-101/images/clam_chowder/1871872.jpg',\n", " 'food-101/images/clam_chowder/1942294.jpg',\n", " 'food-101/images/clam_chowder/1945594.jpg',\n", " 'food-101/images/clam_chowder/1993404.jpg',\n", " 'food-101/images/clam_chowder/2018024.jpg',\n", " 'food-101/images/clam_chowder/2024817.jpg',\n", " 'food-101/images/clam_chowder/2027156.jpg',\n", " 'food-101/images/clam_chowder/2113399.jpg',\n", " 'food-101/images/clam_chowder/2140406.jpg',\n", " 'food-101/images/clam_chowder/2140703.jpg',\n", " 'food-101/images/clam_chowder/2148133.jpg',\n", " 'food-101/images/clam_chowder/2219398.jpg',\n", " 'food-101/images/clam_chowder/2248997.jpg',\n", " 'food-101/images/clam_chowder/2305126.jpg',\n", " 'food-101/images/clam_chowder/2340686.jpg',\n", " 'food-101/images/clam_chowder/2361787.jpg',\n", " 'food-101/images/clam_chowder/2396225.jpg',\n", " 'food-101/images/clam_chowder/2398168.jpg',\n", " 'food-101/images/clam_chowder/2426821.jpg',\n", " 'food-101/images/clam_chowder/2476027.jpg',\n", " 'food-101/images/clam_chowder/2499937.jpg',\n", " 'food-101/images/clam_chowder/2508514.jpg',\n", " 'food-101/images/clam_chowder/2509774.jpg',\n", " 'food-101/images/clam_chowder/2523293.jpg',\n", " 'food-101/images/clam_chowder/2542800.jpg',\n", " 'food-101/images/clam_chowder/2599110.jpg',\n", " 'food-101/images/clam_chowder/2602712.jpg',\n", " 'food-101/images/clam_chowder/2603953.jpg',\n", " 'food-101/images/clam_chowder/2673628.jpg',\n", " 'food-101/images/clam_chowder/2676197.jpg',\n", " 'food-101/images/clam_chowder/2685745.jpg',\n", " 'food-101/images/clam_chowder/2742139.jpg',\n", " 'food-101/images/clam_chowder/2770581.jpg',\n", " 'food-101/images/clam_chowder/2789238.jpg',\n", " 'food-101/images/clam_chowder/2862215.jpg',\n", " 'food-101/images/clam_chowder/2869301.jpg',\n", " 'food-101/images/clam_chowder/2871657.jpg',\n", " 'food-101/images/clam_chowder/2888266.jpg',\n", " 'food-101/images/clam_chowder/2894611.jpg',\n", " 'food-101/images/clam_chowder/2897057.jpg',\n", " 'food-101/images/clam_chowder/2926608.jpg',\n", " 'food-101/images/clam_chowder/2961270.jpg',\n", " 'food-101/images/clam_chowder/298228.jpg',\n", " 'food-101/images/clam_chowder/2992252.jpg',\n", " 'food-101/images/clam_chowder/3018696.jpg',\n", " 'food-101/images/clam_chowder/3045182.jpg',\n", " 'food-101/images/clam_chowder/3073323.jpg',\n", " 'food-101/images/clam_chowder/3081370.jpg',\n", " 'food-101/images/clam_chowder/3107862.jpg',\n", " 'food-101/images/clam_chowder/3109973.jpg',\n", " 'food-101/images/clam_chowder/3126055.jpg',\n", " 'food-101/images/clam_chowder/3137773.jpg',\n", " 'food-101/images/clam_chowder/3142771.jpg',\n", " 'food-101/images/clam_chowder/3228022.jpg',\n", " 'food-101/images/clam_chowder/3264840.jpg',\n", " 'food-101/images/clam_chowder/3291238.jpg',\n", " 'food-101/images/clam_chowder/3322877.jpg',\n", " 'food-101/images/clam_chowder/3395059.jpg',\n", " 'food-101/images/clam_chowder/3549975.jpg',\n", " 'food-101/images/clam_chowder/3588064.jpg',\n", " 'food-101/images/clam_chowder/3656593.jpg',\n", " 'food-101/images/clam_chowder/3689947.jpg',\n", " 'food-101/images/clam_chowder/3830343.jpg',\n", " 'food-101/images/clam_chowder/3853148.jpg',\n", " 'food-101/images/clam_chowder/390727.jpg',\n", " 'food-101/images/clam_chowder/3914755.jpg',\n", " 'food-101/images/clam_chowder/513498.jpg',\n", " 'food-101/images/clam_chowder/546975.jpg',\n", " 'food-101/images/clam_chowder/596255.jpg',\n", " 'food-101/images/clam_chowder/655847.jpg',\n", " 'food-101/images/clam_chowder/686716.jpg',\n", " 'food-101/images/clam_chowder/75800.jpg',\n", " 'food-101/images/clam_chowder/758162.jpg',\n", " 'food-101/images/clam_chowder/762499.jpg',\n", " 'food-101/images/clam_chowder/771369.jpg',\n", " 'food-101/images/clam_chowder/772778.jpg',\n", " 'food-101/images/clam_chowder/777422.jpg',\n", " 'food-101/images/clam_chowder/795839.jpg',\n", " 'food-101/images/clam_chowder/804904.jpg',\n", " 'food-101/images/clam_chowder/854517.jpg',\n", " 'food-101/images/clam_chowder/903815.jpg',\n", " 'food-101/images/clam_chowder/906900.jpg',\n", " 'food-101/images/clam_chowder/907742.jpg',\n", " 'food-101/images/clam_chowder/924933.jpg',\n", " 'food-101/images/clam_chowder/947484.jpg',\n", " 'food-101/images/clam_chowder/967946.jpg',\n", " 'food-101/images/creme_brulee/1107026.jpg',\n", " 'food-101/images/creme_brulee/1207812.jpg',\n", " 'food-101/images/creme_brulee/1447258.jpg',\n", " 'food-101/images/creme_brulee/1453226.jpg',\n", " 'food-101/images/creme_brulee/157134.jpg',\n", " 'food-101/images/creme_brulee/1742194.jpg',\n", " 'food-101/images/creme_brulee/1816938.jpg',\n", " 'food-101/images/creme_brulee/1916393.jpg',\n", " 'food-101/images/creme_brulee/2186943.jpg',\n", " 'food-101/images/creme_brulee/2218106.jpg',\n", " 'food-101/images/creme_brulee/2418653.jpg',\n", " 'food-101/images/creme_brulee/2602002.jpg',\n", " 'food-101/images/creme_brulee/2610691.jpg',\n", " 'food-101/images/creme_brulee/312639.jpg',\n", " 'food-101/images/creme_brulee/332369.jpg',\n", " 'food-101/images/creme_brulee/3684311.jpg',\n", " 'food-101/images/creme_brulee/3686304.jpg',\n", " 'food-101/images/creme_brulee/3739967.jpg',\n", " 'food-101/images/creme_brulee/480234.jpg',\n", " 'food-101/images/creme_brulee/59534.jpg',\n", " 'food-101/images/escargots/3587901.jpg',\n", " 'food-101/images/french_onion_soup/1020156.jpg',\n", " 'food-101/images/french_onion_soup/1062494.jpg',\n", " 'food-101/images/french_onion_soup/1161976.jpg',\n", " 'food-101/images/french_onion_soup/1350426.jpg',\n", " 'food-101/images/french_onion_soup/1679160.jpg',\n", " 'food-101/images/french_onion_soup/2205878.jpg',\n", " 'food-101/images/french_onion_soup/2455859.jpg',\n", " 'food-101/images/french_onion_soup/245738.jpg',\n", " 'food-101/images/french_onion_soup/2482062.jpg',\n", " 'food-101/images/french_onion_soup/2582633.jpg',\n", " 'food-101/images/french_onion_soup/2623210.jpg',\n", " 'food-101/images/french_onion_soup/2740521.jpg',\n", " 'food-101/images/french_onion_soup/2796610.jpg',\n", " 'food-101/images/french_onion_soup/2810153.jpg',\n", " 'food-101/images/french_onion_soup/2962330.jpg',\n", " 'food-101/images/french_onion_soup/397460.jpg',\n", " 'food-101/images/french_onion_soup/539296.jpg',\n", " 'food-101/images/french_onion_soup/759629.jpg',\n", " 'food-101/images/french_onion_soup/883844.jpg',\n", " 'food-101/images/gnocchi/3791682.jpg',\n", " 'food-101/images/hot_and_sour_soup/1000486.jpg',\n", " 'food-101/images/hot_and_sour_soup/1008701.jpg',\n", " 'food-101/images/hot_and_sour_soup/1055929.jpg',\n", " 'food-101/images/hot_and_sour_soup/1069789.jpg',\n", " 'food-101/images/hot_and_sour_soup/1070065.jpg',\n", " 'food-101/images/hot_and_sour_soup/1083094.jpg',\n", " 'food-101/images/hot_and_sour_soup/1122875.jpg',\n", " 'food-101/images/hot_and_sour_soup/1126455.jpg',\n", " 'food-101/images/hot_and_sour_soup/1134579.jpg',\n", " 'food-101/images/hot_and_sour_soup/113560.jpg',\n", " 'food-101/images/hot_and_sour_soup/1151861.jpg',\n", " 'food-101/images/hot_and_sour_soup/1167380.jpg',\n", " 'food-101/images/hot_and_sour_soup/1168184.jpg',\n", " 'food-101/images/hot_and_sour_soup/1198713.jpg',\n", " 'food-101/images/hot_and_sour_soup/1214786.jpg',\n", " 'food-101/images/hot_and_sour_soup/1222219.jpg',\n", " 'food-101/images/hot_and_sour_soup/12248.jpg',\n", " 'food-101/images/hot_and_sour_soup/1226605.jpg',\n", " 'food-101/images/hot_and_sour_soup/1231074.jpg',\n", " 'food-101/images/hot_and_sour_soup/1235094.jpg',\n", " 'food-101/images/hot_and_sour_soup/1240584.jpg',\n", " 'food-101/images/hot_and_sour_soup/1285071.jpg',\n", " 'food-101/images/hot_and_sour_soup/1288010.jpg',\n", " 'food-101/images/hot_and_sour_soup/1312635.jpg',\n", " 'food-101/images/hot_and_sour_soup/1313048.jpg',\n", " 'food-101/images/hot_and_sour_soup/1316758.jpg',\n", " 'food-101/images/hot_and_sour_soup/1337126.jpg',\n", " 'food-101/images/hot_and_sour_soup/1347323.jpg',\n", " 'food-101/images/hot_and_sour_soup/1363828.jpg',\n", " 'food-101/images/hot_and_sour_soup/1364804.jpg',\n", " 'food-101/images/hot_and_sour_soup/1364912.jpg',\n", " 'food-101/images/hot_and_sour_soup/1400511.jpg',\n", " 'food-101/images/hot_and_sour_soup/1406237.jpg',\n", " 'food-101/images/hot_and_sour_soup/1428041.jpg',\n", " 'food-101/images/hot_and_sour_soup/1446456.jpg',\n", " 'food-101/images/hot_and_sour_soup/1449680.jpg',\n", " 'food-101/images/hot_and_sour_soup/1474021.jpg',\n", " 'food-101/images/hot_and_sour_soup/1479709.jpg',\n", " 'food-101/images/hot_and_sour_soup/1484098.jpg',\n", " 'food-101/images/hot_and_sour_soup/148669.jpg',\n", " 'food-101/images/hot_and_sour_soup/1502582.jpg',\n", " 'food-101/images/hot_and_sour_soup/1519845.jpg',\n", " 'food-101/images/hot_and_sour_soup/154363.jpg',\n", " 'food-101/images/hot_and_sour_soup/1548700.jpg',\n", " 'food-101/images/hot_and_sour_soup/1556137.jpg',\n", " 'food-101/images/hot_and_sour_soup/1557701.jpg',\n", " 'food-101/images/hot_and_sour_soup/1563900.jpg',\n", " 'food-101/images/hot_and_sour_soup/1577046.jpg',\n", " 'food-101/images/hot_and_sour_soup/1583224.jpg',\n", " 'food-101/images/hot_and_sour_soup/1607385.jpg',\n", " 'food-101/images/hot_and_sour_soup/1611156.jpg',\n", " 'food-101/images/hot_and_sour_soup/1617113.jpg',\n", " 'food-101/images/hot_and_sour_soup/162622.jpg',\n", " 'food-101/images/hot_and_sour_soup/1633728.jpg',\n", " 'food-101/images/hot_and_sour_soup/1633890.jpg',\n", " 'food-101/images/hot_and_sour_soup/1635802.jpg',\n", " 'food-101/images/hot_and_sour_soup/1653899.jpg',\n", " 'food-101/images/hot_and_sour_soup/1670529.jpg',\n", " 'food-101/images/hot_and_sour_soup/1683364.jpg',\n", " 'food-101/images/hot_and_sour_soup/1805255.jpg',\n", " 'food-101/images/hot_and_sour_soup/1827236.jpg',\n", " 'food-101/images/hot_and_sour_soup/184398.jpg',\n", " 'food-101/images/hot_and_sour_soup/1858145.jpg',\n", " 'food-101/images/hot_and_sour_soup/1858236.jpg',\n", " 'food-101/images/hot_and_sour_soup/1864685.jpg',\n", " 'food-101/images/hot_and_sour_soup/1936292.jpg',\n", " 'food-101/images/hot_and_sour_soup/1938773.jpg',\n", " 'food-101/images/hot_and_sour_soup/1942754.jpg',\n", " 'food-101/images/hot_and_sour_soup/1955928.jpg',\n", " 'food-101/images/hot_and_sour_soup/1959025.jpg',\n", " 'food-101/images/hot_and_sour_soup/1966211.jpg',\n", " 'food-101/images/hot_and_sour_soup/2005531.jpg',\n", " 'food-101/images/hot_and_sour_soup/200587.jpg',\n", " 'food-101/images/hot_and_sour_soup/20067.jpg',\n", " 'food-101/images/hot_and_sour_soup/2041812.jpg',\n", " 'food-101/images/hot_and_sour_soup/204285.jpg',\n", " 'food-101/images/hot_and_sour_soup/204679.jpg',\n", " 'food-101/images/hot_and_sour_soup/209424.jpg',\n", " 'food-101/images/hot_and_sour_soup/209469.jpg',\n", " 'food-101/images/hot_and_sour_soup/2112191.jpg',\n", " 'food-101/images/hot_and_sour_soup/2120287.jpg',\n", " 'food-101/images/hot_and_sour_soup/2172589.jpg',\n", " 'food-101/images/hot_and_sour_soup/217569.jpg',\n", " 'food-101/images/hot_and_sour_soup/2184467.jpg',\n", " 'food-101/images/hot_and_sour_soup/2200930.jpg',\n", " 'food-101/images/hot_and_sour_soup/2202463.jpg',\n", " 'food-101/images/hot_and_sour_soup/222193.jpg',\n", " 'food-101/images/hot_and_sour_soup/2256057.jpg',\n", " 'food-101/images/hot_and_sour_soup/2259445.jpg',\n", " 'food-101/images/hot_and_sour_soup/2273383.jpg',\n", " 'food-101/images/hot_and_sour_soup/2278783.jpg',\n", " 'food-101/images/hot_and_sour_soup/2291396.jpg',\n", " 'food-101/images/hot_and_sour_soup/230163.jpg',\n", " 'food-101/images/hot_and_sour_soup/2312653.jpg',\n", " 'food-101/images/hot_and_sour_soup/2323446.jpg',\n", " 'food-101/images/hot_and_sour_soup/2344945.jpg',\n", " 'food-101/images/hot_and_sour_soup/2367229.jpg',\n", " 'food-101/images/hot_and_sour_soup/2368872.jpg',\n", " 'food-101/images/hot_and_sour_soup/2371233.jpg',\n", " 'food-101/images/hot_and_sour_soup/2377494.jpg',\n", " 'food-101/images/hot_and_sour_soup/2377753.jpg',\n", " 'food-101/images/hot_and_sour_soup/2382833.jpg',\n", " 'food-101/images/hot_and_sour_soup/2415615.jpg',\n", " 'food-101/images/hot_and_sour_soup/2450297.jpg',\n", " 'food-101/images/hot_and_sour_soup/2473701.jpg',\n", " 'food-101/images/hot_and_sour_soup/2495594.jpg',\n", " 'food-101/images/hot_and_sour_soup/2515353.jpg',\n", " 'food-101/images/hot_and_sour_soup/2516789.jpg',\n", " 'food-101/images/hot_and_sour_soup/2520927.jpg',\n", " 'food-101/images/hot_and_sour_soup/2524133.jpg',\n", " 'food-101/images/hot_and_sour_soup/2524577.jpg',\n", " 'food-101/images/hot_and_sour_soup/2528354.jpg',\n", " 'food-101/images/hot_and_sour_soup/2531145.jpg',\n", " 'food-101/images/hot_and_sour_soup/2563123.jpg',\n", " 'food-101/images/hot_and_sour_soup/258760.jpg',\n", " 'food-101/images/hot_and_sour_soup/2597181.jpg',\n", " 'food-101/images/hot_and_sour_soup/2615353.jpg',\n", " 'food-101/images/hot_and_sour_soup/2645587.jpg',\n", " 'food-101/images/hot_and_sour_soup/2659777.jpg',\n", " 'food-101/images/hot_and_sour_soup/2675588.jpg',\n", " 'food-101/images/hot_and_sour_soup/2678235.jpg',\n", " 'food-101/images/hot_and_sour_soup/2685865.jpg',\n", " 'food-101/images/hot_and_sour_soup/2687018.jpg',\n", " 'food-101/images/hot_and_sour_soup/2688464.jpg',\n", " 'food-101/images/hot_and_sour_soup/2700418.jpg',\n", " 'food-101/images/hot_and_sour_soup/2718315.jpg',\n", " 'food-101/images/hot_and_sour_soup/2743122.jpg',\n", " 'food-101/images/hot_and_sour_soup/2754869.jpg',\n", " 'food-101/images/hot_and_sour_soup/2756129.jpg',\n", " 'food-101/images/hot_and_sour_soup/2760407.jpg',\n", " 'food-101/images/hot_and_sour_soup/277065.jpg',\n", " 'food-101/images/hot_and_sour_soup/2772349.jpg',\n", " 'food-101/images/hot_and_sour_soup/2781892.jpg',\n", " 'food-101/images/hot_and_sour_soup/2786483.jpg',\n", " 'food-101/images/hot_and_sour_soup/278768.jpg',\n", " 'food-101/images/hot_and_sour_soup/2790249.jpg',\n", " 'food-101/images/hot_and_sour_soup/2790483.jpg',\n", " 'food-101/images/hot_and_sour_soup/2828157.jpg',\n", " 'food-101/images/hot_and_sour_soup/2843321.jpg',\n", " 'food-101/images/hot_and_sour_soup/285163.jpg',\n", " 'food-101/images/hot_and_sour_soup/2863802.jpg',\n", " 'food-101/images/hot_and_sour_soup/2880758.jpg',\n", " 'food-101/images/hot_and_sour_soup/2889298.jpg',\n", " 'food-101/images/hot_and_sour_soup/2922592.jpg',\n", " 'food-101/images/hot_and_sour_soup/2936421.jpg',\n", " 'food-101/images/hot_and_sour_soup/2941188.jpg',\n", " 'food-101/images/hot_and_sour_soup/2943259.jpg',\n", " 'food-101/images/hot_and_sour_soup/2943708.jpg',\n", " 'food-101/images/hot_and_sour_soup/2954801.jpg',\n", " 'food-101/images/hot_and_sour_soup/2955708.jpg',\n", " 'food-101/images/hot_and_sour_soup/2973919.jpg',\n", " 'food-101/images/hot_and_sour_soup/2979433.jpg',\n", " 'food-101/images/hot_and_sour_soup/2998002.jpg',\n", " 'food-101/images/hot_and_sour_soup/3004353.jpg',\n", " 'food-101/images/hot_and_sour_soup/3023744.jpg',\n", " 'food-101/images/hot_and_sour_soup/3033019.jpg',\n", " 'food-101/images/hot_and_sour_soup/3057104.jpg',\n", " 'food-101/images/hot_and_sour_soup/3086202.jpg',\n", " 'food-101/images/hot_and_sour_soup/3113531.jpg',\n", " 'food-101/images/hot_and_sour_soup/3118313.jpg',\n", " 'food-101/images/hot_and_sour_soup/314042.jpg',\n", " 'food-101/images/hot_and_sour_soup/3155003.jpg',\n", " 'food-101/images/hot_and_sour_soup/3191313.jpg',\n", " 'food-101/images/hot_and_sour_soup/3204149.jpg',\n", " 'food-101/images/hot_and_sour_soup/3216709.jpg',\n", " 'food-101/images/hot_and_sour_soup/3235380.jpg',\n", " 'food-101/images/hot_and_sour_soup/3244591.jpg',\n", " 'food-101/images/hot_and_sour_soup/3257336.jpg',\n", " 'food-101/images/hot_and_sour_soup/325791.jpg',\n", " 'food-101/images/hot_and_sour_soup/3265742.jpg',\n", " 'food-101/images/hot_and_sour_soup/3275927.jpg',\n", " 'food-101/images/hot_and_sour_soup/3286625.jpg',\n", " 'food-101/images/hot_and_sour_soup/3320353.jpg',\n", " 'food-101/images/hot_and_sour_soup/3333255.jpg',\n", " 'food-101/images/hot_and_sour_soup/3362238.jpg',\n", " 'food-101/images/hot_and_sour_soup/3376428.jpg',\n", " 'food-101/images/hot_and_sour_soup/3378533.jpg',\n", " 'food-101/images/hot_and_sour_soup/3382309.jpg',\n", " 'food-101/images/hot_and_sour_soup/340111.jpg',\n", " 'food-101/images/hot_and_sour_soup/342455.jpg',\n", " 'food-101/images/hot_and_sour_soup/3428336.jpg',\n", " 'food-101/images/hot_and_sour_soup/3437487.jpg',\n", " 'food-101/images/hot_and_sour_soup/3446030.jpg',\n", " 'food-101/images/hot_and_sour_soup/3452669.jpg',\n", " 'food-101/images/hot_and_sour_soup/3464402.jpg',\n", " 'food-101/images/hot_and_sour_soup/3496274.jpg',\n", " 'food-101/images/hot_and_sour_soup/3539916.jpg',\n", " 'food-101/images/hot_and_sour_soup/3552976.jpg',\n", " 'food-101/images/hot_and_sour_soup/3565393.jpg',\n", " 'food-101/images/hot_and_sour_soup/3567487.jpg',\n", " 'food-101/images/hot_and_sour_soup/3568665.jpg',\n", " 'food-101/images/hot_and_sour_soup/3601021.jpg',\n", " 'food-101/images/hot_and_sour_soup/3613554.jpg',\n", " 'food-101/images/hot_and_sour_soup/3618218.jpg',\n", " 'food-101/images/hot_and_sour_soup/3644289.jpg',\n", " 'food-101/images/hot_and_sour_soup/3658920.jpg',\n", " 'food-101/images/hot_and_sour_soup/3668485.jpg',\n", " 'food-101/images/hot_and_sour_soup/3673317.jpg',\n", " 'food-101/images/hot_and_sour_soup/3703194.jpg',\n", " 'food-101/images/hot_and_sour_soup/3706507.jpg',\n", " 'food-101/images/hot_and_sour_soup/3781731.jpg',\n", " 'food-101/images/hot_and_sour_soup/3798403.jpg',\n", " 'food-101/images/hot_and_sour_soup/3812573.jpg',\n", " 'food-101/images/hot_and_sour_soup/3819269.jpg',\n", " 'food-101/images/hot_and_sour_soup/3824797.jpg',\n", " 'food-101/images/hot_and_sour_soup/382979.jpg',\n", " 'food-101/images/hot_and_sour_soup/384751.jpg',\n", " 'food-101/images/hot_and_sour_soup/3854857.jpg',\n", " 'food-101/images/hot_and_sour_soup/387487.jpg',\n", " 'food-101/images/hot_and_sour_soup/3879326.jpg',\n", " 'food-101/images/hot_and_sour_soup/3879471.jpg',\n", " 'food-101/images/hot_and_sour_soup/3891611.jpg',\n", " 'food-101/images/hot_and_sour_soup/3918910.jpg',\n", " 'food-101/images/hot_and_sour_soup/414581.jpg',\n", " 'food-101/images/hot_and_sour_soup/420717.jpg',\n", " 'food-101/images/hot_and_sour_soup/437801.jpg',\n", " 'food-101/images/hot_and_sour_soup/442387.jpg',\n", " 'food-101/images/hot_and_sour_soup/449812.jpg',\n", " 'food-101/images/hot_and_sour_soup/478316.jpg',\n", " 'food-101/images/hot_and_sour_soup/495719.jpg',\n", " 'food-101/images/hot_and_sour_soup/497575.jpg',\n", " 'food-101/images/hot_and_sour_soup/54981.jpg',\n", " 'food-101/images/hot_and_sour_soup/554192.jpg',\n", " 'food-101/images/hot_and_sour_soup/557591.jpg',\n", " 'food-101/images/hot_and_sour_soup/561161.jpg',\n", " 'food-101/images/hot_and_sour_soup/564763.jpg',\n", " 'food-101/images/hot_and_sour_soup/579590.jpg',\n", " 'food-101/images/hot_and_sour_soup/611992.jpg',\n", " 'food-101/images/hot_and_sour_soup/613398.jpg',\n", " 'food-101/images/hot_and_sour_soup/615327.jpg',\n", " 'food-101/images/hot_and_sour_soup/619528.jpg',\n", " 'food-101/images/hot_and_sour_soup/661084.jpg',\n", " 'food-101/images/hot_and_sour_soup/669191.jpg',\n", " 'food-101/images/hot_and_sour_soup/721769.jpg',\n", " 'food-101/images/hot_and_sour_soup/723517.jpg',\n", " 'food-101/images/hot_and_sour_soup/729559.jpg',\n", " 'food-101/images/hot_and_sour_soup/770250.jpg',\n", " 'food-101/images/hot_and_sour_soup/776124.jpg',\n", " 'food-101/images/hot_and_sour_soup/788666.jpg',\n", " 'food-101/images/hot_and_sour_soup/843978.jpg',\n", " 'food-101/images/hot_and_sour_soup/854589.jpg',\n", " 'food-101/images/hot_and_sour_soup/856938.jpg',\n", " 'food-101/images/hot_and_sour_soup/886216.jpg',\n", " 'food-101/images/hot_and_sour_soup/892175.jpg',\n", " 'food-101/images/hot_and_sour_soup/924340.jpg',\n", " 'food-101/images/hot_and_sour_soup/942430.jpg',\n", " 'food-101/images/hot_and_sour_soup/943151.jpg',\n", " 'food-101/images/hot_and_sour_soup/947500.jpg',\n", " 'food-101/images/hot_and_sour_soup/965202.jpg',\n", " 'food-101/images/hot_and_sour_soup/979455.jpg',\n", " 'food-101/images/hot_and_sour_soup/982582.jpg',\n", " 'food-101/images/hot_and_sour_soup/993036.jpg',\n", " 'food-101/images/hot_and_sour_soup/997971.jpg',\n", " 'food-101/images/lobster_bisque/1003421.jpg',\n", " 'food-101/images/lobster_bisque/1008682.jpg',\n", " 'food-101/images/lobster_bisque/1030246.jpg',\n", " 'food-101/images/lobster_bisque/1079870.jpg',\n", " 'food-101/images/lobster_bisque/1082597.jpg',\n", " 'food-101/images/lobster_bisque/1119173.jpg',\n", " 'food-101/images/lobster_bisque/1135687.jpg',\n", " 'food-101/images/lobster_bisque/1168951.jpg',\n", " 'food-101/images/lobster_bisque/1172357.jpg',\n", " 'food-101/images/lobster_bisque/1181520.jpg',\n", " 'food-101/images/lobster_bisque/120532.jpg',\n", " 'food-101/images/lobster_bisque/1229730.jpg',\n", " 'food-101/images/lobster_bisque/1235427.jpg',\n", " 'food-101/images/lobster_bisque/1346617.jpg',\n", " 'food-101/images/lobster_bisque/1372543.jpg',\n", " 'food-101/images/lobster_bisque/1399541.jpg',\n", " 'food-101/images/lobster_bisque/1404248.jpg',\n", " 'food-101/images/lobster_bisque/1406169.jpg',\n", " 'food-101/images/lobster_bisque/1439107.jpg',\n", " 'food-101/images/lobster_bisque/1505415.jpg',\n", " 'food-101/images/lobster_bisque/1561678.jpg',\n", " 'food-101/images/lobster_bisque/1618042.jpg',\n", " 'food-101/images/lobster_bisque/1623262.jpg',\n", " 'food-101/images/lobster_bisque/1641764.jpg',\n", " 'food-101/images/lobster_bisque/1715382.jpg',\n", " 'food-101/images/lobster_bisque/1789658.jpg',\n", " 'food-101/images/lobster_bisque/1826587.jpg',\n", " 'food-101/images/lobster_bisque/1901930.jpg',\n", " 'food-101/images/lobster_bisque/1914066.jpg',\n", " 'food-101/images/lobster_bisque/1935600.jpg',\n", " 'food-101/images/lobster_bisque/1988235.jpg',\n", " 'food-101/images/lobster_bisque/1989099.jpg',\n", " 'food-101/images/lobster_bisque/1995069.jpg',\n", " 'food-101/images/lobster_bisque/2005142.jpg',\n", " 'food-101/images/lobster_bisque/2059149.jpg',\n", " 'food-101/images/lobster_bisque/2069023.jpg',\n", " 'food-101/images/lobster_bisque/2140413.jpg',\n", " 'food-101/images/lobster_bisque/2169158.jpg',\n", " 'food-101/images/lobster_bisque/2172321.jpg',\n", " 'food-101/images/lobster_bisque/2179169.jpg',\n", " 'food-101/images/lobster_bisque/2197506.jpg',\n", " 'food-101/images/lobster_bisque/2198032.jpg',\n", " 'food-101/images/lobster_bisque/2210776.jpg',\n", " 'food-101/images/lobster_bisque/2222295.jpg',\n", " 'food-101/images/lobster_bisque/2250767.jpg',\n", " 'food-101/images/lobster_bisque/2265716.jpg',\n", " 'food-101/images/lobster_bisque/2268429.jpg',\n", " 'food-101/images/lobster_bisque/2275539.jpg',\n", " 'food-101/images/lobster_bisque/2290514.jpg',\n", " 'food-101/images/lobster_bisque/2294708.jpg',\n", " 'food-101/images/lobster_bisque/2329671.jpg',\n", " 'food-101/images/lobster_bisque/2358791.jpg',\n", " 'food-101/images/lobster_bisque/2371100.jpg',\n", " 'food-101/images/lobster_bisque/2399818.jpg',\n", " 'food-101/images/lobster_bisque/2519337.jpg',\n", " 'food-101/images/lobster_bisque/2556334.jpg',\n", " 'food-101/images/lobster_bisque/2618116.jpg',\n", " 'food-101/images/lobster_bisque/262244.jpg',\n", " 'food-101/images/lobster_bisque/2644365.jpg',\n", " 'food-101/images/lobster_bisque/2683327.jpg',\n", " 'food-101/images/lobster_bisque/2697188.jpg',\n", " 'food-101/images/lobster_bisque/2720610.jpg',\n", " 'food-101/images/lobster_bisque/274916.jpg',\n", " 'food-101/images/lobster_bisque/2750289.jpg',\n", " 'food-101/images/lobster_bisque/2769569.jpg',\n", " 'food-101/images/lobster_bisque/2795771.jpg',\n", " 'food-101/images/lobster_bisque/2848845.jpg',\n", " 'food-101/images/lobster_bisque/2866110.jpg',\n", " 'food-101/images/lobster_bisque/2866436.jpg',\n", " 'food-101/images/lobster_bisque/2878000.jpg',\n", " 'food-101/images/lobster_bisque/2903478.jpg',\n", " 'food-101/images/lobster_bisque/2910636.jpg',\n", " 'food-101/images/lobster_bisque/2917736.jpg',\n", " 'food-101/images/lobster_bisque/2920368.jpg',\n", " 'food-101/images/lobster_bisque/2955008.jpg',\n", " 'food-101/images/lobster_bisque/2973189.jpg',\n", " 'food-101/images/lobster_bisque/2981838.jpg',\n", " 'food-101/images/lobster_bisque/2984841.jpg',\n", " 'food-101/images/lobster_bisque/3004560.jpg',\n", " 'food-101/images/lobster_bisque/3041886.jpg',\n", " 'food-101/images/lobster_bisque/3076308.jpg',\n", " 'food-101/images/lobster_bisque/3109961.jpg',\n", " 'food-101/images/lobster_bisque/3140029.jpg',\n", " 'food-101/images/lobster_bisque/3144702.jpg',\n", " 'food-101/images/lobster_bisque/3165128.jpg',\n", " 'food-101/images/lobster_bisque/3179216.jpg',\n", " 'food-101/images/lobster_bisque/321057.jpg',\n", " 'food-101/images/lobster_bisque/3221087.jpg',\n", " 'food-101/images/lobster_bisque/3254985.jpg',\n", " 'food-101/images/lobster_bisque/3266002.jpg',\n", " 'food-101/images/lobster_bisque/3282626.jpg',\n", " 'food-101/images/lobster_bisque/3282795.jpg',\n", " 'food-101/images/lobster_bisque/3318185.jpg',\n", " 'food-101/images/lobster_bisque/3319694.jpg',\n", " 'food-101/images/lobster_bisque/3354880.jpg',\n", " 'food-101/images/lobster_bisque/3358721.jpg',\n", " 'food-101/images/lobster_bisque/3374012.jpg',\n", " 'food-101/images/lobster_bisque/337714.jpg',\n", " 'food-101/images/lobster_bisque/3414592.jpg',\n", " 'food-101/images/lobster_bisque/3416128.jpg',\n", " 'food-101/images/lobster_bisque/3418148.jpg',\n", " 'food-101/images/lobster_bisque/3466502.jpg',\n", " 'food-101/images/lobster_bisque/350149.jpg',\n", " 'food-101/images/lobster_bisque/3509758.jpg',\n", " 'food-101/images/lobster_bisque/3520107.jpg',\n", " 'food-101/images/lobster_bisque/3527698.jpg',\n", " 'food-101/images/lobster_bisque/3560547.jpg',\n", " 'food-101/images/lobster_bisque/3586140.jpg',\n", " 'food-101/images/lobster_bisque/3604315.jpg',\n", " 'food-101/images/lobster_bisque/3614535.jpg',\n", " 'food-101/images/lobster_bisque/3652100.jpg',\n", " 'food-101/images/lobster_bisque/3656540.jpg',\n", " 'food-101/images/lobster_bisque/369524.jpg',\n", " 'food-101/images/lobster_bisque/3725606.jpg',\n", " 'food-101/images/lobster_bisque/3808144.jpg',\n", " 'food-101/images/lobster_bisque/3826517.jpg',\n", " 'food-101/images/lobster_bisque/3868773.jpg',\n", " 'food-101/images/lobster_bisque/3869474.jpg',\n", " 'food-101/images/lobster_bisque/3898860.jpg',\n", " 'food-101/images/lobster_bisque/475607.jpg',\n", " 'food-101/images/lobster_bisque/533134.jpg',\n", " 'food-101/images/lobster_bisque/562118.jpg',\n", " 'food-101/images/lobster_bisque/588421.jpg',\n", " 'food-101/images/lobster_bisque/606179.jpg',\n", " 'food-101/images/lobster_bisque/629483.jpg',\n", " 'food-101/images/lobster_bisque/641631.jpg',\n", " 'food-101/images/lobster_bisque/651642.jpg',\n", " 'food-101/images/lobster_bisque/65607.jpg',\n", " 'food-101/images/lobster_bisque/678196.jpg',\n", " 'food-101/images/lobster_bisque/68481.jpg',\n", " 'food-101/images/lobster_bisque/688344.jpg',\n", " 'food-101/images/lobster_bisque/689311.jpg',\n", " 'food-101/images/lobster_bisque/699956.jpg',\n", " 'food-101/images/lobster_bisque/728266.jpg',\n", " 'food-101/images/lobster_bisque/782974.jpg',\n", " 'food-101/images/lobster_bisque/813441.jpg',\n", " 'food-101/images/lobster_bisque/820485.jpg',\n", " 'food-101/images/lobster_bisque/825585.jpg',\n", " 'food-101/images/lobster_bisque/830001.jpg',\n", " 'food-101/images/lobster_bisque/844788.jpg',\n", " 'food-101/images/lobster_bisque/924935.jpg',\n", " 'food-101/images/lobster_bisque/948368.jpg',\n", " 'food-101/images/lobster_bisque/974979.jpg',\n", " 'food-101/images/miso_soup/1096698.jpg',\n", " 'food-101/images/miso_soup/120774.jpg',\n", " 'food-101/images/miso_soup/1293849.jpg',\n", " 'food-101/images/miso_soup/146592.jpg',\n", " 'food-101/images/miso_soup/1616699.jpg',\n", " 'food-101/images/miso_soup/1903256.jpg',\n", " 'food-101/images/miso_soup/2285659.jpg',\n", " 'food-101/images/miso_soup/2412822.jpg',\n", " 'food-101/images/miso_soup/2981398.jpg',\n", " 'food-101/images/miso_soup/3350406.jpg',\n", " 'food-101/images/miso_soup/343304.jpg',\n", " 'food-101/images/miso_soup/3454724.jpg',\n", " 'food-101/images/miso_soup/3605736.jpg',\n", " 'food-101/images/miso_soup/451876.jpg',\n", " 'food-101/images/miso_soup/528721.jpg',\n", " 'food-101/images/miso_soup/574828.jpg',\n", " 'food-101/images/miso_soup/854424.jpg',\n", " 'food-101/images/miso_soup/992468.jpg',\n", " 'food-101/images/pho/1070815.jpg',\n", " 'food-101/images/pho/1076831.jpg',\n", " 'food-101/images/pho/1101330.jpg',\n", " 'food-101/images/pho/1101992.jpg',\n", " 'food-101/images/pho/1105040.jpg',\n", " 'food-101/images/pho/1121706.jpg',\n", " 'food-101/images/pho/1129758.jpg',\n", " 'food-101/images/pho/1129765.jpg',\n", " 'food-101/images/pho/1139062.jpg',\n", " 'food-101/images/pho/1156352.jpg',\n", " 'food-101/images/pho/1172480.jpg',\n", " 'food-101/images/pho/1181230.jpg',\n", " 'food-101/images/pho/1216057.jpg',\n", " 'food-101/images/pho/1226579.jpg',\n", " 'food-101/images/pho/1242474.jpg',\n", " 'food-101/images/pho/1252904.jpg',\n", " 'food-101/images/pho/1275324.jpg',\n", " 'food-101/images/pho/1276052.jpg',\n", " 'food-101/images/pho/1332972.jpg',\n", " 'food-101/images/pho/134776.jpg',\n", " 'food-101/images/pho/1391686.jpg',\n", " 'food-101/images/pho/139729.jpg',\n", " 'food-101/images/pho/1496796.jpg',\n", " 'food-101/images/pho/150417.jpg',\n", " 'food-101/images/pho/1512455.jpg',\n", " 'food-101/images/pho/1517043.jpg',\n", " 'food-101/images/pho/1521706.jpg',\n", " 'food-101/images/pho/1529866.jpg',\n", " 'food-101/images/pho/1563592.jpg',\n", " 'food-101/images/pho/1572698.jpg',\n", " 'food-101/images/pho/1610808.jpg',\n", " 'food-101/images/pho/1613158.jpg',\n", " 'food-101/images/pho/1662402.jpg',\n", " 'food-101/images/pho/174023.jpg',\n", " 'food-101/images/pho/1830525.jpg',\n", " 'food-101/images/pho/1833761.jpg',\n", " 'food-101/images/pho/1889793.jpg',\n", " 'food-101/images/pho/1904441.jpg',\n", " 'food-101/images/pho/1945425.jpg',\n", " 'food-101/images/pho/1966123.jpg',\n", " 'food-101/images/pho/197516.jpg',\n", " 'food-101/images/pho/200060.jpg',\n", " 'food-101/images/pho/2011541.jpg',\n", " 'food-101/images/pho/2037578.jpg',\n", " 'food-101/images/pho/2051133.jpg',\n", " 'food-101/images/pho/2055313.jpg',\n", " 'food-101/images/pho/2081921.jpg',\n", " 'food-101/images/pho/21151.jpg',\n", " 'food-101/images/pho/212649.jpg',\n", " 'food-101/images/pho/2146049.jpg',\n", " 'food-101/images/pho/2209230.jpg',\n", " 'food-101/images/pho/2231809.jpg',\n", " 'food-101/images/pho/223558.jpg',\n", " 'food-101/images/pho/2279738.jpg',\n", " 'food-101/images/pho/2291683.jpg',\n", " 'food-101/images/pho/2310154.jpg',\n", " 'food-101/images/pho/2314570.jpg',\n", " 'food-101/images/pho/2338556.jpg',\n", " 'food-101/images/pho/2381828.jpg',\n", " 'food-101/images/pho/2437694.jpg',\n", " 'food-101/images/pho/2488737.jpg',\n", " 'food-101/images/pho/2520742.jpg',\n", " 'food-101/images/pho/2555529.jpg',\n", " 'food-101/images/pho/255974.jpg',\n", " 'food-101/images/pho/256709.jpg',\n", " 'food-101/images/pho/257397.jpg',\n", " 'food-101/images/pho/260112.jpg',\n", " 'food-101/images/pho/264730.jpg',\n", " 'food-101/images/pho/2647478.jpg',\n", " 'food-101/images/pho/2667172.jpg',\n", " 'food-101/images/pho/2749545.jpg',\n", " 'food-101/images/pho/2756142.jpg',\n", " 'food-101/images/pho/2780074.jpg',\n", " 'food-101/images/pho/2790322.jpg',\n", " 'food-101/images/pho/2816303.jpg',\n", " 'food-101/images/pho/2832528.jpg',\n", " 'food-101/images/pho/2841009.jpg',\n", " 'food-101/images/pho/286649.jpg',\n", " 'food-101/images/pho/2870902.jpg',\n", " 'food-101/images/pho/2872388.jpg',\n", " 'food-101/images/pho/2903413.jpg',\n", " 'food-101/images/pho/292752.jpg',\n", " 'food-101/images/pho/2931231.jpg',\n", " 'food-101/images/pho/2932374.jpg',\n", " 'food-101/images/pho/294020.jpg',\n", " 'food-101/images/pho/2952717.jpg',\n", " 'food-101/images/pho/2959196.jpg',\n", " 'food-101/images/pho/2971145.jpg',\n", " 'food-101/images/pho/2981476.jpg',\n", " 'food-101/images/pho/3021661.jpg',\n", " 'food-101/images/pho/3070544.jpg',\n", " 'food-101/images/pho/3080329.jpg',\n", " 'food-101/images/pho/3100035.jpg',\n", " 'food-101/images/pho/3138587.jpg',\n", " 'food-101/images/pho/3157756.jpg',\n", " 'food-101/images/pho/3201152.jpg',\n", " 'food-101/images/pho/3230074.jpg',\n", " 'food-101/images/pho/3230863.jpg',\n", " 'food-101/images/pho/3293078.jpg',\n", " 'food-101/images/pho/3293740.jpg',\n", " 'food-101/images/pho/3299599.jpg',\n", " 'food-101/images/pho/331738.jpg',\n", " 'food-101/images/pho/333256.jpg',\n", " 'food-101/images/pho/3341886.jpg',\n", " 'food-101/images/pho/336954.jpg',\n", " 'food-101/images/pho/341163.jpg',\n", " 'food-101/images/pho/3431393.jpg',\n", " 'food-101/images/pho/3433092.jpg',\n", " 'food-101/images/pho/344153.jpg',\n", " 'food-101/images/pho/3457544.jpg',\n", " 'food-101/images/pho/346294.jpg',\n", " 'food-101/images/pho/3534971.jpg',\n", " 'food-101/images/pho/3538985.jpg',\n", " 'food-101/images/pho/3552809.jpg',\n", " 'food-101/images/pho/357388.jpg',\n", " 'food-101/images/pho/3625863.jpg',\n", " 'food-101/images/pho/370292.jpg',\n", " 'food-101/images/pho/3706201.jpg',\n", " 'food-101/images/pho/3711445.jpg',\n", " 'food-101/images/pho/372921.jpg',\n", " 'food-101/images/pho/3746643.jpg',\n", " 'food-101/images/pho/3769076.jpg',\n", " 'food-101/images/pho/3775540.jpg',\n", " 'food-101/images/pho/3778824.jpg',\n", " 'food-101/images/pho/3815050.jpg',\n", " 'food-101/images/pho/3840244.jpg',\n", " 'food-101/images/pho/3866730.jpg',\n", " 'food-101/images/pho/3883511.jpg',\n", " 'food-101/images/pho/3888773.jpg',\n", " 'food-101/images/pho/3919470.jpg',\n", " 'food-101/images/pho/398590.jpg',\n", " 'food-101/images/pho/399346.jpg',\n", " 'food-101/images/pho/441414.jpg',\n", " 'food-101/images/pho/452855.jpg',\n", " 'food-101/images/pho/457009.jpg',\n", " 'food-101/images/pho/458911.jpg',\n", " 'food-101/images/pho/466678.jpg',\n", " 'food-101/images/pho/472628.jpg',\n", " 'food-101/images/pho/4790.jpg',\n", " 'food-101/images/pho/485017.jpg',\n", " 'food-101/images/pho/485593.jpg',\n", " 'food-101/images/pho/51138.jpg',\n", " 'food-101/images/pho/51198.jpg',\n", " 'food-101/images/pho/514715.jpg',\n", " 'food-101/images/pho/514754.jpg',\n", " 'food-101/images/pho/522453.jpg',\n", " 'food-101/images/pho/536765.jpg',\n", " 'food-101/images/pho/558069.jpg',\n", " 'food-101/images/pho/572785.jpg',\n", " 'food-101/images/pho/596919.jpg',\n", " 'food-101/images/pho/603678.jpg',\n", " 'food-101/images/pho/62482.jpg',\n", " 'food-101/images/pho/651415.jpg',\n", " 'food-101/images/pho/672924.jpg',\n", " 'food-101/images/pho/673462.jpg',\n", " 'food-101/images/pho/70638.jpg',\n", " 'food-101/images/pho/718975.jpg',\n", " 'food-101/images/pho/721739.jpg',\n", " 'food-101/images/pho/730353.jpg',\n", " 'food-101/images/pho/73216.jpg',\n", " 'food-101/images/pho/746268.jpg',\n", " 'food-101/images/pho/819607.jpg',\n", " 'food-101/images/pho/846615.jpg',\n", " 'food-101/images/pho/873123.jpg',\n", " 'food-101/images/pho/881249.jpg',\n", " 'food-101/images/pho/882063.jpg',\n", " 'food-101/images/pho/914961.jpg',\n", " 'food-101/images/pho/92244.jpg',\n", " 'food-101/images/pho/931906.jpg',\n", " 'food-101/images/pho/945282.jpg',\n", " 'food-101/images/pho/96270.jpg',\n", " 'food-101/images/pho/963353.jpg',\n", " 'food-101/images/pho/96525.jpg',\n", " 'food-101/images/pork_chop/3725022.jpg',\n", " 'food-101/images/ramen/117019.jpg',\n", " 'food-101/images/ramen/1216361.jpg',\n", " 'food-101/images/ramen/1568197.jpg',\n", " 'food-101/images/ramen/1652565.jpg',\n", " 'food-101/images/ramen/23747.jpg',\n", " 'food-101/images/ramen/254816.jpg',\n", " 'food-101/images/ramen/3183909.jpg',\n", " 'food-101/images/ramen/3216527.jpg',\n", " 'food-101/images/ramen/333940.jpg',\n", " 'food-101/images/ramen/3518484.jpg',\n", " 'food-101/images/ramen/3882171.jpg',\n", " 'food-101/images/ramen/5025.jpg',\n", " 'food-101/images/shrimp_and_grits/1556701.jpg',\n", " 'food-101/images/shrimp_and_grits/2406502.jpg',\n", " 'food-101/images/shrimp_and_grits/537365.jpg',\n", " 'food-101/images/shrimp_and_grits/889512.jpg',\n", " 'food-101/images/shrimp_and_grits/932399.jpg',\n", " 'food-101/images/club_sandwich/122236.jpg',\n", " 'food-101/images/club_sandwich/1243806.jpg',\n", " 'food-101/images/club_sandwich/1297247.jpg',\n", " 'food-101/images/club_sandwich/1318118.jpg',\n", " 'food-101/images/club_sandwich/1413794.jpg',\n", " 'food-101/images/club_sandwich/1719340.jpg',\n", " 'food-101/images/club_sandwich/1775789.jpg',\n", " 'food-101/images/club_sandwich/1811271.jpg',\n", " 'food-101/images/club_sandwich/1818664.jpg',\n", " 'food-101/images/club_sandwich/1840706.jpg',\n", " 'food-101/images/club_sandwich/1886101.jpg',\n", " 'food-101/images/club_sandwich/191288.jpg',\n", " 'food-101/images/club_sandwich/2163422.jpg',\n", " 'food-101/images/club_sandwich/2214212.jpg',\n", " 'food-101/images/club_sandwich/2272423.jpg',\n", " 'food-101/images/club_sandwich/2778614.jpg',\n", " 'food-101/images/club_sandwich/3106065.jpg',\n", " 'food-101/images/club_sandwich/3132951.jpg',\n", " 'food-101/images/club_sandwich/3251770.jpg',\n", " 'food-101/images/club_sandwich/3310968.jpg',\n", " 'food-101/images/club_sandwich/3526250.jpg',\n", " 'food-101/images/club_sandwich/3543955.jpg',\n", " 'food-101/images/club_sandwich/3550782.jpg',\n", " 'food-101/images/club_sandwich/3646665.jpg',\n", " 'food-101/images/club_sandwich/3664710.jpg',\n", " 'food-101/images/club_sandwich/574816.jpg',\n", " 'food-101/images/club_sandwich/588478.jpg',\n", " 'food-101/images/crab_cakes/3467918.jpg',\n", " 'food-101/images/fish_and_chips/1195292.jpg',\n", " 'food-101/images/fish_and_chips/2484873.jpg',\n", " 'food-101/images/fish_and_chips/3616236.jpg',\n", " 'food-101/images/french_fries/1049122.jpg',\n", " 'food-101/images/french_fries/1071481.jpg',\n", " 'food-101/images/french_fries/1074446.jpg',\n", " 'food-101/images/french_fries/1080215.jpg',\n", " 'food-101/images/french_fries/109401.jpg',\n", " 'food-101/images/french_fries/1099260.jpg',\n", " 'food-101/images/french_fries/1150103.jpg',\n", " 'food-101/images/french_fries/1165966.jpg',\n", " 'food-101/images/french_fries/1221000.jpg',\n", " 'food-101/images/french_fries/1295274.jpg',\n", " 'food-101/images/french_fries/1327248.jpg',\n", " 'food-101/images/french_fries/1361604.jpg',\n", " 'food-101/images/french_fries/1369116.jpg',\n", " 'food-101/images/french_fries/1370900.jpg',\n", " 'food-101/images/french_fries/1384733.jpg',\n", " 'food-101/images/french_fries/1394977.jpg',\n", " 'food-101/images/french_fries/147628.jpg',\n", " 'food-101/images/french_fries/1504529.jpg',\n", " 'food-101/images/french_fries/1509620.jpg',\n", " 'food-101/images/french_fries/1538500.jpg',\n", " 'food-101/images/french_fries/1552191.jpg',\n", " 'food-101/images/french_fries/1610240.jpg',\n", " 'food-101/images/french_fries/1622796.jpg',\n", " 'food-101/images/french_fries/166821.jpg',\n", " 'food-101/images/french_fries/168924.jpg',\n", " 'food-101/images/french_fries/1692353.jpg',\n", " 'food-101/images/french_fries/1700344.jpg',\n", " 'food-101/images/french_fries/1712331.jpg',\n", " 'food-101/images/french_fries/1724805.jpg',\n", " 'food-101/images/french_fries/1740113.jpg',\n", " 'food-101/images/french_fries/1745333.jpg',\n", " 'food-101/images/french_fries/1763119.jpg',\n", " 'food-101/images/french_fries/1810352.jpg',\n", " 'food-101/images/french_fries/184954.jpg',\n", " 'food-101/images/french_fries/1850310.jpg',\n", " 'food-101/images/french_fries/1875469.jpg',\n", " 'food-101/images/french_fries/1897333.jpg',\n", " 'food-101/images/french_fries/1928660.jpg',\n", " 'food-101/images/french_fries/1936184.jpg',\n", " 'food-101/images/french_fries/1969264.jpg',\n", " 'food-101/images/french_fries/2001959.jpg',\n", " 'food-101/images/french_fries/2011974.jpg',\n", " 'food-101/images/french_fries/2041482.jpg',\n", " 'food-101/images/french_fries/2056007.jpg',\n", " 'food-101/images/french_fries/2073415.jpg',\n", " 'food-101/images/french_fries/2080959.jpg',\n", " 'food-101/images/french_fries/2127530.jpg',\n", " 'food-101/images/french_fries/2189231.jpg',\n", " 'food-101/images/french_fries/2212633.jpg',\n", " 'food-101/images/french_fries/2226284.jpg',\n", " 'food-101/images/french_fries/2239352.jpg',\n", " 'food-101/images/french_fries/2246387.jpg',\n", " 'food-101/images/french_fries/2248373.jpg',\n", " 'food-101/images/french_fries/2269039.jpg',\n", " 'food-101/images/french_fries/2278380.jpg',\n", " 'food-101/images/french_fries/2348229.jpg',\n", " 'food-101/images/french_fries/2369999.jpg',\n", " 'food-101/images/french_fries/2371960.jpg',\n", " 'food-101/images/french_fries/2452990.jpg',\n", " 'food-101/images/french_fries/2485920.jpg',\n", " 'food-101/images/french_fries/2493036.jpg',\n", " 'food-101/images/french_fries/2521012.jpg',\n", " 'food-101/images/french_fries/2521724.jpg',\n", " 'food-101/images/french_fries/2624567.jpg',\n", " 'food-101/images/french_fries/2649729.jpg',\n", " 'food-101/images/french_fries/2672237.jpg',\n", " 'food-101/images/french_fries/2700217.jpg',\n", " 'food-101/images/french_fries/2708944.jpg',\n", " 'food-101/images/french_fries/275673.jpg',\n", " 'food-101/images/french_fries/2761796.jpg',\n", " 'food-101/images/french_fries/2788461.jpg',\n", " 'food-101/images/french_fries/2814658.jpg',\n", " 'food-101/images/french_fries/2816416.jpg',\n", " 'food-101/images/french_fries/2879174.jpg',\n", " 'food-101/images/french_fries/2885926.jpg',\n", " 'food-101/images/french_fries/2892637.jpg',\n", " 'food-101/images/french_fries/2893262.jpg',\n", " 'food-101/images/french_fries/2901987.jpg',\n", " 'food-101/images/french_fries/290229.jpg',\n", " 'food-101/images/french_fries/2913911.jpg',\n", " 'food-101/images/french_fries/2936284.jpg',\n", " 'food-101/images/french_fries/2951206.jpg',\n", " 'food-101/images/french_fries/2984923.jpg',\n", " 'food-101/images/french_fries/3012487.jpg',\n", " 'food-101/images/french_fries/3030853.jpg',\n", " 'food-101/images/french_fries/303809.jpg',\n", " 'food-101/images/french_fries/3061458.jpg',\n", " 'food-101/images/french_fries/3069835.jpg',\n", " 'food-101/images/french_fries/3084187.jpg',\n", " 'food-101/images/french_fries/3156237.jpg',\n", " 'food-101/images/french_fries/3184259.jpg',\n", " 'food-101/images/french_fries/319830.jpg',\n", " 'food-101/images/french_fries/3199467.jpg',\n", " 'food-101/images/french_fries/3225836.jpg',\n", " 'food-101/images/french_fries/3293202.jpg',\n", " 'food-101/images/french_fries/3303248.jpg',\n", " 'food-101/images/french_fries/3307428.jpg',\n", " 'food-101/images/french_fries/3325881.jpg',\n", " 'food-101/images/french_fries/3359887.jpg',\n", " 'food-101/images/french_fries/3377570.jpg',\n", " 'food-101/images/french_fries/3401318.jpg',\n", " 'food-101/images/french_fries/3405511.jpg',\n", " 'food-101/images/french_fries/3423618.jpg',\n", " 'food-101/images/french_fries/3483916.jpg',\n", " 'food-101/images/french_fries/3499831.jpg',\n", " 'food-101/images/french_fries/3499988.jpg',\n", " 'food-101/images/french_fries/3569228.jpg',\n", " 'food-101/images/french_fries/3586758.jpg',\n", " 'food-101/images/french_fries/3620351.jpg',\n", " 'food-101/images/french_fries/3627636.jpg',\n", " 'food-101/images/french_fries/3669402.jpg',\n", " 'food-101/images/french_fries/3673168.jpg',\n", " 'food-101/images/french_fries/3681905.jpg',\n", " 'food-101/images/french_fries/3684873.jpg',\n", " 'food-101/images/french_fries/3697215.jpg',\n", " 'food-101/images/french_fries/370151.jpg',\n", " 'food-101/images/french_fries/3708237.jpg',\n", " 'food-101/images/french_fries/3746805.jpg',\n", " 'food-101/images/french_fries/3764329.jpg',\n", " 'food-101/images/french_fries/3765135.jpg',\n", " 'food-101/images/french_fries/3832957.jpg',\n", " 'food-101/images/french_fries/3857595.jpg',\n", " 'food-101/images/french_fries/3870897.jpg',\n", " 'food-101/images/french_fries/3871541.jpg',\n", " 'food-101/images/french_fries/3894916.jpg',\n", " 'food-101/images/french_fries/3907871.jpg',\n", " 'food-101/images/french_fries/3919772.jpg',\n", " 'food-101/images/french_fries/459162.jpg',\n", " 'food-101/images/french_fries/467106.jpg',\n", " 'food-101/images/french_fries/474317.jpg',\n", " 'food-101/images/french_fries/483574.jpg',\n", " 'food-101/images/french_fries/51678.jpg',\n", " 'food-101/images/french_fries/580652.jpg',\n", " 'food-101/images/french_fries/622359.jpg',\n", " 'food-101/images/french_fries/655989.jpg',\n", " 'food-101/images/french_fries/753350.jpg',\n", " 'food-101/images/french_fries/778600.jpg',\n", " 'food-101/images/french_fries/782182.jpg',\n", " 'food-101/images/french_fries/806895.jpg',\n", " 'food-101/images/french_fries/80865.jpg',\n", " 'food-101/images/french_fries/839871.jpg',\n", " 'food-101/images/french_fries/845350.jpg',\n", " 'food-101/images/french_fries/872441.jpg',\n", " 'food-101/images/french_fries/889641.jpg',\n", " 'food-101/images/french_fries/913020.jpg',\n", " 'food-101/images/french_fries/923140.jpg',\n", " 'food-101/images/french_fries/973682.jpg',\n", " 'food-101/images/french_fries/999957.jpg',\n", " 'food-101/images/grilled_cheese_sandwich/700997.jpg',\n", " 'food-101/images/grilled_salmon/1522348.jpg',\n", " 'food-101/images/grilled_salmon/2342537.jpg',\n", " 'food-101/images/hamburger/3685652.jpg',\n", " 'food-101/images/hamburger/3805565.jpg',\n", " 'food-101/images/hot_dog/2081342.jpg',\n", " 'food-101/images/hummus/1971072.jpg',\n", " 'food-101/images/lobster_roll_sandwich/1898958.jpg',\n", " 'food-101/images/lobster_roll_sandwich/1957723.jpg',\n", " 'food-101/images/lobster_roll_sandwich/699569.jpg',\n", " 'food-101/images/onion_rings/1220595.jpg',\n", " 'food-101/images/pad_thai/1709738.jpg',\n", " 'food-101/images/pad_thai/2202979.jpg',\n", " 'food-101/images/pad_thai/2358685.jpg',\n", " 'food-101/images/pad_thai/3059603.jpg',\n", " 'food-101/images/pad_thai/3442087.jpg',\n", " 'food-101/images/pho/314157.jpg',\n", " 'food-101/images/poutine/1023967.jpg',\n", " 'food-101/images/poutine/1114784.jpg',\n", " 'food-101/images/poutine/1406427.jpg',\n", " 'food-101/images/poutine/154691.jpg',\n", " 'food-101/images/poutine/1714430.jpg',\n", " 'food-101/images/poutine/1729075.jpg',\n", " 'food-101/images/poutine/1773777.jpg',\n", " 'food-101/images/poutine/2170579.jpg',\n", " 'food-101/images/poutine/2517939.jpg',\n", " 'food-101/images/poutine/2678875.jpg',\n", " 'food-101/images/poutine/2919368.jpg',\n", " 'food-101/images/poutine/3267969.jpg',\n", " 'food-101/images/poutine/3445288.jpg',\n", " 'food-101/images/poutine/3666236.jpg',\n", " 'food-101/images/poutine/3677778.jpg',\n", " 'food-101/images/poutine/3727579.jpg',\n", " 'food-101/images/poutine/3774878.jpg',\n", " 'food-101/images/poutine/592600.jpg',\n", " 'food-101/images/poutine/793878.jpg',\n", " 'food-101/images/ramen/360647.jpg',\n", " 'food-101/images/scallops/3730608.jpg',\n", " 'food-101/images/spaghetti_bolognese/1071591.jpg',\n", " 'food-101/images/spaghetti_bolognese/1156737.jpg',\n", " 'food-101/images/spaghetti_bolognese/119565.jpg',\n", " 'food-101/images/spaghetti_bolognese/1236785.jpg',\n", " 'food-101/images/spaghetti_bolognese/1394656.jpg',\n", " 'food-101/images/spaghetti_bolognese/1507164.jpg',\n", " 'food-101/images/spaghetti_bolognese/1791456.jpg',\n", " 'food-101/images/spaghetti_bolognese/1800348.jpg',\n", " 'food-101/images/spaghetti_bolognese/1819184.jpg',\n", " 'food-101/images/spaghetti_bolognese/1914631.jpg',\n", " 'food-101/images/spaghetti_bolognese/2000506.jpg',\n", " 'food-101/images/spaghetti_bolognese/2085786.jpg',\n", " 'food-101/images/spaghetti_bolognese/2143708.jpg',\n", " 'food-101/images/spaghetti_bolognese/2172243.jpg',\n", " 'food-101/images/spaghetti_bolognese/219958.jpg',\n", " 'food-101/images/spaghetti_bolognese/2219555.jpg',\n", " 'food-101/images/spaghetti_bolognese/2333818.jpg',\n", " 'food-101/images/spaghetti_bolognese/2344380.jpg',\n", " 'food-101/images/spaghetti_bolognese/2388664.jpg',\n", " 'food-101/images/spaghetti_bolognese/2552612.jpg',\n", " 'food-101/images/spaghetti_bolognese/2558717.jpg',\n", " 'food-101/images/spaghetti_bolognese/2570327.jpg',\n", " 'food-101/images/spaghetti_bolognese/2605155.jpg',\n", " 'food-101/images/spaghetti_bolognese/2737792.jpg',\n", " 'food-101/images/spaghetti_bolognese/2756020.jpg',\n", " 'food-101/images/spaghetti_bolognese/2766811.jpg',\n", " 'food-101/images/spaghetti_bolognese/278181.jpg',\n", " 'food-101/images/spaghetti_bolognese/299220.jpg',\n", " 'food-101/images/spaghetti_bolognese/2995456.jpg',\n", " 'food-101/images/spaghetti_bolognese/3007698.jpg',\n", " ...]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list_of_duplicates" ] }, { "cell_type": "markdown", "id": "911d5559", "metadata": { "tags": [] }, "source": [ "## Outliers\n", "\n", "Visualize a gallery of outliers. Lower `Distance` value indicates higher chances of outliers." ] }, { "cell_type": "code", "execution_count": 15, "id": "392c5fea", "metadata": { "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 23458.08it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Stored outliers visual view in work_dir/galleries/outliers.html\n" ] }, { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Outliers Report\n", " \n", " \n", "\n", "\n", "\n", "
\n", "
\n", "
\n", " \n", " \"logo\"\n", " \n", "
\n", " \n", "
\n", "
\n", "
\n", "

Outliers Report

Showing image outliers, one per row

\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance0.457944
Path/macarons/2117640.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance0.457944
Path/breakfast_burrito/462294.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance0.54816
Path/shrimp_and_grits/1047420.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance0.556056
Path/sushi/3100962.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
Distance0.574433
Path/pho/1840846.jpg
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " \n", "
\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.vis.outliers_gallery(num_images=5)" ] }, { "cell_type": "markdown", "id": "c32a8c22", "metadata": {}, "source": [ "# List of Outliers\n", "Let's first get the outliers `DataFrame`:" ] }, { "cell_type": "code", "execution_count": 17, "id": "6c0976ee", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
outliernearestdistancefilename_outlierindex_xerror_code_outlieris_valid_outlierfd_index_outlierfilename_nearestindex_yerror_code_nearestis_valid_nearestfd_index_nearest
09797961310.379365food-101/images/breakfast_burrito/462294.jpg9797VALIDTrue9797food-101/images/tacos/1505262.jpg96131VALIDTrue96131
163325382900.429240food-101/images/macarons/2117640.jpg63325VALIDTrue63325food-101/images/fish_and_chips/2079080.jpg38290VALIDTrue38290
296131836670.515785food-101/images/tacos/1505262.jpg96131VALIDTrue96131food-101/images/red_velvet_cake/3143813.jpg83667VALIDTrue83667
389017254210.528563food-101/images/shrimp_and_grits/1047420.jpg89017VALIDTrue89017food-101/images/club_sandwich/2465517.jpg25421VALIDTrue25421
495577225750.546918food-101/images/sushi/3100962.jpg95577VALIDTrue95577food-101/images/chocolate_mousse/303176.jpg22575VALIDTrue22575
\n", "
" ], "text/plain": [ " outlier nearest distance filename_outlier index_x error_code_outlier is_valid_outlier fd_index_outlier filename_nearest index_y error_code_nearest is_valid_nearest fd_index_nearest\n", "0 9797 96131 0.379365 food-101/images/breakfast_burrito/462294.jpg 9797 VALID True 9797 food-101/images/tacos/1505262.jpg 96131 VALID True 96131\n", "1 63325 38290 0.429240 food-101/images/macarons/2117640.jpg 63325 VALID True 63325 food-101/images/fish_and_chips/2079080.jpg 38290 VALID True 38290\n", "2 96131 83667 0.515785 food-101/images/tacos/1505262.jpg 96131 VALID True 96131 food-101/images/red_velvet_cake/3143813.jpg 83667 VALID True 83667\n", "3 89017 25421 0.528563 food-101/images/shrimp_and_grits/1047420.jpg 89017 VALID True 89017 food-101/images/club_sandwich/2465517.jpg 25421 VALID True 25421\n", "4 95577 22575 0.546918 food-101/images/sushi/3100962.jpg 95577 VALID True 95577 food-101/images/chocolate_mousse/303176.jpg 22575 VALID True 22575" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "outlier_df = fd.outliers()\n", "outlier_df.head()" ] }, { "cell_type": "markdown", "id": "410cc84e", "metadata": {}, "source": [ "Let's treat all images with `distance<0.68` as outliers." ] }, { "cell_type": "code", "execution_count": 18, "id": "a47d3763", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['food-101/images/breakfast_burrito/462294.jpg',\n", " 'food-101/images/macarons/2117640.jpg',\n", " 'food-101/images/tacos/1505262.jpg',\n", " 'food-101/images/shrimp_and_grits/1047420.jpg',\n", " 'food-101/images/sushi/3100962.jpg',\n", " 'food-101/images/pho/2399877.jpg',\n", " 'food-101/images/pho/1840846.jpg',\n", " 'food-101/images/chocolate_cake/2518457.jpg',\n", " 'food-101/images/red_velvet_cake/2894652.jpg',\n", " 'food-101/images/macarons/3785921.jpg',\n", " 'food-101/images/waffles/720603.jpg',\n", " 'food-101/images/pad_thai/2614597.jpg',\n", " 'food-101/images/prime_rib/587532.jpg',\n", " 'food-101/images/macarons/2591602.jpg',\n", " 'food-101/images/tacos/1091159.jpg',\n", " 'food-101/images/hamburger/1608876.jpg',\n", " 'food-101/images/frozen_yogurt/3577020.jpg',\n", " 'food-101/images/peking_duck/388951.jpg',\n", " 'food-101/images/steak/2788759.jpg',\n", " 'food-101/images/ice_cream/1837798.jpg',\n", " 'food-101/images/grilled_salmon/795787.jpg',\n", " 'food-101/images/macaroni_and_cheese/912672.jpg',\n", " 'food-101/images/miso_soup/881247.jpg',\n", " 'food-101/images/creme_brulee/1661605.jpg',\n", " 'food-101/images/ice_cream/1793992.jpg',\n", " 'food-101/images/onion_rings/2447676.jpg',\n", " 'food-101/images/cup_cakes/1005580.jpg',\n", " 'food-101/images/bread_pudding/1375816.jpg',\n", " 'food-101/images/chicken_curry/2523126.jpg',\n", " 'food-101/images/pho/3642399.jpg',\n", " 'food-101/images/miso_soup/3215987.jpg',\n", " 'food-101/images/takoyaki/914304.jpg',\n", " 'food-101/images/macarons/3537376.jpg',\n", " 'food-101/images/edamame/2979095.jpg',\n", " 'food-101/images/apple_pie/484038.jpg',\n", " 'food-101/images/apple_pie/236966.jpg',\n", " 'food-101/images/macarons/639127.jpg',\n", " 'food-101/images/cup_cakes/451074.jpg',\n", " 'food-101/images/ceviche/2363511.jpg',\n", " 'food-101/images/cheesecake/2160930.jpg',\n", " 'food-101/images/spaghetti_bolognese/2117379.jpg',\n", " 'food-101/images/bruschetta/1883187.jpg',\n", " 'food-101/images/miso_soup/2887493.jpg',\n", " 'food-101/images/cup_cakes/2590269.jpg',\n", " 'food-101/images/bibimbap/3230839.jpg',\n", " 'food-101/images/churros/1440917.jpg',\n", " 'food-101/images/clam_chowder/759125.jpg',\n", " 'food-101/images/tiramisu/1612518.jpg',\n", " 'food-101/images/donuts/794976.jpg',\n", " 'food-101/images/pizza/2693334.jpg',\n", " 'food-101/images/frozen_yogurt/75792.jpg',\n", " 'food-101/images/churros/1972000.jpg',\n", " 'food-101/images/baklava/1553505.jpg',\n", " 'food-101/images/sushi/1073711.jpg',\n", " 'food-101/images/cup_cakes/630654.jpg',\n", " 'food-101/images/chocolate_cake/3600589.jpg',\n", " 'food-101/images/red_velvet_cake/1279265.jpg',\n", " 'food-101/images/chicken_wings/693809.jpg',\n", " 'food-101/images/foie_gras/235773.jpg',\n", " 'food-101/images/hot_dog/3222202.jpg',\n", " 'food-101/images/lobster_bisque/197569.jpg',\n", " 'food-101/images/macarons/2546580.jpg',\n", " 'food-101/images/lasagna/3787908.jpg',\n", " 'food-101/images/grilled_cheese_sandwich/185843.jpg',\n", " 'food-101/images/chocolate_mousse/2177988.jpg',\n", " 'food-101/images/chocolate_cake/1749296.jpg',\n", " 'food-101/images/hummus/1635471.jpg',\n", " 'food-101/images/mussels/3238774.jpg',\n", " 'food-101/images/shrimp_and_grits/894844.jpg',\n", " 'food-101/images/steak/2412263.jpg',\n", " 'food-101/images/dumplings/1339572.jpg',\n", " 'food-101/images/sushi/3499178.jpg',\n", " 'food-101/images/frozen_yogurt/3480831.jpg',\n", " 'food-101/images/prime_rib/125282.jpg',\n", " 'food-101/images/ice_cream/2012607.jpg',\n", " 'food-101/images/cheesecake/811556.jpg',\n", " 'food-101/images/hot_dog/607289.jpg',\n", " 'food-101/images/grilled_cheese_sandwich/235659.jpg',\n", " 'food-101/images/hummus/3065799.jpg',\n", " 'food-101/images/chocolate_cake/1646662.jpg',\n", " 'food-101/images/macarons/1671595.jpg',\n", " 'food-101/images/baby_back_ribs/1676135.jpg',\n", " 'food-101/images/fried_calamari/440673.jpg',\n", " 'food-101/images/churros/2327883.jpg',\n", " 'food-101/images/hot_dog/607287.jpg',\n", " 'food-101/images/apple_pie/2008772.jpg',\n", " 'food-101/images/caprese_salad/2719211.jpg',\n", " 'food-101/images/sashimi/1245652.jpg',\n", " 'food-101/images/ice_cream/2458209.jpg',\n", " 'food-101/images/cannoli/1070382.jpg',\n", " 'food-101/images/chocolate_cake/1244445.jpg',\n", " 'food-101/images/tiramisu/3180815.jpg',\n", " 'food-101/images/donuts/921183.jpg',\n", " 'food-101/images/beignets/726875.jpg',\n", " 'food-101/images/mussels/3139903.jpg',\n", " 'food-101/images/pizza/979998.jpg',\n", " 'food-101/images/chocolate_mousse/2287892.jpg',\n", " 'food-101/images/churros/1944265.jpg',\n", " 'food-101/images/ice_cream/3748853.jpg',\n", " 'food-101/images/macarons/2742302.jpg',\n", " 'food-101/images/macarons/2552948.jpg',\n", " 'food-101/images/macarons/3056648.jpg',\n", " 'food-101/images/ice_cream/2170604.jpg',\n", " 'food-101/images/macarons/3531807.jpg',\n", " 'food-101/images/ice_cream/583540.jpg',\n", " 'food-101/images/takoyaki/3612623.jpg',\n", " 'food-101/images/lasagna/1142842.jpg',\n", " 'food-101/images/churros/679673.jpg',\n", " 'food-101/images/beignets/3506219.jpg',\n", " 'food-101/images/deviled_eggs/2854885.jpg',\n", " 'food-101/images/seaweed_salad/2030169.jpg',\n", " 'food-101/images/french_toast/1040076.jpg',\n", " 'food-101/images/tacos/3702371.jpg',\n", " 'food-101/images/baklava/3278527.jpg',\n", " 'food-101/images/churros/1971765.jpg',\n", " 'food-101/images/macarons/536991.jpg',\n", " 'food-101/images/donuts/3316195.jpg',\n", " 'food-101/images/gnocchi/3622492.jpg',\n", " 'food-101/images/tiramisu/511137.jpg',\n", " 'food-101/images/tiramisu/746685.jpg',\n", " 'food-101/images/frozen_yogurt/2506976.jpg',\n", " 'food-101/images/hot_dog/3050381.jpg',\n", " 'food-101/images/bread_pudding/2585974.jpg',\n", " 'food-101/images/pizza/1576248.jpg',\n", " 'food-101/images/ravioli/1449254.jpg',\n", " 'food-101/images/cup_cakes/601989.jpg',\n", " 'food-101/images/macarons/2710408.jpg',\n", " 'food-101/images/chocolate_cake/577717.jpg',\n", " 'food-101/images/cheesecake/2465886.jpg',\n", " 'food-101/images/grilled_salmon/3165921.jpg',\n", " 'food-101/images/steak/3460673.jpg',\n", " 'food-101/images/cheesecake/1955517.jpg',\n", " 'food-101/images/beef_tartare/1054197.jpg',\n", " 'food-101/images/macarons/336707.jpg',\n", " 'food-101/images/samosa/2282122.jpg',\n", " 'food-101/images/filet_mignon/2685908.jpg',\n", " 'food-101/images/french_fries/1337938.jpg',\n", " 'food-101/images/fish_and_chips/1566646.jpg',\n", " 'food-101/images/hot_dog/1114633.jpg',\n", " 'food-101/images/peking_duck/288881.jpg',\n", " 'food-101/images/churros/1658982.jpg',\n", " 'food-101/images/filet_mignon/2685899.jpg',\n", " 'food-101/images/fried_rice/816153.jpg',\n", " 'food-101/images/macarons/3720891.jpg',\n", " 'food-101/images/donuts/42188.jpg',\n", " 'food-101/images/chocolate_mousse/2048999.jpg',\n", " 'food-101/images/seaweed_salad/852275.jpg',\n", " 'food-101/images/ice_cream/2278442.jpg',\n", " 'food-101/images/chocolate_cake/2067510.jpg',\n", " 'food-101/images/gnocchi/3038388.jpg',\n", " 'food-101/images/frozen_yogurt/466433.jpg',\n", " 'food-101/images/macarons/3031940.jpg',\n", " 'food-101/images/donuts/1452592.jpg',\n", " 'food-101/images/macarons/2090509.jpg',\n", " 'food-101/images/churros/1995090.jpg',\n", " 'food-101/images/french_fries/1929717.jpg',\n", " 'food-101/images/donuts/456541.jpg',\n", " 'food-101/images/hummus/1741658.jpg',\n", " 'food-101/images/chocolate_cake/2479257.jpg',\n", " 'food-101/images/ice_cream/2473809.jpg',\n", " 'food-101/images/churros/3447996.jpg',\n", " 'food-101/images/caprese_salad/2298180.jpg',\n", " 'food-101/images/macarons/1693277.jpg',\n", " 'food-101/images/churros/3690003.jpg',\n", " 'food-101/images/baby_back_ribs/2083106.jpg',\n", " 'food-101/images/baklava/1452465.jpg',\n", " 'food-101/images/ice_cream/1705429.jpg',\n", " 'food-101/images/ice_cream/693398.jpg',\n", " 'food-101/images/bibimbap/2011447.jpg',\n", " 'food-101/images/chocolate_cake/1060965.jpg',\n", " 'food-101/images/waffles/401374.jpg',\n", " 'food-101/images/creme_brulee/202057.jpg',\n", " 'food-101/images/beignets/3568316.jpg',\n", " 'food-101/images/cup_cakes/3691610.jpg']" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list_of_outliers = outlier_df[outlier_df.distance < 0.68].filename_outlier.tolist()\n", "list_of_outliers" ] }, { "cell_type": "markdown", "id": "abff2b94", "metadata": { "tags": [] }, "source": [ "## Dark, Bright and Blurry Images\n", "\n", "Visualize image with statistical metrics." ] }, { "cell_type": "markdown", "id": "faf7e835", "metadata": {}, "source": [ "Visualize dark images from the dataset in ascending order." ] }, { "cell_type": "code", "execution_count": 19, "id": "b542c80a", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 296.83it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Stored mean visual view in work_dir/galleries/mean.html\n" ] }, { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Dark Image Report\n", " \n", " \n", "\n", "\n", "\n", "
\n", "
\n", "
\n", " \n", " \"logo\"\n", " \n", "
\n", " \n", "
\n", "
\n", "
\n", "

Dark Image Report

Showing example images, sort by ascending order

\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean2.0994
filenamefood-101/images/breakfast_burrito/462294.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean8.9502
filenamefood-101/images/spring_rolls/182658.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean11.9448
filenamefood-101/images/ramen/1222396.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean12.5025
filenamefood-101/images/samosa/987023.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean12.873
filenamefood-101/images/oysters/933713.jpg
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " \n", "
\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.vis.stats_gallery(metric='dark', num_images=5)" ] }, { "cell_type": "markdown", "id": "d5fd2243", "metadata": {}, "source": [ "# List of Dark Images\n", "Get a `DataFrame` of image statistics." ] }, { "cell_type": "code", "execution_count": 20, "id": "b373ce95", "metadata": {}, "outputs": [], "source": [ "stats_df = fd.img_stats()" ] }, { "cell_type": "markdown", "id": "d8189b6d", "metadata": {}, "source": [ "If an image has a mean<13 then we conclude it's a dark image:" ] }, { "cell_type": "code", "execution_count": 21, "id": "c3360b38", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
indeximg_wimg_huniqueblurmeanminmaxstdvfile_sizecontrastfilenameerror_codeis_validfd_index
97979797511512207.31652.09940.030.01.029086931.0food-101/images/breakfast_burrito/462294.jpgVALIDTrue9797
5917159171512512125121.577512.88610.0221.016.3158265561.0food-101/images/lasagna/1675979.jpgVALIDTrue59171
69974699745125128271.193512.87300.092.010.0110249291.0food-101/images/oysters/933713.jpgVALIDTrue69974
8108081080512512179257.040711.94480.0255.011.9857251531.0food-101/images/ramen/1222396.jpgVALIDTrue81080
8599585995512384159253.313012.50250.0201.016.5239313681.0food-101/images/samosa/987023.jpgVALIDTrue85995
922469224651238417072.22008.95020.0255.011.8936181731.0food-101/images/spring_rolls/182658.jpgVALIDTrue92246
\n", "
" ], "text/plain": [ " index img_w img_h unique blur mean min max stdv file_size contrast filename error_code is_valid fd_index\n", "9797 9797 511 512 20 7.3165 2.0994 0.0 30.0 1.0290 8693 1.0 food-101/images/breakfast_burrito/462294.jpg VALID True 9797\n", "59171 59171 512 512 125 121.5775 12.8861 0.0 221.0 16.3158 26556 1.0 food-101/images/lasagna/1675979.jpg VALID True 59171\n", "69974 69974 512 512 82 71.1935 12.8730 0.0 92.0 10.0110 24929 1.0 food-101/images/oysters/933713.jpg VALID True 69974\n", "81080 81080 512 512 179 257.0407 11.9448 0.0 255.0 11.9857 25153 1.0 food-101/images/ramen/1222396.jpg VALID True 81080\n", "85995 85995 512 384 159 253.3130 12.5025 0.0 201.0 16.5239 31368 1.0 food-101/images/samosa/987023.jpg VALID True 85995\n", "92246 92246 512 384 170 72.2200 8.9502 0.0 255.0 11.8936 18173 1.0 food-101/images/spring_rolls/182658.jpg VALID True 92246" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dark_images = stats_df[stats_df['mean'] < 13] \n", "dark_images" ] }, { "cell_type": "markdown", "id": "f364c35f", "metadata": {}, "source": [ "To get a list of the dark images:" ] }, { "cell_type": "code", "execution_count": 22, "id": "38fee209", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['food-101/images/breakfast_burrito/462294.jpg',\n", " 'food-101/images/lasagna/1675979.jpg',\n", " 'food-101/images/oysters/933713.jpg',\n", " 'food-101/images/ramen/1222396.jpg',\n", " 'food-101/images/samosa/987023.jpg',\n", " 'food-101/images/spring_rolls/182658.jpg']" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list_of_dark_images = dark_images['filename'].to_list()\n", "list_of_dark_images" ] }, { "cell_type": "markdown", "id": "ca329771", "metadata": {}, "source": [ "# List of Bright Images" ] }, { "cell_type": "markdown", "id": "39d16740", "metadata": {}, "source": [ "Visualize bright images from the dataset in descending order." ] }, { "cell_type": "code", "execution_count": 23, "id": "de041906", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 353.10it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Stored mean visual view in work_dir/galleries/mean.html\n" ] }, { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Bright Image Report\n", " \n", " \n", "\n", "\n", "\n", "
\n", "
\n", "
\n", " \n", " \"logo\"\n", " \n", "
\n", " \n", "
\n", "
\n", "
\n", "

Bright Image Report

Showing example images, sort by descending order

\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean234.6172
filenamefood-101/images/foie_gras/71445.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean233.1629
filenamefood-101/images/foie_gras/3267247.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean231.8362
filenamefood-101/images/risotto/71446.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean230.4804
filenamefood-101/images/crab_cakes/445057.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
mean227.8502
filenamefood-101/images/beef_carpaccio/3169022.jpg
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " \n", "
\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.vis.stats_gallery(metric='bright', num_images=5)" ] }, { "cell_type": "markdown", "id": "2240707b", "metadata": {}, "source": [ "Let's set that if `mean>220.5` we will conclude it's a bright image. You can set your own mean threshold depending on your data." ] }, { "cell_type": "code", "execution_count": 24, "id": "15432bf2", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
indeximg_wimg_huniqueblurmeanminmaxstdvfile_sizecontrastfilenameerror_codeis_validfd_index
36513651512289171557.1130227.850285.0255.041.1604211880.5food-101/images/beef_carpaccio/3169022.jpgVALIDTrue3651
10338103385125122561419.4502226.02710.0255.060.5383249751.0food-101/images/bruschetta/2275519.jpgVALIDTrue10338
26809268095123842561015.4831230.48040.0255.037.7391216191.0food-101/images/crab_cakes/445057.jpgVALIDTrue26809
39668396683415122561823.7296233.16290.0255.051.8224412861.0food-101/images/foie_gras/3267247.jpgVALIDTrue39668
3974439744512384256453.5893222.35610.0255.052.7906194941.0food-101/images/foie_gras/35694.jpgVALIDTrue39744
\n", "
" ], "text/plain": [ " index img_w img_h unique blur mean min max stdv file_size contrast filename error_code is_valid fd_index\n", "3651 3651 512 289 171 557.1130 227.8502 85.0 255.0 41.1604 21188 0.5 food-101/images/beef_carpaccio/3169022.jpg VALID True 3651\n", "10338 10338 512 512 256 1419.4502 226.0271 0.0 255.0 60.5383 24975 1.0 food-101/images/bruschetta/2275519.jpg VALID True 10338\n", "26809 26809 512 384 256 1015.4831 230.4804 0.0 255.0 37.7391 21619 1.0 food-101/images/crab_cakes/445057.jpg VALID True 26809\n", "39668 39668 341 512 256 1823.7296 233.1629 0.0 255.0 51.8224 41286 1.0 food-101/images/foie_gras/3267247.jpg VALID True 39668\n", "39744 39744 512 384 256 453.5893 222.3561 0.0 255.0 52.7906 19494 1.0 food-101/images/foie_gras/35694.jpg VALID True 39744" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bright_images = stats_df[stats_df['mean'] > 220.5]\n", "bright_images.head()" ] }, { "cell_type": "markdown", "id": "7904e23b", "metadata": {}, "source": [ "Get a list of bright images" ] }, { "cell_type": "code", "execution_count": 25, "id": "69c02774", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['food-101/images/beef_carpaccio/3169022.jpg',\n", " 'food-101/images/bruschetta/2275519.jpg',\n", " 'food-101/images/crab_cakes/445057.jpg',\n", " 'food-101/images/foie_gras/3267247.jpg',\n", " 'food-101/images/foie_gras/35694.jpg',\n", " 'food-101/images/foie_gras/583722.jpg',\n", " 'food-101/images/foie_gras/71445.jpg',\n", " 'food-101/images/foie_gras/71461.jpg',\n", " 'food-101/images/hamburger/1585333.jpg',\n", " 'food-101/images/hamburger/3578279.jpg',\n", " 'food-101/images/hamburger/3783650.jpg',\n", " 'food-101/images/ice_cream/3214424.jpg',\n", " 'food-101/images/risotto/71446.jpg']" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list_of_bright_images = bright_images['filename'].to_list()\n", "list_of_bright_images" ] }, { "cell_type": "markdown", "id": "62410173", "metadata": {}, "source": [ "# List of Blurry Images" ] }, { "cell_type": "markdown", "id": "668007b5", "metadata": {}, "source": [ "Visualize blurry images from the dataset in ascending order." ] }, { "cell_type": "code", "execution_count": 26, "id": "2afb1d2c", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 366.89it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Stored blur visual view in work_dir/galleries/blur.html\n" ] }, { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Blurry Image Report\n", " \n", " \n", "\n", "\n", "\n", "
\n", "
\n", "
\n", " \n", " \"logo\"\n", " \n", "
\n", " \n", "
\n", "
\n", "
\n", "

Blurry Image Report

Showing example images, sort by ascending order

\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
blur7.3165
filenamefood-101/images/breakfast_burrito/462294.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
blur18.8621
filenamefood-101/images/bread_pudding/444890.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
blur31.4164
filenamefood-101/images/clam_chowder/908590.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
blur32.0443
filenamefood-101/images/macarons/2117640.jpg
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", "
Info
blur32.3273
filenamefood-101/images/pizza/2412970.jpg
\n", "
\n", "
\n", "
\n", " \n", "
\n", "
\n", " \n", "
\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fd.vis.stats_gallery(metric='blur', num_images=5)" ] }, { "cell_type": "code", "execution_count": 27, "id": "4535ccef", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
indeximg_wimg_huniqueblurmeanminmaxstdvfile_sizecontrastfilenameerror_codeis_validfd_index
2768276838451218340.321278.012640.0226.022.2391187400.6992food-101/images/baklava/3681797.jpgVALIDTrue2768
6924692438451217646.619559.03380.0176.023.8672190181.0000food-101/images/beignets/726875.jpgVALIDTrue6924
884288423065128018.862116.18800.081.07.392397151.0000food-101/images/bread_pudding/444890.jpgVALIDTrue8842
97979797511512207.31652.09940.030.01.029086931.0000food-101/images/breakfast_burrito/462294.jpgVALIDTrue9797
221862218651238410936.723018.39120.0115.017.1823164281.0000food-101/images/chocolate_mousse/1653769.jpgVALIDTrue22186
\n", "
" ], "text/plain": [ " index img_w img_h unique blur mean min max stdv file_size contrast filename error_code is_valid fd_index\n", "2768 2768 384 512 183 40.3212 78.0126 40.0 226.0 22.2391 18740 0.6992 food-101/images/baklava/3681797.jpg VALID True 2768\n", "6924 6924 384 512 176 46.6195 59.0338 0.0 176.0 23.8672 19018 1.0000 food-101/images/beignets/726875.jpg VALID True 6924\n", "8842 8842 306 512 80 18.8621 16.1880 0.0 81.0 7.3923 9715 1.0000 food-101/images/bread_pudding/444890.jpg VALID True 8842\n", "9797 9797 511 512 20 7.3165 2.0994 0.0 30.0 1.0290 8693 1.0000 food-101/images/breakfast_burrito/462294.jpg VALID True 9797\n", "22186 22186 512 384 109 36.7230 18.3912 0.0 115.0 17.1823 16428 1.0000 food-101/images/chocolate_mousse/1653769.jpg VALID True 22186" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "blurry_images = stats_df[stats_df['blur'] < 50]\n", "blurry_images.head()" ] }, { "cell_type": "markdown", "id": "316edecf", "metadata": {}, "source": [ "Get list of blurry images" ] }, { "cell_type": "code", "execution_count": 28, "id": "34730787", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['food-101/images/baklava/3681797.jpg',\n", " 'food-101/images/beignets/726875.jpg',\n", " 'food-101/images/bread_pudding/444890.jpg',\n", " 'food-101/images/breakfast_burrito/462294.jpg',\n", " 'food-101/images/chocolate_mousse/1653769.jpg',\n", " 'food-101/images/clam_chowder/1730633.jpg',\n", " 'food-101/images/clam_chowder/2250407.jpg',\n", " 'food-101/images/clam_chowder/908590.jpg',\n", " 'food-101/images/dumplings/2174768.jpg',\n", " 'food-101/images/gnocchi/3167362.jpg',\n", " 'food-101/images/gnocchi/574058.jpg',\n", " 'food-101/images/gyoza/2712704.jpg',\n", " 'food-101/images/hot_dog/3050169.jpg',\n", " 'food-101/images/hummus/3707400.jpg',\n", " 'food-101/images/macarons/2117640.jpg',\n", " 'food-101/images/miso_soup/2757717.jpg',\n", " 'food-101/images/miso_soup/3215987.jpg',\n", " 'food-101/images/pizza/2412970.jpg',\n", " 'food-101/images/spring_rolls/406134.jpg',\n", " 'food-101/images/strawberry_shortcake/3363461.jpg',\n", " 'food-101/images/tacos/1505262.jpg']" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list_of_blurry_images = blurry_images['filename'].to_list()\n", "list_of_blurry_images" ] }, { "cell_type": "markdown", "id": "f1386ae9", "metadata": { "tags": [] }, "source": [ "## Summary\n", "\n", "Lets print out a summary of the list of files we got from above." ] }, { "cell_type": "code", "execution_count": 29, "id": "2c1d0992", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Broken: 0\n", "Duplicates: 2893\n", "Outliers: 174\n", "Dark: 6\n", "Bright: 13\n", "Blurry: 21\n", "Total unique images: 3099\n" ] } ], "source": [ "print(f\"Broken: {len(list_of_broken_images)}\")\n", "print(f\"Duplicates: {len(list_of_duplicates)}\")\n", "print(f\"Outliers: {len(list_of_outliers)}\")\n", "print(f\"Dark: {len(list_of_dark_images)}\")\n", "print(f\"Bright: {len(list_of_bright_images)}\")\n", "print(f\"Blurry: {len(list_of_blurry_images)}\")\n", "\n", "problem_images = list_of_duplicates + list_of_broken_images + list_of_outliers + list_of_dark_images + list_of_bright_images + list_of_blurry_images\n", "\n", "print(f\"Total unique images: {len(set(problem_images))}\")" ] }, { "cell_type": "markdown", "id": "46b28a8b", "metadata": {}, "source": [ "## Wrap Up\n", "\n", "That's a wrap! In this notebook we showed how you can run fastdup on a dataset or any folder of images. \n", "\n", "We've seen how to use fastdup to find:\n", "\n", "+ Find various dataset issues with fastdup.\n", "+ Export a list of problematic images for further action.\n", "\n", "For each problem we got a list of file names for further action. Depending on use cases, you might choose to delete the image, relabel them or simply move the image elsewhere.\n", "\n", "Next, feel free to check out other tutorials -\n", "\n", "+ ⚡ [**Quickstart**](https://nbviewer.org/github/visual-layer/fastdup/blob/main/examples/quick-dataset-analysis.ipynb): Learn how to install fastdup, load a dataset and analyze it for potential issues such as duplicates/near-duplicates, broken images, outliers, dark/bright/blurry images, and view visually similar image clusters. If you're new, start here!\n", "+ 🧹 [**Clean Image Folder**](https://nbviewer.org/github/visual-layer/fastdup/blob/main/examples/cleaning-image-dataset.ipynb): Learn how to analyze and clean a folder of images from potential issues and export a list of problematic files for further action. If you have an unorganized folder of images, this is a good place to start.\n", "+ 🖼 [**Analyze Image Classification Dataset**](https://nbviewer.org/github/visual-layer/fastdup/blob/main/examples/analyzing-image-classification-dataset.ipynb): Learn how to load a labeled image classification dataset and analyze for potential issues. If you have labeled ImageNet-style folder structure, have a go!\n", "+ 🎁 [**Analyze Object Detection Dataset**](https://nbviewer.org/github/visual-layer/fastdup/blob/main/examples/analyzing-object-detection-dataset.ipynb): Learn how to load bounding box annotations for object detection and analyze for potential issues. If you have a COCO-style labeled object detection dataset, give this example a try. \n" ] }, { "cell_type": "markdown", "id": "dcf65974", "metadata": {}, "source": [ "\n", "## VL Profiler\n", "If you prefer a no-code platform to inspect and visualize your dataset, [**try our free cloud product VL Profiler**](https://app.visual-layer.com) - VL Profiler is our first no-code commercial product that lets you visualize and inspect your dataset in your browser. \n", "\n", "[Sign up](https://app.visual-layer.com) now, it's free.\n", "\n", "[![image](https://raw.githubusercontent.com/visual-layer/fastdup/main/gallery/vl_profiler_promo.svg)](https://app.visual-layer.com)\n", "\n", "As usual, feedback is welcome! \n", "\n", "Questions? Drop by our [Slack channel](https://visualdatabase.slack.com/join/shared_invite/zt-19jaydbjn-lNDEDkgvSI1QwbTXSY6dlA#/shared-invite/email) or open an issue on [GitHub](https://github.com/visual-layer/fastdup/issues)." ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.11" }, "vscode": { "interpreter": { "hash": "5b6e8fba36db23bc4d54e0302cd75fdd75c29d9edcbab68d6cfc74e7e4b30305" } } }, "nbformat": 4, "nbformat_minor": 5 }