{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "aXpIRSAQ-cw1" }, "source": [ "# Getting started with DeepMatcher\n", "\n", "Note: you can run **[this notebook live in Google Colab](https://colab.research.google.com/github/anhaidgroup/deepmatcher/blob/master/examples/getting_started.ipynb)** and use free GPUs provided by Google.\n", "\n", "This tutorial describes how to effortlessly perform entity matching using deep neural networks. Specifically, we will see how to match pairs of tuples (also called data records or table rows) to determine if they refer to the same real world entity. To do so, we will need labeled examples as input, i.e., tuple pairs which have been annotated as matches or non-matches. This will be used to train our neural network using supervised learning. At the end of this tutorial, you will have a trained neural network as output which you can easily apply to unlabeled tuple pairs to make predictions." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "19SozaN8QJOf" }, "source": [ "As an overview, here are the 4 steps to use `deepmatcher` which we will go through in this tutorial:\n", "\n", "
\n", " | id | \n", "label | \n", "left_Song_Name | \n", "left_Artist_Name | \n", "left_Album_Name | \n", "left_Genre | \n", "left_Price | \n", "left_CopyRight | \n", "left_Time | \n", "left_Released | \n", "right_Song_Name | \n", "right_Artist_Name | \n", "right_Album_Name | \n", "right_Genre | \n", "right_Price | \n", "right_CopyRight | \n", "right_Time | \n", "right_Released | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "448 | \n", "0 | \n", "Baby When the Light ( David Guetta & Fred Rist... | \n", "David Guetta | \n", "Pop Life ( Extended Version ) [ Bonus Version ] | \n", "Dance , Music , Rock , Pop , House , Electroni... | \n", "$ 1.29 | \n", "‰ ãÑ 2007 Gum Records | \n", "6:17 | \n", "18-Sep-07 | \n", "Revolver ( Madonna Vs. David Guetta Feat . Lil... | \n", "David Guetta | \n", "One Love ( Deluxe Version ) | \n", "Dance & Electronic | \n", "$ 1.29 | \n", "( C ) 2014 Swedish House Mafia Holdings Ltd ( ... | \n", "3:18 | \n", "August 21 , 2009 | \n", "
1 | \n", "287 | \n", "1 | \n", "Outversion | \n", "Mark Ronson | \n", "Version | \n", "Pop , Music , R&B / Soul,Soul,Dance,Rock,Jazz,... | \n", "$ 0.99 | \n", "2007 Mark Ronson under exclusive license to SO... | \n", "1:50 | \n", "10-Jul-07 | \n", "Outversion | \n", "Mark Ronson | \n", "Version [ Explicit ] | \n", "Pop | \n", "$ 0.99 | \n", "( c ) 2011 J'adore Records | \n", "1:50 | \n", "July 10 , 2007 | \n", "
2 | \n", "534 | \n", "0 | \n", "Peer Pressure ( feat . Traci Nelson ) | \n", "Snoop Dogg | \n", "Doggumentary | \n", "Hip-Hop/Rap , Music , Rock , Gangsta Rap , Wes... | \n", "$ 1.29 | \n", "‰ ãÑ 2011 Capitol Records , LLC . All rights r... | \n", "4:07 | \n", "29-Mar-11 | \n", "Boom ( ( Feat . T-Pain ) [ Edited ] ) | \n", "Snoop Dogg | \n", "Doggumentary [ Edited ] | \n", "Rap & Hip-Hop , West Coast | \n", "$ 1.29 | \n", "( C ) 2011 Capitol Records , LLC | \n", "3:50 | \n", "March 29 , 2011 | \n", "
3 | \n", "181 | \n", "1 | \n", "Stars Come Out ( Tim Mason Remix ) | \n", "Zedd | \n", "Stars Come Out ( Remixes ) - EP | \n", "Dance , Music , Electronic , House | \n", "$ 1.29 | \n", "2012 Dim Mak Inc. | \n", "5:49 | \n", "20-May-14 | \n", "Stars Come Out ( Dillon Francis Remix ) | \n", "Zedd | \n", "Stars Come Out [ Dillon Francis Remix ] | \n", "Dance & Electronic | \n", "$ 1.29 | \n", "2012 Dim Mak Inc. | \n", "4:08 | \n", "May 20 , 2014 | \n", "
4 | \n", "485 | \n", "0 | \n", "Jump ( feat . Nelly Furtado ) | \n", "Flo Rida | \n", "R.O.O.T.S. ( Deluxe Version ) | \n", "Hip-Hop/Rap , Music | \n", "$ 1.29 | \n", "‰ ãÑ 2009 Atlantic Recording Corporation for t... | \n", "3:28 | \n", "30-Mar-09 | \n", "Yayo [ Feat . Brisco , Billy Blue , Ball Greez... | \n", "Flo Rida | \n", "R.O.O.T.S. ( Route Of Overcoming The Struggle ... | \n", "Rap & Hip-Hop | \n", "$ 1.29 | \n", "( C ) 2012 Motown Records , a Division of UMG ... | \n", "7:53 | \n", "March 30 , 2009 | \n", "
\n", " | id | \n", "label | \n", "left_Song_Name | \n", "left_Artist_Name | \n", "left_Album_Name | \n", "left_Genre | \n", "left_Price | \n", "left_CopyRight | \n", "left_Time | \n", "left_Released | \n", "right_Song_Name | \n", "right_Artist_Name | \n", "right_Album_Name | \n", "right_Genre | \n", "right_Price | \n", "right_CopyRight | \n", "right_Time | \n", "right_Released | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "448 | \n", "0 | \n", "baby when the light ( david guetta & fred rist... | \n", "david guetta | \n", "pop life ( extended version ) [ bonus version ] | \n", "dance , music , rock , pop , house , electroni... | \n", "$ 1.29 | \n", "‰ ãñ 2007 gum records | \n", "6:17 | \n", "18-sep-07 | \n", "revolver ( madonna vs. david guetta feat . lil... | \n", "david guetta | \n", "one love ( deluxe version ) | \n", "dance & electronic | \n", "$ 1.29 | \n", "( c ) 2014 swedish house mafia holdings ltd ( ... | \n", "3:18 | \n", "august 21 , 2009 | \n", "
1 | \n", "287 | \n", "1 | \n", "outversion | \n", "mark ronson | \n", "version | \n", "pop , music , r & b / soul , soul , dance , ro... | \n", "$ 0.99 | \n", "2007 mark ronson under exclusive license to so... | \n", "1:50 | \n", "10-jul-07 | \n", "outversion | \n", "mark ronson | \n", "version [ explicit ] | \n", "pop | \n", "$ 0.99 | \n", "( c ) 2011 j'adore records | \n", "1:50 | \n", "july 10 , 2007 | \n", "
2 | \n", "534 | \n", "0 | \n", "peer pressure ( feat . traci nelson ) | \n", "snoop dogg | \n", "doggumentary | \n", "hip-hop/rap , music , rock , gangsta rap , wes... | \n", "$ 1.29 | \n", "‰ ãñ 2011 capitol records , llc . all rights r... | \n", "4:07 | \n", "29-mar-11 | \n", "boom ( ( feat . t-pain ) [ edited ] ) | \n", "snoop dogg | \n", "doggumentary [ edited ] | \n", "rap & hip-hop , west coast | \n", "$ 1.29 | \n", "( c ) 2011 capitol records , llc | \n", "3:50 | \n", "march 29 , 2011 | \n", "
3 | \n", "181 | \n", "1 | \n", "stars come out ( tim mason remix ) | \n", "zedd | \n", "stars come out ( remixes ) - ep | \n", "dance , music , electronic , house | \n", "$ 1.29 | \n", "2012 dim mak inc . | \n", "5:49 | \n", "20-may-14 | \n", "stars come out ( dillon francis remix ) | \n", "zedd | \n", "stars come out [ dillon francis remix ] | \n", "dance & electronic | \n", "$ 1.29 | \n", "2012 dim mak inc . | \n", "4:08 | \n", "may 20 , 2014 | \n", "
4 | \n", "485 | \n", "0 | \n", "jump ( feat . nelly furtado ) | \n", "flo rida | \n", "r.o.o.t.s . ( deluxe version ) | \n", "hip-hop/rap , music | \n", "$ 1.29 | \n", "‰ ãñ 2009 atlantic recording corporation for t... | \n", "3:28 | \n", "30-mar-09 | \n", "yayo [ feat . brisco , billy blue , ball greez... | \n", "flo rida | \n", "r.o.o.t.s . ( route of overcoming the struggle... | \n", "rap & hip-hop | \n", "$ 1.29 | \n", "( c ) 2012 motown records , a division of umg ... | \n", "7:53 | \n", "march 30 , 2009 | \n", "
\n", " | match_score | \n", "
---|---|
id | \n", "\n", " |
141999 | \n", "0.165365 | \n", "
302034 | \n", "0.349144 | \n", "
126354 | \n", "0.698185 | \n", "
714676 | \n", "0.186895 | \n", "
659997 | \n", "0.770332 | \n", "
\n", " | match_score | \n", "left_Song_Name | \n", "left_Artist_Name | \n", "left_Album_Name | \n", "left_Genre | \n", "left_Price | \n", "left_CopyRight | \n", "left_Time | \n", "left_Released | \n", "right_Song_Name | \n", "right_Artist_Name | \n", "right_Album_Name | \n", "right_Genre | \n", "right_Price | \n", "right_CopyRight | \n", "right_Time | \n", "right_Released | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
id | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
141999 | \n", "0.165365 | \n", "Erase ( Samantha Ronson Remix ) [ feat . Priya... | \n", "The Chainsmokers | \n", "Erase ( Samantha Ronson Remix ) [ feat . Priya... | \n", "Pop , Music | \n", "$ 1.29 | \n", "2012 Interscope Records | \n", "4:09 | \n", "4-Dec-12 | \n", "Auuh Remix ( feat . Pacho , Cirilo & Angel Doze ) | \n", "Javy the Flow | \n", "Auuh Remix ( feat . Pacho , Cirilo & Angel Doze ) | \n", "R&B , Dance & Electronic , Rap & Hip-Hop , Con... | \n", "$ 0.99 | \n", "2015 Javy the Flow | \n", "5:05 | \n", "June 13 , 2015 | \n", "
302034 | \n", "0.349144 | \n", "True Colors ( feat . Nicki Minaj ) | \n", "Wiz Khalifa | \n", "Blacc Hollywood | \n", "Hip-Hop/Rap , Music , Rap , East Coast Rap , H... | \n", "$ 1.29 | \n", "2014 Atlantic Recording Corporation for the Un... | \n", "4:15 | \n", "19-Aug-14 | \n", "House In The Hills ( feat . Curren $ y ) [ Cle... | \n", "Wiz Khalifa | \n", "Blacc Hollywood [ Clean ] | \n", "Rap & Hip-Hop | \n", "$ 0.99 | \n", "2014 Atlantic Recording Corporation for the Un... | \n", "4:52 | \n", "August 19 , 2014 | \n", "
126354 | \n", "0.698185 | \n", "Biggest Man in Los Angeles | \n", "Andy Grammer | \n", "Andy Grammer | \n", "Pop , Music , Rock , Adult Alternative | \n", "$ 1.29 | \n", "‰ ãÑ 2011 S-Curve Records | \n", "3:55 | \n", "14-Jun-11 | \n", "Biggest Man In Los Angeles | \n", "Andy Grammer | \n", "Andy Grammer [ + Video ] [ + Digital Booklet ] | \n", "Pop | \n", "$ 0.99 | \n", "( C ) 2011 S-Curve Records | \n", "3:54 | \n", "June 14 , 2011 | \n", "
714676 | \n", "0.186895 | \n", "Good for You ( feat . A$ AP Rocky ) [ KASBO Re... | \n", "Selena Gomez | \n", "Good for You ( feat . A$ AP Rocky ) [ Remixes ... | \n", "Pop , Music | \n", "$ 1.29 | \n", "2015 Interscope Records | \n", "3:41 | \n", "4-Sep-15 | \n", "Forget Forever ( STå # FAN Remix ) | \n", "Selena Gomez | \n", "For You | \n", "Pop | \n", "$ 1.29 | \n", "( C ) 2014 Hollywood Records , Inc. | \n", "3:46 | \n", "November 24 , 2014 | \n", "
659997 | \n", "0.770332 | \n", "Rewind ( feat . Monty ) | \n", "Fetty Wap | \n", "Fetty Wap | \n", "Hip-Hop/Rap , Music , Rap | \n", "Album Only | \n", "2015 300 Entertainment/RGF Productions | \n", "5:36 | \n", "NaN | \n", "Rewind ( feat . Monty ) [ Clean ] | \n", "Fetty Wap | \n", "Fetty Wap ( Deluxe ) [ Clean ] | \n", "Rap & Hip-Hop | \n", "$ 1.29 | \n", "( C ) 2000 Mute Records Ltd. , a BMG Company ,... | \n", "5:36 | \n", "September 25 , 2015 | \n", "
\n", " | match_score | \n", "label | \n", "left_Song_Name | \n", "left_Artist_Name | \n", "left_Album_Name | \n", "left_Genre | \n", "left_Price | \n", "left_CopyRight | \n", "left_Time | \n", "left_Released | \n", "right_Song_Name | \n", "right_Artist_Name | \n", "right_Album_Name | \n", "right_Genre | \n", "right_Price | \n", "right_CopyRight | \n", "right_Time | \n", "right_Released | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
id | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
299 | \n", "0.210376 | \n", "0 | \n", "Blame ( feat . John Newman ) [ BURNS Remix ] | \n", "Calvin Harris | \n", "Blame ( Remixes ) [ feat . John Newman ] - EP | \n", "Dance , Music | \n", "$ 1.29 | \n", "2014 Sony Music Entertainment UK Limited | \n", "4:17 | \n", "10-Nov-14 | \n", "Pray to God ( R3hab Remix ) | \n", "Calvin Harris feat . HAIM | \n", "Pray to God ( Remixes ) | \n", "Dance & Electronic | \n", "$ 1.29 | \n", "( C ) 2015 Third Pardee Records , LLC under ex... | \n", "4:31 | \n", "September 25 , 2015 | \n", "
48 | \n", "0.991802 | \n", "1 | \n", "Afire Love | \n", "Ed Sheeran | \n", "x | \n", "Singer/Songwriter , Music , Pop , Rock , Conte... | \n", "$ 1.29 | \n", "2014 Asylum Records UK , a Warner Music UK Com... | \n", "5:14 | \n", "20-Jun-14 | \n", "Afire Love | \n", "Ed Sheeran | \n", "x | \n", "Pop | \n", "$ 1.29 | \n", "( C ) 2014 mau5trap Recordings Ltd | \n", "5:14 | \n", "June 20 , 2014 | \n", "
34 | \n", "0.187756 | \n", "0 | \n", "Lifted ( feat . Emeli Sand ' © & Professor Gre... | \n", "Naughty Boy | \n", "Hotel Cabana ( Deluxe Version ) | \n", "Pop , Music , Rock , R&B / Soul , Contemporary... | \n", "$ 1.29 | \n", "2013 Naughty Boy Recordings Ltd under exclusiv... | \n", "4:15 | \n", "6-May-14 | \n", "Never Been The Same [ feat . Thabo ] | \n", "Naughty Boy | \n", "Hotel Cabana ( Deluxe Version ) [ Explicit ] | \n", "Pop | \n", "$ 1.29 | \n", "( C ) 2014 Naughty Boy Recordings Ltd under ex... | \n", "3:29 | \n", "May 6 , 2014 | \n", "
215 | \n", "0.986806 | \n", "1 | \n", "Jack Daniels and Jesus | \n", "Chase Rice | \n", "Ignite the Night | \n", "Country , Music , Urban Cowboy , Contemporary ... | \n", "$ 1.29 | \n", "2014 Dack Janiels Records | \n", "3:55 | \n", "19-Aug-14 | \n", "Jack Daniels and Jesus | \n", "Chase Rice | \n", "Ignite the Night ( Party Edition ) [ Explicit ] | \n", "Country | \n", "$ 0.89 | \n", "( C ) 2014 Dack Janiels Records , under exclus... | \n", "3:55 | \n", "August 19 , 2014 | \n", "
376 | \n", "0.172195 | \n", "0 | \n", "Turn Around ( 5,4,3,2,1 ) [ AK Remix ] | \n", "Flo Rida | \n", "Turn Around ( 5,4,3,2,1 ) [ Remixes ] | \n", "Dance,Music,Hip-Hop / Rap , Dirty South , Rap ... | \n", "$ 1.29 | \n", "2011 Atlantic Recording Corporation for the Un... | \n", "5:09 | \n", "19-Apr-11 | \n", "Good Feeling ( Jaywalker Remix ) | \n", "Flo Rida | \n", "Good Feeling ( Remixes ) | \n", "Dance & Electronic | \n", "$ 1.29 | \n", "( C ) 2009 MPL Tours Inc. under exclusive lice... | \n", "4:51 | \n", "November 15 , 2011 | \n", "