{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Comparison of two LDA models & visualize difference"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## In this notebook, I want to show how you can compare models with itself and with other model and why you need it."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## First, clean up 20 newsgroups dataset. We will use it for fitting LDA."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from string import punctuation\n",
"from nltk import RegexpTokenizer\n",
"from nltk.stem.porter import PorterStemmer\n",
"from nltk.corpus import stopwords\n",
"from sklearn.datasets import fetch_20newsgroups\n",
"\n",
"\n",
"newsgroups = fetch_20newsgroups()\n",
"eng_stopwords = set(stopwords.words('english'))\n",
"\n",
"tokenizer = RegexpTokenizer('\\s+', gaps=True)\n",
"stemmer = PorterStemmer()\n",
"translate_tab = {ord(p): u\" \" for p in punctuation}\n",
"\n",
"def text2tokens(raw_text):\n",
" \"\"\"\n",
" Convert raw test to list of stemmed tokens\n",
" \"\"\"\n",
" clean_text = raw_text.lower().translate(translate_tab)\n",
" tokens = [token.strip() for token in tokenizer.tokenize(clean_text)]\n",
" tokens = [token for token in tokens if token not in eng_stopwords]\n",
" stemmed_tokens = [stemmer.stem(token) for token in tokens]\n",
" \n",
" return [token for token in stemmed_tokens if len(token) > 2] # skip short tokens\n",
"\n",
"dataset = [text2tokens(txt) for txt in newsgroups['data']] # convert a documents to list of tokens"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from gensim.corpora import Dictionary\n",
"dictionary = Dictionary(documents=dataset, prune_at=None)\n",
"dictionary.filter_extremes(no_below=5, no_above=0.3, keep_n=None) # use Dictionary to remove un-relevant tokens\n",
"dictionary.compactify()\n",
"\n",
"d2b_dataset = [dictionary.doc2bow(doc) for doc in dataset] # convert list of tokens to bag of word representation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Second, fit two LDA models."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 2min 31s, sys: 9.94 s, total: 2min 40s\n",
"Wall time: 4min 54s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"from gensim.models import LdaMulticore\n",
"num_topics = 15\n",
"\n",
"lda_fst = LdaMulticore(\n",
" corpus=d2b_dataset, num_topics=num_topics, id2word=dictionary,\n",
" workers=4, eval_every=None, passes=10, batch=True\n",
")\n",
"\n",
"lda_snd = LdaMulticore(\n",
" corpus=d2b_dataset, num_topics=num_topics, id2word=dictionary,\n",
" workers=4, eval_every=None, passes=20, batch=True\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## It's time to cases with visualisation, Yay!"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/vnd.plotly.v1+html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import plotly.offline as py\n",
"import plotly.graph_objs as go\n",
"\n",
"py.init_notebook_mode()\n",
"\n",
"def plot_difference(mdiff, title=\"\", annotation=None):\n",
" \"\"\"\n",
" Helper function for plot difference between models\n",
" \"\"\"\n",
" annotation_html = None\n",
" if annotation is not None:\n",
" annotation_html = [\n",
" [\n",
" \"+++ {}
--- {}\".format(\", \".join(int_tokens), \", \".join(diff_tokens)) \n",
" for (int_tokens, diff_tokens) in row\n",
" ] \n",
" for row in annotation\n",
" ]\n",
" \n",
" data = go.Heatmap(z=mdiff, colorscale='RdBu', text=annotation_html)\n",
" layout = go.Layout(width=950, height=950, title=title, xaxis=dict(title=\"topic\"), yaxis=dict(title=\"topic\"))\n",
" py.iplot(dict(data=[data], layout=layout))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In gensim, you can visualise topic different with matrix and annotation. For this purposes, you can use method `diff` from LdaModel.\n",
"\n",
"This function return matrix with distances mdiff and matrix with annotations annotation. Read the docstring for more detailed info.\n",
"\n",
"In cells mdiff[i][j] we can see a distance between topic_i from the first model and topic_j from the second model.\n",
"\n",
"In cells annotation[i][j] we can see [tokens from intersection, tokens from difference] between topic_i from first model and topic_j from the second model."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"LdaMulticore.diff?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Case 1: How topics in ONE model correlate with each other."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Short description:\n",
"- x-axis - topic;\n",
"- y-axis - topic;\n",
"- almost red cell - strongly decorrelated topics;\n",
"- almost blue cell - strongly correlated topics.\n",
"\n",
"In an ideal world, we would like to see different topics decorrelated between themselves. In this case, our matrix would look like this:\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"linkText": "Export to plot.ly",
"plotlyServerURL": "https://plot.ly",
"showLink": true
},
"data": [
{
"colorscale": "RdBu",
"type": "heatmap",
"uid": "973157fc-4482-42a4-9058-906b13c21939",
"z": [
[
0,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
0,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
0,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
0,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
0,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
0,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
0,
1,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
0,
1,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
0,
1,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
1,
0,
1,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
0,
1,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
0,
1,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
0,
1,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
0,
1
],
[
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
0
]
]
}
],
"layout": {
"height": 950,
"title": "Topic difference (one model) in ideal world",
"width": 950,
"xaxis": {
"title": "topic"
},
"yaxis": {
"title": "topic"
}
}
},
"text/html": [
""
],
"text/vnd.plotly.v1+html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import numpy as np\n",
"\n",
"mdiff = np.ones((num_topics, num_topics))\n",
"np.fill_diagonal(mdiff, 0.)\n",
" \n",
"plot_difference(mdiff, title=\"Topic difference (one model) in ideal world\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Unfortunately, in real life, not everything is so good, and the matrix looks different."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Short description (annotations):\n",
"- +++ make, world, well - words from the intersection of topics;\n",
"- --- money, day, still - words from the symmetric difference of topics."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"linkText": "Export to plot.ly",
"plotlyServerURL": "https://plot.ly",
"showLink": true
},
"data": [
{
"colorscale": "RdBu",
"text": [
[
"+++ said, well, two, american, right, arab, say, palestinian, mani, want
--- ",
"+++ even, could, make, well, time, question, right, also, say, want
--- two, mani, give, thing, sgi, chip, world, need, 000, jewish",
"+++ could, time, make, also, want
--- well, two, right, say, mani, thank, give, speed, simm, chip",
"+++ even, world, could, peopl, make, well, time, right, say, want
--- car, two, give, thing, also, happen, need, 000, jewish, question",
"+++ want, time, think, make
--- car, well, two, c8v, right, say, mani, b8f, okz, give",
"+++ even, world, could, said, kill, well, time, two, peopl, right
--- turkey, went, mani, give, also, serdar, need, number, 000, jewish",
"+++ even, said, make, well, time, two, right, also, say, think
--- hockey, presid, mani, give, stephanopoulo, nhl, score, world, 000, jewish",
"+++ could, make, well, time, two, question, also, want, year
--- current, right, say, mani, give, sale, world, need, 000, jewish",
"+++ make, time, also, want, call, way
--- server, client, well, two, right, say, mani, resourc, thank, display",
"+++ way, world, peopl, make, well, time, two, question, right, also
--- mani, thank, access, give, opinion, great, 000, jewish, dod, much",
"+++ even, way, peopl, could, make, well, time, two, right, also
--- car, give, thing, world, number, 000, jewish, question, amend, polici",
"+++ want, also, time
--- well, two, right, say, mani, thank, give, sale, inform, world",
"+++ said, well, two, say, mani, want, even, make, also, see
--- point, law, right, arab, american, palestinian, live, believ, argument, govern",
"+++ said, well, right, say, mani, want, even, make, also, see
--- atheist, realli, two, point, must, arab, palestinian, live, believ, object",
"+++ could, peopl, make, time, two, right, also, think, govern, year
--- well, presid, say, mani, develop, access, give, inform, chip, world"
],
[
"+++ even, could, make, well, time, question, right, also, say, want
--- two, mani, give, thing, sgi, chip, world, need, 000, jewish",
"+++ realli, well, law, right, say, indiana, want, run, disk, state
--- ",
"+++ want, run, disk, tri, make, also, new, problem, version, chip
--- realli, well, right, say, mac, thank, ide, thing, speed, simm",
"+++ realli, well, right, say, want, state, tri, even, make, thing
--- car, point, law, repli, indiana, mani, run, disk, system, got",
"+++ need, make, time, new, look, distribut, want, good, think, anyon
--- car, well, c8v, right, say, b8f, okz, thing, also, sgi",
"+++ even, could, need, time, well, right, new, say, look, want
--- two, turkey, went, thing, also, serdar, sgi, chip, world, number",
"+++ even, make, well, time, right, new, say, also, look, think
--- hockey, two, presid, stephanopoulo, nhl, thing, sgi, score, chip, need",
"+++ could, need, make, well, time, question, new, also, look, distribut
--- current, two, right, say, thing, sale, sgi, chip, much, softwar",
"+++ need, make, time, bit, also, look, distribut, want, run, problem
--- server, client, well, right, say, resourc, thank, display, thing, sourc",
"+++ well, right, say, want, run, state, tri, make, new, also
--- realli, two, thank, access, opinion, thing, sgi, chip, world, need",
"+++ well, law, right, say, state, even, make, thing, new, also
--- car, realli, two, file, weapon, mani, run, health, crime, sgi",
"+++ need, time, drive, new, also, look, distribut, want, run, system
--- well, right, say, thank, thing, sale, sgi, inform, chip, question",
"+++ well, law, say, want, even, make, thing, also, new, see
--- realli, two, right, mani, run, church, sgi, claim, chip, world",
"+++ realli, well, right, say, want, tri, even, make, thing, also
--- atheist, said, point, law, must, indiana, mani, run, believ, object",
"+++ could, need, make, time, key, law, new, right, also, distribut
--- well, two, presid, say, develop, access, thing, sgi, inform, clipper"
],
[
"+++ could, time, make, also, want
--- well, two, right, say, mani, thank, give, speed, simm, chip",
"+++ want, run, disk, tri, system, make, also, new, problem, version
--- realli, well, law, right, say, repli, indiana, monitor, mac, color",
"+++ repli, monitor, want, mac, run, thank, disk, mous, ide, color
--- ",
"+++ could, need, make, time, anyon, look, repli, distribut, want, problem
--- car, well, right, say, mani, thank, thing, also, speed, simm",
"+++ need, make, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, speed, simm, uchicago, chip",
"+++ could, need, time, new, look, want, work
--- well, two, turkey, right, say, went, thank, also, speed, serdar",
"+++ make, time, also, new, look, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, speed",
"+++ could, need, make, time, new, also, repli, look, distribut, want
--- well, current, two, thank, speed, simm, sale, chip, question, nasa",
"+++ repli, want, color, run, thank, set, tri, make, help, also
--- server, client, motif, applic, monitor, mous, resourc, mac, data, disk",
"+++ make, time, new, also, repli, look, distribut, want, run, thank
--- well, two, right, say, access, opinion, speed, simm, chip, world",
"+++ could, make, time, also, new, problem, scsi, control, system
--- car, well, two, right, say, mani, thank, thing, speed, simm",
"+++ repli, want, mac, run, thank, system, help, new, also, version
--- imag, price, monitor, color, mous, netcom, disk, data, ide, instal",
"+++ could, make, time, new, also, want
--- well, two, say, mani, thank, thing, speed, simm, chip, world",
"+++ could, make, time, also, want, system, problem, tri
--- well, right, say, mani, thank, thing, speed, simm, chip, world",
"+++ could, need, make, time, comput, also, new, distribut, work, system
--- two, presid, right, thank, develop, access, speed, simm, inform, clipper"
],
[
"+++ even, world, could, peopl, make, well, time, right, say, mani
--- car, two, give, thing, also, happen, need, 000, jewish, question",
"+++ realli, well, right, say, want, state, tri, even, make, thing
--- car, point, law, first, repli, indiana, mani, run, disk, got",
"+++ could, need, make, time, anyon, look, repli, distribut, want, problem
--- car, well, right, say, mani, thank, thing, also, speed, simm",
"+++ car, realli, well, point, first, right, say, repli, want, mani
--- ",
"+++ need, car, make, time, look, repli, distribut, want, good, think
--- well, c8v, right, say, mani, b8f, okz, thing, uchicago, happen",
"+++ even, world, could, need, time, well, peopl, right, look, say
--- car, two, turkey, went, mani, thing, serdar, happen, number, dod",
"+++ even, make, well, time, point, right, look, say, good, think
--- car, hockey, two, presid, mani, stephanopoulo, nhl, also, thing, score",
"+++ could, need, gov, well, make, time, point, look, repli, sun
--- car, current, two, right, say, mani, thing, also, sale, happen",
"+++ need, make, time, look, repli, distribut, want, problem, tri, way
--- server, client, car, well, right, say, mani, resourc, thank, display",
"+++ well, right, say, repli, want, state, tri, make, problem, year
--- car, realli, two, point, first, mani, run, thank, net, bnr",
"+++ car, well, right, say, mani, state, even, make, thing, problem
--- realli, two, point, law, repli, weapon, want, health, system, case",
"+++ need, time, look, repli, distribut, want, anyon
--- car, well, right, say, mani, thank, thing, also, sale, inform",
"+++ well, point, say, mani, want, even, make, thing, see, world
--- said, car, realli, two, first, law, right, repli, believ, argument",
"+++ realli, well, point, right, say, mani, want, tri, even, make
--- atheist, said, car, first, must, repli, believ, object, state, got",
"+++ could, need, make, time, peopl, right, distribut, think, year, state
--- car, well, two, presid, say, mani, develop, access, thing, also"
],
[
"+++ want, time, think, make
--- car, well, two, right, c8v, say, mani, b8f, okz, give",
"+++ need, make, time, new, look, distribut, want, good, think, anyon
--- car, well, right, c8v, say, b8f, okz, thing, also, sgi",
"+++ need, make, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, speed, simm, uchicago, chip",
"+++ need, car, make, time, look, repli, distribut, want, good, think
--- well, right, c8v, say, mani, b8f, okz, thing, uchicago, happen",
"+++ 2tm, 1eq, car, c8v, 2di, repli, want, b8f, 1d9, 7ey
--- ",
"+++ need, time, new, look, want, think
--- car, well, two, turkey, right, c8v, say, went, b8f, okz",
"+++ time, make, new, look, good, think
--- car, hockey, well, two, presid, right, c8v, say, b8f, okz",
"+++ need, make, time, new, look, repli, distribut, want, good, engin
--- car, well, current, two, c8v, b8f, okz, also, sale, uchicago",
"+++ need, make, time, look, repli, distribut, want
--- server, client, car, c8v, b8f, resourc, thank, okz, display, also",
"+++ make, time, new, look, repli, usa, want, distribut, good, think
--- car, well, two, right, c8v, say, b8f, thank, okz, access",
"+++ car, time, make, new, good, think
--- well, two, right, c8v, say, mani, b8f, okz, thing, also",
"+++ need, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, sale, uchicago, inform, softwar",
"+++ time, make, new, want, good, think
--- car, well, two, c8v, say, mani, b8f, okz, thing, also",
"+++ time, make, want, good, think
--- car, well, right, c8v, say, mani, b8f, okz, thing, also",
"+++ need, time, make, new, distribut, think
--- car, two, presid, right, c8v, b8f, develop, okz, access, also"
],
[
"+++ even, world, could, said, kill, well, time, two, peopl, right
--- turkey, went, mani, give, also, serdar, need, number, 000, jewish",
"+++ even, could, need, time, well, right, new, say, look, want
--- two, turkey, went, thing, also, serdar, sgi, chip, world, number",
"+++ could, need, time, new, look, want, work
--- well, two, turkey, right, say, went, thank, also, speed, serdar",
"+++ even, world, could, need, time, well, peopl, right, look, say
--- car, two, turkey, went, mani, thing, serdar, happen, number, dod",
"+++ need, time, new, look, want, think
--- car, well, two, turkey, c8v, right, say, went, b8f, okz",
"+++ said, armenia, well, two, turkey, right, say, went, want, live
--- ",
"+++ even, said, time, well, two, right, new, say, look, think
--- hockey, presid, turkey, went, stephanopoulo, nhl, also, serdar, score, world",
"+++ could, need, time, well, two, new, look, want, year, work
--- current, turkey, right, say, went, also, serdar, sale, world, number",
"+++ program, need, time, number, work, look, want, name, file, call
--- server, client, well, two, turkey, right, say, went, resourc, thank",
"+++ world, peopl, time, well, two, right, new, say, look, want
--- turkey, went, thank, access, opinion, also, serdar, need, great, number",
"+++ even, peopl, could, time, well, number, two, right, new, say
--- car, turkey, went, mani, thing, also, serdar, world, need, much",
"+++ program, need, time, new, look, want, file, work
--- well, two, turkey, right, say, went, thank, also, serdar, sale",
"+++ even, world, could, said, time, well, two, new, say, want
--- turkey, right, went, mani, thing, also, serdar, need, number, question",
"+++ even, world, could, said, time, well, right, say, want, think
--- two, turkey, went, mani, thing, also, serdar, need, number, question",
"+++ program, could, need, time, peopl, two, new, right, year, think
--- well, presid, turkey, say, went, develop, access, also, serdar, inform"
],
[
"+++ even, said, make, well, time, two, right, also, say, think
--- hockey, presid, mani, give, stephanopoulo, nhl, score, world, 000, jewish",
"+++ even, make, well, time, right, also, say, new, look, think
--- hockey, two, presid, stephanopoulo, thing, nhl, sgi, score, chip, need",
"+++ make, time, also, look, new, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, speed",
"+++ even, make, well, time, point, right, look, say, good, think
--- car, hockey, two, presid, mani, stephanopoulo, thing, nhl, also, score",
"+++ time, make, new, look, good, think
--- car, hockey, well, two, presid, c8v, right, say, b8f, okz",
"+++ even, said, time, well, two, right, new, say, look, think
--- hockey, presid, turkey, went, stephanopoulo, nhl, also, serdar, score, world",
"+++ game, said, play, hockey, well, two, point, presid, hit, right
--- ",
"+++ make, well, time, two, point, new, also, look, good, run
--- hockey, current, presid, right, say, stephanopoulo, nhl, sale, score, need",
"+++ make, time, also, look, run
--- server, client, hockey, well, two, presid, right, say, resourc, thank",
"+++ make, well, time, two, right, new, say, also, look, good
--- hockey, presid, thank, access, opinion, stephanopoulo, nhl, score, world, great",
"+++ day, even, make, well, time, two, right, new, say, also
--- car, hockey, presid, mani, stephanopoulo, thing, nhl, score, number, season",
"+++ time, also, new, look, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, sale",
"+++ day, even, said, make, well, time, two, point, also, new
--- hockey, presid, right, mani, stephanopoulo, thing, nhl, score, world, question",
"+++ even, said, make, well, time, point, right, also, say, think
--- hockey, two, presid, mani, stephanopoulo, thing, nhl, score, world, question",
"+++ make, time, two, presid, right, new, also, year, think, first
--- hockey, well, say, develop, access, stephanopoulo, nhl, inform, chip, score"
],
[
"+++ could, make, well, time, two, question, also, want, year
--- current, right, say, mani, give, sale, world, need, 000, jewish",
"+++ could, need, make, well, time, question, also, new, look, distribut
--- current, two, right, say, thing, sale, sgi, chip, much, softwar",
"+++ could, need, make, time, also, new, repli, look, distribut, want
--- well, current, two, thank, speed, simm, sale, chip, question, nasa",
"+++ could, need, make, well, gov, time, point, look, repli, sun
--- car, current, two, right, say, mani, thing, also, sale, happen",
"+++ need, make, time, new, look, repli, distribut, want, good, engin
--- car, well, current, two, c8v, b8f, okz, also, sale, uchicago",
"+++ could, need, time, well, two, new, look, want, year, work
--- current, turkey, right, say, went, also, serdar, sale, world, number",
"+++ make, well, time, two, point, new, also, look, good, run
--- hockey, current, presid, right, say, stephanopoulo, nhl, sale, score, need",
"+++ well, imag, two, point, current, wire, repli, want, pitt, run
--- ",
"+++ need, make, time, also, look, repli, includ, want, distribut, run
--- server, client, well, current, two, resourc, thank, display, sourc, sale",
"+++ make, well, time, two, question, new, also, repli, look, distribut
--- current, right, say, thank, access, opinion, sale, world, need, great",
"+++ could, make, well, time, two, first, new, also, good, year
--- car, current, right, say, mani, thing, sale, need, number, question",
"+++ need, time, imag, new, also, repli, look, includ, want, interest
--- well, current, two, thank, inform, question, softwar, address, list, nasa",
"+++ could, make, well, scienc, two, point, question, also, new, time
--- current, say, mani, thing, sale, world, need, much, work, nasa",
"+++ could, make, well, time, point, question, also, want, good, system
--- current, two, right, say, mani, thing, sale, world, need, work",
"+++ could, need, make, time, two, also, new, space, distribut, nasa
--- well, current, presid, right, develop, access, sale, inform, chip, clipper"
],
[
"+++ make, time, also, want, call, way
--- server, client, well, two, right, say, mani, resourc, thank, display",
"+++ need, make, time, bit, also, look, distribut, want, run, problem
--- server, client, well, right, say, resourc, thank, display, thing, sourc",
"+++ repli, want, color, run, thank, set, tri, make, help, also
--- server, client, motif, applic, file, monitor, mac, disk, mous, ide",
"+++ need, make, time, look, repli, distribut, want, problem, tri, way
--- server, client, car, well, right, say, mani, resourc, thank, display",
"+++ need, make, time, look, repli, distribut, want
--- server, client, car, c8v, b8f, resourc, thank, okz, display, also",
"+++ program, need, time, number, work, look, want, name, file, call
--- server, client, well, two, turkey, right, say, went, resourc, thank",
"+++ make, time, also, look, run
--- server, client, hockey, well, two, presid, right, say, resourc, thank",
"+++ need, make, time, also, look, repli, includ, want, distribut, run
--- server, client, well, current, two, resourc, thank, display, sourc, sale",
"+++ server, client, motif, applic, repli, want, color, run, thank, resourc
--- ",
"+++ make, time, also, look, repli, distribut, want, run, thank, problem
--- server, client, well, two, right, say, resourc, display, access, opinion",
"+++ make, time, number, also, problem, file, system, way
--- server, client, car, well, two, right, say, mani, resourc, thank",
"+++ repli, want, run, thank, data, program, help, also, avail, version
--- server, client, imag, price, motif, applic, mac, resourc, netcom, color",
"+++ make, time, also, want, may, way
--- server, client, well, two, say, mani, resourc, thank, display, thing",
"+++ make, time, also, want, system, problem, may, tri, way
--- server, client, well, right, say, mani, resourc, thank, display, thing",
"+++ way, program, need, make, time, also, distribut, may, data, work
--- server, client, two, presid, right, resourc, thank, develop, display, access"
],
[
"+++ world, peopl, make, well, time, two, question, right, also, say
--- mani, thank, give, access, opinion, great, 000, jewish, dod, much",
"+++ well, right, say, want, run, state, tri, make, also, new
--- realli, two, thank, access, opinion, thing, sgi, chip, world, need",
"+++ make, time, also, new, repli, tri, look, want, distribut, run
--- well, two, right, say, access, opinion, speed, simm, chip, world",
"+++ well, right, say, repli, want, state, tri, make, problem, year
--- car, realli, two, mani, run, thank, got, access, put, opinion",
"+++ make, time, new, look, repli, usa, want, distribut, good, think
--- car, well, two, c8v, right, say, b8f, thank, okz, access",
"+++ world, peopl, time, well, two, right, new, say, look, want
--- turkey, went, thank, access, opinion, also, serdar, need, number, great",
"+++ make, well, time, two, right, new, say, also, look, good
--- hockey, presid, thank, access, opinion, stephanopoulo, nhl, score, world, great",
"+++ make, well, time, two, question, new, also, repli, look, distribut
--- current, right, say, thank, access, opinion, sale, world, need, great",
"+++ make, time, also, look, repli, distribut, want, run, thank, problem
--- server, client, well, two, right, say, resourc, display, access, opinion",
"+++ well, two, right, say, repli, want, run, thank, net, state
--- ",
"+++ peopl, make, well, time, two, right, new, say, also, good
--- car, mani, thank, access, opinion, thing, world, number, great, question",
"+++ time, new, also, repli, look, distribut, want, run, thank, pleas
--- well, two, right, say, access, opinion, sale, inform, world, need",
"+++ world, peopl, make, well, time, two, question, also, new, say
--- right, mani, thank, access, opinion, thing, great, dod, much, bmw",
"+++ world, peopl, make, well, time, question, right, also, say, human
--- realli, two, mani, run, object, thank, access, opinion, thing, christ",
"+++ access, peopl, make, time, two, right, new, also, propos, distribut
--- well, presid, say, thank, develop, opinion, inform, chip, world, clipper"
],
[
"+++ even, way, could, peopl, make, well, time, two, right, also
--- car, arab, weapon, health, give, crime, kill, thing, new, control",
"+++ well, law, right, say, state, even, make, thing, also, new
--- car, realli, two, weapon, mani, run, health, crime, sgi, control",
"+++ could, make, time, new, also, problem, scsi, control, system
--- car, well, two, right, say, mani, thank, thing, speed, simm",
"+++ car, well, right, say, mani, state, even, make, thing, problem
--- realli, two, weapon, health, got, put, crime, gov, new, also",
"+++ car, time, make, new, good, think
--- well, two, c8v, right, say, mani, b8f, okz, thing, also",
"+++ even, year, could, peopl, time, well, number, two, right, new
--- car, turkey, went, mani, thing, also, serdar, world, need, much",
"+++ day, even, make, well, time, two, right, new, say, also
--- car, hockey, presid, mani, stephanopoulo, nhl, thing, score, number, season",
"+++ could, make, well, time, two, new, also, good, system, year
--- car, current, right, say, mani, thing, sale, need, number, question",
"+++ make, time, number, also, problem, file, system, way
--- server, client, car, well, two, right, say, mani, resourc, thank",
"+++ peopl, make, well, time, two, right, new, say, also, good
--- car, weapon, mani, run, thank, health, access, crime, opinion, thing",
"+++ car, well, two, first, law, right, file, say, weapon, mani
--- ",
"+++ time, also, new, file, system
--- car, well, two, right, say, mani, thank, thing, sale, inform",
"+++ well, two, law, say, mani, even, make, thing, also, new
--- said, car, point, first, right, weapon, want, believ, argument, health",
"+++ well, right, say, mani, even, make, thing, also, problem, see
--- car, realli, two, weapon, object, health, crime, christ, new, control",
"+++ could, peopl, make, time, two, public, law, right, new, also
--- car, well, presid, say, mani, develop, access, thing, inform, chip"
],
[
"+++ want, also, time
--- well, two, right, say, mani, thank, give, sale, inform, world",
"+++ need, time, drive, also, new, look, distribut, want, run, system
--- well, right, say, thank, thing, sale, sgi, inform, chip, question",
"+++ repli, want, mac, run, thank, help, also, new, version, window
--- imag, price, monitor, netcom, disk, mous, ide, color, instal, set",
"+++ need, time, look, repli, distribut, want, anyon
--- car, well, right, say, mani, thank, thing, also, sale, inform",
"+++ need, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, sale, uchicago, inform, softwar",
"+++ program, need, time, new, look, want, file, work
--- well, two, turkey, right, say, went, thank, also, serdar, sale",
"+++ time, also, new, look, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, sale",
"+++ need, time, imag, new, also, repli, look, includ, want, interest
--- well, current, two, thank, inform, question, softwar, address, list, nasa",
"+++ repli, want, run, thank, data, program, help, also, avail, version
--- server, client, imag, motif, price, applic, color, netcom, resourc, mac",
"+++ time, new, also, repli, look, distribut, want, run, thank, pleas
--- well, two, right, say, access, opinion, sale, inform, world, need",
"+++ time, also, new, file, system
--- car, well, two, right, say, mani, thank, thing, sale, inform",
"+++ imag, price, file, repli, want, mac, run, thank, netcom, data
--- ",
"+++ want, new, also, time
--- well, two, say, mani, thank, thing, sale, inform, world, need",
"+++ want, system, also, time
--- well, right, say, mani, thank, thing, sale, inform, world, need",
"+++ program, need, time, also, new, distribut, data, work, system, inform
--- two, presid, right, thank, develop, access, sale, chip, clipper, softwar"
],
[
"+++ said, well, two, say, mani, want, even, make, also, see
--- point, law, right, arab, palestinian, live, govern, believ, argument, state",
"+++ well, law, say, want, even, make, thing, also, new, see
--- said, realli, two, point, right, indiana, mani, run, believ, disk",
"+++ could, make, time, new, also, want
--- well, two, say, mani, thank, thing, speed, simm, chip, world",
"+++ well, point, say, want, mani, even, make, thing, see, world
--- car, realli, two, right, got, put, gov, also, new, sun",
"+++ time, make, new, want, good, think
--- car, well, two, c8v, say, mani, b8f, okz, thing, also",
"+++ even, world, could, said, time, well, two, new, say, want
--- turkey, right, went, mani, thing, also, serdar, need, number, question",
"+++ day, even, said, make, well, time, two, point, new, also
--- hockey, presid, right, mani, stephanopoulo, nhl, thing, score, world, question",
"+++ could, make, scienc, well, two, point, question, new, also, time
--- current, say, mani, thing, sale, world, need, much, work, nasa",
"+++ make, time, also, want, may, way
--- server, client, well, two, say, mani, resourc, thank, display, thing",
"+++ world, peopl, make, well, time, two, question, new, also, say
--- right, mani, thank, access, opinion, thing, great, dod, much, bmw",
"+++ well, two, law, say, mani, even, make, thing, new, also
--- car, said, point, right, weapon, want, believ, argument, health, state",
"+++ want, also, new, time
--- well, two, say, mani, thank, thing, sale, inform, world, need",
"+++ said, well, two, point, law, say, mani, want, believ, argument
--- ",
"+++ said, well, point, say, mani, want, believ, come, even, make
--- atheist, realli, two, law, right, must, argument, object, tri, scienc",
"+++ way, could, make, time, two, law, also, new, peopl, think
--- well, presid, right, say, mani, develop, access, thing, inform, chip"
],
[
"+++ said, well, right, say, mani, want, even, make, also, see
--- atheist, realli, two, point, must, arab, palestinian, live, govern, believ",
"+++ realli, well, right, say, want, tri, even, make, thing, also
--- atheist, said, point, law, must, indiana, mani, run, believ, disk",
"+++ could, make, time, also, tri, want, problem, system
--- well, right, say, mani, thank, thing, speed, simm, chip, world",
"+++ realli, well, point, right, say, want, mani, tri, even, make
--- atheist, car, said, must, repli, believ, object, state, system, got",
"+++ time, make, want, good, think
--- car, well, c8v, right, say, mani, b8f, okz, thing, also",
"+++ even, world, could, said, time, well, right, say, want, think
--- two, turkey, went, mani, thing, also, serdar, need, number, question",
"+++ even, said, make, well, time, point, right, also, say, think
--- hockey, two, presid, mani, stephanopoulo, nhl, thing, score, world, question",
"+++ could, make, well, time, point, question, also, want, good, system
--- current, two, right, say, mani, thing, sale, world, need, work",
"+++ way, make, time, also, want, system, problem, tri, may
--- server, client, well, right, say, mani, resourc, thank, display, thing",
"+++ way, world, peopl, make, well, time, question, right, also, say
--- two, mani, thank, access, opinion, thing, great, dod, bmw, find",
"+++ well, right, say, mani, even, make, thing, also, problem, see
--- atheist, car, said, realli, two, point, law, must, weapon, want",
"+++ want, system, also, time
--- well, right, say, mani, thank, thing, sale, inform, world, need",
"+++ said, well, point, say, mani, want, believ, come, even, make
--- atheist, realli, two, law, right, must, object, argument, tri, scienc",
"+++ atheist, said, realli, well, point, right, must, say, mani, want
--- ",
"+++ could, make, time, also, right, may, peopl, think, system, way
--- well, two, presid, say, mani, develop, access, thing, inform, chip"
],
[
"+++ could, peopl, make, time, two, right, also, think, govern, year
--- well, presid, say, mani, develop, give, access, inform, chip, world",
"+++ could, need, make, time, key, law, also, right, new, distribut
--- well, two, presid, say, develop, access, thing, sgi, inform, clipper",
"+++ could, need, make, time, comput, new, also, distribut, work, system
--- two, presid, right, thank, develop, access, speed, simm, inform, clipper",
"+++ could, need, make, time, peopl, right, distribut, think, year, state
--- car, well, two, presid, say, mani, develop, access, thing, also",
"+++ need, time, make, new, distribut, think
--- car, two, presid, c8v, right, b8f, develop, okz, access, also",
"+++ program, could, need, time, peopl, two, new, right, year, think
--- well, presid, turkey, say, went, develop, access, also, serdar, inform",
"+++ make, time, two, presid, right, new, also, year, think, first
--- hockey, well, say, develop, access, stephanopoulo, nhl, score, inform, chip",
"+++ could, need, make, time, two, also, new, space, nasa, distribut
--- well, current, presid, right, develop, access, sale, inform, chip, clipper",
"+++ way, program, need, make, time, also, distribut, may, data, work
--- server, client, two, presid, right, resourc, thank, develop, display, access",
"+++ access, peopl, make, time, two, right, new, also, propos, distribut
--- well, presid, say, thank, develop, opinion, inform, chip, world, clipper",
"+++ peopl, could, make, time, two, public, law, right, new, also
--- car, well, presid, say, mani, develop, access, thing, inform, chip",
"+++ program, need, time, also, new, distribut, data, work, system, inform
--- two, presid, right, thank, develop, access, sale, chip, clipper, softwar",
"+++ way, could, make, time, two, law, new, also, peopl, think
--- well, presid, right, say, mani, develop, access, thing, inform, chip",
"+++ could, make, time, also, right, may, peopl, think, system, way
--- well, two, presid, say, mani, develop, access, thing, inform, chip",
"+++ encrypt, commun, two, presid, law, right, govern, privaci, data, develop
--- "
]
],
"type": "heatmap",
"uid": "990d6b8a-163e-4b47-9fcb-74acb393f776",
"z": [
[
0,
0.8353596757852078,
0.9776035834266518,
0.8205588310689567,
0.9889184397163121,
0.8053969901401142,
0.8777207141110296,
0.9298573766658873,
0.9660479855138072,
0.8353596757852078,
0.8053969901401142,
1,
0.773936170212766,
0.773936170212766,
0.8911992263056093
],
[
0.8353596757852078,
0,
0.773936170212766,
0.6879432624113476,
0.91725768321513,
0.8498122653316645,
0.8639287481444828,
0.8353596757852078,
0.8353596757852078,
0.7576084029086992,
0.7898607827685843,
0.8639287481444828,
0.7408619749045281,
0.706047032474804,
0.8205588310689567
],
[
0.9776035834266518,
0.773936170212766,
0,
0.9043748505857041,
0.9298573766658873,
0.9542438801189659,
0.9660479855138072,
0.8639287481444828,
0.773936170212766,
0.8498122653316645,
0.9298573766658873,
0.7576084029086992,
0.9660479855138072,
0.9421831637372804,
0.9043748505857041
],
[
0.8205588310689567,
0.6879432624113476,
0.9043748505857041,
0,
0.9043748505857041,
0.8205588310689567,
0.8205588310689567,
0.8498122653316645,
0.91725768321513,
0.7898607827685843,
0.7576084029086992,
0.9542438801189659,
0.7898607827685843,
0.7236805747444045,
0.8911992263056093
],
[
0.9889184397163121,
0.91725768321513,
0.9298573766658873,
0.9043748505857041,
0,
0.9660479855138072,
0.9660479855138072,
0.91725768321513,
0.9542438801189659,
0.91725768321513,
0.9660479855138072,
0.9421831637372804,
0.9660479855138072,
0.9776035834266518,
0.9660479855138072
],
[
0.8053969901401142,
0.8498122653316645,
0.9542438801189659,
0.8205588310689567,
0.9660479855138072,
0,
0.8498122653316645,
0.9043748505857041,
0.9043748505857041,
0.8639287481444828,
0.8353596757852078,
0.9421831637372804,
0.8353596757852078,
0.8498122653316645,
0.8777207141110296
],
[
0.8777207141110296,
0.8639287481444828,
0.9660479855138072,
0.8205588310689567,
0.9660479855138072,
0.8498122653316645,
0,
0.8911992263056093,
0.9776035834266518,
0.8498122653316645,
0.8353596757852078,
0.9776035834266518,
0.8353596757852078,
0.8777207141110296,
0.91725768321513
],
[
0.9298573766658873,
0.8353596757852078,
0.8639287481444828,
0.8498122653316645,
0.91725768321513,
0.9043748505857041,
0.8911992263056093,
0,
0.8777207141110296,
0.8353596757852078,
0.9043748505857041,
0.8353596757852078,
0.8639287481444828,
0.9043748505857041,
0.8353596757852078
],
[
0.9660479855138072,
0.8353596757852078,
0.773936170212766,
0.91725768321513,
0.9542438801189659,
0.9043748505857041,
0.9776035834266518,
0.8777207141110296,
0,
0.8777207141110296,
0.9421831637372804,
0.7236805747444045,
0.9660479855138072,
0.9298573766658873,
0.8911992263056093
],
[
0.8353596757852078,
0.7576084029086992,
0.8498122653316645,
0.7898607827685843,
0.91725768321513,
0.8639287481444828,
0.8498122653316645,
0.8353596757852078,
0.8777207141110296,
0,
0.8353596757852078,
0.8911992263056093,
0.8498122653316645,
0.8053969901401142,
0.8353596757852078
],
[
0.8053969901401142,
0.7898607827685843,
0.9298573766658873,
0.7576084029086992,
0.9660479855138072,
0.8353596757852078,
0.8353596757852078,
0.9043748505857041,
0.9421831637372804,
0.8353596757852078,
0,
0.9776035834266518,
0.7576084029086992,
0.7898607827685843,
0.8353596757852078
],
[
1,
0.8639287481444828,
0.7576084029086992,
0.9542438801189659,
0.9421831637372804,
0.9421831637372804,
0.9776035834266518,
0.8353596757852078,
0.7236805747444045,
0.8911992263056093,
0.9776035834266518,
0,
0.9889184397163121,
0.9889184397163121,
0.9043748505857041
],
[
0.773936170212766,
0.7408619749045281,
0.9660479855138072,
0.7898607827685843,
0.9660479855138072,
0.8353596757852078,
0.8353596757852078,
0.8639287481444828,
0.9660479855138072,
0.8498122653316645,
0.7576084029086992,
0.9889184397163121,
0,
0.5236583042235631,
0.9043748505857041
],
[
0.773936170212766,
0.706047032474804,
0.9421831637372804,
0.7236805747444045,
0.9776035834266518,
0.8498122653316645,
0.8777207141110296,
0.9043748505857041,
0.9298573766658873,
0.8053969901401142,
0.7898607827685843,
0.9889184397163121,
0.5236583042235631,
0,
0.91725768321513
],
[
0.8911992263056093,
0.8205588310689567,
0.9043748505857041,
0.8911992263056093,
0.9660479855138072,
0.8777207141110296,
0.91725768321513,
0.8353596757852078,
0.8911992263056093,
0.8353596757852078,
0.8353596757852078,
0.9043748505857041,
0.9043748505857041,
0.91725768321513,
0
]
]
}
],
"layout": {
"height": 950,
"title": "Topic difference (one model) [jaccard distance]",
"width": 950,
"xaxis": {
"title": "topic"
},
"yaxis": {
"title": "topic"
}
}
},
"text/html": [
""
],
"text/vnd.plotly.v1+html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"mdiff, annotation = lda_fst.diff(lda_fst, distance='jaccard', num_words=50)\n",
"plot_difference(mdiff, title=\"Topic difference (one model) [jaccard distance]\", annotation=annotation)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you compare a model with itself, you want to see as many red elements as possible (except diagonal). With this picture, you can look at the not very red elements and understand which topics in the model are very similar and why (you can read annotation if you move your pointer to cell).\n",
"\n",
"\n",
"Jaccard is stable and robust distance function, but this function not enough sensitive for some purposes. Let's try to use Hellinger distance now."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"linkText": "Export to plot.ly",
"plotlyServerURL": "https://plot.ly",
"showLink": true
},
"data": [
{
"colorscale": "RdBu",
"text": [
[
"+++ said, well, two, american, right, arab, say, palestinian, mani, want
--- ",
"+++ even, could, make, well, time, question, right, also, say, want
--- two, mani, give, thing, sgi, chip, world, need, 000, jewish",
"+++ could, time, make, also, want
--- well, two, right, say, mani, thank, give, speed, simm, chip",
"+++ even, world, could, peopl, make, well, time, right, say, want
--- car, two, give, thing, also, happen, need, 000, jewish, question",
"+++ want, time, think, make
--- car, well, two, c8v, right, say, mani, b8f, okz, give",
"+++ even, world, could, said, kill, well, time, two, peopl, right
--- turkey, went, mani, give, also, serdar, need, number, 000, jewish",
"+++ even, said, make, well, time, two, right, also, say, think
--- hockey, presid, mani, give, stephanopoulo, nhl, score, world, 000, jewish",
"+++ could, make, well, time, two, question, also, want, year
--- current, right, say, mani, give, sale, world, need, 000, jewish",
"+++ make, time, also, want, call, way
--- server, client, well, two, right, say, mani, resourc, thank, display",
"+++ way, world, peopl, make, well, time, two, question, right, also
--- mani, thank, access, give, opinion, great, 000, jewish, dod, much",
"+++ even, way, peopl, could, make, well, time, two, right, also
--- car, give, thing, world, number, 000, jewish, question, amend, polici",
"+++ want, also, time
--- well, two, right, say, mani, thank, give, sale, inform, world",
"+++ said, well, two, say, mani, want, even, make, also, see
--- point, law, right, arab, american, palestinian, live, believ, argument, govern",
"+++ said, well, right, say, mani, want, even, make, also, see
--- atheist, realli, two, point, must, arab, palestinian, live, believ, object",
"+++ could, peopl, make, time, two, right, also, think, govern, year
--- well, presid, say, mani, develop, access, give, inform, chip, world"
],
[
"+++ even, could, make, well, time, question, right, also, say, want
--- two, mani, give, thing, sgi, chip, world, need, 000, jewish",
"+++ realli, well, law, right, say, indiana, want, run, disk, state
--- ",
"+++ want, run, disk, tri, make, also, new, problem, version, chip
--- realli, well, right, say, mac, thank, ide, thing, speed, simm",
"+++ realli, well, right, say, want, state, tri, even, make, thing
--- car, point, law, repli, indiana, mani, run, disk, system, got",
"+++ need, make, time, new, look, distribut, want, good, think, anyon
--- car, well, c8v, right, say, b8f, okz, thing, also, sgi",
"+++ even, could, need, time, well, right, new, say, look, want
--- two, turkey, went, thing, also, serdar, sgi, chip, world, number",
"+++ even, make, well, time, right, new, say, also, look, think
--- hockey, two, presid, stephanopoulo, nhl, thing, sgi, score, chip, need",
"+++ could, need, make, well, time, question, new, also, look, distribut
--- current, two, right, say, thing, sale, sgi, chip, much, softwar",
"+++ need, make, time, bit, also, look, distribut, want, run, problem
--- server, client, well, right, say, resourc, thank, display, thing, sourc",
"+++ well, right, say, want, run, state, tri, make, new, also
--- realli, two, thank, access, opinion, thing, sgi, chip, world, need",
"+++ well, law, right, say, state, even, make, thing, new, also
--- car, realli, two, file, weapon, mani, run, health, crime, sgi",
"+++ need, time, drive, new, also, look, distribut, want, run, system
--- well, right, say, thank, thing, sale, sgi, inform, chip, question",
"+++ well, law, say, want, even, make, thing, also, new, see
--- realli, two, right, mani, run, church, sgi, claim, chip, world",
"+++ realli, well, right, say, want, tri, even, make, thing, also
--- atheist, said, point, law, must, indiana, mani, run, believ, object",
"+++ could, need, make, time, key, law, new, right, also, distribut
--- well, two, presid, say, develop, access, thing, sgi, inform, clipper"
],
[
"+++ could, time, make, also, want
--- well, two, right, say, mani, thank, give, speed, simm, chip",
"+++ want, run, disk, tri, system, make, also, new, problem, version
--- realli, well, law, right, say, repli, indiana, monitor, mac, color",
"+++ repli, monitor, want, mac, run, thank, disk, mous, ide, color
--- ",
"+++ could, need, make, time, anyon, look, repli, distribut, want, problem
--- car, well, right, say, mani, thank, thing, also, speed, simm",
"+++ need, make, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, speed, simm, uchicago, chip",
"+++ could, need, time, new, look, want, work
--- well, two, turkey, right, say, went, thank, also, speed, serdar",
"+++ make, time, also, new, look, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, speed",
"+++ could, need, make, time, new, also, repli, look, distribut, want
--- well, current, two, thank, speed, simm, sale, chip, question, nasa",
"+++ repli, want, color, run, thank, set, tri, make, help, also
--- server, client, motif, applic, monitor, mous, resourc, mac, data, disk",
"+++ make, time, new, also, repli, look, distribut, want, run, thank
--- well, two, right, say, access, opinion, speed, simm, chip, world",
"+++ could, make, time, also, new, problem, scsi, control, system
--- car, well, two, right, say, mani, thank, thing, speed, simm",
"+++ repli, want, mac, run, thank, system, help, new, also, version
--- imag, price, monitor, color, mous, netcom, disk, data, ide, instal",
"+++ could, make, time, new, also, want
--- well, two, say, mani, thank, thing, speed, simm, chip, world",
"+++ could, make, time, also, want, system, problem, tri
--- well, right, say, mani, thank, thing, speed, simm, chip, world",
"+++ could, need, make, time, comput, also, new, distribut, work, system
--- two, presid, right, thank, develop, access, speed, simm, inform, clipper"
],
[
"+++ even, world, could, peopl, make, well, time, right, say, mani
--- car, two, give, thing, also, happen, need, 000, jewish, question",
"+++ realli, well, right, say, want, state, tri, even, make, thing
--- car, point, law, first, repli, indiana, mani, run, disk, got",
"+++ could, need, make, time, anyon, look, repli, distribut, want, problem
--- car, well, right, say, mani, thank, thing, also, speed, simm",
"+++ car, realli, well, point, first, right, say, repli, want, mani
--- ",
"+++ need, car, make, time, look, repli, distribut, want, good, think
--- well, c8v, right, say, mani, b8f, okz, thing, uchicago, happen",
"+++ even, world, could, need, time, well, peopl, right, look, say
--- car, two, turkey, went, mani, thing, serdar, happen, number, dod",
"+++ even, make, well, time, point, right, look, say, good, think
--- car, hockey, two, presid, mani, stephanopoulo, nhl, also, thing, score",
"+++ could, need, gov, well, make, time, point, look, repli, sun
--- car, current, two, right, say, mani, thing, also, sale, happen",
"+++ need, make, time, look, repli, distribut, want, problem, tri, way
--- server, client, car, well, right, say, mani, resourc, thank, display",
"+++ well, right, say, repli, want, state, tri, make, problem, year
--- car, realli, two, point, first, mani, run, thank, net, bnr",
"+++ car, well, right, say, mani, state, even, make, thing, problem
--- realli, two, point, law, repli, weapon, want, health, system, case",
"+++ need, time, look, repli, distribut, want, anyon
--- car, well, right, say, mani, thank, thing, also, sale, inform",
"+++ well, point, say, mani, want, even, make, thing, see, world
--- said, car, realli, two, first, law, right, repli, believ, argument",
"+++ realli, well, point, right, say, mani, want, tri, even, make
--- atheist, said, car, first, must, repli, believ, object, state, got",
"+++ could, need, make, time, peopl, right, distribut, think, year, state
--- car, well, two, presid, say, mani, develop, access, thing, also"
],
[
"+++ want, time, think, make
--- car, well, two, right, c8v, say, mani, b8f, okz, give",
"+++ need, make, time, new, look, distribut, want, good, think, anyon
--- car, well, right, c8v, say, b8f, okz, thing, also, sgi",
"+++ need, make, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, speed, simm, uchicago, chip",
"+++ need, car, make, time, look, repli, distribut, want, good, think
--- well, right, c8v, say, mani, b8f, okz, thing, uchicago, happen",
"+++ 2tm, 1eq, car, c8v, 2di, repli, want, b8f, 1d9, 7ey
--- ",
"+++ need, time, new, look, want, think
--- car, well, two, turkey, right, c8v, say, went, b8f, okz",
"+++ time, make, new, look, good, think
--- car, hockey, well, two, presid, right, c8v, say, b8f, okz",
"+++ need, make, time, new, look, repli, distribut, want, good, engin
--- car, well, current, two, c8v, b8f, okz, also, sale, uchicago",
"+++ need, make, time, look, repli, distribut, want
--- server, client, car, c8v, b8f, resourc, thank, okz, display, also",
"+++ make, time, new, look, repli, usa, want, distribut, good, think
--- car, well, two, right, c8v, say, b8f, thank, okz, access",
"+++ car, time, make, new, good, think
--- well, two, right, c8v, say, mani, b8f, okz, thing, also",
"+++ need, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, sale, uchicago, inform, softwar",
"+++ time, make, new, want, good, think
--- car, well, two, c8v, say, mani, b8f, okz, thing, also",
"+++ time, make, want, good, think
--- car, well, right, c8v, say, mani, b8f, okz, thing, also",
"+++ need, time, make, new, distribut, think
--- car, two, presid, right, c8v, b8f, develop, okz, access, also"
],
[
"+++ even, world, could, said, kill, well, time, two, peopl, right
--- turkey, went, mani, give, also, serdar, need, number, 000, jewish",
"+++ even, could, need, time, well, right, new, say, look, want
--- two, turkey, went, thing, also, serdar, sgi, chip, world, number",
"+++ could, need, time, new, look, want, work
--- well, two, turkey, right, say, went, thank, also, speed, serdar",
"+++ even, world, could, need, time, well, peopl, right, look, say
--- car, two, turkey, went, mani, thing, serdar, happen, number, dod",
"+++ need, time, new, look, want, think
--- car, well, two, turkey, c8v, right, say, went, b8f, okz",
"+++ said, armenia, well, two, turkey, right, say, went, want, live
--- ",
"+++ even, said, time, well, two, right, new, say, look, think
--- hockey, presid, turkey, went, stephanopoulo, nhl, also, serdar, score, world",
"+++ could, need, time, well, two, new, look, want, year, work
--- current, turkey, right, say, went, also, serdar, sale, world, number",
"+++ program, need, time, number, work, look, want, name, file, call
--- server, client, well, two, turkey, right, say, went, resourc, thank",
"+++ world, peopl, time, well, two, right, new, say, look, want
--- turkey, went, thank, access, opinion, also, serdar, need, great, number",
"+++ even, peopl, could, time, well, number, two, right, new, say
--- car, turkey, went, mani, thing, also, serdar, world, need, much",
"+++ program, need, time, new, look, want, file, work
--- well, two, turkey, right, say, went, thank, also, serdar, sale",
"+++ even, world, could, said, time, well, two, new, say, want
--- turkey, right, went, mani, thing, also, serdar, need, number, question",
"+++ even, world, could, said, time, well, right, say, want, think
--- two, turkey, went, mani, thing, also, serdar, need, number, question",
"+++ program, could, need, time, peopl, two, new, right, year, think
--- well, presid, turkey, say, went, develop, access, also, serdar, inform"
],
[
"+++ even, said, make, well, time, two, right, also, say, think
--- hockey, presid, mani, give, stephanopoulo, nhl, score, world, 000, jewish",
"+++ even, make, well, time, right, also, say, new, look, think
--- hockey, two, presid, stephanopoulo, thing, nhl, sgi, score, chip, need",
"+++ make, time, also, look, new, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, speed",
"+++ even, make, well, time, point, right, look, say, good, think
--- car, hockey, two, presid, mani, stephanopoulo, thing, nhl, also, score",
"+++ time, make, new, look, good, think
--- car, hockey, well, two, presid, c8v, right, say, b8f, okz",
"+++ even, said, time, well, two, right, new, say, look, think
--- hockey, presid, turkey, went, stephanopoulo, nhl, also, serdar, score, world",
"+++ game, said, play, hockey, well, two, point, presid, hit, right
--- ",
"+++ make, well, time, two, point, new, also, look, good, run
--- hockey, current, presid, right, say, stephanopoulo, nhl, sale, score, need",
"+++ make, time, also, look, run
--- server, client, hockey, well, two, presid, right, say, resourc, thank",
"+++ make, well, time, two, right, new, say, also, look, good
--- hockey, presid, thank, access, opinion, stephanopoulo, nhl, score, world, great",
"+++ day, even, make, well, time, two, right, new, say, also
--- car, hockey, presid, mani, stephanopoulo, thing, nhl, score, number, season",
"+++ time, also, new, look, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, sale",
"+++ day, even, said, make, well, time, two, point, also, new
--- hockey, presid, right, mani, stephanopoulo, thing, nhl, score, world, question",
"+++ even, said, make, well, time, point, right, also, say, think
--- hockey, two, presid, mani, stephanopoulo, thing, nhl, score, world, question",
"+++ make, time, two, presid, right, new, also, year, think, first
--- hockey, well, say, develop, access, stephanopoulo, nhl, inform, chip, score"
],
[
"+++ could, make, well, time, two, question, also, want, year
--- current, right, say, mani, give, sale, world, need, 000, jewish",
"+++ could, need, make, well, time, question, also, new, look, distribut
--- current, two, right, say, thing, sale, sgi, chip, much, softwar",
"+++ could, need, make, time, also, new, repli, look, distribut, want
--- well, current, two, thank, speed, simm, sale, chip, question, nasa",
"+++ could, need, make, well, gov, time, point, look, repli, sun
--- car, current, two, right, say, mani, thing, also, sale, happen",
"+++ need, make, time, new, look, repli, distribut, want, good, engin
--- car, well, current, two, c8v, b8f, okz, also, sale, uchicago",
"+++ could, need, time, well, two, new, look, want, year, work
--- current, turkey, right, say, went, also, serdar, sale, world, number",
"+++ make, well, time, two, point, new, also, look, good, run
--- hockey, current, presid, right, say, stephanopoulo, nhl, sale, score, need",
"+++ well, imag, two, point, current, wire, repli, want, pitt, run
--- ",
"+++ need, make, time, also, look, repli, includ, want, distribut, run
--- server, client, well, current, two, resourc, thank, display, sourc, sale",
"+++ make, well, time, two, question, new, also, repli, look, distribut
--- current, right, say, thank, access, opinion, sale, world, need, great",
"+++ could, make, well, time, two, first, new, also, good, year
--- car, current, right, say, mani, thing, sale, need, number, question",
"+++ need, time, imag, new, also, repli, look, includ, want, interest
--- well, current, two, thank, inform, question, softwar, address, list, nasa",
"+++ could, make, well, scienc, two, point, question, also, new, time
--- current, say, mani, thing, sale, world, need, much, work, nasa",
"+++ could, make, well, time, point, question, also, want, good, system
--- current, two, right, say, mani, thing, sale, world, need, work",
"+++ could, need, make, time, two, also, new, space, distribut, nasa
--- well, current, presid, right, develop, access, sale, inform, chip, clipper"
],
[
"+++ make, time, also, want, call, way
--- server, client, well, two, right, say, mani, resourc, thank, display",
"+++ need, make, time, bit, also, look, distribut, want, run, problem
--- server, client, well, right, say, resourc, thank, display, thing, sourc",
"+++ repli, want, color, run, thank, set, tri, make, help, also
--- server, client, motif, applic, file, monitor, mac, disk, mous, ide",
"+++ need, make, time, look, repli, distribut, want, problem, tri, way
--- server, client, car, well, right, say, mani, resourc, thank, display",
"+++ need, make, time, look, repli, distribut, want
--- server, client, car, c8v, b8f, resourc, thank, okz, display, also",
"+++ program, need, time, number, work, look, want, name, file, call
--- server, client, well, two, turkey, right, say, went, resourc, thank",
"+++ make, time, also, look, run
--- server, client, hockey, well, two, presid, right, say, resourc, thank",
"+++ need, make, time, also, look, repli, includ, want, distribut, run
--- server, client, well, current, two, resourc, thank, display, sourc, sale",
"+++ server, client, motif, applic, repli, want, color, run, thank, resourc
--- ",
"+++ make, time, also, look, repli, distribut, want, run, thank, problem
--- server, client, well, two, right, say, resourc, display, access, opinion",
"+++ make, time, number, also, problem, file, system, way
--- server, client, car, well, two, right, say, mani, resourc, thank",
"+++ repli, want, run, thank, data, program, help, also, avail, version
--- server, client, imag, price, motif, applic, mac, resourc, netcom, color",
"+++ make, time, also, want, may, way
--- server, client, well, two, say, mani, resourc, thank, display, thing",
"+++ make, time, also, want, system, problem, may, tri, way
--- server, client, well, right, say, mani, resourc, thank, display, thing",
"+++ way, program, need, make, time, also, distribut, may, data, work
--- server, client, two, presid, right, resourc, thank, develop, display, access"
],
[
"+++ world, peopl, make, well, time, two, question, right, also, say
--- mani, thank, give, access, opinion, great, 000, jewish, dod, much",
"+++ well, right, say, want, run, state, tri, make, also, new
--- realli, two, thank, access, opinion, thing, sgi, chip, world, need",
"+++ make, time, also, new, repli, tri, look, want, distribut, run
--- well, two, right, say, access, opinion, speed, simm, chip, world",
"+++ well, right, say, repli, want, state, tri, make, problem, year
--- car, realli, two, mani, run, thank, got, access, put, opinion",
"+++ make, time, new, look, repli, usa, want, distribut, good, think
--- car, well, two, c8v, right, say, b8f, thank, okz, access",
"+++ world, peopl, time, well, two, right, new, say, look, want
--- turkey, went, thank, access, opinion, also, serdar, need, number, great",
"+++ make, well, time, two, right, new, say, also, look, good
--- hockey, presid, thank, access, opinion, stephanopoulo, nhl, score, world, great",
"+++ make, well, time, two, question, new, also, repli, look, distribut
--- current, right, say, thank, access, opinion, sale, world, need, great",
"+++ make, time, also, look, repli, distribut, want, run, thank, problem
--- server, client, well, two, right, say, resourc, display, access, opinion",
"+++ well, two, right, say, repli, want, run, thank, net, state
--- ",
"+++ peopl, make, well, time, two, right, new, say, also, good
--- car, mani, thank, access, opinion, thing, world, number, great, question",
"+++ time, new, also, repli, look, distribut, want, run, thank, pleas
--- well, two, right, say, access, opinion, sale, inform, world, need",
"+++ world, peopl, make, well, time, two, question, also, new, say
--- right, mani, thank, access, opinion, thing, great, dod, much, bmw",
"+++ world, peopl, make, well, time, question, right, also, say, human
--- realli, two, mani, run, object, thank, access, opinion, thing, christ",
"+++ access, peopl, make, time, two, right, new, also, propos, distribut
--- well, presid, say, thank, develop, opinion, inform, chip, world, clipper"
],
[
"+++ even, way, could, peopl, make, well, time, two, right, also
--- car, arab, weapon, health, give, crime, kill, thing, new, control",
"+++ well, law, right, say, state, even, make, thing, also, new
--- car, realli, two, weapon, mani, run, health, crime, sgi, control",
"+++ could, make, time, new, also, problem, scsi, control, system
--- car, well, two, right, say, mani, thank, thing, speed, simm",
"+++ car, well, right, say, mani, state, even, make, thing, problem
--- realli, two, weapon, health, got, put, crime, gov, new, also",
"+++ car, time, make, new, good, think
--- well, two, c8v, right, say, mani, b8f, okz, thing, also",
"+++ even, year, could, peopl, time, well, number, two, right, new
--- car, turkey, went, mani, thing, also, serdar, world, need, much",
"+++ day, even, make, well, time, two, right, new, say, also
--- car, hockey, presid, mani, stephanopoulo, nhl, thing, score, number, season",
"+++ could, make, well, time, two, new, also, good, system, year
--- car, current, right, say, mani, thing, sale, need, number, question",
"+++ make, time, number, also, problem, file, system, way
--- server, client, car, well, two, right, say, mani, resourc, thank",
"+++ peopl, make, well, time, two, right, new, say, also, good
--- car, weapon, mani, run, thank, health, access, crime, opinion, thing",
"+++ car, well, two, first, law, right, file, say, weapon, mani
--- ",
"+++ time, also, new, file, system
--- car, well, two, right, say, mani, thank, thing, sale, inform",
"+++ well, two, law, say, mani, even, make, thing, also, new
--- said, car, point, first, right, weapon, want, believ, argument, health",
"+++ well, right, say, mani, even, make, thing, also, problem, see
--- car, realli, two, weapon, object, health, crime, christ, new, control",
"+++ could, peopl, make, time, two, public, law, right, new, also
--- car, well, presid, say, mani, develop, access, thing, inform, chip"
],
[
"+++ want, also, time
--- well, two, right, say, mani, thank, give, sale, inform, world",
"+++ need, time, drive, also, new, look, distribut, want, run, system
--- well, right, say, thank, thing, sale, sgi, inform, chip, question",
"+++ repli, want, mac, run, thank, help, also, new, version, window
--- imag, price, monitor, netcom, disk, mous, ide, color, instal, set",
"+++ need, time, look, repli, distribut, want, anyon
--- car, well, right, say, mani, thank, thing, also, sale, inform",
"+++ need, time, new, look, repli, distribut, want, anyon
--- car, c8v, b8f, thank, okz, also, sale, uchicago, inform, softwar",
"+++ program, need, time, new, look, want, file, work
--- well, two, turkey, right, say, went, thank, also, serdar, sale",
"+++ time, also, new, look, run
--- hockey, well, two, presid, right, say, thank, stephanopoulo, nhl, sale",
"+++ need, time, imag, new, also, repli, look, includ, want, interest
--- well, current, two, thank, inform, question, softwar, address, list, nasa",
"+++ repli, want, run, thank, data, program, help, also, avail, version
--- server, client, imag, motif, price, applic, color, netcom, resourc, mac",
"+++ time, new, also, repli, look, distribut, want, run, thank, pleas
--- well, two, right, say, access, opinion, sale, inform, world, need",
"+++ time, also, new, file, system
--- car, well, two, right, say, mani, thank, thing, sale, inform",
"+++ imag, price, file, repli, want, mac, run, thank, netcom, data
--- ",
"+++ want, new, also, time
--- well, two, say, mani, thank, thing, sale, inform, world, need",
"+++ want, system, also, time
--- well, right, say, mani, thank, thing, sale, inform, world, need",
"+++ program, need, time, also, new, distribut, data, work, system, inform
--- two, presid, right, thank, develop, access, sale, chip, clipper, softwar"
],
[
"+++ said, well, two, say, mani, want, even, make, also, see
--- point, law, right, arab, palestinian, live, govern, believ, argument, state",
"+++ well, law, say, want, even, make, thing, also, new, see
--- said, realli, two, point, right, indiana, mani, run, believ, disk",
"+++ could, make, time, new, also, want
--- well, two, say, mani, thank, thing, speed, simm, chip, world",
"+++ well, point, say, want, mani, even, make, thing, see, world
--- car, realli, two, right, got, put, gov, also, new, sun",
"+++ time, make, new, want, good, think
--- car, well, two, c8v, say, mani, b8f, okz, thing, also",
"+++ even, world, could, said, time, well, two, new, say, want
--- turkey, right, went, mani, thing, also, serdar, need, number, question",
"+++ day, even, said, make, well, time, two, point, new, also
--- hockey, presid, right, mani, stephanopoulo, nhl, thing, score, world, question",
"+++ could, make, scienc, well, two, point, question, new, also, time
--- current, say, mani, thing, sale, world, need, much, work, nasa",
"+++ make, time, also, want, may, way
--- server, client, well, two, say, mani, resourc, thank, display, thing",
"+++ world, peopl, make, well, time, two, question, new, also, say
--- right, mani, thank, access, opinion, thing, great, dod, much, bmw",
"+++ well, two, law, say, mani, even, make, thing, new, also
--- car, said, point, right, weapon, want, believ, argument, health, state",
"+++ want, also, new, time
--- well, two, say, mani, thank, thing, sale, inform, world, need",
"+++ said, well, two, point, law, say, mani, want, believ, argument
--- ",
"+++ said, well, point, say, mani, want, believ, come, even, make
--- atheist, realli, two, law, right, must, argument, object, tri, scienc",
"+++ way, could, make, time, two, law, also, new, peopl, think
--- well, presid, right, say, mani, develop, access, thing, inform, chip"
],
[
"+++ said, well, right, say, mani, want, even, make, also, see
--- atheist, realli, two, point, must, arab, palestinian, live, govern, believ",
"+++ realli, well, right, say, want, tri, even, make, thing, also
--- atheist, said, point, law, must, indiana, mani, run, believ, disk",
"+++ could, make, time, also, tri, want, problem, system
--- well, right, say, mani, thank, thing, speed, simm, chip, world",
"+++ realli, well, point, right, say, want, mani, tri, even, make
--- atheist, car, said, must, repli, believ, object, state, system, got",
"+++ time, make, want, good, think
--- car, well, c8v, right, say, mani, b8f, okz, thing, also",
"+++ even, world, could, said, time, well, right, say, want, think
--- two, turkey, went, mani, thing, also, serdar, need, number, question",
"+++ even, said, make, well, time, point, right, also, say, think
--- hockey, two, presid, mani, stephanopoulo, nhl, thing, score, world, question",
"+++ could, make, well, time, point, question, also, want, good, system
--- current, two, right, say, mani, thing, sale, world, need, work",
"+++ way, make, time, also, want, system, problem, tri, may
--- server, client, well, right, say, mani, resourc, thank, display, thing",
"+++ way, world, peopl, make, well, time, question, right, also, say
--- two, mani, thank, access, opinion, thing, great, dod, bmw, find",
"+++ well, right, say, mani, even, make, thing, also, problem, see
--- atheist, car, said, realli, two, point, law, must, weapon, want",
"+++ want, system, also, time
--- well, right, say, mani, thank, thing, sale, inform, world, need",
"+++ said, well, point, say, mani, want, believ, come, even, make
--- atheist, realli, two, law, right, must, object, argument, tri, scienc",
"+++ atheist, said, realli, well, point, right, must, say, mani, want
--- ",
"+++ could, make, time, also, right, may, peopl, think, system, way
--- well, two, presid, say, mani, develop, access, thing, inform, chip"
],
[
"+++ could, peopl, make, time, two, right, also, think, govern, year
--- well, presid, say, mani, develop, give, access, inform, chip, world",
"+++ could, need, make, time, key, law, also, right, new, distribut
--- well, two, presid, say, develop, access, thing, sgi, inform, clipper",
"+++ could, need, make, time, comput, new, also, distribut, work, system
--- two, presid, right, thank, develop, access, speed, simm, inform, clipper",
"+++ could, need, make, time, peopl, right, distribut, think, year, state
--- car, well, two, presid, say, mani, develop, access, thing, also",
"+++ need, time, make, new, distribut, think
--- car, two, presid, c8v, right, b8f, develop, okz, access, also",
"+++ program, could, need, time, peopl, two, new, right, year, think
--- well, presid, turkey, say, went, develop, access, also, serdar, inform",
"+++ make, time, two, presid, right, new, also, year, think, first
--- hockey, well, say, develop, access, stephanopoulo, nhl, score, inform, chip",
"+++ could, need, make, time, two, also, new, space, nasa, distribut
--- well, current, presid, right, develop, access, sale, inform, chip, clipper",
"+++ way, program, need, make, time, also, distribut, may, data, work
--- server, client, two, presid, right, resourc, thank, develop, display, access",
"+++ access, peopl, make, time, two, right, new, also, propos, distribut
--- well, presid, say, thank, develop, opinion, inform, chip, world, clipper",
"+++ peopl, could, make, time, two, public, law, right, new, also
--- car, well, presid, say, mani, develop, access, thing, inform, chip",
"+++ program, need, time, also, new, distribut, data, work, system, inform
--- two, presid, right, thank, develop, access, sale, chip, clipper, softwar",
"+++ way, could, make, time, two, law, new, also, peopl, think
--- well, presid, right, say, mani, develop, access, thing, inform, chip",
"+++ could, make, time, also, right, may, peopl, think, system, way
--- well, two, presid, say, mani, develop, access, thing, inform, chip",
"+++ encrypt, commun, two, presid, law, right, govern, privaci, data, develop
--- "
]
],
"type": "heatmap",
"uid": "f56df2f1-5247-4d40-969e-fdba578be822",
"z": [
[
0,
0.7595438359657607,
0.9262841492358087,
0.7884700368007593,
1,
0.7323566346435393,
0.8170752433154621,
0.8413158254151436,
0.8740155079503418,
0.7695480873771401,
0.709199118658843,
0.8879803722003395,
0.7314161717683719,
0.7237986193866011,
0.8038717033978985
],
[
0.7595438359657607,
0,
0.7318068179280713,
0.7160213176898725,
0.931461318117774,
0.7726170600952607,
0.7862904411360011,
0.7437804773160451,
0.7261304682479388,
0.7535871571515667,
0.6712180651685388,
0.7162615966682004,
0.6791077745495656,
0.6720945460948236,
0.6964932824724144
],
[
0.9262841492358087,
0.7318068179280713,
0,
0.8574415951298031,
0.9854159083689508,
0.9069175150192125,
0.9046739989611283,
0.8043455907707084,
0.7465496685414171,
0.8585050034234649,
0.844819756902363,
0.6933520111362702,
0.8380803175475514,
0.8430082043112994,
0.8796770147813993
],
[
0.7884700368007593,
0.7160213176898725,
0.8574415951298031,
0,
0.9355864990614852,
0.8123575778010284,
0.7531174879151491,
0.7726893543131144,
0.841589742333695,
0.7803722520664951,
0.7108866861290872,
0.8319749623897799,
0.7790096616692483,
0.7570699362092709,
0.8428560212776922
],
[
1,
0.931461318117774,
0.9854159083689508,
0.9355864990614852,
0,
0.9943633453585028,
0.96633290846124,
0.9427710170298542,
0.9842359174618569,
0.9565789495122544,
0.9252482154378123,
0.9546269085033189,
0.9831865525817131,
0.976673603156099,
0.9862908354943184
],
[
0.7323566346435393,
0.7726170600952607,
0.9069175150192125,
0.8123575778010284,
0.9943633453585028,
0,
0.8369554935234337,
0.8334980316244139,
0.81754589609866,
0.8444443421224488,
0.7641296194848691,
0.8514311288067665,
0.7710627082125688,
0.7681500144434829,
0.8247767768912454
],
[
0.8170752433154621,
0.7862904411360011,
0.9046739989611283,
0.7531174879151491,
0.96633290846124,
0.8369554935234337,
0,
0.8285319686301731,
0.8739669989287112,
0.8128104170459997,
0.812028432574059,
0.8763913774589871,
0.8364217703073451,
0.8332002259898788,
0.8746609598118593
],
[
0.8413158254151436,
0.7437804773160451,
0.8043455907707084,
0.7726893543131144,
0.9427710170298542,
0.8334980316244139,
0.8285319686301731,
0,
0.7698580319588464,
0.7980862901604131,
0.7527906457466673,
0.724541349001037,
0.8089451356075532,
0.8178669870572579,
0.760409513739808
],
[
0.8740155079503418,
0.7261304682479388,
0.7465496685414171,
0.841589742333695,
0.9842359174618569,
0.81754589609866,
0.8739669989287112,
0.7698580319588464,
0,
0.8306979226984069,
0.8043458026246182,
0.6430984098928708,
0.8108465393365341,
0.8313197770928223,
0.7982346696285947
],
[
0.7695480873771401,
0.7535871571515667,
0.8585050034234649,
0.7803722520664951,
0.9565789495122544,
0.8444443421224488,
0.8128104170459997,
0.7980862901604131,
0.8306979226984069,
0,
0.7686319141956662,
0.8025391599365734,
0.7810887188379143,
0.7900886236487616,
0.8371214921530137
],
[
0.709199118658843,
0.6712180651685388,
0.844819756902363,
0.7108866861290872,
0.9252482154378123,
0.7641296194848691,
0.812028432574059,
0.7527906457466673,
0.8043458026246182,
0.7686319141956662,
0,
0.7921819700961124,
0.7034290766439234,
0.6881510253531813,
0.7233383725539672
],
[
0.8879803722003395,
0.7162615966682004,
0.6933520111362702,
0.8319749623897799,
0.9546269085033189,
0.8514311288067665,
0.8763913774589871,
0.724541349001037,
0.6430984098928708,
0.8025391599365734,
0.7921819700961124,
0,
0.8235993983490295,
0.8508846565675093,
0.7915239430267434
],
[
0.7314161717683719,
0.6791077745495656,
0.8380803175475514,
0.7790096616692483,
0.9831865525817131,
0.7710627082125688,
0.8364217703073451,
0.8089451356075532,
0.8108465393365341,
0.7810887188379143,
0.7034290766439234,
0.8235993983490295,
0,
0.5398098970501584,
0.7811421666690115
],
[
0.7237986193866011,
0.6720945460948236,
0.8430082043112994,
0.7570699362092709,
0.976673603156099,
0.7681500144434829,
0.8332002259898788,
0.8178669870572579,
0.8313197770928223,
0.7900886236487616,
0.6881510253531813,
0.8508846565675093,
0.5398098970501584,
0,
0.7854428745787172
],
[
0.8038717033978985,
0.6964932824724144,
0.8796770147813993,
0.8428560212776922,
0.9862908354943184,
0.8247767768912454,
0.8746609598118593,
0.760409513739808,
0.7982346696285947,
0.8371214921530137,
0.7233383725539672,
0.7915239430267434,
0.7811421666690115,
0.7854428745787172,
0
]
]
}
],
"layout": {
"height": 950,
"title": "Topic difference (one model)[hellinger distance]",
"width": 950,
"xaxis": {
"title": "topic"
},
"yaxis": {
"title": "topic"
}
}
},
"text/html": [
""
],
"text/vnd.plotly.v1+html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"mdiff, annotation = lda_fst.diff(lda_fst, distance='hellinger', num_words=50)\n",
"plot_difference(mdiff, title=\"Topic difference (one model)[hellinger distance]\", annotation=annotation)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You see that everything has become worse, but remember that everything depends on the task.\n",
"\n",
"You need to choose the function with which your personal point of view about topics similarity and your task (from my experience, Jaccard is fine)."
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"## Case 2: How topics from DIFFERENT models correlate with each other."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Sometimes, we want to look at the patterns between two different models and compare them. \n",
"\n",
"You can do this by constructing a matrix with the difference."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"linkText": "Export to plot.ly",
"plotlyServerURL": "https://plot.ly",
"showLink": true
},
"data": [
{
"colorscale": "RdBu",
"text": [
[
"+++ world, could, peopl, make, well, time, two, nazi, right, also
--- car, azeri, turkey, say, mani, give, serdar, german, 000, question",
"+++
--- well, two, c8v, right, say, mani, b8f, okz, give, qtm",
"+++ want, time, also, call
--- server, well, two, right, say, mani, thank, display, give, inform",
"+++ well, two, right, say, mani, want, govern, state, even, make
--- atheist, said, evid, point, must, law, arab, american, live, palestinian",
"+++ even, could, said, kill, well, make, two, time, peopl, right
--- presid, went, mani, give, made, stephanopoulo, thing, happen, world, 000",
"+++ even, world, could, make, well, time, right, say, mani, think
--- two, give, opinion, thing, also, sgi, 000, jewish, question, much",
"+++ world, could, time, also, nation, year
--- well, two, right, say, mani, develop, lunar, news, give, project",
"+++ even, could, make, time, two, also, say, mani, govern, peopl
--- well, right, develop, give, inform, chip, world, clipper, need, number",
"+++ even, make, well, time, two, right, say, think, much, year
--- hockey, mani, give, nhl, also, score, world, 000, jewish, question",
"+++ two, time, also, want
--- modem, well, floppi, right, say, mani, thank, access, give, speed",
"+++ jewish, year
--- well, two, right, check, say, mani, give, char, also, sourc",
"+++ right, arab, say, live, mani, want, state, israel, even, fact
--- said, well, two, point, law, american, palestinian, believ, govern, case",
"+++ even, could, make, well, time, two, right, also, say, want
--- car, mani, give, thing, speed, world, need, 000, jewish, question",
"+++ year, world, time, 000, also, nation, peopl, state
--- armi, well, two, turkey, right, say, mani, kent, sandvik, give",
"+++ world, could, make, time, also, columbia, want, peopl, insur
--- server, cunixb, well, two, right, say, newsgroup, mani, thank, news"
],
[
"+++ could, make, well, time, key, law, right, new, also, look
--- car, two, azeri, turkey, say, thing, serdar, sgi, german, chip",
"+++
--- well, c8v, right, say, b8f, okz, qtm, wm4u, thing, also",
"+++ need, time, bit, anyon, also, new, look, distribut, want, run
--- server, well, right, say, thank, display, thing, sgi, inform, chip",
"+++ well, law, right, say, want, state, tri, even, make, thing
--- atheist, realli, two, evid, point, must, indiana, mani, run, govern",
"+++ well, right, say, want, state, tri, even, make, thing, also
--- said, realli, two, presid, law, went, indiana, run, believ, disk",
"+++ realli, well, right, say, state, tri, even, make, thing, sgi
--- atheist, caltech, optilink, law, keith, show, indiana, mani, gay, want",
"+++ could, time, new, also, distribut, problem, softwar, work, system
--- well, right, say, develop, lunar, news, project, thing, moon, sgi",
"+++ law, say, even, make, new, also, chip, need, good, work
--- realli, well, two, right, mani, netcom, run, data, develop, thing",
"+++ even, make, well, time, take, right, new, say, look, think
--- hockey, two, nhl, thing, also, sgi, score, chip, need, question",
"+++ need, time, drive, also, new, look, distribut, want, system, problem
--- modem, well, two, floppi, right, say, thank, access, thing, speed",
"+++ read, need, tri, may
--- well, right, check, say, char, thing, also, sourc, sgi, inform",
"+++ law, right, say, want, state, even, make, thing, also, see
--- realli, well, point, arab, live, mani, indiana, run, believ, disk",
"+++ realli, well, right, say, want, run, state, tri, even, make
--- car, two, first, law, wire, indiana, turn, disk, system, got",
"+++ time, also, new, org, peopl, state, work
--- armi, well, turkey, right, say, kent, sandvik, thing, sourc, serdar",
"+++ could, make, time, also, distribut, want, org, run, system, peopl
--- server, cunixb, well, right, say, newsgroup, thank, news, thing, sgi"
],
[
"+++ could, make, time, also, new, repli, look, distribut, want, work
--- car, well, two, azeri, turkey, right, thank, speed, serdar, simm",
"+++
--- c8v, b8f, thank, okz, qtm, wm4u, also, speed, simm, chip",
"+++ want, color, run, thank, mac, mous, set, tri, system, help
--- server, imag, applic, repli, monitor, vga, disk, ide, data, instal",
"+++ time, make, also, want, tri
--- well, two, right, say, mani, thank, without, thing, speed, simm",
"+++ could, make, time, also, look, want, work, tri
--- well, two, presid, right, say, went, thank, made, stephanopoulo, thing",
"+++ could, time, make, system, tri
--- well, right, say, mani, thank, opinion, thing, also, speed, simm",
"+++ could, time, new, also, distribut, problem, scsi, work, system
--- thank, develop, lunar, news, project, speed, simm, moon, inform, chip",
"+++ could, need, make, time, comput, bit, also, new, work, system
--- two, say, mani, thank, develop, speed, simm, inform, clipper, number",
"+++ make, time, new, look, run
--- hockey, well, two, right, say, thank, nhl, also, speed, simm",
"+++ repli, monitor, want, thank, disk, ide, system, pin, new, also
--- modem, price, two, point, floppi, motherboard, origin, mac, run, vga",
"+++ need, help, pleas, tri, comput
--- check, thank, char, also, sourc, speed, simm, inform, chip, number",
"+++ could, time, make, also, want
--- right, say, mani, thank, thing, speed, simm, chip, world, need",
"+++ could, need, make, time, drive, new, also, speed, look, distribut
--- car, well, two, right, say, thank, thing, simm, chip, dod",
"+++ time, appl, also, new, repli, work
--- armi, turkey, thank, kent, sandvik, sourc, speed, serdar, simm, chip",
"+++ window, could, make, time, help, also, repli, distribut, want, run
--- server, cunixb, newsgroup, news, speed, simm, inform, chip, world, need"
],
[
"+++ world, could, car, make, well, time, peopl, right, look, repli
--- armenia, realli, two, azeri, law, turkey, ingr, pass, greek, mani",
"+++
--- car, well, c8v, right, say, mani, b8f, okz, qtm, wm4u",
"+++ need, time, anyon, look, distribut, want, problem, tri
--- server, car, well, right, say, mani, thank, display, thing, also",
"+++ well, point, right, say, mani, want, state, tri, even, make
--- atheist, car, realli, two, evid, first, must, repli, law, govern",
"+++ well, right, say, want, state, tri, got, even, make, thing
--- said, car, realli, two, presid, point, repli, went, mani, believ",
"+++ realli, well, right, say, mani, state, tri, even, make, thing
--- car, show, gay, frank, object, got, cramer, put, opinion, gov",
"+++ world, could, gov, time, distribut, problem, year, first
--- car, well, right, say, mani, develop, lunar, news, project, thing",
"+++ even, could, need, make, time, peopl, say, mani, good, think
--- car, well, two, right, develop, thing, also, inform, chip, happen",
"+++ well, point, right, say, even, make, year, see, better, look
--- period, game, car, play, hockey, two, realli, hit, repli, weapon",
"+++ need, time, point, look, repli, distribut, want, problem, good, anyon
--- modem, car, well, two, floppi, right, say, mani, thank, access",
"+++ need, tri, year, read, first
--- car, well, right, check, say, mani, char, thing, sourc, inform",
"+++ point, right, say, mani, want, state, even, make, thing, see
--- car, realli, well, law, first, arab, repli, live, believ, tri",
"+++ car, realli, well, right, say, want, state, tri, got, even
--- two, point, wire, repli, mani, turn, run, put, gov, new",
"+++ year, world, time, repli, peopl, state, first
--- armi, car, well, turkey, right, say, mani, kent, sandvik, thing",
"+++ world, could, make, time, repli, distribut, want, peopl, anyon
--- server, car, cunixb, well, right, say, newsgroup, mani, thank, news"
],
[
"+++ car, make, time, new, look, repli, distribut, want, think, engin
--- well, two, azeri, turkey, right, c8v, b8f, okz, also, serdar",
"+++ 2tm, 1eq, c8v, 2di, b8f, 1d9, 7ey, max, okz, b8e
--- dyer, car, 1d9l, repli, want, 7ez, r8f, f9d, brake, mile",
"+++ need, time, new, look, distribut, want, anyon
--- server, car, c8v, b8f, thank, okz, display, also, uchicago, inform",
"+++ time, make, want, good, think
--- car, well, two, right, c8v, say, mani, b8f, without, okz",
"+++ time, make, look, want, good, think
--- car, well, two, presid, right, c8v, say, went, b8f, okz",
"+++ time, think, make, good
--- car, well, right, c8v, say, mani, b8f, okz, opinion, thing",
"+++ time, new, engin, distribut
--- car, c8v, b8f, develop, okz, lunar, news, project, also, moon",
"+++ need, time, make, new, good, think
--- car, two, c8v, say, mani, b8f, develop, okz, also, uchicago",
"+++ time, make, new, look, good, think
--- car, hockey, well, two, right, c8v, say, b8f, okz, nhl",
"+++ need, time, new, look, repli, usa, want, distribut, good, anyon
--- modem, car, two, floppi, c8v, b8f, thank, okz, access, also",
"+++ need
--- car, c8v, check, b8f, okz, char, sourc, uchicago, inform, number",
"+++ time, make, want, good, think
--- car, right, c8v, say, mani, b8f, okz, thing, also, uchicago",
"+++ car, make, need, time, new, look, usa, want, distribut, good
--- well, two, right, c8v, say, b8f, okz, thing, also, speed",
"+++ time, new, repli, umd, eng
--- armi, car, turkey, c8v, b8f, kent, sandvik, okz, also, sourc",
"+++ time, make, repli, distribut, want, anyon
--- server, car, cunixb, c8v, newsgroup, b8f, thank, okz, news, also"
],
[
"+++ armenia, well, two, turkey, right, greek, want, turk, new, serdar
--- car, said, azeri, law, file, repli, ingr, pass, live, say",
"+++
--- well, two, turkey, c8v, right, say, went, b8f, okz, qtm",
"+++ program, need, time, work, new, look, want, file, call
--- server, well, two, turkey, right, say, went, thank, display, also",
"+++ even, time, well, two, right, say, want, think, peopl, see
--- turkey, went, mani, without, thing, also, serdar, world, need, number",
"+++ said, well, two, right, say, went, want, start, come, even
--- armenia, presid, turkey, file, azerbaijani, live, greek, believ, state, return",
"+++ even, world, could, time, well, right, say, think, someth, peopl
--- two, turkey, went, mani, opinion, thing, serdar, sgi, need, number",
"+++ program, world, could, time, new, year, work, first
--- well, two, turkey, right, say, went, develop, lunar, news, project",
"+++ even, could, need, time, number, two, work, new, say, peopl
--- well, turkey, right, went, mani, develop, also, serdar, inform, chip",
"+++ even, time, well, two, right, new, say, look, think, start
--- hockey, turkey, went, nhl, serdar, score, world, need, number, season",
"+++ need, time, two, new, look, want, work
--- modem, well, turkey, floppi, right, say, went, thank, access, also",
"+++ rule, program, need, number, output, file, build, name, year, return
--- well, two, turkey, right, check, say, went, char, sourc, serdar",
"+++ even, world, could, kill, time, right, say, live, want, think
--- well, two, turkey, went, mani, thing, also, serdar, need, number",
"+++ even, could, need, time, well, two, right, new, say, look
--- car, turkey, went, thing, also, speed, serdar, world, number, dod",
"+++ armenia, turkey, file, muslim, turk, new, serdar, year, world, number
--- armi, david, well, two, right, say, went, greek, popul, kent",
"+++ world, could, time, file, want, name, peopl
--- server, cunixb, well, two, turkey, right, say, newsgroup, went, thank"
],
[
"+++ make, well, time, two, new, right, also, look, year, think
--- car, hockey, azeri, turkey, presid, say, stephanopoulo, nhl, serdar, german",
"+++
--- hockey, well, two, presid, c8v, right, say, b8f, okz, qtm",
"+++ time, also, look, new, run
--- server, hockey, well, two, presid, right, say, thank, display, stephanopoulo",
"+++ even, make, well, time, two, point, right, also, say, think
--- hockey, presid, mani, without, stephanopoulo, thing, nhl, score, question, constitut",
"+++ said, well, two, presid, right, say, start, got, come, even
--- game, play, hockey, point, hit, went, want, win, run, believ",
"+++ even, make, well, time, right, say, think, good, see
--- hockey, two, presid, mani, opinion, stephanopoulo, thing, nhl, also, sgi",
"+++ time, new, also, year, first
--- hockey, well, two, presid, right, say, develop, lunar, news, project",
"+++ even, make, time, two, new, also, say, good, think
--- hockey, well, presid, right, mani, develop, stephanopoulo, nhl, inform, chip",
"+++ game, play, hockey, well, two, point, right, hit, say, win
--- period, said, presid, weapon, next, got, come, stephanopoulo, also, day",
"+++ time, two, point, new, look, also, good
--- modem, hockey, well, presid, floppi, right, say, thank, access, stephanopoulo",
"+++ first, year
--- hockey, well, two, presid, right, check, say, char, stephanopoulo, nhl",
"+++ day, even, make, time, point, right, also, say, think, good
--- hockey, well, two, presid, mani, stephanopoulo, thing, nhl, score, world",
"+++ well, two, right, say, run, got, even, make, new, also
--- game, car, realli, said, play, wire, hockey, point, want, turn",
"+++ time, also, new, toronto, year, first
--- armi, hockey, well, two, presid, turkey, right, say, kent, sandvik",
"+++ also, run, make, time
--- server, cunixb, hockey, well, two, presid, right, say, newsgroup, thank"
],
[
"+++ could, make, well, time, two, new, also, repli, sun, look
--- car, current, azeri, turkey, right, serdar, sale, german, world, need",
"+++
--- well, current, two, c8v, b8f, okz, qtm, wm4u, also, sale",
"+++ need, time, imag, also, new, look, includ, want, distribut, run
--- server, well, current, two, thank, display, sale, inform, question, softwar",
"+++ make, well, time, two, point, question, also, want, good, may
--- current, right, say, mani, without, thing, sale, need, constitut, much",
"+++ could, make, well, time, two, also, look, want, good, year
--- current, presid, right, say, went, made, stephanopoulo, thing, sale, happen",
"+++ could, time, make, well, good, system
--- current, two, right, say, mani, opinion, thing, also, sale, sgi",
"+++ imag, system, gov, scienc, also, new, year, space, center, includ
--- well, fund, current, two, point, wire, satellit, repli, 1993, want",
"+++ could, need, make, scienc, time, two, work, new, also, pitt
--- well, current, say, mani, develop, sale, inform, chip, clipper, number",
"+++ make, well, time, two, point, new, look, good, run, year
--- hockey, current, right, say, nhl, also, sale, score, need, question",
"+++ two, point, repli, want, offer, new, also, power, sale, need
--- modem, imag, current, price, first, well, floppi, motherboard, wire, monitor",
"+++ need, includ, year, first, may, comput
--- well, current, two, check, char, also, sourc, sale, inform, number",
"+++ could, make, time, point, question, also, want, book, good, may
--- well, current, two, right, say, mani, thing, sale, world, need",
"+++ well, two, wire, want, run, make, new, also, year, need
--- car, realli, imag, current, right, say, point, repli, turn, pitt",
"+++ time, also, new, repli, center, year, work, first
--- armi, well, current, two, turkey, kent, sandvik, sourc, serdar, sale",
"+++ could, make, time, also, repli, distribut, want, run, system, may
--- server, cunixb, current, two, well, newsgroup, thank, news, sale, inform"
],
[
"+++ make, time, also, look, repli, distribut, want, work, way
--- server, client, car, well, two, azeri, turkey, right, resourc, thank",
"+++
--- server, client, c8v, b8f, resourc, thank, okz, display, qtm, wm4u",
"+++ server, applic, want, color, run, thank, data, set, tri, display
--- client, imag, motif, repli, resourc, mac, mous, make, graphic, new",
"+++ way, make, time, also, want, tri, may
--- server, client, well, two, right, say, mani, resourc, thank, without",
"+++ make, time, also, look, want, work, tri, way
--- server, client, well, two, presid, right, say, went, resourc, thank",
"+++ time, system, tri, make
--- server, client, well, right, say, mani, resourc, thank, display, opinion",
"+++ program, time, softwar, also, group, includ, distribut, avail, problem, data
--- server, client, resourc, thank, develop, display, lunar, news, project, sourc",
"+++ need, make, time, number, bit, work, also, may, data, call
--- server, client, two, say, mani, resourc, thank, develop, display, sourc",
"+++ time, make, look, run, way
--- server, client, hockey, well, two, right, say, resourc, thank, display",
"+++ need, time, also, look, repli, includ, want, distribut, thank, problem
--- modem, server, client, two, floppi, resourc, display, access, sourc, speed",
"+++ program, need, help, number, sourc, includ, name, mail, may, file
--- server, client, check, resourc, thank, display, char, also, lib, jewish",
"+++ way, make, time, also, want, may
--- server, client, right, say, mani, resourc, thank, display, thing, sourc",
"+++ way, need, make, time, also, look, distribut, want, run, problem
--- server, client, car, well, two, right, say, resourc, thank, display",
"+++ time, number, also, sourc, repli, file, work
--- armi, server, client, turkey, resourc, thank, kent, sandvik, display, serdar",
"+++ server, repli, want, run, thank, make, help, also, inform, window
--- client, anonym, cunixb, motif, applic, newsgroup, color, resourc, privaci, data"
],
[
"+++ well, two, right, repli, want, state, make, new, also, convex
--- car, armenia, azeri, law, turkey, say, ingr, pass, greek, run",
"+++
--- well, two, c8v, right, say, b8f, thank, okz, access, opinion",
"+++ time, also, new, look, distribut, want, run, thank, problem, pleas
--- server, well, two, right, say, display, access, opinion, inform, world",
"+++ peopl, make, well, time, two, question, right, also, say, human
--- mani, thank, without, access, opinion, thing, world, great, constitut, dod",
"+++ peopl, make, well, time, two, right, also, say, look, want
--- presid, went, run, thank, ask, got, tax, access, kill, opinion",
"+++ world, peopl, opinion, make, well, time, right, say, think, system
--- two, mani, thank, access, thing, also, sgi, great, question, dod",
"+++ year, world, time, new, also, distribut, problem, base, system
--- well, two, right, say, thank, develop, lunar, news, access, opinion",
"+++ peopl, make, time, two, also, new, say, good, think, system
--- well, right, mani, thank, develop, access, opinion, inform, chip, world",
"+++ make, well, time, two, right, new, say, look, good, run
--- hockey, thank, access, opinion, nhl, also, score, world, great, question",
"+++ access, time, two, new, also, repli, look, usa, want, distribut
--- modem, well, floppi, right, say, opinion, speed, sale, world, need",
"+++ comput, pleas, tri, year
--- well, two, right, check, say, thank, access, char, opinion, also",
"+++ world, peopl, make, time, question, right, also, say, human, want
--- well, two, mani, thank, access, opinion, thing, great, jewish, dod",
"+++ well, two, right, say, want, run, state, tri, make, new
--- car, realli, first, wire, repli, turn, thank, net, bnr, pyron",
"+++ world, peopl, time, also, new, repli, org, year, state
--- armi, well, two, turkey, right, say, thank, kent, sandvik, access",
"+++ world, make, time, also, repli, distribut, want, org, run, thank
--- server, cunixb, well, two, right, say, newsgroup, news, access, opinion"
],
[
"+++ crime, could, car, make, well, time, two, peopl, law, right
--- armenia, azeri, turkey, file, say, pass, weapon, greek, mani, health",
"+++
--- car, well, two, c8v, right, say, mani, b8f, okz, qtm",
"+++ time, new, also, problem, file, system
--- server, car, well, two, right, say, mani, thank, display, thing",
"+++ well, two, law, right, say, mani, state, even, make, thing
--- atheist, car, evid, point, first, must, file, weapon, want, govern",
"+++ well, two, right, say, state, even, make, thing, also, year
--- car, presid, weapon, went, mani, health, ask, got, tax, crime",
"+++ even, could, peopl, make, well, time, person, thing, right, say
--- car, realli, two, show, weapon, gay, frank, object, health, cramer",
"+++ could, time, new, also, system, problem, scsi, year, first
--- car, well, two, right, say, mani, develop, lunar, news, project",
"+++ two, law, say, mani, case, even, make, new, also, number
--- car, well, right, weapon, netcom, data, develop, health, crime, thing",
"+++ even, make, well, firearm, two, time, gun, right, new, say
--- car, hockey, file, mani, run, health, crime, nhl, also, thing",
"+++ time, two, also, new, problem, scsi, good, control, system
--- modem, car, well, floppi, right, say, mani, thank, access, thing",
"+++ file, first, number, year
--- car, well, two, right, check, say, mani, char, thing, also",
"+++ day, even, could, peopl, make, time, person, law, right, thing
--- car, well, two, arab, weapon, health, crime, kill, christ, new",
"+++ car, well, two, right, say, state, even, make, thing, new
--- realli, law, wire, file, weapon, want, turn, mani, run, health",
"+++ peopl, report, time, number, public, also, new, file, year, state
--- armi, car, well, two, turkey, right, say, mani, kent, sandvik",
"+++ could, time, make, also, file, peopl, system
--- server, car, cunixb, well, two, right, say, newsgroup, mani, thank"
],
[
"+++ time, also, new, look, repli, distribut, want, work, comput
--- car, well, two, azeri, turkey, right, thank, serdar, sale, german",
"+++
--- c8v, b8f, thank, okz, qtm, wm4u, also, sale, inform, 3di",
"+++ imag, file, want, mac, run, thank, data, system, program, help
--- server, price, applic, repli, color, netcom, mous, set, display, tri",
"+++ want, also, time
--- well, two, right, say, mani, thank, without, thing, sale, inform",
"+++ time, also, look, want, work
--- well, two, presid, right, say, went, thank, made, stephanopoulo, thing",
"+++ system, time
--- well, right, say, mani, thank, opinion, thing, also, sale, sgi",
"+++ program, time, imag, softwar, new, also, graphic, includ, distribut, avail
--- thank, develop, lunar, news, project, sale, moon, world, need, group",
"+++ need, time, also, new, netcom, data, work, system, inform, comput
--- two, say, mani, thank, develop, sale, chip, clipper, number, softwar",
"+++ new, run, look, time
--- hockey, well, two, right, say, thank, nhl, also, sale, score",
"+++ price, repli, want, thank, data, system, new, also, sale, need
--- modem, imag, two, point, floppi, motherboard, file, monitor, origin, mac",
"+++ info, program, need, address, list, help, includ, send, pleas, mail
--- check, thank, char, also, sourc, sale, number, lib, jewish, titl",
"+++ want, also, time
--- right, say, mani, thank, thing, sale, inform, world, need, question",
"+++ need, time, drive, new, also, look, distribut, want, run, work
--- car, well, two, right, say, thank, thing, speed, sale, inform",
"+++ time, also, new, repli, file, work
--- armi, turkey, thank, kent, sandvik, sourc, serdar, sale, inform, world",
"+++ repli, want, run, thank, system, help, also, inform, window, pleas
--- server, anonym, cunixb, imag, price, newsgroup, mac, netcom, privaci, data"
],
[
"+++ world, could, make, well, time, two, law, new, also, want
--- car, azeri, turkey, right, say, mani, thing, serdar, german, question",
"+++
--- well, two, c8v, say, mani, b8f, okz, qtm, wm4u, thing",
"+++ want, new, also, time
--- server, well, two, say, mani, thank, display, thing, inform, world",
"+++ well, two, point, law, say, mani, want, believ, argument, even
--- atheist, said, evid, right, must, govern, state, without, tri, come",
"+++ said, well, two, say, want, believ, come, even, make, thing
--- presid, point, right, law, went, mani, argument, start, state, seem",
"+++ well, say, mani, believ, even, make, thing, see, world, god
--- atheist, said, caltech, optilink, realli, two, point, right, keith, show",
"+++ world, could, time, scienc, new, also
--- well, two, say, mani, develop, lunar, news, project, thing, moon",
"+++ even, could, peopl, make, scienc, time, two, law, new, also
--- well, develop, thing, inform, chip, world, clipper, need, number, question",
"+++ even, make, well, time, two, point, new, say, think, much
--- hockey, right, mani, nhl, thing, also, score, world, question, season",
"+++ time, two, point, new, also, want, good
--- modem, well, floppi, say, mani, thank, access, thing, speed, sale",
"+++ read, may
--- well, two, check, say, mani, char, thing, also, sourc, inform",
"+++ point, law, say, mani, want, believ, come, even, make, thing
--- said, well, two, right, arab, live, argument, state, israel, fact",
"+++ day, even, way, could, make, well, time, two, thing, new
--- car, realli, said, first, point, right, wire, law, turn, mani",
"+++ world, time, also, new, peopl
--- armi, well, two, turkey, say, mani, kent, sandvik, thing, sourc",
"+++ world, could, make, time, also, want, peopl, may
--- server, cunixb, well, two, say, newsgroup, mani, thank, news, thing"
],
[
"+++ world, could, make, well, time, also, right, want, peopl, think
--- car, two, azeri, turkey, say, mani, thing, serdar, german, question",
"+++
--- well, c8v, right, say, mani, b8f, okz, qtm, wm4u, thing",
"+++ time, also, want, system, problem, tri
--- server, well, right, say, mani, thank, display, thing, inform, world",
"+++ atheist, well, point, right, must, say, mani, want, believ, tri
--- said, realli, two, evid, law, govern, object, argument, state, without",
"+++ said, well, right, say, want, believ, tri, come, even, make
--- atheist, realli, two, presid, point, must, went, mani, object, start",
"+++ atheist, realli, well, right, say, mani, object, believ, tri, even
--- said, caltech, optilink, point, must, keith, show, want, gay, frank",
"+++ world, could, time, also, problem, system
--- well, right, say, mani, develop, lunar, news, project, thing, moon",
"+++ even, could, peopl, make, time, also, say, mani, good, think
--- well, two, right, develop, thing, inform, chip, world, clipper, need",
"+++ even, make, well, time, take, point, right, say, think, good
--- hockey, two, mani, nhl, thing, also, score, world, question, season",
"+++ time, point, also, want, problem, good, system
--- modem, well, two, floppi, right, say, mani, thank, access, thing",
"+++ must, tri, may, follow
--- well, right, check, say, mani, char, thing, also, sourc, inform",
"+++ point, right, say, mani, want, believ, come, even, make, thing
--- atheist, said, realli, well, law, must, arab, live, object, state",
"+++ realli, well, right, say, want, tri, even, make, thing, also
--- atheist, car, said, two, first, point, must, wire, turn, mani",
"+++ world, also, time, peopl
--- armi, well, turkey, right, say, mani, kent, sandvik, thing, sourc",
"+++ world, could, make, find, time, also, want, peopl, system, may
--- server, cunixb, well, right, say, newsgroup, mani, thank, news, thing"
],
[
"+++ could, peopl, make, time, key, two, law, right, new, also
--- car, well, azeri, turkey, presid, develop, access, serdar, german, inform",
"+++
--- two, presid, c8v, right, b8f, develop, okz, access, qtm, wm4u",
"+++ program, need, time, new, also, distribut, data, work, system, inform
--- server, two, presid, right, thank, develop, display, access, chip, clipper",
"+++ way, make, time, two, law, right, also, think, govern, peopl
--- well, presid, say, mani, develop, without, access, thing, inform, chip",
"+++ could, peopl, make, time, two, presid, right, also, think, year
--- well, say, went, develop, access, made, stephanopoulo, thing, inform, chip",
"+++ could, time, make, right, think, peopl, state, system
--- well, two, presid, say, mani, develop, access, opinion, thing, also",
"+++ year, program, could, time, launch, also, data, new, technolog, space
--- two, presid, right, lunar, news, access, project, moon, chip, world",
"+++ encrypt, commun, two, law, govern, develop, data, escrow, secret, make
--- effect, presid, first, right, say, mani, clinton, pitt, netcom, privaci",
"+++ make, time, two, new, right, year, think, first, way
--- hockey, well, presid, say, develop, access, nhl, also, score, inform",
"+++ access, need, time, two, new, also, space, distribut, data, work
--- modem, presid, floppi, right, thank, develop, speed, sale, inform, chip",
"+++ program, need, inform, year, first, may, comput
--- two, presid, right, check, develop, access, char, also, sourc, chip",
"+++ could, make, time, law, also, right, think, may, peopl, state
--- two, presid, say, mani, develop, access, thing, inform, chip, world",
"+++ way, could, need, make, time, two, right, new, also, distribut
--- car, well, presid, say, develop, access, thing, speed, inform, chip",
"+++ year, time, public, also, new, peopl, state, work, first
--- armi, two, presid, turkey, right, develop, kent, sandvik, access, sourc",
"+++ could, make, time, also, messag, distribut, privaci, may, peopl, system
--- server, cunixb, two, presid, right, newsgroup, thank, develop, news, access"
]
],
"type": "heatmap",
"uid": "a3ce4eba-5101-4544-850d-7b2963eb6250",
"z": [
[
0.8095238095238095,
1,
0.9583333333333334,
0.75,
0.7804878048780488,
0.8235294117647058,
0.9361702127659575,
0.8505747126436781,
0.8636363636363636,
0.9583333333333334,
0.9795918367346939,
0.6486486486486487,
0.8095238095238095,
0.9130434782608696,
0.9010989010989011
],
[
0.8095238095238095,
1,
0.8095238095238095,
0.7341772151898734,
0.7341772151898734,
0.717948717948718,
0.9010989010989011,
0.7654320987654322,
0.8372093023255813,
0.8235294117647058,
0.9583333333333334,
0.7341772151898734,
0.6301369863013699,
0.9247311827956989,
0.8636363636363636
],
[
0.8764044943820225,
1,
0.6111111111111112,
0.9473684210526316,
0.9130434782608696,
0.9473684210526316,
0.9010989010989011,
0.8764044943820225,
0.9473684210526316,
0.6486486486486487,
0.9473684210526316,
0.9473684210526316,
0.8235294117647058,
0.9361702127659575,
0.8235294117647058
],
[
0.7804878048780488,
1,
0.9130434782608696,
0.7654320987654322,
0.6666666666666667,
0.7654320987654322,
0.9130434782608696,
0.8888888888888888,
0.7654320987654322,
0.8888888888888888,
0.9473684210526316,
0.7654320987654322,
0.5507246376811594,
0.9247311827956989,
0.9010989010989011
],
[
0.8888888888888888,
0.5915492957746479,
0.9247311827956989,
0.9473684210526316,
0.9361702127659575,
0.9583333333333334,
0.9583333333333334,
0.9361702127659575,
0.9361702127659575,
0.8888888888888888,
0.98989898989899,
0.9473684210526316,
0.8636363636363636,
0.9473684210526316,
0.9361702127659575
],
[
0.7012987012987013,
1,
0.9010989010989011,
0.8764044943820225,
0.6842105263157895,
0.8764044943820225,
0.9130434782608696,
0.8636363636363636,
0.8372093023255813,
0.9247311827956989,
0.8636363636363636,
0.8372093023255813,
0.7951807228915663,
0.7654320987654322,
0.9247311827956989
],
[
0.8888888888888888,
1,
0.9473684210526316,
0.8505747126436781,
0.717948717948718,
0.9010989010989011,
0.9473684210526316,
0.9010989010989011,
0.33333333333333337,
0.9247311827956989,
0.9795918367346939,
0.8636363636363636,
0.7341772151898734,
0.9361702127659575,
0.9583333333333334
],
[
0.8095238095238095,
1,
0.8505747126436781,
0.8764044943820225,
0.8636363636363636,
0.9361702127659575,
0.7341772151898734,
0.8095238095238095,
0.8764044943820225,
0.75,
0.9361702127659575,
0.8888888888888888,
0.7341772151898734,
0.9130434782608696,
0.8764044943820225
],
[
0.9010989010989011,
1,
0.48484848484848486,
0.9247311827956989,
0.9130434782608696,
0.9583333333333334,
0.8505747126436781,
0.8636363636363636,
0.9473684210526316,
0.8372093023255813,
0.8636363636363636,
0.9361702127659575,
0.8505747126436781,
0.9247311827956989,
0.717948717948718
],
[
0.75,
1,
0.8505747126436781,
0.8095238095238095,
0.7951807228915663,
0.8505747126436781,
0.9010989010989011,
0.8764044943820225,
0.8372093023255813,
0.7804878048780488,
0.9583333333333334,
0.8235294117647058,
0.7341772151898734,
0.9010989010989011,
0.8372093023255813
],
[
0.8095238095238095,
1,
0.9361702127659575,
0.7341772151898734,
0.7654320987654322,
0.7804878048780488,
0.9010989010989011,
0.7654320987654322,
0.7804878048780488,
0.9010989010989011,
0.9583333333333334,
0.7804878048780488,
0.717948717948718,
0.8764044943820225,
0.9247311827956989
],
[
0.9010989010989011,
1,
0.5294117647058824,
0.9690721649484536,
0.9473684210526316,
0.9795918367346939,
0.8372093023255813,
0.8888888888888888,
0.9583333333333334,
0.75,
0.8372093023255813,
0.9690721649484536,
0.8888888888888888,
0.9361702127659575,
0.717948717948718
],
[
0.8505747126436781,
1,
0.9583333333333334,
0.5074626865671642,
0.75,
0.7654320987654322,
0.9361702127659575,
0.8235294117647058,
0.8505747126436781,
0.9247311827956989,
0.9795918367346939,
0.46153846153846156,
0.7804878048780488,
0.9473684210526316,
0.9130434782608696
],
[
0.8764044943820225,
1,
0.9361702127659575,
0.5507246376811594,
0.717948717948718,
0.6301369863013699,
0.9361702127659575,
0.8636363636363636,
0.8636363636363636,
0.9247311827956989,
0.9583333333333334,
0.5294117647058824,
0.7654320987654322,
0.9583333333333334,
0.8888888888888888
],
[
0.7804878048780488,
1,
0.8764044943820225,
0.8636363636363636,
0.8235294117647058,
0.9130434782608696,
0.7804878048780488,
0.5507246376811594,
0.9010989010989011,
0.8636363636363636,
0.9247311827956989,
0.8764044943820225,
0.8095238095238095,
0.9010989010989011,
0.8636363636363636
]
]
}
],
"layout": {
"height": 950,
"title": "Topic difference (two models)[jaccard distance]",
"width": 950,
"xaxis": {
"title": "topic"
},
"yaxis": {
"title": "topic"
}
}
},
"text/html": [
""
],
"text/vnd.plotly.v1+html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"mdiff, annotation = lda_fst.diff(lda_snd, distance='jaccard', num_words=50)\n",
"plot_difference(mdiff, title=\"Topic difference (two models)[jaccard distance]\", annotation=annotation)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Looking at this matrix, you can find similar and different topics (and relevant tokens which describe the intersection and difference)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}