Latent Semantic Analysis workflows

Wolfram Language Conference
Saint Petersburg
June 2023

Anton Antonov
MathematicaForPrediction at GitHub
MathematicaForPrediction at WordPress

Abstract

In this presentation we outline typical step in Latent Semantic Analysis (LSA) workflows and demonstrates with several types of examples:

Topic extraction from manuals and novels

And use in language translations.

Deconstruction of collection of images

And generation of new ones with the obtained bases.

Making certain types of visual effects.

Like in the movie "Watchmen" (2009)

Rorschach's mask

Connections with other mathematical and data science subcultures are going to be discussed.

LSA via Questions and Answers

- What data is going used?

- What is a typical workflow example?

- What are the typical applications of LSA?

- Why use LSA?

- What it the fundamental philosophical or scientific assumption for LSA?

- What is the most important and/or fundamental step of LSA?

- What is the difference between LSA and Latent Semantic Indexing (LSI)?

- What are the alternatives?
  - Using Neural Networks instead?

- How is LSA used to derive similarities between two given texts?

- How is LSA used to evaluate the proximity of phrases?

- How the main dimension reduction methods compare?

SVD, NNMF, ICA

Flowchart

Import["~/Downloads/LSA-workflows.pdf","PageGraphics"][[1]]

First examples

"MonadicLatentSemanticAnalysis"

speeches=ResourceData[ResourceObject["Presidential Nomination Acceptance Speeches"]];
texts=Normal[speeches[[All,"Text"]]];

res=
({{, LSAMonUnit }})[texts]⟹
({{, LSAMonMakeDocumentTermMatrix }})[{},Automatic]⟹
({{, LSAMonApplyTermWeightFunctions }})[]⟹
({{, LSAMonExtractTopics }})["NumberOfTopics"->20,Method->"SVD"]⟹
({{, LSAMonEchoTopicsTable }})[Dividers->All]⟹
({{, LSAMonExtractStatisticalThesaurus }})[{"war","nuclear","health","education"},10]⟹
({{, LSAMonEchoStatisticalThesaurusTable }});

Topic extraction over Raku manual

Procedure outline

The Raku manual is given in three different languages: English, Spanish, Russian.

The the manuals are split into chapters and the corresponding chapters from the different languages are spliced together

Typical LSA workflow is applied:

Document-term matrix

Topic extraction

Statistical thesaurus extraction

The data

aDocs[[21]]

Document-term matrix

AbsoluteTiming[
lsaRakuGuide=
LSAMonUnit[aDocs]⟹
LSAMonMakeDocumentTermMatrix[{},Automatic]⟹
LSAMonEchoDocumentTermMatrixStatistics["ParetoPrinciplePlots"->True,PlotTheme->"Scientific"]⟹
LSAMonApplyTermWeightFunctions["IDF","None","Cosine"];
]

Topic extraction

AbsoluteTiming[
lsaRakuGuide=
lsaRakuGuide⟹
LSAMonExtractTopics["NumberOfTopics"->60,"MinNumberOfDocumentsPerTerm"->1,Method->"NNMF",MaxSteps->12];
]

lsaRakuGuide⟹
LSAMonEchoTopicsTable["NumberOfTableColumns"->8,"NumberOfTerms"->20];

Statistical thesaurus extraction

lsaRakuGuide⟹
LSAMonEchoStatisticalThesaurus["Words"->{"regex","interface","expression","operator","object","programming","read","recommended","variable","tmtowtdi"}];

Deconstruction of random mandalas collection

"Comparison of dimension reduction algorithms over mandala images generation"

"LSA methods comparison over random mandalas deconstruction -- WL"

"LSA methods comparison over random mandalas deconstruction -- Python"

"Re-exploring the structure of Chinese character images"

Making Rorschach's mask (from "Watchmen" 2009)

"Rorschach mask animations projected over 3D surfaces"

"Details in Rorschach mask animations making"

References

Articles

[AA1] Anton Antonov, A monad for Latent Semantic Analysis workflows, (2019), MathematicaForPrediction at GitHub.

Packages and paclets

[AA2] Anton Antonov, Monadic Latent Semantic Analysis Mathematica package, (2017), MathematicaForPrediction at GitHub.
    URL: https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicLatentSemanticAnalysis.m .

Videos

[AAv1] Anton Antonov, "Making of Rorschach mask 2D and 3D animations", (2022), YouTube/@AAA4Prediction.

Setup

Install paclets

PacletInstall["AntonAntonov/MonadMakers"]

PacletInstall["AntonAntonov/MonadicLatentSemanticAnalysis"]

PacletInstall["AntonAntonov/DimensionReducers"]

PacletInstall["AntonAntonov/DataReshapers"]

Load paclets

Needs["AntonAntonov`MonadicLatentSemanticAnalysis`"]
Needs["AntonAntonov`DimensionReducers`"]
Needs["AntonAntonov`DataReshapers`"]

Raku manuals

aDocs=;