Guides: Text Mining & Analysis @ Pitt: Topic Modeling (2024)

Topic modelingis used to analyze clustersof "topics" or co-occurring words in a text or series of texts, often with the aim of understanding recurring themes.

Tools

Out-of-the-Box

• MALLET
  For statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text

• Topic Modeling Tool
  For Latent Dirichlet Allocation (LDA)topic modeling

• Factorie
  For natural language processing and information integration such as segmentation, tokenization, part-of-speech tagging, named entity recognition, dependency parsing, mention finding, coreference, lexicon-matching, and latent Dirichlet allocation

• jsLDA
  For in-browser topic modeling

Programmatic

Python

• Genism
  For latent semantic analysis (LSA, LSI, SVD), unsupervised topic modeling (Latent Dirichlet allocation; LDA), embeddings (fastText, word2vec, doc2vec), non-negative matrix factorization (NMF), and term frequency–inverse document frequency (tf-idf)

• NLTK (Natural Language Toolkit)
  For accessing corpora and lexicons, tokenization, stemming, (part-of-speech) tagging, parsing, transformations, translation, chunking, collocations, classification, clustering, topic segmentation, concordancing, frequency distributions, sentiment analysis, named entity recognition, probability distributions, semantic reasoning, evaluation metrics, manipulating linguistic data (in SIL Toolbox format), language modeling, and other NLP tasks

  See Also

  Is LDA Topic Modeling Dead?Choose Number of Topics for LDA Model - MATLAB & Simulink

• spaCy
  For tokenization, named entity recognition, part-of-speech tagging, dependency parsing, sentence segmentation, text classification, lemmatization, morphological analysis, entity linking and more

• scikit-learn
  For classification, regression, clustering, dimensionality reduction, model selection, and preprocessing

• NLP Architect
  For word chunking, named entity recognition, dependency parsing, intent extraction, sentiment classification, language models, transformations, Aspect Based Sentiment Analysis (ABSA), joint intent detection and slot tagging, noun phrase embedding representation (NP2Vec), most common word sense detection, relation identification, cross document coreference, noun phrase semantic segmentation, term set expansion, topics and trend analysis, optimizing NLP/NLU models

• Top2Vec
  For topic modeling,semantic search, andword and document embeddings

R

• tidytext
  For converting to and from non-tidy formats, word and document frequency analysis (tf-idf), n-grams and correlations, sentiment analysis with tidy data, and topic modeling

• topicmodels
  For Latent Dirichlet Allocation (LDA) models and Correlated Topics Models (CTM) by David M. Blei and co-authors and the C++ code for fitting LDA models using Gibbs sampling by Xuan-Hieu Phan and co-authors;provides an interface to the C code

• BTM
  For identifying topics in texts from term-term cooccurrences (hence 'biterm' topic model, BTM)

• topicdoc
  ForLDA and CTM topic models to assist in evaluating topic quality; provide topic-specific diagnostics

• lda
  For Latent Dirichlet Allocation and related models similar to LSA and topic models

• stm(Structural Topic Model)
  For implementinga topic model derivate that can include document-level meta-data; also includes tools for model selection, visualization, and estimation of topic-covariate regressions

• text2vec
  For text vectorization, topic modeling (LDA, LSA), word embeddings (GloVe), and similarities

• mscstexta4r
  For sentiment analysis, topic detection, language detection, and key phrase extraction;provides an interface to the Microsoft Cognitive Services Text Analytics API

Java

• Weka
  For data preprocessing (e.g., stemming, data resampling,transformation),classification, regression, clustering, latent semantic analysis (LSA, LSI),association rules, visualization, filtering, and anonymization

Helpful Resources

• Brett, Megan R. 2012."Topic Modeling: A Basic Introduction."Journal of Digital Humanities12 (1).

• Chang, Jonathan, Sean Gerrish, Chong Wang, Jordan L. Boyd-graber, and David M. Blei. 2009. “Reading Tea Leaves: How Humans Interpret Topic Models.” In Advances in Neural Information Processing Systems 22, edited by Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta, 288–296. Curran Associates, Inc.

• Graham, Shawn, Scott Weingart, and Ian Milligan. 2012. “Getting Started with Topic Modeling and MALLET.” Programming Historian, September2012.

• Posner, Miriam. 2012. “Very Basic Strategies for Interpreting Results from the Topic Modeling Tool.” Miriam Posner’s Blog, October 29, 2012.

• Řehůřek, Radim. “models.wrappers.ldamallet – Latent Dirichlet Allocation via Mallet." Gensim: Topic Modelling for Humans.

• Underwood, Ted. 2012."Topic Modeling Made Just Simple Enough."The Stone and the Shell,April 7, 2012.

• Veas, Edurardo, and Cecilia di Sciascio. 2015.“Interactive Topic Analysis with Visual Analytics and Recommender Systems.” Association for the Advancement of Artificial Intelligence.

• Silge, Juliaand David Robinson."Topic Modeling."InText Mining with R:ATidy Approach. Beijing; Boston: O’Reilly Media, April 6, 2020.

Example Projects

• Blevins, Cameron. 2010.“Topic Modeling Martha Ballard’s Diary.” Cameron Blevins.April 1, 2010.

• Nelson, Robert K. n.d.Mining the Dispatch.

• Goldstone, Andrew, and Ted Underwood. 2014. “Metadata and Modeling Outputs for Study ‘The Quiet Transformations of Literary Studies’Data.”

  • Goldstone, Andrew, and Ted Underwood. 2014. “The Quiet Transformations of Literary Studies: What Thirteen Thousand Scholars Could Tell Us.” New Literary History 45, no. 3: 359–84.