Structural Topic Model (STM)
STM discovers latent topics using probabilistic modeling while incorporating document metadata as covariates. It models how topics vary across documents based on metadata like time, author, or category.
Best For
- Metadata analysis: Relate topics to document characteristics
- Covariate effects: Test how metadata affects topics
- Interpretability: Clear word-probability distributions
Usage
Set topic count, add covariates, and evaluate model quality with metrics.
Quality Metrics
Choosing K: Prefer models in the upper-right of the semantic-coherence vs. exclusivity scatter (high on both).
- Semantic Coherence: Measures word co-occurrence within topics (Mimno et al., 2011)
- Exclusivity: Measures how unique words are to each topic
- NPMI: Normalized Pointwise Mutual Information on top terms; alternative coherence axis less sensitive to rare words.
- Topic Diversity: Proportion of unique words across topics' top-K terms (higher = less topic overlap).
- Held-out Likelihood: Predictive performance on unseen documents
- Residuals: Model fit diagnostic
References
- Roberts, M. E., Stewart, B. M., & Tingley, D. (2019). stm: An R package for structural topic models. Journal of Statistical Software, 91(2), 1-40.
- Roberts, M. E., Stewart, B. M., Tingley, D., et al. (2014). Structural topic models for open-ended survey responses. American Journal of Political Science, 58(4), 1064-1082.
- Mimno, D., Wallach, H., Talley, E., Leenders, M., & McCallum, A. (2011). Optimizing semantic coherence in topic models. EMNLP 2011.
- Hoyle, A., Goel, P., Hian-Cheong, A., Peskov, D., Boyd-Graber, J., & Resnik, P. (2021). Is automated topic model evaluation broken? The incoherence of coherence. NeurIPS 2021. arXiv:2107.02173
- Dieng, A. B., Ruiz, F. J. R., & Blei, D. M. (2020). Topic modeling in embedding spaces. Transactions of the Association for Computational Linguistics, 8, 439–453. https://aclanthology.org/2020.tacl-1.29/