Continuous-Time Infinite Dynamic Topic Models: The Dim Sum Process for Simultaneous Topic Enumeration and Formation

Abstract

Topic models are probabilistic models for discovering topical themes in collections of documents. These models provide us with the means of organizing what would otherwise be unstructured collections. The first wave of topic models developed was able to discover the prevailing topics in a big collection of documents spanning a period of time. These time-invariant models were not capable of modeling 1) the time varying number of topics they discover and 2) the time changing structure of these topics. Few models were developed to address these two deficiencies. The online-hierarchical Dirichlet process models the documents with a time varying number of topics, and the continuous-time dynamic topic model evolves topic structure in continuous-time. In this chapter, the authors present the continuous-time infinite dynamic topic model that combines the advantages of these two models. It is a probabilistic topic model that changes the number of topics and topic structure over continuous-time.