What is Topic Modeling?
Topic modeling is an unsupervised machine learning technique. It involves scanning documents, identifying word and phrase patterns, and clustering word groups and similar expressions to discover the topic or set of topics that describe the document in the most appropriate manner. Essentially, topic modeling aids you in identifying topics that are the most effective at describing a document or a set of documents.
It helps you cut through the clutter and identify the signal (main topics) of your document. It is essentially a kind of statistical modeling that is used to discover the topics that occur in a collection of documents.
Topic modeling uses quantitative algorithms to detect the key topics that your body of text revolves around. You could, as an example, use topic modeling for the purpose of discovering the topics that are in a set of customer reviews.
As mentioned earlier, Topic modeling is an unsupervised machine learning technique. This basically means that this machine learning technique does not require training. It does not need a predefined list of tags or training data that’s been classified by humans in advance. On the other hand, topic classification is a supervised machine learning technique. It requires training before it is able to automatically analyze texts.
Due to the fact that topic modeling is an unsupervised machine learning technique and does not require training, you won’t have to wait too long or make too much effort before you can start analyzing your data. Topic modeling basically offers you a rather fast and easy way for you to get started on analyzing your data, but there’s a catch involved with it. When you use topic modeling, you sure can get started quicker and get the job done faster, but there is no way of being sure that the results that you get from the topic modeling task will be accurate at all.
Topic classification models need to know the topics of a set of texts before they analyze them. With these topics, the data is tagged manually, which makes it possible for a topic classification model to learn and later make predictions by itself.
Because of the fact that accuracy of the results is not guaranteed with the use of topic modelling, a very large number of businesses choose to invest their time into training a topic classification model.
Topic modeling has its origins in latent semantic indexing (LSI). But since LSI does is not a probabilistic model, it is not an authentic topic model. Probabilistic latent semantic analysis (PLSA), however, was proposed by Hofmann in 2001 and is an authentic topic model. It was based on latent semantic indexing.
Why is Topic Modeling important?
Topic modeling empowers us to organize, understand and summarize vast collections of text data. It helps us discover topical patterns that are hidden across the document, annotate those documents according to the topics discovered, and organize, search and summarize by using those annotations.
You could consider topic modeling to be a form of text mining.
What are the two widely used Topic Modeling techniques?
You can use a range of techniques to perform topic modeling tasks. Here are two commonly used techniques:
1. Latent Dirichlet Allocation (LDA)
Latent Dirichlet Allocation is an extremely popular technique for topic modeling. It considers every document to be a mixture of topics that are present in the corpus. It says that every word in the document can be attributed to one of the main topics of the document. Basically, Latent Dirichlet Allocation (LDA) builds a topic per document model and words per topic model, modeled as Dirichlet distributions.
The Latent Dirichlet Allocation (LDA) model identifies the various topics present in the document and even shows to what extent the document deals with a particular topic.
One way for LDA models to identify the topics and topic representations of every document is to use collapsed Gibbs sampling.
It first scans through documents and assigns each word randomly to one of K topics (with K being predetermined). Now there are topic representations for all the documents, along with word distributions for all the topics, but these are not accurate.
To get better results, we have to go through every word w in every document d. Now we need to find:
- p(topic t | document d) - the proportion of words in the document d that are assigned to topic t.
- p(word w| topic t) - the proportion of assignments to topic t in all documents d that come from word w
Now we reassign a new topic t’ to word w. To do this, we need to choose topic t’ with probability
p(topic t’ | document d) * p(word w | topic t’)
Doing this, the model can predict the probability that word w was generated by topic t’.
After performing the last step multiple times, the topic assignments will be fairly accurate. Then the topic mixtures in each document can be found using the topic assignments.
2. TextRank
TextRank is based on Google’s PageRank algorithm. It is extensively used for extractive text summarization.
First, the goes through the documents and extracts the text from them. After that, it divides the text into sentences and converts the sentences into vectors. It then finds the vector representations for every sentence. It could do this by averaging word vectors.
After that, it calculates the similarities between sentence vectors and stores them in a similarity matrix, which is then converted into a graph. The graph is used to calculate sentence ranks and the highest ranked sentences form the final summary.
What are Topic Modeling toolkits?
There are many toolkits available for the application of topic models. They are predominantly used in Natural Language Processing (NLP).
Here are three popular topic modeling toolkits:
- Gensim
- Stanford topic modeling toolbox (TMT)
- MALLET
