Article Preview
Top1. Introduction
Sentiment Analysis is the computational study of public opinions, emotions, sentiments, attitudes, and appraisals towards entities. The entity can be an issue, events, services, individuals, products, etc. Everyday more reviews are posted on the internet due to advancements in technology. The objective of Sentiment Analysis is to conclude the polarity of the review as positive (good) or negative (bad). So, decision making plays an important role in Sentiment Analysis.
Decision making process helps the customer to get the right product and helps the organization to sell the right product to the customer. It benefits the consumer and corporates based on the decision. Applying a machine learning algorithms to the decision making process on the opinion is a challenging task. The goal behind this is to apply deep learning algorithms to build models that allow automatic extraction of features from the text. When Deep Learning Methods (DLM) is used in feature engineering, automatically the high-level features are learned without any human bias. In deep learning, the features are learned during the training process and no specialized domain knowledge is required by the researchers.
Convolutional Neural Network (CNN) (Yoav Goldberg, 2015) has a local pattern of connection between the neurons of adjacent layers. This connection helps to maintain a special spatially local correlation. This characteristic is helpful in the classification of the sentences in NLP. It finds strong local clues that appear in the different places of inputs regardless of input class membership. The local indicators are nothing but the key phrases that helps to identify the sentiment of a sentence. Consider the movie review from the IMDB dataset
A convolutional layer extracts the local features from the movie review includes “actor”, “resolve”, “social”, “issue” etc. Self-Attention distinguishes relevant and un-relevant parts of a movie review based on Parts of Speech (POS). It correlates the distinct parts of a longer sequence to compute the weight of a part of a sequence. In the above review, “actor”, “resolve”, “social”, “issue” holds noun, verb, adjective, noun tag of POS. These tags are assigned higher weights by the Self-Attention layer and improve (Sindoori et al., 2017) the prediction score of the movie review. GRU (Chen Tao et al., 2017) is divergent of the recurrent network that doesn’t have internal memory and has only two gates when compared with LSTM Sepp Hochreiter et al., 1997). The internal design of GRU is simple and takes less training time than LSTM. Bidirectional GRU scans the review in the forward and reverse direction (Li Zhang et al., 2017 and Jianqiao Hu et al., 2017). It has higher learning power to better understand contextual information. It relates the features that are located in distinct parts of the sentence.
The term “movie” is located on the left side of the sentence, while “social issue” is located on the right side. BGRU understand contextually and correlates the term or phrases located on two extreme ends of the movie review. We have conducted four experiments on the IMDB dataset viz., Unidirectional Neural Network (CNN, LSTM, GRU), Bidirectional Neural Network (BLSTM (Yu Zhao et al., 2017), BGRU), Hybrid Neural Network (CNN+LSTM, CNN+BGRU) and Attention Based Neural Network. The proposed attention based architecture is compared with baseline architecture. It obtained better results than other architecture. The contribution of the research work is listed below:
- •
We have designed a new architecture by integrating the attention layer with a hybrid convolution bidirectional recurrent neural network (ACBRNN).
- •
The proposed architecture extracts more relevant terms and assigns high weights to those terms based on a context that influences the polarity of the review.
- •
We have highlighted the importance of different layers in the proposed architecture with a movie review.