Abstract
This chapter provides a basic understanding of processes and models needed to investigate the data posted by users on social networking sites like Facebook, Twitter, Instagram, etc. Often the databases of social networking sites are large and can't be handled using traditional methodology for analysis. Moreover, the data is posted in such a random manner that can't be used directly for the analysis purpose; therefore, a considerable preprocessing is needed to use that data and generate important results that can help in decision making for various areas like sentiment analysis, customer feedback, customer reviews for brand and product, prevention management, risk management, etc. Therefore, this chapter is discussing various aspects of text and its structure, various machine learning algorithms and their types, why machine learning is better for text analysis, the process of text analysis with the help of examples, issues associated with text analysis, and major application areas of text analysis.
TopStructure Of Text
According to the situation and usage the text can have any kind of organization (format) like the chapter in a book, article in a newspaper, a blog, tweet etc. everywhere text is organized in an inherited way. Moreover with the advancement of World Wide Web (WWW) many users are getting associated with social media,(Manyika et al., 2011) reading e-newspapers, e-books, shop online, filling e-forms and posting their views, feelings, emotions, thinking and expression in the languages they are familiar most probably their mother tongue, English, Hindi, Telgu, Kannad, Spanish, German etc. further they may use multiple languages in a single sentence to make the sentence more impactful which give rises to code switching. The properties of these categories are summarized in a Table 1 below:
Key Terms in this Chapter
Semi-Supervised Machine Learning Algorithms: In this category either the model is developed in such a way that either there are labels exist for all kind of observations or there is no label exist.
Supervised Machine Learning Algorithms: Supervised algorithms mean that a system is developed or modeled on predetermined set of sample data.
N-Gram: Making group of ‘n’ words from a sequence to convey some meaningful things.
Skip-Gram: Inverse process of Word2Vec, here based on targeted words, prediction of context word is done.
Term Frequency/Inverse Document Frequency (TF/IDF): To identify the occurrence of a word in document and finding the most probable word in the text.
Facebook: An online social networking site.
Instagram: A video and photo sharing social site owned by Facebook.
Stemming: Stemming is the process of reducing word to stem. By removing unnecessary inflammation this is done by removing suffix.
Text Analysis: Text analysis is used to parse the unstructured data into machine readable form.
Word2Vec: One method of word embedment is word2Vec where similar words have same vector representation.
Predictive Analysis: To predict the future based on historical data.
Semi-Structured Data: Data that have some organizational property, but not having some row and column relationship.
Continuous Bag of Words: A process of taking words as a input and predict the next word in the sequence.
Natural Language Processing (NLP): Natural language processing is the ability of computer program to understand human language as it is spoken or handwritten.
Bag of Word (BOW): To map the words into a fixed length vector according to the predefined dictionary.
Unsupervised Machine Learning Algorithms: Unsupervised algorithms mean that a program is provided with some collection of data, with no predetermined dataset being available.
Structured Data: Data is organized properly into a formatted repository.
Reinforcement Machine Learning Algorithm: This model gathered information based on the interaction of system with the environment to take actions that would maximize the rewards and in turn minimizes the risk.
Part of Speech: Used to identify the theme of sentence.
Data Cleaning: A sub-process in data preprocessing, where we remove punctuation, stop words, etc. from the text.
Sequential Analysis: Based on the pattern of historical data, to identify what will be the next pattern in the sequence.
Social Media (SM): The interactive computer-based technologies that facilitate the creation and sharing of information, ideas, feelings, etc.
Twitter: An online news and social networking service on which user can share message called tweets.
Machine Learning: Machine learning is branch of data science which has concern with the design and development of algorithm to develop a system that can learn from data, identify the complex patterns and provide intelligent, reliable, repeatable decisions and results with minimal human interaction based on the provided input.
Feature Extraction: A process of finding features of words and map them to vector space.
Lemmatization: A process to reduce words into its root, generally by reducing second form and third form of verb to first form, etc.
Word Embedment: A process to find context of words, here similar words have same representation.
Deep Learning: An extension of machine learning approach, which uses neural network.
Data Preprocessing: A process for making data ready for analysis purpose by eliminating unwanted things from data.
Unstructured Data: Data structure or organization can’t be predicted.
GloVe: The extension of word2Vec method for efficiently learning the relationship between input and the target words.
Tokenization: A process of converting a sentence into small identifiable units.
Business Intelligence: A technological driven process for analyzing data and presenting information, in such a way that user can take immediate actions and unable decision making.
Hadoop: A framework that allow for the distributed processing for large datasets.
Data Collection: A process of storing and managing data.