Using sentiment analysis & machine learning for security price forecasting

Abstract

We worked with sentiment analysis and supervised machine learning to forecast the security movements in stock market and benefit from it. Text rich data sources like newspapers, blogs, stock market related internet forums, social networking websites contain relevant and updated information about the publicly listed companies. Sentiment analysis can help us to extract usable information from these texts to understand the overall sentiment of the articles. In our research, we used two sentiment analyzed database provided by Accern [1] & Sentdex [19] and tried to see how positive is the relation between the market sentiment and market movement of S&P 100 index listed companies. We also implemented machine learning agent trained on the price data to find a comparable result. With our implementation we have been able to consistently perform better than the benchmark with low beta and sharpe which suggests that algorithms based on state-of-theart sentiment analyzed data can follow the market movement stably. We have also seen that machine learning agent trained on the price data can move with the market given a higher initial investment.