Sentiment Monitoring of Social Media from Oceania

In our work, we have been investigating whether we can process emotion-related information from social media in real time, to understand how people react to different events and circumstances and potentially also help further research in mental health. To this end, we developed We Feel, a tool that analyses emotions on Twitter and presents them through an interactive visualization [1]. We Feel constantly monitors the Twitter stream, looking for tweets (in English) containing any emotional content [2,3]. The platform aims at monitoring the regional elevated risks of suicide by assessing the mood of people in that region. Figure 1 shows a screen shot of We Feel. The set of emotions that are captured is shown on the left, displayed as an “emotion wheel”. A map of the world is on the right. Both of these elements are interactive: one can select a region in the world, or a specific emotion, and the visualisation in the centre will focus on the chosen attributes (location or emotion) and change accordingly. In Figure 1, a specific date (May 21-27, the week of the Manchester attack), region (Oceania) and emotion (sadness) have been chosen. The visualisation shows the emotions as reflected in the tweets being processed, colour-coded by emotions, matching the wheel.


Introduction
In our work, we have been investigating whether we can process emotion-related information from social media in real time, to understand how people react to different events and circumstances and potentially also help further research in mental health. To this end, we developed We Feel, a tool that analyses emotions on Twitter and presents them through an interactive visualization [1]. We Feel constantly monitors the Twitter stream, looking for tweets (in English) containing any emotional content [2,3]. The platform aims at monitoring the regional elevated risks of suicide by assessing the mood of people in that region. Figure 1 shows a screen shot of We Feel. The set of emotions that are captured is shown on the left, displayed as an "emotion wheel". A map of the world is on the right. Both of these elements are interactive: one can select a region in the world, or a specific emotion, and the visualisation in the centre will focus on the chosen attributes (location or emotion) and change accordingly. In Figure 1, a specific date (May 21-27, the week of the Manchester attack), region (Oceania) and emotion (sadness) have been chosen. The visualisation shows the emotions as reflected in the tweets being processed, colour-coded by emotions, matching the wheel.
In this paper we use We Feel to explore the mood of the people in Oceania (Australia and New Zealand) over the period running from1 June 2014 to the end of November 2016 ( Figure 1). This paper uses statistical process control to flag significant changes in the mood of a region and understand its implication on the society in that region. We are interested in what events influenced the mood. An event may dominate conversation and so the number of people that talk about an event increases significantly after the event and then subsides as people either loose interest or all the issues of the event are covered. The monitoring technology in this paper is more interested in isolating the dominate sentiment for an event. An event is determined by the significant increase of the number of tweets. The dominate sentiment for an event is found by monitoring the proportion of tweets with sentiments either classified as expressing anger, fear, surprise, sadness, joy or love. The final aim is to understand when people respond to events, why they respond with certain sentiments and how quickly does the event stop influencing the mood of people, or in other words how quickly do people move on with their lives after an event.

Event detection
We start by detecting an event. An event is defined as an unusual increase in the number of tweets per hour. To start we firstly need to define what is usual before we can establish what is usual. This will be done firstly respectively. Firstly we used the total tweets per hour ( Figure 2) as a response variable with explanatory variables lag logarithm hourly counts, time, harmonics to model both seasonal trends and within hour trends, and day-of-the week influences. Public holidays are ignored because the region does not have consistent public holidays. We assumed that the harmonic for season and day interacted. This model fitted quite well with the Pearson residuals showing no significant autocorrelation. The EWMA chart applied to the Pearson residuals of this model looked very strange with it mostly hugging the centreline and with no high-sided signal. Further investigation revealed that the lag 1 autocorrelation in the hourly counts was not very high at 0.54, and the coefficient for the logarithm lag counts in the fitted model was 0.308. This autocorrelation was driven by the events where counts ramped up. However while they communicated with friends between events there was no apparent autocorrelation until they the next event. For this reason we decided to fit the above model without the explanatory variable lag logarithm hourly counts included, and used this model to define usual behaviour. This meant that we would Cos (2*pi*t/365.25): Sin (2*pi*h/24) 4.421e-01 9.518e-02 4.645 3.41e-06 *** Sin (2*pi*t/365.25): Cos (2*pi*h/24) 1.502e-01 3.481e-02 4.316 1.59e-05 *** Sin (2*pi*t/365.25): Sin (2*pi*h/24) 9.408e-01 1.690e-01 5.566 2.60e-08 *** Significance code: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.  Figure 3 provides the a qq-plot for the model indicating a reasonable fir to the negative binomial model except for outliers in the tail, which on the high-side would correspond to events of interest to twitters that cause the unusually high number of hourly tweets while this event holds the twitters attention.
These total hourly tweets appear to be over dispersed with a number of low and high sided outliers. We are interested in detected the high sided outliers which we try to associate with a historical event that we believe created the significantly elevated interest amongst twitters. To achieve this we apply the EWMA chart to the Pearson residuals for the model above. Firstly we establish expected total hourly tweets by fitting a negative binomial regression model defined above. Estimate the Pearson residuals for this model and then apply the EWMA chart with exponential weights given by 0.4 because most events seem to wane very quickly in the social media context, and most of the events we are looking at are fairly large shifts. We believe that this is appropriate because most twitters attention span is fairly short less than an hour.
We applied an EWMA control chart to the Pearson residuals to flag the unusual events of the study period using a retrospective surveillance approach. The in-control Average Run Length (ARL) for this EWMA was taken as 365 in designing the plan. The threshold was found by simulation but we could have used the spc package in R [4] to provide a live with a slightly higher over-dispersion in the model than is justified, because we have included all events in the model without accounting for their autocorrelation, but we were happy to live with that and only focus on the major events ( Figure 2).
In such cases it is really difficult to define what usual behaviour is because there is no natural in-control situation. In this paper we define in-control behaviour as the predicted values using the negative binomial regression model below; glm.nb (formula=Total ~ time+WD+(cos(2*pi*t/365.25)+sin (2*pi*t/365.25))*(cos(2*pi*h/24)+sin(2*pi*h/24)), data=tem, init. theta=62.12038824, link=log).  very similar threshold. Since we are dealing with hourly data this gives us roughly 24 false alarms on average per year. Figure 3 provides the results of this chart by signalling unusual events if exceed the upper dashed red line. We will ignore the low sided signals in Figure 4.

Understanding the twitters sentiment for events
Each twitters' message is classified as having one (or more) of the following sentiments: anger, fear, joy, love, sadness or surprise. There are two senarios we are interested in the first whether tere is a change in sentiment independent of whether there is an event of not. The second is to explore the sentiments for the events discovered in the previous section. The first scenario is now explored in this section and the second demands a multivariate approach which will be explored in the next section. Here we are interested whether the sentiments change significantly over time independently. To carry this out we fit the following model using fear as an example but the modelling is identical for all other sentiments.
glm.nb(formula=fear~lag.fear+time+(cos(2*pi*t/365.25)+sin(2*pi *t/365.25))*(cos(2*pi*h/24)+sin(2*pi*hr/24))+WD+offset(log(Total)), data=tem, init.theta=194.9292045, Surprise: Now we explore tweets that express a higher than expected proportion of tweets with sentiment surprise (Figure 7). The first surprise I am guessing is with the protests at the G20 summit in Brisbane. The second is when 2 of the Bali 9 drug smugglers into Indonesia where executed by firing squad. The third was Johnny Depp illegally smuggling his dogs into Australia from the USA. The forth is Penrith teenager caught with a gun in school in a western suburb of Sydney. The fifth was Russia starting to attack ISIL in Syria. The sixth is the climate pact agreement which seem to last a long-time when most other events seem to dissipate quite quickly. The last is massive shift from low surprise to massive surprise on the BREXIT election outcome. Figure 8 signals the unusual proportion of tweets that express sadness. The first is the shooting of Michael Brown; the second is the Martin place siege although it is not signalled as unusual; the third is the Germanwings plane crash into the Alps; and the last Multiple attacks by ISIL.

Sadness:
Love: Figure 9 can be a little confusing because I can see why twitters respond in love to the Duchess of Cambridge's birth of a child, on Christmas day with family and with a tribute to Malcolm Fraser, but the other two are less clear unless sarcasm is being used. Joy: Figure 10

Multivariate Views of the Sentiment Analysis
In order to understand the mood of Australian during the study period we need a multivariate view of the sentiment monitoring process. The first multivariate view of the sentiment counts is achieved using parallel coordinate plots and an example is displayed in Figure  11. Figure 11 displays the full list of sentiment counts for 6 days jointly using a parallel coordinate plot. This allows us to jointly view trends for all sentiment counts in a single plot, displaying trend information for all sentiment counts relative to their expected values. The lines go from black being the most recent date 14 November 2016, followed by red, green, blue, light blue, magenta and yellow (9 November 2016). The confidence bounds are the thresholds for the EWMA statistic for the sentiment scores. This plot helps us identify that there is a rough trend towards greater volumes of anger, fear and sadness while a reduction in joy and love. Note that love started in the unusually high number of counts region. This plot is easy to interpret and help interpret the full picture of the sentiment scores but does not make the best use of the relationships between the variables (Figure 11). To do this we propose using the dynamic biplot of Sparks et al. [5].
The dynamic biplot of Sparks et al. [5] monitors changes in location of the counts as well as changes in correlation between the tweet counts and changes in dispersion of the counts in a single plot making it quite useful in interpreting the twitters responses to certain events. For example, Figure 11 describes the response to the shooting down of flight MH17 over Ukraine. Note that Figure 11 describes 85% of the variation in two dimensional display; 58% in the first dimension and 27% in the second dimension ( Figure 12).
How to interpret these plots is well covered in Sparks et al. [6] Figure  13 indicates a significant increase in anger and fear (note that many expressing anger and fear at the same time because these are close to being collinear). While other twitters had significant increased sadness ( Figure 12). There was also a simultaneous reduction in the expression of joy mostly for those that expressed sadness. The correlation between these sentiments counts have changed significantly by the colours in the matrix below the variable plot. Figure 13 indicates the initial response to Phillip Hughes's death by being hit on the head by a cricket ball was a mixture of sadness and anger but later as people like Michael Clarke (the then Australian cricket captian) expressed his mate ship for Phillip Hughes and then this changed to the dominate response being love for the man who had so tragically had his life taken away from him. Note that there was no change in correlation structure by the matrix of boxes below the variable plot not being coloured. At Hughes death the dominate response was anger for a while but then is soon dissipated.   Figure 14 indicates that the dominate response to Rosie Batty was made Australian of the year was one of surprise and all other sentiments were orthogonal to this indicating that no other sentiment increased. This is fascinating but it is unclear whether people were surprised about Tony Abbott Australian then Prime Minister making such a call or whether they were surprised by that the choice of Rosie. This choice did raise the serious issue of domestic violence within Australia and Rosie was the perfect ambassador fighting against domestic violence seeing she had experienced it first-hand. Note that there was no change in correlation structure by the matrix of boxes below the variable plot not being coloured.
In Figure 15, the dominant response to the energy debate after the South Australia energy crises was one of increased sadness and no other sentiment increased. The issue was one where severe weather cut the supply of energy to the entire state and this started a national debate about the state relying too much on renewable energy sources. The interesting feature of this response was that there was no increase in angry tweets because of the state government's decision on the percentage of renewable energy that is to be used. I think this means that the South Australian residents don't strongly disagree with state energy policy. Note that there was a change in correlation structure love and joy became less correlated. The other colours indicate warnings.
In Figure 16, the dominant, but weak response to the news that Australian Air force Jets were starting to operate in Syria for the first time was initially one of anger but this did not last long; no more than a few hours before the response was an increase in joy was dominant and remain so for more than the next 24 hours.   This increase in joy was not massively significant because the mean square error for joy did not flag as significant because the joy line in the vector plot was not coloured red but the sausage shape in the middle of this vector indicate a significant increase in joyful responses. Note that there was a change in correlation structure love and joy became less correlated and love and anger became more positively correlated as these counts both decreased simultaneously.     The twitter response to arrest of Gino & Mark Stocco (Father and son) after being on the run for 8 years was a strong response of sadness and this is mostly driven by two hours of the day at about 6 and 7 pm at night when the arrest was probably reported. This does not make a whole lot of sense but there was a non-significant reduction in the surprise, love and joy tweets which make more sense when harden criminals are arrested. Potentially this was a case of things going wrong for two Aussie battlers.
In Figure 17, the response to phone data retention laws for internet service providers in Australia was one of increased sadness and reduced joy but the observation plot does not flag a multivariate shift in location and so this response is not very strong. There is no change in correlations.

Conclusion
We have demonstrated ways of monitoring tweet sentiment scores for a region as a way of understanding how the region responds to events. These first defined events as those period where the number of tweets for the region significantly increased. We then monitored how unusual the counts of these tweets were after correcting for the volume of tweets. This was achieved for each sentiment independently, however these sentiment counts are correlated and monitoring them independently makes interpreting the response to events quite difficult. The parallel coordinate plots are relatively easy to understand and display trends in a reasonable way but ignore correlations. Therefore we prefer the dynamic biplot which monitors changes in location, dispersion and correlations simultaneously in one plot. It is also efficient at displaying trends in the observation plot. Although its interpretation is complex we believe the rich information it presents makes it a reasonable tool for monitoring and understanding events.