What is Twitter telling us?
140 characters is not a lot. 2,800,000,000 characters is a different story. In my recent research with Ryne Sherman, we analyzed 20,000,000 Tweets (i.e. 2,800,000,000 characters) to discover what people all over the U.S. experience in their everyday lives.
Frustrated with small scale studies using undergraduate samples, we knew their had to be a better way. Researchers were starting to uncover the power of digital foot prints, and these studies were ground breaking. However, most the studies at this time were looking at what digital footprints could tell us about the individual. We wanted to know what this can tell us about people in general: What do people do? When are they the happiest? Saddest? What is life like in the city compared to the country?
People have general ideas about the answers to these questions, but most these ideas basically amount to intuitive guesses. No one has empirically answered this question. The key insight for this study came when we realized that people share millions of experiences on Twitter. We realized we could use Twitter to collect the largest collection of experiences that anyone has analyzed, using a comprehensive measure of psychological experiences.
What did we find?
The situations that people share are, on average, more positive than negative. That was a pleasant finding. Further, people are happier on the weekends than during the week. No surprise there! People who are Tweeting in the late night night hours are usually not having a good time. (Ryne likes to say “Nothing good happens after midnight.”) Also, we compared Urban to Rural areas. It turns out there are very few psychological differences between life in the city and life in the country.
Finally, we found gender differences. Females experience both more Negativity and more pOsitivity than males (Fig 1.). Here you can see that Females experienced more Negativity (red) than males (orange), and more pOsitivity (green) than males (blue).
What does this mean?
Aside from being one of the most descriptive studies to date exploring what people actually experience in their daily lives, we also developed a new method for assessing textual data sources. By using machine learning to approximate human judgments we assessed a corpus that would be completely inaccessible using traditional methods. By our calculations, it would have taken about 500,000 hours to rate these 20 million Tweets by hand. We were able to assess four thousand times as many experiences using this method than we were able to using traditional human coders. This study demonstrates the application of Big Data analytics to an obviously important psychological question, highlighting the potential of these methods for researchers in any number of domains.