Detailed NLP results

High Level Data Summary

To begin our analysis, we conducted some high level data exploration to gain an understanding of dataset statistics more relevant to working with NLP than in our EDA segment of the work.

Figure 1

We can see some disparity between environmental and non-environmental posts simply by looking at the posts statistics: posts flagged as “environmental” based upon our set of keyword search terms are typically longer than those without the relevant keywords.This pattern holds true across all the subreddits we are considering in this analysis, as can be seen in Figure 2.

Figure 2

When examining the change of comment lengths over time by both the subreddit and whether or not the comment is environment related, there is a less discernible pattern. There does not appear to have been a significant change over the time period in question regarding general comments, but there’s some volatility across months. Looking at the Environmental space, we’re seeing an overall trend of increase in comment length in r/climateskeptics, but a decrease in the other two spaces.

Figure 3

We then looked into the words themselves to get a better grasp of the topics being discussed. For this, we used word clouds to visualize the most frequently occurring words, as shown in Figure 4.

Figure 4

While mostly mundane words like ‘people’ and ‘comment’ dominate, there are some interesting words used heavily in the discussions. Most notably, vaccines were discussed heavily, using different variations of the word.

We then analyzed frequently used words across the three subreddits, as shown in Figure 5.

Figure 5

There are some interesting overlaps between the r/climatechange and r/climateskeptic subreddits. Global warming, greenhouse gases, emissions and temperatures (specifically the word warm) are discussed heavily. Similarly, the words ‘research’ and ‘scientist’ seem to be mentioned frequently in both subreddits as well.

The word cloud for the r/conspiracy subreddit is a bit less interesting, though vaccines are discussed heavily. More generic words like ‘means’, ‘think’, ‘should’ ‘government’ and ‘media’ appear frequently. While they do not reveal much on their own, when we understand the context, we can conclude that the users are speculating about conspiracies related to vaccines, government and media more than anything.

Goal V.

Understand the overall sentiment of posts, and how the tone of posts relate to controversiality and environmental themes.

Results:

To deepen our understanding of these comments, we also ran a sentiment analysis to see if general and environmental posts differ. Interestingly, we’ve found that environmental posts tended to be more positive than general posts, while controversial posts tended to have more negativity in both categories. Across both controversial and uncontroversial posts, comments pertaining to environmental terms tend to have be positive at a higher rate than general comments. Figure 6 shows this breakdown. One reason for this difference might be either people trying to be hopeful, or people denying climate change and reaffirming false positivities.

Figure 6

We then wanted to investigate how the sentiments differ across the three subreddits, for environmental posts. Figures 7.1-3 show this breakdown, in the respective subreddit. The interactive chart can be used to zoom in on any subsection.

Figure 7

Overall, there are far more non-controversial posts, than controversial. An interesting insight is that within controversial posts, there seems to be a balance between positive and negative posts, with a slight tilt towards positivity. This is surprising because we would have expected negative emotion dominating.

However, a counter argument could be that there are posts that argue that the world is perfectly fine (i.e. positive sentiment) that are perceived as controversial. At the same time, there are probably many negative posts that also being flagged as controversial (e.g., derisive to others’ opinions). Ironic posts might also be picked up as positive.

Goal VI.

Investigate trends over time to see if any particular claim is gaining in popularity. Relate temporal changes in prevalent claims or subclaims to identified changes in controversy.

Results:

Building upon the sentiment analysis, we wanted to understand how the sentiment of comments have been chaning over time across the different subreddits, in both the general and the environmental space, and how controversiality relate to all of these. Figure 8 shows the average sentiment across the investigated time period for all subreddits. We calculated the average sentiment by weighting negative sentiment as -1, positive as 1, and neutral as 0, multiplying these with the number of comments, and then dividing with the total number of comments for that given time period / area. We’ve found two key things. One, in most subreddits across all time period, non-controversial comment sentiment was relatively stable, while controversial can differ quite drastically month-by-month. But, when looking at the r/climatechange and r/climatesceptic spaces, we see that controversial and non-controversial posts have a relatively similar sentiment - granted, controversial sentiment is unstable with no clear pattern, but it generally speaking is in line with ‘normal’ comments.

On the other hand, for the r/conspiracy space, we can see different trends. For one, in the general space, controversial comments are always more negative. Two, in the environmental space, regardless of controversy, the sentiment of comments have been more volatile than in other spaces. In fact, we can see a negative trend in sentiment around the time when we’ve also perceived an overall decrease in the percentage of controversial comments in the r/conspiracy space. And, at the same time as the percentage of controversial environmental (and general, though not by the same rate) comments started to increase, their sentiment also started to increase - indicating that in recent times, there is not only an increased number of controversial environmental posts, but their sentiment also tends to be somewhat more positive than it was earlier, though there’s a lot of volatility in this. For reference, we included key environmental policy events on this timeline as well (Build back better debate, Supreme Court debates and rules on EPA regulatory power, Inflation Reduction Act passes).

Figure 8

Goal VII.

Understand the role of external information in climate misinformation: how prevalent it is to include external sources, and how these sources influence controversy or sentiment

Results

Finally, we wanted to understand how sentiment and climate misinformation are connected, and we investigated this in the environmental space. When looking at the split between controversial and non-controversial comments, we’ve found that posts that contain climate misinformation are generally more positive, and this difference is even greater if the post is controversial. Also, controversial posts tend to be much more likely to be neutral if they contain climate misinformation. This suggest that these misinformation materials spread through using positive or neutral (potentially academic-like) voice.

Figure 9