| Controversial: No | Controversial: Yes | |
|---|---|---|
| Environmental: No | 54679 | 1258 |
| Environmental: Yes | 26655 | 783 |
Detailed EDA results
Goal I.
Determine the basic characteristics of the climate-related reddit spaces, and understand the prevalence of controversial and climate-related controversial posts across the different spaces.
Results:
First of all, we wanted to understand the basic characteristics of the three investigated spaces: r/conspiracy, r/climateskeptic, and r/climatechange. Table 1 provides a summary of all posts, submissions and comments per subreddit. Overall, we can see that r/conspiracy submissions engage the community the most: the share of comments were the largest here. r/climateskeptic is also more engaging for the community than the r/climatechange submissions. This seems to corroborate the narrative that negative, sensational, controversial topics hold our attention better.
In this initial phase, we also wanted to make sure we clean the data from erroneous records: we filtered out empty comments, and comments with a missing timestamp. Table 1 also contains the final number of comments we’re using for further analysis.
Summary statistics
Table 1
Next, we wanted to understand the prevalence of both environmental topics and controversy in comments. Since we assume misinformation is mostly spreading in comments, most of the EDA focuses on comments, apart from the section on authors, where submissions are also included in our research. Controversial comments were identified based on the data’s own controversial feature: if a comment receives a similar amount of upvotes and downvotes, it’s identified as controversial. Regarding environmental comments, we’ve experimented with a couple of selections and finally aligned on a list of words, and if any of these were in the text of the comment, we’ve identified it as environmental. Examples are ‘environment’, ‘renewable’, ‘solar’, or ‘global warming’. We’re fully aware that some of these comments aren’t necessarily what we’re looking for, and this simple search might miss comments dealing with the topic without mentioning these specific words - a more nuanced NLP method will be implemented later to more accurately identify these comments, and to identify the ones spreading climate misinformation.
Table 2 contains the results of these two analyses. As can be expected, controversial comments are mostly present in the r/conspiracy subreddit. A large proportion of the r/climatechange and r/climateskeptic comments were identified as environment-related. While only about 1% of the r/conspiracy subreddit seems to be related to environmental topics, it’s still almost 200 thousand records, which gives a good basis for further analysis.
Presence of controversial and environmental comments
Table 2
Finally, we wanted to understand the explicit relationship between these two features: more precisely, if a comment is related to environmental topics, is it more likely to be also controversial? To analyze this, we conducted three chi-square tests on the respective subreddits. The related contingency tables can be seen in Table 3, Table 4 and Table 5. For all three subreddits, we found a significant relationship (99% confidence level) between controversial posts and environmental topics. In other words, if a comment is related to the environment, there’s a higher proportion of it that’s also controversial than what we would normally expect. This difference was relatively small in the r/climateskeptic space, but a sizable one in the r/climatechange space, and especially large in the r/conspiracy space. Environmental topics such as climate change seem to be quite contested even though the scientific consensus behind them.
Table 3
| Controversial: No | Controversial: Yes | |
|---|---|---|
| Environmental: No | 125009 | 1832 |
| Environmental: Yes | 53472 | 867 |
Table 4
| Controversial: No | Controversial: Yes | |
|---|---|---|
| Environmental: No | 14652472 | 1148803 |
| Environmental: Yes | 167375 | 15337 |
Table 5
Goal II.
Determine and compare the evolution of the presence of environmental concepts across various subreddits.
Results:
When looking at environmental and controversial topics across the three investigated years, we wanted to identify if there’s a difference in trends across any of the subreddits over time. Figure 1 displays the total number of comments per subreddit per month, split left to right between General and Environmental themed comments. Dotted lines refer to controversial comments, and the scale is logarithmic for easier comparison. When looking at environmental versus general comments, the trends seem to be relatively similar. One notable exception is the amount of controversial environmental posts in the r/conspiracy reddit space, where there seems to be an uptick after 2022 January.
Figure 1
To investigate this further, we zoomed in on the controversial space, and calculated the percentages of controversial comments in both general and environmental topics. The results can be seen in Figure 2. When looking at the r/climatechange space, the share of controversial comments seem to be declining in both topics. On the other hand, controversy is slowly on the rise in the r/climateskeptic space. Both of these trends are relatively minor. The r/conspiracy subreddit has two sudden changes. First of all, there seem to be a negative trend already ongoing or starting around early 2021, a trend that suddenly reversed around the beginning of 2022, and controversy has been on the rise ever since. When looking at environmental controversy, the change is even more pronounced: before mid-2022, general and environmental controversy had a similar level, but after that, controversy disproportionately increased in environmental comments. There seems to be a connection between controversy and environmental topics in the r/conspiracy space, and it’s even more pronounced than in other, general topics.
To investigate this phenomenon, we looked at critical environmental policy events in the United States to see if we can find any events that might explain these shifts. On the decline starting at the beginning of 2021, one explanation could be the end of the Covid-19 crisis that potentially led to a decrease in controversial claims. Another explanation could be a stronger moderation or increased activity elsewhere following January 6. In any case, the increase in controversial and especially in environment-related controversial posts seem to coincide in time with the time period when US policy debate around climate investment and pollution control has also intesified. Around October-November 2021, the House passed legislation related to infrastructure, social benefits, and climate, called Build Back Better. That bill did not clear the Senate, but further discussions led to the passing of the Inflation Reduction Act in August 2022, a bill focused heavily on climate subsidies. At the same time, the Supreme Court started hearing arguments in February 2022 and made a decision in June 2022 about the Environmental Protection Agency (EPA)’s right to control pollution, another key environmental issue that was debated in that time period. What we’re seeing in the r/conspiracy subreddit might be activity reflecting on these events. We’ve also seen a general increase in the total number of comments in the r/climateskeptic and r/climatechange spaces (can be seen on Figure 1), which might also be related to these policy debates.
Figure 2
We also wanted to understand if well-known sources of climate misinformation were present in these spaces. To investigate this, we used a list of 53 outlets (websites, newspapers, blogs), collected by Coen et. al (2021) to see if these specific sites are mentioned and referenced in the submissions and the comments. We focused on both submissions and comments to gain a more nuanced understanding, and we also looked at the number of authors referencing these media outlets. The top 10 most common climate denial sources, and the associated statistics, can be found in Table 6.
Top 10 climate denial sources
Table 6
In general, it seems like these outlets are used frequently, but by a small number of authors, and they generate a sizable community engagement and many replies. Understanding in more detail the prevalence of these links and sources, and how it influences conversations, will be a focus point of our further research. The cumulative presence of these outlets can be seen on Figure 3.
Figure 3
Goal III.
Understand more about the posting patterns of sources of environmental controversiality, and if there are any users spreading climate-related controversy.
Results:
Similarly to our earlier analysis, we looked at the number of distinct authors in all three subreddit spaces (regarding comments), and the percentage of distinct authors of either general or environmental comments. The results can be seen on Figure 4. The difference is especially striking in the controversy space: it seems like a smaller percentage of distinct authors generate environmental controversy. As we’ve seen in Figure 2, the percentage of environmental comments is similar or even higher than the general comments, so this indicates the presence of central authors who spread controversial content and thus likely spread misinformation.
Figure 4
To build on our findings regarding the misinformation-related external sources, we looked at individual authors, and how many posts and distinct sites are related to them, as well as the engagement they generated. The top 10 authors associated with these sites can be seen in Table 7.
Author-level presence of known misinformation links
Table 7
In total, we’ve found 93 authors who posted and cited these sources. The number of posts related to the most frequent posters can be seen in Figure 5. When looking at proportions, 73% of posts spreading misinformation were created by just two authors. This indicates these authors are major nodes of misinformation, and investigating the networks they build and how they disseminate this information will be a future focus of this research.
Figure 5
Goal IV.
Understand the presence and prevalence of specific environment-related words and phrases across subreddits in both general and controversial comments.
Finally, we wanted to understand the presence of specific environmental phrases across all three subreddits, specifically for comments identified as controversial. As a reminder, the overall higher likelihood of environmental-related comments being controversial was already proven; our goal was to understand if any specific phrase or word had a notable frequency. Figure 7 showcases the results, normalized for comparability. Interestingly, out of all the phrases, climate was the most prevalent, indicating a strong correlation between climate-related posts and controversiality. Nuclear was also a high frequency keyword, but only in r/conspiracy, while fossil fuel and global warming were more likely to be found in r/climatechange and r/climatesceptic. This indicates that while controversial environmental and climate comments are present in all three subreddits, the specific content of these comments might be quite different. A more sophisticated NLP method is needed to compare the three spaces.
Figure 6