Internet public opinion on climate change: a world views analysis of online reader comments

In development and ex ante (before implementation) assessment of environmental and sustainability policies, scenario analysis is a popular method (Alcamo, 2008). Famous examples are the scenario analyses performed as part of the Millennium Ecosystem Assessment and the Global Environmental Outlooks (Carpenter et al., 2005; UNEP, 2002, 2007). More specifically focussed on climate change are IPCC's well-known SRES global emission scenarios (Nakicenovic et al., 2000), which have also been adapted to national levels, for example in the UK and The Netherlands (Hulme et al., 2002; RIVM, 2005).

Scenarios are plausible descriptions of how the future may develop, based on a coherent and internally consistent set of assumptions about key relationships and driving forces (Alcamo et al., 2005). Often, these assumptions are based on a set of archetypical world views, i.e. combinations of values – determining what aims are important – and beliefs – concerning how the world functions and how aims can best be achieved (De Vries and Petersen, 2009). In relation to environmental and sustainability issues, these world views define both the nature of a problem and how it should be solved (Verweij et al., 2006). In scenario analysis, the performance of policies can be assessed ex ante under different future conditions (Alcamo, 2008). This helps to determine the opportunities and risks associated with these policies from the perspective of different world views and to identify “robust” policies. Robust policies are policies that perform well both in terms of effectiveness and societal support under the various scenarios, and consist usually of a mix of measures preferred by different world views (Verweij et al., 2006; De Vries and Petersen, 2009).

In scenario analysis, the different archetypical world views are usually defined by social scientists, largely on a theoretical basis (Offermans, 2012). For an assessment of the robustness of policies it is therefore important to know to what extent these world views are represented in the public opinion. The common way to determine this is by means of surveys covering a representative sample of the population with written questionnaires or interviews. Based on their answers to the questions or statements, respondents are assigned to pre-defined world views, resulting in a frequency distribution of these world views among the population of, for example, the USA (Pendergraft, 1998) and European countries (PBL, 2008).

Recently, internet-based opinions have entered the climate change arena as new and powerful voices, strongly influencing public opinion and public and political support for climate policies, with the turmoil caused in 2010 by bloggers reporting on errors in IPCC's 4th Assessment Report as a telling example (Berkhout, 2010). As a consequence, politicians are nowadays extremely sensitive to internet-based opinions, which thus can have a disproportionately strong effect on the support for and robustness of climate policies. It is therefore important for scenario analysts to know how the world views they use for their policy assessments are represented in the public opinion on the internet.

In this paper, we report on a world views analysis of lay public opinion on the internet on climate change. We did this by means of a content analysis of publicly expressed opinions in the form of online reader comments to articles on climate change. Based on keywords, these reader comments were assigned to the different world views of two commonly used typologies in scenario analyses of climate policies. Our specific objectives were:

• to determine in the first place whether online reader comments can be assigned to different pre-defined world views and whether the two typologies differed in this respect;

• to determine the frequency distribution of online reader comments over the different world views, comparable to the results from surveys of the general public; and

• to determine whether this frequency distribution was affected by characteristics of the article to which the readers responded.

In the following sections, we will first present the two world view typologies we used in our study and give more details about our methods of data collection and analysis. We will then present a description of the material and our findings vis-à-vis our specific research objectives. We conclude with a discussion of the methodology and the results in comparison with other studies and discuss the implications for ex ante assessment of climate policies.

We conducted a world views analysis of internet public opinion on climate change by collecting online lay reader comments to newspaper articles on climate change and assigning these to the different world views of two typologies. In this section, we will first describe both world view typologies used in our analysis, and then proceed to describe our methods of data collection and analysis in more detail.

In scenario analyses in the domain of climate change, two typologies of world views are frequently used: IPCC's AB-typology and Cultural Theory's HEIF-typology. A typology is here a limited set of archetypical world views, structured according to a few organizing principles.

The best-known and most frequently used typology has been developed for the IPCC's Special Report on Emission Scenarios (SRES, Nakicenovic et al., 2000). The organizing principles are a focus on globalisation versus regionalisation and a focus on economic concerns and market forces versus environmental and social concerns and governmental regulation. The four resulting world views have been termed A1, A2, B1 and B2 in the IPCC report and have become widely known as such. In the national adaptations in the UK and The Netherlands, more telling names have been given, such as “Global Market” (A1), “Safe Region” (A2), “Global Solidarity” (B1), “Caring Region” (B2) (Hulme et al., 2002; RIVM, 2005). Figure 1 shows this “AB-typology” of world views, with their typical values and beliefs, including those pertaining to climate change.

The second world view typology included in our study, is less widely used in the domain of climate change, but has a longer history. Thompson et al. (1990) developed this typology based on an extended version of Mary Douglas' Cultural Theory. Since the 1990s, it has been used to describe and analyse societal views on climate change (Pendergraft, 1998; O'Riordan and Jordan, 1999). More recently, the typology has been put forward by prominent social scientists for assessment of climate policies (Verweij et al., 2006) and used to inform scenario analysis for climate adaptation in the UK (Walsh, 2007). The organizing principles are a focus on the group versus the individual and a focus on strong versus weak social regulation. This results in four archetypical world views, known as the “Hierarchist”, “Individualist”, “Egalitarian”, and “Fatalist” perspectives. This “HEIF-typology” of world views is shown in Figure 2, with the typical values and beliefs, including those pertaining to climate change.

Online reader comments to articles on climate change were collected from the web sites of 30 Dutch newspapers, including all national and regional daily newspapers and two major weekly newspapers. Collection followed a stepwise procedure:

• with the search function available at each of the newspaper web sites, articles on climate change were identified using the search terms “climate change”, “climate”, “climate summit” or “greenhouse effect”^[1];

• the contents of these articles were screened and articles not dealing with physical climate change were excluded;

• articles without online reader comments were also excluded; and

• the remaining articles including online reader comments were stored as separate electronic files.

Collection of articles and reader comments started in September 2008 and ended in December 2009, but articles published in previous years – when still available on the web sites – were also collected. This resulted in a collection of 3,360 online reader comments, associated with 295 articles on climate change, published on 30 Dutch newspaper web sites between August 2002 and December 2009.

The analysis of the data was conducted in several steps. First, all 295 articles were read and characterized in terms of year of publication, source newspaper, tone (explicitly supportive, critical or neutral with regard to the idea of anthropogenic climate change) and scope (national or international). Second, all reader comments were characterized in terms of date of publication and name of the reader. In addition, also the apparent sex of the reader was recorded in case the name used was obviously male or female. Multiple comments to a given article from the same reader were treated and counted as one single comment. For the third step, content analysis of the reader comments and assigning these to world views, a two-stage approach was chosen. In the first stage, a subset of the collection, consisting of 1,212 comments from a random selection of 11 out of 30 newspaper sites, was read and compared with the world view descriptions shown in Figures 1 and 2. From this comparison, sets of keywords (or brief key-phrases) were derived from the comments for each world view. The keywords can be considered as translations of the abstract terms used to describe the different world views in the scientific literature, to the actual language used in the online lay reader comments. In addition to the keywords, the beliefs of the different world views pertaining to anthropogenic climate change were classified as either convinced (B1, B2, Hierarchist, Egalitarian) or critical, not (fully) convinced (A1, A2, Individualist, Fatalist), based on RIVM (2005), Verweij et al. (2006), PBL (2008), Walsh (2007) and Offermans (2012). We refer here to the latter four world views as “climate sceptic” world views, which include but are not identical to views that explicitly deny anthropogenic climate change. On the basis of an interpretation of the position taken concerning anthropogenic climate change and the keywords, the remaining 2,148 reader comments were assigned to the different world views of both the AB and HEIF typologies. In case the position on climate change was not clear or no keywords were found, the comment was not assigned to a world view. In the fourth step of the data analysis, the distribution of the reader comments over the various world views was calculated per typology, and expressed as percentages of the total number of assigned comments. These frequency distributions were compared between typologies and according to article characteristics.

From a set of 2,148 online reader comments in response to 168 articles on climate change found on the web sites of 19 Dutch newspapers, 314 comments (15 per cent) could be assigned to a particular world view following the procedure described above. Only in a few cases, comments could not be assigned because the interpretation of the world view based on the position taken on climate change (e.g. “climate sceptic”) conflicted with the interpretation based on the keywords (e.g. Egalitarian). The majority of the comments could not be assigned because many were simply too short (consisting of just one or two lines) or completely off topic. Moreover, a substantial part of the longer, relevant comments could not be assigned because the keywords (or approximations) used to distinguish between world views were not found in the text, even though the position taken on climate change could be determined. These cases represented an estimated 5-10 per cent of the total set of 2,148 online reader comments. In this section we will now first present the general characteristics of the online reader comments and then present the distribution of the reader comments over the various world views in detail.

We refer to the sequence of online comments to an article, as a “discussion”. A discussion consisted on average of 11 comments, but this could vary substantially, from 1 up to 413 per discussion. In most cases, readers contributed only once (82 per cent) or twice (10 per cent) to a given discussion. The maximum number of contributions from a single person to a discussion was 46. As stated before, multiple contributions to a discussion from the same person were treated and counted as one single comment. Out of 3,360 comments, 1,650 (=49 per cent) were signed with a unique name. Based on the (first) names, we classified 42 per cent of the contributors as male and 5 per cent as female. The sex of the remaining 53 per cent could not be classified, due to the fact that the online reader comments were often signed with a made-up name or without a first name.

In terms of the content of the online reader comments, a few aspects stood out. First, the level of knowledge among the contributors about the issue of climate change appeared in general to be low, for example, climate change was frequently confused with the hole in the ozone layer, which is a mistake commonly found (De Boer, 2007). Second, despite the apparent lack of detailed knowledge, many contributors were quite outspoken in their opinions on climate change. Third, depletion of fossil fuels was a concern broadly shared among the contributors, including the need to reduce energy use and find alternative sources. Fourth, many of the comments displayed a sense of distrust, not only against politicians, environmentalists and the media, but also against climate scientists.

From the total of 314 online reader comments that could be assigned to a certain world view, 263 could be assigned to one of the world views of the AB-typology and 278 to one of the world views of the HEIF-typology. For both typologies, the resulting distribution of the reader comments over the different world views was highly uneven, with the two “climate sceptic” world views scoring 40-50 per cent each, and the other two less than 10 per cent each of the assigned comments (Table I). In the part of the reader comments that displayed a position on climate change, but could not be assigned to a particular world view because no relevant keywords were found, the share of “climate sceptic” comments was about 75-80 per cent.

When broken down per year of publication of the article, the distribution of the comments over the different world views appears to be rather stable over the years (Figure 3). More variation shows when comparing the percentages per world view between the different newspapers. The six newspapers included in Figure 4 vary from conservative to liberal and from right wing to left wing and thus adequately cover the Dutch political spectrum. Yet, the differences in the percentage per world view between the newspapers have no obvious connection with differences in political colour. For example, Volkskrant has the most deviant distribution of reader comments over world views. Traditionally, the readers of this newspaper are left wing voters, which could explain that the Global Market and Individualist world views are relatively weakly represented. However, the dominance of the Safe Region and Fatalist world views is less easily explained. Overall, the strong dominance of the “climate sceptic” world views is consistent across all newspapers. No useful comparison could be made according to the tone of the article to which readers commented, because more than 90 per cent of the articles were classified as “neutral” in tone. This indicates that the newspapers generally avoid to express a clear position concerning the issue of anthropogenic climate change. With regard to the scope of the article (national or international), the distribution of reader comments over world views varied in a perhaps predictable way for the AB-typology, with a relatively stronger representation of the “global” (A) world views when the article was international in scope, and of the “regional” (B) world views when the scope was national (Figure 5). A similar deviation can be seen for the Individualist and Fatalist world views, respectively, in case of the HEIF-typology, which is more difficult to explain. Again, irrespective of the scope of the article, the dominance of the “climate sceptic” world views remains.

In this section we compare our findings with the outcomes of public surveys and discuss the suitability of online reader comments as a data source for the purpose of our study. We conclude with a discussion of the implications of our results for ex ante assessment of climate policies.

Our results indicate that contributors of online comments to articles on climate change, pre-dominantly support “climate sceptic” world views. For the AB-typology, these results are in stark contrast with the outcomes of opinion surveys commissioned by PBL, The Netherlands Environmental Assessment Agency (Table II). According to these surveys, only a minority of the population has a preference for the “climate sceptic” A1 and A2 world views. The PBL (2008) outcomes for The Netherlands were relatively stable over the three years during which the surveys were conducted, and comparable with the outcomes for six other European countries studied. The only exception was Poland, where in 2007 over 60 per cent of the respondents displayed a preference for the two “climate sceptic” world views.

For the HEIF-typology, no public surveys of world view preferences have been conducted in The Netherlands, so no direct comparison can be made. The only broad public survey using the HEIF-typology known to us, is the one conducted by Pendergraft (1998) in the USA. In this survey, the Fatalist perspective was not included, because Pendercraft thought it unlikely that fatalists would bother to respond to his questionnaires. Nevertheless, also his results contrast strongly with our findings, as the Hierarchist, Individualist and Egalitarian perspectives were equally represented among his respondents (Pendergraft, 1998). When broken down to subgroups (e.g. according to sex or ethnic group) more variation showed, but deviations from the grand mean were always less than 10 per cent, and the Individualist, “climate sceptic” world view scored never more than 40 per cent.

A wide range of possible explanations can be thought of for the strong dominance of the “climate sceptic” world views in our study as compared to public surveys. A number of these explanations can be excluded on the basis of the available information, however. First of all, our results would have been biased towards “climate sceptic” world views, in case climate sceptical readers were more inclined to use world view related keywords in their comments, which would therefore have a higher probability of being assigned to a particular world view. This appears to have been the case, but not to the extent that it affects our conclusions because, with a share of about 75-80 per cent, the “climate sceptics” were also quite dominant in the part of the comments that displayed a position on climate change, but could not be assigned to a particular world view because no relevant keywords were found. Another explanation is that the dominance of “climate sceptic” comments was the result of active manipulation by an organized lobby of “climate sceptics”, but this is highly unlikely, given the many unique contributors to the online discussions. This dominance can also not be explained by an over representation of “climate sceptic” newspaper sites in our results, because the dominance of “climate sceptic” comments was independent of the political colour of the newspapers. Nor can it be explained as a reaction to dominance of either explicitly critical or supportive articles on anthropogenic climate change, because more than 90 per cent of the newspaper articles was neutral in tone. Finally, selective moderation of the online discussions is an unlikely cause as well, because, although most of the web sites included in our study state that comments that are offensive and insulting will be removed, one would not expect this to result in a dominance of “climate sceptic” comments.

Several other explanations for the strong dominance of the “climate sceptic” world views in our study seem to make more sense. First of all, due to the anonymity and immediacy of online comments, such comments may provide a more authentic insight into people's opinions than interviews or questionnaires (Henrich and Holmes, 2013). In non-anonymous interviews, there is a general tendency towards socially desirable answers, whereas in anonymous questionnaires, a selective part of the population may not care to respond. Second, the majority of the comments included in our results (nearly 80 per cent) dated from 2008 and 2009, whereas the latest PBL survey was conducted in 2007. According to Verhue and Koenen (2010), concern about climate change among the Dutch population has declined substantially between 2007 and 2010. Third, research has shown that people do not adhere to one single world view all the time, but that their world view preferences differ according to issue and context (Offermans, 2012). It might be that in the specific case of climate change, people prefer other world views than in the more general context of environment or sustainability, which were the contexts of the survey questions and statements by Pendergraft (1998) and PBL (2008). However, although these three factors may certainly have contributed to some extent to the strong dominance of “climate sceptic” world views in our results, it seems unlikely that public surveys yield a fundamentally wrong picture of public opinion or that a complete landslide in public opinion on climate change has occurred in The Netherlands between 2007 and 2009.

A much more simple and plausible explanation for the large differences in outcomes between our study of online reader comments and public opinion surveys, is that the contributors of these online comments are not representative of the Dutch population at large. Our finding that the very large majority of the contributors is male, is a strong indication in this respect. Furthermore, other studies of online reader comments present a typical profile of the contributors: “blindly opinionated” (Richardson and Stanyer, 2011), “often challenging institutions” (McCluskey and Hmielowski, 2012), and “using comments to vent and bloviate” (Eisinger, 2011), “or even solely with the purpose of attacking other readers personally” (Paskin, 2010). According to Mitchelstein (2011), posting comments in online newspapers is usually associated with self-expression needs and not with discussion and socialization motives, whereas Myers (2010) concluded that online posters produce increasingly more provocative comments to gain distinction in the crowdedness of online space. These characteristics of contributors make it likely that adherents to “climate sceptic” world views, with outspoken opinions and a strong need to express their dissent with the prevailing, institutionalized views on climate change, will be overrepresented in online reader comments as compared to public surveys.

We studied the prevalence of pre-defined world views in public opinion on the internet by content analysis of online reader comments. Content analysis has been used before, e.g. by Beumer and Martens (2010) and by Offermans and Cörvers (2012) who studied the prevalence of world views of the HEIF-typology in policy-oriented documents on biodiversity conservation and water management, respectively. In both cases, this method yielded useful results, and in the case of Offermans et al. (2013), the results appeared comparable to the outcomes of a survey of water management professionals. The novelty in our approach was to apply content analysis for this purpose to online reader comments. Online reader comments present a rapidly growing volume of text that increasingly attracts the attention of scholars from media studies, science communication and sociology of science, but the methodologies to study this new data source are still under development (Henrich and Holmes, 2013). The primary interest of media studies concerns the question in how far online reader comments and discussions match ideals of participatory journalism or open public debate (Paskin, 2010; Richardson and Stanyer, 2011), whereas researchers of science communication and sociology are interested in laypeople's discourses on science and scientific issues (Brossard and Scheufele, 2013; Henrich and Holmes, 2011). A very recent example of the latter in the field of climate science is the study by Jaspal et al. (2012) and Koteyko et al. (2013). These authors conducted a discourse analysis of online reader comments, posted on the web site of a UK newspaper before and after “climategate”, and concluded that it yielded insightful results revealing how readers' constructions of climate science have changed after “climategate”. We conclude that online reader comments are also a useful data source for a world views analysis. Despite the fact that in our case 85 per cent of the comments could not be assigned to a certain world view, the volume of this data source is so big that the remaining 15 per cent still represented a sufficiently large sample size (n>300). The material appears equally suited for both world view typologies, as the numbers of comments that could be assigned to world views of both typologies was about the same. A minor difference was that for the HEIF typology it turned out to be more difficult to find keywords matching the abstract descriptions of the world views, especially for the Hierarchist and Egalitarian perspectives. Further research on how to operationalize the abstract descriptions of world views and how to translate them to specific issues, such as climate change, taking lay vocabulary into account may help to increase the “useful” proportion of online reader comments in a world views analysis.

Discussions on newspaper web sites in response to articles on climate change appear to be strongly dominated by people with world views that can be characterized as “climate sceptic”. The contributors to these discussions, however, cannot be considered as representative of the population at large. It would nevertheless be unwise to ignore this group just for this reason. In the recent past, internet-based opinions have proven to be powerful voices in the climate change arena, with the potential to strongly influence public opinion and support for climate policies (Berkhout, 2010). In case of online comments, studies have shown that readers' opinions were more influenced by the comments than by the article itself (Yang, 2008). Although the people actively contributing comments represent a relatively small group – in the UK in 2008 less than 0.1 per cent of the unique web site visitors per month according to Richardson and Stanyer (2011) – their comments are read by a large part of the population (Henrich and Holmes, 2013). Therefore, it is advisable for scenario analysts to pay due attention to “climate sceptic” world views in ex ante assessment of the degree of societal support for climate policies (i.e. their social robustness). Even though public opinion surveys are probably correct in indicating that only a minority of the population in The Netherlands and many other European countries has a preference for “climate sceptic” world views, resistance to climate policies that are based on “climate supportive” world views is potentially very strong, which seriously affects the social robustness of these policies. An entry to identifying socially robust policies, i.e. policies likely to be supported by a large majority of the population, is perhaps our finding that concern over the depletion of fossil fuels was broadly shared across all world views. This concern could therefore provide common ground for climate policies targeting renewable energy resources.

The purpose of our study was to assess the representation of different world views with respect to climate change in public opinion on the internet in the form of online reader comments to articles published on Dutch newspaper web sites. We conclude that it proved possible to assign online reader comments to different pre-defined world views and that these comments are a useful source for a world views analysis of public opinion. The material appears equally suited for both world view typologies we applied, as the numbers of comments that could be assigned to world views of both typologies was about the same. For both typologies, the distribution of comments over the different world views was highly uneven, with world views characterized as “climate sceptic” scoring more than 90 per cent of the assigned comments. The strong dominance of these “climate sceptic” world views was independent of year, newspaper, and scope of the article to which the readers responded. These findings are in stark contrast with the outcomes of public opinion surveys which indicated that only a minority of the population has a preference for a “climate sceptic” world view. The most plausible explanation for this difference is that the contributors of online reader comments are not representative for the Dutch population at large. However, as internet-based opinions have a proven potential to strongly influence the opinion of the general public and politicians on climate change, we advise analysts to pay due attention to “climate sceptic” world views in ex ante assessment of the societal support for climate policies.

Joop De Kraker is an Associate Professor of environmental sciences and Research Coordinator of the School of Science at Open Universiteit (The Netherlands). His research focuses on the interface between science, policy and society in sustainability issues, in particular on tools for multi-actor learning and joint knowledge production. He holds a PhD in agricultural and environmental sciences from Wageningen University. Joop De Kraker is the corresponding author and can be contacted at: joop.dekraker@ou.nl

Sacha Kuijs currently works as an Advisor on the environmental aspects of spatial planning for a medium-sized town. The study described in this article was part of her Master thesis project in environmental sciences at Open Universiteit. She also holds an MSc in computer science from Leiden University.

Ron Cörvers is an Associate Professor in Governance and Sustainable Development and Director of the International Centre for Integrated Assessment and Sustainable Development (ICIS) at Maastricht University. His current research is on governance and sustainable development, global certifying partnerships, and learning for sustainable development. He holds a PhD in environmental policy from Utrecht University.

Astrid Offermans is a postdoctoral Researcher at the International Centre for Integrated Assessment and Sustainable Development (ICIS) at Maastricht University. In her PhD research, she developed a method to integrate social dynamics into scenarios in order to explore the sustainability and social robustness of water management strategies. Currently, she is working on the topic of (joint) knowledge production in certifying partnerships.