Sharing expert group decisions: Examining television meteorologists' tweets of a severe weather forecasting team’s warnings

As climate continues to change, severe weather events continue to increase in both severity and frequency. The U.S. National Weather Service (NWS) established storm prediction centers around the country to monitor and produce predictions, warnings, and watches about weather events. However, the NWS relies mostly on local television stations to communicate this information to the public. Although many storm prediction centers have Twitter accounts, residents often turn to local news stations for information on these weather events. In this study we analyzed one year of tweets from a small prediction team as well as tweets from the lead meteorologist twitter accounts from ABC, CBS, FOX, and NBC stations. We focused on tweets sent on days that severe weather occurred ( N = 17,259). Agenda setting theory served as a lens to examine these results and advance our understanding of weather communication in the digital age. We found that tweets from television meteorologists differed significantly from those of the NWS for clout, analytical thinking, and positive emotional valence. Tweets were also significantly different for authenticity and negative emotional valence, but only when individual stations were compared to the NWS. This paper contributes to small group literature the idea that expert teams, who rely on the media to report their decisions, may have their decisions reported in differing manners based on the motivations of the media.


Introduction
The ability to accurately predict severe weather events is becoming increasingly challenging, as climate change becomes more severe. The need to communicate life-saving forecasts about severe weather to the public is more important now than ever, as these storms have increased in severity and frequency. The U.S. National Weather Service (NWS) has developed a network of prediction centers staffed by forecasters trained to decipher complex weather data and produce weather predictions. These forecasters often work in teams of 3-10, depending on the population density of the region, and the decisions that these forecasting teams make result in severe storm watches and warnings throughout the country.
Although the NWS makes these predictions, the NWS has traditionally relied on local television and radio stations to make residents affected by severe weather aware of impending threats. However, the rise of social networking sites, such as Twitter, has created an alternative means for this information distribution. Many NWS storm prediction teams have Twitter accounts, but a casual examination of the number of followers of NWS accounts reveals a smaller following than that of television meteorologists. Therefore, if television meteorologists do not echo the message of the NWS, but reinterpret the message, then the decisions of this expert team could become distorted. Such a distortion could result in those in the path of severe weather to receive misleading information about the danger of severe weather. Underreporting a threat could lead residents to not be prepared for severe weather about to hit. Conversely, overreporting a threat could lead residents to begin to distrust weather predictions and ignore future watches and warnings.
In this study we sought to better understand how television stations in a large metropolitan area reported the weather compared to reports from the NWS field office. We first pulled one year's worth of tweets, and then removed tweets not related to severe weather, from NWS, FOX, ABC,NBC,and CBS (N = 17,259) to examine how weather predictions were presented and to see if significant differences existed. The stations selected for analysis (a) covered the same geographical region as the NWS field office and (b) were freely accessible to the public just like the NWS. Furthermore, we analyzed and limited the tweets of the NWS field office to days of severe weather (e.g., tornadoes, fire, ice storms) and then limited the tweets from the local television stations to just those time periods. We theorized, based on agenda setting theory, that local news stations would distribute messages about weather events in a differential manner than the NWS, as we will explain later. We found there were significant differences between tweets of the NWS and local stations, but we also found that these stations differed significantly from each other. The results of this study have implications for small group, science, news media, and organizational communication as we explain in the discussion.

Theoretical framework
There are several theoretical lenses we could have used to aid in understanding this topic and interpreting the results of this study. Agenda setting theory research supplies both a breadth and depth that allowed us to build a solid foundation upon which to base our hypotheses (Kim, et al., 2017), as agenda setting theory is employed in media studies across a variety of contexts. We first present a general understanding of agenda setting theory and then move to a more nuanced view of agenda setting that built the basis for specific hypotheses. The effects of agenda setting theory provide useful insight to how audiences interact with the messages they receive. Given previous findings of how influence credibility and emotional language affect viewer opinions of media (Lao, et al., 2019), this theoretical framework offers a clear way to predict the pattern of messages sent and provides a possible explanation for why news media outlets, or television stations, may be differentially motivated in their production of weather-related messages compared to the National Weather Service (NWS).

Agenda setting theory
First theorized by Lippmann (1922), agenda setting theory was refined and formalized by McCombs and Shaw (1972). Agenda setting theory highlights the ability of news outlets to create (or not) social awareness of any topic based decisions to include a given story in their messaging. By deciding what to include and what to exclude, media outlets influence what information their audience believes is important. Lao, et al., (2019) conducted a meta-analysis of studies using agenda setting theory published between 1972 and 2015.
They found a strong presence of agenda setting effects over time. Namely, the overall perceived importance an audience will attribute to an issue is affected by the content and style of the messages themselves.
For a clearer interpretation of existing literature, as it applies to the present study, we must define media networks within the context of this research. We examined several weather news outlets for the purpose of this study, but it is important to differentiate between them for both consistency and clarity. Outside of the previously mentioned NWS tweets, we also collected data from local stations affiliated with national networks. We categorize these local stations as either family-owned or network-owned. When applying Lao, et al.'s (2019) findings to this study we believe that how local television stations communicate about the weather will directly affect the decisions viewers make regarding weather preparedness. Considering the number of severe weather events that occur in the state where this study was conducted, the accurate reporting of weather predictions is crucial. How local stations advise viewers about the weather influences viewers' evaluations of the severity and importance of weather, thereby playing a deciding role in the actions the viewers take, or do not take, and the consequences of these actions.
In addition to the generalized findings of agenda setting theory research, Hester and Gibson (2007) expanded the scope of agenda setting theory to examine more specific facets of the media at large, suggesting that local news stations are more prone to agenda setting effects when reporting on issues that are more locally relevant. Moreover, they suggest examining agenda setting effects for local and national media separately because of these differences. Germane to this study is the understanding that local stations and national news networks may present weather reports differently in content and in style based on the goals of the outlet. Audiences in the same geographic area may receive weather reports and forecasts in diverse ways, based on the goals of the media outlet supplying the information. Agenda setting effects may thus alter audience opinions of weather predictions based on which news outlet they are viewing tweets from. More succinctly, news media or television outlets may feel a greater need to sensationalize stories than the NWS as these organizations rely on viewership for income generation and the NWS does not.

Credibility
The credibility of a media source plays a significant role in determining what media source viewers will rely on for their weather as well as how viewers will interpret the predictions. According to O'Keefe (2002), credibility is the combination of ability and trustworthiness, or the ability to "know the truth" and "tell the truth" [1]. Viewers are likely to see a news source as more credible if their level of ability and trustworthiness is high. Considering the NWS is the source of weather data, it may follow that viewers would be more likely to think their information is more credible than information from national or local weather reporters. However, there are factors which affect the credibility of a media source outside of the trustworthiness and expertise of the source: hedging and uncertain language. Audiences typically consider news coverage of scientific topics more credible if the journalists and the scientists who originate the data address the limitations or uncertainty about the data being reported (Jensen, 2008). Therefore, if one news outlet uses certain language, they may be deemed less credible than another news outlet which uses uncertain language. Stations that address the unpredictable nature of weather in their messages may be deemed more credible than those who report the weather with absolute certainty.
Credibility is a determinant of viewers' preferences for types of media. Hamilton (2004) explains that most viewers rely on local stations for their news, citing only 23 percent of viewers relying on national networks such as CNN, MSNBC, and FOX News. However, Elareshi and Gunter (2012) found national news sources were rated more credibly than local sources. Additionally, Rao and Ravi (2015) identified national news to be the most preferred medium for news but noted that viewers relied on local news for local events. National news is preferred and considered more credible, due to the viewers' desire to appear well educated and worldly as well as the volume of content national networks can produce. In lieu of this, there may exist stronger viewership and credibility attributed to weather messages broadcast on national networks than local stations.

Characteristics of credible tweets
In this study we examine four characteristics of these severe weather tweets (clout, emotional language, analytical soundness, and authenticity) that may be used to differing degrees by both the NWS and local television stations. We believe that these four factors play a key role in establishing source credibility.

Clout
The perceived credibility of a weather reporting outlet and their messages affects the viewers' evaluation of the outlet's clout. In contrast to uncertain or hedging language, clout encompasses confident language and conveys the authority of the outlet to disseminate information (Pennebaker, et al., 2015). We believe that experts at the NWS will use more hedging language as they are more aware of the changing nature of their predictions than television personalities distributing the information, and we therefore predict:

H1:
The news media will score higher on clout than the NWS. H2: Local stations and national networks will score higher on clout than the NWS.

Analytical thinking
Analytical thinking is the reliance on fact in messages (Jordan, et al., 2018). It stands in contrast with narrative construction tactics to formally present information as it is, creating logical and clear messages. Furthermore, it can indicate distance between the viewer and the station which produced the message, resulting in a less emotional tone being included. Considering the importance of emotional valence in weather messages, viewers may avoid highly analytical messages from one station in lieu of more emotional ones from another station. Due to the scientific stance of the NWS compared to media weather we predict: H3: The news media will score lower on analytical thinking than the NWS. H4: Local stations and national networks will score lower on analytical thinking than the NWS.

Authenticity
Message authenticity is marked by honesty and candor. Jordan, et al. (2018) note messages which score high on authenticity include more first-person perspective, insightful language, and "differentiation and relativity words" [2]. Authenticity differs from clout; in that it is more about being genuine than authoritative. Authentic messages are likelier to address limitations and uncertainty. Consumers that understand the complexity of weather prediction may find outlets which produce more authentic messages to be deemed more credible by viewers. For this reason, it is likely that: H5: The media will score higher on authenticity than the NWS. H6: Local stations and national networks will score higher on authenticity than the NWS.

Emotional language
The use of emotion in weather forecasts was a point of study within this research. The effects of emotional language within media have been examined across a variety of contexts, including agenda setting theory. In contrast to the previous accessibility explanation of agenda setting, Miller (2007) examined the role of emotional valence in agenda setting and viewer attention to news stories and identified a content-based explanation. The actual content of the news stories had greater influence on the viewers' evaluation of issue importance, especially when that content was framed with significant emotional affect. This may indicate a tendency for weather reporting to include emotional language. As referenced earlier, in cases of extreme or dangerous weather events the action, or inaction, of individual residents could be a matter of life and death. Therefore, in severe weather it may benefit viewers, and stations, to construct weather messages which reflect appropriate emotional valence and heighten the importance of weather events to viewers. However, we believe that trained NWS forecasters may be more neutral in emotional language because they are relying more on the science of forecasting. We therefore predict that: H7: The media will score higher on (a) positive emotions and (b) negative emotions than the NWS. H8: Local stations and national networks will score higher on (a) positive emotions and (b) negative emotions than the NWS.

Method
The researchers downloaded tweets using a proprietary program that allows access to the last year of tweets based on a hashtag or Twitter handle. The researchers downloaded one year of tweets from the National Weather Service (NWS) Twitter account of the office responsible for the region of study, as well as one year of tweets from each of the four major television station's (ABC, CBS, FOX, NBC) meteorology teams. The first researcher coded all tweets from the NWS for mentions of severe weather events. The first researcher also marked tweets that were retweets. The second researcher then verified these codes and removed all non-severe weather tweets and retweets from the NWS data. The second researcher then removed tweets from local stations that occurred on days when no NWS tweets were present, as well as retweets, from the dataset. This data reduction was performed to ensure that tweets in the dataset were original and about severe weather.

Descriptive statistics of tweets
The final dataset had 17,259 tweets: NWS (n = 3,059), ABC (n = 5,316), CBS (n = 3,100), FOX (n = 3,354), and (NBC n = 2,430). Tweets averaged 30.94 words per tweet (SD = 6.06) and 7.50 words per sentence (SD = 4.30). Linguistic Inquiry and Word Count (LIWC) 2015 was used to score the tweets using the composite summary variables of analytical thinking , clout (Kacewicz, et al., 2014), authenticity (Newman, et al., 2003;Pennebaker, 2011), and emotional tone (Cohn, et al., 2004). LIWC assigns scores on a range from 0-100. In the overall sample, analytic thinking had a mean of 89.71 (SD = 11.33), clout 55.91 (SD = 16.29), and authenticity 15.90 (SD = 34.02). The composite scale of emotional tone is scored uniquely in that 0 equals fully negative tone, 100 equals a fully positive tone, and 50 is no emotional tone. To conduct linear quantitative analysis the researchers used the sub-dimensions of positive emotion and negative emotion to evaluate emotional tone. Positive emotions had an average score of 13.33 (SD = 1.74) and negative emotions had an average score of 11.43 (SD = 1.34).

Results
The researchers used IBM's SPSS v. 25 package for all hypothesis testing reported below. Multivariate analysis of variance (MANOVA) was conducted with the NWS and news channels listed as a dichotomous grouping variable and analytic thinking, clout, authenticity, positive emotions, and negative emotions listed as the dependent variables. H1 predicted that the media would score lower on clout than the NWS and this hypothesis was supported [F(1, 17,258) = 221.07, p < .001, η 2 = .013]. H3 predicted that the news media would score lower on analytical thinking than the NWS. This hypothesis was also supported as media stations used less analytical thinking than the NWS in their tweets [F(1, 17,258) = 69.27, p < .001, η 2 = .004]. H5 predicted that the media would score higher on authenticity than the NWS and this was not supported [F(1, 17,258) = 0.05, p = .823, η 2 = .000]. H7a predicted that the media would score higher on positive emotions than the NWS, and this finding was significant [F(1, 17,258) = 8.61, p < .01, η 2 = .001], but in the opposite direction than predicted. H7b predicted that the media would score higher on negative emotions than the NWS and was not supported [F(1, 17,258) = 0.95, p = .329, η 2 = .000]; see Table 1 for complete statistics. MANOVA was then conducted with the NWS and news channels grouped individually and analytic thinking, clout, authenticity, positive emotions, and negative emotions listed as dependent variables. H2 predicted that news channels would score lower on clout than the NWS. Significant differences were noted [F(1, 17,258) = 153.73, p < .001, η 2 = .034] and a post hoc test with Bonferroni correction revealed that ABC, CBS, and FOX were all significantly lower than the NWS, but NBC had no significant difference. H4 predicted that news channels would score lower on analytical thinking than the NWS and significant differences were noted [F(1, 17,258) = 114.33, p < .001, η 2 = .026]. Post hoc test with Bonferroni correction revealed that NBC and ABC scored significantly higher than the NWS while FOX and CBS had no significant differences. H6 predicted that news channels will score higher on authenticity than the NWS and this was not supported [F(1, 17,258) = 53.26, p < .001, η 2 = .012] post hoc test with Bonferroni correction revealed that FOX and NBC scored significantly lower than the NWS, but ABC scored significantly higher, while CBS had no significant difference. H8a predicted that news channels would score higher on positive emotions than the NWS and this finding was significant [F(1, 17,258) = 54.03, p < .001, η 2 = .012], but post hoc test with Bonferroni correction revealed that only CBS was significantly higher. It is worth noting that CBS also scored significantly higher than every other television station. H8b predicted that news channels would score higher on negative emotions than the NWS and significant differences were found [F(1, 17,258) = 0.95, p = .329, η 2 = .000]. Post hoc test with Bonferroni correction revealed that FOX and CBS scored significantly lower than the NWS while ABC scored significantly higher and NBC had no significant difference. Please see Table 2 for complete details.

Discussion and conclusion
In this study we sought to examine how media outlets reported the decisions of expert weather forecasters and to see if severe weather messages differed when they came from the U.S. National Weather Service (NWS) compared to television weather forecasters. Considering the importance of weather warnings, as severe weather events increase and climate change becomes a greater focus of social consciousness, presenting prompt and correct, as possible, severe weather messages to viewers is crucial. The content of weather messages viewers receive affects their actions, or lack thereof. During severe weather threats, resident's actions have implications for their own personal safety as well as for society at large.
The results of this study illustrate why future researchers should approach media studies differentially based on the outlet's vulnerability to agenda setting. The ability of the NWS and television weather outlets to set the agenda for what weather messages viewers receive is clear, but we found the individual methods or modes of agenda setting to be more nuanced than initially thought. Agenda setting alters the messages distributed from the NWS and local television stations in diverse ways. Notably, we found that agenda setting resulted in differences not only between the NWS and local television stations, but also among the television stations themselves. Namely, the family-owned television station (CBS) was more prone to tweet information differentially than the network-owned stations of ABC, FOX, and NBC.
We specifically examined the role of clout, analytical thinking, authenticity, and emotional language in the messages reported by the NWS and television weather forecasters. In aggregate, the NWS scored lower on clout than media stations. The NWS also scored higher on analytical thinking and positive emotion language use than media stations. Thus, the results supported H1 and H3, but H7a was significant in the opposite direction. H5 and H7b were non-significant. We were interested in better understanding the unsupported hypotheses and believe that the further examination of the individual differences between local stations and the NWS may shed light on this issue.
Further investigation into the differences between the NWS and local television stations revealed that one station, CBS, differed significantly in every category from the other television stations. We believe this may be because CBS, in this market, is the only family-owned station while ABC, FOX, and NBC are all network-owned stations. Building from the ideas of Hester and Gibson (2007), we believe that not only may local stations be producing messages differently than national stations, but that family-owned stations may be differentially motivated in how they produce messages than network-owned stations. We reason that a family-owned network may feel more competition in the marketplace to keep viewers and this may be why we found that CBS uses more clout, more analytical thinking, less authenticity, and less positive emotions, and more negative emotions than the network-owned stations. By crafting messages in this manner CBS may be trying to build more credibility with viewers through their tweets.
Furthermore, the results of this study show different trends among media stations beyond the noted difference with CBS. When examined individually, media stations had significant differences from the NWS that often differed from the media comparison. For example, media stations overall did not score significantly different on authenticity than the NWS, but individually ABC had a significantly higher score on authenticity, as we predicted, but FOX and NBC had significantly lower scores and CBS had no significant difference. This offset of scores from FOX and NBC with ABC explains why in aggregate no significant differences were found. We reason that ABC forecasters must have some mandate or embedded norm for reporting weather events in a more authentic nature, although researchers must conduct more studies to determine if this is the case or not.
The results of this study suggest different relationships between news stations and weather communication than initially hypothesized. These differences between the messages the NWS and television weather forecasters communicate presents implications for both viewers and future academic research of weather communication. The mixed results of this study make it even more clear that members of the community may not be receiving correct information about severe weather events. When the NWS and media stations tweet differing messages about the weather, and when television stations communicate differently from each other as well, viewers that rely solely on one outlet over others may not respond appropriately to severe weather scenarios. If the NWS is reporting weather as more severe than media stations, and more viewers follow tweets of the media stations and consider these messages more correct, the safety of residents could be in jeopardy.
This study contributes to scientific literature the idea that media sources may misrepresent the predictions of expert teams. We have shown how the NWS and local media stations differentially communicate weather information. Such knowledge could further future academic research of message fidelity and message accuracy between and among the NWS and media stations. Furthermore, when creating weather messages, media forecasters could consider the implications of these reports and craft messages which not only accurately reflect the severity of the weather event, but also align with the forecasts presented by the expert teams of the NWS. Although media stations may wish to tweet about weather events in ways that engage more viewers, a cognizance of the meteorological content provided by the NWS could aid in preserving the expertise used to develop and refine severe weather forecasting. If the messages of expert teams are distorted, then that expertise could be lost, and the public could be misinformed or uninformed about the dangers presented by severe weather.

Limitations and future directions
Although accessing publicly available tweets from the NWS and local television stations provides the benefit of gathering messages that are communicated directly to the public, we cannot assume that these tweets match the weather communication that is occurring on air. Tweets sent out are limited in character length and therefore may not be able to capture as much contextual information as recording forecasters distribution of information during a severe weather event on air. However, we do note that the NWS does not have an on-air personality that could be used to compare with television stations and tweets are therefore the best way to compare and contrast these organizations at this time. Also, the use of LIWC as a computer aided coding system could prove problematic in that we do not know the exact algorithms used to generate scores for the variables of interest. We do acknowledge, though, that using LIWC has the benefit of limiting researcher biases in the coding process.
Based on the results of this study we believe that researchers should continue to investigate the discrepancies between how the NWS forecasts weather and how the media provides this information to the public. An ethnographic study of an NWS field office with video recording while simultaneously recording how weather is communicated from local television stations could supply more insight into why these differences exist in how weather events are communicated to the public. Furthermore, these findings may suggest a new way to think about the way media reports expert team findings in contexts beyond weather, such as the COVID-19 pandemic, and the effect that these reports have on public perception of the seriousness of the issue.