Assessing the Accuracy of Tornado Watches across the Contiguous United States from 2007-2015

During 2007-2015, a total of 2,359 tornado watches were issued by the Storm Prediction Center and 10,840 tornadoes were confirmed. The highest concentration of tornado watches occurred in Mississippi, Alabama, Louisiana, and Arkansas. Tornado watches are issued when conditions are favorable for the development of severe thunderstorms and tornadoes over a larger-scale region. This study analyzed the accuracy of tornado watches for the nine year period of 2007-2015. In addition to accuracy, fatalities, lead times and valid watch times were calculated for each tornado watch. 58.80% of the tornado watches had at least one tornado inside the tornado watch and 27.43% had at least one tornado outside the tornado watch. Of the 10,840 tornadoes, 56.70% were inside a tornado watch, 9.69% were outside a tornado watch, and 33.62% occurred when there was no tornado watch in effect. The average valid time for a tornado watch was 6 hours and 50 minutes and the average lead time for a tornado was 2 hours and 8 minutes. Tornado watches are designed to alert the public to the potential threat to life and property, a mission of the National Weather Service. This study demonstrates the difficulty of accurately forecasting tornadoes, both spatially and temporally, as well as the significance of preparedness and mitigation strategies. It should be noted that the accuracy of tornado watches were analyzed, not the effectiveness of tornado watches.


Introduction
A tornado is defined as "a rotating column of air, in contact with the surface, and often visible as a funnel cloud and/or circulating debris/ dust at the ground" [1]. On average, the United States experiences over 1,100 tornadoes per year [2]. Tornadoes can occur throughout the year, at any time of day, and produce winds ranging from 30 meters per second (m s -1 ) (67 miles per hour ((mph)) to 135 m s -1 (300 mph) [1]. From 1985-2014, tornadoes have been responsible for an average of 73 fatalities annually [2] as well as additional economic, societal, and physiological impacts. The stated goal of the Storm Prediction Center (SPC) is to have a tornado watch issued two hours prior to the first tornado [3]. Tornado watches are issued by the SPC in Norman, Oklahoma "when conditions are favorable for the development of severe thunderstorms and tornadoes over a larger-scale region" [4]. A watch is defined by the area inside a parallelogram. Tornado warnings are issued by the local National Weather Service office (NWS) "when there is evidence based on radar or a reliable spotter report that a tornado is imminent or occurring" [4].
A common tool for evaluating tornado warning accuracy is a 2 × 2 contingency table, which compares tornadic events to forecasts [5][6][7][8]. A hit is assigned to the event if a tornado was reported and a tornado warning was in effect. A miss is assigned to the event if a tornado was reported, but a tornado warning was not in effect. The Probability of Detection (POD) is calculated by dividing the number of hits by the total number of hits and misses [5][6][7][8][9]. This method will also be applied to better assess the accuracy of tornado watches.
From 1982-1988, more than 80% of all significant tornadoes (killer tornadoes as well as those rated F3 or higher on the Fujita scale [10]) and 40% of all tornadoes occurred within or near a tornado watch [10]. Keene et al. and Hales found that tornado watches tend to give local NWS office meteorologist the confidence to issue tornado warnings on severe local storms as opposed to when no watch is issued. For example, from 1997-2007, the POD drastically increased for tornado warned storms when tornados watch was in effect [7]. Keene et al. found that a 20 year improvement of tornado warning performance can be seen when a tornado watch is in effect.
The SPC can also issue Particularly Dangerous Situation (PDS) tornado watches when "long-lived intense tornadoes are likely" [11]. PDS tornado watches are uncommon, but "three out of every four should verify with multiple intense tornadoes" [3]. From 1996-2005, a F2-F5 tornado was three times as likely to occur in a PDS tornado watch compared to a typical tornado watch and four times as likely when there was at least two F2-F5 tornadoes [12].
The objective of this study was to determine the accuracy and usefulness of tornado watches across the contiguous United States from 2007-2015. Geographic Information Systems (GIS) was used to assess the accuracy of tornado watches, both spatially and temporally. Tornado watch data and tornado track data were used to determine the accuracy of 2,359 tornado watches.

Tornado event tracks
All tornado tracks from 2007-2015 from the severe report database (SVR) were extracted into GIS as annual shapefiles dating back to 1950. Each shapefile contains information about the tornadoes including date, time, location, strength, injuries, fatalities, starting and ending latitude, starting and ending longitude, length in miles, and width in yards. Tornado tracks were generated by connecting the beginning and ending latitude and longitude with a line. Tornado track data were used, instead of touchdown points, in order to better assess the tornadoes proximity to a watch polygon during its lifecycle.

Tornado watches
Tornado watch data from 2007-2015 were obtained from the SPC Severe Weather Events Archive page. This page contains information about all severe weather events dating back to 3 January 2000. Every entry that is classified as a severe weather day contains information about the events of the day, including storm reports, observations and mesoanalysis, skew-t/log-p charts, outlooks, mesoscale discussions, watches, and public outlooks. Tornado watch polygons were generated by importing the four latitude and longitude points of each tornado watch from the SPC Severe Weather Events Archive into GIS. Geospatial Modeling Environment (GME) was used to connect the four points of each tornado watch (Figure 1).

Methods and Materials
This study analyzed the accuracy of tornado watches for the nine year period of 2007-2015. A similar study by Schaefer, 2000 analyzed tornado watches from 1970-2000. For this study, data prior to 2007 were not analyzed due to the advent of the EF tornado rating scale in 2007. To calculate the accuracy of tornado watches, each year was analyzed separately [9,10,[12][13][14][15][16][17]. The accuracy was calculated by determining if each tornado watch was a 'hit' or a 'miss.' A 'hit' was defined as at least one tornado occurring inside the tornado watch during the valid time frame [9,10,[14][15][16][17]. If any portion of the tornado track occurred inside the tornado watch polygon then the watch was considered a 'hit.' A 'miss' was defined as at least one tornado occurring outside the tornado watch, up to 161 kilometers (km) (100 miles (mi)), and within the valid time frame. 'No watch' was defined as a tornado that occurred where no watch was in effect or outside the valid time frame. Each tornado watch was classified as a 'hit' or a 'miss.' A tornado watch was considered both a 'hit' and a 'miss' if at least one tornado occurred inside the watch and outside the watch during the valid time frame. It should be noted that a tornado could occur inside or outside one or more watches, if two or more watches overlapped or were within 161 km of each other.
In addition to accuracy, fatalities, lead times and valid watch times were calculated for each tornado watch. Lead times were calculated by comparing the time of the first tornado to the start time of the tornado watch [17]. Lead times were not calculated for watches with no tornadoes. Valid watch times were calculated from the initial tornado watch issuance; early cancellations were not taken into consideration.

Results and Discussion
During 2007-2015, a total of 2,359 tornado watches were issued by the SPC with 10,840 confirmed tornadoes for the same period. 2008 had the greatest number of tornadoes, tornado watches, and PDS tornado watches with 1,694, 405, and 22, respectively (Table 1). Of the 10,840 confirmed tornadoes, 88.02% (9,541) were weak (6,187 EF0s and 3,354 EF1s), 11.28% (1,223) were strong (953 EF2s and 270 EF3s), and 0.70% (76) were violent (67 EF4s and 9 EF5s). 74.19% (707) of the EF2 tornadoes occurred inside a tornado watch, while 9.76% (93) occurred outside a tornado watch. 85.93% (232) of the EF3 tornadoes were inside a tornado watch and 5.19% (14) were outside a tornado watch. All of the EF4 and EF5 tornadoes were inside a tornado watch ( Table 2). 89.02% (308) of the 346 EF3 or stronger (EF3+) tornadoes occurred inside a tornado watch, an increase from Hales (1990).      27.43% (647) of the tornado watches had at least one tornado outside the tornado watch. 2013 had the lowest percentage of tornado watches with at least one tornado outside the tornado watch with 24.14%, followed by 2014 and 2007 with 24.59% and 24.66%, respectively. The highest percentage, 35.21%, occurred in 2015 ( Figure  2). 4.83% (114) of the tornado watches had at least one EF2+ tornado occur outside the watch. Of the 90 PDS tornado watches, 45.56% (41) had at least one tornado outside the tornado watch and 11.11% (10) had at least one EF2+ tornado outside the tornado watch.
A slight increasing trend in the accuracy of tornado watches occurred from 2007-2015. The greatest percentage of tornado watches with at least one tornado inside the tornado watch occurred in 2011, while the lowest percentage occurred in 2009 with 65.75% and 51.42%, respectively. Previous studies have shown an overall increasing trend in the accuracy of tornado watches for the respective time period [9,14,16,17]. A more substantial increase in the accuracy of tornado watches occurred from 1970-2000. The lowest percentage occurred in 1970 with only 21%, while the highest occurred in 1996 with 63% [17] ( Figure 3). During 1980-1989, the lowest percentage of verified tornado watches occurred in 1987 and the highest occurred in 1984 with around 36% and 54%, respectively [9,14]. An extended study, from 1967-1990, showed similar results but the lowest percentage occurred in 1970 with 29% [9,14]. During 1967During -1977, the lowest percentage occurred in 1970 with 29% and the highest occurred in 1977 with 47% [16]. During 2007-2015, 1,034 fatalities occurred from 216 tornadoes. 968 (93.62%) fatalities occurred inside a tornado watch, 18 (1.74%) occurred outside a tornado watch, and 48 (4.64%) occurred when there was no tornado watch in effect. 667 (64.51%) of the 1,034 fatalities were caused by violent tornadoes, 337 (32.59%) were caused by strong tornadoes, and 30 (2.90%) were caused by weak tornadoes. Even though an increasing trend in the accuracy of tornado watches has occurred over the last 45 years, fatalities are still prevalent.
In conclusion, 2,359 tornado watches were issued by the Storm Prediction Center and 10,840 tornadoes were confirmed during a nine year period from 2007-2015. Of the 2,359 tornado watches, 58.80% were confirmed with at least one tornado occurring inside the tornado watch polygon, while 27.43% had at least one tornado occur outside the tornado watch. 56.70% of the 10,840 tornadoes occurred inside a tornado watch, 9.69% occurred outside a tornado watch, and 33.62% occurred when there was no tornado watch in effect. The average valid time for a tornado watch was 6 hours and 50 minutes and the average lead time for a tornado was 2 hours and 8 minutes. Mississippi, Alabama, Louisiana, and Arkansas had the highest concentration of tornado watches from 2007-2015. Future research will continue to assess the accuracy of tornado watches, fatalities, lead times, and valid watch times as well tornado watch size.