What's more important in a weather forecast: Accuracy? Or consistency?
There was an interesting twist to the big snowstorm that hit parts of the Northeast on Tuesday. While some spots inland off the coast did indeed receive a major snowfall of 20-40 inches, the millions of people in coastal populated areas like Washington, D.C., Philadelphia and New York City received quite a bit less than the 7-24 inches predicted in those areas (the lower forecasted amounts were toward Washington, D.C,; higher amounts toward NYC and Philadelphia.)
What happened: The line between where temperatures were cold enough for snow versus rain inside the storm turned out to settle just inland off the Atlantic Coastline, bringing coastal areas more in the way of sleet and rain instead of snow, instead of the predicted position just offshore, which would have brought those hefty snow totals that fell inland right to the coast as well, affecting tens of millions of people.
While forecasts had been very consistent most of the time in the snowier scenario, on Monday night, the models began backing off, correctly predicting the rain/snow line would be farther inland and less snow would fall in the major cities.
Here are two ensemble snow total forecast maps from Monday morning showing, yes a major snowstorm still on for the inland areas, but note the lack of color right along the coast, particularly Washington, DC, Philadelphia and New York City:
But the forecasters there made a conscious decision to stick with the greater snowfall totals and corresponding warnings in their forecast.
WASHINGTON (AP) - Before the first snow fell, U.S. meteorologists realized there was a good chance the late-winter storm wasn't going to produce giant snow totals in big Northeast cities as predicted.
But they didn't change their forecasts because they said they didn't want to confuse the public.
National Weather Service meteorologists in Boston, New York, Philadelphia and Washington held a conference call Monday afternoon about computer models that dramatically cut predicted snow amounts. They decided to keep the super snowy warnings.
"Out of extreme caution we decided to stick with higher amounts," Greg Carbin, chief of forecast operations at the Weather Prediction Center in suburban Maryland, told The Associated Press. "I actually think in the overall scheme that the actions (by states and cities) taken in advance of the event were exceptional."
On Monday, the weather service predicted 18 to 24 inches of snow in New York City. By late Tuesday afternoon, Central Park was covered with a little more than 7 inches of snow with rain and sleet still falling. Other areas, including upstate New York and Connecticut, received more than a foot and a half of snow. Swaths of Pennsylvania were walloped by 20 to 30 inches of snow.
Carbin said a last-minute change downgrading snowfall totals might have given people the wrong message that the storm was no longer a threat. It still was, but real danger was from ice and sleet in places like New York City and Washington, he said.
Dramatically changing forecasts in what meteorologists call "the windshield wiper effect" only hurts the public, said Bob Henson, a meteorologist for the private Weather Underground.
Carbin stood by the decision.
"The nature of the beast is that there's always uncertainty in every forecast and we have to get better at describing that," Carbin said.
The right amount of precipitation fell, but it came down as rain and sleet because the rain-snow line moved inland, according to Carbin and private forecasters.
Private forecast outfits said the National Weather Service did a good job forecasting a tough storm despite the beating the federal agency took on social media.
"Overall the range of the forecast was very solid. It ended up being on the low end," Henson said. "I understand why people can be frustrated when the expectation is the big storm."
But was not backing off the forecast the right call?
Seattle-area meteorologists can sympathize with their East Coast brethren as the storm came nearly exactly five months after our own forecast difficulties in a predicted major wind storm that fizzled. (Although as mentioned, at least the northeast storm did bring a lot of snow and winter storm issues, just not as much in the mega populated regions.)
In the wake of our "wind" storm, we've done a lot of soul searching on how to handle these kind of storms in the future, mainly centered on trying to create better ways to communicate probabilities and uncertainties. We've even created an "Integrated Weather Team" made up of Seattle- and Portland-area meteorologists and emergency officials to tackle these issues.
Also on that team is UW psychologist Dr. Susan Joslyn, who has been researching how the public absorbs and analyzes forecasts and has been instrumental in being a part of the process of guiding the future of severe weather forecasts. And coincidentally, she did a presentation at the recent Northwest Weather Workshop on her study just completed that aimed to determine what has more of an impact on the public's trust of the forecast: Inconsistency, or inaccuracy?
She set up hypothetical forecast companies and gave an impending snow storm scenario where one company was more consistent -- as in the forecasted snow totals remained steady at each forecast period even if forecast models were changing their forecasted amounts, and another source where the forecast bounced around more "chasing the models".
She split the hypothetical forecast providers into four groups. Half were consistent (but only half of the forecasts were accurate) and half were inconsistent (and half of their forecasts were accurate.)
What she found was that, of course, consistent, accurate forecasts give you the best "trust" rating, with accurate "inconsistent" forecasts coming in second. There was obviously less trust for being wrong, but no statistically significant "bump" in trust for at least being consistent about an ultimately inaccurate forecast. And being inconsistent but ultimately right scored having higher trust in the forecasters than being consistent, but wrong.
"If you think more recent information is more accurate: use it," and "consistency does not buy you anything in terms of trust of if you are wrong."
It would be interesting to see additional research if this changes when the stakes are raised to major storms and whether the northeastern Weather Service forecasters indeed would have been better off lowering the snow forecasts to be more accurate, and then should be seen as more trustworthy, or if their fears would have been validated that people would take a still-dangerous storm less seriously. There, the thought process could be over-simplistically boiled down to: Save face or potentially save lives?
Most meteorologists will likely choose the latter. Although that comes with the peril of a population who may not take your next storm warning as seriously.
We faced similar fears in our windstorm -- even as it was becoming apparent that morning that the wind storm was not going according to plan, that significantly backing off the wind speeds would then have everyone believe the dangers were over and give up on any safety precautions taken (although one difference between us and the Nor'Easter --much of our models even up to the morning of the wind storm were still predicting widespread damaging wind speeds, even if not at the lofty speeds they had been predicting).
Ultimately, the windstorm in Western Washington ended up tame enough that there was no widespread damage but we, like New York and Philadelphia for their Nor'Easter -- were on the razor's edge of a significantly dangerous storm where a small tweak on such a large scale event would have had drastically different outcomes.
The discussions continue...