Sandy: How Good Is Our Weather Prediction Ability?

I have been critical here on numerous occasions about "global warming."

Recently I have been interested in matching the predictions of hurricane Sandy against the actuality.  The site I used for the predictions was the National Hurricane Center.

There isn't a lot of concise, direct prediction, e.g., the path it will take, and so on.

But there is a lot of "probability maps" showing, for example, where the most likely rain fall will be.

The main predictions as I saw it were (I am using "consumer" predictions - not any special weather data):

1) The hurricane will move up and along the coast from the Bahamas (I only really paid attention as the hurricane left the Caribbean).

2) The hurricane will shift inland to mid-Pennsylvania.

3) The hurricane will move off to the north.

(Edit - here is an image from the prediction period...)

The destruction in NYC and NJ were a given as long as the hurricane went inland below New York harbor.

So how did the predictions work out?

Well, at a gross level they were generally accurate in terms of basic direction and movement.

The storm moved up along the coast as predicted.  It moved inland starting Monday morning.

By about 8 PM Monday night the "eye" (rotation center) reached the NJ/Pennsylvania border.

By Tuesday the center reached Pittsburgh.

Today, Wednesday the "eye" has moved to western New York state.

So over all this was what was predicted.

The low-level details, however, were all over the place.  The "eye" reaching NJ from the traversal up the coast was much more rapid than predicted.  There wasn't a lot of precision where the storm would reach the shore.

Once the storm reached the shore a lot of strange things happened (I talked about the "shock wave" yesterday).

The storm changed from a hurricane to something else as it moved inland and its unclear what the path it took was.

The rain and snow levels shifted around and changed dramatically from day to day (the predictions are posted periodically on the site but I checked them once or twice a day).

So in general each event was predicted as something of a probability curve.  The higher the curve the higher the probability.  So the storm followed along within the curve but not always along the highest probability predictions.

The key game changer was a low pressure system which had a front along the western Pennsylvania/Ohio border.  This created a low pressure "trough" that Sandy slipped into as it approached.  Sandy then followed this inland.

The only assessment of all this is that in terms of gross prediction things went well - but in terms of precise detail things were all over the place and there was little correspondence between predictions and actuality.

Now there was talk about how the "low pressure" affected the hurricane model's prediction ability.

The model seemed to work only after the hurricane was on-track to move up the coast, i.e., before that point it was unclear what the hurricane would do exactly (a slight change in course, for example, would have simply sent it out to see as often happens with hurricanes that move up the coast).

So clearly there was a prediction "model" involved which worked at a gross level.

So let's compare this to recent talk we see about global warming (here for example).  This article talks about "improved" models that provide "less certainty."

Advances in computing power, for example, allow models to use more data points, provide more complex and sophisticated modeling, and so on.

However, the "output" of the model shows less certainty about what will happen.  Temperature goes up but what this means is less clear.

Not really much different than what is accomplished for the hurricane.

On the other hand, the hurricane model doesn't work too far in the future, i.e., determining where it will go once it enters the Caribbean.

The question becomes this:

1) If we cannot predict what a major storm system will do over the course of say, two weeks, how can we predict anything more accurately at a larger time scale?

2) How can we be certain about the precision of, for example, small temperature changes from climate models when even within a day or two we cannot precisely predict landfall for something simpler (in terms of climate) like a hurricane?

This is like saying while I cannot predict the precision of each shot in a pool game I can say who will win based on a large collection of such unpredictable shot predictions.

One thing that to me reenforces the foolishness of this "climate modeling" is how nothing available to the public shows the relative accuracy of past weather predictions.

For example, each day sites like AccuWeather show predictions for the next day, week, and so on.

But where is the historic comparison, i.e., we predicted X for next week but Y actually happened?

No industry I know of operates in this way, particularly when public safety is involved.

It would be like never analyzing failures.  For example, as suspension bridges were built over the years various bridges failed quite publicly (for example, the Tacoma Narrows bridge below).

Similarly for financial investments and any other human endeavor that involves learning from past mistakes.

But with weather the past weather predictions just seem to vanish.

What I don't understand is why?  How is this science?  Where is the regression comparison of what we predicted in the past and what actually happened?

My guess is that the public would have little confidence in the science of "weather prediction" if they could see the accuracy.

Some science, such as chaos theory, tells us that we cannot accurately predict the weather.

Yet we continue to try?

The real question is more basic: Can we even reliably predict the weather beyond several hours?

I don't think anyone knows the answer.  But from what I know about chaos theory I would say no, its simply impossible to make accurate long term predictions.

Here is a good NY Times article about this very subject: "The Weatherman is not a Moron."

Personally I think of weather prediction, at least at the consumer level, as more of a "comfort food."

I've noticed that radio stations, for example, always try and put a good spin in "weekend weather."

For me, I simply look at the radar.  Is something "green" (for rain) coming?  You can predict fairly accurately (even given the diddling that removes the precision from the radar) when you are going out for the afternoon.

Predicting a full day or a weekend, though, is less simple.

Perhaps as a society we need to think about how our education system, particularly a government-backed one such as we have, can be used to assist us.  Instead of turning out poets with $100K USD in debt perhaps we should look at building a reliable weather prediction system over a decade - as JFK did with the "race to the moon" in the 1960's.