How wrong are the weather services?

I've had an idea for launching a web service here at home.milkmandan.org for a while now and since I haven't gotten off my ass to actually do it (read: no time) I guess I'll give the idea away beforehand.

I've always been really curious just how wrong the weather prediction systems are. In specific, I'd like to know how much they change their estimates of what the weather will be like from the time that they start forecasting (usually five to ten days, depending on the service) up until the day of the weather itself. I strongly suspect that the numbers range all over the place and that aggregating the data together is likely to reveal trends as far as who gets this sort of thing correct and incorrect.

I don't know if anything like this already exists—at least, publically. Does it?

Presuming that it doesn't (I'm not sure...I haven't looked very hard) I plan on implementing this sometime in the next month or so and plan on using Web Services to do it.

The idea is that I'll assemble a database (probably MySQL, just because that's what I had to install to get Movable Type to work) and write some thin Web Services wrappers to expose the data publically. Then, I'll write a few more WS apps that scrape data together from existing WS weather sources (of which probably a few exist) and traditional web sources (of which probably a bunch exist) and dump the data into the DB. The idea is that for a given day (say, November 15^th) you can query who predicted what weather on what days—on the 5^th, for example, Yahoo predicted a high of H₁ and a low of L₁ and that on the 10^th they revised that prediction to a high of H₂ and a low of L₂. Finally, I figure I'll throw together another quick-and-dirty WS app to expose the data in a human-readable format as a regular web page.

The explanation above is long and drawn-out, and doesn't really cover all the genuinely neat shit that can be done with these numbers. I don't think I'll have the statistical base to do anything completely sweet with it until it's been scraping data for at least a year—but that's OK. I figure assembling the apps will be the really fun part&mdas;especially since I plan on doing it using the Windows Communication Foundation and C#. Hot.

So there you go. That's the idea I've had. And if I don't get bogged down with other stuff that needs doing, I'll probably be implementing it within the next month or two.