Tuesday, February 16, 2010
Subscribe to:
Post Comments (Atom)
skip to main |
skip to sidebar
Maintained Pages
Search This Blog
Recent Comments
Blogroll
-
-
-
-
30 Years of Sea Level Rise2 days ago
-
Books in 20243 days ago
-
Quaternions and spherical trigonometry3 days ago
-
The Syrian Experiments6 days ago
-
-
Nature 2023: Part II2 weeks ago
-
Wind and Solar Can’t Support the Grid2 weeks ago
-
-
-
Climate risk2 months ago
-
A Measurement of Climate Sensitivity3 months ago
-
-
-
-
-
-
PIOMAS December 20195 years ago
-
Sea Level Rise Catastrophe ?5 years ago
-
-
72. Odd recent evolution of the QBO8 years ago
-
Posts
Now this is a work of art. I thought the original topic on WUWT was simply stupid, but it has inspired much new insight here.
ReplyDeleteWe see for example that Colombia maybe only updates twice a year, so if anybody gets worried about them the next few months, we now know that.
Meanwhile, Greece and France stay up-to-date, but have a track record of reporting very late in the month.
That 91 month station in Ivory Coast tells us that no station should ever be left for dead. You should tell EM Smith about it, though based on past form, he'll only try to make a conspiracy theory out of it. But by far most interesting are the backfills ending Aug/Sep 2009. Sudan is active, just lagging.
The negative Ethiopian stations are well-represented in the QC file.
I'm having trouble downloading it, but there looks to be an old archived v2.mean in the 'hold' directory. Could be worth a look?
ReplyDeletecarrot eater: "Now this is a work of art. I thought the original topic on WUWT was simply stupid, but it has inspired much new insight here."
ReplyDeleteThis does illustrate a principle of science...namely there are no really harmful questions. For all of the problems with his work, EM Smith has raised awareness of certain issues. As I've said on other blogs, his argument about latitude effect is a real issue, especially if you don't use gridded averages (or equivalent techniques to remove latitude bias).
It can be shown for CRUTEMP3v that post 1950 there is essentially no latitude effect by the addition or removal of stations, when one first bins by 5°x5° grid points. (There is a shift prior to 1950, which tends to accentuate warming over land, since warming prior to 1950 is not considered a signature for AGW, that isn't critical here, but probably the CRUTEMP3v data should be adjusted for it.)
Hey Nick,
ReplyDeleteNice work.
Have a look at the work Roman did on combining stations. He tried to post a question to tamino but Tammy would not take a question from a statistician. maybe you can look at both approaches and do a fair assessment?
http://statpad.wordpress.com/2010/02/13/rejected-again-open-mind-isn’t/#comment-261
CE,
ReplyDeleteThanks for the suggestion. I've downloaded it - it's dated 4 June 1998. It may not be very suited to the analysis done in this post; the interest will probably be that it represents the situation at the start of regular updating. Some may wish to check that the numbers haven't changed.
Carrick This does illustrate a principle of science...namely there are no really harmful questions.
ReplyDeleteYes, I agree, and though I don't like EMS's deductive style, he does burrow down into a lot of stuff.
Steven,
ReplyDeleteThanks for the suggestion. I read Roman's post. Probably T and R both know more about the topic than I do, but I'll see what I can do.
Carrick: That would be more of a point if any of the global temperature records didn't use spatial averaging and anomalies. But they all do.
ReplyDeleteSmith's obsession with cold locations is distracting from the real issue, which is undersampling. If you remove a station in a region where you already have undersampling, your error will go up. How this will affect the year-year variations or the trend at that grid point is hard to predict, though. Removing a cold-location station could just as easily cause a false cooling in the result, especially in the Arctic.
Finally, a question is harmless, but taking a half-baked thought, making a bunch of averages of absolute temperatures, publishing it someplace for the lay audience and publicising it is harmful, as it rather decreases the general understanding of the issue.
Nick, that's about what I had in mind. One could see what corrections or updates have been made to the pre-1998 data. I'd imagine at least a few.
ReplyDeleteAs for people complaining about comment moderation.. try getting a challenging comment through at EM Smith's blog.
Carrot Eater: Carrick: That would be more of a point if any of the global temperature records didn't use spatial averaging and anomalies. But they all do.
ReplyDeleteThe use of anomalies doesn't help much on this issue, because the temperature trend is affected by latitude, not just temperature offset. The temperature trend does increase as you move towards the North Pole, no way to get around that.
The spatial averaging algorithm does help: Since most of the variability is South-to-North, averaging over bands of constant latitude appears to be sufficient to remove most of the effect of station loss since 1950.
The effect prior to 1950 is real and probably should be modeled for and corrected:
Here's the effective latitude bias in CRUTEMP3v using their 5°x5° gridding method.
And here's the temperature trend as a function of latitude.
Removing a cold-location station could just as easily cause a false cooling in the result, especially in the Arctic
I think you're trying to argue that the variability in the mean temperature in a latitude band goes up as the number of stations is reduced. That is true of course true, but for the arctic there is no getting around the cos(latitude) weighting that reduces the importance of dense geographical sampling for that region.
In practice, dropping a station minutely increases the uncertainty in the ensemble average over all stations, but if the ensemble is weighted average using the grid method, it has virtually no effect on the global trend.
Finally, a question is harmless, but taking a half-baked thought, making a bunch of averages of absolute temperatures, publishing it someplace for the lay audience and publicising it is harmful, as it rather decreases the general understanding of the issue.
Critics, even poorly reasoning ones, are a fact of life. Railing about it doesn't change anything, except maybe to make your own arguments appear less credible to a neutral observer.
The issue is not whether people raise unreasonable or erroneous criticisms, but whether they are dealt with in a credible fashion by the scientific community.
Language such as you use is more dangerous than what EM Smith uses, because it sounds like you are advocating the stifling of debate. That is far more harmful to the scientific cause than anything he has every said. We need to encourage debate, not stifle it.
Steven,
ReplyDeleteI've looked more carefully through both the T & R posts, and I do think they are both right. Roman's comment does add something, so there's certainly no reason to suppress it.
Tamino takes a generic least squares view; forms the SS, differentiates and solves by matrix methods. I tend to do that too, and if you're comfortable with the process and solving methods, it may be the most natural and informative.
ANOVA is an elegant and powerful framework that statisticians are familiar with. If you express it as Roman did, a lot of thought processes fall into place, and you can use R functions. There's a lot to be said for that. The underlying arithmetic is the same, but you don't have to write the code.
So it comes down really to what you are most familiar with. I've been for long periods a mathematician thinking like Tamino surrounded by statisticians thinking like Roman, and I've learnt from them, as R says T should. But Tamino's way of thinking is effective too.
"The use of anomalies doesn't help much on this issue, because the temperature trend is affected by latitude, not just temperature offset. The temperature trend does increase as you move towards the North Pole, no way to get around that."
ReplyDeleteWhat? The use of anomalies makes EM Smith's argument ridiculous on the face of it. As you correctly identify, it is the temperature trend that matters. And there are stronger warming trends around the Arctic than elsewhere. This recognition is completely opposite to EM Smith's claim, which is that removing Arctic stations would cause a false warming, just because those stations are cold.
Carrot Eater: This recognition is completely opposite to EM Smith's claim, which is that removing Arctic stations would cause a false warming, just because those stations are cold.
ReplyDeleteYou are correct, of course...it would have the opposite effect post 1980 (when the number of stations started to decline), in that it would tend to impose a cooling bias on the real global mean temperature.
There is a bit of amusing irony here, isn't it? As I've pointed out, though, appropriately gridding the data mostly fixes that problem, post 1980.
There are problems prior to 1950, which I believe are still present in the current CRUTEMP3 program. These however mostly affect how much natural warming we would assume is present for that period [remember than the models suggest that anthropogenic CO2 was pretty much balanced by anthropogenic sulfates prior to the mid-1970s, AGW starts circa 1980, not 1850 as Smith appears to assume].
Fixing this problem decreases how much warming we observe prior to 1950, so the only real effect it would have would be to elevate the significance of the AGW warming observed since 1980. Ironically it would make it look more "hockey stick" shaped were we to fix the problem.
Hmm... never mind. I see no irony there. Nada.
Carrick: I have no idea how you have defined latitude bias, or indeed just what you mean by it.
ReplyDeleteIn any case, the further back you go, the bigger the error bars get due to undersampling, especially in the SH. And I wouldn't be surprised if there was still some mismatch in the old SST data, between buckets and whatever else.
carrot eater: Carrick: I have no idea how you have defined latitude bias, or indeed just what you mean by it.
ReplyDeleteIn any case, the further back you go, the bigger the error bars get due to undersampling, especially in the SH. And I wouldn't be surprised if there was still some mismatch in the old SST data, between buckets and whatever else
CRUTEMP3 is land only so bucket errors play no role.
I examined the mean latitude in the CRUTEMP3 using this formula. Here N(theta) is the number of grid points that are filled for a given latitude theta. This definition follows the way that the CRUTEMP3 algorithm works, which is basically to average over all non-zero grid points, weighting them by latitude.
Note that mean latitude bias should be 15.5°N (this is a result of more land-mass in the Northern hemisphere than Southern hemisphere). Since 1950, it is pretty much bang on.
The way GISTEMP does it---as I understand it at least---it should be immune to this bias, as it averages over each 5° latitude stripe, and further infills missing grid points. Changing the number of nonzero cells should have no effect on this algorithm.
In any case, the further back you go, the bigger the error bars get due to undersampling, especially in the SH.
ReplyDeleteAlso remember that undersampling can lead to a net bias in addition to larger uncertainties. The paucity of SH stations is the origin in the bias to high latitudes in the early data.
I don't know what CRU does, so I'll not comment on that. But it doesn't seem relevant to GISS, as you say.
ReplyDeleteI finally found what I've been looking for - Peterson published an illustration of what a station drop does, based on how you combine stations.
See Fig 2 in Peterson et al, 1998. GRL 103: 25967-25974.
I wonder if EM Smith is banging on about those discontinuities in the GISS method. Right at the point of the discontinuity, you get a spurious jump.
Again, it has nothing to do with whether the stations are from cold or warm locations. But you will get those jumps.
Carrick,
ReplyDeleteThere is a legitimate criticism of Smith that can't be dismissed as trying to "stifle debate" per se. It toughes on the larger issue of peer review and its diminishing role (at least for some) in the age of blog science. Specifically, I imagine that more people will read Smith's stuff (published by Watts and promoted by Heartland et al) than will ever read, say, Menne et al or the Peterson articles discussing the way global temp records actually work! With the power of making scientific claims comes the responsibility of ensuring basic quality control and review before making those claims public. There is a difference between a blog post and a "published report", and the latter should require at least a bit of review.
Oh, and if you guys haven't seen it, Chad's latest work is quite germane: http://treesfortheforest.wordpress.com/
ReplyDeleteAlong the lines of what Zeke said: Asking questions and having a discussion is one thing. Doing a half-hearted incomplete analysis, putting on an unjustified spin and giving unjustified conclusions, and then publishing it widely for the lay audience, is another altogether.
ReplyDeleteIn my view, people are smart enough to recognize spin when they see it. [This goes for everybody's writings, not just critics like Smith.]
ReplyDeleteAnd if Smith's work is as poor as you are intimating people can generally tell. Those who get "sold" on his work, had already made up their minds and aren't going to get confused by mundane things like facts. Remember that established scientists at major universities have a big advantage going into any debate in terms of credibility with the general public. As long as they don't blow it by shrill rhetoric and half-cocked arguments.
"In my view, people are smart enough to recognize spin when they see it."
ReplyDeleteHardly. Just for example, look at all the people parroting the line that "Jones admitted global warming stopped in 1995" or whatever. They don't stop to think, or go back and see what the original quote was. They just run with the tabloid spin.
The average person has never opened up an issue of GRL or Nature. They just pick up some rough ideas from whichever websites or newspapers they look at. So the tabloid spin is the flavor of what they're getting.
"Remember that established scientists at major universities have a big advantage going into any debate in terms of credibility with the general public."
Hardly. The opposite is actually true among a good chunk of the American populace, at least. There is some suspicion of the ivory tower and the so-called 'elites', and you can see the effort the sceptic movement takes to convince people that peer review, grants and other aspects of academic life are somehow corrupted. You see this across a range of movements, actually. Once you decide science is just a big conspiracy, it's easy to just wave away whatever you don't like.
IN any case, what we are seeing from EM Smith and the like (say, pages of charts of absolute temperatures averaged together) are simply going to earn him a poor reputation. Perhaps he's OK with that; I don't know.
Nick: This fine fellow has made a map of these data.
ReplyDeleteI really need to write a script that maps stuff. It is really useful.
http://climatewtf.blogspot.com/2010/03/stations-reporting-in-past-month.html
"In my view, people are smart enough to recognize spin when they see it."
ReplyDeleteIf that were true, politics and sales would be completely different than they are. If spin weren't effective, who'd bother with it?