Update
I've developed the diagram of this post into an interactive trend-viewer here.With the conversation about the new BEST data, there has been an uptick in the posts showing plots from WoodForTrees with trends proving something or other. Here is a recent duel.
So I decided to expand the plots from the previous post to maybe make all this redundant. You'll see a plot of all the popular time intervals, with trends presented in a colorful array. Now all you have to do is spot the color you like, and look up the endpoints on the axes. Or if your disposition is wonkier you can try to spot patterns in the way the trends vary.
Some math details - I've taken each of nine indices and done an OLS regression of the monthly temperature against time for each pair of start and end points an various ranges. No prior smoothing, no annual averaging. There is a little colored square indicating the trend, in °C/century, for each pair. I haven't included periods shorter than four years. I've used rainbow colors, except for two ranges which help to calibrate by eye. These are zero trend, in gray, and 1.7°C, representing a recent average, in brown.
The plot itself is an interactive compendium of 36 images, for the 9 datasets and four time ranges. The times are from 1999 to present, 1989 to present, 1960 to present, and 1901 to present. They are presenting different fragments of the same trend array. I chose 1999 to present a popular contention at the moment that over a decade the trend is zero or less. For the land/sea indices there are certainly ranges for which that is true - I think this plot gives a fuller picture.
To use the interactive aspect, you can click on either of the legends in red to get any combination of index and time. Your current choice will be shown in black. As with all these plots that I do, the images are downloaded when requested, so when you first click there is a short pause - however, when you revisit that image, it's now in cache.
Below the jump, I'll say a bit more about some stationary images.
Update - I see now that {'ve posted that the interactive part doesn't work in IE (this seems to be an issue of security settings). It works in Firefox, and I expect that it will work in Safari and Chrome. I have added at the bottom of the post a table of links to the still images.
Firstly, a few more details about the plots. The faint diagonal lines show the trend period, as enumerated by the axis on the right. I don't go below four years, and part of the purpose of this presentation is to show how noisy short term trends can be. You'll see that the BEST plot stops in March 2010 - I did not include the data-deprived months of April and May, and there is no data for later months.
The data sources for the plots are:
- HADCRUT3
- GISS Land/Sea
- NOAA
- UAH Satellite
- MSU-RSS
- CRUTEM3
- GISS MET stations
- BEST all land monthly averages
Here is the GISS Land/Ocean plot shown over a 20-year range. This image is not interactive. You can see some horizontal and vertical patterns. The vertical stripe at 2008/9 shows that this was a relatively cold year, and so trends that terminate there are likely to be negative, There was indeed a flurry of trend-watching at that time, and this plot shows why. Remember gray is approx zero trend, so the short-term trends based on 2008 go negative.
Another strong feature is the 1998 hot year, showing as a horizontal stripe. That stripe has lower trends, because the peak sits at the start of the trend period. One would expect the vertical bar below to have markedly positive trends, and it doesn't. I think the reason is that there is another feature - the positive horizontal bar produced by the cold Pinatubo years around 1993. The result is positive, but the pattern by eye goes with the horizontal rather than vertical.
Here's HADCRUT3 over the century. The longer time shows how the noise gathers on the shrt-term diagonal, with more sustained patterns as you get away from that. Right in the middle there is a big zero to negative trend region corresponding to the post-1940 cooling. The warm '30's show out as a vertical bar. And towards the top right, the shorter term recent segments have a strong tendency to be positive.
Well, I could go on, but I think this may help explain how to use the plots. So click away!
Here is a table of links to the still pictures:
| Time | HADCRUT | GISS LO | NOAA | UAH | MSU-RSS | TempLS | BEST | GISS Ts | CRUTEM3 |
| 1999- | Link | Link | Link | Link | Link | Link | Link | Link | Link |
| 1989- | Link | Link | Link | Link | Link | Link | Link | Link | Link |
| 1960- | Link | Link | Link | Link | Link | Link | Link | Link | Link |
| 1901- | Link | Link | Link | Link | Link | Link | Link | Link | Link |
Posts
I think the proper technical term to describe this kind of post is "awesome". :)
ReplyDeleteYes, it looks like the sort of thing which would be very informative for the non-chromatically challenged :P
ReplyDeleteKevin C
p.s. Only joking. Don't waste time doing a mouseover version for me.
I agree with Toto and Kevin C, very nice job.
ReplyDeleteThanks, guys. Kevin, that's actually a very good idea. To make the left legend the active one, so a particular level stands out. Helpful for everyone, not just with color issues. I could only do one index, though.
ReplyDeleteI did something similar back in September, only using start year and length-of-trend instead of start and end year. I think our methods are almost identical, but the results get tilted by 45 degrees. These plots are indeed a great way of picking out cherry-picks. I'm not sure which plot is more intuitive.. I think your way of plotting is more useful for picking out dates that have an outsized influence on trends, but mine is a bit more useful in seeing that trends average out to warming over longer time periods.
ReplyDeleteExample
Cheers, and nice work,
----Paul from VA
Paul,
ReplyDeleteYes, that's well done. For many purposes your combination of variables is best. There are three things of interest - start, end and duration, and you can put any two of them on the axes and the third ends up as 45 deg lines. In my previous post, which got me started on this, I had end date on the x-axis and duration on the y-axis.
I thought here that end date would be on people's minds, so should go on the x-axis. Duration on the y-axis has the advantage that for the shorter time ranges, you don't need the triangle cutoff and can use the whole space. But I needed space for the legends anyway ;)
I just downloaded the "BEST all land monthly averages" with the link on this page. I have searched for these BEST temperatures for some time, and I am happy to have found them on your blog. On the Berkeley web page I have only find the temperatures for all the stations; much data that need to be processed before giving any meaning. Have you received your temperatures from Berkeley, or have you processed them yourself based on the stations temperatures. If you have processed them yourself, why does not Berkely make the temperatures available in the same human-friendly way as you have done ?
ReplyDeleteHans,
ReplyDeleteYes, the BEST web site is not well set out for finding things. I used just the monthly averages compiled by BEST - the file called Full_Database_Average_complete.txt in analysis-data.zip.
I have downloaded the big data set, and I'm planning to put it through TempLS, which is my least squares program. I've been holding off because the data has been de-seasonalised in some unexplained way. It shouldn't affect my analysis, but I'd like to know. Also it's about the time of month when I usually download the new GHCN and SST data to produce the October mean and plots, tho this month I may be late.
Robert Rohde of BEST was noted for his work with GlobalWarmingArt, so I'm expecting there will be better graphics from the team when he gets time.
Nick,
ReplyDeleteThe de-seasonalising in the current Preliminary data seems to be work in progress.
The current released analysis code simply discards data that seems not to be properly processed (too high std deviation):
(from BerkelyAverageCore.m)
%%%%%%%%
% Temporary Fix: Remove ultra-high variability created by bad seasonality
% procedure on sparse data. This will be removed in the future versions
% when the seasonality procedure is fixed.
parfor k = 1:length(se)
data = se(k).data;
if std(data) > 7
orig_map(k) = false;
end
end
%%%%%%%%
A year gone by and Steve still has not redone his messed up calculations on MMH. Or opened the thread where he locked comments after finding he messed up.
ReplyDeleteI'm a little disappointed I've missed this until now. Great work Nick!
ReplyDeleteThanks, Tim
ReplyDelete