Wednesday, November 20, 2013

Pausebusting NOAA


In previous posts here, here and here, I have been discussing issues in infilling cells in the HADCRUT 4 dataset, following a new much-discussed paper by Kevin Cowtan and Robert Way in QJRoyMetSoc. They used satellite data and kriging to extend the data; I looked at a much simpler approach of infilling with latitude averages from the same data (rather than the implicit infilling with global average). I found that it gave rather similar results, although with more limited effect.

Now I want to try it with the similar NOAA gridded data set which is used for their index. They give a detailed description at this download site. It is a similar 5°x5° grid, so the same analysis should work.

I show below first plots of both HADCRUT 4 and NOAA coverage. NOAA has greatly fewer gaps, but also fairly extensive omissions near the poles. Then I show the latitude breakdowns, with seasons. I have included the C&W and UAH plots for comparison.

Coverage plots

This is an active plot - you can choose dataset and view. It counts data in each cell/month from 1979 to 2012. Red is for no data, cyan is complete, and there is shading for intermittent data.


NOAA N
NOAA S
HAD 4 N
HAD 4 S

NOAA coverage is more extensive than HADCRUT, with some odd inclusions (eg central Greenland) and omissions (S Pole). Still, I found using the latitude average method described in previous posts with smoothing parameter r=0.1, that I had to include an extra level of bands to get enough data for lat 85-90S.

Latitude average trends

This is a similar active plot to the one in the previous post. NOAA data replaces HADCRUT 4, and I have increased the width of those curves, since they are new. UAH and the C&W hybrid HADCRUT 4 result are included for comparison.


Annual
DJF
MAM
JJA
SON

(Update - again I made an error with latitude sign (the problem is that HADCRUT orders from N Pole, NOAA from S) So NOAA were reversed, Fixed now.)
The background stripes indicate the latitude bands; N pole is on the right. There's a faint red vertical marking multiples of 30°. In the centre is a little figure showing the global totals, on the same scale.

Here is a table of the global trends, 1997-2012 in C/decade:






SeasonNOAANOAA AvHADCRUT 4HAD 4 Lat AvUAHC&W Hybrid
DJF-0.0519-0.0049-0.04890.00970.02760.0536
MAM0.06880.11230.0660.10190.06390.1276
JJA0.06290.07890.08920.09660.120.1187
SON0.10840.13470.10810.12770.15750.1688
Annual0.04850.08140.05390.08440.09360.1187

The results of NOAA and HADCRUT 4 are very similar, in effect on trend and in seasonal variation. Both show a substantial trend increase in trend over this period, as a result of simply using the appropriate latitude average to infill rather than the global.


5 comments:

  1. NOAA does not per say have "greater coverage" than HadCRUT4 - simply at mid-to-low latitudes they infill.

    ReplyDelete
    Replies
    1. As I understand it, they use an optimal interpolation scheme to infill. This is certainly more desirable than leaving the cell out (which is equivalent to infilling with the global value).

      So technically not "greater coverage" but certainly "better treatment".

      Delete
  2. Interesting. Do I interpret you plots right that for HadCRUT the interpolation affects the winter strongly, whereas in NOAA it affects all seasons similarly. Such differences show the dangers of interpolation. I am glad that C&W made a careful cross-validation.

    ReplyDelete
    Replies
    1. Oops, no, I thin k I reversed the latitudes again. I'll fix. The underlying problem is that HAD orders from N Pole and NOAA from S, and when I try to use the same program for each...

      Delete
    2. Fixed now - the problem was that I originally applied a correction to the C&W files to bring S Pole to the left, and wrongly applied it to the NOAA files as well. So the NOAA files only were reversed. There's a big correction spike in winter for NH, and also a substantial one for SH winter.

      Delete