Tuesday, February 7, 2012

A study of GHCN V3 homogeneity adjustments.

Invisible Pin
Yellow Pink Cyan
Selector
Clear
StartYr
EndYr
Duration
Altitude
Trend Adjustment C/Cen
Urban
Airport
All
Pause:sec
Start:  End:

11 comments:

  1. Its worth noting the the breakpoints are detected by a comparison to nearby stations. Those stations that show irregularities post-homogenization are likely cases where the irregularities show up in nearby stations as well (and may actually be a signal). Similarly, inhomogenities in stations with no nearby counterparts will generally not be detected.

    ReplyDelete
  2. Nick, good to hear you have done this.
    I started writing a code to look through all the stations systematically but it's quite fiddly and I haven't had time.
    I disagree with your final point. No reading is required to see that the Iceland adjustments are wrong.


    I think Zeke needs to read the previous comments about the Iceland adjustments.
    There was a sharp cooling in 1965 in all the Iceland raw data sets.
    This was deleted in almost all cases by the erroneous GHCN adjustment algorithm.

    PaulM

    ReplyDelete
  3. Zeke,
    Yes, and that would affect the interpretations here. But I think you do have to have a station irregularity first - then it is checked with neighbors.

    One of the objections raised about Iceland was that in about 1965 several of them were corrected in a similar way. It wasn't easy to see that as a neighbor-forced thing. I think that is what Paul is referring to.

    Paul, I don't know if R code helps but I'll try to scrub mine up and get it online.

    ReplyDelete
  4. Paul,
    My point is, though, that a statistical evaluation is what is needed. There is a cost in noise to correcting apparent bias, as here. This may well be a false positive. The objective is to ensure that the noise introduced is less harmful (biased) than the noise removed.

    ReplyDelete
  5. Nick,

    I meant that neighbors are used to detect irregularities in the first place (and later correct for them). Metadata is also used if available, though it simply reduces the threshold for breakpoint detection rather than forces a breakpoint, given that metadata can be wrong.

    PaulM,

    I was simply pointing out how the Menne and Williams algorithm detects breakpoints. I'll read over your prior Iceland comments, but if multiple stations in the region show the same behavior at the same time it is unlikely that they will be corrected.

    ReplyDelete
  6. Zeke, this is exactly what the fuss is about! If multiple sites in the same area show the same behaviour, then clearly they shouldn't be "corrected". But this is exactly what happens in the GHCN adjustments for Iceland. As well as the 1965 issue, there is the consistent warm period around 1940 where temperature was similar today, apparent in all the raw data and in the literature. But the GHCN adjustments put in a cooling here. See the sequence of posts at Paul Homewood's blog.
    There is something wrong - either the adjustment algorithm is 'over-zealous' or there is a coding error.

    I have the Lawrimore et al paper if anyone wants a copy.

    Paul

    ReplyDelete
  7. A noted example occurred in late 2009, when Willis Eschenbach became excited about GHCN V2 adjustments to the temperature at Darwin.

    Just wait until he discovers the upward trend adjustment at WILLIS ISLAND (off the northeast coast of Australia).

    Ned

    ReplyDelete
  8. Nick,
    GHCN have changed their version!
    A comment in the Changelog file says

    ********************************************************************************

    02/07/2012

    GHCNM v3.1.1 is released with changes in scripts due to asynchronous execution
    problems. Also added ability to add new stations automatically when their
    period of record increased enough to process and improved testing when not
    enough neighbors to estimate adjustments.

    ********************************************************************************

    I'm not sure what 'asynchronous execution problems' means.
    Anyway the adjustments are now significantly different, but no better.
    The erroneous 1965 adjustment now appears in all 8 iceland stations (before it was only 7/8)
    and the fabricated warming in Reykjavik is worse.
    But some stations eg Stykkisholmur are better.

    The warming adjustment at Corona NM is now "only" about 4 degrees rather than 5.
    Can you download the new dataset and run your code again to see where the biggest adjustments are now?

    Paul

    ReplyDelete
  9. Thanks for the note, Paul. Yes, I'll re-run.

    ReplyDelete
  10. . . . Mean 0.0196 °C/Decade . . .
    That's a tad high, if you've only got +0.7°C over the last century or so. I think they're over-adjusting there.

    ReplyDelete
  11. Anon,
    Yes, the adjustment does have some upward effect, and the updated algorithm (next post) raises it a bit more. But it's nothing like the impression you get from these extreme examples.

    The thing is, it may be right.

    ReplyDelete