Tuesday, April 6, 2010

Ver 1.2.1 - Various results

20 comments:

  1. Cool, Should I just download a new version?

    ReplyDelete
  2. opps I see the bug fixes..

    Anyway I did a fresh download of R since I was a couple versions out of date. Will apply your fix and revert.

    ReplyDelete
  3. Those '90/91 values in the dropped series look suspicious. None of the rest of us got anything quite that dramatic.

    ReplyDelete
  4. Zeke,
    Yes, I think the transition to zero stations needs more work. It's possibly a weakness with this implementation of least squares. A graph point which is determined by only a few stations will be relatively lightly weighted, unless the weight function is varied.

    The weighting takes account of spatial sparsity (through the grid) - maybe something similar is needed in time for series like this.

    ReplyDelete
  5. Zeke,
    I think now the issue is more that as the plot winds down, there's dependence on very few stations, so what you see depends a lot on the particular year you stop. I've included an update above.

    ReplyDelete
  6. Ahh, its also in part because most of us (myself, Tamino, etc.) used 1992 instead of 1994 as the cutoff, so we didn't have those two years.

    ReplyDelete
  7. Just incase there are other curious Canadians lurking here:

    The Canadian Province abbreviations are a total mess. ie there were more than BC & B.C including, off the top of my head- ,space BC; B; nothing at all, B.C. and maybe even BC.

    Many stations had no province but had CANADA.

    So beware, if any one else is doing this.

    ReplyDelete
  8. Have you tried cities > 2 million people. It'd be nice if someone confirms my UHI result :)

    ReplyDelete
  9. Joseph - not yet. I'll put it on the list for the next run.

    ReplyDelete
  10. Joseph: there is always a danger of having spatial coverage issues predominate when you get too selective a station set. E.g. the low pop density stations in http://rankexploits.com/musings/wp-content/uploads/2010/03/Picture-78.png

    ReplyDelete
  11. @Zeke: Sure, but I've actually looked at disjoint population size groups, e.g. 10 million to 12 million, 12 million to 15 million, etc. There's a statistically significant temperature slope trend once you pass the 1 million mark.

    It would be difficult to explain that regression trend as a regional bias.

    Details here.

    ReplyDelete
  12. Joseph,
    I've put up a new post with BigCity (>2M) and MedCity (>0.5M) results. I'll post the numbers at the document store

    ReplyDelete
  13. Hmm, is conus in the XLS file?

    ReplyDelete
  14. The ConUS numerical results are in the file
    Output_2010-04-06:12:50_Global_ConUS_Cut_Kept_BC_Africa_.txt
    in All-Files.zip, under dir GHCN Glob Average v1.2.1

    I'll update those zip files soon.

    ReplyDelete
  15. ok,

    I just commented out the graphics and ran the code. still no word from the sig on the X11 problem. anyways. It matches GISS nicely correlation about .97. A bit of weirdness out in the past few years.

    Zeke should put together a US comparsion. maybe joseph would contribute as well

    ReplyDelete
  16. A U.S. comparison of what exactly?

    ReplyDelete
  17. We have Nick's version of Conus, your version, GISS, add josephs..
    maybe you guys could convince tamino to do one, I'm working on asking Jeff.. or In a pinch We could subset v2.mean to just Conus and feed it to jeff's algorithm rather than moding his code.

    By doing that we come pretty close to comparing apples to apples.
    GISS do some fiddling with GHCN and USHCN and drop some US stations entirely.

    In the end what you have is a "proof" of the claim ( which we all know to be largely true) that the various methods of averaging produce the "same" answer.

    ReplyDelete
  18. Mosh,

    Given that the U.S. is much smaller than the globe and pretty densely sampled with mostly complete records, it really shouldn't matter much what gridding method and anomaly calculation method you use. It will make a difference if you use GHCN or USHCN, and what version you use (e.g. USHCN F52/GHCN v2.mean_adj vs. raw versions) since the U.S. has a known cooling bias in the raw data (from TOBs and MMTS at least).

    ReplyDelete
  19. Here is an old image of mine that shows GISS and my results for GHCN raw, GHCN adj, USHCN raw, and USHCN adj:

    http://rankexploits.com/musings/wp-content/uploads/2010/03/Picture-112.png

    ReplyDelete
  20. Ya Zeke, I see that now, If you get a chance to post an Xls that would be great.

    I imagine then checking other parts of the world. for example africa which Nick has done

    ReplyDelete