tag:blogger.com,1999:blog-7729093380675162051.post489501139116584697..comments2024-03-28T13:56:47.604+11:00Comments on moyhu: USA Temperatures; comparison of Moyhu results with NOAA.Nick Stokeshttp://www.blogger.com/profile/06377413236983002873noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-7729093380675162051.post-22214671124641002792020-11-11T08:39:16.836+11:002020-11-11T08:39:16.836+11:00Andy,
The source data directory is here. They use ...Andy,<br />The source data directory is <a href="ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/" rel="nofollow">here</a>. They use the GHCN Daily station set; they don't seem to make a separate file for that. I know only of the ascii data in that directory for divisions. I haven't dealt with that divisional data.<br />Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-16686062242375343292020-11-11T06:22:09.313+11:002020-11-11T06:22:09.313+11:00Do you know if the nClimGrid data is available in ...Do you know if the nClimGrid data is available in NetCDF? Or is it just ascii?Andy Mayhttp://andymaypetrophysicist.comnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-37691410239091731672020-11-10T04:06:18.919+11:002020-11-10T04:06:18.919+11:00Yes, this is Andy, sorry I didn't realize I co...Yes, this is Andy, sorry I didn't realize I could change my name in the reply box. Thanks for the quick response. It does appear that at some point they started drawing data from GHCN-D, do you know when that happened? The earlier GHCN-M record is almost identical to USCHN. I need to check and see just how close they are. maybe, I can pick a time where they begin to diverge. I sure would like to see a history of what was done and when. The documentation I've been able to find is confusing and often contradictory. My working assumption is the contradictions are because different portions of the record are done differently. But, maybe you are correct, maybe they redid all the old data from scratch. I need to dig deeper.Andy Mayhttp://andymaypetrophysicist.comnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-30509446468628351492020-11-09T10:42:28.146+11:002020-11-09T10:42:28.146+11:00Hi Andy (I presume)
First, sorry that I've had...Hi Andy (I presume)<br />First, sorry that I've had to put moderation on the older posts. I am beset by a very persistent spammer. But I'll watch out for your comments.<br /><br />As you mentioned <a href="https://wattsupwiththat.com/2020/11/06/comparing-uscrn-and-nclimdiv-to-uschn/#comment-3120187" rel="nofollow">here</a> GHCN-M comes with two current files - QCU and QCF. QCU are unadjusted (by GHCN). They may have been adjusted at source, but that is rare, since the source is usually a met office just recording what was observed. US data would be unadjusted. qcf is adjusted by the pairwise algorithm PHA. That doesn't include an explicit TOBS adjustment, but relies on PHA to pick up changes.<br /><br />GHCN-M was originally mostly derived from a project <a href="https://moyhu.blogspot.com/2012/10/new-isti-temperature-database-stations.html" rel="nofollow">ISTI</a>. GHCN-D was a somewhat separate development. Not all known monthly records have corresponding reliable daily data. I think for new data the two are integrated, and I'm sure all GHCN-D records that are good enough are included in GHCN-M.<br /><br /><i>"Is that your understanding? What a mess."</i><br />No, it isn't a mess. The primary data is the unadjusted, which would correspond to USHCN raw in that dataset. Everything GHCN does works from that, so there is no TOBS to be undone. They record the unadjusted (which doesn't change) in QCU, and put into QCF the current PHA calculation of the adjustments, going right back through the record (same algorithm for all years). That is why you read complaints about adjusted records changing.<br /><br />In fact GHCN is fussy about not changing QCU. The original V1 was distributed on DVD, and you can check that numbers still haven't changed. They won't even change numbers that are obviously wrong; there are some Bolivian months that show about 80°C, obviously a decimal point dropped. They flag it, but don't make the obvious change. They will only change if there is a change at source.<br />Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-76062936793248897042020-11-09T07:52:56.921+11:002020-11-09T07:52:56.921+11:00Hi Nick,
Thanks for sending me to your web site. I...Hi Nick,<br />Thanks for sending me to your web site. I'm enjoying reading what you've done, it helps me a lot.<br /><br />I'm very confused about what is actually done to the data in GHCN-D and GHCN-M and I strongly suspect the historical data is not consistent from beginning to end. It seems the later data in GHCN-Daily has very few corrections applied, is that true of the earlier data as well? GHCN-M takes its data from GHCN-D now, but the older GHCN-M data is from multiple sources. GHCN-M is homogenized, presumably with PHA, but GHCN-D is not. No mention of TOB corrections. Were they done to earlier data, then they stopped doing them? Or were they removed from of the older data? The documentation is pretty bad and often inconsistent. My interpretation of the inconsistency is that the settled older data, along with whatever was done to it, is left alone. The newer data follows the procedures they now have on their web site. Is that your understanding? What a mess.Anonymoushttps://www.blogger.com/profile/13588306783949399516noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-20803818785540846942020-03-07T05:39:45.708+11:002020-03-07T05:39:45.708+11:00Thanks, StevenThanks, StevenNick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-2733764335101476452020-03-06T23:52:28.413+11:002020-03-06T23:52:28.413+11:00https://journals.ametsoc.org/doi/full/10.1175/JAMC...https://journals.ametsoc.org/doi/full/10.1175/JAMC-D-13-0248.1<br /><br />Just for reference, you are correct abt climdiv<br /><br />"GHCN-Daily temperature records do not contain adjustments for historical changes in observing practice. Consequently, bias adjustments were computed specifically for version 2 to account for changes in observation time, station location, temperature instrumentation, and siting conditions. The first step in this process entailed using the method of Karl et al. (1986) to address documented changes in observation time at COOP stations and to adjust the records to a midnight local standard time (LST) observation schedule (matching ASOS, RAWS, and SNOTEL). COOP station histories were obtained from the NCDC Historical Observing Metadata Repository (HOMR) and the U.S. Historical Climatology Network (HCN; Menne et al. 2009). The second step in the adjustment process involved using the “pairwise” method of Menne and Williams (2009) to address all other documented and undocumented changes at any station in any network. For example, the pairwise approach was applied to account for documented changes in station location and temperature instrumentation at COOP and ASOS stations, again using HOMR and HCN for station history information. Likewise, the pairwise approach was used to address undocumented changes in observation time, station location, and temperature instrumentation across all networks, including COOP and ASOS. Because the pairwise method largely accounts for local, unrepresentative trends that arise from changes in siting conditions (Menne et al. 2010; Hausfather et al. 2013), version 2 contains no separate adjustment in that regard."stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.com