tag:blogger.com,1999:blog-7729093380675162051.post9020064088679527448..comments2024-03-28T13:56:47.604+11:00Comments on moyhu: GISS UHI adjustmentsNick Stokeshttp://www.blogger.com/profile/06377413236983002873noreply@blogger.comBlogger44125tag:blogger.com,1999:blog-7729093380675162051.post-33920010022153081012010-05-13T08:21:43.979+10:002010-05-13T08:21:43.979+10:00Dear Anon,
No, it's Matanuska. The misspelling...Dear Anon,<br />No, it's <a href="http://en.wikipedia.org/wiki/Matanuska-Susitna_Valley" rel="nofollow">Matanuska</a>. The misspelling is in the GHCN list.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-77947164367124063462010-05-13T05:40:26.000+10:002010-05-13T05:40:26.000+10:00FYI: Its Mantanuska, with an "Mant" not...FYI: Its Mantanuska, with an "Mant" not a "Mat". Typos can be frustrating for those of us who go back to look at the raw data and fail to find the station... =)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-30808910304120092382010-03-29T09:53:16.536+11:002010-03-29T09:53:16.536+11:00Not to beat a dead horse but there are also rules ...<i>Not to beat a dead horse but there are also rules for throwing out the two legged fit and doing a simple linear fit.</i><br />Yes, I think such a rule could be applied here. It doesn't actually matter when you aggregate. The linear fit that you force should have the same contribution as the two-legged. But it looks better.<br /><br /><i>BTW, you should post up your stuff or link to it on jeffid temp page</i><br />Yes. I'm currently working on a coupled global trend method. If that works. I'll post and then link everything.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-55676085849518484442010-03-29T06:07:57.842+11:002010-03-29T06:07:57.842+11:00Nick,
Not to beat a dead horse but there are al...Nick,<br /><br /> Not to beat a dead horse but there are also rules for throwing out the two legged fit and doing a simple linear fit.<br /><br />BTW, you should post up your stuff or link to it on jeffid temp page<br /><br />http://noconsensus.wordpress.com/2010/03/28/new-surface-temperature-page/stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-32521126609679212032010-03-29T06:06:30.500+11:002010-03-29T06:06:30.500+11:00Eli,
I've read Imhoff 97 and I'm not the...Eli,<br /><br /> I've read Imhoff 97 and I'm not the least bit convinced that nightlights serves as a better indicator of urbanity than population and population is not 'all that' as well. <br /><br />Lets just start with basics. UHI is a meso scale phenomena. You dont rule it out by looking at the pixel that is "coincident" with the station.<br /><br />1. The station locations are demonstrably wrong in many cases.<br />2. The UHI feild meso scale.<br /><br />You could I suppose use Nightlights to ELIMINATE sites. but you can't reliably use it to QUALIFY sites as rural.<br /><br />A good qualification will involve a manual check of every station.<br /><br />1. Check the population<br />2. check the nightlights<br />3. check the urban density.<br />4. look at the satillite photo<br />5. look at the land use ( global records go back a long way)<br />6. Look at the "land under irrgation" satillite products.<br />7. Look at the Spot 3D products.<br />8. look at the site photos.<br /><br />Detailed work.stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-87473766368256962112010-03-18T13:21:46.868+11:002010-03-18T13:21:46.868+11:00Steven, the reason they did it first in the US is ...Steven, the reason they did it first in the US is they had good demographic data to cross check it against. It's in one of their papers (2001??) In other words cautionEliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-62228963722422317312010-03-13T18:40:07.333+11:002010-03-13T18:40:07.333+11:00CE
Where is the GISS formula for calculating t...CE<br /><br /> Where is the GISS formula for calculating the error due to spatial coverage? Trying to get my mind wrapped around the problem..<br /><br />Eli,<br /><br /> ya we know. It was one of those things that dopes like me asked stupid questions about "why does nightlights only get used in the USA where you have the best population data. made no sense, since nightlights was supposed to be a proxy of density.<br /><br />Still, this is a really cool problem. climate science will NOT change, but its a really cool problem. Glad folks are looking at it. <br /><br />Ideally, we would fund<br /><br /> 1. site surveys<br /> 2. SPOT aquistion. just do a 3D DTM. cost is not that bad.<br /><br />50 years from now it will come in handy.stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-37728431925234128972010-03-13T13:49:06.101+11:002010-03-13T13:49:06.101+11:00It's worth pointing out that until Jan 16 this...It's worth pointing out that until Jan 16 this year night lights were only used to identify urban and somewhat urban sites in the US.EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-90397643141097414652010-03-13T12:56:00.357+11:002010-03-13T12:56:00.357+11:00GISS does it too, using a GCM. It's where th...GISS does it too, using a GCM. It's where the green error bars come from. NCDC also uses a model field, I think.<br /><br />ISH is an interesting alternative. I wonder if it has enough density in the SH though to serve as a perfect field. Large chunks of Africa barely bother to issue SYNOPs, so that may not help, either.carrot eaternoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-87826264650289482892010-03-13T10:35:41.070+11:002010-03-13T10:35:41.070+11:00Nick,
I think we agree. personally I think I'...Nick,<br /><br /> I think we agree. personally I think I'd approach the problem by selecting the tightest screen I could to eliminate urban stations. Just to avoid all the crowing about stuff that in the end doesnt matter much.<br /><br />As CE notes that is going to give you ( perhaps) a noisey product, but<br />for somepeople you simply cant have enough stations ( I hate those nonsense arguments.. arrg)<br /><br />Then I suppose after you have these golden stations you can start the process of adding more stations.. There is no point in oversampling the US. <br /> <br />Nick, In the past at CA folks have started to discuss the jones approach<br />to calculating a error due to spatial coverage. I pointed Roman at some stuff from Shen. <br /><br />That might make an interesting post.. How many stations are required?<br />or rather error due to spatial coverage versus number of stations..<br />although number of stations is not the right term. <br /><br />Shen's paper looked at a couple things ( dont trust me) one was doing<br />and estimate based on GCM and the other was an estimate based on <br />decomposing hadcrut.. I think it was something like looking at EOFs<br />needed to capture the global climate feild.. <br /><br />Anyway with the ISH database I'm wondering if that could be used to<br />estimate the number required..<br /><br />neat problem.stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-48472829232755863472010-03-12T22:50:33.064+11:002010-03-12T22:50:33.064+11:00it'd be more interesting to look at this in a ...it'd be more interesting to look at this in a data-sparse part of the world. leaving the urbans in lets you keep the short term variations; this is already redundant anywhere in the US but maybe someplace else it helps reduce the noise a bitcarrot eaternoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-17873244980463122672010-03-12T19:39:57.056+11:002010-03-12T19:39:57.056+11:00Steven:
Anyways, I'm not sure what the urban a...Steven:<br /><i>Anyways, I'm not sure what the urban adjust gets you except an additional data point which really...</i><br /><br />Yes, that's my view too. They might as well just leave the urban stations out. Especially in Cali, where there are plenty of rurals.<br /><br />They're saying, let's take out the urban trend and try to salvage the rest. I think that salvaging does no harm, but no real good.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-68825805510270007262010-03-12T16:49:45.716+11:002010-03-12T16:49:45.716+11:00here's One problem with Orland. A couple mile...here's One problem with Orland. A couple miles to the south of the USCHN site is a another site. A site that records data hourly. It's in a field.<br />A few miles north of the USCHN station is another hourly logger in a field.<br />and then 10s of Km away is a third hourly logger in a field.<br /><br />From 1987 to today I can compare these 4. correlations on the tmax (97+%) Tmin(97+) and Tave 98%+ the trends for all 4<br /><br />gerber, durham, orland1, orland ushcn are all within .003F per month.<br /><br />Anyways, I'm not sure what the urban adjust gets you except an additional data point which really insn't a sample of the temp. but a munged piece of what ever you wanna call itstevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-52494986344305250922010-03-12T12:08:43.192+11:002010-03-12T12:08:43.192+11:00"yet, it gets cooled by comparing its record ..."yet, it gets cooled by comparing its record to the record at catalina island!"<br /><br />But this is what I'm talking about. Regardless of the correctness of your diagnosis of what the adjusting stations were, you're mesmerized by the mechanics of the thing. Which makes you forget that the mechanics are essentially tossing the station out. <br /><br />Now, there could be times where the two-legged thing doesn't do a good job of canceling the station. But you aren't even saying that here.carrot eaternoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-15968681568913638342010-03-12T08:00:11.880+11:002010-03-12T08:00:11.880+11:00it gets cooled by comparing its record to the reco...<i>it gets cooled by comparing its record to the record at catalina island!</i><br />It probably didn't. I haven't looked up the criterion GISS uses to switch from 500 to 1000 km, but they almost certainly would have found that 500km gave enougfh stations.<br /><br />And even at 1000km, catalina would have very little weight in the average,Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-39571057439453951192010-03-12T07:48:05.152+11:002010-03-12T07:48:05.152+11:00Carrot WRT2
Im talking about a situation like Or...Carrot WRT2<br /><br /> Im talking about a situation like Orland. Its 137 people per sq km.<br />by that measure a small town. By nightlights dim. By inspection<br /> probably rural/small town.<br /><br />When can be fairly confident that prior to 1960 the area was more rural<br />as the town population ( today 7k) was less than 3K ( 6.6sqkm)<br /><br />And definately prior to 1940 it was rural as the census shows electrification at less than 100%. and at the turn of the century with less than 1000 people its even more rural. yet, it gets cooled by comparing its record to the record at catalina island!<br /><br />not sure how you fix this problemstevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-74076744213770109582010-03-10T00:23:07.951+11:002010-03-10T00:23:07.951+11:00"1. does nightlights pick out rural/small tow..."1. does nightlights pick out rural/small town/urban with More accuracy<br />than other methods? No evidence of this.<br />2. Does hansen's UHI adjustment proceedure make sense?"<br /><br />These two are basically the same question. On 2, Hansen is essentially deleting a station if it gets flagged as urban. People get mesmerized by precisely how this is implemented, but they forget that's what it's doing. So the focus reverts back to question 1. <br /><br />The advantage of nightlights is that it's a lot easier to collect the data. How well it works is a continuing question.carrot eaternoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-7571612756975380122010-03-07T22:48:38.455+11:002010-03-07T22:48:38.455+11:00Steven,
Here's a list. Number, distance (km), ...Steven,<br />Here's a list. Number, distance (km), name. My count of 121 rural stations was wrong - there was irrelevant stuff in the vector whose length I used. The graphs I posted should be right, except that I counted Orland itself, so the numbers for each year are inflated by one. It didn't add to the weighted sum.<br /><br /> 1 966 VICTORIA MARINE,BC <br /> 2 815 SAN CLEMENTE/ISLAND NAAS<br /> 3 763 SAN NICHOLAS/ISLAND <br /> 4 905 CUYAMACA<br /> 5 782 SANTA CATALINA/CATALINA ARPT<br /> 6 869 NEEDLES FAA AP <br /> 7 978 LEES FERRY <br /> 8 850 ZION NATIONAL PARK <br /> 9 883 ALTON<br />10 957 SELIGMAN<br />11 972 GRAND CANYON NATL PARK 2<br />12 653 FAIRMONT<br />13 635 SANDBERG/WSMO<br />14 607 TEJON RANCHO<br />15 642 MERCURY/DESER<br />16 466 LEMON COVE <br />17 908 MANTI<br />18 958 HIAWATHA<br />19 922 LOA <br />20 868 SCIPIO <br />21 820 DESERET <br />22 481 TONOPAH/AIRPORT <br />23 385 MINA<br />24 318 YOSEMITE PARK HEADQUARTERS <br />25 749 MODENA <br />26 335 HOLLISTERUSA <br />27 904 MORGAN COMO SPRINGS <br />28 952 WOODRUFF<br />29 944 LAKETOWN<br />30 875 MALAD CITY <br />31 750 OAKLEY <br />32 910 IDAHO FALLS/42 NW WB<br />33 959 DUBOIS/FAA AIRPORT <br />34 493 BEOWAWE USA <br />35 439 AUSTIN <br />36 313 LOVELOCK/FAA AIRPORT<br />37 422 GOLCONDA<br />38 262 CEDARVILLE <br />39 27 WILLOWS 6W <br />40 206 ORLEANS <br />41 248 HAPPY CAMP RS<br />42 312 BLY 3NW <br />43 334 SEXTON SUMMIT<br />44 332 PROSPECT 2SW<br />45 350 CRATER LAKE NPS HQ <br />46 412 FREMONT 5NW <br />47 952 LIFTON PUMPING STATION <br />48 977 BORDER 3N<br />49 539 DANNER <br />50 674 ARROWROCK DAM<br />51 703 CAMBRIDGE<br />52 504 BURNS,OR.<br />53 357 PAISLEY <br />54 482 MALHEUR REFUGE HDQ <br />55 573 DAYVILLEUSA <br />56 711 MEACHAM <br />57 493 MCKENZIE BRIDGE RS <br />58 518 CASCADIA<br />59 597 THREE LYNX <br />60 634 HEADWORKS PORTLAND WTRB <br />61 640 DUFUR<br />62 649 MORO<br />63 889 FENN RANGER STN <br />64 933 WILBUR <br />65 744 WESTON USA <br />66 964 STEHEKIN 4NW<br />67 998 CONCONULLY <br />68 777 RAYMOND 2S <br />69 840 STAMPEDE PASS/WSMO <br />70 854 CEDAR LAKE <br />71 931 QUILLAYUTE,WA<br />72 981 TATOOSH ISLAND WASHINGTON<br />73 875 ELBERTA <br />74 987 OLGA 2SE<br />75 779 LONGMIRE RAINIER NPS<br />76 841 GRAPEVIEW 3SW<br />77 702 HOLLISTER<br />78 746 HAZELTON<br />79 718 GOODING/2 S <br />80 674 GLENNS FERRY<br />81 114 NEVADA CITY USA <br />82 206 ELECTRA PH <br />83 139 BLUE CANYON <br />84 143 LAKE SPAULDINGNick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-58234916528901243602010-03-07T21:55:04.769+11:002010-03-07T21:55:04.769+11:00Are those all the rural stations that adjust Orlan...Are those all the rural stations that adjust Orland?<br /><br /> Hmm.<br /><br />Can you dump a list. looks wrong.<br /><br />I guess the other question is this. Does it really make sense to<br />weight this stations with rural stations from 500km away? like<br />how many many do you need? is 20 better than 10? <br /><br />Whats the sensitivity of 500Km to the adjustment?<br /><br />Clearly something is messed up with this.stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-90405624546737305992010-03-07T21:47:28.820+11:002010-03-07T21:47:28.820+11:00Thanks Nick I'll have a look.
Ron Did some ni...Thanks Nick I'll have a look.<br /><br />Ron Did some nice work pulling a density database out of columbia.<br />It's nice to see people on the warmist side taking an interest in this and actually doing numbers. The baggage of hansen and jones ( even if unjustified in your view) is a great thing to drop<br /><br />How do I put this. you, ron, zeke have more credibility than Jones,hansen, tamino. At least in my mind.stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-48941296568406865762010-03-07T10:32:38.061+11:002010-03-07T10:32:38.061+11:00Steven,
Thanks for the Imhoff ref. Here is a paywa...Steven,<br />Thanks for the Imhoff ref. <a href="http://cat.inist.fr/?aModele=afficheN&cpsidt=2814639" rel="nofollow">Here</a> is a paywall ref - I couldn't find a free.<br /><br />I can't say much nore about Orland, but I can about the rural stations. There's a lot of them; in fact, they broke my array limit. 121 in all - I've plotted the number for each year in an update above, along with the weighted sum. That sum is the numbers each multiplied by the wt function (0;1) tapering with distance.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-20155141525852703022010-03-07T06:49:51.950+11:002010-03-07T06:49:51.950+11:00Nick
Imhoff, M.L., W.T. Lawrence, D.C. Stutzer, a...Nick<br /><br />Imhoff, M.L., W.T. Lawrence, D.C. Stutzer, and C.D. Elvidge. A technique for using composite DMSP/OLS “city lights” <br />satellite data to map urban area<br /><br /><br /><br />http://www.ngdc.noaa.gov/dmsp/pubs.php<br /><br />( good resource with other papers )<br /><br />#<br />www.isprs.org/proceedings/XXXVI/8-W27/smalletal.pdf -<br />#<br /><br />Essentially the study looks at How well nightlights captures Population density. The issue I was concerned about was Blooming, amongst other things. Further, it seemed that if one was "after" population density then why not just use population density. I pointed folks to the sedac data as an example of a more direct measure of density.<br />A couple spot checks of nighlights with places I knew ( shasta, orland, mineral california) was enough for me to question the methods<br />ability to pick out density accurately. <br /><br />Anyways, From the review above:<br /><br />Specifically, the area and intensity of illumination is known to vary significantly with <br />energy availability, economic development and density of settlement... <br /><br />"Previous analyses have revealed a consistent disparity between various spatial measures of urban <br />extent and the spatial extent of lighted areas in the night lights datasets ... Specifically, the lighted areas detected by the OLS are consistently larger than the <br />geographic extents of the settlements they are associated with. The larger spatial extent of lighted <br />area, relative to developed land area, is sometimes referred to as “blooming”. The blooming is the <br />result of three primary phenomena, including: 1) the relatively coarse spatial resolution of the OLS <br />sensor and the detection ofdiffuse and scattered light over areas containing no light souce , 2) large <br />overlap in the footprints of adjacent OLS pixels, and the accumulation of geolocation errors in the <br />compositing process (Elvidge et al., 2004). In the context of this study, blooming refers to spurious <br />indication of light in a location that does not contain a light source. "<br /><br /><br />WRT Orland. Orland loos to be an example of blooming. The 1 sq km<br />surrounding the site has a density of 137 people. The area is flat<br />with single story dwellings located more than 100 feet away from the sensor. The site is on the outskirst of a small town, population 7K<br />The density of the town is roughly 950 per sq km. The total town is 6.6sq km. <br /><br />here are some selected population figures for the town.<br /><br />2000 : 6K<br />1990: 5K<br />1980: 3.5K<br />1970 3K<br />1912 1K<br />1910 800<br />1880 292<br /><br />The sensor in question has not moved.<br />If its 137 people per sq km there today and the other 5.6KM contain<br />over 6000 residents, its probably safe to assume that in 1970 when the town population was half of what it is today that the density was lower than 137. and in 1960 what magically happened? Nothing. But that is the kneepoint.<br /><br /> bottom line is that This adjustment proceedure has nothing to do with the physical process of UHI. Orland is rural, has probably always been rural. For some reason its temps prior to 1960 showed a cooling<br />while other sites ( judged as rural today) showed a warming. <br /><br />Now, Orland and two other stations very close by are used by the agriculture department of the state of california. All the stations in this system have to meet fairly rigid standards for placement. They are located in open fields and collect more than just temps <br /><br />Does the orland adjustment matter? That's the wrong question.<br />The question is <br /><br />1. does nightlights pick out rural/small town/urban with More accuracy<br /> than other methods? No evidence of this.<br />2. Does hansen's UHI adjustment proceedure make sense?<br /> No.<br /><br />The next question is what sites was Orland compared to?<br /><br />And what happened in this area from 1880 to 1960?<br /><br />Any guesses?stevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-9640928041837153012010-03-06T10:57:39.322+11:002010-03-06T10:57:39.322+11:00Steven, Thanks. I did the Orland plot which I'...Steven, Thanks. I did the Orland plot which I've put as an update at the end of the main post. It shows Orland warmer than its rural neighbors back before 1900. However, there weren't many of them then - less than 10 i9n the early years. So the adjustment, which seeks to remove the Orland trend, has a warming effect uo till 1960, then cooling. The cooling is what you'd expect if it was really urban.<br /><br />It isn't easy for me to to the equivalent Eschenbach graph of actual adjustments made.<br /><br />I take it by imhoff you mean <a href="http://www.citeulike.org/article-posts/6191906" rel="nofollow"> this recent paper</a>? Do you know of a non-paywall version?Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-65196557723014435272010-03-05T18:23:18.249+11:002010-03-05T18:23:18.249+11:00Let me see if I can explain one of the issues.
Ni...Let me see if I can explain one of the issues.<br /><br />Nightlights in one case I found classified an urban site as rural<br /><br />( Nightlight = Dark And pop< some figure) takes care of that.<br /><br />Nightlights also rated some rural sites as small towns.<br /><br />Take the case of orland. ( as I recall) it was classified as dim.<br /><br />When you compare it to rural sites it gets adjusted up. This makes no sense especially when you look at the site.<br /><br />That opens the question : are the rural sites really rural or are they<br />infected with microsite bias.<br /><br />Don't know. <br /><br />Anyways, this is a really interesting question. Suggest that everybody read imhoffstevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-26540516093573272242010-03-05T18:08:52.621+11:002010-03-05T18:08:52.621+11:00Nice work Nick,
One of the sites that has always...Nice work Nick,<br /><br /> One of the sites that has always interested me is Orland California. I used to drive through the area every year, so I was kind familiar with it. GISS adjustment never seemed to make much sense on this site which is unmoved for 100 years. And there is good population data all the way back before 1900. it would interesting to see what your approach says about that site. I say this becuase I do have access to pristine rural area data close to orland from about 1982 to present. State of califorina data ( wind insolation soil temp ) It's always good to test a algorithm blindlystevenhttps://www.blogger.com/profile/06920897530071011399noreply@blogger.com