Monday, October 31, 2011

GWPF is wrong, warming has not stopped.

The Daily Mail says that BEST results show that global warming has stopped, citing this graph from the GWPF. And there's an interview with Judith Curry talking about someone "hiding the decline".

Tamino has quite correctly taken this apart. This is his version of the GWPF graph:



As he points out, the claim of no warming is dependent on the dip in April/May 2010. Without it, there is warming, pretty much as expected.

So I checked the BEST data.txt to see why these month data had such large error bars, and were so out of line. It turns out that all the data they have for those months is from 47 Antarctic stations. By contrast, in March 2010 they have 14488 stations.

How this could have happened, I don't know. Anyway, a list of those 47 stations below the jump.



NameLat
MIRNYJ-66.5400
SYOW-69.0000
Elizabeth-82.6000
AMUNDSEN-SCOTT-90.0000
NICO-89.0000
LETTAU-82.5180
GILL-80.0090
BYRD-80.0000
MARILYN -79.9500
SCHWERDTFEGER -79.8750
MINNA BLUFF -78.5500
VOSTOK-78.4800
LINDA -78.4755
PEGASUS NORTH -77.9250
FERRELL -77.8840
MCMURDO SOUND NAF -77.8734
NAVY OPERATED(AMOS) -77.5000
CAPE ROSS -76.7170
HALLEY BAY-75.5000
DOME C II -75.1210
MANUELA -74.9460
BUTLER ISLAND -72.2060
POSSESSION ISLAND -71.8910
NOVOLAZAREVSKAJA/LAZREV-1/61-70.7670
GEORG VON NEUMAYER, FRG ANT.. -70.6608
ZHONGSHAN WX OFFICECI -69.3670
DAVIS -68.5777
ROTHERA POINT -67.6062
MAWSON-67.6000
LAW DOME SUMMIT -66.7330
DUMONT D'URVILLE, FRANCE ANT. -66.6670
CASEY -66.3320
B. A. VICECOMODORO MARAMBIO -64.2330
BASE ESPERANZA-63.4000
ISLAS ORCADAS B.N.-60.7360
MACQUARIE ISLAND-54.4941
MARION ISLAND -46.8738
GOUGH ISLAND-40.3500
Henry -89.0000
Limbert -75.4000
Mario Zucchelli -74.7000
GENERAL SAN MARTIN B.E. -68.1300
Larsen Ice Shelf-66.9000
BELLINGSHAUSEN-62.2000
TENIENTE JUBANY E. C., ARG. -62.2000
GRYTVIKEN S. GEORGIA IS.-54.2700
Campbell-52.0000







Thursday, October 27, 2011

A Javascript index for Moyhu


Tuesday, October 25, 2011

BEST has same data as GHCN pre-1850

BEST extended its temperature series back a lot further than its predecessors, to about 1800. Many have assumed that this was because they have more early data, but this is not true. Their data set in this period is pretty much identical with GHCN.

You can see this from the KMZ file or the interactive Javascript plot. But I thought I should check it out in detail, so I have below a table of the actual stations, in one case from GHCNV2 (V3 is the same) and in the other BEST, taken from their data.txt file.

There are very few discrepancies. Out of about 275 pre-1850 stations, I count only 6 which BEST has and GHCN doesn't. There seem to be 9 than GHCN has that BEST doesn't, and 8 that BEST has included apparently twice. Details below the jump.





Here is a complete table of stations with pre-1850 data.. The color scheme is:
BlackBEST
RedGHCN
PurpleBEST unique
BlueGHCN unique
GreenBEST duplicate


The first table is just of the exceptions. I have colored STYKKISHOLMU but not counted it, because it is in both, though BEST seems to have about 20 more years data. Generally I did not color if there is a discrepancy of less than three years in start date. I have shortened names to 12 characters.

NameLatLonAltStartEnd
BERLIN-DAHLE52.4713.35817692009
Wien - Hohe 48.216.420317752009
UCCLE BELGIU50.84.410017942009
SHIP V 34164-99918081829
EAST MILTON 42.22-71.1219318112010
KIEV GMO50.430.5317018122010
SAMSUN 41.2836.33418192010
FORT SNILLIN44.9-93.224518201982
NEW YORK AVE40.63-73.96718222009
CHARLESTON I32.83-79.97818232010
STYKKISHOLMU65.08-22.721018232010
MACDILL AFB/27.85-82.52618252010
ITHACA CORNE42.46-76.4629218272010
KIRKWALL59-2.92618272009
Helsinki/Kai60.225418292000
BLUE HILL OB42.22-71.1219518312006
STENNIS INTE30.33-89.44618332010
WAVELAND 30.3-89.38218332006
AMHERST 42.38-72.534518362010
Graz-Univers47.115.536618372001
SALZBURG-FLU47.81343918422010
WEST CHESTER39.97-75.6313718432006
STYKKISHOLMU65.08-22.73818462010
SAN FRANCISC37.62-122.38518472010
SATA FE COUN35.64-106.0320391849


And here is the full table, including exceptions, ordered by start date:

NameLatLonAltStartEnd
BERLIN-DAHLE52.4813.355217012010
BERLIN-TEMPE52.4713.44917012010
DE BILT 52.15.19617062010
DE BILT52.15.181517062010
UPPSALA & 59.8817.62417392010
UPPSALA59.8817.64117391970
BOSTON LOGAN42.36-71.04617432010
BOSTON/LOGAN42.37-71.03917432010
LENINGRAD /T59.9730.3417432010
ST.PETERBURG59.9730.3617432010
TURKU 60.5222.285617502010
TURKU60.5222.275917502010
GENEVE-COINT46.256.1242017532010
GENEVE-COINT46.256.1341617532010
LUNDSWEDEN55.713.27317531773
LUNDSWEDEN 55.713.27317531773
TORINO/CASEL45.227.6528717531981
TORINO/CASEL45.227.6528717531981
BASEL/BINNIN47.67.631717551990
BASEL/BINNIN47.67.631817551980
STOCKHOLM/BR59.3318.034017562010
STOCKHOLM59.3318.055217561994
FRANKFURT A 50.18.710917571961
FRANKFURT A 50.18.710917571961
PARIS/LE BOU48.822.426217572010
PARIS/LE BOU48.82.55317571995
PHILADELPHIA39.87-75.23617582010
PHILADELPHIA40-75.2917582010
LEIPZIG E.GE51.412.413717591935
LEIPZIG E.GE51.412.413717591935
TRONDHEIM/TY63.410.4511517611981
TRONDHEIM/TY63.410.511517611981
GREENWICH/MA51.50717631969
GREENWICH/MA51.50717631969
MILANO/LINAT45.439.2910317632010
MILANO/LINAT45.439.2810317631987
EDINBURGH/RO55.9-3.213417641990
EDINBURGH/RO55.9-3.213417641960
KREMSMUENSTE48.0514.1238517672010
KREMSMUENSTE48.0514.1338917672010
CHURCHILL FA58.8-9428717681847
CHURCHILL FA58.8-9428717681858
KOBENHAVN/LA55.6812.54917682010
KOBENHAVN/ 55.6812.55917682010
BERLIN-DAHLE52.4713.35817692009
JENAE.GERMAN50.911.615517701935
JENAE.GERMAN50.911.615517701935
PRAHA/RUZYNE50.114.2536517712010
PRAHA/RUZYNE50.114.2536517712009
REGENSBURG49.0512.137117732010
REGENSBURG 49.0512.137117732010
YORK FACTORY57-92.38617742004
YORK FACTORY57-92.312617741910
Wien - Hohe 48.216.420317752009
WIEN/HOHE WA48.2616.3720717752010
WIEN/HOHE WA48.2516.3720917752010
OULU FINLAND64.9425.411417762010
OULU 64.9325.371517762010
INNSBRUCK/UN47.2711.3958417772010
INNSBRUCK/UN47.311.458217772000
VIL'NJUS U.S54.6325.2316017772010
VILNIUS54.6325.115617772010
KARLSRUHEGER49.038.3813017792008
KARLSRUHE49.038.3714517792008
MOSKOW AGRO 55.837.615817792010
MOSKVA 55.8337.6215617792010
WARSZAWA-OKE52.1720.9710717792010
WARSZAWA-OKE52.1720.9710717792010
BUDAPEST/MET47.5219.0312917801995
BUDAPEST/47.5219.0312917801991
PADOVA (CIV/45.411.851417801990
PADOVA 45.411.851417801827
GORDON CASTL57.6-3.13217811975
GORDON CASTL57.6-3.13217811975
HOHENPEISSEN47.811.0297917812010
HOHENPEISSEN47.811.0298617812010
MUNCHEN/RIEM48.111.752917811992
MUNCHEN/RIEM48.111.752917811991
NEW HAVEN UN41.3-72.9717811992
NEW HAVEN/TW41.3-72.9717811970
MONTDIDIERFR49.72.69017841869
MONTDIDIERFR49.72.69017841869
TOULOUSE BLA43.631.3815217842010
TOULOUSE/BLA43.631.3715317842010
PAMPLEMOUSSE-20.157.575517871960
TRIER-PETRIS49.756.6723717882010
TRIER-PETRIS49.756.6727317882010
VERONA-VILLA45.3810.876817882010
VERONA/VILLA45.3810.876817882005
PALERMO ITAL38.113.422917911868
PALERMO ITAL38.113.422917911868
STUTTGART/CA48.839.231417922000
STUTTGART/ 48.839.231117922010
WROCLAW II-S51.116.8912317922010
WROCLAW II 51.116.8812117922010
MANCHESTER A53.35-2.297517942004
MANCHESTER A53.35-2.277817942004
UCCLE BELGIU50.84.410017942009
ALBANY COUNT42.75-73.88617952010
ALBANY/ALBAN42.75-73.88917952006
RIGA U.S.S.R56.9624.06917952010
RIGA 56.9724.05717951997
MADRAS/MINAM1380.181617962010
MINAMBAKKAM 1380.191617962010
NATCHEZ 31.56-91.375917992010
NATCHEZ31.55-91.385917992006
WOROFINLAND 63.222NA18001824
WOROFINLAND63.222-99918001824
STRASBOURG-E48.557.6315418012010
STRASBOURG 48.557.6315418012010
TORNEOFINLAN66.423.8NA18011832
TORNEOFINLAN66.423.8-99918011832
UDINE/CAMPOF4613.19218032005
UDINE/CAMPOF4613.19218032005
LEOBSCHUTZCZ50.217.835718051849
LEOBSCHUTZCZ50.217.835718051849
CHALONS FRAN48.94.48918061848
CHALONS FRAN48.94.48918061848
NICE/COTE DA43.657.21318062010
NICE 43.657.21018062010
TALLINN 59.4224.84118062010
TALLIN 59.4224.84418062010
GDANSK-WRZES54.418.61218071982
GDANSK-WRZES54.418.61218071984
BOLOGNA/BORG44.5311.34918081981
BOLOGNA/BORG44.5311.34918081981
MIKOLAIV 47.0331.955018081990
NIKOLAYEV 47.0131.965218081998
SHIP V 34164-99918081829
EAST MILTON 42.22-71.1219318112010
ROMA/CIAMPIN41.8312.438018111996
ROMEITALY41.812.610718111991
KAZAN'55.7249.239518122010
KAZAN' 55.649.2811618122010
KIEV GMO50.430.5317018122010
KIEV50.430.517918122009
KYIV 50.430.5716718122010
NEW BEDFORD 41.69-70.932018122010
NEW BEDFORD41.63-70.932118122002
ARHANGEL'SK 64.5540.56818132010
ARHANGEL'SK64.540.73818132010
AUGSBURGW.GE48.410.449018132009
AUGSBURGW.GE48.410.449018131834
KLAGENFURT-F46.6514.3445818132010
KLAGENFURT-F46.6514.3347618132010
BAYREUTHW.GE49.911.632018141930
BAYREUTHW.GE49.911.632018141930
BERGEN-FREDR60.385.313018162010
BERGEN/FREDR60.45.34418162010
CALCUTTA/ALI22.5388.32618162010
CALCUTTA/ALI22.5388.33618162010
OSLO-BLINDER59.9210.719518162010
OSLO/BLINDER59.910.79618161991
PETROZAVODSK61.8234.288218162010
PETROZAVODSK61.8234.2711018162010
TRENTOITALY 46.111.131218161866
TRENTOITALY46.111.131218161866
ARNSBERGW.GE51.48.121218171851
ARNSBERGW.GE51.48.121218171851
BALTIMORE CU39.28-76.62518172010
BALTIMORE WS39.28-76.62418171999
CARLO FINLAN65.0324.71718172010
CARLO FINLAN6524.7-99918171836
HOHENFURTHAU48.615.3NA18171843
HOHENFURTHAU48.615.3-99918171843
HOHENELBCZEC50.615.6NA18171850
HOHENELBCZEC50.615.6-99918171850
LANDSKRON CZ49.916.6102518171840
LANDSKRON CZ49.916.6102518171840
SMETSCHNA CZ50.214107718171847
SMETSCHNA CZ50.214107718171847
SYKTYVKAR 61.7150.8510518172010
SYKTYVKAR61.7250.8311918172004
KOBLENZ W.GE50.47.66918181831
KOBLENZ W.GE50.47.66918181831
MAASTRICHT-B50.925.7811518182010
MAASTRICHT A50.925.7811618182010
MUNSTER W.GE527.66418181868
MUNSTER W.GE527.66418181868
SAINT BERNHA45.86.1207018181868
SAINT BERNHA45.86.1207018181868
ST. BERNARDS45.76.9246018181985
ST. BERNARDS45.76.9246018181985
MINNEAPOLIS-44.88-93.2225618192010
MINNEAPOLIS/44.88-93.2225518192006
SAMSUN 41.2836.33418192010
BOCHUMW.GERM51.57.2NA18201851
BOCHUMW.GERM51.57.2-99918201851
BRESCIA/GHED45.4210.289718202010
BRESCIA/GHED45.4210.289718202010
CHAPEL HILL 35.91-79.0815218202010
CHAPEL HILL 35.92-79.115218202006
FORT SNILLIN44.9-93.224518201982
HALLE E.GERM51.511.911118201868
HALLE E.GERM51.511.911118201868
IRKUTSK OBSE52.27104.3447518202010
IRKUTSK52.27104.3246918202010
PORTLAND INT43.65-70.311618202010
PORTLAND/INT43.65-70.321918202006
SOVETSK USSR55.121.83718201930
SOVETSK USSR55.121.83718201930
WASHINGTON D38.85-77.031218202010
WASHINGTON/N38.85-77.032018202010
NEW YORK CEN40.78-73.974218212010
NEW YORK CEN40.78-73.973918212006
ODESSA GMO46.4730.675118212010
ODESA46.4330.774218212010
SIMFEROPOLUS45.0233.9821418212010
SIMFEROPOLUS453420518212010
BATON ROUGE 30.52-91.151818222010
BATON ROUGE 30.53-91.131918222006
EASTPORT ME.44.91-66.992418222010
EASTPORT ME44.92-672318222006
NEW YORK AVE40.63-73.96718222009
PRAIRIE DU C43.03-91.1520018222010
PRAIRIE DU C43.03-91.1520018222006
SOUTHPORT 5 33.99-78.01618222010
SOUTHPORT 5N34-78.02618222006
ARNSTADTE.GE50.811.389818231867
ARNSTADTE.GE50.811.389818231867
CHARLESTON I32.83-79.97818232010
CHARLESTON32.9-801518232009
CHARLESTON C32.78-79.93318232006
KOTHENE.GERM51.811.9NA18231847
KOTHENE.GERM51.811.9-99918231847
KREUZBURG SW6018.262118231849
KREUZBURG SW6018.262118231849
MITAU USSR56.723.71018231872
MITAU USSR 56.723.71018231872
NEURODE CZEC50.516.523718231843
NYSAPOLAND50.317.358118231851
NYSAPOLAND 50.317.358118231851
PRINCESS ANN38.26-75.66618232010
PRINCESS ANN38.22-75.68618232002
STYKKISHOLMU65.08-22.721018232010
FELLINUSSR58.425.66118241847
FELLINUSSR 58.425.66118241847
L'VOV49.8223.9532518242010
L'VIV49.8223.9532318242010
POLTAVA49.634.5616018242010
POLTAVA49.634.5516018242010
PORTSMOUTH S38.75-82.8816418242010
PORTSMOUTH-S38.75-82.8816418242006
SSEVASTOPOL 44.633.54018241875
SSEVASTOPOL 44.633.54018241875
WEST POINT 41.38-73.979718242006
BRAUNSCHWEIG52.310.458418252010
BRAUNSCHWEIG52.310.458818252010
KISINEV U.S.47.0228.9312418252010
KISINEV47.0228.9817318252010
KRAKOW/BALIC50.0819.823418252010
KRAKOW 50.0819.823718252010
MACDILL AFB/27.85-82.52618252010
METZ/FRESCAT49.086.1319218252010
METZ/FRESCAT49.086.1319118251868
NEURODE CZEC50.516.523718251843
SINGAPORE SI1.34103.931718252010
SINGAPORE SI1.3103.91818251984
TAMPA INTERN27.97-82.53518252010
TAMPA/INT. F27.97-82.53318252010
WEST POINT41.38-73.969818252010
BERNSWITZERL477.4NA18261849
BERNSWITZERL477.4-99918261851
UTICA HARBOR43.11-75.2314218261991
UTICA43.08-75.217618261991
AUBURN42.92-76.5423518272010
AUBURN 42.92-76.5323418272006
ITHACA CORNE42.46-76.4629218272010
ITHACA44.5-76.829318271990
ITHACA CORNE42.45-76.4529218272006
KIRKWALL59-2.92618272009
LOWVILLE43.8-75.4826218272010
LOWVILLE 43.8-75.4826218272006
MONTPELLIERF43.583.952518272010
MONTPELLIER43.583.97618272010
ORKNEYUK59.1-3.32218271906
ORKNEYUK 59.1-3.32218271906
BURLINGTON I44.47-73.1510218282010
BURLINGTON/I44.47-73.1510418282006
DEUTSCHBROD 49.615.6123818281866
DEUTSCHBROD 49.615.6123818281866
DORPATUSSR58.424.51018281875
DORPATUSSR 58.424.51018281875
OXFORDUK51.7-1.26318281980
OXFORDUK 51.7-1.26318281980
POUGHKEEPSIE41.64-73.94618282010
POUGHKEEPSIE41.63-73.925118282006
SITKA MAGNET57.06-135.351618282010
SITKA57.07-135.352018281993
STRALSUND E.54.313.1NA18281852
STRALSUND E.54.313.1-99918281852
TETSCHENCZEC50.814.228818281842
TETSCHENCZEC50.814.228818281842
AACHEN50.786.119518292010
AACHEN 50.786.120518292010
BREMEN GERMA53.058.8418292010
BREMEN 53.058.8518292010
ELBLAG POLAN54.1719.424318292010
ELBLAG 54.1719.434318292010
FREDONIA42.45-79.2723118292009
FREDONIA 42.45-79.323118292006
Helsinki/Kai60.225418292000
HELSINKI/SEU60.3255618292010
HELSINKI/SEU60.3255818292010
HOULTON 5 N 46.18-67.8312118292010
HOULTON 5N 46.2-67.8311818292006
JAKUTSK62.02129.7210118292010
PENN YAN 8 W42.67-77.1830418291994
PENN YAN 8W42.67-77.1830318291994
TROY LOCK AN42.75-73.68718292010
TROY LOCKDAM42.75-73.68718292006
VARDOE70.3731.11518292010
VARDO70.3731.11518292010
WILTENAUSTRI47.311.418418291858
WILTENAUSTRI47.311.418418291858
YAKUTSK GMO 62.06129.7410018292010
ARYSPOLAND53.822.145018301865
ARYSPOLAND 53.822.145018301865
DARMSTADT W.49.98.715718301930
DARMSTADT W.49.98.715718301930
KARESUANDO A68.4522.4832818302010
KARESUANDO 68.4522.532718302010
KEY WEST BOC24.56-81.75618302010
KEY WEST/INT24.55-81.75618302006
LEAVENWORTH 39.3-94.9126818302010
LEAVENWORTH39.32-94.9327718302006
ROCHESTER GR43.14-77.6715018302010
ROCHESTER AI43.13-77.6718218302006
SYLTE.GERMAN54.112.765018301863
SYLTE.GERMAN54.112.765018301863
BLUE HILL OB42.22-71.1219518312006
BUFFALO GREA42.94-78.7321518312010
BUFFALO/GREA42.93-78.7321518312006
DUBLIN-PHOEN53.4-6.297718312010
DUBLIN AIRPO53.43-6.258518312010
PROVIDENCE T41.77-71.431818312010
PROVIDENCE W41.73-71.431518312006
SVERDLOVSKUS56.8360.6126018312010
SVERDLOVSKUS56.860.623718312004
FLORENCEITAL43.811.224718322010
FLORENCEITAL43.811.37518321981
OCALA TAYLOR29.17-82.092218322010
OCALA29.2-82.082218322006
ORENBURG/CHK51.7455.111218322010
ORENBURG 51.6855.111718322004
ROSTOCK E.GE54.112.2NA18321868
ROSTOCK E.GE54.112.2-99918321868
SAVANNAH CHA32.08-81.171418322010
SAVANNAH/MUN32.13-81.21518322006
TOBOLSK AS-258.1568.244718322010
TOBOL'SK 58.1568.255018322010
ENISSALAUSSR44.934.646018331872
ENISSALAUSSR44.934.646018331872
KURSK U.S.S.51.7236.1821518332010
KURSK51.7736.1724718331990
STENNIS INTE30.33-89.44618332010
UCCLE 50.84.3610418332010
UCCLE50.84.3510418332010
WAVELAND 30.3-89.38218332006
BELFAST/ALDE54.65-6.217218342010
BELFAST/ALDE54.65-6.228118342010
HANOVER 43.7-72.2818418342009
HANOVER43.7-72.2818318342006
ST.JOHN'S NF47.62-52.7314018341990
ST.JOHN'S NF47.62-52.7314018341990
BARCELONA SP41.42.29518351985
BARCELONA SP41.42.29518351985
GUETERSLOH51.938.347318352010
GUETERSLOH 51.928.37218351920
HANOVER 43.7-72.318318351990
NEW-YORK-CEN40.8-743918351990
NOVGOROD58.5231.262418351991
NOVGOROD 58.5231.252418351993
WEST-POINT41.4-749718351990
AMHERST 42.38-72.534518362010
AMHERST 42.4-72.54518361990
AMHERST42.38-72.534518362006
GOERLITZ 51.1714.9523818362010
GORLITZGERMA51.1714.9523718362010
HILLSBORO 39.2-83.6233518362010
HILLSBORO39.2-83.6233518362006
PERPIGNAN 42.732.884618362010
PERPIGNAN42.732.874818362010
SARATOV 51.5746.0314918362010
SARATOV51.5746.0315618362004
ST LOUIS LAM38.75-90.3818018362010
ST.LOUIS/LAM38.75-90.3717218362010
SZCZECIN-DAB53.414.61418362010
SZCZECIN 53.414.62318362010
ZURICH/TOWN/47.388.5856518362010
ZURICH (TOWN47.388.5756918362010
ASTRAKHAN'46.2848.04-1418372010
ASTRAHAN'46.2848.05-2318372010
CHARLOTTESVI38.03-78.5223518372010
CHARLOTTESVI38.03-78.5226518372006
GARDINER44.22-69.784318372009
GARDINER 44.22-69.784218372006
Graz-Univers47.115.536618372001
LUGAN USSR48.639.36018371880
LUGAN USSR 48.639.36018371880
TOMSK56.584.9213918371990
THIRUVANANTH8.4876.956418372010
TRIVANDRUMOB8.4876.955418372010
USSOLJE USSR59.356.810718371853
USSOLJE USSR59.356.810718371853
ZLATOUST55.1759.6755118371993
ZLATOUST 55.259.6753818371875
BARNAUL 53.3783.6117918382010
BARNAUL53.4383.5218418382004
BOGOSLOWSKUS59.860.119018381880
BOGOSLOWSKUS59.860.119018381880
MARSEILLE/MA43.445.261418382010
MARSEILLE/MA43.35.4818382010
OGDENSBURG 444.73-75.448518382010
OGDENSBURG 444.73-75.438518382006
SCHOSSL CZEC50.513.5NA18381865
SCHOSSL CZEC50.513.5-99918381865
SSAREPTAUSSR48.544.65018381855
SSAREPTAUSSR48.544.65018381855
BALTISCHPORT59.424.11018391880
BALTISCHPORT59.424.11018391880
DETROIT MICH42.41-83.0119118392010
DETROIT MICH42.4-8319118392010
EISENHOWER N39.83-77.2416318392010
EISENHOWER N39.8-77.2715818392006
ISTANBUL/GOZ40.9729.093618392010
ISTANBUL/GOZ40.9729.083318392010
NERCHINSKIJ 51.32119.6262018392010
NERCINSKIJ Z51.32119.6261918392010
NIZHNYJ TAGI57.8860.0825818392010
NIZHNYJ TAGI57.8860.0726018391865
PLATTSBURGH 44.65-73.476018392010
PLATTSBURGH 44.65-73.475018391995
ST PAUL ISLA57.16-170.22818392010
ST. PAUL 57.15-170.22918392010
STANISLAV RU45.924.820518391854
STANISLAV RU45.924.820518391854
TOMSK 56.4684.9513418392010
URALSK KAZA51.2551.333518392010
URALSK 51.2551.283718392010
BANGKOK METR13.73100.551118402010
BANGKOK13.73100.572018402010
FORT PIERCE 27.47-80.35718402010
FORT PIERCE27.47-80.35718402006
HARRISBURG C40.22-76.8510518402010
HARRISBURG C40.22-76.8510318401991
LUEBECK-BLAN53.8210.71518401868
LUEBECK-BLAN53.8210.71518401868
MADRID/RETIR40.4-3.766018402009
MADRID/RETIR40.4-3.765718401991
TORONTO CANA43.67-79.411318402010
TORONTO ON 43.67-79.411218401990
BEIJING 39.9116.335018412010
BEIJING39.93116.285518412010
HOBART REGIO-42.85147.422918412010
HOBART AIRPO-42.83147.482718412010
NANCY/ESSEY 48.686.2121518412002
NANCY/ESSEY48.686.2221718412002
TRIESTE 45.6513.771518412010
TRIESTE45.6513.752018412010
BALACHNAUSSR56.543.66018421875
BALACHNAUSSR56.543.66018421875
BOMBAY/COLAB18.972.811118422010
BOMBAY / COL18.972.821118422010
BORDEAUX/MER44.83-0.75318422010
BORDEAUX/MER44.83-0.76118422010
SALZBURG-FLU47.81343918422010
BEYROUTH /AE33.8435.492018432003
BEYROUTH (AE33.8235.481918431987
FORT SCOTT37.86-94.7125818432010
FORT SCOTT 37.85-94.725718432006
IZMIR TURKEY38.427.32518431991
IZMIR TURKEY38.427.32518431991
SYRACUSE HAN43.11-76.1112518432010
SYRACUSE/HAN43.12-76.1212418432006
WEST CHESTER39.97-75.6313718432006
EMDEN_FLUGPL53.357.2418442004
EMDEN-HAFEN 53.37.2618441997
KRONSTADT US6029.82018441875
KRONSTADT US6029.82018441875
OSWEGO EAST 43.47-76.510718442009
OSWEGO EAST43.47-76.510618442006
TBILISY GMO 41.6844.9145218442010
TBILISI41.6844.9549018441994
CAYENNE/ROCH4.83-52.37918452010
CAYENNE/ROCH4.83-52.37918452010
DIJON 47.275.0922618452010
DIJON47.275.0822718452010
KIROV (VYATK58.6249.5615918452010
KIROV58.5249.3214618451994
NAGASAKI32.73129.883218452010
NAGASAKI 32.73129.873518452010
ROUEN/BOOS F49.381.1915618452010
ROUEN49.381.1815718452010
TAMBOV U.S.S52.7641.4113918452010
TAMBOV 52.841.3312818451990
GEORGETOWN6.8-58.15218462009
GEORGETOWN 6.8-58.15218462007
SAN ANTONIO/29.53-98.4724218462006
STYKKISHOLMU65.08-22.73818462010
DURHAMUK54.8-1.610218471981
DURHAMUK 54.8-1.610218471981
SAN FRANCISC37.62-122.38518472010
SHANGHAICHIN31.26121.43518472010
SHANGHAICHIN31.2121.4718472010
BAKU (DENDRO40.4249.786118481875
BAKU (DENDRO40.4249.786118481875
DARJEELING I27.0588.29212818481997
DARJEELING 27.0588.27212818481999
ERFURT/BINDE50.9810.9732018482010
ERFURT-BINDE50.9810.9732218482010
FORT SHEVTCH44.5550.23-1418482010
FORT SEVCENK44.5550.25-2518482009
FRANKFURT AM52.314.64818481868
FRANKFURT AM52.314.64818481868
HAMMERFESTNO70.723.7NA18481863
HAMMERFESTNO70.723.7-99918481863
KALININGRAD 54.7220.582218482010
KALININGRAD54.7220.552118481993
KOLNW.GERMAN5174518481858
KOLNW.GERMAN5174518481858
KOSZALIN POL54.216.173418482010
KOSZALIN 54.216.153418481989
LOCKPORT 2 S43.15-78.6717718481999
LOCKPORT 2NE43.18-78.6515818481999
OBIRAUSTRIA 46.514.5204418481944
OBIRAUSTRIA46.514.5204418481944
STONYHURSTUK53.8-2.511518481969
STONYHURSTUK53.8-2.511518481969
TORGAUE.GERM51.6138018481868
TORGAUE.GERM51.6138018481868
EAGLE PASS 328.72-100.4924718492010
EAGLE PASS 28.7-100.4824518492006
KIEL HOLTENA54.3710.143118492010
KIEL-HOLTENA54.3810.153118491926
LUNGI 8.62-13.22718492010
LUNGI8.62-13.22718491998
RIO GRANDE C26.38-98.845318492010
RIO GRANDE C26.38-98.875318492006
SAN ANTONIO 29.46-98.4722218492010
SAN DIEGO LI32.73-117.17618492010
SANTIAGO/COM42.87-8.3136718492010
SAN DIEGO/LI32.73-117.17918492010
SANTIAGO/COM42.8-8.1-99918492010
SATA FE COUN35.64-106.03203918492010
SOCORRO 34.08-106.88139718492009
SOCORRO34.08-106.88139718492006
VANCOUVER 4 45.68-122.656418492010
VANCOUVER 4N45.68-122.656418492006







Monday, October 24, 2011

World coverage by decade of BEST, GHCN, GSOD and CRUTEM3

Update: I see that the plot does not show in Internet Explorer - I'm trying to find out why. It works in Firefox, Chrome and Safari.

In the previous post, I talked about a KMZ file which would display the stations of the four temperature databases, BEST, GHCN, GSOD and CRUTEM3. They were set in folders so that different starrting dates could be displayed.

This post provides a JavaScript interactive display with the same general intent. There are a total of 52 images, each showing a single database in a single decade (approx). It shows the stations that returned any data in that decade.

Here you see just a single image. There are two legends, one with decade and one with database. You can click on the legend to bring up any combination. There is a control with a square and four triangles. The triangles just navigate up and down the menus in the way they point. The square enables you to toggle rapidly between the last two images shown. The intent is that you can set up pairings and compare.



The images are quite high resolution (1600x960 pixels) so you can use the Ctrl+ and Ctrl- controls to enlarge and see more detail. When you click on an image for the first time, there is a short pause while it downloads.



As you'll see, the daily data, GSOD, has good coverage at present, but doesn't go back far. GHCN goes a long way back, and the BEST coverage seems to use much the same data, with just a few more ststions. But try it out - there is a lot you can test.


Saturday, October 22, 2011

A combined KMZ file for BEST, GHCN, GSOD and CRUTEM3

Update I have modified the ALL4.kmz file, which you can download here. I updated the data.txt file, where BEST had mistakenly posted the MAX file. And I found the problem which had led to the previous version showing very little early BEST data. BEST had a  column saying how many days in each month had readings, and I set a filter to require at least 10 days. However, much of the early period had this set to -99, which meant the data was rejected. I have removed the filter, and now there are a lot of pre-1850 sites.


This is a development foreshadowed in the previous post. I have put a combined .kmz file with data from 4 land station datasets. GHCN is actually v2, but there is very little difference at this level between v2 and v3. CRUTEM3 is the version released in July, discussed here. GSOD was discussed here.

There is now much more information - in fact, when you open the file in Google Earth, it looks colorful but cluttered. The pushpins are colored according to dataset - yellow for BEST, green for GHCN, red for GSOD, and a sort of dull green for CRUTEM3. They also vary in size - the smallest has 0-30 years of data, next has 31-60, and the largest has more than 60.

But the key to looking it is that the data is stored in folders. At the top level, there is a folder for each dataset. At the next level down, they are classified according to start year of data. The ranges are 0-1850, 1851-80, 1881-1900,1901-1920, 1921-30, 1931-40, 1841-50, 1951-60, 1961-70, 1971-80, 1981-90, 1991-2000, and 2001-2011. As I'll show in the next picture, in Google earth you can toggle on/off at any level. If a dataset is on, you can toggle the year folders. If you want to see in any set the years before 1921, just toggle off the later folders.

Update - I had a warning briefly that there were spurious sites in the BEST folder. These had start years of -9.9999 and so went into the pre-1950 folder. That is fixed, but there's a new problem that the pre-1850 folder is almost empty. That could be real, but the BEST Team have done analyses for this period. See below for a discussion of the data.Fixed

The Start year, End year and Duration have been added to the pop-up balloon that you get by clicking on a station. The file is called ALL4.kmz, and can be found here.

Here's a GE snapshot of the toggle facility. You have to click on a few +s to see this. GSOD and CRUTEM3 are not visible. BEST shows only stations with data before 1971. GHCN is visible, but you'd have to open that menu to see which years. I had it matching BEST.

Added: To get the data years for BEST, I used the data.txt file in their PreliminaryTextDataset folder. That looks right, but I need to investigate to see if it includes everything. There seem to be early stations missing.


A KMZ file for the BEST stations

Update - the latest post points to a more comprehensive KMZ file
In the BEST text data zip (warning - 200 Mb), there is a listing of 36736 stations in the file site_detail.txt. I've made a KMZ file (1300 Kb), which is in the file repository under the name "best.kmz". If you download it and click on it, it will bring up Google Earth (if you have it installed) with all the stations marked with yellow pushpins.

If you click on a pushpin, a balloon will pop up with some minimal data (Name,ID's, Altitude, Lat/Lon). Later if I get some analysis done, I'll produce versions with folders, colors and more info. Here's a GE snapshot:





Friday, October 21, 2011

The Berkeley Surface Temperature (BEST) analysis

I woke up this morning and saw that the BEST analysis papers and data where online. And there were already posts at Judith Curry's, WUWT, Tamino's and Stoat, and soon one by Zeke at the Blackboard. The dynamic blogroll here put Zeke's upbeat "Some interesting results from BEST" directly above Stoat's pithier "BEST is boring".

My expectations had been somewhat more in line with Stoat's, and indeed the concensus seems to be that it confirms what had been known. But I was interested in the analysis, mainly because the claimed novelty was the least squares method that TempLS uses, which I thought David Brillinger would have improved considerably.

One of the minor surprises was that DB was not on the list of authors of the main analysis paper, despite being one of the big names on the Team. However, he is acknowledged handsomely, and the sophistication of the statistics does indicate his contribution.

There have been a number of other analyses in the last two years which, like BEST, confirm the major indices. Some were land-only, others included sea surface temperatures.

So, with that preamble, here is a very preliminary discussion, mainly of the paper Berkeley Earth Temperature Averaging Process.



The main thing that has puzzled me about this project is that they have restricted themselves to land-only. I heard that the funding ran out before they could include oceans, but I thought the priority was odd. There isn't much point in fancy stats when 2/3 of the globe is missing. They go into a discussion of the different ways the land-only indices handle this problem. They, like NOAA, treat the stations as representing land only. GISS weights the stations in such a way that they attempt to represent the whole globe. BEST says the issue has had insufficient discussion. I think that the BEST discussion fills a much-needed gap in the literature. The land-only indices should really be of only historical interest.

Anyway, they have done some things in the modelling I had been putting off, specifically kriging. TempLS analysis is based on OLS, which effectively assumes that the residuals are independent, identically distributed. Of course that isn't true, and it is possible to improve. I had that on the back burner, mainly because I suspected it would make no real difference to the mean estimate, though it gives a better handle on the error. I think that BEST have affirmed that, which is valuable.

Their model (eq (4)) is
\(d_i(t_j) = \theta(t_j)+b_i+W(\vec(x)_i,t_j) + \epsilon_{i,j}\)
for the temperature in the i-th station at the j-th timestep, and bi is the baseline temp at station i. θ is the global variation. The TempLS equivalent (omitting the noise ε) is:
dsmy  ~  bsm  +  θy

 TempLS decomposes timestep into years y, months m and s corresponds to i above. A big apparent difference is that TempLS includes seasonality in the baseline. It's not currentrly clear to me what BEST does here. The online summary says that they allow for it at a later stage. The TempLS global function can be assumed to vary with year only or with year and month. For trends it makes little difference to the result.

BEST's weather term W  is estimated from correlations. I'm not sure why it is done this way rather than including the correlation in the weight function.

TempLS does a weighted fit in which the weight is by area density (and zero for no record). BEST put a lot mote into their weighting. They use a kind of kriging which adds spatial structure, and they weight by the confidence they have in the reading. Naturally this is only very approximately estimated.

An interesting sub-discussion is on the number of degrees of freedom. They say that 180 well-placed stations should be enough, echoing a figure of Phil Jones. Others  say even sixty would do, and TempLS did this analysis, fairly successfully.

TempLS avoided homogenization (with little apparent effect); BEST introduce a new technique - the scalpel. Or fairly new - they break the series into two when they suspect a station discontinuity, and reduce the weighting nearby to compensate for the greater uncertainty. I suspect the loss of information here is rather great - with the lower weighting, of course, but also the much lower confidence in the baseline means of the fragments (assumed independent) rather than the higher confidence you would have in a longer period.

They also intervene to identify outliers - not by removing them but by reducing the weighting (with similar effect). There are some dangers in this, and one needs to think with a dataset like GHCN how it interacts with the efforts that have already been made to remove outliers. "Outliers" in a preprocessed data set are much more likely to represent real data.

They do a much more sophisticated uncertainty analysis than TempLS, and I like their jackknife approach, and will probably use it. On the other hand, I think their spatial uncertainty analysis is rather pointless when they cover land only.

They break the station baseline temperatures (climatologies) down by latitude and altitude. I'm not sure why, as it isn't helpful for an analysis of global temperature. It is of independent interest, but the analysis could have been done separately on the derived baselines.

There will be several posts to come. I've only just started to look at the Matlab code. An ambition is to port it to R, but there is quite a lot of it. Currently the zip file of text data is somewhat corrupted - I've managed to get all but the flag.txt file, which probably is not needed to get started. Of course, the paper discussed here uses GHCN data only. Maybe I'll be able to compare BEST and GISS on the same data.








Monday, October 17, 2011

GISS Sep 11 - down 0.13°C

Giss Data is out, so, as I have done recently, I'll compare with the TempLS calculation. The monthly global mean anomaly average was down from 0.61°C to 0.48°C (1951-1980 base). TempLS had a small decrease (0.025°C). NOAA had an even smaller one, UAH slightly larger, and RSS had a minute rise. Numbers and plots are here.

Below the jump are the GISS global plot and TempLS. Similar features, but GISS showed larger excursions. This is to be expected, as the TempLS fitting process has a smoothing effect.


Here is the GISS Plot:


And here is the TempLS plot. It comes from this post  which also showed an interactive spherical projection.






Saturday, October 8, 2011

September GMST - TempLS down from 0.444 to 0.42

The TempLS analysis, based on GHCNV3 land temperatures and the ERSST sea temps, showed a slight cooling from August. The August temp itself came down from 0.45 °C to 0.444°C with inclusion of late data, and September's mean land/sea temperature anomaly came out as 0.42°C, relative to the 1961-1990 base period. The data and plots are at the latest ice and temperature data page.

Below is the graph (lat/lon) of temperature distribution for September. There is also an interactive world map.




This is done with the GISS colors and temperature intervals, and I'll post a comparison when GISS comes out.

And here, from the data page, is the plot of the last four months:



Finally, here is the interactive worldview of Sept surface temperatures. Just click on the yellow squares to see different views of the Earth. The bottom square is the S pole view. The first click takes a second or two for loading.