Thursday, October 18, 2012

New ISTI dataset - duplicates

This is my third post on the new beta release of the ISTI temperature database. In the first post, a Google Maps display, I noticed a number of stations which appeared to be duplicates. So I thought I'd check more comprehensively.

I first ordered the inventory alphabetically by name. A complication here is that 430 have no name. Some still showed up as duplicates.

The next step was to collect pairs of adjacent stations whose data began in the same or adjacent year. Then I did a rough distance check and retained pairs for which the sum of lat and longitude differences (absolute value) was less than 1°. That's within about 70 km at most near the equator, requiring greater closeness near the poles. In fact most pairs at this stage have near identical coordinates.

That left 1077 pairs. I've made a list as a zipped CSV file here.

There will be some missing. I suspect Vienna/Wien are duplicates, but are missed alphabetically. The two Trondheims I noticed are assigned coords too far apart. And of course, my test doesn't prove duplication - just flags for checking.

Tuesday, October 16, 2012

New ISTI temperature dataset - station trends map

This is my second post on the new beta release of the ISTItemperature database. The first post was a Google Maps interactive map of the stations. This time we have an interactive global trend map in the style I did for GHCN.

Again, the dataset is large, and takes a few seconds to load, so I have put it below the fold. It is a globe that shows individual station trends with shading on a triangular mesh. The shading color is accurate at the stations themselves. You can display the stations and mesh, and click to pick up the numerical information. There is a little navigator map that lets you reorient the globe as you wish. Maps are available for periods of 30, 45 and 60 years to present. You can magnify 2x, 4x or 8x.

I should emphasise that these are trends for individual stations - there is no modelling or smoothing, except for the triangle shading interpolation. I find that valuable in that it shows the spatial correlation (or lack of). I was interested to see if the larger ISTI set gave a similar result to GHCN.

I think it does. A notable feature of all these plots, whether for period averages or trends, is that the US seems more of a patchwork than ROW. This is of course partly the higher density of stations, but I think there really is less coherence. Perhaps there is a quality issue (associated with the large numbers).

Anyway, the map is below the jump. There is some further discussion of the methods in this plot from last November. Below the map I've written a little about the numerics..

Monday, October 15, 2012

New ISTI Temperature database - stations in Google Maps with GHCN

A few days ago, Peter Thorne of NOAA noted in a comment that a new Initiative ISTI, with NOAA involvement, has released a large new database of surface temperatures. It's on a similar scale to BEST, and I'll do comparisons in due course. But as an initial step, I thought it would be useful to gather a Google maps presentation, as I did for GHCN.

The beta release is here. I used the recommended merge (3 Oct version). The data combines TMIN, TMAX and TAVG (and is big!); I extracted the TAVG. There were 39430 stations in the inventory.

This is a big set for the GM application, so I've divided it into 8 regions with about 5000 each. You can look at them all at once if you like, but it will run very slowly. Selecting one or two regions is much better. There is a little map at the right of the display showing where they are. Because the data takes several seconds to download, I've put the map beneath the fold.

The idea of the map is that it shows stations with tags with information that you can pop up by clicking. But the main use is that you can filter by categories. You can choose ranges of start date (of data), end date, duration and altitude. The mechanics are that you make these selections, select regions, and then press one of the colors (for tag). What you've asked for will appear in that color, additional to what was there before. A special (and useful) color is invisible. The range choices combine with "or" logic, so you get "and" by making what you don't want go away. Because it is "or", you need to suppress the "All" button to make other choices. The buttons toggle. The middle columns with gt and lt signs also toggle.

I've included GHCN stations for comparison. There is a checkbox for each database. To compare you'll probably want to display in one color with one box ticked, and in another with the other box ticked (only one at a time). The green pin is useful here, in case of overlaps.

The map starts out blank, waiting for you to choose a region (or two). It's below the jump:

Saturday, October 13, 2012

September GISS Temp up 0.03°C

The GISS land/sea monthly anomaly rose from 0.57°C July to 0.60°C in September. This was almost exactly the same rise as shown by TempLS. Time series and graphs are shown here

As usual, I compare below  the previously posted TempLS distribution to the GISS plot.

Thursday, October 11, 2012

Fancy graphics and climate data

Over the last two years I've been exploring various ways of using interactivity to make climate data more accessible and attractive. I've been trying to maintain a gallery, but that takes time, and the effort has been lagging.

So I want to write a post which just summarizes the techniques used, to act mainly as a catalogue with pointers.

Here is a table of headings

KML, Google EarthSpaghetti graphs
Trend plotsEarth projections
HTML 5 and triangle meshesWebGL
Google MapsClimate Plotter

Snafu with comments

Probably in conjunction with my introducing the Captcha system, my system imposed moderation. I didn't realise this, and so the queue built up. My apologies here, especially to Girma. I've released all the comments. I've also removed the Captcha for the moment. I'll try to make sure moderation is not re-imposed.

Wednesday, October 10, 2012

WebGL - dynamic global temperature maps

This is an advance on HTML 5 that I've been using for some previous dynamic temperature plots. Some Web browsers now support WebGL, a Javascript version of openGL for 3D plotting. I've been playing with it for monthly temperature maps - here's September.

What you'll notice is a globe that responds to mouse dragging just like Google Earth - in fact I believe GE, at least initially, used a version of WebGL. The main advantage for presentation is better shading - Gouraud shading rather than the rather kludgy HTML5 canvas version.

A downside is that support for WebGL is quite patchy, and implementation depends also on your graphics card. I've found Chrome is fine; Firefox produces a fragment of picture and then gives an error message, and of course IE is way behind. Some other browsers have the capability but disable it by default. I understand the reason is that it creates a vulnerability to DOS attacks which send dynamic but very slow pictures.

I don't think WebGL will replace my HTML 5 versions for a while. I don't have as much control, so I can't, for example, allow you to click on the picture to bring up local station info.

The other downside is that the files are fairly large, and take a few seconds to download. So I've put them below the jump. There is the WebGL version and a snapshot of the HTML5 version. The color scheme is the same - I haven't figured out yet how to put a bar on the WebGL version. It's a direct plot from the TempLS September station anomalies - exact for each station, and shaded elsewhere.

I've added a technical update describing the methods I used here.

Tuesday, October 9, 2012

September TempLS Global Temp up 0.04°C

The TempLS analysis, based on GHCNV3 land temperatures and the ERSST sea temps, showed a monthly average of 0.53°C for September, up from 0.49 °C in August. Last month also showed a higher rise with late data. There are more details at the latest temperature data page.

Below is the graph (lat/lon) of temperature distribution for September. I've also included a count and map of the stations that have reported to this date.

Thursday, October 4, 2012


I've been battling with some persistent spammers in comments. I've invoked Captcha word verification for the time being - I hope I can remove it when things quieten down.

Monday, October 1, 2012

A necessary adjustment - Time of Observation

Sceptics complain a lot about adjustments made in indexing temperatures. Rarer is an acknowledgement of the argument for the adjustments. The fact is that if an adjustment is appropriate, then it is required. It's not optional.

This post will set out the quantitative basis for one of the larger adjustments to USHCN, a frequent object of this complaint. This is TOBS, the time of observation. It arises because USHCN gets its data from a wide variety of observers, many voluntary. The time at which min/max thermometers are read and reset is recommended but not mandated, but is on record. For many stations it has changed, and this matters.

In this post I take a USCRN station, Boulder, Colorado, with hourly data from 2009-2011. I calculate the effect of varying the notional reading time of a min/max thermometer. There is a positive bias of about 1.3°F if it is read in mid-afternoon, tapering to nearly nil around midnight. There is potentially a cooling bias in the morning, though for this site it was small.

But firstly, a discussion of why temperature measurement is relevant to the climate debate, and what kind of measure should be used.