Merry Christmas - and a new gallery

It's time for November temperatures with TempLS. The headline number is a cooling - from 0.48°C to 0.36°C. I'll show the usual comparative plot below.

But what I really want to talk about is a new style of presentation. The last few posts have tracked my learning curve with Javascript; now I'm coupling it with the canvas feature of HTML 5. This means I can do client-side graphics - much more flexible and interactive

What this means for November GHCN temperatures is that I can show the anomalies directly - without cell aggregation or smoothing with orthogonal functions. This is done by a triangular mesh and HTML5's neat linear color gradient function. Every station has it's anomaly directly represented with the correct color.

I found this very interesting, as it shows graphically how correlated the anomalies are. Most of the color picture is actually quite smooth. There are exceptions, and because these can be associated with large triangles, it gives an exaggerated effect there. But for the most part, large scale patterns show through well.

So here's the plot of November average temperature anomalies, in °C. It is of course interactive - you can rearrange it, magnify, show the stations, click to see station detail and numbers etc. Use Ctrl+. Ctrl- to get it the right size for your screen. The mechanics are explained below the jump.

How it works

The flat map at top right is your navigator. If you click a point in that, the sphere will rotate so that point appears in the centre. The buttons below allow modification. Set what you want, and press refresh. You can show stations, and the mesh, and magnify 2×, 4×, or 8× (by setting both). You can click again to unset (and press refresh).

Then you can click in the sphere. At the bottom on the right, the nearest station name and anomaly will appear. You may want to have stations displayed here. You'll see two faint numbers next to "stations". This indicates how much your clicked missed the station by (in pixels). It's not really a test of your mousing, but of my getting the alignment right (fiddly).

I'm continuing to work on the shading. The HTML5 function is good, but it works by RGB linear interpolation. The rainbow scheme that I use isn't linear (linear schemes are monochromatic and boring), so they get out of alignment. This sometimes means that one of the corners of the triangle does not get quite the right color. That can happen when there is a lot of change within a triangle.

Comparative plot of recent months, and the usual spherical harmonics fit.

and the Spherical Harmonics map

Previous Months

October
September
August

More data and plots

November GISS is out - down 0.07°C from October

The other surface indices were in notable unison. TempLS was down by 0.124°C, NOAA by 0.13°C and HADCrut by 0.122°C. Satellites showed less change. Here's the plot:


I had been using the GISS index based on V2 of GHCN, but that was discontinued this month. This caused a few days delay as my script, looking at the V2 file, said there was no change. There doesn't seem to be any great discontinuity caused by the change. Of course I'll be using the V3 version in the future.

Below the jump I'll show the comparison of the GISS and TempLS spatial distributions. This month I did another much more detailed spatial distribution based directly on station readings.



Here is the GISS plot of temperature distribution for November 2011:




And here is the TempLS version, using the GISS base years and levels.colors. I use a standard scale, and this is the first time I have overrun - the white patch over the Caucasus. I was reassured to find that GISS also used it's extreme color there. I did the map fairly this month (about 8th) but it still works fairly well.

Previous Months

October
September
August

More data and plots

Significant trends in Foster/Rahmstorf

Having recently finished a post on trends in unadjusted temperature indices, I see that Tamino has posted his code and results for the indices after adjustment for known exogenous effects.

This is of great interest to the kind of study I was doing. There the variations were assumed random, and assessed for significance on that basis. But of course we don't know they are random - it's just an model in the absence of better information. F&R have used more information, so I want to see what the effect is.

First a review of the methods. Some posts starting here looked at the pattern of temperature trends you could create with all possible start and end points over a period. Then I looked at how allowance for statistical significance changed the picture, and then at how a similar picture could be drawn of upper and lower CI's.

I want to show principally the latter analysis. Here's an example which will be enlarged later:


On the left you see the trends marked in color. The x-axis shows the end year of the trend period; the y-axis shows the start. The faint white lines at 45° show constant trend period, shown on the right axis.

On the right you see, in this case, the lower bound trends. That is the highest trend which allows you to say that the observed trend is significantly greater, at 95% confidence. It gives a cool side check to the trend. You can take a value, look up its color, and say that where you see that color or redder, you know the trend significantly exceeds that value.

Next I'll show the plot for all 5 indices used by Foster/Rahmstorf. This comes with the Javascript gadgetry I developed in previous posts. You can click on any point, and on the right you'll see printed the corresponding period and trends. On the corresponding time series plot, the red and blue balls will jump into position to show the trend. On that plot, there are controls that let you move the balls around to different locations. Below the graph, there are a set of radio button controls which allow you to switch the plot to any of the five datasets (giss,noaa,cru,rss,uah) used by Foster and Rahmstorf.

giss  ncdc cru  rss uah

Now the lower bound plot. It shows that trend which is less than the observed trend, but is the highest of those for which the difference from the observed is significant. In most plots, for the longer periods, the color corresponds to about 1.3°C, so the observed (adjusted) trend is significantly higher than this.

giss ncdc cru rss uah

Now the upper bound plot, converse of the above. It shows that trend which is greater than the observed trend, but is the least of those for which the difference from the observed is significant. In most plots, for the longer periods, the color corresponds to about 1.8°C, so the observed (adjusted) trend is significantly lower than this.

giss ncdc cru rss uah

Significant trend differences

In earlier posts I have shown plots of all possibly linear trends in about a century of surface temperature data from various sources. In this post I modified the plot by fading regions where the trend was not significantly different from zero.

But as commenter Frank noted, this is not the only significant difference that might be of interest. So in this post, I'll show some plots that help to bound the range of signficance.

The first plot shows, for each start and end month, the lowest trend that the observed trend would be significantly less than. That is useful for testing predictions that might be failing on the cool side. For example, if you think that there is a claim that the trend over a period should have been 2°C/century, you can see where the actual trend was significantly (at 95%) below that.

The second plot, below the jump, shows the converse. What is the highest trend that was significantly (95%) below the observed. This is probably mainly of interest in establishing whether a trend was significantly above zero, but you might be interested in other values - if you think a theory significantly under-predicts.

The final plot is of the t-statistic - the trend normalized by its standard error. This lets you look at other degrees of significance - mainly relative to zero, but in conjunction with the other plots, you can work out other trend comparisons too.

I've retained the apparatus whereby you can check each point (by clicking) against its plot, echoing the numerical trend value (and period). I've changed from earlier plots of this kind by allowing trend periods down to 1 year.

Purpose

I should at this stage say that my purpose here is to show how the much invoked arithmetic of trend fitting works out. I'm not saying that it is always a good thing to do, and there are cautions about what significance means.

Saying that a trend is significantly different from a base trend is saying that the on the null hypothesis that the data is formed from the base trend plus random noise the observed result is improbable. Cautions:

• Lack of significance does not mean the base trend is right. It just means it is consistent with this data. Many other possibilities would also be consistent.
• Significance does not mean that any physics, say AGW, is disproved. It just means that there may be something more than random variation plus trend.
• And it may not even mean that. It says the result, on the null hypothesis, is improbable. But improbable things happen. There are about half a million dots on the longest plots. If they were independent, and each had a 5% chance of being in a certain range, then that means 25000 significant dots. Even with correlation, you might still expect something like 5% of the area to show as significant. 

Plots

So here is the first plot, showing which trends the observed trend  would be significantly less than. To use it, pick a trend (color) you want to test from the legend. That and bluer redder colors indicate regions where the trend observed was significantly less than the test (color).  I have reversed the coloring custom of earlier posts, marking zero with dark brown, and 1.7°C/Century with gray. For the associated plot and mode of operation, see this post. But note that you can click anywhere on the plot to show the real trend there (shown in text and also by the time series plot).

1999-now 1989-now 1960-now 1901-now

Land and Ocean Hadcrut GISSLO NOAA UAH MSU.RSS TempLS

Land Only BEST GissTs CRUTEM NOAAland Sea Surface HADSST2

And here is the second plot, showing trends where the observed would be significantly greater. The brown marks the edge of the area where the trend is significantly greater than zero.

1999-now 1989-now 1960-now 1901-now

Land and Ocean Hadcrut GISSLO NOAA UAH MSU.RSS TempLS

Land Only BEST GissTs CRUTEM NOAAland Sea Surface HADSST2

And the third plot, which shows the t-statistic, or ratio of the trend to its standard error. For the number of degrees of freedom here, this is distributed normally, and 1.96, marked in brown, is the level of 95% significance. 1.64 is 90%, and 2.58 is 99%. One observation here is that there is only a small fringe region where a choice of a different test level would alter the result..

1999-now 1989-now 1960-now 1901-now

Land and Ocean Hadcrut GISSLO NOAA UAH MSU.RSS TempLS

Land Only BEST GissTs CRUTEM NOAAland Sea Surface HADSST2

Google Map problem fixed

Well, I'm hoping. I posted a while ago a Google Maps rendition of the GHCN surface temperature network. The Javascript mechanics made it possible to select and display all kinds of subsets, and even show a movie.

I heard some reports that people were having trouble making it display. I'd been struggling a bit to make robust Javascript, but that was getting better. There's a complication that I have to have a valid GM ID, but that seemed OK - users shouldn't need one.

Today I found what I think was the problem. I've recently started using an Amazon bucket, and using a newly acquired Web address www.moyhu.org. But it seems that DNS servers don't always get to the right place using that. So I'm using the full Amazon address.

It seems to work now. It's here.

Nov temps displayed with HTML 5

It's time for November temperatures with TempLS. The headline number is a cooling - from 0.48°C to 0.36°C. I'll show the usual comparative plot below.

But what I really want to talk about is a new style of presentation. The last few posts have tracked my learning curve with Javascript; now I'm coupling it with the canvas feature of HTML 5. This means I can do client-side graphics - much more flexible and interactive

What this means for November GHCN temperatures is that I can show the anomalies directly - without cell aggregation or smoothing with orthogonal functions. This is done by a triangular mesh and HTML5's neat linear color gradient function. Every station has it's anomaly directly represented with the correct color.

I found this very interesting, as it shows graphically how correlated the anomalies are. Most of the color picture is actually quite smooth. There are exceptions, and because these can be associated with large triangles, it gives an exaggerated effect there. But for the most part, large scale patterns show through well.

So here's the plot of November average temperature anomalies, in °C. It is of course interactive - you can rearrange it, magnify, show the stations, click to see station detail and numbers etc. Use Ctrl+. Ctrl- to get it the right size for your screen. The mechanics are explained below the jump.

How it works

The flat map at top right is your navigator. If you click a point in that, the sphere will rotate so that point appears in the centre. The buttons below allow modification. Set what you want, and press refresh. You can show stations, and the mesh, and magnify 2×, 4×, or 8× (by setting both). You can click again to unset (and press refresh).

Then you can click in the sphere. At the bottom on the right, the nearest station name and anomaly will appear. You may want to have stations displayed here. You'll see two faint numbers next to "stations". This indicates how much your clicked missed the station by (in pixels). It's not really a test of your mousing, but of my getting the alignment right (fiddly).

I'm continuing to work on the shading. The HTML5 function is good, but it works by RGB linear interpolation. The rainbow scheme that I use isn't linear (linear schemes are monochromatic and boring), so they get out of alignment. This sometimes means that one of the corners of the triangle does not get quite the right color. That can happen when there is a lot of change within a triangle.

Comparative plot of recent months, and the usual spherical harmonics fit.

and the Spherical Harmonics map

Previous Months

October
September
August

More data and plots

A Javascript Gallery

My last post had a Javascript worldmap of GHCN stations. I think this approach is very promising, and I'm hoping to improve with HTML 5 features soon.

Something new - I have an Amazon S3 bucket which I'm using for backup storage. This caused a hiccup with the last post, when I had not adjusted the permissions properly. I could see it, but probably no-one else could. I hope it works now. Please let me know if there are problems.

Anyway, this post is about a gallery that I've compiled with the various recent Javascript and other interactive graphics collected. It's a page on the blog, which you can also get to through the links on top right. There is a table of contents. I'm hoping to keep adding to it.

Google Maps display of GHCN stations, with Javascript

I have been using KML files with Google Earth for displaying temperature stations so that they can be colored and selected according to inventory information. Steven Mosher got me started on this, and he has again pointed me towards using Google Maps interactively.

This is the next in my Javascript learning process. V3 of the Google maps API is well integrated with JS, and the language gives much more flexible control than KML. So in the display presented here, you can select subsets of the GHCN V3 stations according to inventory and data properties, changing their color or making them disappear. Properties include start, end and duration of data, and urban and airport status.

There is also an animation capability. You can set up a movie to run over a period, and you will see the stations in the chosen category appaer and fade reflecting the availability of data in those years.

You can click on each marker to bring up a balloon with information. Rolling the mouse over shows the name of the station.



Implementation of Javascript is not wholly reliable - IE requires tweaking for permission, and there can be clashes with the Blogger software etc. You may find that looking at the original html from here will work better, and will also give an uncluttered screen. An option is to download the html. Use Ctrl+ and Ctrl- to fit well on the screen.






















































More detailed usage information is below the jump.



You have available all the Google Maps controls. On the right are the controls for this widget, in three sections. You should make selections from the middle section. For the selection to be effective, the left button should be checked. All regular buttons act as toggles, and the label reflects the current state; clicking moves to an opposite state. You can write numbers in the text boxes, and toggle the "<" sign to affect the interpretation. The clear button clears the choices, and All overrides.

When you click an action button, all stations which satisfy any one of the chosen conditions will be re-rendered in that color. You may want to make a note of your choice, because the logic can get tangles after a few choices. Of course, to can just use the All button to restore.

To work the movies, choose a time interval and a pause (time between frames in sec). Then click the movie button, where the label will switch to "Show". Then click an action button; currently selected stations will be rendered in that color in each year for which they have data. The years tick over just above the Movie button. Normally you'll start from no stations visible, and they'll become invisible again after data finishes. Even if they start colored, they will still disappear after data ends.

Here are some suggestions:

• At start, the All button will show. Try just changing the colors with an action button.
• Try coloring by urban status. First All and yellow, then clicked the Mixed button, to show Not M. That means Mixed, the GHCN class between Urban and Rural, will stay yellow. Then, with Urban showing (and All not showing - toggle if it is), click the left radio button beside it, and then click pink. Urban stations will then be pink. Then change Urban to Rural, and click cyan. Now you will have a display showing the three different classes.
• Then click Clear, and then set Duration to less than 70. Click Invisible, and you'll be left with the Urban/Rural coloring for stations with at least 70 years data.
• Finally, a movie. Click Clear, All, Invisible. Then choose some movie years - say 1900-2011, and a pause of 1 sec. Click the Movie button; it changes to Show. Then click a color (Yellow, say) to run the movie. It doesn't matter what order you click buttons prior to the Action.

You can run a movie on, say, airports, but be aware that it is using current classifications, so you'll see 19th century airports.

I'll produce more of these, and maybe one for all stations, but data size (loading time) may be a drag.