Thursday, July 30, 2015

Moyhu data updates

For a while now, I have been maintaining updated data tables and graphics. Many are collected in the latest data page, but there are also the trend viewer, GHCN stations monthly map, and the hi-res NOAA SST (with updated movies). These mostly just grew, with a patchwork of daily and weekly updates.

I'm now trying to be more systematic, and to test for new data every hour. My scheme downloads only if there is new data, and consumes few resources otherwise. My hope is that all data will be processed and visible within an hour of its appearance.

I have upgraded the log at the bottom of the latest data page. This is supposed to record new data on arrival, with some diagnostics. Column 1 is date, which is actually the date listed at origin, translated to Melbourne time. The column headed "Delay" is the difference between this date and the date when processing is finished and the result should be on the website. I'm using this to find bugs. The date in the first column isn't totally reliable; it is the outcome of various systems, and may predate the actual availability of the data on the web. The second column is the name with link to the actual data. For the bigger files (size, col 3) a dialog box will ask whether to download. The "Time taken" is the time used by my computer in processing (again, for my diagnostics). Where several datasets are processed in the same hourly batch, this time is written against each of them. Currently, only the top few most recent lines of the log are useful, but new data should be correctly recorded in future.

NOAA temperature is a special case. It doesn't have the files I use in a NOAA ftp directory, but serves them with the current time attached. I have to use roundabout methods to decide whether they are new and need to be downloaded (I use their RSS file). By default they show as new every hour - I have measures to correct this, but they may not be perfect. Anyway, the times in the log for NOAA are not meaningful.

I have a scheme for doing the hourly listening only when an update is likely (assuming approx periodicity). If data arrives unexpectedly, it will be caught in a nightly processing.

It is still a bit experimental - I can't conveniently test some aspects other than just waiting for new (monthly) data to appear and be processed. But I think the basics are working.

Wednesday, July 22, 2015

Record warm periods

In the previous post, I cited the NOAA June temperatures report. This noted various periods of time with average temperatures that exceeded anything comparable in the record, particularly recent twelve month periods.

Sou has been tracking the average so far in this calendar year. Steve Bloom doesn't like non-physical calendar periods, and prefers the running average. I think Sou's analysis makes sense. It is the best guide to the 2015 year average, since it uses data from that year only. One could say people shouldn't focus on arbitrary year divisions, but they do.

However, if you want running twelve month averages, they are available at the maintained active plot. Just click the buttons on the side table headed Sm. twelve month running is the default (and only) smooth. Here is an example:

However, in this context, I'm a fan of polar, or radial, plots. Here curves track like a clock with time, with radial distance indicating temperature or other plot variable. I use it for ice extent here. Then there is a natural period of a year, but that isn't essential. The point is that by rolling it up, a long stretch of time can be covered with good resolution (but crowding).

So below the fold I'll show radial plots (with decade winding period) for the main surface indices, from 1950 to now. They will show clearly how warming has continued, as the curve spirals outward, even during the "hiatus". Now they are at record radius, and that is very likely to increase, at least for a little while. The reason is that the change in moving average depends on both the new readings, and the old readings that they replace. Mid-2014 was a relatively cool period, so as long as 2015 is warmer, the running mean will increase.

Tuesday, July 21, 2015

NOAA up 0.02°C in June; GISS now up 0.04°C

I don't normally post separately for the NOAA NCEI global temperature anomaly index, but this month is the first where everyone is using the new V4 ERSST. There is also a revision of GISS post-V4 (thanks to GISS for the acknowledgement). So it is an opportunity too to review how well TempLS matches in the new environment.

The NOAA report has June at 0.88°C relative to 20Cen; up from 0.86°C in May. GISS is now 0.8°C, up from 0.76°C. Both these are the hottest June in the respective records, and would be close to the hottest month ever, if it were not for Feb-Mar of this year. I have described in my GISS post where it was hot and cold, with more quantitative information here; the NOAA report presents a similar account in more detail.

I have noted earlier how, as expected, TempLS mesh and GISS tend to go together, being interpolated, and NOAA and TempLS grid also have an affinity. Indeed, at times, TempLS and NOAA have been eerily close. With V4 that has been somewhat interrupted, although the general pattern still holds, and the correspondence is currently close. I'll show plots below the fold.

Incidentally, the revised GISS now shows nothing unexpected in the plot of recent monthly differences (earlier version here). Here how it looks now:

Sunday, July 19, 2015

A breakdown of monthly temperature variation

A by-product of the new TempLS mesh is the analysis that can be done using the arrays of residuals and weights. The weights are mesh areas assigned to stations, and the global anomaly is the thus-weighted sum of residuals. This sum can be partitioned into regions. Two calculations are then of interest:
AveT = sum_R(w*r)/sum_R(w)
which is the average T for region R, and
Contrib_r = sum_R(w*r)/sum_G(w)
which is the proportional contribution R makes to the global T. That means you can see how those contributions add. So instead of saying it was cold in N America, you can see just how much that cold pulled down the global average.

Below the fold I'll show both of those for continent-sized regions, and also the ocean, for the months of 2015. Note that they are not exactly sums over the continent areas, but over the areas that are assigned to nodes within the regions. Each month has a different mesh, so these area assignations vary slightly - I'll show a third plot to quantify that. Of course, the regions don't change (except for ocean freezing), so to the extent it otherwise varies, it's an error.

I'll show how you can see what exactly contributed to the ups and downs of temperature this year.

Thursday, July 16, 2015

GISS (new version) up by 0.03°C in June

The anomaly for GISS in June was 0.76°C (h/t JCH). This was just as I had expected, since TempLS mesh rose from 0.618°C to (now) 0.673°C. However, meanwhile the GISS May number increased from 0.71°C to 0.73°C, so the rise was not quite as great.

Behind that lies a story. As Olof noted, GISS this month switched from using ERSST V3b to ERSST v4 (I switched last month). They show some plots of the differences here. Here is the difference plot

In the new file April 2015 is the same as before, and May only increased by 0.02°C. But the plot shows an annual difference of more than 0.05°C in recent years. I plotted the monthly differences (GISS v4 - 3b) over the last five years:

April and May do look quite different. I wonder if GISS is still using v3b there? Anyway, the usual maps and commentary are below the fold:

Wednesday, July 8, 2015

TempLS shows June anomaly up by 0.05°C

Most of the GHCN V3 data is now in, and ERSST v4, and TempLS mesh (the new V3)  shows a rise from 0.618 to 0.665°C, relative to a 1961-90 base period. TempLS grid rose from 0.674 to 0.711 °C. Usually the mesh weighted version is more likely to agree with GISS and the grid version with NOAA, though this time they both are similar.

The rise is somewhat at variance with the NCEP/NCAR index, which suggested a fairly cool month, although that is muddied somewhat by the May oddity when that index suggested more warming than TempLS mesh and GISS showed. Anyway the TempLS calc is certainly warm, and for TempLS grid, seems to be almost a record, falling just short of the 0.718°C for Feb 1998.

The warm places were western US and central Russia, with cold in Antarctica and the E Mediterranean, pretty much as indicated in the NCEP/NCAR report. As I mentioned in my previous post, you can get more detail in the regular WebGL map here, which shows the actual station anomalies with shading between.

There was a curious delay with the Canada data which slightly set back this report. The initial posted data was a duplicate (mostly) of May, and flagged as such, so TempLS rejected it. That seems to have been corrected, and now 4266 stations (incl SST cells) have reported, which is almost all that we can expect.

In other news, Arctic ice has held up well in the last month, but the last two days of JAXA show rising melt, and Neven says more heat is on the way.

Tuesday, July 7, 2015

Maintained WebGL map of past GHCN/SST station anomaliess

I have a maintained page here which shows anomalies for individual land stations and SST cells, averaged over each month. Land data comes from GHCN V3 unadjusted, and SST from ERSST, formerly ver 3b. It is a shaded color map, with true color for each station, and linear interpolated color in between. I think it's the best way of seeing the raw data.

Now that I am doing daily runs of TempLS mesh, and this produces much of the necessary data, I have decided to make production of these maps a subsidiary process. This saves duplication, and keeps up to date with versions (using now ERSST 4). But what may be more useful is the daily updating. That means that the latest month can be tracked progressively as data comes in. So today, I am waiting on data from Canada and some Latin American countries before posting the TempLS result for June. But you can see it here, and it shows how TempLS calculates its interim result with interpolation. Of course, you could wait till tomorrow when Canada will probably be in.

I have also extended the range back to 1900. You can examine any month in detail. I haven't yet done the annual and decade averages back beyond 1990, but soon.