Adjusting global temperatures for ENSO, Solar and volcanoes

This post follows a flurry of activity in the spirit of the paper of Foster and Rahmstorf (2011). There multiple regression was used to remove from various datasets the effects of what could be seen as exogenous variables - the ENSO osciaaltion, solar flux and volcanic eruption aerosols. The result was a much more regular temperature rise, with most of the recent "slowdown" gone. In other words, the exogenous variables appeared to be responsible for the slowdown.

The method was a multiple regression in which the exogenous variables were lagged.
Update - I have a new post with a common lag parameter which seems to work better. 

I blogged about this at the time, and did a display of the trends with significance, showing the great improvement that came with removing the exogenous effects.

Troy Masters took this up in a series of posts, in communication with KevinC of SkS. An improvement was the use of exponential smoothing to achieve the lag effect. Troy found that his version still left some "slowdown" in the recent decade.

A few days ago, SteveF used similar methods on Hadcrut 4, over a longer period, back to 1950. He found a more substantial slowdown than Troy, since 1997, although the trend was still positive.

Going back to 1950 is controversial. Tamino stopped at 1979 because he felt that the linear trend which was used to fit the endogenous part could not be justified going further back. I thought so to, in comments at SteveF's post, and noted the "dip in the middle" in the detrended curve. Tamino wrote a recent more emphatic post on this.

In this post, I have done a similar analysis, but trying quadratic as well as linear, and using the intervals 1979-2011 as well as 1950-2011. But I've added some features. I've used the R non-linear optimiser nlm() to optimise the lags, which are individual to each variable. And instead of detrending, I've just included the trend in a multiple regression.

Update There is a problem pointed out by SreveF that the solar component is sometimes shown with a secular trend. I have tracked down the reason - it happens because nlm() sometimes finds an optimum with a negative exponential trend coefficient. That means that in the recurrence, instead of decaying, errors grow, especially the effect rather arbitrary starting point. This potentially affects all variables, adding a growing exponential component. . I'm working on a remedy.

I have now got reasonable results by constraining the solar delay coefficient to be not less than 0.03 - SteveF's value. That keeps it away from the problem areas. I have posted new images and table.

Data

For exogenous variables I used:

• ENSO - I followed SteveF in using Nino3.4
  
• Volcanic Aerosols - I used the GISS forcing Stratospheric aerosols optical depth.
  
• Solar - I used the SIDC sunspot count monthssn.dat"
  

For the temperature variables, sources are listed here

Optimisation

The R function nlm() requires that you pass a function with prescribed parameters. My function just created the sum of squares of residuals, used a recurrence relation like that of SteveF to create the delay. There were up to 9 parameters - three coefficients of the exogenous variables, three coefficients for the delay, and coefficients for 1, t and t^2 (if used). I used as a starter the coefficients for linear regression with lag - 0.031 (SteveF's number). In all cases, nlm() completed with apparent convergence, although there are a few cases I'm not sure about.

Results

Here is a table showing the trend since 1997, the regression coefficients, incl lags, and the SS. Trend means trend in °C/century from 1997-2011. The next 6 are the regression coefs of the variables, including 1,t,t^2, where t is time normalised to -1;1 on the range. Where the t^2 is NA, it means that it is linear. The next three are the fitted coefficients of the exponential smoothing. The last is the SS of residuals from the nlm() fit.

	Start	Trend	1	Vol	Sol	ENSO	t	t^2	Vol	Sol	ENSO	SS
HADCRUT4	1950	0.318	-0.00129	-5.94377	-2e-05	0.16166	0.30708	NA	-0.03457	0.06661	0.11558	9.629
HADCRUT4	1950	0.446	-0.04143	-5.91488	0.00042	0.1628	0.31052	0.11991	-0.03444	0.03	0.1145	8.799
HADCRUT4	1979	0.965	-0.01416	-2.97676	0.00047	0.12968	0.4747	NA	0.07659	0.89907	0.12001	3.911
HADCRUT4	1979	0.407	0.01882	-5.61282	2e-04	0.13134	0.37974	-0.21538	0.03504	0.29079	0.09831	4.019
GISS	1950	0.81	-0.00104	-5.09307	4e-05	0.13374	0.37035	NA	0.04418	0.08153	0.11195	10.104
GISS	1950	1.152	-0.04222	-3.18862	3e-04	0.12183	0.37283	0.12432	0.06878	-0.11881	0.12758	9.532
GISS	1979	1.202	-0.0126	-3.46822	0.00041	0.12581	0.46437	NA	0.06986	1.27341	0.10493	5.073
GISS	1979	0.869	0.00016	-5.38912	0.00025	0.14397	0.38489	-0.11805	0.04297	0.46433	0.08087	5.145
NOAA	1950	0.266	-0.00123	-5.86604	0.00018	0.14008	0.35172	NA	-0.03394	0.03	0.11379	7.77
NOAA	1950	0.253	-0.01369	-5.89777	0.00031	0.12957	0.36056	0.03824	-0.03167	0.03	0.1235	7.708
NOAA	1979	0.81	-0.01258	-3.0283	0.00037	0.12022	0.44312	NA	0.07208	1.21223	0.11321	3.568
NOAA	1979	0.559	0.00211	-4.23165	0.00029	0.11534	0.40219	-0.09781	0.04273	0.6245	0.10207	3.587

I think some of the NOAA cases may not have converged properly - the SS is anomalously high. Possibly the quadratics starting 1979 should be discounted, since the extra regressor is really redundant.

An observation - there are some signs of non-convergence, where adding the quadratic actually raises (slightly) the sun of squares. That happens when starting in 1979 and confirms that those cases should be discounted. The quadratic didn't help there.

Images

Here are the various results. For plotting, the temperatures have been smoothed with a twelve month running average. SteveF has noted that the solar component sometimes has an unexpected secular component. I think this must be an error in the exponential smoothing. Checking.


HADCRUT 4 Start 1950 Linear Trend Components


Start 1950 Linear Trend Time series

Start 1950 Quadratic Trend Components

Start 1950 Quadratic Trend Time series

Start 1979 Linear Trend Components

Start 1979 Linear Trend Time series

Start 1979 Quadratic Trend Components

Start 1979 Quadratic Trend Time series

GISS Start 1950 Linear Trend Components

Start 1950 Linear Trend Time series

Start 1950 Quadratic Trend Components

Start 1950 Quadratic Trend Time series

Start 1979 Linear Trend Components

Start 1979 Linear Trend Time series

Start 1979 Quadratic Trend Components

Start 1979 Quadratic Trend Time series

NOAA Start 1950 Linear Trend Components

Start 1950 Linear Trend Time series

Start 1950 Quadratic Trend Components

Start 1950 Quadratic Trend Time series

Start 1979 Linear Trend Components

Start 1979 Linear Trend Time series

Start 1979 Quadratic Trend Components

Start 1979 Quadratic Trend Time series