This post follows a flurry of activity in the spirit of the paper of Foster and Rahmstorf (2011). There multiple regression was used to remove from various datasets the effects of what could be seen as exogenous variables - the ENSO osciaaltion, solar flux and volcanic eruption aerosols. The result was a much more regular temperature rise, with most of the recent "slowdown" gone. In other words, the exogenous variables appeared to be responsible for the slowdown.

The method was a multiple regression in which the exogenous variables were lagged.

Update - I have a new post with a common lag parameter which seems to work better.

I blogged about this at the time, and did a display of the trends with significance, showing the great improvement that came with removing the exogenous effects.

Troy Masters took this up in a series of posts, in communication with KevinC of SkS. An improvement was the use of exponential smoothing to achieve the lag effect. Troy found that his version still left some "slowdown" in the recent decade.

A few days ago, SteveF used similar methods on Hadcrut 4, over a longer period, back to 1950. He found a more substantial slowdown than Troy, since 1997, although the trend was still positive.

Going back to 1950 is controversial. Tamino stopped at 1979 because he felt that the linear trend which was used to fit the endogenous part could not be justified going further back. I thought so to, in comments at SteveF's post, and noted the "dip in the middle" in the detrended curve. Tamino wrote a recent more emphatic post on this.

In this post, I have done a similar analysis, but trying quadratic as well as linear, and using the intervals 1979-2011 as well as 1950-2011. But I've added some features. I've used the R non-linear optimiser nlm() to optimise the lags, which are individual to each variable. And instead of detrending, I've just included the trend in a multiple regression.

Update There is a problem pointed out by SreveF that the solar component is sometimes shown with a secular trend. I have tracked down the reason - it happens because nlm() sometimes finds an optimum with a negative exponential trend coefficient. That means that in the recurrence, instead of decaying, errors grow, especially the effect rather arbitrary starting point. This potentially affects all variables, adding a growing exponential component. . I'm working on a remedy.

I have now got reasonable results by constraining the solar delay coefficient to be not less than 0.03 - SteveF's value. That keeps it away from the problem areas. I have posted new images and table.

#### Data

For exogenous variables I used:- ENSO - I followed SteveF in using Nino3.4

- Volcanic Aerosols - I used the GISS forcing Stratospheric aerosols optical depth.

- Solar - I used the SIDC sunspot count monthssn.dat"

#### Optimisation

The R function nlm() requires that you pass a function with prescribed parameters. My function just created the sum of squares of residuals, used a recurrence relation like that of SteveF to create the delay. There were up to 9 parameters - three coefficients of the exogenous variables, three coefficients for the delay, and coefficients for 1, t and t^2 (if used). I used as a starter the coefficients for linear regression with lag - 0.031 (SteveF's number). In all cases, nlm() completed with apparent convergence, although there are a few cases I'm not sure about.#### Results

Here is a table showing the trend since 1997, the regression coefficients, incl lags, and the SS. Trend means trend in °C/century from 1997-2011. The next 6 are the regression coefs of the variables, including 1,t,t^2, where t is time normalised to -1;1 on the range. Where the t^2 is NA, it means that it is linear. The next three are the fitted coefficients of the exponential smoothing. The last is the SS of residuals from the nlm() fit.Start | Trend | 1 | Vol | Sol | ENSO | t | t^2 | Vol | Sol | ENSO | SS | |

HADCRUT4 | 1950 | 0.318 | -0.00129 | -5.94377 | -2e-05 | 0.16166 | 0.30708 | NA | -0.03457 | 0.06661 | 0.11558 | 9.629 |

HADCRUT4 | 1950 | 0.446 | -0.04143 | -5.91488 | 0.00042 | 0.1628 | 0.31052 | 0.11991 | -0.03444 | 0.03 | 0.1145 | 8.799 |

HADCRUT4 | 1979 | 0.965 | -0.01416 | -2.97676 | 0.00047 | 0.12968 | 0.4747 | NA | 0.07659 | 0.89907 | 0.12001 | 3.911 |

HADCRUT4 | 1979 | 0.407 | 0.01882 | -5.61282 | 2e-04 | 0.13134 | 0.37974 | -0.21538 | 0.03504 | 0.29079 | 0.09831 | 4.019 |

GISS | 1950 | 0.81 | -0.00104 | -5.09307 | 4e-05 | 0.13374 | 0.37035 | NA | 0.04418 | 0.08153 | 0.11195 | 10.104 |

GISS | 1950 | 1.152 | -0.04222 | -3.18862 | 3e-04 | 0.12183 | 0.37283 | 0.12432 | 0.06878 | -0.11881 | 0.12758 | 9.532 |

GISS | 1979 | 1.202 | -0.0126 | -3.46822 | 0.00041 | 0.12581 | 0.46437 | NA | 0.06986 | 1.27341 | 0.10493 | 5.073 |

GISS | 1979 | 0.869 | 0.00016 | -5.38912 | 0.00025 | 0.14397 | 0.38489 | -0.11805 | 0.04297 | 0.46433 | 0.08087 | 5.145 |

NOAA | 1950 | 0.266 | -0.00123 | -5.86604 | 0.00018 | 0.14008 | 0.35172 | NA | -0.03394 | 0.03 | 0.11379 | 7.77 |

NOAA | 1950 | 0.253 | -0.01369 | -5.89777 | 0.00031 | 0.12957 | 0.36056 | 0.03824 | -0.03167 | 0.03 | 0.1235 | 7.708 |

NOAA | 1979 | 0.81 | -0.01258 | -3.0283 | 0.00037 | 0.12022 | 0.44312 | NA | 0.07208 | 1.21223 | 0.11321 | 3.568 |

NOAA | 1979 | 0.559 | 0.00211 | -4.23165 | 0.00029 | 0.11534 | 0.40219 | -0.09781 | 0.04273 | 0.6245 | 0.10207 | 3.587 |

I think some of the NOAA cases may not have converged properly - the SS is anomalously high. Possibly the quadratics starting 1979 should be discounted, since the extra regressor is really redundant.

An observation - there are some signs of non-convergence, where adding the quadratic actually raises (slightly) the sun of squares. That happens when starting in 1979 and confirms that those cases should be discounted. The quadratic didn't help there.

#### Images

Here are the various results. For plotting, the temperatures have been smoothed with a twelve month running average. SteveF has noted that the solar component sometimes has an unexpected secular component. I think this must be an error in the exponential smoothing. Checking.HADCRUT 4 Start 1950 Linear Trend Components

Start 1950 Linear Trend Time series

Start 1950 Quadratic Trend Components

Start 1950 Quadratic Trend Time series

Start 1979 Linear Trend Components

Start 1979 Linear Trend Time series

Start 1979 Quadratic Trend Components

Start 1979 Quadratic Trend Time series

GISS Start 1950 Linear Trend Components

Start 1950 Linear Trend Time series

Start 1950 Quadratic Trend Components

Start 1950 Quadratic Trend Time series

Start 1979 Linear Trend Components

Start 1979 Linear Trend Time series

Start 1979 Quadratic Trend Components

Start 1979 Quadratic Trend Time series

NOAA Start 1950 Linear Trend Components

Start 1950 Linear Trend Time series

Start 1950 Quadratic Trend Components

Start 1950 Quadratic Trend Time series

Start 1979 Linear Trend Components

Start 1979 Linear Trend Time series

Start 1979 Quadratic Trend Components

Start 1979 Quadratic Trend Time series

Would it improve accuracy to treat ocean and land separately, and possibly divide again by latitude zone?

ReplyDeleteCCE,

ReplyDeleteI think the problem is more towards overfitting; too many parameters, and subdividing wouldn't help there. I've been thinking more of jointly fitting to the three indices.

Tropics and extra-tropical zones respond so differently I really don't think this sort of thing will work on a global conglomerate temperature.

ReplyDeletehttp://climategrog.wordpress.com/?attachment_id=310

http://climategrog.wordpress.com/?attachment_id=312

NH land shows the strongest response to volcano events tropics start late and bounce back quick.

KNMI makes it pretty easy to get the necessary regional subsets, I suggest you try that way. It may provide more consistent results.

BTW if you labelled the figures it would be easier to comment.

"Going back to 1950 is controversial. Tamino stopped at 1979 because he felt that the linear trend which was used to fit the endogenous part could not be justified going further back. "

But Grant Foster aka Tamino sees no problem "justifying" using a linear detrend post 2000. But that is typical of his disingenuous unscientific science.

Perhaps a more objective period would be 17 years before 1975 to match the 17 years since rapid rise ended. That may allow a third order polynomial detrend to work. Just guessing and trying to help. Personally I don't think approach is going to be fruitful but if you want to give it a try, I'd like you to have the best chance of making it work.

I'll be interested what you get.

Greg,

DeleteMy top priority at the moment is to see why the late trends vary so much, and of course to find the reason for the secular part creeping into the solar, which SteveF spotted. Subdividing into regions would make this worse.

"Subdividing into regions would make this worse."

DeleteYou seems sure of that before having tested it.

I think your results are not stable because what you are trying to fit is not a suitable model (globally). Others fit something but have not been as open and rigorous you are being and so have not hit the irregularity.

I think splitting into regions _may_ separate out what may fit and what does not. Thus giving you a chance of more repeatable results.

I may be mistaken but its at least worth a test on 20N-70N , I would have thought. That is where you have the best chance of a stable regression fit because that is where there's the clearest volcanic response.

Greg,

Delete'But Grant Foster aka Tamino sees no problem "justifying" using a linear detrend post 2000. But that is typical of his disingenuous unscientific science. 'No, it's silly to be so judgmental. He's actually exactly right. He wants to see if the post-2000 period can be explained by ENSO etc. For that you have to use the default regressors. If you put a kink in the line to accommodate that, you'll never find out whether it could be natural.

Don't know if my suggestion was clear.

ReplyDeleteTry 20N-70N first, best chance of a clear correlation to volcanoes. Then look at 0-20N; 0-20S and 20S-70S

HTH

Nick,

ReplyDeleteSince ca 1997, a period in which volcanic forcing is somewhat flat and solar is slightly decreasing except for slight upturn post 2009, your adjustments must primarily reflect ENSO events. However, I notice that El Nino adjustments drop the temperature substantially, but the La Nina adjustments have little effect. Thus, in the quadratically detrended GISS, the big 2008 La Nina gets hardly any adjustment upwards, but the 2010 El Nino gets a big downward adjustment. Same for 1998 El Nino and the 1999-2002 La Nina. This does not seem right.

Owen,

DeleteIt should be just arithmetic. The three components add to give the green curve on the following graph, which then adds to the red to give the black.

I think the reason for your observation is that the Nina34 index is largely positive in recent years. So in La Nina it drops to zero, and the adjustment is zero.

Possibly Nino34 is not a good index to use. I followed SteveF here, but in fact it contains temperature trend itself, and when subtracted reduces the trend overall. That's why the end result recently is a more linear rise, but reaching a lower level. SOI might be better.

Nick, what are you using for the solar forcing?

ReplyDeleteEli - SIDC sunspot count

ReplyDeletehttp://www.esrl.noaa.gov/psd/gcos_wgsp/Timeseries/SUNSPOT/