Impulse responses and the Spencer/Dessler Cloud models

I've been involved in discussions at this CA thread, which has gone into FFT analysis of the relation between temperature and cloud contribution to TOA energy flux, as discussed a lot re papers from Dessler and Spencer and Braswell.

I should hasten to add that the analysis at CA is based on the use of all CERES data, which Dessler did not do. He gave good reasons for his choice to use reanalysis instead. Consequently, I don't think this alternative approach is telling us anything useful about clouds. However, there is interesting maths.

Commentator Bart there proposed a model in which an impulse function is found by FFT which, when convolved with surface temp (Hadcrut3) will reproduce that "cloud forcing" function, in his notation dR (for δR_cloud). He has produced a lot of Matlab analysis. His contention is that the low frequency behaviour of the impulse response shows strong negative feedback. I don't now agree with that - I think he is looking at very low frequency results which can't be supported by the short period (about 10 years) of data. Furthermore the integral of the impulse response that he cites is the low frequency limit where it becomes just the ratio of the time gradients of the two data sets as determined by OLS regression.

Anyway, I translated his code into R and did some comparison calculations, which I put n a page attached to this blog. I'm now transferring it to this post, in case people want to comment. I'm hoping to write a more detailed analysis soon. The data being used, referred to in the code as flux.csv, is here.



I have converted Bart's matlab code to R - see below. I've only done as far as the impulse response so far.
I'm testing a Hanning window - a cos^2 function which tapers to zero at both ends of the data.

When you FFT a finite set of data, it's as if you did the full set with a gate function. You have to accept low frequency problems with finite data, but the gate adds HF signal as well. A Hanning window tapers much faster in the freq domain, and mitigates this extra problem.

First a check that my R code is getting Bart's result. I've commented out the taper line:





Now after applying the Hanning taper to temp and dR

Update. Steve at CA asked to see if these impulse responses could be convolved with the temp to give dR, which is really what they are for. It's a useful check. In R, you have to use type="circular" and conj=F in the convolve() options. That done, it works. In the plots below, black is the original dR (cloud radiance), red uses the unsmoothed impulse response, and the cyan is the smoothed impulse response. I made the black wider so you can see the red superimposed.

The plot on the right is the same, but smoothed with a 10 month triangle filter. This was done without the Hann tapering of the data. I should clarify that the smoothed impulse response referred to here is the top graph. I have not showed the unsmoothed response - perhaps Bart has. 

Update I've added in dark gold the result of convolving with the taper (w) that Bart used on h. This taper makes h effectively one-sided in time.

Here's the R code. I've tried to follow Bart's notation and steps.
Update - the code originally here deviated in minor ways - I've added fixes

data=read.csv("flux.csv",skip=0);
data[is.na(data)]=0;
temp=data[,9];
dR=data[,5]-data[,8];
N=length(dR);
h=abs(1:N-64.5)/62.5
h[h>1]=1;Ha=cos(h*(pi/2))^2;  #  Hanning Window
temp=temp*Ha; dR=dR*Ha; #  Tapering
dT=1/12;
Nsamp=8192/2;
Npad=Nsamp-N;
X=fft(c(temp,rep(0,Npad)))+1.0e-9;
Y=fft(c(dR,rep(0,Npad)));
h = Re(fft(Y/X,inv=T))/dT/Nsamp;
Nc=Nsamp/8;
w=c(rep(1,Nc),(1-cos(pi*(1:Nc-1)/(Nc-1)/2)),rep(0,Nsamp-6*Nc))
hw=h*w;
f1=c(1:15,15:1); f1=f1/sum(f1);
hs=filter(h,f1)
t=(0:599)*dT
png("impresHanning.png");
plot(t,hs[1:600],type="l",ylab="Impulse Response W/m2/C/yr",xlab="Years",main="Cloud-Temperature System Smoothed Impulse Response/Hanning")
dev.off()

Late final extra

Carrick asked to see the mag plot of h. I've followed Bart's scheme, with log axes. It's weighted towards low frequencies. This h has neither my Hann taper of data nor Bart's h truncation. Mag and phase: