tag:blogger.com,1999:blog-7729093380675162051.post9044368779405172565..comments2022-09-24T23:35:44.074+10:00Comments on moyhu: Trends, breakpoints and derivatives - part 2Nick Stokeshttp://www.blogger.com/profile/06377413236983002873noreply@blogger.comBlogger28125tag:blogger.com,1999:blog-7729093380675162051.post-35271421149647440722015-02-26T02:35:03.470+11:002015-02-26T02:35:03.470+11:00"Also look at graphs of 15y trends in M&F...<i>"Also look at graphs of 15y trends in M&F paper. Clearly loads of sub 15y variability in the results. Spiky as hell. "</i><br /><br /><br />What's the deal with these tossed-off remarks? As if you are such an authority that you can casually project confidence in what you are asserting. <br /><br /><br />I am still looking at factorizing the temperature time-series. The basic CSALT factors (CO2-SOI-Aerosols-LOD-TSI) generate this fit, where blue is the model<br /><br />http://imageshack.com/i/hjU14bjMg<br /><br />In keeping with the point of the top-level post, I am only applying a 12-month sliding window filter to the GISS data. If you look closely you probably notice that fine detail occasionally pops through as shoulders and minor peaks and valleys in the model fit. This is not a coincidence as it reveals the impact of ENSO on the global temperature, even when strong El Nino or La Nina conditions do not prevail.<br /><br />Further, what is interesting is that the residual noise contains spectral components consisting of higher frequency harmonics of the 21-22 year Hale cycle, particularly N=3, N=4. <br />I probably will take a look at this in more depth at the risk of being accused of over-fitting.<br /><br />After that the residual will probably lie somewhere between red and white noise.<br /><br />@whuthttps://www.blogger.com/profile/18297101284358849575noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-36922340477726880102015-02-25T11:24:29.735+11:002015-02-25T11:24:29.735+11:00"
It can be "jiggered" if you do no...<i>"<br />It can be "jiggered" if you do not account for the proper relationship between radiative forcing and surface temperature. <br />"</i><br /><br />Actually it's just a first-order perturbation. Get used to using differentials.<br /><br />How can you remove a "spurious apparent warming" -- just because you don't like what you see?<br /><br />LOD anomaly is useful because it reveals the trendless long-term temperature variability. The significance is that the trend is removed. Just like the trend is missing in the SOI as well. This leaves the trend in the residual, and you can either attribute that to an effective log(CO2) or you are going to have to get very creative.<br /><br /><br />But then when you say <i>"I haven't seen anything that impressive since Pratt's global warming to within a millikelvin farce."</i>, I realize that you have truly jumped the shark, as Pratt applied a smoothing filter that essentially removed all the fine detail that I am trying to model. And you apparently are now trying to do the same thing with these multi-year filters that you are playing around with.<br /><br />Are you really that deceitful? At least I am honest in not removing the interesting part of the time-series. Get serious and use the data as is with minimal filtering, just as I asserted in my first comment today. <br /><br />Or are you afraid of what you are going to find?<br /><br /><br /> <br /><br /><br /><br /><br /><br /><br />@whuthttps://www.blogger.com/profile/18297101284358849575noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-38544573311199828052015-02-25T10:57:54.644+11:002015-02-25T10:57:54.644+11:00"It is also well known that strong volcanic e..."It is also well known that strong volcanic eruptions can cool the climate for a few years."<br /><br />I did not say anything to the contrary, in fact I linked you to an article I've just done investing this very thing. It can be "jiggered" if you do not account for the proper relationship between radiative forcing and surface temperature. This is what is most commonly done as I documented in a list of references in that article. You appear to make the same mistake. <br /><br />One of the results is an spurious apparent warming several years after and eruption after ENSO and volcanoes have been supposedly removed. This was noted in Santer 2014. I explained where this comes from. <br /><br />In doing your kind of scatter-gun multivariate analysis this will get falsely attributed to another variable , likely CO2. That will not prevent you getting a high correlation coeff. if you have enough variables.<br /><br />BTW, dLOD is mostly transfer of momentum from solid earth to atmosphere, via evaporation and rainfall, so you are using a symptom of temperature change as a causative variable. <br /><br />But don't worry all that is just FUD, you're definitely onto a "winner" with a score like 98% . I haven't seen anything that impressive since Pratt's global warming to within a millikelvin farce. <br /><br /><br /><br /><br />Greg Goodmanhttps://climategrog.wordpress.com/2015/01/17/on-determination-of-tropical-feedbacks/noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-48660023908958852252015-02-25T07:43:06.359+11:002015-02-25T07:43:06.359+11:00Yeah, I am sure I am "up to about a dozen var...Yeah, I am sure I am "up to about a dozen variables now", when I count 6 factors that comprise essentially fixed time-series. <br /><br />I really don't care for your FUD. Everything you say is the exact opposite to the truth. <br /><br />Everyone knows that ENSO cycles correlate strongly to warm and cool periods throughout the globe. That is not under discussion, and it is certainly not a "spurious correlation". Matching an erratic time-series that represents ocean oscillations to an erratic time series representing global temperature can not be jiggered.<br /><br />It is also well known that strong volcanic eruptions can cool the climate for a few years. This also can not be jiggered.<br /><br />Dish this data up on a silver platter and Goodmann still wouldn't know what to do with it.<br /><br />@whuthttps://www.blogger.com/profile/18297101284358849575noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-44033678689009458222015-02-25T07:16:10.047+11:002015-02-25T07:16:10.047+11:00Well, Pukeite you can do Ornstein-Uhlenbeck if you...Well, Pukeite you can do Ornstein-Uhlenbeck if you think it makes you sound smarter but then just saying "And that isn't going to cut it." is hardly a response. You don't seem to realise that most your 98% of the jigsaw puzzle is probably spurious correlation. You seem to be up to about a dozen variables now. <br /><br />Then you tell me to "do it correctly" whatever you think that means. You certainly know enough about ODEs and system behaviour to know that you should not be regressing a radiative forcing directly against temp. <br /><br />Correlating ENSO or SOI to global mean temperature is a joke since about half the planet shows negative correlations and everything in between with varying lags. <br /><br />You know as little about me as I do about you, so your pathetic last comment about my motivations has probably more to do with projection than anything else.<br /><br />Now if you wanted to show that your magic 98% is more than spurious correlation you know how to do so. But you won't will you, because you are scared of what you might find. <br /><br />You prefer sounding off, being insulting and trying to impress with your long words. <br /><br />You talk the talk but you won't walk the walk. <br /><br /><br /><br />Greg Goodmanhttps://climategrog.wordpress.com/?attachment_id=283noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-30339510658244071602015-02-25T03:42:12.116+11:002015-02-25T03:42:12.116+11:00A quick test doing diff and RM on an impulse confi...A quick test doing diff and RM on an impulse confirms freq resp is sin(f) with zeroes every 15y. data inverted between 15,30y :(<br /><br />No good.<br /><br />Also look at graphs of 15y trends in M&F paper. Clearly loads of sub 15y variability in the results. Spiky as hell. <br /><br />Greg Goodmanhttps://climategrog.files.wordpress.com/2015/02/cmip5_hi_grp_low_grp.png?w=800noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-8839898870380036822015-02-25T03:39:23.782+11:002015-02-25T03:39:23.782+11:00I don't know what your problem is Goodmann, ot...I don't know what your problem is Goodmann, other than you have problems spelling your own name.<br /><br />So you want me to go through your "very valid points", eh? <br /><br />First, I don't do AR1, I do model an Ornstein-Uhlenbeck random walk process. And that isn't going to cut it. <br /><br />Next, what exactly is the problem with the well-known factors that contribute to natural variability? You call them exploratory variables. Yes, I have seen how you are thrashing and wailing about trying to make sense of the volcanic aerosols. My recommendation to you is to take a deep breath and just do it correctly. The result pops right out, just as the SOI factor pops out. Adding in the log(CO2), TSI, and LOD anomaly brings up the correlation coefficient to at least 0.98. <br /><br />That is essentially the starting point for further analysis. <br /><br />If you really want to ignore 98% of the jigsaw puzzle because you are beholden to some political agenda, that is your problem. It really is painful to watch you abnegators try to make things more difficult than they actually are, for the sole purpose of creating FUD.<br /><br /><br /><br /><br /><br /><br /><br /><br /><br /><br />@whuthttps://www.blogger.com/profile/18297101284358849575noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-89445976883677098642015-02-25T03:09:21.354+11:002015-02-25T03:09:21.354+11:00How about you learn some manners PUKE-ITE. ?
Yes,...How about you learn some manners PUKE-ITE. ?<br /><br />Yes, I misread fine. <br /><br />Now as usual you ignore the very valid points I make about your ill-conceived model and how you need to test it and why your regressions are wrong, and simply declare it "the winner in all this". <br /><br />Very powerful argument that.<br /><br />You're right, I'm really, really afraid of the power of your model and what it is going to do to the world. LOL. <br /><br /><br /><br /><br /><br /><br />Greg Goodmanhttp://judithcurry.com/2015/02/06/on-determination-of-tropical-feedbacks/noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-35154466942753501032015-02-25T02:07:33.670+11:002015-02-25T02:07:33.670+11:00Bad news Goodmann, you have a reading comprehensio...Bad news Goodmann, you have a reading comprehension problem. I clearly wrote "intra-annual", which you somehow read as "inter-annual".<br /><br />The CSALT model is the winner in all this and you are afraid of what it can do with absolutely no over-fitting. <br />@whuthttps://www.blogger.com/profile/18297101284358849575noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-8140194502682407832015-02-25T01:19:56.086+11:002015-02-25T01:19:56.086+11:00"All I really need is a window of about 1 yea..."All I really need is a window of about 1 year to suppress the intra-annual noise. "<br /><br />Bad new Webby, you can't possibly filter noise with "inter-annual" components using a window of "about a year", think about it. <br /><br />Your final para has fundamental errors of logic. Firstly that your CSALT thing means anything. It is basically an exercise in over-fitting using a now very large number of natural variables as possible explanatory variables. <br /><br />It is you that is being delusional, not your "usual suspects".<br /><br />Make yourself a similar number AR1 times series and repeat the exercise. You will probably get a very similar level of fit. Then try a Monte Carlo on your fit to real data. chop out a few 10 ro 15y segments and swap them over, then redo you fit and check the correlation. Do that 1000 time or so and do some stats. Find out which variables have any objective explanatory power. <br /><br />You may get a confirmed result on one or two. Volcanoes probably. Even that you are not regressing correctly since the climate response in not in phase with the forcing so you will not be correctly scaled. See my recent post at Curry's on that question. <br />http://judithcurry.com/2015/02/06/on-determination-of-tropical-feedbacks/<br /><br />I hope those suggestions help you beef up your model. <br /><br /><br /><br />Greg Goodmanhttp://judithcurry.com/2015/02/06/on-determination-of-tropical-feedbacks/noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-38689939571919871712015-02-25T01:01:45.806+11:002015-02-25T01:01:45.806+11:00OK, the formula you give just above is the derivat...OK, the formula you give just above is the derivative of the sinc fn. <br /><br /> (sin(x)-x*cos(x))/x²<br /><br />so my first question is why diff of sinc.? We are considering diff in TD that is not equivalent to diff in FD. <br /><br />If you are doing runing mean of diff, or diff of RM in TD. you are convoluting a two pt diff kernel with a rectangular pulse to in FD thus you multiply: <br /><br />FT(diff) * FT(pulse) = f . sinc(f) = f*sin(f)/f = sin(f) !!<br /><br />ie running mean of diff has ZERO roll-off and periodic pass-bands ( with the nice feature of inverting every other band). <br /><br />Now if you invoke of Carrick's 1/f type spectrum for the signal being analyise you may get close to your red line as the spectrum of the output. <br /><br />I should note in passing that this noise model is totally integrated white noise and does not apply to climate. Climate is not totally random as looking at any spectrum of real climate data will show.( not just temps). There is not infinitely strong long frequencies. Any real data levels off as you approach zero freq. <br /><br />If it is a truly random component it may not matter too much inverting it. If it is the strong volcanic signal that is one of the major features of both models and climate it "matters", in spades. <br /><br />Maybe this is the difference between RM of diff and a sliding trend which is min variance condition, not min deviation.<br /><br />I've been trying to check what you posted but I'm having trouble reconciling your integrals with integration by parts. You seem to assume certain things that are probably obvious to you but not necessarily to the reader. <br /><br />I'll post on the relevant thread about that. <br /><br /><br /><br />Greg Goodmanhttps://climategrog.files.wordpress.com/2015/02/cmip5_hi_grp_low_grp.png?w=800noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-59121195092220549912015-02-24T23:27:18.572+11:002015-02-24T23:27:18.572+11:00I likely will never use filters with that large a ...I likely will never use filters with that large a window. All I really need is a window of about 1 year to suppress the intra-annual noise. It is becoming more and more clear that a time-series such as the global GISS record is composed by ENSO, volcanic, CO2, etc factors and that one uses these signatures as a model-based compensation or filtering. And for the long-term variations, the temperature compensation is empirically established by the LOD anomaly.<br /><br />So the approach is to apply the same small-window filter to both the SOI and the GISS time series and compose that way.<br /><br />http://contextearth.com/2015/01/30/csalt-re-analysis/<br /><br />Moreover, this approach points out how delusional the current hysteria is over manipulation of temperature data -- see Curry, WUWT, and the usual suspects. There is no possible way that manipulation of temperature and of a metric such as SOI can be aligned so perfectly as a result of a conspiratorial strategy. <br /><br /><br />@whuthttps://www.blogger.com/profile/18297101284358849575noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-12072181943620544492015-02-23T21:36:38.272+11:002015-02-23T21:36:38.272+11:00Oops
x-x^3.6+... - (x-x^3/2+...)Oops<br />x-x^3.6+... - (x-x^3/2+...)Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-8024906464643063142015-02-23T21:35:43.816+11:002015-02-23T21:35:43.816+11:00Greg,
I'm not sure what to fix. Could you be m...Greg,<br />I'm not sure what to fix. Could you be more specific?<br /><br />The formula (sin(x)-x*cos(x))/x² tends to x/3 as x->0, and is well defined at 0. You can check that by power series expansion<br />x+x^3.6+... - (x+x^3/2+...)<br />It doesn't depend on n=10<br /><br />Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-38224712574016941392015-02-23T19:57:40.534+11:002015-02-23T19:57:40.534+11:00You seem to assume that temperature is random: int...You seem to assume that temperature is random: integrated white noise. If it was we would not all be arguing about "forcings". <br /><br />There were two major "forcing" events: El Chichon and Mt Pinatubo that inserted a very significant circa 10y signal in the latter half of the 20th. c. Inverting part of that is definitely not desirable. <br /><br />There are some errors in Nick's formula as give since it only works n=10. I would not start doing dimensional analysis until he's fixed. it. <br /><br />The transfer should not be undefined at zero, the difference operation will take out any constant term. It is correctly zero and should tend to zero in a well defined manner.<br />Greg Goodmanhttps://climategrog.wordpress.com/?attachment_id=1313noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-41347391589523980032015-02-23T17:49:39.514+11:002015-02-23T17:49:39.514+11:00One thing I should add is that Nick's transfer...One thing I should add is that Nick's transfer function is not dimensionless. This is the ratio of the trend amplitude to temperature amplitude.<br /><br />If you want to look at the dimensionless version, you need to divide by 1/f (the value at f=0 is undefined as is appropriate).<br /><br />Also, you have to look at the noise spectrum when looking at the relative magnitude of the signal that passes through the first lobe as compared to the pass band of the filter.<br /><br />Since this is also 1/f to some power, in practice, the amount of energy in the first band is much less than the 50% that Greg is quoting.Carrickhttps://www.blogger.com/profile/03476050886656768837noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-71489548175911250182015-02-23T02:01:40.681+11:002015-02-23T02:01:40.681+11:00He, he !! I exaggerated "much faster" ....He, he !! I exaggerated "much faster" . In fact I have a pretty good hunch that this is converging to gaussian as the order increases. All roads lead to Rome. Central limit theorem ? <br /><br />Forth order may still be a useful tool since it does have zero in the FD. That can be a notable selling point in many situations. For the price of small neg. lobe this could be good.<br /><br />It seems to settle a lot better than a crudely cropped 2-sig gaussian, too. <br />Greg Goodmanhttps://climategrog.files.wordpress.com/2015/02/dgauss-5y_freq_resp.png?w=800noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-49309408910201637412015-02-23T01:44:37.791+11:002015-02-23T01:44:37.791+11:00Thanks for the code. I extended that index to 4 an...Thanks for the code. I extended that index to 4 and it starts to look like a useful response curve: only a moderate neg. lobe and much faster roll-off than GD. <br /><br />That makes it good choice where G is too slow. <br /><br />Can your method be extended to produce something with that freq resp?<br />Greg Goodmanhttp://climategrog.files.wordpress.com/2015/02/dgauss-5y_freq_resp.png?w=800noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-68009215418443491552015-02-22T22:33:13.380+11:002015-02-22T22:33:13.380+11:00Greg,
I'm working through the multiplier; π²/...Greg,<br />I'm working through the multiplier; π²/6 might not be right.<br />There is interesting theory. The three curves that I plotted are spherical bessel J functions of order 1,2,3. The following R code duplicates my original plot (that you showed at CA) except for the y-scaling:<br /><br />bess=function(x,n){<br />2*(x/2)^(.5-n)*besselJ(x,n+.5)*gamma(n+1.5)<br />}<br />x=1:150/10<br />a=pi^2/6<br />y=NULL<br />for(i in 1:3)y=cbind(y,a*bess(x,i))<br />x=x*(10/pi)<br />png("bess.png",width=800)<br />plot(range(x),range(y),type="n")<br />for(i in 1:3)lines(x,y[,i],col=i+1)<br />lines(x,0*x)<br />dev.off()<br /><br />Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-70181719678100938272015-02-22T20:57:09.337+11:002015-02-22T20:57:09.337+11:00Thanks Nick,
that works for n=10 but if I put n=...Thanks Nick, <br /><br />that works for n=10 but if I put n=20 it blows up at zero. I think there's another n missing somewhere.<br /><br />Greg Goodmannhttp://climategrog.wordpress.com/noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-64842636072182920892015-02-22T18:20:00.197+11:002015-02-22T18:20:00.197+11:00Greg,
The analytic form of the spectrum is
π²/6(...Greg,<br />The analytic form of the spectrum is <br />π²/6(sin(x)-x*cos(x))/x²<br />where x=f*π/10, f=freq in 1/Cen<br />Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-55760877380818010502015-02-22T17:32:26.901+11:002015-02-22T17:32:26.901+11:00Here is the response of the gaussian-derivative. S...Here is the response of the gaussian-derivative. Since the freq resp of a gaussian is itself a gaussian the result of diff and gauss is the linear ramp of diff x gaussian and gaussian wins at high freq.<br /><br />http://climategrog.files.wordpress.com/2015/02/dgauss-5y_freq_resp.png?w=800<br /><br />The example I plotted is for sigma=5y which is a suitable replacement for the M&F 15y sliding-trend .<br /><br />It would be good to see this (with suitable frequency scaling) on the same plot as your graph in this post to allow direct comparison.<br /><br />Roll-off is slow and even but most importantly does not have negative lobes. It looks like your blue line filter may have some merit of a much faster roll-off but the neg. lobe, though smaller, is still a bit ugly. <br /><br />presumably you could take the process further and get a still improved version. <br /><br />I would like to try a Lanczos-derivative, in the same way as GD. That should have minimum overshoot/ripple and be faster roll-off than the blue line. <br /><br /><br />Greg Goodmanhttps://climategrog.files.wordpress.com/2015/02/dgauss-5y_freq_resp.png?w=800noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-60372421221570310652015-02-22T15:13:00.353+11:002015-02-22T15:13:00.353+11:00Thanks Nick,
what are the magnitude and frequenc...Thanks Nick, <br /><br />what are the magnitude and frequency of main peak and the peak in the negative lobe? Eye-balling your graph I'd guess about 11/century and 22/century..<br /><br />Do you have an analytic expression for sliding trend 'filter'. That's the red line right?<br /><br />I'd like to plot this against gaussian-derivative that I detailed in your new post.<br />Greg Goodmanhttp://climategrog.files.wordpress.com/2015/02/cmip5-xs.png?w=800noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-18364068710425205812015-02-21T20:29:19.255+11:002015-02-21T20:29:19.255+11:00ps Greg,
I have a new post up on this.ps Greg,<br />I have a new post up on this.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-57335633767198344572015-02-21T20:22:17.266+11:002015-02-21T20:22:17.266+11:00Greg,
Yes, it is. That is why I am trying to reduc...Greg,<br />Yes, it is. That is why I am trying to reduce noise with Welch filters.<br /><br />It is necessarily worse than the mean. It is a differentiating filter. That means among other things that the big central peak of the spectrum is missing, so the lobes have to look bigger.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.com