Sunday, October 30, 2016

Chaos, CFD and GCMs.

There has been a flurry of skeptic blogging (and commentary from me) on chaos and climate models. It's generally along the lines that chaos renders GCMs unworkable because of small changes magnifying or some such, with words like coupled and non-linear. Kip Hansen has a series at WUWT, finishing here. Like many such, it shows the Lorenz trajectories produced by a set of three slightly non-linear equations. I'll develop that with a gadget to explore these curves and their attractor in a future post. Tomas Milanovic has one of an intermittent series of posts (latest "Determinism and predictability") at Climate Etc, of which the general theme is the unsolvability of Navier-Stokes equations due to some effect of non-linearity negating proof of existence and uniqueness, or some such.

My standard response to all this is, look at Computational Fluid Dynamics (CFD, which has been my professional activity for the last thirty years). It is a major established engineering tool based on numerically solving the Navier Stokes equations, and has dealt with the chaos (turbulence) from the beginning. And the climate models are just large scale CFD. There are certainly difficulties with the solution, mainly to do with the necessary sub-grid modelling (in both CFD and GCMs). But they aren't to do with the fact that the solutions don't relate to initial conditions. In fact, that is a benefit, since initial conditions are hardly ever known accurately.

And the theoretical issues of existence and uniqueness etc don't impinge on practice. Algorithms are used which generate solutions on a gridded or meshed space with time stepping. These solutions satisfy on that scale the conservation laws of momentum, mass and energy, which are also expressed by the N-S equations. If you find such a solution, it doesn't matter whether it's existence could be proved in advance. As for uniqueness, the solution procedure itself will generally indicate whether different solution pathways are possible. One CFD scientist, David Young, has been objecting that some recent work, in which he has a part, does show non-unique solutions. But as far as I see, this is in situations like near-stall on a wing, where reality itself is far from predictable.

The CE post had an odd answer to this - yes, CFD works, but only on a scale of up to a few metres. This is of course unphysical - there is no such restriction on the physical laws, nor in the discretised algorithms is any physical scale limitation built in. And of course, GCM's are just Numerical Weather Prediction (NWP) programs, run for longer periods. Most sensible people concede that these work quite well, despite the many km scale.

What people who like to show fancy chaos pictures rarely dwell on is the nature of attractors. These are what distinguish chaos from randomness. And they are typically the results that are sought from CFD analysis. In CFD, initial conditions are usually just a nuisance (because you rarly have good data, and when you try and specify them, there is usually something that will generate unintended disturbances). The standard remedy is to run the program for a while to let these settle out. This takes advantage of the fact that initial conditions are swept away in chaos. GCM's do the same. They typically "wind back" to start at some time well before the period of interest. This would be bad if initial conditions mattered, because data back then is less reliable. But it isn't bad, because they don't. Again, it is better to let artefacts settle before the solutions are needed.

This lack of concern with initial conditions in a search for attractors, relates to the frequent criticism of GCMs as predictors. GCM's find out about climate (attractor), but don't predict the trajectories that converge to them (weather). That relates to the initial condition issue - models can only generate trajectories that are possible in the circumstances, not ones that will reproducibly happen.

When trying to explain why GCMs do really work, and attractors are the key, I often post this GFDL video of modelled ocean SST over seasons. I say that it shows many transient effect, from various eddies to longer term events like ENSO. None of these are predictions for Earth. The actual eddies won't happen, nor will the ENSO events, at least not at the stated times. But this solution which just came from specifying bottom topography and various long term forcings (energy input) comes up with familiar patterns like Gulf Stream and other major ocean currents. The wiggles vary, but the current is there. There is underlying physics which determines the transfer of heat from the Caribbean to the North Atlantic. And GCMs can tell you how that effect of physics will relate to changes in forcing. Anyway, here is the video:

In my next post, I'll develop the notion of an attractor using the simple Lorenz system of differential equations. These show two important things. Trajectories follow a path with a pattern that is, after some convergence from the initial point, similar for all cases. This is the attractor, and in contrast to the hypersensitive dependence on initial conditions, the dependence of that trajectory on the three parameters of the system is gradual, although over the full range allows many very different shapes. To do this, I'll show a Javascript/WebGL gadget that allows you to vary initial conditions and parameters, and visualise the trajectories in 3D.


  1. Thanks for taking up the topic here at this time. I look forward to it. A quick (for now) question: are probability and statistics (and geostatistics) viable concepts in characterizing attractors? is ergodicity? I am just looking for a little guidance--are these topics obvious or deep waters?

    1. Ergodicity seems like a key property. The existence of an attractor is the antithesis of equiprobable. With a long sequence, you could count occurrences of location, so I guess that is a kind of frequentist probability.

      This was part of my interest in developing the WebGL gadget for the Lorenz butterfly. I want to see how I can actually trace the attractor. My idea was to identify a region in space through which the attractor was passing, with estimated direction, and then to plot dots on a normal plane where trajectories went through, to see if they converge to an identifiable point, or if I have to settle for some sort of fuzzy average. That is a bit of mechanics I haven't yet got working. I think it's important, because it relates to whether you can have a unique climate state derived from an ensemble average.

      I don' think anything about your question is obvious. There is a lot of theory, going back to Poincaré and Boltzmann before, with a lot from Kolmogorov and colleagues in the '50s. But I'm sure you know all this.

    2. Is it a biennial attractor? Which is a way of saying that the ocean is modulated by a metastable 2-year period, subject to the known angular momentum variation forcings (Chandler wobble plus periodic lunisolar gravity ).

      No longer any need for the toy Lorenz models. Maybe its time to go with the real geophysics?

  2. Nick, I'm not going to comment at length except to dispute your statement about our multiple solutions paper. I suggest you look at AIAA 2005-4729 by Mavriplis to see what can happen in attached mildly transonic flow. Such are usually hidden and the literature is very misleading. I don't know what experience you have with NS but if you rely on the literature you could be wrongly informed

    1. David,
      Again the context for my comments is where people are claiming that there is something fundamentally wrong with modelling fluid flow. There is chaos and N-S equations can't be solved. or some such. And I look at Mavriplis paper, where the workshop is getting agreement with experiment to within 10% for flows at far higher mach number than found in the atmosphere, and worrying about discrepancies between grids of similar order. The very existence of the workshop proves again that CFD is a major and successful engineering activity.

      I'll note again that Mavriplis' study inolves separated flow - not a big issue with GCMs. And he says in the introduction:
      "Generally, for cases with minimal amounts of separated flow, state-of-the-art Reynolds averaged Navier-Stokes (RANS) solvers can be used with a high degree of confidence. "

    2. David Young really has no clue on how to do geophysics. He makes some claim about air flow over a wing and therefore concludes that all modeling of atmospheres and oceans is somehow faulty.

      I think David Young's problem is that he confuses computational math with formulating a physical model and solving that. Get the physics right and one can make lots of progress.

      As one example, reformulate the denier Richard Lindzen's model of quasi-biennial oscillation (QBO) of equatorial stratospheric winds so that the correct boundary conditions are applied to Laplace's tidal equations, and the math becomes workable. The result is a Sturm-Liouville equation that has a surprisingly trivial analytical solution.

      As another example, ignore the warnings of the denier Anastasios Tsonis that the El Nino Southern Oscillation (ENSO) is chaotic and instead work out the Mathieu sloshing model of a thermocline. You will find that all the seemingly random oscillations are simply the result of biennial frequency folding with the known lunar tidal periods and the wobble in the earth's rotation acting as a boundary condition forcing.

      More in this ground-breaking article in the Journal of Fluid Mechanics, something that David Young should be familiar with but apparently is not

      J. Rajchenbach and D. Clamond, “Faraday waves: their dispersion relation, nature of bifurcation and wavenumber selection revisited,” Journal of Fluid Mechanics, vol. 777, p. R2, 2015.

      The issue with Young, Lindzen, Tsonis and the rest of the deniers is that they no longer have anything meaningful to add to the discussion.

    3. David Young has been trying to comment, but it hasn't been getting through the system. He send this response:
      "Be careful Nick in quoting Mavriplis especially a single sentence. Dmitri has a history of not reporting negative results. This paper really made me change my mind about him, but some of his other work is a marketing exercise for his code. The test cases for this paper are simple wing bodies. At the specified flow conditions, one is completely attached. The other has a very small side of body separation bubble that should simply make very little difference. The two different “solutions” he showed were 10% different in both lift and drag. Separation has nothing to do with this. What is almost certainly going on has to do with the trailing edge pressures. In any case, he also ran a lower Mach number so the flow is subsonic.!!

      Another paper to look at is Jameson’s AIAA 2015-2447 which I believe implements a method similar to what is in GCM’s, i.e., higher order methods with heavy filtering of “spurious” wiggles using grids that are really too coarse to resolve the detailed dynamics. They are too smart to claim the method gives credible results compared to more modern methods, but do suggest it as a stable way to get an initial guess for a finer grid. If you look at their results you will see the dynamics are very washed out.

      GCM’s do almost certainly do a better job on some things than others. Energy balance may be close to right, modulo all the tuning of feedbacks via sub grid models. However, the washed out dynamics does influence the energy balance and can lead to large energy losses due to false entropy. That’s why I advocate a clean sheet of paper GCM. It is just not credible to me anyway that GCM's are very skillful at things like regional climate which after all depends on the details of the dynamics."

    4. That’s why I advocate a clean sheet of paper GCM.
      What does this even mean? Different equations? Different integrators? Different results? I would expect that most GCMs are modular and so you can implement changes and modifications without needing to rewrite the whole code from scratch.

      I think there are some really interesting issues related to climate modelling (how complex, varying initial conditions, perturbed physics ensembles versus multi-model ensembles) but I don't really think that some kind of "start again" idea is really something many would regard as a suitable way forward.

    5. Ken, I suggest reading the Jameson paper. Higher order finite element methods are dramatically better than the heavy filtering old methods.

    6. David,
      I suggest you read my comment properly. It would be a first if you did, but - as they say - there's always a first time for everything.

    7. Ken Rice, There you go again.!! You have no comment on the papers I referenced?

    8. Young says:
      "You have no comment on the papers I referenced?"

      Why would anyone look at that when we are trying solve problems in geophysics?

      Young is sounding a lot like a Trump bully. He claims he's got the greatest stuff ever but it's all talk and no action. Like Trump, he wants to change the system completely yet has no clue on the implications.

    9. David,
      There you go again.!! You have no comment on the papers I referenced?
      That's probably because the papers you reference have virtually nothing to do with my comment. I have no doubt that modern higher order methods are better than old lower-order methods. What I fail to see is how that has anything to do with what I said. It's almost as if you automatically assume that everyone is too stupid to consider that we should use use newer improved methods, rather than older methods.

      Here's a thought. Why don't you try reading my comment, thinking about it, responding to what I actually said, rather than responding to what you think I said. It's worth a shot (although, I think I may have suggested this to you before and you appear to have little interest in actually doing so, so it's probably not worth my time pointing it out any more).

    10. Why don't you try to use the older methods such as Laplace's tidal equations? And then apply some geophysics approximations to actually solve them?

      Amazing what you can do if all you have is math and Newton's laws at your disposal. After all, that's all Laplace had access to.

      I feel like Bernie Sanders watching a catfight between Donald Trump and Hillary Clinton. Between an insane person and one that plays it safe by echoing the status quo. That's climate ball for you I suppose, or whatever those dudes at There's Physics are calling it.

    11. Why don't you try to use the older methods such as Laplace's tidal equations? And then apply some geophysics approximations to actually solve them?

      Amazing what you can do if all you have is math and Newton's laws at your disposal. After all, that's all Laplace had access to.

      I feel like Bernie Sanders watching a catfight between Donald Trump and Hillary Clinton. Between an insane person and one that plays it safe by echoing the status quo. That's climate ball for you I suppose, or whatever those dudes at There's Physics are calling it.

    12. I said earlier:
      "I feel like Bernie Sanders watching a catfight between Donald Trump and Hillary Clinton. Between an insane person and one that plays it safe by echoing the status quo."

      Lindzen is wrong. Tsonis is wrong. Consensus climate science made a big mistake in implicitly endorsing by peer review what these two dolts have to say about QBO and ENSO. When science normalizes crackpots like Lindzen and Tsonis in much the same way that the political press normalizes a likely insane despot, there are indeed consequences that we will eventually have to face.

      What we need to do is completely marginalize Lindzen and Tsonis and go with the simple and concise geophysics. What Lindzen and Tsonis have done is completely bogus, but unfortunately its not straightforward to unwind their models from the consensus, because that would expose the weakness of goodoldboy peer review.

      What's even worse, with the way things are looking, it's possible that all the funding for climate science will dry up and we all will have to resort to any means at our disposal to figure this stuff out.

      As for the other dolts ... at WUWT they are begging for money to go to AGU and of course their even more doltish readership is pitching in because they don't realize that they are being played like Trump chumps.

      I will be presenting my own ENSO and QBO models at AGU (and would never resort to pleading to offset the costs because I am not some kid collecting money for a junior high band trip)

      Like our politics, the status quo is pretty bad in climate science but the alternative is even worse. Us Bernie-bro's are committed to getting this done correctly. Either AGW is gonna hit us or Peak Oil will, and it make sense to understand climate science if for no other reason than to understand how we can extract renewable energy from the system.

      This evening sucks pretty bad but we will have to make do. At least I don't need climate science funding.

    13. Here's another clueless denier scientist who has been normalized

      "The second co-winner was Dr. Peter J. Webster (Georgia Institute of Technology) who won for his work on ocean-atmosphere interactions and their effect on monsoon strength, which is used to provide one to two-week lead time forecasts of monsoonal floods"

      The guy gets an award for a 2 week lead time on a prediction.

    14. Ken Rice, I reread your comment and still can't see anything of a technically relevant nature. It is often the case with software that it is easier to start over again especially if you want to use different numerical methods that require very different software structures. The problem here is that old codes develop tremendous complexity over time to make things run faster or stabilize calculations. If you look at any CFD code, you will see what I mean. These are not academic toy codes.

    15. Young said:
      "The problem here is that old codes develop tremendous complexity over time to make things run faster or stabilize calculations. ...
      These are not academic toy codes. "

      As an aside you can tell that Young is an old-timey software hack. Current day software engineers never use the archaic term "codes" for the software they write. That's a relic of a bygone Fortran era.

  3. What I have never understood about ENSO/mid-century cooling and the chaos brigade is how they flip from it's a nonlinear system, IPCC morons - and their condescension flows like Niagra Falls - to it ain't gonna warm for another 2 to 3 decades. They're roosting over at CE waiting for the AMO to flip negative, which is really really odd.

    1. We can likely tell exactly what will happen with ENSO many years in advance, with the caveat of not being able to predict when a odd-vs-even biennial phase inversion will occur

      Discussions at the Azimuth Project forum:

  4. This is interesting, but mostly a defense of CFDs which work on the principle of allowing engineers to tweak their designs to prevent them from straying into chaotic regimes of the dynamical system at work.

    GCMs do not end up presenting the modeler with "the attractor" at all -- look at the CESM-LE project and their paper running 30 runs of a 50 year projection of temperature trends for North American winters -- with less-than-rounding errors changes to s single initial condition. This study produced 30 different climates.

    See their press release here ->

    1. Maybe it does mean what you think it means. Or, would that be a first?

    2. JCH,
      Kip Hansen needs to stay in his playpen at Judy's or Tony's. That was a load of rubbish that he wrote on chaos.

      Tony: "Judy, Judy, Judy ..." LOL

    3. Kip,
      " defense of CFDs which work on the principle of allowing engineers to tweak their designs"
      The key word there is work. They do. And it's not by avoiding chaos. It's by seeking results that correspond to the attractor.

      I wonder if by tweaking ou are referring to Mavriplis saying that the design can and should be tweaked to avoid separation. That's a rather special case, and separation is avoided for reasons other than facilitating CFD. In many areas of engineering CFD usage you can't tweak design.

      You might like to consider the effect of Reynolds averaging. The equations then use that average as the unknown, which takes out a level of subgrid chaos. But there is plenty left.

      Your case of 30 different results corresponds to trajectories. The point the paper was making was that with all that variability, the average over runs still is reasonable. But it also points to the recognised issue that GCMs do worse with regional climate than global. This partly reflects that global is constrained by energy conservation, but regional isn't. You can have one place hot, one cold, nd still satisfy that. And there is nothing much to say whether it is one place or another.

    4. Nick rightly says:
      The key word there is work. They do.

      Yet the bigger question concerns why we can't easily map out the silly oscillations of ENSO like we can with tides. For heaven's sakes, ENSO is just a standing wave dipole of ocean sloshing and should have been figured out long ago, maybe not as long ago as the last Cubbies championship but certainly by now.

      What makes it that hard to figure out? The key is that it isn't necessarily a chaotic system but one that is defined more by metastability. Everything about ENSO points to an underlying periodicity governed by a period doubling of the annual cycle. Yet period doubling from 1 to 2 years implies that something has to set the bifurcation parity to either an odd-year cycle or an even-year cycle. Energetically, there is nothing that obviously determines the even vs odd cycle ... except perhaps how tidal gravitational forcing interact with the seasonal cycle. In fact, as I have shown on my blog there is a distinct biennial parity for each of the three classes of lunar tides (nodal, anomalistic, and tropical), which occurs after a nonlinear mixing with the seasonal cycle.

      The metastability revolves around how easily this balance is tipped. From what I have been able to discern this metastability has only flipped once, and that was during the 1980 to 1996 time interval. This can explain why standard signal processing techniques have not uncovered the metastability but the one described recently by Astudillo has detected the disturbance at 1980 as well:

      H. Astudillo, R. Abarca-del-Río, and F. Borotto, “Long-term potential nonlinear predictability of El Niño–La Niña events,” Climate Dynamics, pp. 1–11, 2016.

      Cubs finally win, and perhaps we are nearing an understanding of ENSO

    5. If you look at who is using Clara Deser's model results, there are a large number of papers about multi-decadal variability and regional climate. Some have hit the news: a prediction of a Mediterranean desert; speculation that the IPO may have recently flipped, etc. Blank-paper restart versus the march of science: incremental improvements in the understanding of nature. This versus Tsonis and Curry and Wyatt and their converts sitting in their pews praying for the AMO to flip negative and restore the pause and a final victory over the pinko greens.

  5. The issue with Tsonis is that he just assumes ENSO is a chaotically driven process without acknowledging the possible geophysics at work.

    I come from a laboratory characterization background and have long ago learned that any behavior that you can measure can tell you something about the driving forces at play. The ocean's thermocline is actually an acutely sensitive amplifier of slight changes in the angular momentum in the Earth's rotation. Change it ever so slightly and it will start the thermocline interface sloshing, just like a lava lamp wave machine in action.

    So if you take the ENSO data and wave transform it to extract the periodic driving forces, they rather precisely match up to the known angular momentum variations tied to the lunar cycles and the Chandler wobble.

    This is not so much a model fitting exercise but a validation of the physical effect that must occur. If it didn't happen, one would have to explain why nature conveniently decided to prevent the effect from happening.

    Tsonis goes on about all these nebulous hand-wavy explanations for ENSO invoking chaos because he never laid the fundamental geophysics groundwork. Tsonis, Curry and their converts are essentially building castles made of sand that won't even remain as footnotes in history when ENSO is ultimately characterized. Uncertainty? What's that?

    For some reason, I am reminded here of the story of King Canute, which is also often misrepresented. Like King Canute, but unlike Tsonis, we need to realize that the forces at work are inexorable and not borne of some whim.