8 A Tutorial on Data-Driven Methods in Nonlinear Dynamics 53 analytical solutions. To be more precise, nonlinear equations of motion almost never have exact closed-form solutions. Suppose one were to take the linear equation in Eq.(8.1) and add the ‘simplest’ nonlinear term (which turns out to be cubic in y), the result is Duffing’s equation [16]: .m¨y +c˙y +ky +k3y 3 =x(t) (8.2) To this day, there are no exact solutions of this equation, except in the very restricted undamped (.c =0) and unforced (.x(t)) case. For the restricted case, Duffing himself provided an exact solution involving elliptic functions in 1918; however, there has been no progress to speak of on the general case since then. Unfortunately, this general lack of exact solutions means that all of the fascinating nonlinear phenomena discovered in the twentieth century—like bifurcations and chaos— present intractable problems for exact analytical methods. One way around this issue has been to rely on simulations; one numerically simulates the system responses of interest and characterises the responses. Initially these simulations were carried out using analogue computers incorporating bespoke nonlinear circuits, but with the advent of (and explosion in) digital computers, it became possible to solve the nonlinear initial value problems in order to simulate samples of response data for further computational analysis. For example, if one wishes to analyse the stability or chaotic nature of a system, the Lyapunov exponents can be estimated from a time series [17] (in fact, the algorithm in that paper was designed for use with experimental data, but the principle is the same). Discrete wavelet analysis has also proved powerful in the analysis of chaotic time series [18]; apart from estimating certain chaotic invariants, the analysis can also detect coherent structures. Finally, on the subject of coherent structures, the interesting book [19] details how coherent structures and low-dimensional structure can be found in data from turbulent flows; one of the main algorithms discussed is the Karhunen-Loéve expansion or principal orthogonal decomposition (POD). Perhaps a little ironically, the POD is basically a variant of principal component analysis (PCA), which is a linea r projection method often used in applied machine learning for data visualisation and dimension reduction. Turbulence is of course one of the major outstanding problems in nonlinear science generally. Even within the restricted scope of NLSI, this paper will omit one very important class of models—the ‘black-box’ models mentioned earlier. In principle, such models can work without any physical insight whatsoever; they work by proposing some basis of functions which spans the function space of interest, much as a Fourier expansion does for the space of periodic functions on some finite interval. The important point of such models is not that they encode some prior physical knowledge, but rather that they have a universal approximation property, which is to say that they can represent some target function with arbitrary accuracy (as long as they include enough terms). Such ‘learners’ include neural networks (deep or shallow), radialbasis function (RBF) networks, Gaussian processes, etc. [2]. In principle, one could download software for such a learner, input some data and run the software with default settings, and out would pop a model. Such a model would be ‘pitch black’.4 The authors would always caution against this practice; it seldom produces good research and could produce catastrophic results when applied in real life. Black boxes will surface later in the discussion of grey boxes and physics-informed machine learnin g (PIML). The layout of the paper is as follows. The next section presents the arguments for adopting a Bayesian approach to NLSI and this is followed by a discussion of a specific parameter estimation method based around Markov chain Monte Carlo (MCMC). Section 8.4 then presents a case study of MCMC applied to the identification of a Duffing oscillator system. Section 8.5 discusses the combined problem of parameter estimation and model selection and is followed by another case study involving equation discovery for hysteretic oscillators. Section 8.7 then presents some ideas on combining physicsbased and data-based approaches in the form or grey-box modelling, or more generally physics-informed machine learning. The paper ends with brief conclusions. 8.2 Bayesian Inference and System Identification This section will give a precise definition of the system identification problem and will outline the advantages of taking a Bayesian probabilistic viewpoint. The input (stimulus) variable will be denoted .x(t) and the output (response) variable by .y(t). Suppose that one has a set of data .D = {(xi,yi), i = 1, . . . ,N} of sampled system inputs . xi and outputs . yi. If one assumes that there is no measurement noise, and the model structure is known, then the application of an identification algorithm will yield (assuming that the problem is well-conditioned) a deterministic estimate of the system parameters . w: .w=id(D) (8.3) 4 The authors thank Johan Schoukens for introducing them to this term.
RkJQdWJsaXNoZXIy MTMzNzEzMQ==