Elsevier

Automatica

Volume 60, October 2015, Pages 65-78
Automatica

Non-asymptotic model quality assessment of transfer functions at multiple frequency points

https://doi.org/10.1016/j.automatica.2015.06.040Get rights and content

Abstract

In this paper we develop methods for evaluating uncertainties in the frequency response of a dynamical system based on finitely many input–output data points. We extend the “Leave-out Sign-dominant Correlation Regions” (LSCR) algorithm to deliver confidence regions with a guaranteed probability for the frequency response at multiple frequencies, and we introduce a computationally efficient scheme that enables the confidence regions to be constructed frequency by frequency. Simulation examples illustrating the usefulness of the developed algorithm are provided.

Introduction

In system identification, providing a description of the uncertainties associated with the nominal system model is as important as obtaining the nominal model itself, especially for the synthesis of robust controllers. A popular technique for evaluating the model quality is based on constructing confidence regions using asymptotic system identification theory. This is a mature approach and the confidence regions can be computed relatively easily (see  Ljung, 1999). However, in some cases using asymptotic theory may lead to unreliable results (see  Garatti, Campi, & Bittanti, 2004) when applied to a finite number of data points.

In this paper, we consider a method for constructing confidence regions based on finitely many data points as, e.g., considered in Bayard (1993), Campi and Weyer (2005), den Dekker, Bombois, and Van den Hof (2008), Goodwin, Gevers, and Ninnes (1992) and  Hjalmarsson and Ninness (2006). Unlike methods based on asymptotic theory, the developed method generates guaranteed confidence regions for a finite number of data points. The developed approach is based on the LSCR method introduced in  Campi and Weyer (2005) (see also  Campi, Ko, & Weyer, 2009 and  Campi & Weyer, 2010), and it is extended to produce guaranteed confidence regions for the frequency response of a dynamical system. As a finite number of data points does not provide any information about the tail of the impulse response, prior information, such as exponentially decaying bounds, is introduced and incorporated in the algorithm in order to deal with tail effects. Moreover, an experimental scheme is derived that allows the confidence regions to be constructed separately frequency by frequency. This reduces the computational burden significantly.

In the next subsection we give simple preview examples that illustrate the main ideas of the proposed approach. Then, in Section  2, the procedure used in the preview examples is generalized to construct simultaneous confidence regions when the system is excited by a multi-sine input signal. In Section  3 an experimental scheme and an algorithm that allow the confidence regions to be constructed at low computational costs are introduced. Two simulation examples demonstrating the usefulness of the proposed approach are given in Section  4.

In this section we first introduce a simple example illustrating the main ideas of LSCR by generating a confidence interval for the amplitude of a sinusoid, before moving on to the construction of a confidence set for the frequency response of a dynamical system at a given frequency. For further descriptions of the main ideas in the LSCR algorithm, the reader is referred to  Campi and Weyer (2006) and Section 1.2 of  Campi et al. (2009).

The signal of interest is a sinusoid observed in noise yt=A0cosωt+nt. We have observations yt, t=1,,N=60. nt is a sequence of zero mean independent random variables, symmetrically distributed about zero. The frequency ω=0.2 is known, but the amplitude A0 is unknown. The observed signal is shown in Fig. 1. We wish to construct a confidence interval for A0. Given the signal model yˆt(A)=Acosωt, we compute the observation error εt(A)=ytyˆt(A)=(A0A)cosωt+nt, and correlate it with cosωt, which gives ft(A)=εt(A)cosωt=(A0A)cos2ωt+ntcosωt. We note that E{t=1Nft(A)}=0 for A=A0, and is different from zero for AA0. The idea is now to use random subsamples of ft(A) to form empirical estimates of the correlation between the observation error and cosωt. To this end we compute M=20 empirical subsample estimates gi(A)=t=1Nhi,tft(A)=t=1Nhi,tεt(A)cosωt,i=0,1,,M1, where hi,t are independent and identically distributed (i.i.d.) random variables taking on the values 0 and 1 with probability 1/2 each. The exception is h0,t which is equal to zero for all t, and hence g0(A)0. This means that hi,t determines whether sample t is used when gi(A) is computed.

The M1 non-zero gi(A) functions are shown in Fig. 2. Corresponding to the true amplitude A0, gi(A0) is a sum of zero mean random variables. It is therefore unlikely that nearly all of the gi(A) functions are positive or negative for A=A0, and hence we exclude those values of A where all the gi(A) functions take on positive or negative values. Thus, the confidence interval marked with a thick line in Fig. 2 is obtained by keeping those values of A for at which at least q=1 of the gi(A) functions are positive and at least q=1 are negative. It is shown in Theorem 1 that the constructed confidence interval contains the true amplitude (A0=1) with probability 12q/M=0.9.

Next we move onto a more realistic situation where also the phase is unknown and transient effects need to be taken into account.

Suppose that the true continuous-time system is given by y(t)=0g0(τ)u(tτ)dτ+v(t), where g0(τ) is the impulse response function, and v(t) is additive noise. The transfer function G0(s) of the system (1) is the Laplace transform of g0(τ) given by G0(s)=0g0(t)estdt and in this example it is given by G0(s)=2.5s+2.5. This information about the true system is given for completeness of description but is unknown to the user.

The input to the system is a sinusoid u(t)={cos(t),t00,t<0. The output is given by y(t)=0tg0(τ)cos(tτ)dτ+v(t)=Re{0g0(τ)ejτdτejttg0(τ)ejτdτejt}+v(t)=Re{G0(j)ejttg0(τ)ejτdτejt}+v(t)=a0costb0sint+ȳ(t)+v(t), where a0Re{G0(j1)}, b0Im{G0(j1)} and ȳ(t)Re{tg0(τ)ejτdτejt} represents the transient effects due to initial conditions.

Our task is to construct a confidence region for a0 and b0, the frequency response parameters at 1 rad/s based on a finite number of data points obtained by sampling the input and output at time instants t=kT for k=1,2,,365 with T=0.02s. We assume that the sampled noise v(kT), k=1,2,,365, is a sequence of independent (but not necessarily identically distributed) random variables with symmetric distributions around zero and all v(kT) admit densities.

One way to estimate the frequency response parameters and to construct a confidence region is to measure the output once the transients have died out. In order to avoid the transient phase of the response we wait =50 samples before starting the measurements of the output y(kT) (see Fig. 3). We take 315 samples of the input and output. This corresponds to approximately one cycle of the input signal.

From finite-length input and output data we cannot obtain full information about the frequency response of the system since the data do not carry any information about the tail of the impulse response. The only way we can bound the uncertainty due to the tail is via a priori knowledge and assumptions. Here we assume that a bound on the impulse response is available, i.e., parameters Mg and ρ are known such that |g0(τ)|Mgeρτ,for some  0<Mg<andρ>0. For this example, we use the following prior information Mg=3,ρ=1.7, which is shown in Fig. 4. Using this information, we can bound the unknown value ȳ(t) as follows |ȳ(t)|t|g0(τ)|dτtMgeρτdτ=Mgeρtργ(t). We can compute the predictions of the output and the prediction error for k=51,,365 as follows yˆk(θ)=acos(kT)bsin(kT),εk(θ)=y(kT)yˆk(θ)=ãcos(kT)b̃sin(kT)+ȳ(kT)+v(kT), where ãa0a, b̃b0b and θ=[a,b]T denotes the parameter vector. Using random subsamples of the data set we calculate the following M=400 scaled empirical correlation functions Cia(θ) and Cib(θ) between the prediction error and sines and cosines of the same frequency as the input signal Cia(θ)=k=51365hi,kεk(θ)cos(kT)=k=51365hi,k[ãcos2(kT)b̃sin(kT)cos(kT)+ȳ(kT)cos(kT)+v(kT)cos(kT)],Cib(θ)=k=51365hi,kεk(θ)sin(kT)=k=51365hi,k[ãcos(kT)sin(kT)b̃sin2(kT)+ȳ(kT)sin(kT)+v(kT)sin(kT)], where hi,k for i=1,,399 and k=51,,365 are i.i.d. with distribution hi,k={0,with probability0.51,with probability0.5, and are independent of the noise sequence v(kT). The first string is given by h0,k=0 for k=51,,365. Note that, using (6), we have |k=51365hi,kȳ(kT)cos(kT)|k=51365hi,kγ(kT)|cos(kT)|Γia,|k=51365hi,kȳ(kT)sin(kT)|k=51365hi,kγ(kT)|sin(kT)|Γib, and, hence, at the true parameter, θ=θ0, for all i we obtain Cia(θ0)Γiak=51365hi,kv(kT)cos(kT)Cia(θ0)+Γia,Cib(θ0)Γibk=51365hi,kv(kT)sin(kT)Cib(θ0)+Γib. Since v(kT) is zero-mean, it is unlikely that nearly all of the sums k=51365hi,kv(kT)cos(kT), i=1,,399=M1, take on positive values or that nearly all of them take on negative values, hence, it is unlikely that nearly all Cia(θ0)+Γia take on negative values or that nearly all Cia(θ0)Γia take on positive values, and the same holds for Cib(θ0)+Γib and Cib(θ0)Γib. Based on this observation, in order to construct a confidence region we discard those regions in the parameter space where Cia(θ)+Γia or Cib(θ)+Γib take on negative value too many times and also the regions where Cia(θ)Γia or Cib(θ)Γib take on positive value too many times, and hence the name of the algorithm “Leave-out Sign-dominant Correlation Regions”.

Therefore, in order to find a confidence set, we excluded the regions in the parameter space where less than q=10 empirical correlation function satisfies either Cia(θ)Γia<0, Cib(θ)Γib<0, Cia(θ)+Γia>0 or Cib(θ)+Γib>0. The obtained confidence set is shown as the blank area in Fig. 5, and according to Theorem 2 it contains the true parameters with probability at least 122qM=0.9. In the figure, the region where at most 9 of the Cia(θ)Γia functions were negative is marked with , and the region where at most 9 of the Cia(θ)+Γia were positive is marked with . Likewise, × and + represents the regions where at most 9 values of Cib(θ)Γib and Cib(θ)+Γib were negative and positive, respectively. As we can see, each correlation excludes a particular region of the parameter space. Note that in order to construct the confidence region we have not made any assumptions on the noise other than it should be symmetrically distributed around 0. Still the algorithm constructs a confidence region with guaranteed probability with a finite number of data points.

Section snippets

Main algorithm

Here we extend the approach in the preview example to a multi sine input signal.

Computational aspects

Using the procedure in the previous section, we can construct non-asymptotic confidence regions for the frequency response at multiple frequencies. However, each of the empirical correlation functions (20) depends on the whole set of parameters a1,b1,,aL,bL, and thus the resulting confidence regions Θra and Θrb are not only dependent on ar and br, but also on all other parameters.

In this section we develop an experiment procedure and a method for the generation of decoupling binary strings

Simulation example

In this section, we present two simulation examples to illustrate the procedures for constructing confidence regions developed in the previous sections. We consider the same first-order system as described by (1), (3) in the preview example in Section  1.1 and construct simultaneous confidence regions for a two-frequency and a ten-frequency case.

Conclusion

In this paper, we have extended the LSCR algorithm introduced in Campi and Weyer (2005) to the problem of constructing confidence regions for the frequency response at multiple frequencies using a finite number of input–output data points. No information about the tail of the impulse response can be obtained from a finite number of data points, and hence a priori information has been used to bound the effects of the tail. Three theorems have been established describing the probabilistic

Acknowledgments

The research of S. Ko and E. Weyer was partly supported by the Australian Research Council under the Discovery Grant Scheme, Project DOP0558579 and the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIP) (NRF-2013M1A3A3A02042434). The research of M.C. Campi was supported by MIUR under the project “Identification and Adaptive Control for Industrial Systems”.

Sangho Ko is an associate professor at the School of Aerospace and Mechanical Engineering, Korea Aerospace University, South Korea. He received his Ph.D. degree from the Department of Mechanical and Aerospace Engineering of the University of California, San Diego (UCSD) in 2005. From 2005 to 2006, he was a post-doctoral scholar in UCSD to research system identification techniques. From March 2006 to February 2008 he was a research fellow in the Department of Electrical and Electronic

References (17)

There are more references available in the full text version of this article.

Cited by (3)

Sangho Ko is an associate professor at the School of Aerospace and Mechanical Engineering, Korea Aerospace University, South Korea. He received his Ph.D. degree from the Department of Mechanical and Aerospace Engineering of the University of California, San Diego (UCSD) in 2005. From 2005 to 2006, he was a post-doctoral scholar in UCSD to research system identification techniques. From March 2006 to February 2008 he was a research fellow in the Department of Electrical and Electronic Engineering of the University of Melbourne, Australia, where he worked on finite sample quality assessment of system identification. From 1992 to 1999, he was with Samsung Aerospace Industries, Ltd., Kyungnam, Korea, where he was a research and development engineer for digital flight control systems. His research interests include: state estimation and control, system identification, flight control, propulsion system control.

Erik Weyer is a professor in the Department of Electrical and Electronic Engineering at the University of Melbourne. He received the Siv. Ing. degree in 1988 and the Ph.D. in 1993, both from the Norwegian Institute of Technology, Trondheim, Norway. From 1994 to 1996 he was a Research Fellow at the University of Queensland, and since 1997 he has been with the Department of Electrical and Electronic Engineering at the University of Melbourne. He has held visiting positions at the University of Brescia, Italy, the Technical University of Vienna, Austria, and Politecnico di Milano, Italy. From 2010 to 2012 he was an associate editor of IEEE Transactions of Automatic Control, and he is currently an associate editor of Automatica. His research interests are in the areas of system identification and control, with particular emphasis on finite sample properties of system identification methods, and modeling and control of irrigation channels and rivers. He was a co-recipient of the IEEE CSS Control System Technology Award in 2014.

Marco Claudio Campi is Professor of Automatic Control at the University of Brescia, Italy.

In 1988, he received the Doctor degree in electronic engineering from the Politecnico di Milano, Milano, Italy. From 1988 to 1989, he was a Lecturer at the Department of Electrical Engineering of the Politecnico di Milano. From 1989 to 1992, he was a Research Fellow at the Centro di Teoria dei Sistemi of the National Research Council (CNR) in Milano and, in 1992, he joined the University of Brescia, Brescia, Italy. He has held visiting and teaching appointments at the Australian National University, Canberra, Australia; the University of Illinois at Urbana-Champaign, USA; the Centre for Artificial Intelligence and Robotics, Bangalore, India; the University of Melbourne, Australia; the Kyoto University, Japan.

Marco Campi is the chair of the Technical Committee IFAC on Modeling, Identification and Signal Processing (MISP). He has been in various capacities on the Editorial Board of Automatica, Systems and Control Letters and the European Journal of Control. Marco Campi is a recipient of the “Giorgio Quazza” prize, and, in 2008, he received the IEEE CSS George S. Axelby outstanding paper award for the article The Scenario Approach to Robust Control Design. He has delivered plenary and semi-plenary addresses at major conferences including SYSID, MTNS, and CDC, and has been a distinguished lecturer of the Control Systems Society. Marco Campi is a Fellow of IEEE, a member of IFAC, and a member of SIDRA.

The research interests of Marco Campi include: system identification, stochastic systems, randomized methods, adaptive and data-based control, robust optimization, and learning theory.

The material in this paper was partially presented at the 2007 International Conference on Control, Automation and Systems, October 17–20, 2007, Seoul, Korea and at the 17th IFAC World Congress, July 6–11, 2008, Seoul, Korea. This paper was recommended for publication in revised form by Associate Editor Alessandro Chiuso under the direction of Editor Torsten Söderström.

1

Tel.: +82 2 300 0119; fax: +82 2 3158 2191.

View full text