The least squares method is one of the most effective approaches for estimating an impulse response of discrete-time systems. However, when colored smooth input signals are utilized for parameter identification, the least squares estimates tend to fluctuate seriously and converge very slowly to their true values. In these circumstances, the corresponding normal equation becomes ill-conditioned and the mean square error (MSE) of estimates increases under the influence of small eigenvalues of the input autocovariance matrix.
In this paper, we consider an identification method in which a small positive number (we call it a weighting coefficient) is added to diagonal elements of the input autocovariance matrix so as to mitigate the ill-conditioning. Effects and the optimal choice of the weighting coefficient are analyzed by use of the eigenvalue decomposition. It is also clarified that the weighting coefficient can be determined so as to minimize the MSE of the estimates. Next, we extend the identification method to a recursive form and apply the idea of the proposed method to optimal choice of an initial value of the covariance matrix in the recursive least squares method. It is shown that, in a case of white input signals, the optimal initial value is given analytically in terms of variance of input signal, output SNR and an order of the impulse response.