Practical realization of discrete-time Volterra series for high-order nonlinearities

Abstract

Full realization of all versions of Volterra series like pure, truncated, and doubly finite Volterra series, and especially, the realization of their high orders is an intractable problem. Hence, practical implementation of Volterra series for high-order nonlinearities is not feasible with reasonable computational cost. For this reason, mathematicians, neuroscientists, and especially, biomedical and electrical engineers are forced to use only the low-order Volterra series. In this paper, we provide a full realization of off-repetitive discrete-time Volterra series (ORDVS) by departure from a traditional approach in favor of choosing a hierarchical structure. The proposed method is named fast full tantamount of off-repetitive discrete-time Volterra series (FFT-ORDVS). We have proven that the proposed off-repetitive discrete-time Volterra series approximates the basic discrete-time Volterra series very well and with much less computational complexity. In a conventional method, if ${M} +1$ is considered as the memory length of the ORDVS, around $2^{M}$ math operations are needed for the full realization of it. In most cases, M is a large number and consequently, $2^{M}$ is too large. To solve this problem, we have proposed a simple polynomial time solution and using the proposed method, the same task is done only by 6M math operations. It means that we have found a shortcut to change an intractable problem (${O}(2^{M}))$ to a simple P problem (O(M)). This achievement enables researchers to use high-order kernels and consequently covers high-order nonlinearities with the lowest possible computational load. We have proven our claims mathematically and validated the performance of the proposed method using two numerical examples and a real problem.

Appendices

Appendices

Appendix A

By mathematical simplification of the rth-order Volterra functional, it is straightforward to show that there is no noticeable difference in the existence or absence of the repetitive terms in the realization of Volterra series. Therefore, in order to estimate the rth-order Volterra functional in Eqs. (1) and (2), $N_{{\mathrm{BDVS}}} \left( r \right)$ and $N_{{\mathrm{ORDVS}}} \left( r \right)$ parameters should be estimated for the basic discrete-time Volterra series (Volterra series with repetitive and non-repetitive components) and for the off-repetitive discrete-time Volterra series (Volterra series with non-repetitive components), respectively. Consequently, we reach Eqs. (A.1) and (A.2) as follows.

$$\begin{aligned} N_{{\mathrm{BDVS}}} (r)= & {} \frac{\left( {M+r} \right) !}{r!M!} \end{aligned}$$

(A.1)

$$\begin{aligned} N_{{\mathrm{ORDVS}}} (r)= & {} \frac{\left( {M+1} \right) !}{r!\left( {M+1-r} \right) !} \end{aligned}$$

(A.2)

where $M+1$ is the memory length.

We define the ratio of the number of the non-repetitive components to the number of all components (repetitive and non-repetitive) as a criterion for comparison between exploited information in the basic Volterra series (BDVS) and the ORDVS.

$$\begin{aligned} \% \psi (r)=\left( {\frac{N_{{\mathrm{ORDVS}}} (r)}{N_{{\mathrm{BDVS}}} (r)}} \right) 100 \end{aligned}$$

(A.3)

By substituting Eqs. (A.1) and (A.2) into Eq. (A.3) and after some mathematical simplification, we obtain the following equation.

$$\begin{aligned} \% \psi (r)=\left( {\frac{\Gamma \left( {M+1} \right) \Gamma \left( {\frac{M+2-r}{1+r}} \right) }{\Gamma \left( {\frac{M+r}{1+r}} \right) \Gamma \left( {M+2-r} \right) }} \right) 100 \end{aligned}$$

(A.4)

where $\Gamma$ is the Gamma function. In an ideal state, if this ratio equals to 100, we can conclude that all exploited information in the basic discrete-time Volterra series (BDVS) have been used exactly in the off-repetitive discrete-time Volterra series (ORDVS). Since the value of M is much greater than the value of r ($M>>r)$, it is obvious the value of $\% \psi \left( r \right)$ tends to reach 100. For example, if the input signal discretized by a 10 MS/s A/D and using just 5 ms memory length, the results that are shown in Table 4 are obtained. Results shown in Table 4 indicate that almost all information of the BDVS (Volterra series with repetitive and non-repetitive components) have appeared in different orders of the ORDVS (Volterra series with non-repetitive components). This means that there is no noticeable difference in the existence or absence of the repetitive terms in Volterra series.

Appendix B

Equation (48) was calculated using Stirling’s approximation and Laplace’s method by following procedures.

We know

$$\begin{aligned} \max (N\,(r)) = N\,(r) \quad \forall \,r=\left( \frac{M}{2}+1\right) . \end{aligned}$$

(B.1)

Then by substituting $r=\left( \frac{M}{2}+1\right)$ into Eq. (43) we have

$$\begin{aligned} \max (N\,(r)) = N\,\left( \frac{M}{2}+1\right) = \frac{(M+1)!}{\left( \frac{M}{2}\right) !\left( \frac{M}{2}\right) !}. \end{aligned}$$

(B.2)

An appropriate alternative for $\theta !$ is Gamma function.

$$\begin{aligned} \theta != & {} \Gamma (\theta +1) \end{aligned}$$

(B.3)

$$\begin{aligned} \theta != & {} \int \nolimits ^{\infty }_{0} x^{\theta }\hbox {e}^{-x} \hbox {d}x \Rightarrow \theta ! = \int \nolimits ^{\infty }_{0} \hbox {e}^{\theta \ln x-x} \hbox {d}x \end{aligned}$$

(B.4)

By putting $x-\theta y$, we have:

$$\begin{aligned} \theta != & {} \hbox {e}^{\theta \ln \theta } \theta \int \nolimits ^{\infty }_{0} \hbox {e}^{\theta (\ln y-y)} \hbox {d}y \,\underrightarrow{\psi (y)=\ln \,y-y}\,\, \theta !\nonumber \\= & {} \hbox {e}^{\theta \ln \,\theta } \theta \int \nolimits ^{\infty }_{0} \hbox {e}^{\theta \psi (y)} \hbox {d}y. \end{aligned}$$

(B.5)

The function $\psi (y)$ has a unique global maximum at $y_{0}$ and by applying Laplace’s method, we can solve the integral term in (B.5) as follows:

$$\begin{aligned} \int \nolimits ^{\infty }_{0} \hbox {e}^{\theta \psi (y)} dy\approx \sqrt{\frac{2\pi }{\theta |\psi ''(y_{0})|}}\hbox {e}^{\theta \psi (y_{0})}. \end{aligned}$$

(B.6)

It is obvious that:

$$\begin{aligned} \psi (y)= & {} \ln \,y-y\Rightarrow \psi '(y) =\frac{1}{y}-1\Rightarrow y_{0}=1\nonumber \\ \psi ''(y)= & {} -\frac{1}{y^{2}}\Rightarrow |\psi ''(y_{0})|=1. \end{aligned}$$

(B.7)

And then

$$\begin{aligned} \int \nolimits ^{\infty }_{0} \hbox {e}^{\theta \psi (y)}dy \approx \sqrt{\frac{2\pi }{\theta }}\hbox {e}^{\theta }. \end{aligned}$$

(B.8)

By substituting (B.8) into (B.5) and after simplification, we get (B.9).

$$\begin{aligned} \theta ! \approx \hbox {e}^{\theta (\ln \,\theta -1)}\sqrt{2\pi \theta } \end{aligned}$$

(B.9)

Now called the Stirling’s series

$$\begin{aligned} \theta !\approx & {} \sqrt{2\pi \theta }\left( \frac{\theta }{e}\right) ^{\theta } \underbrace{\left( 1+\frac{1}{12\theta }+\frac{1}{288\theta ^{2}}+\cdots \right) }_{\approx 1} \rightarrow \theta !\nonumber \\\approx & {} \sqrt{2\pi \theta }\left( \frac{\theta }{e}\right) ^{\theta }. \end{aligned}$$

(B.10)

By substituting (B.10) into (B.2), we obtain

$$\begin{aligned}&\max (N(r))\nonumber \\&\quad = \frac{\sqrt{2\pi (M+1)}\left( \frac{M+1}{e}\right) ^{M+1}}{\sqrt{2\pi \left( \frac{M}{2}\right) }\left( \frac{\left( \frac{M}{2}\right) }{e}\right) ^{\frac{M}{2}}\,\,\sqrt{2\pi \left( \frac{M}{2}\right) }\left( \frac{\left( \frac{M}{2}\right) }{e}\right) ^{\frac{M}{2}}}.\nonumber \\ \end{aligned}$$

(B.11)

With simplification and a little approximation, we have

$$\begin{aligned} \max (N(r)) \approx \frac{1}{\sqrt{\pi M}}2^{M+\frac{1}{2}}. \end{aligned}$$

(B.12)