Multi-factor authentication model based on multipurpose speech watermarking and online speaker recognition

Abstract

In this paper, a Multi-Factor Authentication (MFA) method is developed by a combination of Personal Identification Number (PIN), One Time Password (OTP), and speaker biometric through the speech watermarks. For this reason, a multipurpose digital speech watermarking applied to embed semi-fragile and robust watermarks simultaneously in the speech signal, respectively to provide tamper detection and proof of ownership. Similarly, the blind semi-fragile speech watermarking technique, Discrete Wavelet Packet Transform (DWPT) and Quantization Index Modulation (QIM) are used to embed the watermark in an angle of the wavelet’s sub-bands where more speaker specific information is available. For copyright protection of the speech, a blind and robust speech watermarking are used by applying DWPT and multiplication. Where less speaker specific information is available the robust watermark is embedded through manipulating the amplitude of the wavelet’s sub-bands. Experimental results on TIMIT, MIT, and MOBIO demonstrate that there is a trade-off among recognition performance of speaker recognition systems, robustness, and capacity which are presented by various triangles. Furthermore, threat model and attack analysis are used to evaluate the feasibility of the developed MFA model. Accordingly, the developed MFA model is able to enhance the security of the systems against spoofing and communication attacks while improving the recognition performance via solving problems and overcoming limitations.

Appendix A

Discrete Fourier Transform (DFT) is assumed as Weibull distribution. However, the distribution of the DWPT sub-bands is assumed as a Generalized Gaussian Distribution (GGD) [2]. GGD can be defined as in Eq. (14), if μ ² _s  = 0 and σ ² _s are assumed.

$$ {f}_s\left(s;\mu, {\sigma}_s,v\right)=\frac{1}{2\varGamma \left(1+\frac{1}{v}\right)A\left({\sigma}_sv\right)} exp\left\{-{\left|\frac{s-\mu }{A\left({\sigma}_sv\right)}\right|}^v\right\} $$

(14)

where Γ(.) corresponds to Gamma function which is expressed by $\varGamma (x)={\displaystyle {\int}_0^{\infty }{t}^{x-1}{e}^{-t}dt\cong}\sqrt{2\pi }{x}^{x-\frac{1}{2}}{e}^{-x},v$ corresponds to the shape of the distribution which can be estimated by statistical moment of the signal.

If the watermarked speech signal is passing through AWGN channel, it is possible to formulate the watermarked speech signal at receiver based on Eqs. (15) and (16).

$$ {r}_i=\alpha \times {s}_i+{n}_i\ if\ {m}_i=1 $$

(15)

$$ {r}_i=\frac{1}{\alpha}\times {s}_i+{n}_i\ if\ {m}_i=0 $$

(16)

where n _i corresponds to the amount of noise which is contaminated the watermarked speech signal. To estimate the probability of the watermark bits when it is 1, Eq. (17) is expressed:

$$ \left.R\right|1=\frac{{\displaystyle {\sum}_A}{\left(\alpha \times {s}_i+{n}_i\right)}^4}{{\displaystyle {\sum}_B}{\left({s}_i+{n}_i\right)}^4}\Rightarrow \left.R\right|1=\frac{\alpha^4{\displaystyle {\sum}_A}{s}_i^4+4{\alpha}^3{\displaystyle {\sum}_A}{s}_i^3{n}_i+6{\alpha}^2{\displaystyle {\sum}_A}{s}_i^2{n}_i^2+4\alpha {\displaystyle {\sum}_A}{s}_i{n}_i^3+{\displaystyle {\sum}_A}{n}_i^4}{{\displaystyle {\sum}_B}{s}_i^4+4{\displaystyle {\sum}_B}{s}_i^3{n}_i+6{\displaystyle {\sum}_B}{s}_i^2{n}_i^2+4{\displaystyle {\sum}_B}{s}_i{n}_i^3+{\displaystyle {\sum}_B}{n}_i^4} $$

(17)

As seen, the summation of different parameters in Eq. (17) are affected the amount of the detection threshold. By considering Central Limit Theorem (CLT), there is possible to compute different series in nominator and denominator based on Normal distribution. Due to large value for μ and long length of the speech frames, the Normal distribution is often generated positive numbers which can modeled parameters like ∑ _A n ⁴ _i which is always positive. Equations (18) and (19) are computed the mean and variance respectively.

$$ E\left\{\sum {s}_i^4\right\}=\sum E\left\{{s}_i^4\right\}=M{\mu}_4 $$

(18)

$$ \begin{array}{l}var\left({\displaystyle \sum {s}_i^4}\right)=E{\left\{\left({\displaystyle \sum \left({s}_i^4-M{\mu}_4\right)}\right)\right\}}^2=E{\left\{\left({\displaystyle \sum \left({s}_i^4-{\mu}_4\right)}\right)\right\}}^2=\hfill \\ {}{\displaystyle \sum E{\left\{\left(\left({s}_i^4-{\mu}_4\right)\right)\right\}}^2}={\displaystyle \sum \left(E\left\{{s}_i^8-{\mu}_4^2\right\}\right)}=M{\mu}_8-M{\mu}_4^2\hfill \end{array} $$

(19)

where M corresponds to the length of each set of A and B. By applying the moment of GGD for r = 4 and r = 8, Eqs. (20) and (21) are estimated.

$$ {\mu}_4=\frac{\sigma_s^4\ \Gamma \left(\frac{1}{v}\right)\ \Gamma \left(\frac{5}{v}\right)\ }{\Gamma^2\left(\frac{3}{v}\right)} $$

(20)

$$ {\mu}_8=\frac{\sigma_s^8\ {\Gamma}^3\left(\frac{1}{v}\right)\ \Gamma \left(\frac{9}{v}\right)\ }{\Gamma^4\left(\frac{3}{v}\right)} $$

(21)

By considering Eqs. (18) and (19), Eq. (22) is formulated.

$$ \sum {s}_i^4\sim \mathcal{N}\left(M{\mu}_4,M{\mu}_8-M{\mu}_4^2\right) $$

(22)

If the mean of the noise is assumed as zero, Eq. (23) can be expressed.

$$ {n}_i\sim \mathcal{N}\left(0,{\sigma}_n^2\right)\ \Rightarrow E\left\{{n}_i^m\right\}=\left\{\begin{array}{ll}0\hfill & for\ m=2k+1\hfill \\ {}\left(m-1\right)\left(m-3\right)\dots \times 1\times {\sigma}_n^m\hfill & for\ m=2k\hfill \end{array}\begin{array}{c}\hfill\ \hfill \\ {}\hfill\ \hfill \end{array}\right. $$

(23)

Then, the Normal distribution of 4 moment noise component can be estimated as in Eq. (24).

$$ {\displaystyle \sum {n}_i^4\sim \mathcal{N}}\left(3M{\sigma}_n^4,96M{\sigma}_n^8\right) $$

(24)

The other parameters in Eq. (17) can be computed from Eq. (25) to (27).

$$ {\displaystyle \sum {s}_i^3{n}_i\sim \mathcal{N}}\left(0,M{\mu}_6{\sigma}_n^2\right)\ \&\ {\mu}_6=\frac{\sigma_s^6\ {\Gamma}^2\left(\frac{1}{v}\right)\ \Gamma \left(\frac{7}{v}\right)\ }{\Gamma^3\left(\frac{3}{v}\right)} $$

(25)

$$ {\displaystyle \sum {s}_i^2{n}_i^2\sim \mathcal{N}}\left(M{\sigma}_s^2{\sigma}_n^2,3M{\mu}_4{\sigma}_n^4-M{\sigma}_s^4{\sigma}_n^4\right) $$

(26)

$$ {\displaystyle \sum {s}_i{n}_i^3\sim \mathcal{N}}\left(0,15M{\sigma}_s^2{\sigma}_n^6\right) $$

(27)

In order to simplify the computation, two free auxiliary parameters p and q are used in Eq. (28). Therefore, R|1,p,q can formulated as in Eq. (29).

$$ p={\displaystyle {\sum}_B{s}_i^4\ \&\ q}=\frac{{\displaystyle {\sum}_A{s}_i^4}}{{\displaystyle {\sum}_B{s}_i^4}} $$

(28)

$$ \left.R\right|1,p,q=\frac{\alpha^4pq+4{\alpha}^3{\displaystyle {\sum}_A{s}_i^3{n}_i+6{\alpha}^2}{\displaystyle {\sum}_A{s}_i^2{n}_i^2+4\alpha }{\displaystyle {\sum}_A{s}_i{n}_i^3+}{\displaystyle {\sum}_A{n}_i^4}}{p+4{\displaystyle {\sum}_B{s}_i^3{n}_i+6}{\displaystyle {\sum}_B{s}_i^2{n}_i^2+4}{\displaystyle {\sum}_B{s}_i{n}_i^3+{\displaystyle {\sum}_B{n}_i^4}}}=\frac{u}{w} $$

(29)

where u and w are defined themselves by Eqs. (30) and (31).

$$ \begin{array}{l}{f}_U(u)\sim \mathcal{N}\left({\alpha}^4pq+6{\alpha}^2M{\sigma}_s^2{\sigma}_n^2+3M{\sigma}_n^4,\ 16{\alpha}^6M{\mu}_6{\sigma}_n^2+36{\alpha}^4\right.\left(3M{\mu}_4{\sigma}_n^4-M{\sigma}_s^4{\sigma}_n^4\right)+16{\alpha}^2\times 15M{\sigma}_s^2{\sigma}_n^6+\hfill \\ {}\left.96M{\sigma}_n^8\right)\hfill \end{array} $$

(30)

$$ {f}_W(w)\sim \mathcal{N}\left(p+6M{\sigma}_s^2{\sigma}_n^2+3M{\sigma}_n^4,\ 16M{\mu}_6{\sigma}_n^2+36\left(3M{\mu}_4{\sigma}_n^4-M{\sigma}_s^4{\sigma}_n^4\right)+16\times 15M{\sigma}_s^2{\sigma}_n^6+96M{\sigma}_n^8\right) $$

(31)

The density of $\frac{u}{w}$ is computed to estimate the pdf of R|1,p,q. By considering independency and normal distribution for two parameters of u and w, it is possible to express Eq. (32):

$$ {f}_{R\Big|1,p,q}(r)={\displaystyle {\int}_{-\infty}^{\infty}\left|w\right|{f}_{U,W}\left(wr,w\right)\ dw} $$

(32)

Also, if U and W are assumed as normal distribution and independent, then f _U,W (u, w) is formulated as in Eq. (33):

$$ {f}_{U,W}\left(u,w\right)={f}_U(u)\times {f}_W(w) $$

(33)

Equation (34) is closed-form solution for Eq. (31) which has already discussed in literature [14].

$$ D(r)=\frac{b(r)c(r)}{a^3(r)}\ \frac{1}{\sqrt{2\pi }{\sigma}_u{\sigma}_w}\left[2\Phi \left(\frac{b(r)}{a(r)}\right)-1\right]+\frac{1}{a^3(r)\pi {\sigma}_u{\sigma}_w}{e}^{-\frac{1}{2}\left(\frac{\mu_u^2}{\sigma_u^2}+\frac{\mu_w^2}{\sigma_w^2}\right)} $$

(34)

Each parameter in Eq. (34) is defined based on Eqs. (35) to (38):

$$ a(r)=\sqrt{\frac{r^2}{\sigma_u^2}+\frac{1}{\sigma_w^2}} $$

(35)

$$ b(r)=\frac{\mu_u}{\sigma_u^2}r+\frac{\mu_w}{\sigma_w^2} $$

(36)

$$ c(r)= exp\left\{\frac{1}{2}\frac{b^2(r)}{a^2(r)}-\frac{1}{2}\left(\frac{\mu_u^2}{\sigma_u^2}+\frac{\mu_w^2}{\sigma_w^2}\right)\ \right\} $$

(37)

$$ \Phi (r)={\displaystyle {\int}_{-\infty}^r\frac{1}{\sqrt{2\pi }}\ {e}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.{u}^2}\ du} $$

(38)

As a result, Eq. (39) formulate the density of R|1:

$$ {f}_{R\Big|1}\left(r\Big|1\right)={\displaystyle {\int}_L^U{\displaystyle {\int}_{-\infty}^{\infty }{f}_{R\Big|1,p,q}}}\left(r\Big|1,p,q\right)\ {f}_P(p)\ {f}_Q(q) $$

(39)

The lowest bound and the highest bound are applied to restrict the energy ration between two A and B sets within L and U which is stated as in Eq. (40):

$$ L<\frac{{\displaystyle {\sum}_A{r}_i^4}}{{\displaystyle {\sum}_B{r}_i^4}}<U $$

(40)

Although Eq. (22) is expressed the density of parameter P, Eq. (41) is formulated the density of parameter q based on the ratio between independent and normal distribution.

$$ {f}_Q(q)=\frac{D(q)}{{\displaystyle {\int}_L^UD(q)\ dq}} $$

(41)

With using same manner in Eq. (17), the probability of r|0 is also computable. Therefore, Eq. (42) can estimate the probability of detected error:

$$ {P}_e=\frac{1}{2}{\displaystyle {\int}_T^{\infty }f\left(r\Big|0\right)}\ dr+\frac{1}{2}{\displaystyle {\int}_{-\infty}^Tf\left(r\Big|1\right)}\ dr $$

(42)

The threshold is estimated by minimizing the error as in Eq. (43):

$$ \frac{\partial {P}_e}{\partial T}=0\ \Rightarrow\ {f}_r\left(T\Big|0\right)={f}_r\left(T\Big|1\right) $$

(43)