1 Introduction

Since the discovery of the Higgs boson [1, 2] its decay channels have been extensively studied. In general, the decay pattern and properties of the \(h\) boson are consistent [3] with the quantum numbers \(J^{PC} = 0^{++}\) of the boson in the standard model (SM). Yet the nature of \(h\) needs to be clarified and will be investigated in detail in the next run of the LHC after its upgrade.

In many extensions of the SM a more complicated Higgs sector can exist, and some of the Higgs bosons may not have definite CP parity [46]. This aspect of the Higgs-boson physics is important for clarification of the origin of the CP violation, and possible additional mechanisms beyond the CP violation via the CKM matrix which can contribute to the observed matter–antimatter asymmetry in the Universe [7].

The CP properties of the Higgs boson were addressed for the two-photon decay \(h \rightarrow \gamma \, \gamma \) in a model with vectorlike fermions [8]. It was shown that the mutual orientation of linear polarizations of the photons carries information on the CP violation. This idea was elaborated in Ref. [9], where the Bethe–Heitler conversion on nuclei of the two photons to electron–positron pairs was suggested as a means to probe the CP violation in the Higgs coupling to photons. In Ref. [10] the author analyzed possibilities of observation of the CP violation effects in the Higgs decays \( h \rightarrow V_1 \, V_2 \, \rightarrow (f_1 \, \bar{f}_2) \, (f_3 \, \bar{f}_4)\) to various final lepton and quark pairs, where \(V = W^\pm , \, Z\).

In Refs. [11, 12] the authors suggested to study the CP violation effects in the Higgs sector in the decay \(h \rightarrow \gamma Z\) via the polarization parameters of the photon, or \(Z\) boson. A direct way for this is the forward–backward (FB) asymmetry in the decay \(h \rightarrow \gamma Z \rightarrow \gamma f \bar{f}\), where the \(Z\) boson on the mass shell decays to fermions. This observable vanishes in the SM and therefore it carries information on physics beyond the SM. Estimates of CP violation effects in some models of new physics were made in [11, 12].

The invariant-mass distributions in the Higgs decay to the \(\gamma \ell ^+ \ell ^- \) and \(\gamma q \bar{q}\) final states were intensively explored in Refs. [1320]. The first experimental study of the process \(h \rightarrow \gamma \mu ^+ \mu ^- \) by the CMS collaboration was recently reported in [21]. The analysis in [21] was performed for dimuon invariant mass less than 20 GeV.

In Refs. [22, 23] the authors discussed the angular distribution of the leptons \(\ell = (e, \, \mu , \, \tau )\) in the decay \(h \rightarrow \gamma \ell ^+ \ell ^-\) in the framework of the SM. The importance of the FB asymmetry was emphasized, and its nonzero values were found. At the same time, beyond the SM, in Ref. [24] the FB asymmetry was proposed as a probe for CP-violating Higgs coupling to \(Z \gamma \) and \(\gamma \gamma \) states.

The FB asymmetry sure enough is an informative observable which can be of interest for future experiments at the LHC. In the present paper we address the decay \(h \rightarrow \gamma \ell ^+ \ell ^-\) in some detail. In addition to the loop mechanism \(h \rightarrow \gamma Z^* \rightarrow \gamma \ell ^+ \ell ^-\) considered in [11, 12], we include here the photon bremsstrahlung off leptons, i.e. tree-level amplitudes for \(h \rightarrow \gamma \ell ^+ \ell ^- \), and the loop amplitude \(h \rightarrow \gamma \gamma ^* \rightarrow \gamma \ell ^+ \ell ^-\). We remark that in the framework of the SM the FB asymmetry is equal to zero as a consequence of the scalar nature of the Higgs boson. This asymmetry can take nonzero values only in models beyond the SM and therefore this observable is sensitive to possible CP violation in the Higgs sector.

To estimate values of this asymmetry we apply a model in which the Higgs boson couples to fermions with a mixture of the scalar (S) and pseudoscalar (PS) interactions. The strength of the S and PS couplings, \(1 + s_{f}\) and \(p_{f}\), respectively, are partly constrained from the LHC measurements of the decay rates \(h \rightarrow \ell ^+ \ell ^- \) and \(h \rightarrow q \bar{q}\) (where \(q=(c, \, b)\)) [25, 26]. As for the Higgs interaction with the top quark, the corresponding couplings are chosen from experimental information on the two-photon, \(h \rightarrow \gamma \gamma \), and two-gluon, \(h \rightarrow g g\), decay widths.

In this model, for the decays \(h \rightarrow \gamma \ell ^+ \ell ^-\) we derive the distribution over the angle \(\theta \) between the momentum of the lepton (in the rest frame of the pair \(\ell ^+ \ell ^-\)) and momentum of the photon (in the rest frame of \(h\)). The presence of the PS \(h f \bar{f}\) coupling gives rise to the linear in \(\cos \theta \) terms in this distribution, and thereby to a FB asymmetry. We calculate the differential decay width and FB asymmetry as functions of the dilepton invariant mass squared \(q^2 = (q_+ + q_-)^2\) (\(q_+\) and \(q_-\) are the four-momenta of leptons). The widths and FB asymmetries integrated over the invariant mass are also discussed.

The paper is organized as follows. In Sect. 2 amplitudes and angular distribution in \(h \rightarrow \gamma \ell ^+ \ell ^-\) are presented. The loop contributions are defined for the S and PS Higgs couplings to the fermions. The FB asymmetry is discussed. In Sect. 3 the differential decay width and FB asymmetry for various leptons are calculated. The results of the calculation are discussed. Section 4 contains the conclusions. In Appendix A the loop integrals are defined, and in Appendix B vanishing of the contribution from axial-vector \(Z f \bar{f}\) coupling to the fermion-loop diagrams is shown.

2 Formalism

2.1 Amplitudes and angular distribution

There are models with more than one Higgs doublet which induce CP violation due to the specific coupling of neutral Higgs bosons to fermions. We assume that the couplings of \(h\) boson to the fermion fields, \(\psi _{f}\), are given by the Lagrangian including both scalar and pseudoscalar parts,

$$\begin{aligned} \mathcal{L}_\mathrm{hff}=-\sum _{f = \ell , \, q} \frac{m_{f}}{v}\,h\,{\bar{\psi }_{f}}\left( 1+s_{f}+i\,p_{f} \gamma _5\right) \psi _{f}, \end{aligned}$$
(1)

where \(v=(\sqrt{2}G_{f})^{-1/2}\approx 246\) GeV is the vacuum expectation value of the Higgs field, \(G_{f} = 1.166378 \times 10^{-5}\) GeV\(^{-2}\) is the Fermi constant [27], \(m_f\) is the fermion mass, and \(s_{f}\), \(p_{f}\) are real parameters (\(s_{f}=p_{f}=0\) corresponds to the SM). Equation (1) can be considered as a phenomenological parametrization of effects of new physics. As for the Higgs interaction with the \(W^\pm \) and \(Z\) bosons, it is assumed to be the same as in the SM.

We consider the decay of the zero-spin Higgs \(h\) boson

$$\begin{aligned} h(p) \rightarrow \gamma (k, \, \epsilon (k) ) +\ell ^+ (q_+) + \ell ^- (q_-) , \end{aligned}$$
(2)

where the four-momenta of the \(h\) boson, photon, and leptons are \(p\), \(k\), \(q_+\), \(q_-\), respectively, and \(\epsilon (k)\) is the polarization four-vector of the photon.

The differential decay width can be written as

$$\begin{aligned} \frac{ \mathrm{d} \Gamma }{\mathrm{d} q^2 \, \mathrm{d} \cos \theta }= \frac{\beta _{\ell } (m_h^2 -q^2)}{(8 \pi )^3 \, m_h^3} \, |\mathcal{M}|^2 , \end{aligned}$$
(3)

where \(m_h\) is the mass of the \(h\) boson, \(q \equiv q_+ + q_- \), \(q^2\) is the invariant mass squared of the lepton pair, \(\beta _{\ell } = \sqrt{1-4 m_{\ell }^2/q^2}\) is the lepton velocity in the rest frame of the lepton pair. The polar angle \(\theta \) is defined in this frame and it is the angle between the momentum of lepton \(l^+\) and the axis opposite to the direction of the Higgs-boson momentum.

The amplitude of the decay is

$$\begin{aligned} \mathcal{M} = \mathcal{M}_\mathrm{tree} + \mathcal{M}_\mathrm{loop}, \end{aligned}$$
(4)

where the tree-level amplitude (Fig. 1) is

(5)

where

$$\begin{aligned} c_0 = e m_{\ell } Q_{\ell } \, ({\sqrt{2} G_{f}})^{1/2}, \end{aligned}$$
(6)

\(e = \sqrt{4 \pi \alpha _{G_{f}}}\) is the positron charge, \(Q_{\ell } = -1\) (lepton charge in units of \(e\)) and \(m_{\ell }\) is the lepton mass. The electromagnetic coupling in the \(G_{f}\)-scheme [28] is \(\alpha _{G_{f}} = \sqrt{2}G_{f} m_W^2(1-m_W^2 /m_Z^2)/ \pi \), where \(m_W \, (m_Z)\) is the mass of the \(W\) (\(Z\)) boson. For the rest we follow the standard definition of the \(\gamma \) matrices and lepton spinors (see, e.g. [29]).

Fig. 1
figure 1

Diagrams for the process \(h \rightarrow \gamma \ell ^+ \ell ^-\). The upper row shows the tree-level (bremsstrahlung) amplitudes, and the loop diagrams are drawn below. Fermions \(f\) are indicated by the solid lines, gauge bosons \(W^\pm , \, Z, \, \gamma \) by the wavy lines, and \(h\) boson by the dashed lines

The loop contributions \(h \rightarrow \gamma \, \gamma ^* / Z^* \rightarrow \gamma \ell ^+ \ell ^-\) (see Fig. 1) can be written in the form

$$\begin{aligned} \mathcal{M}_\mathrm{loop}&= \epsilon _\mu ^* (k) \, [ \, (q^\mu k^\nu - g^{\mu \nu } k\cdot q) \nonumber \\&\times \bar{u}(q_-) ( c_1 \gamma _\nu + c_2 \gamma _\nu \gamma _5) v(q_+) \nonumber \\&- \epsilon ^{\mu \nu \alpha \beta } k_\alpha q_\beta \, \bar{u}(q_-) ( c_3 \gamma _\nu + c_4 \gamma _\nu \gamma _5 ) v(q_+)], \end{aligned}$$
(7)

with coefficients \(c_1, \ldots , c_4 \), which are specified below in terms of the loop functions, and \(\epsilon _{0123} = +1\). Here we follow the notation in Refs. [22, 23].

Note that we do not take into account the loop contributions of the type \(h \rightarrow \gamma \ell ^+ \ell ^- \) (the so-called box diagrams). The contribution of these diagrams to the considered decay is negligibly small in the SM [13, 14]. Also the processes \(h \rightarrow \gamma V \rightarrow \gamma \ell ^+ \ell ^-\), where \(V\) is intermediate vector resonance decaying into the \(\ell ^+ \ell ^-\) pair, can contribute to the decay \(h \rightarrow \gamma \ell ^+ \ell ^-\). In particular, resonant production of the quarkonium states \(J/ \psi \, (c \bar{c})\) and \(\Upsilon (1S) \, (b \bar{b})\) is of interest for studying the \(h q \bar{q}\) coupling (see, for example [3033]). The account of such mechanisms lies beyond the scope of the present work.

We evaluate the amplitude (4) squared, sum over the lepton and photon polarizations, and obtain the following result in the model (1):

$$\begin{aligned} |\mathcal{M}|^2&= c_0^2 \, \bigl [ (1+s_{\ell })^2 \, A + p_{\ell }^2 \, \widetilde{A} \bigr ] \nonumber \\&+ 2 \, c_0 \, \bigl [ (1+s_{\ell }) \, \mathrm{Re} (c_1) \, B + p_{\ell } \, \mathrm{Im} (c_2) \, \widetilde{B} \, \nonumber \\&+ (1+s_{\ell }) \, \mathrm{Im} (c_4) \, C + p_{\ell } \, \mathrm{Re}(c_3) \, \widetilde{C} \bigr ] \nonumber \\&+ \bigl ( |c_1|^2 + |c_3|^2 \bigr ) \, D + \bigl ( |c_2|^2 + |c_4|^2 \bigr ) \, E \nonumber \\&+ 2 \, \mathrm{Im} \bigl (c_1 c_4^* + c_2 c_3^*\bigr ) \, F . \end{aligned}$$
(8)

The fact that \(c_0\) is real while \(c_1, \ldots , c_4\) are generally complex-valued is used in derivation of (8).

The coefficients in Eq. (8) are defined as follows (we use the notation \(z \equiv \cos \theta \)):

$$\begin{aligned} A&= \frac{16}{(1- \beta _{\ell }^2 z^2 )^2 (m_h^2-q^2)^2} \left[ \, (m_h^4 + q^4 \right. \nonumber \\&\left. - 8 m_{\ell }^2 q^2)(1-\beta _{\ell }^2 z^2) +32 m_{\ell }^4 - 8 m_h^2 m_{\ell }^2 \, \right] , \end{aligned}$$
(9)
$$\begin{aligned} \widetilde{A}&= \frac{16}{(1- \beta _{\ell }^2 z^2 )^2 (m_h^2-q^2)^2} \left[ \, (m_h^4+q^4) \right. \nonumber \\&\left. \times \, (1-\beta _{\ell }^2 z^2) - 8 m_h^2 m_{\ell }^2\, \right] , \end{aligned}$$
(10)
$$\begin{aligned} B&= - \frac{8 m_{\ell } }{(1- \beta _{\ell }^2 z^2 )} [\, m_h^2 -q^2 +q^2 \beta _{\ell }^2 (1-z^2 )\, ] , \end{aligned}$$
(11)
$$\begin{aligned} \widetilde{B}&= - \frac{8 m_{\ell } }{(1- \beta _{\ell }^2 z^2 )} \, (m_h^2-q^2) \, \beta _{\ell } \, z , \end{aligned}$$
(12)
$$\begin{aligned} C&= - \frac{8 m_{\ell } }{(1- \beta _{\ell }^2 z^2 )} \, (m_h^2-q^2) \, \beta _{\ell } \, z , \end{aligned}$$
(13)
$$\begin{aligned} \widetilde{C}&= \frac{8 m_{\ell } }{(1- \beta _{\ell }^2 z^2 )} (m_h^2-q^2), \end{aligned}$$
(14)
$$\begin{aligned} D&= \frac{1}{2} (m_h^2-q^2)^2 \, [\, q^2 (1 + \beta _{\ell }^2 z^2 ) +4 m_{\ell }^2 \,], \end{aligned}$$
(15)
$$\begin{aligned} E&= \frac{1}{2} (m_h^2-q^2)^2 \, q^2 \, \beta _{\ell }^2 \, (1 + z^2 ), \end{aligned}$$
(16)
$$\begin{aligned} F&= - (m_h^2-q^2)^2 \, q^2 \, \beta _{\ell } \, z . \end{aligned}$$
(17)

The FB asymmetry is defined as (see, e.g. [11, 12, 22, 23])

$$\begin{aligned} A_\mathrm{FB} (q^2) =\left( {\frac{\mathrm{d} \Gamma _F }{\mathrm{d} q^2} - \frac{\mathrm{d} \Gamma _B }{\mathrm{d} q^2}} \right) {\left( \frac{\mathrm{d} \Gamma _F }{\mathrm{d} q^2} + \frac{\mathrm{d} \Gamma _B }{\mathrm{d} q^2} \right) ^{-1} }, \end{aligned}$$
(18)

where

$$\begin{aligned} \frac{\mathrm{d} \Gamma _F }{\mathrm{d} q^2}&\equiv \int _0^1 \frac{\mathrm{d} \Gamma }{\mathrm{d} q^2 \, \mathrm{d} \cos \theta }\, \mathrm{d} \cos \theta , \nonumber \\ \frac{\mathrm{d} \Gamma _B }{\mathrm{d} q^2}&\equiv \int _{-1}^0 \frac{ \mathrm{d} \Gamma }{ \mathrm{d} q^2 \, \mathrm{d} \cos \theta }\, \mathrm{d}\cos \theta . \end{aligned}$$
(19)

As only the coefficients \(\widetilde{B}, \, C\), and \(F\) are linear in \(\cos \theta \), then it is seen from Eqs. (9)–(17) that the numerator of the asymmetry (18) is determined by the imaginary part of the terms \(c_2\), \(c_4\), and the combination \(c_1 c_4^* + c_2 c_3^*\):

$$\begin{aligned}&\frac{\mathrm{d} \Gamma _F }{\mathrm{d} q^2} - \frac{\mathrm{d} \Gamma _B }{\mathrm{d} q^2} = -\frac{2 (m_h^2-q^2)^2 }{(8 \pi )^3 \, m_h^3} \nonumber \\&\times \left[ c_0 \, \left( p_\ell \, \mathrm{Im}(c_2) + (1+s_\ell ) \, \mathrm{Im} (c_4) \right) \, 8 m_\ell \ln \left( \frac{q^2}{4m_\ell ^2} \right) \right. \nonumber \\&\left. + \, \mathrm{Im} \left( c_1 c_4^* + c_2 c_3^*\right) \, (q^2 - 4 m_\ell ^2) \, (m_h^2-q^2) \right] . \end{aligned}$$
(20)

It may be instructive to analyze the asymmetry (18) in the limit of zero lepton masses. Putting \(m_{\ell } =0\) in (8), (11)–(17), and (20) one obtains the distribution over the \(\ell ^+ \ell ^-\) invariant mass

$$\begin{aligned} \frac{\mathrm{d} \Gamma }{\mathrm{d} q^2} \equiv \frac{\mathrm{d} \Gamma _F }{\mathrm{d} q^2} + \frac{\mathrm{d} \Gamma _B }{\mathrm{d} q^2} = \frac{ (m_h^2 -q^2)^3 \, q^2 }{6 \, (4 \pi )^3 \, m_h^3 } \, \sum _{j=1}^4 \, |c_j|^2 \, \end{aligned}$$
(21)

and the FB asymmetry

$$\begin{aligned} A_\mathrm{FB}(q^2) = -\frac{3}{2} \frac{\mathrm{Im}\big (c_1 c_4^* +c_2 c_3^*\big )}{|c_1|^2 + |c_2|^2 + |c_3|^2 + |c_4|^2} . \end{aligned}$$
(22)

For further reference we also introduce the integrated-over-\(q^2\) asymmetry [22]:

$$\begin{aligned} \langle A_\mathrm{FB} \rangle = \int \left( {\frac{\mathrm{d} \Gamma _F }{\mathrm{d} q^2} - \frac{\mathrm{d} \Gamma _B }{\mathrm{d} q^2}} \right) \, \mathrm{d} q^2 \, \left( \int \frac{\mathrm{d} \Gamma }{\mathrm{d} q^2} \mathrm{d} q^2\right) ^{-1}, \end{aligned}$$
(23)

for the appropriate integration limits \(q^2_\mathrm{min} \ge 4 m_\ell ^2 \) and \(q^2_\mathrm{max} \le m_h^2\).

2.2 Loop contributions

Let us specify the loop contributions in Fig. 1 to the coefficients \(c_1, \ldots , c_4\). We introduce below the Weinberg angle \(\theta _W\) and the notation \(s_W \equiv \sin \theta _W\) and \(c_W \equiv \cos \theta _W\).

We evaluate the loop diagrams using the Lagrangian (1) for the \(h f \bar{f}\) vertex. The scalar coupling of the Higgs to fermions contributes to the coefficients \(c_1, \, c_2\), which read

$$\begin{aligned} c_1&= \frac{1}{2} \, \frac{g_{V, \ell }}{q^2 - m_Z^2 + i m_Z \Gamma _Z} \, \Pi _{ Z} + \frac{Q_{\ell }}{q^2} \, \Pi _{\gamma } , \end{aligned}$$
(24)
$$\begin{aligned} c_2&= - \frac{1}{2} \, \frac{g_{A, \ell }}{q^2 - m_Z^2 + i m_Z \Gamma _Z} \, \Pi _{ Z} , \end{aligned}$$
(25)

where \(m_Z\) (\(\Gamma _Z\)) is the mass (total decay width) of the \(Z\) boson, and

$$\begin{aligned} \Pi _{ Z}&= \frac{e g^3}{16 \pi ^2 m_W } \, \left[ (1+s_{f}) \, \frac{2 g_{V, f}}{c_W^2} \, N_f Q_f A_f (\lambda _f^\prime , \lambda _f) \right. \nonumber \\&\left. + A_W (\lambda _W^\prime , \lambda _W) \right] , \end{aligned}$$
(26)
$$\begin{aligned} \Pi _{\gamma }&= \frac{e^3 g}{16 \pi ^2 m_W} \, \bigl [ (1+s_{f}) \, 4 Q_f^2 N_f \, A_f (\lambda _f^\prime , \lambda _f) \nonumber \\&+ A_W (\lambda _W^\prime , \lambda _W) \bigr ] . \end{aligned}$$
(27)

Here \(g = 2 m_W (\sqrt{2}G_{f})^{1/2}\) is the \(SU(2)_L\) coupling, \(Q_f\) is the charge of the fermion \(f\) in units of \(e\), \(N_f = 1 (3)\) for leptons (quarks), \(g_{V, f} = t_{3L, f} - 2Q_f s_W^2\) and \(g_{A, f} = t_{3L, f}\) are the vector and axial-vector couplings of \(Z\) boson to the fermion, where \(t_{3L, f}\) is the projection of the weak isospin, and

$$\begin{aligned} \lambda _{f, \, W} \equiv \frac{4m_{f, \, W}^2}{q^2 }, \quad \lambda ^{\prime }_{f, \, W} \equiv \lambda _{f, \, W} |_{q^2 = m_h^2} . \end{aligned}$$
(28)

The loop integrals for fermions, \(A_f (\lambda _f^\prime , \lambda _f)\), and \(W\) bosons, \(A_W (\lambda _W^\prime , \lambda _W)\), are expressed in terms of the loop functions \(I_{1} (\lambda ^\prime , \, \lambda )\) and \(I_{ 2} (\lambda ^\prime , \, \lambda )\) [34] (see Appendix A).

The coefficients \(c_3, \, c_4\) in the amplitude (7) come only from the PS coupling of the Higgs boson to fermions in the loops. We obtain

$$\begin{aligned}&c_3 = \frac{1}{2} \, \frac{g_{V, \ell }}{q^2 - m_Z^2 + i m_Z \Gamma _Z} \, \widetilde{\Pi }_{Z} + \frac{Q_{\ell }}{q^2} \, \widetilde{\Pi }_{\gamma } , \end{aligned}$$
(29)
$$\begin{aligned}&c_4 = - \frac{1}{2} \, \frac{ g_{A, \ell }}{q^2 - m_Z^2 + i m_Z \Gamma _Z} \, \widetilde{\Pi }_{ Z} \, \end{aligned}$$
(30)
$$\begin{aligned}&\widetilde{\Pi }_{ Z} = \frac{e g^3}{16 \pi ^2m_W } \, p_{f} \, \frac{2 g_{V,f}}{c_W^2} N_f Q_f I_2 (\lambda _f^\prime , \lambda _f) , \end{aligned}$$
(31)
$$\begin{aligned}&\widetilde{\Pi }_{ \gamma } = \frac{e^3 g}{16 \pi ^2 m_W} \, p_{f} \, 4 Q_f^2 \, N_f \, I_2 (\lambda _f^\prime , \lambda _f) . \end{aligned}$$
(32)

Of course, the sum over all fermions \(f = (\ell , q)\) in (26), (27), (31), and (32) is implied.

2.3 Forward–backward asymmetry in the SM

In the SM the angular distribution in Eq. (8) simplifies. Indeed, one sets \(s_{\ell } = p_{\ell }=0\) in Eq. (8) and \(s_{f} = p_{f} =0\) in Eqs. (26), (27), (31), and (32). Then \(c_{3, \, SM} = c_{4,\, SM} = 0\) and (8) turns into

$$\begin{aligned} |\mathcal{M}|^2_{SM}&= c_0^2 \, A + 2 \, c_0 \, \mathrm{Re} (c_{1, \, SM}) \, B \, \nonumber \\&+ |c_{1, \, SM}|^2 \, D + |c_{2, \, SM}|^2 \, E , \end{aligned}$$
(33)

where \(c_{1, \, SM} = c_{1}|_{s_{f} =0}\) and \(c_{2, \, SM} = c_{2}|_{s_{f} =0}\) in (24) and (25).

In follows from Eq. (20) that in the SM

$$\begin{aligned} A_\mathrm{FB } (q^2)_{SM} = 0. \end{aligned}$$
(34)

Therefore a nonzero value of the FB asymmetry can arise only in certain models beyond the SM. A similar conclusion for the decay \(h \rightarrow \gamma Z \rightarrow \gamma \ell ^+ \ell ^-\) with on-mass-shell \(Z\) boson has been inferred in [11, 12] and is used there to estimate the magnitude of possible CP violation effect.

The result (34) is at variance with the conclusion of Refs. [22, 23], where the authors have found a nonzero FB asymmetry in the framework of the SM. The origin of a nonzero asymmetry in Ref. [22] is related to the axial-vector coupling of the \(Z\) boson to the fermions in the loop diagrams.

In fact, the axial-vector \(Z f \bar{f}\) coupling does not contribute to the process \(h \rightarrow \gamma Z^*\) (for real or virtual \(Z\)). This was noticed long ago in the framework of the SM in Refs. [3537] on the basis of charge-conjugation parity arguments. As an alternative argument, in Appendix B we show in the model (1) and in the SM explicit cancelation of contributions from axial-vector \(Z f \bar{f}\) coupling to the fermion-loop diagrams for \(h \rightarrow \gamma ^* Z^*\).

3 Results of calculations and discussion

Let us discuss the choice of parameters \(s_{f}\) and \(p_{f}\) for the Higgs coupling to the fermions in (1). In terms of these parameters the decay width of the Higgs to fermions, except the top quark, is equal to

$$\begin{aligned} \Gamma (h \rightarrow f \bar{f}) = \frac{N_f G_{f}}{4 \sqrt{2} \pi } \, m_f^2 \, m_h \, \beta _f \bigl ( |1+s_{f}|^2 \beta _f^2 + |p_{f}|^2 \bigr ) , \nonumber \\ \end{aligned}$$
(35)

where \(\beta _f = \sqrt{1- 4m_f^2/m_h^2} \) is the fermion velocity in the rest frame of \(h\). Apparently, one can put \(\beta _f \approx 1\). Then in order to keep the Higgs decay widths to fermions equal to their SM values we impose the following constraint on the parameters \(s_{f}, \, p_{f}\):

$$\begin{aligned} |1+s_{f}|^2 + |p_{f}|^2 = 1 . \end{aligned}$$
(36)

In this case, in order to ascertain the exact values of the parameters \(s_{f}\) and \(p_{f}\) one would need to measure polarization characteristics of the leptons, which is not accessible at present.

Although Eq. (36) does not uniquely determine the parameters we choose the tentative values as in Ref. [11]:

$$\begin{aligned} s_{f} = 1/\sqrt{2}-1 , \quad p_{f} = \pm \,1/\sqrt{2}\, \end{aligned}$$
(37)

for all fermions. These values imply an equal weight of \(1/2\) of the S and PS couplings.

Regarding the Higgs couplings to the top quark, we will choose them by requiring that the ratios

$$\begin{aligned} \mu _{gg} = \frac{\Gamma (h\rightarrow gg) }{\Gamma _\mathrm{SM}(h\rightarrow gg)} , \quad \mu _{\gamma \gamma } = \frac{\Gamma (h\rightarrow \gamma \gamma ) }{\Gamma _\mathrm{SM}(h \rightarrow \gamma \gamma )} \end{aligned}$$
(38)

are consistent with the recent CMS results [38],

$$\begin{aligned} \mu _{ggh, \, t \bar{t}h} = 1.13^{+0.37}_{-0.31}, \quad \mu _{\gamma \gamma } = 1.14^{+0.26}_{-0.23} . \end{aligned}$$
(39)

This allows us to choose the following values of parameters \(s_t\) and \(p_t\):

$$\begin{aligned} s_t = -0.3 , \quad |p_t | = 0.55 . \end{aligned}$$
(40)

With these parameters, values of \(\mu _{gg}\) and \(\mu _{\gamma \gamma }\) appear to be, respectively, 1.2 and 1.23.

As the interaction of the Higgs boson with the \(W^\pm \) and \(Z\) is not modified compared to the interaction in the SM, the observables in the decays \(h \rightarrow Z Z \rightarrow 4 \ell \) and \(h \rightarrow W W \rightarrow \ell \nu _\ell \ell \nu _\ell \), where \(\ell = (e, \, \mu )\), are consistent with the ATLAS and CMS data and spin–parity analyses [39, 40].

Numerical values of the SM parameters are taken from [27], namely, the gauge boson masses, widths, and \(Z f \bar{f}\) couplings. The quark masses are chosen according to [28, 41], and \(\sin ^2 \theta _W = 1- m_W^2/m^2_Z\).

In Fig. 2 we show the differential decay width for \(h \rightarrow \ell ^+ \ell ^- \gamma \) for various leptons \(\ell = (e, \, \mu , \, \tau )\) calculated in the SM model and in the model of new physics (1) with parameters \( s_{f} = 1/\sqrt{2}-1, \, p_{f} = + 1/\sqrt{2}\) and \(s_t = -0.3, \, p_t = +0.55\). This choice of parameters is called hereafter NP1. The photon minimal energy in the Higgs-boson rest frame is taken \(E_\gamma = 1\) GeV in order to cut-off infrared divergence, so that \(q_{max} = (m_h^2 - 2 m_h E_\gamma )^{1/2} \approx m_h - E_\gamma \).

Fig. 2
figure 2

Differential decay width for various final lepton pairs as a function of the dilepton invariant mass \(q \equiv \sqrt{q^2}\). Solid lines are calculated in the SM, dashed lines in the model (1) (parameters NP1; see the text)

As is seen from Fig. 2, there is a deviation from the prediction of the SM with the chosen parameters \(s_{f}, \, p_{f}\) of new physics. Integration over the invariant mass within the interval \([q_\mathrm{min}, \, q_\mathrm{max}]\) leads to the widths shown in Table 1.

Table 1 Decay width \(\Gamma (h \rightarrow \gamma \ell ^+ \ell ^-)\) in keV for various lepton states in the interval of invariant masses from \(q_\mathrm{min}\) to \(q_\mathrm{max}\) (in GeV)

The effect of new physics appears on the level 10–20%, if the invariant-mass interval lies below 30 GeV. Although the decay width in this interval is very small compared, for example, to the two-photon decay width of the Higgs boson in the SM \(\Gamma (h \rightarrow \gamma \gamma )=9.28\) keV (Ref. [28]; see Table A.10 therein).

As one can also see from Fig. 2, for the decay \(h \rightarrow \gamma e^+ e^-\) the dominant contribution to the width in Table 1 comes from the loop amplitude. For the \(h \rightarrow \gamma \mu ^+ \mu ^-\) decay, the tree-level and loop contributions are comparable, while for the \(h \rightarrow \gamma \tau ^+ \tau ^-\) decay, the tree-level amplitude gives the dominant contribution.

In Fig. 3 the FB asymmetry (18) is presented as a function of \(q\). As mentioned above, the FB asymmetry can take nonzero values only in models beyond the SM, though not all models of new physics lead to nonzero FB asymmetry. In the model (1), \(A_\mathrm{FB}(q^2)\) is proportional to Eq. (20). All terms in (20) are proportional to the parameters \(p_{f}\) which characterize PS couplings of the Higgs to the fermions. So in this model the FB asymmetry is a direct measure of a possible CP violation in the \(h f \bar{f}\) coupling.

Fig. 3
figure 3

Forward–backward asymmetry for various final leptons, calculated in the model (1). Solid lines correspond to parameters NP1 (\(p_t = +0.55\)), dashed lines to parameters NP2 (\(p_t = +0.55 \, i\)). The sign of the asymmetry is indicated at the curve

For the light final leptons, \(e^+ e^-\) and \(\mu ^+ \mu ^-\), the dominant contribution to \(A_\mathrm{FB} (q^2)\) comes from the term in (20) proportional to the imaginary part of the combination \(c_1 c_4^* + c_2 c_3^*\). This imaginary part in turns originates from the \(Z\)-boson propagators in Eqs. (26), (27), (31), and (32), and the loop contributions \(\Pi _Z, \Pi _\gamma , \widetilde{\Pi }_{ Z}\) and \(\widetilde{\Pi }_{ \gamma }\). The latter have small imaginary parts arising due to the intermediate on-mass-shell fermion–antifermion pairs with the masses \(m_f \le m_h/2\). These imaginary parts come mainly from the bottom, charm quarks, and the \(\tau \) lepton (this fact was also noticed in [11]).

As is seen from Fig. 3, for real values of the parameters \(s_{f}, \, p_{f}\) (see solid lines) the FB asymmetry takes values less than \(1\,\%\) for the electrons and muons, with maximum value \(0.8\,\%\) at the dilepton invariant mass around the \(Z\)-boson. For the \(\tau \) leptons, the FB asymmetry is bigger, with a maximum value of about \(2.5\,\%\). In principle, observation of a nonzero FB asymmetry will point to CP violation in the Higgs coupling to fermions, though its small values make the corresponding experimental task difficult.

Let us emphasize that real parameters \(s_{f}, \, p_{f}\) follow from the requirement of Hermiticity of the Lagrangian \(\mathcal{L}_{h f f}\) in Eq. (1). Note that Hermiticity of Hamiltonian is a necessary condition, in addition to Lorentz invariance, locality, and the connection between spin and statistics, in the proof of the \(CPT\) theorem in quantum field theory [42]. It is of interest to explore how a possible non-Hermiticity of the Lagrangian (1) will influence the FB asymmetry. Noticeable sensitivity of the FB asymmetry to non-Hermiticity of the \(h f \bar{f}\) Lagrangian, in principle, can be used for testing the \(CPT\) symmetry.

For the purpose of this we change for the top quark the parameter \(p_t\) from real value \(0.55\) to the imaginary value \(0.55\, i\), while keeping the rest of parameters equal to their values in model NP1. This model is hereafter called NP2. Note that this choice of parameters does not affect the values of \(\mu _{gg}\) and \(\mu _{\gamma \gamma }\) calculated above.

As a result, the FB asymmetry increases substantially, up to 22  % for electron and 14  % for muon, while for \(\tau \) lepton the maximal value of the asymmetry remains on the level of 2.5  % (see dashed lines in Fig. 3).

In general, \(A_\mathrm{FB}(q^2)\) changes sign as a function of the invariant mass, therefore the integrated FB asymmetry (23) over the whole interval of \(q\) is rather small and is not a suitable observable. In particular, in the model NP1 (NP2) the FB asymmetry integrated over the interval \([1, \; 124]\) GeV for electrons and muons is \(\langle A_\mathrm{FB} \rangle = -0.4\,\,\%\) (\(+1\,\,\%\)), and integrated over the interval \([4, \; 124]\) GeV for \(\tau \) leptons is \(\langle A_\mathrm{FB} \rangle = -0.06\,\,\%\) (\(-0.04\,\,\%\)).

However, the integrated asymmetry increases for an appropriately chosen interval of invariant mass, in which \(A_\mathrm{FB}(q^2)\) does not change sign. For example, within the interval \([37.5, \; 75]\) GeV, for the \(e^+ e^-\) pair \(\langle A_\mathrm{FB} \rangle = -0.4\,\,\%\) (\(+17\,\,\%\)) in the model NP1 (NP2). Within the same interval, for the \(\mu ^+ \mu ^-\) pair \(\langle A_\mathrm{FB} \rangle = - 0.3\,\,\%\) (\(+12\,\,\%\)) in the model NP1 (NP2).

4 Conclusions

The differential decay width and forward–backward asymmetry have been calculated for the decay of Higgs boson to the photon and lepton–antilepton pair, \(h \rightarrow \gamma \ell ^+ \ell ^-\), where \(\ell = (e,\, \mu , \, \tau )\). The calculations were performed in the framework of the SM and in a model of new physics, in which the Higgs boson interacts with fermions via a mixture of scalar and pseudoscalar couplings. Both the tree-level amplitudes and the one-loop \(h \rightarrow \gamma Z^* \rightarrow \gamma \ell ^+ \ell ^-\) and \(h \rightarrow \gamma \gamma ^* \rightarrow \gamma \ell ^+ \ell ^-\) diagrams have been included.

We noted that the FB asymmetry vanishes identically in the SM. In models of new physics, which include effects of CP violation in the \(h f \bar{f}\) interaction, this asymmetry takes nonzero values. The experimental study of the FB asymmetry is of interest in the search for effects of new physics in the Higgs–fermion interaction.

In numerical estimates of the decay width and FB asymmetry, the model parameters \(s_{f}, \, p_{f}\) have been chosen by requiring that the \(h \rightarrow f \bar{f}\) decay widths coincide with the widths in the SM for all leptons and quarks, except the top quark. For the latter the parameters \(s_t, \, p_t\) were constrained from the conditions that the rates of the \(h \rightarrow \gamma \gamma \) and \(h \rightarrow g g\) decays are consistent with the CMS data [38].

In the differential decay widths effects of new physics appear on the level of 10–20  %, especially at relatively small values of dilepton invariant mass \(\lesssim 30\) GeV.

As for the FB asymmetry, it takes nonzero values; however, these values are small. In particular, \(A_\mathrm{FB}(q^2)\) reaches 1  % for electrons and muons and 2.5  % for \(\tau \) leptons in the region of invariant mass \(q \sim m_Z\).

We have also shown that the FB asymmetry increases considerably if the parameter \(p_t\) for the pseudoscalar \(h t \bar{t}\) coupling becomes complex. Specifically, for the imaginary value \(p_t = 0.55\, i\) the asymmetry rises up to 22  % for electrons and 14  % for muons in the region of invariant mass \(q \sim 50\)\(60\) GeV. Hence the FB asymmetry for the \(e^+ e^-\) and \(\mu ^+ \mu ^-\) pairs turns out to be sensitive to the non-Hermiticity of the \(h t \bar{t}\) interaction Lagrangian. Since the requirement of Hermiticity underlies the proof of the \(CPT\) theorem, the FB asymmetry may also be used for testing the \(CPT\) symmetry.

For the \(\tau \) leptons, the FB asymmetry is sensitive to non-Hermiticity of the \(h t \bar{t}\) interaction Lagrangian in the region of relatively small \(q \sim 20\) GeV, staying less than 1  %. Its maximal value 2.5  % remains the same with the real and imaginary parameter \(p_t\).

In our opinion experimental study of the differential decay width and FB asymmetry in the \(h \rightarrow \gamma \ell ^+ \ell ^-\) decays may give additional information on the couplings of the Higgs boson to fermions.