Anomaly-free models for flavour anomalies

Abstract

We explore the constraints imposed by the cancellation of triangle anomalies on models in which the flavour anomalies reported by LHCb and other experiments are due to an extra U(1)$^\prime$ gauge boson Z$^\prime$. We assume universal and rational U(1)$^\prime$ charges for the first two generations of left-handed quarks and of right-handed up-type quarks but allow different charges for their third-generation counterparts. If the right-handed charges vanish, cancellation of the triangle anomalies requires all the quark U(1)$^\prime$ charges to vanish, if there are either no exotic fermions or there is only one Standard Model singlet dark matter (DM) fermion. There are non-trivial anomaly-free models with more than one such ‘dark’ fermion, or with a single DM fermion if right-handed up-type quarks have non-zero U(1)$^\prime$ charges. In some of the latter models the U(1)$^\prime$ couplings of the first- and second-generation quarks all vanish, weakening the LHC $Z^\prime$ constraint, and in some other models the DM particle has purely axial couplings, weakening the direct DM scattering constraint. We also consider models in which anomalies are cancelled via extra vector-like leptons, showing how the prospective LHC Z$^\prime$ constraint may be weakened because the $Z^\prime \rightarrow \mu ^+ \mu ^-$ branching ratio is suppressed relative to other decay modes.

1 Introduction

The LHCb Collaboration and other experiments have reported a number of anomalies in semileptonic B decays, including apparent violations of $\mu$–e universality in $B \rightarrow K^{(*)} \ell ^+ \ell ^-$ decays [1,2,3,4], and apparent deviations from the Standard Model (SM) predictions for the $P_5^\prime$ angular variable in $B \rightarrow K^* \ell ^+ \ell ^-$ decay [5,6,7,8,9] and the $m_{\mu ^+ \mu ^-}$ distribution in $B_s \rightarrow \phi \mu ^+ \mu ^-$ decay [10, 11]. These anomalies have reached a high level of statistical significance. There are ongoing discussions of the possible systematic effects and the uncertainties in the SM calculations, so the jury is still out on the significances of these flavour anomalies [12]. In the meantime, it is interesting to explore possible interpretations and look for other phenomenological signatures that might corroborate them [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].

In the wake of this recent report of $\mu$–e non-universality in $B \rightarrow K^{*} \ell ^+ \ell ^-$ decay [1, 2], several phenomenological analyses favour an anomalous non-SM contribution to the coefficient of the dimension-6 operator $O_9^\mu \equiv ({{\bar{s}}} \gamma _\mu P_L b)({{\bar{\mu }}} \gamma ^\mu \mu )$, but do not exclude a smaller non-SM contribution to the coefficient of $O_{10}^\mu \equiv ({{\bar{s}}} \gamma _\mu P_L b)({{\bar{\mu }}} \gamma ^\mu \gamma _5 \mu )$ or $O_{9 \prime }^\mu \equiv ({{\bar{s}}} \gamma _\mu P_R b)({{\bar{\mu }}} \gamma ^\mu \mu )$ [13,14,15,16,17,18,19, 27]. A popular interpretation of this anomaly is that it is due to the exchange of a U(1)$^\prime$ gauge boson $Z'$ with non-universal couplings to both quarks and leptons [15, 22,23,24, 27, 28, 31, 33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52]. It is possible to take a purely phenomenological attitude to this possibility, and not (yet) concern oneself about the theoretical consistency of such a $Z'$ model. However, any gauge theory should be free of triangle anomalies, which in general need to be cancelled by fermions with masses comparable to that of the $Z'$. Thus, not only does this requirement have the potential to constrain significantly the possible U(1)$^\prime$ couplings of both SM and non-SM particles, but it may also suggest novel signatures that could confirm or disprove such a $Z'$ interpretation of the LHCb $B \rightarrow K^{*} \ell ^+ \ell ^-$ measurements and other flavour anomalies.

We explored recently the impact of the anomaly-cancellation requirement on simplified dark matter (DM) models, assuming generation-independent U(1)$^\prime$ couplings to quarks and leptons [53]. Rather than take a top-down based on some postulated ultraviolet scenario, we proposed some minimal benchmark models with desirable characteristics such as suppressed leptonic couplings (so as to reduce the impact of unsuccessful LHC searches for massive $Z'$ bosons) or axial coupling to quarks (so as to reduce the impact of direct searches for DM scattering).

In this paper we follow an analogous strategy for flavourful $Z'$ models with generation-dependent U(1)$^\prime$ couplings to quarks and leptons, treating DM as a possible optional extra. Motivated by the long-standing discrepancy between experiment and the SM prediction for the anomalous magnetic moment of the muon, $g_\mu - 2$, we also consider models with additional leptons L that are vector-like under the SM gauge group but might have parity-violating U(1)$^\prime$ couplings [54].

Since the LHCb anomaly could be explained by just a left-handed flavour-changing quark coupling $\propto {{\bar{s}}} \gamma _\mu P_L b$, but not by a coupling $\propto {{\bar{s}}} \gamma _\mu P_R b$ alone, we mainly follow [54] in assuming that the right-handed charge -1/3 quarks have vanishing U(1)$^\prime$ charges.¹ However, we also discuss in the appendix a scenario for anomaly cancellation with non-vanishing charges to right-handed charge − 1/3 quarks. In view of the strong upper limits on anomalous flavour-changing interactions of strange quarks, we also assume [54] that the first two generations of left-handed down-type quarks $d_L, s_L$ have identical U(1)$^\prime$ charges, whereas the $b_L$ charge is different, opening the way to the suggested flavour-changing neutral interaction.² The reported $\mu$–e non-universality and LEP constraints lead us also to assume that the electron has vanishing U(1)$^\prime$ couplings [54], but we allow arbitrary U(1)$^\prime$ charges for the left- and right-handed $\mu$ and $\tau$, generalizing the anomaly-free models discussed previously that assumed a $L_\mu - L_\tau$ charge in the lepton sector [33,34,35,36]. The semileptonic B decay data suggest that the couplings of the $\mu$ to the $Z'$ are predominantly vector-like, corresponding to dominance by the $O_9^\mu$ operator over $O_{10}^\mu$, but we do not impose this restriction a priori. We discuss anomaly-free models with muon couplings that are not completely vector-like, but have combinations of $O_9^\mu$ and $O_{10}^\mu$ that are nevertheless consistent with a global fit the flavour anomalies, as seen in Figure 1.

In the absence of U(1)$^\prime$ charges for the right-handed charge 2/3 quarks, we show that the anomaly-cancellation conditions are so restrictive that there are no solutions with non-vanishing U(1)$^\prime$ charges for quarks. This is also the case if we include a single dark sector particle. However, we do find acceptable solutions if we allow for a second ‘dark’ fermion, as illustrated in Table 1.

There are also solutions with a single DM particle if we allow non-vanishing U(1)$^\prime$ charges for the $u_R, c_R$ (assumed to be equal) and $t_R$ (allowed be different). We have scanned for all possible triangle anomaly-free models with charges that can be expressed in the form p / q with $p, q \in [-\, 4, 4]$ when we normalise the left-handed DM charge $Y^{\prime }_{\chi _L} = 1$. Among these, four have vector-like $\mu$ couplings, three have no couplings to the first two generations of quarks, and three have axial couplings for the DM fermion (as required if it is a Majorana particle, and which would suppress DM scattering by a relative velocity factor). These models are all distinct, with the exception of a single model that combines a vector-like $Z'$ coupling to the muon with an axial coupling to the DM particle. We display in Table 2 the U(1)$'$ charges for this model, two other models with vector-like muon couplings and one with vanishing $Z'$ couplings for the first two generations, as benchmarks that illustrate the potential signatures of anomaly-free models of flavourful $Z'$ bosons with DM.

We also explore models in which anomalies are cancelled by extra vector-like leptons [54], exhibiting an example in which the LHC $Z' \rightarrow \mu ^+ \mu ^-$ signal is suppressed because of a small $Z' \rightarrow \mu ^+ \mu ^-$ branching ratio. Such a model may be able to explain the discrepancy between SM calculations and the experimental measurement of the anomalous magnetic moment of the muon, $g_\mu - 2$ [56].

The construction described in the appendix allows for a non-vanishing coupling to all types of quarks, including RH down-type quarks, and solves all the anomaly constraints without the need for exotic fermions. In this case the coefficient of $O^{\mu }_{10}$ vanishes, and there is an admixture of the $O^{\mu }_9$ and $O^{\mu }_{9 \prime }$ operators.

2 Scenarios for anomaly cancellation

The following are the six anomaly-cancellation conditions to be considered:

• (a) [SU(3)$_C^2]\times [{\text {U}}(1)^\prime$], which implies Tr[$\{\mathcal {T}^i,\mathcal {T}^j\}Y^\prime$] = 0,

• (b) [SU(2)$_W^2]\times [{\text {U}}(1)^\prime$], which implies Tr[$\{T^i,T^j\}Y^\prime$] = 0,

• (c) [U(1)$_Y^2]\times [{\text {U}}(1)^\prime$], which implies Tr[Y$^2 Y^\prime$] = 0,

• (d) [U(1)$_Y]\times [{\text {U}}(1)^{\prime ^2}$], which implies Tr[$Y Y^{\prime 2}$] = 0,

• (e) [U(1)$^{\prime 3}]$, which implies Tr[$Y^{\prime 3}$] = 0,

• (f) Gauge-gravity, which implies Tr[$Y^\prime$] = 0.

In general there can be independent U(1)$'$ charges for each of the $3 \times 5 = 15$ multiplets of the Standard Model,³ which we label $q_{L,i}$, $u_{R,i}$, $d_{R,i}$, $l_{L,i}$ and $e_{R,i}$ where i is a generation index, as well as charges for any extra particles beyond the SM. However, as mentioned in the Introduction, we make simplifying assumptions motivated by phenomenological considerations.

SM particles only, no $Z'$ couplings to electrons or right-handed quarks

Motivated by the indication that the flavour anomalies originate in the U(1)$^{\prime }$ couplings to left-handed charge − 1/3 quarks, initially we set the charges of all the right-handed quarks to zero: $Y'_{u_R,i} = 0 =Y'_{d_R,i}$, though this is not mandated by the data. Motivated by the strong upper limits on non-SM flavour-changing interactions between the first two generations of charge − 1/3 quarks, we assume that the left-handed doublets in the first two generations have identical charges $Y'_{q_L,1}=Y'_{q_L,2} \equiv Y'_q$ [54]. These differ from that of the third left-handed doublet $Y'_{q_L,3} \equiv Y'_t$, making possible the desired flavour-changing $\bar{b} \gamma _\mu P_L s$ coupling.

A complete discussion of the implications of constraints on flavour-changing couplings, e.g., from $\Delta F = 2$ processes, is beyond the scope of this work, since it would depend on the structures of the individual matrices that rotate the quark fields into the mass basis. Experimentally, only the combination entering into the CKM matrix is known, and the structures of the individual matrices depend on details of the Higgs representations and Yukawa coupling matrices that are independent of the anomaly cancellation conditions that we consider here.

In order to avoid the experimental constraints from LEP and other electroweak measurements [54], we also assume that the electron charges vanish: $Y'_{l_L,1}=0=Y'_{e_R,1}$. However, we allow independent left- and right-handed couplings for the muon and tau. With these assumptions, and in the absence of any particles beyond the SM, the anomaly-cancellation conditions become

$$\begin{aligned}&2 (2Y'_q + Y'_t) = 0 \end{aligned}$$

(2.1)

$$\begin{aligned}&Y'_{\mu _L} + Y'_{\tau _L} + 3 Y'_{t} + 6 Y'_q = 0 \end{aligned}$$

(2.2)

$$\begin{aligned}&\frac{2}{3} \left( 3 Y'_{\mu _L} - 6 Y'_{\mu _R} + 3 Y'_{\tau _L} - 6 Y'_{\tau _R} + Y'_{t} + 2 Y'_q \right) = 0 \end{aligned}$$

(2.3)

$$\begin{aligned}&2 \left( - Y_{\mu _L}^{\prime \; 2} + Y_{\mu _R}^{\prime \; 2} - Y_{\tau _L}^{\prime \; 2} + Y_{\tau _R}^{\prime \; 2}+ Y_t^{\prime \; 2} + 2 Y_{q}^{\prime \; 2} \right) = 0 , \end{aligned}$$

(2.4)

$$\begin{aligned}&-Y_{\mu _R}^{\prime \; 3} -Y_{\tau _R}^{\prime \; 3} + 2 \left( Y_{\mu _L}^{\prime \; 3} + Y_{\tau _L}^{\prime \; 3} \right) + 6 \left( Y_{t}^{\prime \; 3} + 2 Y_{q}^{\prime \; 3} \right) = 0, \end{aligned}$$

(2.5)

$$\begin{aligned}&-Y_{\mu _R}^{\prime } -Y_{\tau _R}^{\prime } + 2 \left( Y_{\mu _L}^{\prime } + Y_{\tau _L}^{\prime } \right) + 6 \left( Y_{t}^{\prime } + 2 Y_{q}^{\prime } \right) = 0. \end{aligned}$$

(2.6)

Solving the conditions (2.1), (2.2), (2.3) gives the relations

$$\begin{aligned} Y'_t&= -2 Y'_q , \end{aligned}$$

(2.7)

$$\begin{aligned} Y'_{\tau _L}&= - Y'_{\mu _L} , \end{aligned}$$

(2.8)

$$\begin{aligned} Y'_{\tau _R}&= - Y'_{\mu _R} . \end{aligned}$$

(2.9)

Using these relations to solve the conditions (2.4) then yields

$$\begin{aligned} 4 \left( -Y_{\mu _L}^{\prime \; 2} + Y_{\mu _R}^{\prime \; 2} + 3 Y_q^{\prime \; 2} \right)&=0 , \end{aligned}$$

(2.10)

which has rational solutions for any rational value of $Y'_q$, since any odd number can be written as the difference between two squares. Equation (2.10) implies that if $Y'_q \ne 0$ the muon could not have vector-like U(1)$'$ couplings as suggested by the data, but this is a moot point in this scenario, since (2.5) yields

$$\begin{aligned} - 36 Y_q^{\prime \; 3}&= 0 . \end{aligned}$$

(2.11)

Hence $Y'_q = 0$, which in turn implies via (2.7) that $Y'_t$ also vanishes and the $Z'$ decouples from quarks.⁴ Therefore, we must relax the assumptions made above if the flavour anomalies are to be explained by the exchange of a $Z'$ boson.

Including one or two ‘dark’ particles

Adding a single DM particle with vanishing SM couplings does not remedy the situation, as none of the conditions (2.7), (2.8), (2.9) are affected, and condition (2.6) implies $\mathrm {Tr}_{\mathrm {BSM}} [Y^{\prime }] = 0$. Hence, if there is a single DM particle it must have vector-like U(1)$'$ couplings, and would not change the fatal condition (2.11).

The next simplest possibility has two SM-singlet ‘dark’ fermions, A and B, in which case we have, in addition to (2.7, 2.8) and (2.9) the condition that

$$\begin{aligned} Y'_{A_L} = Y'_{A_R} -Y'_{B_L} + Y'_{B_R} \end{aligned}$$

(2.12)

and the remaining anomaly conditions to solve are (2.10) and

$$\begin{aligned} -36 Y^{\prime \; 3}_{q} + 3 (Y'_{B_L} - Y'_{A_R}) (Y'_{B_L} - Y'_{B_R}) (Y'_{A_R} + Y'_{B_R})&=0 , \end{aligned}$$

(2.13)

instead of (2.11) As in the single DM particle case, the condition (2.10) implies that the muon cannot have vector-like U(1)$'$ couplings. Normalizing $Y'_{A_R} = 1$ and restricting our attention to anomaly-free models with U(1)$'$ charges that can be expressed as p / q with $p, q \in [- \, 4, 4]$, we find several solutions with $Y'_{\mu _L}/Y'_{\mu _R} = 2$ .⁵ In these models the ratio of the vector-like and axial muon couplings $Y'_{\mu _V}/Y'_{\mu _A} = - \, 3$, which may be consistent with the ratio of $O_9^\mu$ and $O_{10}^\mu$ coefficients $C_9^\mu , C_{10}^\mu$ allowed by the analyses in [13,14,15,16,17,18,19, 27], as indicated in Fig. 1. We see that the green dot-dashed line corresponding to models with $Y'_{\mu _V}/Y'_{\mu _A} = - \,3$ traverses the region of the $(C_9^\mu , C_{10}^\mu )$ plane preferred in the analysis of [13] at the 1-$\sigma$ level. These models have the same U(1)$'$ charges for the SM particles but different values for Y$'_{A_R, B_{L,R}} = 0, \pm \, 1/3, \pm \, 4/3$. None of these solutions has a DM candidate with a purely axial U(1)$'$ coupling, though we cannot exclude the possibility that the SM-singlet fermions might mix in such a way that the lighter mass eigenstate does have an axial coupling. The U(1)$'$ charges of one representative model are shown in Table 1.

Fig. 1

Regions of the operator coefficients $(C_9^\mu , C_{10}^\mu )$ preferred in the global analysis of flavour anomalies in [13], which includes all relevant flavour observables including, e.g., the branching fraction for $B_s \rightarrow \mu \mu$. We superimpose the predictions of models with $Y'_{\mu _V}/Y'_{\mu _A} = - \, 3$ as in Table 1 (green dot-dashed line), $Y'_{\mu _V}/Y'_{\mu _A} = 7$ (purple dashed line), e.g., model (D) in Table 2, and $Y'_{\mu _V}/Y'_{\mu _A} = -1$ (blue dotted line) as in model (E) in Table 2. We note that $C_{10}^\mu = 0$ in models (A, B) and (C) in Table 2 and in the model in Table 3

Table 1 The U(1)$'$ charges in a benchmark model with two SM-singlet ‘dark’ particles A, B that have couplings only for left-handed quarks and a muon coupling that is dominantly vector-like: $Y'_{\mu _V}/Y'_{\mu _A} = -3$

The identification of the lighter SM-singlet fermion mass eigenstate depends on details of the mixing in the dark sector that we do not discuss here. Various experimental constraints should be considered for this fermion to be a realistic DM candidate: the correct thermal relic density should be obtained, the cross-sections for scattering on nuclei should be below the sensitivities of current direct detection experiments, and LHC and indirect detection bounds should be taken into account where appropriate. Anomaly cancellation constrains only the $Y'$ charges but not the overall magnitude g of the gauge coupling, which could be fixed by the requiring the observed abundance of dark matter. When combined with the $Y'$ charges and mixing patterns in specific models, predictions for the LHC and dark matter experiments could be made, but such a study lies beyond the scope of this work.

Including couplings for right-handed charge 2/3 quarks

As an alternative way to relax our initial assumptions, we allow next for non-vanishing U(1)$'$ charges for the right-handed charge 2/3 quarks $u_{R,i}$. We recall that the flavour anomalies apparently originate from left-handed b and s quarks, but there is no reason to forbid U(1)$'$ couplings to right-handed charge 2/3 quarks. In this Section we assume that the charges of the RH charge − 1/3 quarks vanish, but we relax this assumption in the appendix. We assume that the U(1)$'$ charges of the first two generations are identical, i.e., $Y'_{u_R,1} = Y'_{u_R,2}$, again with the motivation of suppressing flavour-changing neutral interactions.

In the absence of non-SM fermions, one can readily solve the anomaly conditions (a, b, c) and (f) above, and the conditions (d) and (e) then take the following forms:

$$\begin{aligned} Y^{\prime \; 2}_{l_L,2} - Y^{\prime \; 2}_{e_R,2} - 3 Y^{\prime \; 2}_{q_L,1} + 6 Y^{\prime \; 2}_{u_R,1}&= 0 , \end{aligned}$$

(2.14)

$$\begin{aligned} -2 Y^{\prime \; 3}_{q_L,1} + Y^{\prime \; 3}_{u_R,1}&= 0 . \end{aligned}$$

(2.15)

These conditions clearly have non-trivial solutions, but (2.15) does not admit rational values for both $Y^{\prime }_{q_L,1}$ and $Y^{\prime }_{u_R,1}$. All the unification scenarios known to us have rational values for U(1)$'$ charges, so these solutions are not acceptable.

Including couplings for right-handed charge 2/3 quarks and a DM particle

We are therefore led to consider adding a single DM fermion $\chi$ with charges $Y'_{\chi _L}$ and $Y'_{\chi _R}$. Normalizing $Y'_{\chi _L} = 1$, the anomaly conditions (a), (b), (c), (d) and (f) yield the following expressions for the other charges:

$$\begin{aligned} Y'_{t_L}&= -\frac{-2 Y'_{q_L} \big (-3 Y'_{\mu _L}+8 Y'_{q_R}+4 Y'_{\mu _R}\big )+(Y')_{q_L}^2+(Y')_{\mu _L}^2+6 (Y')_{q_R}^2-(Y')_{\mu _R}^2}{2 Y'_{q_L}+3 Y'_{\mu _L}-8 Y'_{q_R}-4 Y'_{\mu _R}} , \end{aligned}$$

(2.16)

$$\begin{aligned} Y'_{t_R}&= \frac{2 \big (Y'_{q_R} \big (4 Y'_{\mu _R}-3 Y'_{\mu _L}\big )-2 Y'_{q_L} Y'_{q_R}+3 (Y')_{q_L}^2-(Y')_{\mu _L}^2+2 (Y')_{q_R}^2+(Y')_{\mu _R}^2\big )}{2 Y'_{q_L}+3 Y'_{\mu _L}-8 Y'_{q_R}-4 Y'_{\mu _R}} , \end{aligned}$$

(2.17)

$$\begin{aligned} Y'_{\tau , L}&= \frac{8 Y'_{\mu _L} Y'_{q_R}-2 Y'_{q_L} Y'_{\mu _L}-9 \left( Y'\right) _{q_L}^2+Y'_{\mu _R} \left( 4 Y'_{\mu _L}-3 Y'_{\mu _R}\right) +18 \left( Y'\right) _{q_R}^2}{2 Y'_{q_L}+3 Y'_{\mu _L}-8 Y'_{q_R}-4 Y'_{\mu _R}} , \end{aligned}$$

(2.18)

$$\begin{aligned} Y'_{\tau , R}&= \frac{-2 Y'_{q_L} Y'_{\mu _R}-12 (Y')_{q_L}^2+Y'_{\mu _L} \big (4 Y'_{\mu _L}-3 Y'_{\mu _R}\big )+8 Y'_{q_R} Y'_{\mu _R}+24 (Y')_{q_R}^2}{2 Y'_{q_L}+3 Y'_{\mu _L}-8 Y'_{q_R}-4 Y'_{\mu _R}}, \end{aligned}$$

(2.19)

$$\begin{aligned} Y'_{\chi _R}&= \frac{-6 (Y')_{q_L}^2+2 Y'_{q_L}+2 (Y')_{\mu _L}^2+3 Y'_{\mu _L}+12 (Y')_{q_R}^2-8 Y'_{q_R}-2 (Y')_{\mu _R}^2-4 Y'_{\mu _R}}{2 Y'_{q_L}+3 Y'_{\mu _L}-8 Y'_{q_R}-4 Y'_{\mu _R}}, \end{aligned}$$

(2.20)

when $2 Y'_{q_L}+3 Y'_{\mu _L}-8 Y'_{q_R}-4 Y'_{\mu _R} \ne 0$.⁶ At this stage $Y'_{q_L}$, $Y'_{q_R}$, $Y'_{\mu _L}$ and $Y'_{\mu _R}$ are undetermined, but we have not yet applied the anomaly condition (e), which yields an additional constraint that is algebraically complicated and unrevealing. Scanning over the four undetermined charges, we find a set of solutions with $Y'_{q_L}=Y'_{q_R}=0$ (which would suppress $Z'$ production at the LHC and direct DM scattering) and either $Y'_{\mu _L}=Y'_{\mu _R}$ (as favoured by the data) or $Y'_{\mu _L}= -Y'_{\mu _R}$. However, these solutions also have vanishing couplings for the third-generation quarks, i.e., $Y'_{t_L} = Y'_{t_R} = 0$, so all the quark charges vanish.

We are therefore forced to make a ‘Solomonic choice’ between models with vector-like couplings to muons, i.e., $Y'_{\mu _L}=Y'_{\mu _R}$, and those with vanishing couplings to first- and second-generation quarks. Scanning over rational values of U(1)$'$ that can be expressed in the form $p/q: p, q \in [-4, 4]$, we find 4 models with vector-like muon couplings and 3 that have vanishing first- and second-generation quark couplings. One of the models with vector-like muon couplings also has a DM particle with a purely axial U(1)$'$ coupling that could be a Majorana particle. The U(1)$'$ charges of this model (A) are listed in Table 2, along with the corresponding charges for some other models that may serve as interesting benchmarks. The charges in the second and third rows are for models (B, C) with vector-like muon couplings but non-axial DM couplings, and the charges in the bottom two rows are for models (D, E) with vanishing first- and second-generation quark couplings and a mixture of vector and axial couplings to the muon.

Table 2 The U(1)$'$ charges in some benchmark models with couplings for right-handed quarks and a single dark matter particle that have interesting properties: (A) vector-like $\mu$ coupling and axial DM coupling, (B, C) vector-like $\mu$ coupling, (D) no first- and second-generation couplings and relatively small axial-vector $\mu$ coupling: $Y'_{\mu _V}/Y'_{\mu _A} = 7$, (E) no first- and second-generation couplings and $Y'_{\mu _V}/Y'_{\mu _A} = -1$

In models such as (D, E), the $Z'$ production mechanisms via first- and second-generation ${{\bar{q}}} q$ annihilations that are usually dominant at the LHC are suppressed, and the constraint on the $Z'$ mass coming from production via ${{\bar{b}}} s + {{\bar{s}}} b$ collisions is much weaker [24]. Moreover, the constraint from searches for direct DM scattering on nuclei is greatly weakened.

Although the $Z'$ coupling to muons is not purely vectorial in model (D), the ratio of the vector and axial muon couplings is 7 in this model, so the axial coupling might be acceptably small. As seen in Fig. 1, the data allow a non-vanishing axial/vector ratio, although they prefer the opposite relative sign. Models with $Y'_{\mu _V}/Y'_{\mu _A} = 7$ (dashed purple line) are compatible with the region of the $(C_9^\mu , C_{10}^\mu )$ preferred in the analysis of [13] at the 2-$\sigma$ level.

Model (E) shares the property of having no coupling to the first two generations of quarks but has a different mixture of axial/vector coupling, $Y'_{\mu _V}/Y'_{\mu _A} = - \, 1$, since it has a purely left-handed muon coupling. This is compatible with the fit shown in Fig. 1 at the $3 \, \sigma$ level.

We also found a model (not shown) with vanishing first- and second-generation quark couplings, but with a muon coupling that is either purely right-handed, which is disfavoured by the data.⁷

Including a vector-like lepton

Finally, we consider a scenario proposed in [54] in which the SM particles are not supplemented by DM, but by extra leptons, a vector-like doublet $(\nu ', \ell ')$ and a vector-like singlet $\mu '$. We assume that only the left-handed quarks have non-zero U(1)$'$ charges, with those for the first two generations being the same. We also assume that the $Z'$ coupling of the muon is purely vectorial. The left-handed components of the doublet and the right-handed component of the singlet are assumed to have identical values of $Y'$, but the U(1)$'$ charges of the right-handed doublet and left-handed singlet are free a priori. Thus the free parameters of the model are $Y'_{q_L}, Y'_{t_L}, Y'_{\mu _L} = Y'_{\mu _R}, Y'_{\tau _{L,R}}, Y'_{\ell '_L} = Y'_{\mu '_R}, Y'_{\ell '_R}$ and $Y'_{\mu '_L}$.

Table 3 The U(1)$'$ charges in a model with extra vector-like leptons $\ell ', \mu '$ and a vector-like $Z'$ muon coupling in which the branching ratio for $Z' \rightarrow \mu ^+ \mu ^-$ is suppressed

We have also scanned rational values of these free parameters that can be expressed in the form $p/q: p, q \in [- \, 4, 4]$. Since one of the objectives of [54] is to explain the discrepancy between SM calculations of $g_\mu -2$ and the experimental measurement [56], via a contribution $\propto 1/M_{Z^{'2}}$, it is desirable to focus on solutions in which the LHC $Z' \rightarrow \mu ^+ \mu ^-$ signal is suppressed. Since the U(1)$'$ charges of the first- and second-generation quarks are non-vanishing, the only way to suppress the prospective LHC signal is to suppress the $Z' \rightarrow \mu ^+ \mu ^-$ branching ratio. We have found several models in which the combined branching ratios for other decays exceed that for $Z' \rightarrow \mu ^+ \mu ^-$ by more than an order of magnitude. Table 3 displays the model in which the branching ratio for $Z' \rightarrow \mu ^+ \mu ^-$ is most suppressed by the U(1)$'$ charges and multiplicities of states, namely to 3/130, assuming that the masses of the extra leptons can be neglected, as is the case if all the fermions are much lighter than $M_{Z^\prime }/2$. Since $M_{Z'}$ may be in the TeV range, this is compatible with the lower limits on the masses of vector-like leptons given by the Particle Data Group [57], which are $\sim 100$ GeV, and with model-dependent recasts of LHC searches [58], which yield limits $\sim$ a few hundred GeV.

3 Summary and conclusions

We have explored in this paper the constraints on $Z'$ interpretations of the flavour anomalies in $B \rightarrow K^{(*)} \ell ^+ \ell ^-$ decays imposed by the cancellation of triangle anomalies, namely the conditions (a) to (f) stated at the beginning of Section 2. We find many models that have not been discussed previously in the literature, and have novel experimental signatures involving new particles and/or non-trivial combinations of the operators $O_9^\mu$ and $O_{10}^\mu$ that are consistent with the reported flavour anomalies.

Motivated by the observed pattern of flavour anomalies, we considered initially models in which the $Z'$ has quark couplings that are purely left-handed (universal for the first two generations, non-universal for the third), and it has no electron coupling. In this case we find no non-trivial solution of the anomaly-cancellation conditions in the absence of non-SM particles, and so are led to introduce ‘dark’ fermions without SM couplings. In the case of a single DM particle, there is again no non-trivial solution, but we do find solutions with 2 ‘dark’ fermions. In none of these does the $Z'$ have a purely vector-like muon coupling, but we find a class of solutions in which $Y'_{\mu _V}/Y'_{\mu _A} = -3$, a ratio that is compatible with the data at the 1-$\sigma$ level as seen in Fig. 1. Examples of these solutions are shown in Table 1.

We then considered models in which the $Z'$ couples to right-handed charge 2/3 quarks, a possibility that is allowed by the data. In the absence of a dark sector we find no solution of the anomaly-cancellation conditions with rational charges, but we do find a number of interesting solutions in the presence of a DM fermion, and we show some examples in Table 2. Some of these have vector-like muon couplings—models (A), (B) and (C)—and in one of these the DM particle has a purely axial $Z'$ coupling—model (A). In models (D) and (E) there are no $Z'$ couplings to first- and second-generation quarks, so production at the LHC is suppressed and the experimental constraints on the dark mass scale are correspondingly reduced. Model (D) is one of a class of models in which $Y'_{\mu _V}/Y'_{\mu _A} = 7$, a ratio that appears compatible with the data at the 2-$\sigma$ level, as also seen in Fig. 1. Model (E) has a purely left-handed muon coupling and so predicts instead $Y'_{\mu _V}/Y'_{\mu _A} = -1$, which is compatible with the data at the 3-$\sigma$ level. We have also considered models in which the triangle anomalies are cancelled by vector-like leptons, exhibiting in Table 3 a model with a vector-like $Z'$ muon coupling in which the branching ratio for $Z' \rightarrow \mu ^+ \mu ^-$ is maximally suppressed.

These examples illustrate that anomaly cancellation is a powerful requirement that could have interesting phenomenological consequences linking flavour anomalies to other observables. Anomaly cancellation requires some extension of the SM spectrum to include, e.g., a dark sector or a vector-like lepton. Moreover, either the dark sector should more than just a single DM particle, or some quarks should have right-handed couplings to the $Z'$ boson. Additionally, we find several classes of models in which $Y'_{\mu _V}/Y'_{\mu _A} \ne 0$ in a way that is compatible with the present data but could be explored in the future. Finally, we have shown that it is possible to cancel the triangle anomalies using vector-like leptons in such a way as to suppress the LHC $Z' \rightarrow \mu ^+ \mu ^-$ signal, potentially facilitating an explanation of the anomaly in $g_\mu - 2$.

Appendix: Non-vanishing couplings to right-handed down-type quarks

For completeness, we investigate in this appendix the effect of anomaly cancellation on models with couplings to all quark fields, including charges for the right-handed down-type quarks. As before, we fix the charges of the first two quark generations to be equal. However, in order to restrict the number of unknowns we take a purely vectorial coupling, $Y_{\mu ,L }'=Y_{\mu ,R }'$, for which $C^{\mu }_{10} = 0$. In this case the anomaly cancellation conditions read

$$\begin{aligned} 0= & {} 2 Y_3'-Y_b'-2 Y_d'+4 Y_q'-Y_t'-2 Y_u' \end{aligned}$$

(3.1a)

$$\begin{aligned} 0= & {} 3 Y_3'+6 Y_q'+Y_{\mu }'+Y_{\tau ,L}' \end{aligned}$$

(3.1b)

$$\begin{aligned} 0= & {} \frac{2}{3} \left( Y_3'-2 Y_b'-4 Y_d'\right. \nonumber \\&\left. +2 Y_q'-8 Y_t'-16 Y_u'\right. \nonumber \\&\left. -3 Y_{\mu }'+3 Y_{\tau ,L}'-6 Y_{\tau ,R}'\right) \end{aligned}$$

(3.1c)

$$\begin{aligned} 0= & {} \left( Y_3'\right) {}^2+\left( Y_b'\right) {}^2\nonumber \\&+2 \left( Y_d'\right) {}^2+2 \left( Y_q'\right) {}^2 - 2 \left( Y_t'\right) {}^2\nonumber \\&-4 \left( Y_u'\right) {}^2 -\left( Y_{\tau ,L}'\right) {}^2+\left( Y_{\tau ,R}'\right) {}^2 \end{aligned}$$

(3.1d)

$$\begin{aligned} 0= & {} 6 \left( Y_3'\right) {}^3-3 \left( Y_b'\right) {}^3-6 \left( Y_d'\right) {}^3\nonumber \\&+12 \left( Y_q'\right) {}^3-3 \left( Y_t'\right) {}^3-6 \left( Y_u'\right) {}^3+\left( Y_{\mu }'\right) {}^3\nonumber \\&+2 \left( Y_{\tau ,L}'\right) {}^3-\left( Y_{\tau ,R}'\right) {}^3 \end{aligned}$$

(3.1e)

$$\begin{aligned} 0= & {} 6 Y_3'-3 Y_b'-6 Y_d'+12 Y_q'-3 Y_t'\nonumber \\&-6 Y_u'+Y_{\mu }'+2 Y_{\tau ,L}'-Y_{\tau ,R}' \end{aligned}$$

(3.1f)

with $Y_3'= Y_{q,L,3}'$, $Y_t' = Y'_{u,R,3}$, $Y_q' = Y_{q,L,1/2}'$ and $Y_u' = Y'_{u,R,1/2}$, etc. Also, we have $Y_{\mu }' = Y_{\mu ,L }'=Y_{\mu ,R }'$.

Tthe anomaly cancellation for all these charges can be solved completely algebraically. We find the following constraints for the unconstrained charges $Y'_q$, $Y'_u$ and $Y'_d$:

$$\begin{aligned} Y'_3= & {} -\frac{2 \left( Y_d'\right) {}^2 Y_q'+\left( Y_d'\right) {}^3-4 Y_q' \left( Y_u'\right) {}^2+\left( Y_u'\right) {}^3}{\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2} \end{aligned}$$

(3.2a)

$$\begin{aligned} Y'_{\tau , L}= & {} \frac{1}{6} \left( \frac{9 Y_u' \left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) +18 \left( \left( Y_d'\right) {}^3-2 \left( Y_q'\right) {}^3\right) }{\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2}\right. \nonumber \\&\left. +\frac{24 \left( Y_u'\right) {}^2 \left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) +24 Y_u' \left( \left( Y_d'\right) {}^3-2 \left( Y_q'\right) {}^3\right) -6 \left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) {}^2}{\left( Y_d'\right) {}^3-2 \left( Y_q'\right) {}^3+\left( Y_u'\right) {}^3}+8 Y_d'-4 Y_q'-Y_u'\right) \end{aligned}$$

(3.2b)

$$\begin{aligned} Y'_{\tau , R}= & {} \frac{3 Y_u' \left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) +6 \left( \left( Y_d'\right) {}^3-2 \left( Y_q'\right) {}^3\right) }{\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2}\nonumber \\&+\frac{4 \left( Y_u'\right) {}^2 \left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) +4 Y_u' \left( \left( Y_d'\right) {}^3-2 \left( Y_q'\right) {}^3\right) -\left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) {}^2}{\left( Y_d'\right) {}^3-2 \left( Y_q'\right) {}^3+\left( Y_u'\right) {}^3} +\frac{2}{3} \left( 2 Y_d'-Y_q'\right) -\frac{5 Y_u'}{3} \end{aligned}$$

(3.2c)

$$\begin{aligned} Y'_{t}= & {} \frac{-2 Y_u' \left( \left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2\right) -4 \left( Y_d'\right) {}^3+8 \left( Y_q'\right) {}^3}{\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2} \end{aligned}$$

(3.2d)

$$\begin{aligned} Y'_{b}= & {} \frac{-2 Y_d' \left( \left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2\right) -4 \left( Y_q'\right) {}^3+2 \left( Y_u'\right) {}^3}{\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2} \end{aligned}$$

(3.2e)

$$\begin{aligned} Y'_{\mu }= & {} \frac{-2 Y_d' \left( \left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2\right) -4 \left( Y_q'\right) {}^3+2 \left( Y_u'\right) {}^3}{\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2} \end{aligned}$$

(3.2f)

This construction is always possible as long as $\left( Y_d'\right) {}^2+\left( Y_q'\right) {}^2-2 \left( Y_u'\right) {}^2 \ne 0$ and $\left( Y_d'\right) {}^2-2 \left( Y_q'\right) {}^2 + \left( Y_u'\right) {}^2 \ne 0$.

This class of models features couplings to the first generation of quarks. However, the $\Delta M_s$ bound in models with a right-handed quark coupling is different from that in the models discussed previously, since it is possible to obtain a negative contribution to $\Delta M_s$. Such a negative contribution can bring $\Delta M_s$ closer to the experimental value than the current SM prediction, as noted in [59], but we leave an exploration of this issue to a future study.