Search for lepton-flavour-violating decays of the Higgs and Z bosons with the ATLAS detector

Abstract

Direct searches for lepton flavour violation in decays of the Higgs and Z bosons with the ATLAS detector at the LHC are presented. The following three decays are considered: $H\rightarrow e\tau$, $H\rightarrow \mu \tau$, and $Z\rightarrow \mu \tau$. The searches are based on the data sample of proton–proton collisions collected by the ATLAS detector corresponding to an integrated luminosity of 20.3 $\mathrm{fb}^{-1}$ at a centre-of-mass energy of $\sqrt{s}=8$ TeV. No significant excess is observed, and upper limits on the lepton-flavour-violating branching ratios are set at the 95$\%$ confidence level: Br$(H\rightarrow e\tau )<1.04\%$, Br$(H\rightarrow \mu \tau )<1.43\%$, and Br$(Z\rightarrow \mu \tau )<1.69\times 10^{-5}$.

1 Introduction

One of the main goals of the Large Hadron Collider (LHC) physics programme at CERN is to discover physics beyond the Standard Model (SM). A possible sign would be the observation of lepton flavour violation (LFV) that could be realised in decays of the Higgs boson or of the Z boson to pairs of leptons with different flavours.

Lepton-flavour-violating decays of the Higgs boson can occur naturally in models with more than one Higgs doublet [1,2,3,4], composite Higgs models [5, 6], models with flavour symmetries [7], Randall–Sundrum models [8] and many others [9,10,11,12,13,14,15,16]. LFV Z boson decays are predicted in models with heavy neutrinos [17], extended gauge models [18] and supersymmetry [19].

The most stringent bounds on the LFV decays of the Higgs and Z bosons other than $H\rightarrow {}\mu {}e$ are derived from direct searches [20]. The CMS Collaboration has performed the first direct search for LFV $H\rightarrow \mu \tau$ decays [21] and reported a small excess (2.4 standard deviations) of data over the predicted background. Their results give a 1.51$\%$ upper limit on Br($H\rightarrow \mu \tau$) at the 95$\%$ confidence level (CL). The ATLAS Collaboration has also performed a search [22] for the LFV $H\rightarrow \mu \tau$ decays in the final state with one muon and one hadronically decaying $\tau$-lepton, $\tau _{\mathrm {had}} {}$, and reported a 1.85$\%$ upper limit on Br($H\rightarrow \mu \tau$) at the 95$\%$ CL. The most stringent indirect constraint on $H\rightarrow e\mu$ decays is derived from the results of searches for $\mu \rightarrow e\gamma$ decays [23], and a bound of Br($H\rightarrow e\mu$) < O($10^{-8}$) is obtained [24, 25]. The bound on $\mu \rightarrow e\gamma$ decays suggests that the presence of a $H\rightarrow \mu \tau$ signal would exclude the presence of a $H\rightarrow e\tau$ signal, and vice versa, at an experimentally observable level at the LHC [25]. It is also important to note that a relatively large Br($H\rightarrow \mu \tau$) can be achieved without any particular tuning of the effective couplings, while a large Br($H\rightarrow e\tau$) is possible only at the cost of some fine-tuning of the corresponding couplings [25]. Upper bounds on the LFV $Z\rightarrow e\mu$, $Z\rightarrow \mu \tau$ and $Z\rightarrow e\tau$ decays were set by the LEP experiments [26, 27]: Br$(Z\rightarrow e\mu )<1.7\times 10^{-6}$, Br$(Z\rightarrow e\tau )<9.8\times 10^{-6}$, and Br$(Z\rightarrow \mu \tau )<1.2\times 10^{-5}$ at the 95$\%$ CL. The ATLAS experiment set the most stringent upper bound on the LFV $Z\rightarrow e\mu$ decays [28]: Br$(Z\rightarrow e\mu )<7.5 \times 10^{-7}$ at 95$\%$ CL.

This paper describes three new searches for LFV decays of the Higgs and Z bosons. The first study is a search for $H\rightarrow e\tau$ decays in the final state with one electron and one hadronically decaying $\tau$-lepton, $\tau _{\mathrm {had}}$. The second analysis is a simultaneous search for the LFV $H\rightarrow e\tau$ and $H\rightarrow \mu \tau$ decays in the final state with a leptonically decaying $\tau$-lepton, $\tau _{\mathrm {lep}} {}$. A combination of results of the earlier ATLAS search for the LFV $H\rightarrow \mu \tau _{\mathrm {had}}$ decays [22] and the two searches described in this paper is also presented. The third study constitutes the first ATLAS search for LFV decays of the Z boson with hadronic $\tau$-lepton decays in the channel $Z\rightarrow \mu {} \tau _{\mathrm {had}} {}$. The search for LFV decays in the $\tau _{\mathrm {lep}} {}$ analysis is based on the novel method introduced in Ref. [29]; the searches in the $\tau _{\mathrm {had}} {}$ analyses are based on the techniques developed for the SM $H\rightarrow \tau _{\mathrm {lep}} {} \tau _{\mathrm {had}} {}$ search. All three searches are based on the data sample of pp collisions collected at a centre-of-mass energy of $\sqrt{s}=8$ TeV and corresponding to an integrated luminosity of 20.3 $\mathrm{fb}^{-1}$. Given the overlap between the analysis techniques used in the $H\rightarrow {}e \tau _{\mathrm {had}}$ search and in the $Z\rightarrow \mu {} \tau _{\mathrm {had}} {}$ search, from here on they are referred to as the $\tau _{\mathrm {had}}$ channels; the $H\rightarrow {}\ell \tau _{\mathrm {lep}}$ search is referred to as the $\tau _{\mathrm {lep}}$ channel, where $\ell {}=e, \mu {}$.

2 The ATLAS detector and object reconstruction

The ATLAS detector¹ is described in detail in Ref. [30]. ATLAS consists of an inner tracking detector (ID) covering the range $|\eta |<2.5$, surrounded by a superconducting solenoid providing a 2 T axial magnetic field, a high-granularity electromagnetic ($|\eta |<3.2$) calorimeter, a hadronic calorimeter ($|\eta |<4.9$), and a muon spectrometer (MS) ($|\eta |<2.7$) with a toroidal magnetic field.

The signatures of LFV searches reported here are characterised by the presence of an energetic lepton originating directly from the boson decay and carrying roughly half of its energy, and the hadronic or leptonic decay products of a $\tau$-lepton. The data in the $\tau _{\mathrm {had}}$ channels were collected with single-lepton triggers: a single-muon trigger with the threshold of $p_{\text {T}} =24$ GeV and a single-electron trigger with the threshold $E_{\text {T}} =24$ GeV. The data in the $\tau _{\mathrm {lep}}$ channel were collected using asymmetric electron-muon triggers with $(p_{\text {T}} ^{\mu },E_{\text {T}} ^{e})>(18, 8)$ GeV and $(E_{\text {T}} ^{e},p_{\text {T}} ^{\mu })>(14,8)$ GeV thresholds. The $p_{\text {T}}$ and $E_{\text {T}}$ requirements on the objects in the presented analyses are at least 2 GeV higher than the trigger requirements.

A brief description of the object definitions is provided below. The primary vertex is chosen as the proton–proton collision vertex candidate with the highest sum of the squared transverse momenta of all associated tracks [31].

Muon candidates are reconstructed using an algorithm that combines information from the ID and the MS [32]. Muon quality criteria such as inner-detector hit requirements are applied to achieve a precise measurement of the muon momentum and to reduce the misidentification rate. Muons are required to have $p_{\text {T}} >10$ GeV and to be within $|\eta |<2.5$. The distance between the z-position of the point of closest approach of the muon inner-detector track to the beam-line and the z-coordinate of the primary vertex is required to be less than 1 cm. In the $\tau _{\mathrm {lep}}$ channel, there is an additional cut on the transverse impact parameter significance, defined as the transverse impact parameter divided by its uncertainty: $|d_0|/\sigma _{d_0}<3$. These requirements reduce the contamination due to cosmic-ray muons and beam-induced backgrounds. Typical reconstruction and identification efficiencies for muons meeting these selection criteria are above 95% [32].

Electron candidates are reconstructed from energy clusters in the electromagnetic calorimeters matched to tracks in the ID. They are required to have transverse energy $E_{\text {T}} {}>15(12)~\mathrm{GeV}{}$ in the $\tau _{\mathrm {had}}$ ($\tau _{\mathrm {lep}}$) channel, to be within the pseudorapidity range $|\eta |<2.47$, and to satisfy the medium shower shape and track selection criteria defined in Ref. [33]. Candidates found in the transition region between the barrel and end-cap calorimeters ($1.37<|\eta |<1.52$) are not considered in the $\tau _{\mathrm {had}}$ channel. Typical reconstruction and identification efficiencies for electrons satisfying these selection criteria range between 80 and 90%, depending on $E_{\text {T}}$ and $\eta$.

Exactly one lepton (electron or muon) satisfying the above identification requirements is allowed in the $\tau _{\mathrm {had}}$ channels. In the $\tau _{\mathrm {lep}}$ channel, only events with exactly one identified muon and one identified electron are retained. All lepton (electron or muon) candidates must be matched to the corresponding trigger objects and satisfy additional isolation criteria, based on tracking and calorimeter information, in order to suppress the background from misidentified jets or from semileptonic decays of charm and bottom hadrons. The calorimeter isolation variable $I(E_{\text {T}},\Delta R)$ is defined as the sum of the total transverse energy in the calorimeter in a cone of size $\Delta {}R$ around the electron cluster or the muon track, divided by the $E_{\text {T}}$ of the electron cluster or the $p_{\text {T}}$ of the muon, respectively. The track-based isolation $I(p_{\text {T}},\Delta R)$ is defined as the scalar sum of the transverse momenta of tracks within a cone of size $\Delta R$ around the electron or muon track, divided by the $E_{\text {T}}$ of the electron cluster or the muon $p_{\text {T}}$, respectively. The contribution due to the lepton itself is not included in either sum. The isolation requirements used in the $\tau _{\mathrm {had}}$ and $\tau _{\mathrm {lep}}$ channels, optimised to reduce the contamination from non-prompt leptons, are listed in Table 1.

Table 1 Summary of isolation requirements applied for the selection of isolated electrons and muons. The isolation variables are defined in the text

Hadronically decaying $\tau$-leptons are identified by means of a multivariate analysis technique [34] based on boosted decision trees, which exploits information about ID tracks and clusters in the electromagnetic and hadronic calorimeters. The $\tau _{\mathrm {had}}$ candidates are required to have $+1$ or $-1$ net charge in units of electron charge, and must be 1- or 3-track (1- or 3-prong) candidates. Events with exactly one $\tau _{\mathrm {had}}$ candidate satisfying the medium identification criteria [34] with $p_{\text {T}} >20$ GeV and $|\eta |<2.47$ are considered in the $\tau _{\mathrm {had}}$ channels. In the $\tau _{\mathrm {lep}}$ channel, events with identified $\tau _{\mathrm {had}}$ candidates are rejected to avoid overlap between $H\rightarrow {}\ell \tau _{\mathrm {had}}$ and $H\rightarrow {}\ell \tau _{\mathrm {lep}}$. The identification efficiency for $\tau _{\mathrm {had}}$ candidates satisfying these requirements is (55–60)%. Dedicated criteria [34] to separate $\tau _{\mathrm {had}}$ candidates from misidentified electrons are also applied, with a selection efficiency for true $\tau _{\mathrm {had}}$ decays (that pass the $\tau _{\mathrm {had}}$ identification requirements described above) of 95%. To reduce the contamination due to backgrounds where a muon mimics a $\tau _{\mathrm {had}}$ signature, events in which an identified muon with $p_{\text {T}} (\mu )>4$ GeV overlaps with an identified $\tau _{\mathrm {had}}$ are rejected [35]. The probability to misidentify a jet with $p_{\text {T}} >20$ GeV as a $\tau _{\mathrm {had}}$ candidate is typically (1–2)% [34].

Jets are reconstructed using the anti-$k_{t}$ jet clustering algorithm [36] with a radius parameter $R=0.4$, taking the deposited energy in clusters of calorimeter cells as inputs. Fully calibrated jets [37] are required to be reconstructed in the range $|\eta |<4.5$ and to have $p_{\text {T}} >30$ GeV. To suppress jets from multiple proton–proton collisions in the same or nearby beam bunch crossings, tracking information is used for central jets with $|\eta |<2.4$ and $p_{\text {T}} <50$ GeV. In the $\tau _{\mathrm {lep}}$ channel, these central jets are required to have at least one track originating from the primary vertex. In the $\tau _{\mathrm {had}}$ channel, tracks originating from the primary vertex must contribute more than half of the jet $p_{\text {T}}$ when summing the scalar $p_{\text {T}}$ of all tracks in the jet; jets with no associated tracks are retained.

In the pseudorapidity range $|\eta |<2.5$, jets containing b-hadrons (b-jets) are selected using a tagging algorithm [38]. These jets are required to have $p_{\text {T}} {}>30~\mathrm{GeV}{}$ in the $\tau _{\mathrm {had}}$ channel, and $p_{\text {T}} {}>20~\mathrm{GeV}{}$ in the $\tau _{\mathrm {lep}}$ channel. Two different working points with $\sim$70 and $\sim$80$\%$ b-tagging efficiencies for b-jets in simulated $t\bar{t}$ events are used in the $\tau _{\mathrm {had}}$ and $\tau _{\mathrm {lep}}$ channels, respectively. The corresponding light-flavour jet misidentification probability is (0.1–1)%, depending on the $p_{\text {T}}$ and $\eta$ of the jet. Only a very small fraction of signal events have b-jets, therefore events with identified b-jets are vetoed in the selection of signal events.

Some objects might be reconstructed as more than one candidate. Overlapping candidates, separeted by $\Delta R < 0.2$, are resolved by discarding one object and selecting the other one in the following order of priority (from highest to lowest): muons, electrons, $\tau _{\mathrm {had}}$, and jet candidates [35].

The missing transverse momentum (with magnitude $E_{\text {T}}^{\text {miss}}$) is reconstructed using the energy deposits in calorimeter cells calibrated according to the reconstructed physics objects (e, $\gamma$, $\tau _{\mathrm {had}}$, jets and $\mu$) with which they are associated [39]. In the $\tau _{\mathrm {had}}$ channels, the energy from calorimeter cells not associated with any physics object is included in the $E_{\text {T}}^{\text {miss}}$ calculation. It is scaled by the scalar sum of $p_{\text {T}}$ of tracks which originate from the primary vertex but are not associated with any objects divided by the scalar sum of $p_{\text {T}}$ of all tracks in the event which are not associated with objects. The scaling procedure achieves a more accurate reconstruction of $E_{\text {T}}^{\text {miss}}$ under high pile-up conditions.

3 Signal and background samples

The LFV signal is estimated from simulation. The major Higgs boson production processes (gluon fusion ggH, vector-boson fusion VBF, and associated production WH / ZH) are considered in the reported searches for LFV $H\rightarrow e\tau$ and $H\rightarrow \mu \tau$ decays. In the $\tau _{\mathrm {lep}}$ channel, all backgrounds are estimated from data. In the $\tau _{\mathrm {had}}$ channels, the $Z/\gamma ^{*}\rightarrow \tau \tau$ and multi-jet backgrounds are estimated from data, while the other remaining backgrounds are estimated from simulation, as described below.

The largely irreducible $Z/\gamma ^{*}\rightarrow \tau \tau$ background is modelled by $Z/\gamma ^{*}\rightarrow \mu \mu$ data events, where the muon tracks and associated energy deposits in the calorimeters are replaced by the corresponding simulated signatures of the final-state particles of the $\tau$-lepton decay. In this approach, essential features such as the modelling of the kinematics of the produced boson, the modelling of the hadronic activity of the event (jets and underlying event) as well as contributions from pile-up are taken from data. Therefore, the dependence on the simulation is minimised and only the $\tau$-lepton decays and the detector response to the $\tau$-lepton decay products are based on simulation. This hybrid sample is referred to as embedded data in the following. A detailed description of the embedding procedure can be found in Ref. [40].

The W+jets, $Z/\gamma ^{*}\rightarrow \mu \mu$ and $Z/\gamma ^{*}\rightarrow ee$ backgrounds are modelled by the ALPGEN [41] event generator interfaced with PYTHIA8 [42] to provide the parton showering, hadronisation and the modelling of the underlying event. The backgrounds with top quarks are modelled by the POWHEG [43,44,45] (for $t\bar{t}$, Wt and s-channel single-top production) and AcerMC [46] (t-channel single-top production) event generators interfaced with PYTHIA8. The ALPGEN event generator interfaced with HERWIG [47] is used to model the WW process, and HERWIG is used for the ZZ and WZ processes.

The events with Higgs bosons produced via ggH or VBF processes are generated at next-to-leading-order (NLO) accuracy in QCD with the POWHEG [48] event generator interfaced with PYTHIA8 to provide the parton showering, hadronisation and the modelling of the underlying event. The associated production (ZH and WH) samples are simulated using PYTHIA8. All events with Higgs bosons are produced with a mass of $m_{H}=125$ GeV assuming the narrow width approximation and normalised to cross sections calculated at next-to-next-to-leading order (NNLO) in QCD [49,50,51]. The SM $H\rightarrow \tau \tau$ decays are simulated by PYTHIA8; the other SM decays of the Higgs boson are negligible. The LFV Higgs boson decays are modelled by the EvtGen [52] event generator according to the phase-space model. In the $H\rightarrow \mu \tau$ and $H\rightarrow e\tau$ decays, the $\tau$-lepton decays are treated as unpolarised because the left- and right-handed $\tau$-lepton polarisation states are produced at equal rates. Finally, the LFV Z boson decays are simulated with PYTHIA8 assuming an isotropic decay. The width of the Z boson is set to its measured value [20].

For all simulated samples, the decays of $\tau$-leptons are modelled with TAUOLA [53] and the propagation of particles through the ATLAS detector is simulated with GEANT4 [54, 55]. The effect of multiple proton–proton collisions in the same or nearby beam bunch crossings is accounted for by overlaying additional minimum-bias events. Simulated events are weighted so that the distribution of the average number of interactions per bunch crossing matches that observed in data.

Background contributions due to non-prompt leptons in the $\tau _{\mathrm {lep}}$ channel and multi-jet events in the $\tau _{\mathrm {had}}$ channel are estimated using data-driven techniques described in Sects. 4.2 and 5.2.

4 Search for $H\rightarrow e\tau$ decays in the $\tau _{\mathrm {had}}$ channel

The search for the LFV $H\rightarrow e\tau$ decays in the $\tau _{\mathrm {had}}$ channel follows exactly the same analysis strategy and utilises the same background estimation techniques as those used in the ATLAS search for the LFV $H\rightarrow \mu \tau$ decays in the $\tau _{\mathrm {had}}$ channel [22]. The only major difference is that a high-$E_{\text {T}}$ electron is required in the final state instead of a muon. A detailed description of the $H\rightarrow e \tau _{\mathrm {had}}$ analysis is provided in the following sections.

4.1 Event selection and categorisation

Signal $H\rightarrow e\tau$ events in the $e \tau _{\mathrm {had}}$ final state are characterised by the presence of exactly one energetic electron and one $\tau _{\mathrm {had}}$ of opposite-sign (OS) charge as well as moderate $E_{\text {T}}^{\text {miss}}$, which tends to be aligned with the $\tau _{\mathrm {had}}$ direction. Same-sign (SS) charge events are used to control the rates of background contributions. Events with identified muons are rejected. Backgrounds for this signature can be broadly classified into two major categories:

• Events with true electron and $\tau _{\mathrm {had}}$ signatures. These are dominated by the irreducible $Z/\gamma ^{*}\rightarrow \tau \tau$ production with some contributions from the $VV\rightarrow e\tau +X$ (where $V=W,Z$), $t\bar{t}$, single-top and SM $H\rightarrow \tau \tau$ production processes. These events exhibit a very strong charge anti-correlation between the electron and the $\tau _{\mathrm {had}}$. Therefore, the expected number of OS events ($N_{\mathrm {OS}}$) is much larger than the number of SS events ($N_{\mathrm {SS}}$).

• Events with a misidentified $\tau _{\mathrm {had}}$ signature. These are dominated by W+jets events with some contribution from multi-jet (many of which have genuine electrons from semileptonic decays of heavy-flavour hadrons), diboson (VV), $t\bar{t}$ and single-top events with $N_{\mathrm {OS}}>N_{\mathrm {SS}}$. Additional contributions to this category arise from $Z(\rightarrow ee)$+jets events, where a $\tau _{\mathrm {had}}$ signature can be mimicked by either a jet (no charge correlation) or an electron (strong charge anti-correlation).

Events with a misidentified $\tau _{\mathrm {had}}$ tend to have a much softer $p_{\text {T}} ( \tau _{\mathrm {had}} )$ spectrum and a larger angular separation between the $\tau _{\mathrm {had}}$ and $E_{\text {T}}^{\text {miss}}$ directions. These properties are exploited to suppress backgrounds and define signal and control regions. Events with exactly one electron and exactly one $\tau _{\mathrm {had}}$ with $E_{\text {T}} (e)>26$ GeV, $p_{\text {T}} ( \tau _{\mathrm {had}} )>45$ GeV and $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|<2$ form a baseline sample as it represents a common selection for both the signal and control regions. The $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|$ cut has $\sim$99% efficiency for signal and rejects a considerable fraction of multi-jet and W+jets events. Similarly as done in Ref. [22], two signal regions are defined using the transverse mass², $m_{\mathrm {T}}$, of the e-$E_{\text {T}}^{\text {miss}}$ and $\tau _{\mathrm {had}}$-$E_{\text {T}}^{\text {miss}}$ systems: OS events with $m_{\mathrm {T}}^{e,E_{\text {T}}^{\text {miss}}}>40$ GeV and $m_{\mathrm {T}}^{ \tau _{\mathrm {had}} ,E_{\text {T}}^{\text {miss}}}<30$ GeV form the signal region-1 (SR1), while OS events with $m_{\mathrm {T}}^{e,E_{\text {T}}^{\text {miss}}}<40$ GeV and $m_{\mathrm {T}}^{ \tau _{\mathrm {had}} ,E_{\text {T}}^{\text {miss}}}<60$ GeV form the signal region-2 (SR2). Both regions have similar sensitivity to the signal (see Sect. 4.4). The dominant background in SR1 is W+jets, while the $Z/\gamma ^{*}\rightarrow \tau \tau$ and $Z\rightarrow ee$+jets backgrounds dominate in SR2. The modelling of the W+jets background is checked in a dedicated control region (WCR) formed by events with $m_{\mathrm {T}}^{e,E_{\text {T}}^{\text {miss}}}>60$ GeV and $m_{\mathrm {T}}^{ \tau _{\mathrm {had}} ,E_{\text {T}}^{\text {miss}}}>40$ GeV. As discussed in detail in Sect. 4.2, the modelling of the $Z/\gamma ^{*}\rightarrow \tau \tau$ and $Z\rightarrow ee$+jets backgrounds is checked in SR2. The choice of $m_{\mathrm {T}}$ cuts to define SR1, SR2 and WCR is motivated by correlations between $m_{\mathrm {T}}^{e,E_{\text {T}}^{\text {miss}}}$ and $m_{\mathrm {T}}^{ \tau _{\mathrm {had}} ,E_{\text {T}}^{\text {miss}}}$ in $H\rightarrow e\tau$ signal and major background (W+jets and $Z/\gamma ^{*}\rightarrow \tau \tau$) events, as illustrated in Fig. 1. No events with identified b-jets are allowed in SR1, SR2 and WCR. The modelling of the $t\bar{t}$ and single-top backgrounds is checked in a dedicated control region (TCR), formed by events that satisfy the baseline selection and have at least two jets, with at least one being b-tagged. Table 2 provides a summary of the event selection criteria used to define the signal and control regions.

Fig. 1

Two-dimensional distributions of the transverse mass of the e-$E_{\text {T}}^{\text {miss}}$ system, $m_{\mathrm {T}}^{e,E_{\text {T}}^{\text {miss}}}$, and that of the $\tau _{\mathrm {had}}$-$E_{\text {T}}^{\text {miss}}$ system, $m_{\mathrm {T}}^{ \tau _{\mathrm {had}} ,E_{\text {T}}^{\text {miss}}}$, in simulated $Z/\gamma ^{*}\rightarrow \tau \tau$ (top left plot), W+jets (top right plot), $H\rightarrow e\tau$ signal (bottom left plot) and data (bottom right plot) events. Magenta, red and yellow boxes on the bottom right plot illustrate SR1, SR2, and WCR, respectively. All events used for these distributions are required to have a well-identified electron and $\tau _{\mathrm {had}}$ (as described in text) of opposite charge with $p_{\text {T}} ( \tau _{\mathrm {had}} )>20$ GeV and $E_{\text {T}} (e)>26$ GeV

The LFV signal is searched for by performing a fit to the mass distribution in data, $m_{e\tau }^{\mathrm {MMC}}$, reconstructed from the observed electron, $\tau _{\mathrm {had}}$ and $E_{\text {T}}^{\text {miss}}$ objects by means of the Missing Mass Calculator [56] (MMC). Conceptually, the MMC is a more sophisticated version of the collinear approximation [57]. The main improvement comes from requiring that the relative orientations of the neutrino and other $\tau$-lepton decay products are consistent with the mass and kinematics of a $\tau$-lepton decay. This is achieved by maximising a probability defined in the kinematically allowed phase-space region. The MMC used in the $H\rightarrow \tau \tau$ analysis [35] is modified to take into account that there is only one neutrino from a hadronic $\tau$-lepton decay in LFV $H\rightarrow e\tau$ events. For a Higgs boson with $m_{H}=125$ GeV, the reconstructed $m_{e\tau }^{\mathrm {MMC}}$ distribution has a roughly Gaussian shape with a full width at half maximum of $\sim$19 GeV. The analysis is performed “blinded” in the 110 GeV$<m_{e\tau }^{\mathrm {MMC}}<$150 GeV regions of SR1 and SR2, which contain 93.5 and 95% of the expected signal events in SR1 and SR2, respectively. The event selection and the analysis strategy are defined without looking at the data in these blinded regions.

Table 2 Summary of the event selection criteria used to define the signal and control regions (see text)

4.2 Background estimation

The background estimation method takes into account the background properties and composition discussed in Sect. 4.1. It also relies on the observation that the shape of the $m_{e\tau }^{\mathrm {MMC}}$ distribution for the multi-jet background is the same for OS and SS events. This observation was made using a dedicated control region, MJCR, with an enhanced contribution from the multi-jet background. Events in this control region are required to meet all criteria for SR1 and SR2 with the exception of the requirement on $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|$, which is reversed: $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|>2$. Therefore, the total number of OS background events, $N_{\mathrm {OS}}^{\mathrm {bkg}}$ in each bin of the $m_{e\tau }^{\mathrm {MMC}}$ (or any other) distribution in SR1 and SR2 can be obtained according to the following formula:

$$\begin{aligned} N_{\mathrm {OS}}^{\mathrm {bkg}} =r_{\mathrm {QCD}}\cdot N_{\mathrm {SS}}^{\mathrm {data}} + \sum _{\mathrm {bkg}{\text {-}}i} N_{\mathrm {OS-SS}}^{\mathrm {bkg}{\text {-}}i}, \end{aligned}$$

(1)

where the individual terms are described below. $N_{\mathrm {SS}}^{\mathrm {data}}$ is the number of SS data events, which contains significant contributions from W+jets events, multi-jet and other backgrounds. The fractions of multi-jet background in SS data events inside the 110 GeV$< m_{e\tau }^{\mathrm {MMC}}<$150 GeV mass window are $\sim$27 and $\sim$64% in SR1 and SR2, respectively. The contributions $N_{\mathrm {OS-SS}}^{\mathrm {bkg}{\text {-}}i}=N_{\mathrm {OS}}^{\mathrm {bkg}{\text {-}}i}-r_{\mathrm {QCD}}\cdot N_{\mathrm {SS}}^{\mathrm {bkg}{\text {-}}i}$ are add-on terms for the different background components (where bkg-i indicates the $i^{\mathrm {th}}$ background source: $Z\rightarrow \tau \tau$, $Z\rightarrow ee$, W+jets, VV, $H\rightarrow \tau \tau$ and events with t-quarks), which also account for components of these backgrounds already included in SS data events.³ The factor $r_{\mathrm {QCD}}=N_{\mathrm {OS}}^{\mathrm {multi{\text {-}}jet}}/N_{\mathrm {SS}}^{\mathrm {multi{\text {-}}jet}}$ accounts for potential differences in flavour composition (and, as a consequence, in jet $\rightarrow \tau _{\mathrm {had}}$ misidentification rates) of final-state jets introduced by the same-sign or opposite-sign charge requirements. The value of $r_{\mathrm {QCD}}= 1.0\pm 0.13$ is obtained from a multi-jet enriched control region in data using a method discussed in Ref. [58]. This sample is obtained by selecting events with $E_{\text {T}}^{\text {miss}} <15$ GeV, $m_{\mathrm {T}}^{e,E_{\text {T}}^{\text {miss}}}<30$ GeV, removing the isolation criteria of the electron candidate and using the loose identification criteria for the $\tau _{\mathrm {had}}$ candidate [34]. The systematic uncertainty on $r_{\mathrm {QCD}}$ is estimated by varying the selection cuts described above. The obtained value of $r_{\mathrm {QCD}}$ is also verified in the MJCR region, which has a smaller number of events but where the electron and $\tau _{\mathrm {had}}$ candidates pass the same identification requirements as events in SR1 and SR2.

The data and simulation samples used for the modelling of background processes are described in Sect. 3. A discussion of each background source is provided below.

The largely irreducible $Z/\gamma ^{*}\rightarrow \tau \tau$ background is modelled by the embedded data sample described in Sect. 3. The $Z/\gamma ^{*}\rightarrow \tau \tau$ normalisation is a free parameter in the final fit to data and it is mainly constrained by events with 60 GeV$<$ $m_{e\tau }^{\mathrm {MMC}}$ $<$90 GeV in SR2.

Events due to the W+jets background are mostly selected when the $\tau _{\mathrm {had}}$ signature is mimicked by jets. This background is estimated from simulation, and the WCR region is used to check the modelling of the W+jets kinematics and to obtain separate normalisations for OS and SS W+jets events. The difference in these two normalisations happens to be statistically significant. An additional overall normalisation factor for the $N_{\mathrm {OS-SS}}^{W+\mathrm {jets}}$ term in Eq. (1) is introduced as a free parameter in the final fit in SR1. By studying WCR events and SR1 events with $m_{e\tau }^{\mathrm {MMC}}>150$ GeV (dominated by W+jets background), it is also found that an $m_{e\tau }^{\mathrm {MMC}}$ shape correction, which depends on the number of jets, $p_{\text {T}} ( \tau _{\mathrm {had}} )$ and $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|$, needs to be applied in SR1. This correction is derived from SR1 events with $m_{e\tau }^{\mathrm {MMC}}>150$ GeV and it is applied to events with any value of $m_{e\tau }^{\mathrm {MMC}}$. The corresponding modelling uncertainty is set to be 50% of the difference of the $m_{e\tau }^{\mathrm {MMC}}$ shapes obtained after applying the SR1-based and WCR-based shape corrections. The size of this uncertainty depends on $m_{e\tau }^{\mathrm {MMC}}$ and it is as large as ±10% for W+jets events with $m_{e\tau }^{\mathrm {MMC}}< 150$ GeV. In the case of SR2, good modelling of the $N_{\mathrm {jet}}$, $p_{\text {T}} ( \tau _{\mathrm {had}} )$ and $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|$ distributions suggests that such a correction is not needed. However, a modelling uncertainty in the $m_{e\tau }^{\mathrm {MMC}}$ shape of the W+jets background in SR2 is set to be 50% of the difference between the $m_{e\tau }^{\mathrm {MMC}}$ shape obtained without any correction and the one obtained after applying the correction derived for SR1 events. The size of this uncertainty is below 10% in the 110 GeV$< m_{e\tau }^{\mathrm {MMC}}<$150 GeV region, which contains most of the signal events. It was also checked that applying the same correction in SR2 as in SR1 would affect the final result by less than 4% (see Sect. 6). The modelling of jet fragmentation and the underlying event has a significant effect on the estimate of the jet $\rightarrow \tau _{\mathrm {had}}$ misidentification rate in different regions of the phase space and has to be accounted for with a corresponding systematic uncertainty. To estimate this effect, the analysis was repeated using a sample of W+jets events modelled by ALPGEN interfaced with the HERWIG event generator. Differences in the W+jets predictions in SR1 and SR2 are found to be ±12 and ±15%, respectively, and are taken as corresponding systematic uncertainties.

In the case of the $Z\rightarrow ee$ background, there are two components: events in which an electron mimics a $\tau _{\mathrm {had}}$ ($e\rightarrow \tau _{\mathrm {had}} ^{\mathrm {misid}} {}$) and events in which a jet mimics a $\tau _{\mathrm {had}}$ (jet$\rightarrow \tau _{\mathrm {had}} ^{\mathrm {misid}} {}$). In the first case, the shape of the $Z\rightarrow ee$ background is obtained from simulation. Corrections from data, derived from dedicated tag-and-probe studies [59], are also applied to account for the variation in the $e\rightarrow \tau _{\mathrm {had}} ^{\mathrm {misid}} {}$ misidentification rate as a function of $\eta$. The normalisation of this background component is a free parameter in the final fit to data and it is mainly constrained by events with 90 GeV$<$ $m_{e\tau }^{\mathrm {MMC}}$ $<$110 GeV in SR2. For the $Z\rightarrow ee$ background where a jet is misidentified as a $\tau _{\mathrm {had}}$ candidate and one of the electrons does not pass the electron identification criteria described in Sect. 2, the normalisation factor and shape corrections, which depend on the number of jets, $p_{\text {T}} ( \tau _{\mathrm {had}} )$ and $|\eta (e)-\eta ( \tau _{\mathrm {had}} )|$, are derived using events with two identified OS electrons with an invariant mass, $m_{ee}$, in the range of 80–100 GeV. Since this background does not have an OS–SS charge asymmetry, a single correction factor is derived for OS and SS events. Half the difference between the $m_{e\tau }^{\mathrm {MMC}}$ shape with and without this correction is taken as the corresponding systematic uncertainty.

The TCR is used to check the modelling and to obtain normalisations for OS and SS events with top quarks. The normalisation factors obtained in the TCR are extrapolated into SR1 and SR2, where $t\bar{t}$ and single-top events may have different properties. To estimate the uncertainty associated with such an extrapolation, the analysis is repeated using the MC@NLO [60] event generator instead of POWHEG for $t\bar{t}$ production.⁴ This uncertainty is found to be ±8% (±14%) for backgrounds with top quarks in SR1 (SR2).

The background due to diboson (WW, ZZ and WZ) production is estimated from simulation, normalised to the cross sections calculated at NLO in QCD [61]. Finally, the SM $H\rightarrow \tau \tau$ events also represent a small background in this search. This background is estimated from simulation and normalised to the cross sections calculated at NNLO in QCD [49,50,51]. All other SM Higgs boson decays constitute negligible backgrounds for the LFV signature.

Figure 2 shows the $m_{e\tau }^{\mathrm {MMC}}$ distributions for data and the predicted backgrounds in each of the signal regions. The backgrounds are estimated using the method described above and their normalisations are obtained in a global fit described in Sect. 4.4. The signal acceptance times efficiencies for passing the SR1 or SR2 selection requirements are 1.8 and 1.4%, respectively, and the combined efficiency is 3.2%. The numbers of observed events in the data as well as the signal and background predictions in the mass region 110 GeV$< m_{e\tau }^{\mathrm {MMC}}<$150 GeV can be found in Table 3.

Fig. 2

Distributions of the mass reconstructed by the Missing Mass Calculator, $m_{e\tau }^{\mathrm {MMC}}$, in SR1 (left) and SR2 (right). The background distributions are determined in a global fit (described in Sect. 4.4). The signal distribution corresponds to Br($H\rightarrow e\tau$) = 25%. The bottom panel of each sub-figure shows the ratio of the observed data to the estimated background. Very small backgrounds due to single top, $t\bar{t}$, VV, $Z\rightarrow ee(\text {jet}\rightarrow \tau _{\mathrm {had}} ^{\mathrm {misid}} {})$ and $H\rightarrow \tau \tau$ events are combined in a single background component labelled as “Other Backgrounds”. The grey band for the ratio illustrates post-fit systematic uncertainties in the background prediction. The statistical uncertainties in the background predictions and data are added in quadrature for the ratios. The last bin in each distribution contains events with $m_{e\tau }^{\mathrm {MMC}}>$ 250 GeV

Table 3 Data yields, signal and post-fit OS–SS background predictions (see Eq. (1)) for the 110 GeV$< m_{e\tau }^{\mathrm {MMC}}<$150 GeV region. The signal predictions are given for Br($H\rightarrow e\tau$) = 1.0%. The background predictions are obtained from the combined fit to SR1, SR2, WCR and TCR. The post-fit values of systematic uncertainties are provided for the background predictions. For the total background, all correlations between various sources of systematic uncertainties and backgrounds are taken into account. The quoted uncertainties represent the statistical (first) and systematic (second) uncertainties, respectively

4.3 Systematic uncertainties

The numbers of signal and background events and the shapes of corresponding $m_{e\tau }^{\mathrm {MMC}}$ distributions are affected by systematic uncertainties. They are discussed below and changes in event yields are provided for major sources of uncertainties. For all uncertainties, the effects on both the total signal and background predictions and on the shape of the $m_{e\tau }^{\mathrm {MMC}}$ distribution are evaluated. Unless otherwise mentioned, all sources of experimental uncertainties are treated as fully correlated across signal and control regions in the final fit which is discussed in Sect. 4.4.

The largest systematic uncertainties arise from the normalisation (±12% uncertainty) and modelling of the W+jets background. The uncertainties on the W+jets normalisation and $m_{e\tau }^{\mathrm {MMC}}$ shape corrections are treated as uncorrelated between SR1 and SR2. The uncertainties in $r_{\mathrm {QCD}}$ (±13%) and in the normalisation (±13%) and modelling of $Z\rightarrow \tau \tau$ also play an important role. The normalisation uncertainty (±7%) for the $Z\rightarrow ee$ (with $e\rightarrow \tau _{\mathrm {had}} ^{\mathrm {misid}} {}$) background has a limited impact on the sensitivity because of a good separation of the signal and $Z\rightarrow ee$ peaks in the $m_{e\tau }^{\mathrm {MMC}}$ distribution. The other major sources of experimental uncertainty, affecting both the shape and normalisation of signal and backgrounds, are the uncertainty in the $\tau _{\mathrm {had}}$ energy scale [34], which is measured with ±(2–4)% precision (depending on $p_{\text {T}}$ and decay mode of the $\tau _{\mathrm {had}}$ candidate), and uncertainties in the embedding method used to model the $Z\rightarrow \tau \tau$ background [35]. Less significant sources of experimental uncertainty, affecting the shape and normalisation of signal and backgrounds, are the uncertainty in the jet energy scale [37, 62] and resolution [63]. The uncertainties in the $\tau _{\mathrm {had}}$ energy resolution, the energy scale and resolution of electrons, and the scale uncertainty in $E_\mathrm {T}^\mathrm {miss}$ due to the energy in calorimeter cells not associated with physics objects are taken into account; however, they are found to be only ±(1–2$\%$). The following experimental uncertainties primarily affect the normalisation of signal and backgrounds: the ±2.8% uncertainty in the integrated luminosity [64], the uncertainty in the $\tau _{\mathrm {had}}$ identification efficiency [34], which is measured to be ±(2–3)% for 1-prong and ±(3–5)% for 3-prong decays(where the range reflects the dependence on $p_{\text {T}}$ of the $\tau _{\mathrm {had}}$ candidate), the ±2.1% uncertainty for triggering, reconstructing and identifying electrons [33], and the ±2% uncertainty in the b-jet tagging efficiency [38].

Theoretical uncertainties are estimated for the Higgs boson production and for the VV background, which are modelled with the simulation and are not normalised to data in dedicated control regions. Uncertainties due to missing higher-order QCD corrections in the production cross sections are found to be [65] ±10.1$\%$ (±7.8$\%$) for the Higgs boson production via ggH in SR1 (SR2), ±1$\%$ for the $Z\rightarrow ee$ background and for VBF and VH Higgs boson production, and ±5% for the VV background. The systematic uncertainties due to the choice of parton distribution functions used in the simulation are evaluated based on the prescription described in Ref. [65] and the following values are used in this analysis: ±7.5$\%$ for the Higgs boson production via ggH, ±2.8$\%$ for the VBF and VH Higgs boson production, and ±4$\%$ for the VV background. Finally, an additional ±5.7% systematic uncertainty [65] on Br($H\rightarrow \tau \tau$) is applied to the SM $H\rightarrow \tau \tau$ background.

4.4 Results of the search for LFV $H\rightarrow e\tau$ decays in the $\tau _{\mathrm {had}}$ channel

Fig. 3

Post-fit combined $m_{e\tau }^{\mathrm {MMC}}$ distribution obtained by adding individual distributions in SR1 and SR2. In the lower part of the figure, the data are shown after subtraction of the estimated backgrounds. The grey band in the bottom panel illustrates the post-fit systematic uncertainties in the background prediction. The statistical uncertainties for data and background predictions are added in quadrature in the bottom part of the figure. The signal is shown assuming Br$(H\rightarrow e\tau )=1.0\%$. Very small backgrounds due to single top, $t\bar{t}$, VV, $Z\rightarrow ee(jet\rightarrow \tau _{\mathrm {had}} ^{\mathrm {misid}} {})$ and $H\rightarrow \tau \tau$ events are combined in a single background component labelled as “Other Backgrounds”. The last bin of the distribution contains events with $m_{e\tau }^{\mathrm {MMC}}>$250 GeV

A simultaneous binned maximum-likelihood fit is performed on the $m_{e\tau }^{\mathrm {MMC}}$ distributions in SR1 and SR2 and on event yields in WCR and TCR to extract the LFV branching ratio Br($H\rightarrow e\tau$). The fit exploits the control regions and the distinct shapes of the W+jets, $Z\rightarrow \tau \tau$ and $Z\rightarrow ee$ backgrounds in the signal regions to constrain some of the systematic uncertainties. This increases the sensitivity of the analysis. The post-fit $m_{e\tau }^{\mathrm {MMC}}$ distributions in SR1 and SR2 are shown in Fig. 2, and the combined $m_{e\tau }^{\mathrm {MMC}}$ distribution for both signal regions is presented in Fig. 3. Figure 2 illustrates the level of agreement between data and background expectations in SR1 and SR2. No statistically significant deviations of the data from the predicted background are observed. An upper limit on the LFV branching ratio Br($H\rightarrow e\tau$) for a Higgs boson with $m_{H}=125$ GeV is set using the CL$_s$ modified frequentist formalism [66] with the test statistic based on the profile likelihood ratio [67]. The observed and the median expected 95% CL upper limits are $1.81$% and $2.07^{+0.82}_{-0.58}$%, respectively. Table 6 provides a summary of all results, including the results of the ATLAS search for the LFV $H\rightarrow \mu \tau$ decays [22].

5 Search for $H\rightarrow e\tau /\mu \tau$ decays in the $\tau _{\mathrm {lep}} {}$ channel

In the $\tau _{\mathrm {lep}}$ channel the background estimate is based on the data-driven method developed in Ref. [29]. This method is sensitive only to the difference between Br($H\rightarrow \mu \tau$) and Br($H\rightarrow e\tau$), and it is based on the premise that the kinematic properties of the SM background are to a good approximation symmetric under the exchange $e\leftrightarrow {}\mu$.

5.1 Event selection and signal region definition

Events selected in the $\tau _{\mathrm {lep}}$ channel must contain exactly two opposite-sign leptons, one an electron and the other a muon. The lepton with the higher $p_{\text {T}}$ is indicated by $\ell _1$ and the other by $\ell _2$. Additional kinematic criteria, based on the $p_{\text {T}}$ difference between the two leptons and on the angular separations between the leptons and the missing transverse momentum, are applied to suppress the SM background events, which are mainly due to the production of $Z/\gamma ^{*}\rightarrow \tau \tau$ and of diboson (VV) events. Two mutually exclusive signal regions are defined: one with no central ($|\eta |<2.4$) light-flavour jets, $\mathrm {SR_{noJets}}$, and the other with one or more central light-flavoured jets, $\mathrm {SR_{withJets}}$. The kinematic criteria defining each signal region, summarised in Table 4, are optimised following two guidelines. The first one is to maximise the signal-to-background ratio. The second one is to have, in each signal region, enough events to perform the data-driven background estimation described in Sect. 5.2.

Table 4 Summary of the selection criteria used to define the signal regions in the $\tau _{\mathrm {lep}}$ channel (see text)

The final discriminant used in the $\tau _{\mathrm {lep}}$ channel is the collinear mass $\mathrm {m_{coll}}$ defined as:

$$\begin{aligned} m_\mathrm{coll}{} = \sqrt{2p_{\mathrm {T}}^{\ell {}_{1}}{} (p_{\mathrm {T}}^{\ell {}_{2}}{}+E_{\text {T}}^{\text {miss}} {}) (\cosh {}\Delta \eta -\cos \Delta \phi )}. \end{aligned}$$

(2)

This quantity is the invariant mass of two massless particles, $\tau {}$ and $\ell _1$, computed with the approximation that the decay products of the $\tau {}$ lepton, $\ell _2$ and neutrinos, are collinear to the $\tau {}$, and that the $E_{\text {T}}^{\text {miss}}$ originates from the $\nu {}$. In the $H \rightarrow \mu {}\tau {}$ ($H \rightarrow {}e\tau {}$) decay, $\ell _1$ is the muon (electron) and $\ell _2$ is the electron (muon). The differences in rapidity and azimuthal angle between $\ell _1$ and $\ell _2$ are indicated by $\Delta \eta {}$ and $\Delta \phi {}$. More sophisticated kinematic variables, such as MMC, do not significantly improve the sensitivity of the $\tau _{\mathrm {lep}}$ channel.

5.2 Background estimation

For simplicity, the symmetry method is illustrated here assuming a $H \rightarrow \mu {}\tau {}$ signal. The same procedure, but with e and $\mu {}$ exchanged, is valid under the $H \rightarrow {}e\tau {}$ assumption. The symmetry method is based on the following two premises:

• 1. SM processes result in data that are symmetric under the exchange of prompt electrons with prompt muons to a good approximation. In other words, the kinematic distributions of prompt electrons and prompt muons are approximately the same;⁵

• 2. flavour-violating decays of the Higgs boson break this symmetry.

Dilepton events in the dataset are divided into two mutually exclusive samples:

• $\varvec{\mu {}e}$ sample: $\ell _1$ is the muon and $\ell _2$ is the electron ($p_{\text {T}} {}^\mu \ge p_{\text {T}} {}^e$)

• $\varvec{e\mu {}}$ sample: $\ell _1$ is the electron and $\ell _2$ is the muon ($p_{\text {T}} {}^e > p_{\text {T}} {}^\mu$)

With these assumptions, the SM background is split equally between the two samples. The $H \rightarrow \mu {}\tau {}$ signal, however, is present only in the $\mu e$ sample because the $p_{\text {T}}$ spectrum of electrons from $H \rightarrow \mu {}\tau {}$ decays is softer then the muon $p_{\text {T}}$ spectrum. The number of $H \rightarrow \mu {}\tau {}$ events in the $e\mu$ sample is negligible with the selection criteria described in Sect. 5.1.

For SM events the distributions of kinematic variables in the two samples are the same with good approximation. In particular, the collinear mass distribution differs between the two samples only for the narrow signal peak. The peak, present only in the distribution of the $\mu e$ sample, is on top of the SM background, which, to a good approximation, can be modelled from the $e\mu$ collinear mass distribution.

5.2.1 Asymmetries in the SM background

Although the $e\mu$-$\mu e$ symmetry hypothesis is a good starting assumption, there are effects that can invalidate it and that need to be accounted for. The first effect is due to events containing misidentified and non-prompt leptons, together referred to as non-prompt in the following. These leptons originate from misidentified jets or from hadronic decays within jets. They contribute differently to the $\mu e$ and $e\mu$ samples because the origin of the non-prompt lepton is different for electrons and for muons. The second effect originates from the different dependencies on $p_{\text {T}}$ and $|\eta |$ that the trigger efficiency and reconstruction efficiency can have for electrons and muons. The non-prompt effect is accounted for by estimating the non-prompt background separately from the other backgrounds. The efficiency effect is accounted for by scaling the $m_\mathrm{coll}$ distribution of the $e\mu$ sample with a scale factor parameterised as a function of the sub-leading lepton $p_{\text {T}}$ , $p_{\mathrm {T}}^{\ell {}_{2}}$. As shown in Sect. 5.5, the $e\mu$-$\mu e$ symmetry is restored when these two effects are taken into account. Smaller effects, which might depend on other parameters such as $\eta$ or $p_{\mathrm {T}}^{\ell {}_{1}}$, are found to be negligible.

Events containing non-prompt leptons The background contribution due to non-prompt leptons is estimated with the matrix method described in Refs. [68, 69], which relies on the difference in identification efficiency between prompt and non-prompt leptons. Two lepton categories are defined: tight leptons, which must satisfy all the lepton identification criteria described in Sect. 2, and loose leptons, which are not required to satisfy the primary vertex and isolation criteria. By measuring separately for prompt and non-prompt leptons the tight-to-loose lepton efficiencies, defined as the fraction of loose leptons that are also tight, one can determine the non-prompt background contribution from the number of data events that have two leptons that are either loose or tight. The efficiencies for prompt and non-prompt leptons, parameterised as a function of $p_{\text {T}}$ and $\eta$, are derived from data with the tag-and-probe method. Prompt efficiencies are derived from an opposite-sign sample enriched in $Z\rightarrow {}e^{\pm }e^{\mp }$ and $Z\rightarrow {}\mu ^{\pm }\mu ^{\mp }$. Non-prompt efficiencies are derived from a same-sign sample ($\mu ^{\pm }e^{\pm }$ or $\mu ^{\pm }\mu ^{\pm }$) where the muon is the tag lepton.

Asymmetry induced by the different trigger and reconstruction efficiency of electrons and muons The efficiency to trigger on and reconstruct an $e\mu$ event, $\varepsilon ^{e\mu }$, is different from the one of a $\mu e$ event, $\varepsilon ^{\mu {}e}$. These two efficiencies can be expressed as a function of the $p_{\text {T}}$ of the two leptons:

$$\begin{aligned} \varepsilon ^{\mu e}= \varepsilon ^{\mu e}_{\text {trig.}}\left( p_{\text {T}} ^{\ell _{2}=e}\right) \times \varepsilon ^{\mu }_{\text {reco.}}\left( p_{\text {T}} ^{\ell _1=\mu }\right) \times \varepsilon ^{e}_{\text {reco.}}\left( p_{\text {T}} ^{\ell _2=e}\right) \end{aligned}$$

$$\begin{aligned} \varepsilon ^{e \mu }= \varepsilon ^{e \mu }_{\text {trig.}}\left( p_{\text {T}} ^{\ell _{2}=\mu }\right) \times \varepsilon ^{e}_{\text {reco.}}\left( p_{\text {T}} ^{\ell _1=e}\right) \times \varepsilon ^{\mu }_{\text {reco.}}\left( p_{\text {T}} ^{\ell _2=\mu }\right) . \end{aligned}$$

In this search, the leading lepton is required to have $p_{\mathrm {T}}^{\ell {}_{1}}{}>35~\mathrm{GeV}{}$, which is on the plateau region of the trigger and reconstruction efficiencies. Hence the ratio of the efficiencies can be approximated as:

$$\begin{aligned} \frac{\varepsilon ^{\mu e}}{\varepsilon ^{e \mu }}&= \frac{ \varepsilon ^{\mu e}_{\text {trig.}}\left( p_{\text {T}} ^{\ell _{2}}\right) \varepsilon ^{\mu }_{\text {reco.}}\left( p_{\text {T}} ^{\ell _1}\right) \varepsilon ^{e}_{\text {reco.}}\left( p_{\text {T}} ^{\ell _2}\right) }{ \varepsilon ^{e\mu }_{\text {trig.}}\left( p_{\text {T}} ^{\ell _{2}}\right) \varepsilon ^{e}_{\text {reco.}}\left( p_{\text {T}} ^{\ell _1}\right) \varepsilon ^{\mu }_{\text {reco.}}\left( p_{\text {T}} ^{\ell _2}\right) } \\&= \frac{ \varepsilon ^{\mu e}_{\text {trig.}}\left( p_{\text {T}} ^{\ell _{2}}\right) \varepsilon ^{e}_{\text {reco.}}\left( p_{\text {T}} ^{\ell _2}\right) }{ \varepsilon ^{e\mu }_{\text {trig.}}\left( p_{\text {T}} ^{\ell _{2}}\right) \varepsilon ^{\mu }_{\text {reco.}}\left( p_{\text {T}} ^{\ell _2}\right) }\times \frac{ \varepsilon ^{\mu }_{\text {reco.}}\left( p_{\text {T}} ^{\ell _1}\right) }{ \varepsilon ^{e}_{\text {reco.}}\left( p_{\text {T}} ^{\ell _1}\right) } \\&= f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right) {}\times \text {Const.} \end{aligned}$$

Therefore, the ratio of the $e\mu$ and $\mu e$ event reconstruction efficiencies can be parameterised as a function of the sub-leading lepton $p_{\text {T}}$ , $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$. Using the fit described in Sect. 5.4, the parameter $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ is determined in three $p_{\mathrm {T}}^{\ell {}_{2}}$ bins, 12–20, 20–30, and $> 30~\mathrm{GeV}{}$.

5.3 Systematic uncertainties

Using the $e\mu {}$ asymmetry technique, the only systematic uncertainty associated with the background prediction is due to the non-prompt background modelling. This uncertainty has two components: the first one is the limited number of tag-and-probe events used to extract the prompt and non-prompt efficiencies; the second one is the difference in kinematics, and therefore in sources of non-prompt leptons, between the events used to extract the non-prompt efficiency and the events in the signal regions. This second component is evaluated by measuring the non-prompt efficiencies in subsets of the nominal tag-and-probe sample. The subsets are obtained by applying, one at a time, the kinematic requirements of the signal regions. The ensuing uncertainties in the estimated number of non-prompt events can be as large as 10–50$\%$ for the non-prompt efficiency and $3\%$ for the prompt efficiency, depending on the signal region.

Uncertainties related to the signal prediction are the same ones described in Sect. 4.3 with one minor difference in the uncertainty in the signal cross section due to higher-order QCD corrections. This uncertainty is split into two anticorrelated components: $\pm {}12\%$ in $\mathrm {SR_{withJets}}$ and $\pm {}20\%$ in $\mathrm {SR_{noJets}}$.

5.4 The statistical model

Assuming that the SM background is completely symmetric when exchanging $e\leftrightarrow \mu {}$ , the likelihood function for the collinear mass distribution of the $e\mu$ and $\mu e$ samples can be written as:

$$\begin{aligned} L(b_i,\mu ) = \prod \limits _i^{N_{m_\mathrm{coll}{}}} \mathrm{Pois}(n_i\mid b_i) \times \mathrm{Pois}(m_i\mid b_i+\mu s_i), \end{aligned}$$

(3)

where $n_i$ $(m_i)$ is the number of $e\mu$ ($\mu e$) events in the i-th of the $N_{m_\mathrm{coll}}$ $m_\mathrm{coll}$ bins. The number of background events in the i-th $m_\mathrm{coll}$ bin is indicated by $b_i$, and $s_i$ is the number of $H \rightarrow \mu {}\tau {}$ events in the i-th mass bin. The number of signal events $\sum \limits _{i}s_i$ is normalised to a branching ratio $\text{ Br }(H \rightarrow \mu {}\tau {}{})=1\%$, multiplied by a signal strength $\mu$. The likelihood for the $m_\mathrm{coll}$ distributions with a $H \rightarrow {}e\tau {}$ signal can be defined in a similar way. The contributions due to non-prompt leptons add to the $e\mu$ and $\mu e$ terms and they are denoted by $N_i^{\text {np}}$ and $M_i^{\text {np}}$, along with their uncertainties, $\sigma _{N_i^{\text {np}}}$ and $\sigma _{M_i^{\text {np}}}$. The numbers of non-prompt events in each bin, $N_i^{\text {np}}$ and $M_i^{\text {np}}$, are treated as Gaussian nuisance parameters.

The $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ correction, described in Sect. 5.2, is implemented by performing the fit separately in $N_{p_{\mathrm {T}}^{\ell {}_{2}}{}}=3$ $p_{\mathrm {T}}^{\ell {}_{2}}$ bins, labelled with the index j. The corrective scale factor $A_j$, corresponding to the $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ value in the $m_\mathrm{coll}$ bin i and $p_{\mathrm {T}}^{\ell {}_{2}}$ bin j, multiplies the $e\mu$ yield $b_{ij}$. These scale factors are treated in the statistical model as unconstrained nuisance parameters.

Adding up the symmetric contribution ($b_{ij}$), the non-prompt contributions ($N_{ij}^{\text {np}}$ and $M_{ij}^{\text {np}}$), the $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right) {}$ correction, and the signal contribution ($s_{ij}$), the likelihood is written as:

$$\begin{aligned} L(\mu , b_{ij}, n_{ij}^{\text {np}}, m_{ij}^{\text {np}})= & {} \prod \limits _i^{N_{m_\mathrm{coll}{}}} \prod \limits _j^{N_{p_{\mathrm {T}}^{\ell {}_{2}}{}}} \mathrm{Pois}(n_{ij}\mid A_jb_{ij} + n_{ij}^{\text {np}})\nonumber \\&\times \mathrm{Pois}(m_{ij}\mid b_{ij} + m_{ij}^{\text {np}} + \mu s_{ij})\nonumber \\&\times \mathrm{Gaus}(n_{ij}^{\text {np}} | N_{ij}^{\text {np}} , \sigma _{N_{ij}^{\text {np}}})\nonumber \\&\times \mathrm{Gaus}(m_{ij}^{\text {np}} | M_{ij}^{\text {np}} , \sigma _{M_{ij}^{\text {np}}}). \end{aligned}$$

(4)

5.5 Background model validation

The symmetry-based method is validated with simulation and with data. The validation with simulated samples is performed by comparing the signal strength measured in the SR with background samples, and with signal samples corresponding to several non-zero LFV branching ratios. The validation with data is performed in a validation region (VR) defined as $\mathrm {SR_{noJets}}$, but with at least one angular requirement reversed, $\Delta \phi (\ell _1,\ell _2)$ or $\Delta \phi (\ell _1,E_{\text {T}}^{\text {miss}})$.

The validation procedure consists of comparing the data, or the sum of the simulated background samples, to the total background estimated from the statistical model. The comparison is done for the $e\mu$ sample and the $\mu e$ one. With the simulated samples, it is also verified that the symmetric background and the $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ do not depend on the presence of an LFV signal.

Generated pseudo-experiments are used to confirm that the statistical model is unbiased. No significant discrepancy was found between the injected signal strength and its fitted value up to LFV branching ratios of $10\%$.

5.6 Results of the search for LFV $H\rightarrow e\tau /\mu \tau$ decays in the $\tau _{\mathrm {lep}} {}$ channel

Figure 4 compares the observed data to the yields expected from the symmetry-based statistical model. The comparison, combining the different $p_{\mathrm {T}}^{\ell {}_{2}}$ bins, shows the symmetric component of the background ($b_{ij}$) as a dashed line, and the total background estimation including the contribution from events containing misidentified and non-prompt leptons as a full line. As can be seen, the background estimation is in good agreement with the data over the full mass range. Table 5 summarises the fit results in the data in $\mathrm {SR_{noJets}}$ and $\mathrm {SR_{withJets}}$: the fitted $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ scale factors, the symmetric background component ($\sum \limits _{i}^{N_{m_\mathrm{coll}}}b_{ij}$) in each $p_{\mathrm {T}}^{\ell {}_{2}}$ bin, and the non-prompt estimate in the $\mu e$ and the $e\mu$ channels. The excellent level of agreement between the fitted number of events and the observed number is due to the many unconstrained parameters in the fit.

Fig. 4

Collinear mass distributions in the $\tau _{\mathrm {lep}} {}$ channel: background estimate compared to the events observed in the data in the $\mathrm {SR_{noJets}}$ (top) and $\mathrm {SR_{withJets}}$ (bottom). Left $e\mu$ channel. Right $\mu e$ channel. In these plots, events from the three $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ bins are combined, although the fit parameters are different in each $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$ bin. The signal expected for a $\text {Br}(H \rightarrow \mu {}\tau {}{})=1\%$ is shown in the $\mu e$ channel

Table 5 A summary of the fit results in the $\tau _{\mathrm {lep}} {}$ channel. The values of the fit parameters $f\left( p_{\mathrm {T}}^{\ell {}_{2}}\right)$, which account for the ratio of the $e\mu$ and $\mu e$ event reconstruction efficiencies described in Sect. 5.2, are obtained from a background-only fit, and reported for each signal region and for each $p_{\mathrm {T}}^{\ell {}_{2}}$ bin. The expected and observed yields correspond to the number of events used in the fit, representing the 0–300 GeV $m_\mathrm{coll}$ range shown in Fig. 4. The quoted uncertainties in the expected yields represent the statistical (first) and systematic (second) uncertainties, respectively. The post-fit values of systematic uncertainties are provided for the background predictions. The signal predictions are given for $\text {Br}(H \rightarrow {}e\tau {}{})=1\%$ in the $e\mu$ sample and for $\text {Br}(H \rightarrow \mu {}\tau {}{})=1\%$ in the $\mu e$ sample

The expected and observed 95$\%$ CL upper limits on branching ratios as well as their best fit values are calculated using the statistical model described in Sect. 5.4. Table 6 presents a summary of results for the individual categories and their combination can be found in Table 6 for both the $H \rightarrow {}e\tau {}$ and $H \rightarrow \mu {}\tau {}$ hypotheses.

6 Combined results of the search for LFV $H\rightarrow e\tau /\mu \tau$ decays

The results of the individual searches for the LFV $H\rightarrow e\tau$ and $H\rightarrow \mu \tau$ decays in the $\tau _{\mathrm {had}}$ (including the result from Ref. [22]) and $\tau _{\mathrm {lep}}$ channels presented in Sects. 4.4 and 5.6 are statistically combined. The two channels use different background estimation techniques, leading to uncorrelated systematic uncertainties in the background predictions. The systematic uncertainties for the LFV signal are treated as 100$\%$ correlated between the two channels. Table 6 presents a summary of results for the expected and observed $95\%$ CL upper limits and the best fit values for the branching ratios for the individual categories and their combination. There is no indication of a signal in the search for the LFV $H\rightarrow e\tau$ decays. The combined observed, and the median expected, 95$\%$ CL upper limits on Br($H\rightarrow e\tau$) for a Higgs boson with $m_{H} = 125$ GeV are 1.04$\%$ and $1.21^{+0.49}_{-0.34}$ $\%$, respectively. A small $\sim$1$\sigma$ excess of data over the predicted background is observed in the search for the LFV $H\rightarrow \mu \tau$ decays. It is mostly driven by a 1.3$\sigma$ excess in the earlier search in the $\mu \tau _{\mathrm {had}}$ channel [22]. This corresponds to a best fit value for the branching ratio of Br($H\rightarrow \mu \tau$) = ($0.53 \pm 0.51$)$\%$. In the absence of any significant signal, an upper limit on the LFV branching ratio Br($H\rightarrow \mu \tau$) for a Higgs boson with $m_\mathrm {H} = 125$ GeV is set. The corresponding observed, and the median expected, 95$\%$ CL upper limits are 1.43$\%$ and $1.01^{+0.40}_{-0.29}$ $\%$, respectively. The upper limits on the LFV decays of the Higgs boson are summarised in Fig. 5.

Fig. 5

Upper limits on LFV decays of the Higgs boson in the $H \rightarrow {}e\tau {}$ hypothesis (left) and $H \rightarrow \mu {}\tau {}$ hypothesis (right). The limits are computed under the assumption that either Br($H\rightarrow {}\mu {}\tau {}$)=0 or Br($H\rightarrow {}e\tau {}$)=0. The $\mu \tau _{\mathrm {had}}$ channel is from Ref. [22]

Table 6 Results of the search for the LFV $H\rightarrow e\tau$ and $H\rightarrow \mu \tau$ decays. The limits are computed under the assumption that either Br($H\rightarrow {}\mu {}\tau {}$)=0 or Br($H\rightarrow {}e\tau {}$)=0. The expected and observed $95\%$ confidence level (CL) upper limits and the best fit values for the branching ratios for the individual categories and their combination. The $\mu \tau _{\mathrm {had}}$ channel is from Ref. [22]

7 Search for $Z\rightarrow \mu \tau$ using the $\tau _{\mathrm {had}}$ channel

The search for $Z\rightarrow \mu \tau$ events is based on $\mu \tau _{\mathrm {had}}$ final state and utilises the same strategy as the $H\rightarrow \mu \tau$ analysis documented in Ref. [22], and applied to the $H\rightarrow e \tau _{\mathrm {had}}$ search described above. The final state is characterised by the presence of an energetic muon and a $\tau _{\mathrm {had}}$ of opposite charge and the presence of moderate $E_\mathrm {T}^\mathrm {miss} {},$ aligned with the $\tau _{\mathrm {had}}$ direction. The typical transverse momenta of the muon and of the $\tau _{\mathrm {had}}$ are somewhat softer than those expected in Higgs boson LFV decay, due to the lower mass of the Z boson. The main backgrounds are the same as those observed in $H\rightarrow \mu \tau _{\mathrm {had}}$ analyses, namely: $Z\rightarrow \tau \tau$, W+jets, multi-jet, $H\rightarrow \tau \tau$, diboson and top backgrounds. The $m_{\mu \tau }^{\mathrm {MMC}}$ variable is used to extract the signal using the same fit procedure and estimation of systematic uncertainties as for the $H\rightarrow \mu \tau _{\mathrm {had}}$ search. The corresponding Higgs boson LFV contribution is assumed to be negligible.

The $Z\rightarrow \mu \tau$ analysis differs from the $H\rightarrow \mu \tau _{\mathrm {had}}$ one as follows:

• The signal and control regions are defined in the same way as in the $H\rightarrow \mu \tau _{\mathrm {had}}$ analysis, but the cut values are lowered to match the kinematics of Z boson decay products. The exact definition is given in Table 7.

• The LFV $H\rightarrow \mu \tau _{\mathrm {had}}$ signal sample is replaced with a LFV $Z\rightarrow \mu \tau$ signal sample.

• The shape correction for W+jets in SR1 is obtained from the $m_{\mu \tau }^{\mathrm {MMC}}>110$ GeV sideband in SR1.

• Due to larger W+jets contribution in SR1 and SR2, the shape corrections for the W+jets samples are calculated using a three-dimensional binning scheme in $p_{\text {T}} ( \tau _{\mathrm {had}} )$, $|\eta (\mu )-\eta ( \tau _{\mathrm {had}} )|$ and $N_{\mathrm {jet}}$.

• The W+jets extrapolation uncertainty, which accounts for the difference between the W+jets ALPGEN PYTHIA and HERWIG samples, is also included as a shape uncertainty.

The numbers of observed events and background in each of the regions are given in Table 8. The efficiencies for simulated $Z\rightarrow \mu \tau$ signal events to pass the SR1 and SR2 selections are 1.2 and 0.8%, respectively. Figure 6 shows the $m_{\mu \tau }^{\mathrm {MMC}}$ distribution for data and predicted background in each of the signal regions. The discrepancy observed in the $m_{\mu \tau }^{\mathrm {MMC}}$ range 80–100 GeV of SR1 was studied carefully. All the other SR1 distributions, including lepton momenta, transverse masses, and missing transverse momentum, are in excellent agreement with the predictions, and the background shapes are constrained in the control regions as well as in SR2. This discrepancy is hence attributed to a statistical fluctuation.

Table 7 Summary of the $Z\rightarrow \mu \tau _{\mathrm {had}}$ event selection criteria used to define the signal and control regions (see text)

Fig. 6

Distributions of the mass reconstructed by the Missing Mass Calculator, $m_{\mu \tau }^{\mathrm {MMC}}$, in $Z\rightarrow \mu \tau$ SR1 (left) and SR2 (right). The background distributions are determined in a global fit. The signal distributions are scaled to a branching ratio of Br($Z\rightarrow \mu \tau$) = $10^{-3}$ to make them visible. The bottom panel of each subfigure shows the ratio of the observed data to the estimated background. The hatched band for the ratio illustrates post-fit systematic uncertainties in the background prediction. The statistical uncertainties for data and background predictions are added in quadrature for the ratios. The last bin of the distribution contains events with $m_{\mu \tau }^{\mathrm {MMC}}>200$ GeV

Table 8 Data yields, signal and post-fit OS–SS background predictions (see Eq. (1)) for the $Z\rightarrow \mu \tau _{\mathrm {had}}$ 80 GeV$<m_{\mu \tau }^{\mathrm {MMC}}<$115 GeV region. The signal predictions are given assuming Br($Z\rightarrow \mu \tau$) = $10^{-5}$. The background predictions are obtained from the combined fit to SR1, SR2, WCR and TCR. To calculate these quantities for SR1 and SR2, the signal strengths are decorrelated in the signal regions and set to zero in the control regions. The post-fit values of systematic uncertainties are provided for the background predictions. For the total background, all correlations between various sources of systematic uncertainties and backgrounds are taken into account. The quoted uncertainties represent the statistical (first) and systematic (second) uncertainties, respectively

No excess of data is observed and the CL$_s$ limit-setting technique is used to calculate the observed and expected limits on the branching ratio for $Z\rightarrow \mu \tau$  decays. The observed 95 % CL limit on Br($Z\rightarrow \mu \tau$) is $1.7 \times 10^{-5}$, which is lower than the expected upper limit of Br($Z\rightarrow \mu \tau$)$= 2.6 \times 10^{-5}$, but still within the $2 \sigma$ band. This corresponds to a best fit value for the branching ratio Br($Z\rightarrow \mu \tau$)$= -1.6_{-1.4}^{+1.3} \times 10^{-5}$. The results for the different signal regions are summarised in Table 9.

Table 9 The expected and observed 95% CL exclusion limits as well as the best fit values for the branching ratio of $\text {Br}(Z\rightarrow \mu \tau )[10^{-5}]\,$ are shown for SR1, SR2 and the combined fit. To calculate these quantities for SR1 and SR2, the signal strengths are decorrelated in the signal regions and set to zero in the control regions

8 Summary

Searches for lepton-flavour-violating decays of the Z and Higgs bosons are performed using a data sample of proton–proton collisions recorded by the ATLAS detector at the LHC corresponding to an integrated luminosity of 20.3 $\mathrm{fb}^{-1}$ at $\sqrt{s}=8$ TeV. Three LFV decays are considered: $H\rightarrow e\tau$, $H\rightarrow \mu \tau$, and $Z\rightarrow \mu \tau$. The search for the Higgs boson decays is performed in the final states where the $\tau$-lepton decays either to hadrons or to leptons (electron or muon). The search for the Z boson decays is performed in the final state with the $\tau$-lepton decaying into hadrons. No significant excess is observed, and upper limits on the LFV branching ratios are set. The observed and the median expected 95$\%$ CL upper limits on Br($H\rightarrow e\tau$) are $1.04$ $\%$ and $1.21^{+0.49}_{-0.34}$ $\%$, respectively. This direct search for the $H\rightarrow e\tau$ decays places significantly more stringent constraints on Br($H\rightarrow e\tau$) than earlier indirect estimates. In the search for the $H\rightarrow \mu \tau$ decays, the observed and the median expected 95$\%$ CL upper limits on Br($H\rightarrow \mu \tau$) are $1.43$ $\%$ and $1.01^{+0.40}_{-0.29}$ $\%$, respectively. A small deficit of data compared to the predicted background is observed in the search for the LFV $Z\rightarrow \mu \tau$ decays. The observed and the median expected 95$\%$ CL upper limits on Br($Z\rightarrow \mu \tau$) are $1.69\times 10^{-5}$ and $2.58\times 10^{-5}$, respectively.