1 Introduction

Measuring triboson final states at the Large Hadron Collider (LHC) [1] provides a test of the non-Abelian structure of the electroweak sector of the Standard Model (SM) of particle physics that predicts quartic gauge couplings. Deviations from the SM can be parametrised in the framework of anomalous quartic gauge couplings (aQGCs). This paper describes a measurement of \(WV \gamma \) production by analysing events containing a \(W\) boson, a vector boson (\(V\)), being either another \(W\)  boson or a \(Z\)  boson, and a photon, using proton–proton collisions at a centre-of-mass energy of \(\sqrt{s} = 8\,\text {TeV}\) corresponding to an integrated luminosity of 20.2 fb\(^{-1}\) recorded by the ATLAS detector [2].

At LEP, \(WW\gamma \) production was studied at centre-of-mass energies ranging from 183 to 207 \(\text {GeV}\) in a variety of photon plus leptonic or hadronic final states [3]. The analysis presented here has a higher energy reach than the results obtained at LEP. The production of \(WV \gamma \) events was studied by the CMS Collaboration in Ref. [4] in final states containing electrons or muons and jets, and using a data set with a similar luminosity and the same centre-of-mass energy as employed here. Other analyses with three bosons in the final state and also sensitive to quartic gauge couplings have been performed by the ATLAS and the CMS collaborations [5,6,7,8]. Furthermore, exclusion limits on new physics beyond the SM described by aQGCs have also been set at the LHC using diboson final states including photons [9,10,11] and in diboson final states including massive gauge bosons only [12,13,14,15,16,17].

In proton–proton collisions, \(WV \gamma \) events are produced through the \(WWZ\gamma \) and \(WW\gamma \gamma \) quartic couplings as depicted in Fig. 1a or through radiation of one or more bosons as exemplified in Fig. 1b, c. The fully leptonic final state (\(e\nu \mu \nu \gamma \)) of \(WW\gamma \) production containing an electron (e), a muon (\(\mu \)), their corresponding neutrinos (\(\nu \)), and a photon is studied as it has a clean experimental signature. The same-flavour final states, \(e\nu e\nu \gamma \) and \(\mu \nu \mu \nu \gamma \), are not studied as they have large backgrounds. Semileptonic final states (\(\ell \nu jj\gamma \)) containing one light lepton (\(\ell = e \text { or } \mu \)), a neutrino, two jets (j), and a photon are also studied. The analysis of the latter profits from the larger hadronic branching ratio of \(W\)- and \(Z\)-boson decays and is performed separately in the electron (\(e\nu jj\gamma \)) and the muon (\(\mu \nu jj\gamma \)) channels. The production of \(WV \gamma \) events whose decays include \(\tau \) leptons is not considered as signal.

Two fiducial regions are defined for all final states: one is optimised for the observation of the process while the other is optimised for a search for new physics beyond the SM. The results obtained in the latter region are interpreted in the context of aQGCs that describe modified triboson production using an effective field theory [18].

This paper is structured as follows. The ATLAS detector and the data employed in this analysis are described in Sect. 2. Section 3 details the Monte Carlo simulations used. The reconstruction of the detector information is outlined in Sect. 4. The analysis of the fully leptonic final state is described in Sect. 5 followed by the description of the semileptonic analysis in Sect. 6. In Sect. 7 the fiducial region of the cross-section measurement is defined and the determination of the production cross-section in the \(e\nu \mu \nu \gamma \) final state is described. The derivation of upper limits on the \(WV \gamma \) production cross-section is also presented. Section 8 discusses the cross-section exclusion limits in the fiducial region optimised for new physics beyond the SM and the interpretation of the results in the framework of aQGCs. A summary of the results is given in Sect. 9.

Fig. 1
figure 1

Examples of Feynman diagrams of \(WV \gamma \) production at the LHC. In a the quartic vertex is shown, while b, c depict the production from radiative processes

2 ATLAS detector and data sample

The ATLAS experiment [2] at the LHC is a multipurpose particle detector with a forward-backward symmetric cylindrical geometry and a near \(4\pi \) coverage in solid angle.Footnote 1 It consists of an inner tracking detector surrounded by a thin superconducting solenoid providing a 2 T axial magnetic field, electromagnetic and hadronic calorimeters, and a muon spectrometer. The inner tracking detector covers the pseudorapidity range \(|\eta | < 2.5\) and consists of silicon pixel, silicon microstrip, and transition radiation tracking detectors. Lead/liquid-argon (LAr) sampling calorimeters provide electromagnetic energy measurements with high granularity in the \(\eta \)\(\phi \) plane and a threefold segmentation in the radial direction. The first of the three layers of the LAr calorimeter has the smallest \(\eta \)-segmentation to discriminate between single photon showers and two overlapping showers coming from the decays of neutral hadrons. A hadronic (steel/scintillator-tile) calorimeter covers the central pseudorapidity range. The endcap and forward regions are instrumented with LAr calorimeters for the energy measurement of electromagnetic and hadronic showers up to \(|\eta | = 4.9\). The muon spectrometer encompasses the calorimeters and includes a system of precision tracking chambers as well as fast detectors for triggering. It comprises three large air-core toroidal superconducting magnets with eight coils each. The field integral of the toroids ranges between 2.0 and 6.0 Tm across most of the detector. A three-level trigger system is used to select events for read-out and storage. The first-level trigger is implemented in hardware and uses a subset of the detector information to reduce the accepted rate to 75 KHz. This is followed by two software-based trigger levels that together reduce the accepted event rate to 400 Hz on average.

This analysis uses data recorded at a centre-of-mass energy of \({8\,\text {TeV}}\), corresponding to an integrated luminosity of \(20.2\,\pm \,0.4\,\text {fb}^{-1}\) [19] after applying basic data quality criteria to ensure the full functionality of all detector subcomponents. Only events that have at least three reconstructed tracks [20] with \(p_{\text {T}} > 500\,\text {MeV}\) associated with the primary vertex are considered for analysis. The primary vertex is defined as the vertex whose associated tracks have the largest sum of squared transverse momenta. Furthermore, events are discarded if they contain jets that are likely to be mismeasured.

Dedicated triggers are used for each final state. The events of the fully leptonic analysis are triggered by requiring three particles in the event: a muon with a transverse momentum (\(p_{\text {T}}\)) of at least 18 \(\text {GeV}\) and two clusters of energy deposits in the electromagnetic calorimeter with a transverse energy (\(E_{\text {T}} \)) of at least 10 \(\text {GeV}\). The efficiency of this trigger for the selection of the signal described in Sect. 5 corresponds to \(0.82 \pm 0.01\)(stat.). For the semileptonic final states, a combination of single-lepton triggers [21] is used to maintain a high efficiency over a wide range of lepton transverse momenta. The \(e\nu jj\gamma \) final state is triggered by either requiring an isolated electron with \(p_{\text {T}} > {24\,\text {GeV}}\) or an electron with \(p_{\text {T}} > {60\,\text {GeV}}\) and no requirement on isolation. The lepton isolation is based on the sum of the transverse momenta of additional tracks in a cone of size \(\Delta R = 0.2\) around the lepton’s track. This trigger combination provides an efficiency of \(0.964 \pm 0.004\)(stat.) for the signal selection described in Sect. 6. Similarly, the \(\mu \nu jj\gamma \) final state is triggered by either requiring an isolated muon with \(p_{\text {T}} > {24\,\text {GeV}}\) or a muon with \(p_{\text {T}} > {36\,\text {GeV}}\) and no requirement on isolation. The efficiency of this trigger combination for the signal corresponds to \(0.772 \pm 0.007\)(stat.).

3 Monte Carlo simulations

The expected signal and background events were simulated with Monte Carlo (MC) event generators. The simulations were used to optimise the selection criteria, to compute efficiencies, and to estimate the contributions of specific background processes. For the simulation of the MC samples, the ATLAS simulation infrastructure [22], which uses the GEANT4 toolkit [23] for the detector simulation, was employed. All simulations described in this section were computed at leading order (LO) in the perturbative expansion of the strong-coupling constant (\(\alpha _{\text {S}}\)) unless otherwise stated.

The \(WV \gamma \) signal process was simulated with the MC event generator SHERPA 2.1.1 [24,25,26,27] with up to one additional parton in the matrix element, using the default tunes. The CT10NLO [28] set of parton distribution functions (PDF) was used. These signal predictions were normalised using the cross-sections of the fiducial regions introduced in Sect. 7, computed at next-to-leading order (NLO) in \(\alpha _{\text {S}}\) using the VBFNLO 2.7.1 [29,30,31,32] program and the CT14NLO [33] PDF set. The renormalisation and factorisation scales were set to the invariant mass of the triboson system. The \(WV \gamma \) processes that contain \(\tau \) leptons in their decay are considered as background in this analysis and were simulated like the signal as just described. For cross-checks and for the estimation of systematic uncertainties associated with the event generation, the \(WV \gamma \) signal process was also simulated using the MadGraph 5.2.2.2 [34] event generator with dynamical renormalisation and factorisation scales. It was interfaced to the PYTHIA 6.427 [35] program for the hadronisation and underlying event simulation with the Perugia 2012 [36] tune and used the CTEQ6L1 [37] PDF set. In addition, five reference samples modelling anomalous quartic gauge couplings were simulated for each studied final state, using the MadGraph event generator as described above and normalised using the corresponding cross-section predictions obtained at NLO with the VBFNLO program.

Backgrounds from WZ, ZZ, and \(Z\gamma \) diboson production were simulated with up to three additional partons in the final state using the SHERPA event generator (versions 1.4.1, 1.4.5, and 1.4.1 with the default tunes respectively) with the CT10NLO PDF set. Top quark pair production in association with a photon (\(t\bar{t}\gamma \)) was generated with the MadGraph 5.2.1.0 event generator using the CTEQ6L1 PDF set and interfaced to PYTHIA 8.183 [38] for the simulation of the hadronisation and the underlying event using the AUET2B [39] tune. The cross-section was normalised using the computations of Ref. [40] which were performed at NLO in \(\alpha _{\text {S}}\). The simultaneous production of top and antitop quarks (\(t\bar{t}\)) and the production of \(W\) bosons in association with top quarks (Wt) were generated at NLO in \(\alpha _{\text {S}}\) with the POWHEG-BOX [41,42,43] program using the CT10f4 PDF set and being interfaced to PYTHIA 6.426 with the Perugia 2011C [36] tune and using the CTEQ6L1 PDF set. The background from \(Z\) bosons produced in association with jets (\(Z\) + jets) and from \(W\)-boson production in association with a photon (\(W\gamma \) + jets) were generated with the ALPGEN [44] program interfaced to the HERWIG 6.520.2 [45] event generator for parton showering and hadronisation and to the JIMMY [46] event generator to simulate the underlying event. The AUET2 [47] tune and the CTEQ6L1 PDF set were employed. All simulations that used the PYTHIA event generator employed the TAUOLA [48] program to compute the \(\tau \) lepton decays. In samples that do not contain a prompt photon in the final state, the PHOTOS [49] program was employed to simulate photon radiation from final-state charged particles.

Contributions from additional proton–proton collisions accompanying the hard-scatter interaction, termed pile-up, were simulated using the PYTHIA 8.160 event generator. The resulting distribution of the mean number of interactions per bunch crossing was corrected to reproduce the distribution measured in data. The level of agreement between simulated and recorded data was further improved by correcting the simulated vertex distribution, object trigger and identification efficiencies, resolution and calibration to agree with the measured values [50,51,52].

4 Event reconstruction

The selection of the \(WV \gamma \) signal events is based on objects that are reconstructed using the same algorithms for simulated and recorded events. The reconstruction of electron and photon candidates employs energy clusters [53] of the calorimeters and their matching to tracks from the inner detector [50, 54]. The measured energies of the electrons and photons are corrected as described in Ref. [55]. Electron or photon candidates reconstructed within \(1.37< |\eta | < 1.52\) are discarded as this corresponds to a transition region between different calorimeter components which has poor energy resolution and identification efficiencies for these objects.

Photon candidates are reconstructed within \(|\eta | < 2.37\) and their transverse energy has to exceed 15 \(\text {GeV}\). They are required to fulfil the tight identification criteria described in Ref. [51]. An isolation requirement is applied to reject hadronic backgrounds: the additional transverse energy deposited in the calorimeter in a cone of size \(\Delta R = 0.4\) around the photon candidate, called \(E_{\text {T}}^{\text {iso}}\), must be less than 4 \(\text {GeV}\) after the median energy density of the event scaled to the cone size is subtracted in order to reduce the effect from pile-up [56].

Electron candidates are reconstructed within \(|\eta | < 2.47\) and their transverse momentum has to exceed 7 \(\text {GeV}\). They are required to fulfil the tight identification criteria described in Ref. [50]. In the fully leptonic analysis the same isolation requirement used for photons is applied to electrons as this facilitates the background estimation with the two-dimensional sideband method (see Sect. 5). The semileptonic analysis imposes a different isolation requirement, as it relies on other background estimation methods (see Sect. 6). For this analysis, the additional transverse energy deposited in the calorimeter in a cone of size \(\Delta R = 0.3\) around the electron is required to be less than 14% of the transverse energy of the electron after the pile-up energy is subtracted as for the photons. Furthermore, a track-based isolation requirement is imposed: the sum of the transverse momenta of the additional tracks in the aforementioned cone is required to be less than 7% of the transverse energy of the electron itself. In addition, the semileptonic analysis requires the electron track to be consistent with coming from the primary vertex.

Muon candidates are reconstructed within \(|\eta | < 2.4\) by combining tracks in the inner detector with tracks in the muon spectrometer. A statistical combination of the track parameters or a global refit of the tracks, described as Chain 3 in Ref. [52], is used. Muon candidates are required to have a transverse momentum larger than 7 \(\text {GeV}\) and to originate from the primary vertex. A track-based isolation requirement is imposed: the sum of the transverse momenta of the additional tracks in a cone of size \(\Delta R = 0.2\) around the muon candidate is required to be less than 10% of the transverse momentum of the muon candidate itself.

Jet candidates are reconstructed within \(|y| < 4.4\) from topological energy clusters [57] using the anti-\(k_{t}\) algorithm [58] with a radius parameter of \(R = 0.4\) implemented in the FastJet software package [59]. The measured energies of the jet candidates are corrected to the hadronic scale using the local cell signal weighting scheme [60] and their transverse momentum has to exceed 25 \(\text {GeV}\). For central jets (\(|\eta | < 2.4\)) with \(p_{\text {T}} < {50\,\text {GeV}}\), the scalar sum of the transverse momenta of tracks associated with the jet and originating from the primary vertex of the interaction is required to be at least 50% of the jet \(p_{\text {T}}\). This requirement suppresses jets originating from pile-up interactions [61].

The possible overlap between the object candidates is removed by applying the following requirements sequentially. Any electron that lies within a cone of size \(\Delta R = 0.1\) around a more energetic electron candidate or a muon candidate is discarded. Photon candidates are rejected if their angular distance to any remaining electron or muon is smaller than \(\Delta R = 0.5\). Apart from the removal of overlapping objects, this requirement also suppresses photons that are radiated from the lepton in the final state. Jets are discarded if they lie within a cone of size \(\Delta R = 0.3\) around an electron or \(\Delta R = 0.5\) around a photon candidate. Finally, muon candidates are rejected if their angular distance to a jet is smaller than \(\Delta R = 0.3\) in order to remove muons originating from heavy-flavour quark decays within jets.

The missing transverse momentum vector (\(\vec {p}_{\text {T}}^{\text {\,miss}}\)) of an event is a measure of the momentum imbalance in the transverse plane. It is calculated as the negative vector sum of the transverse momenta of calibrated leptons, photons, and jets, and additional tracks from the primary vertex that are not associated with any of those objects [62]. The missing transverse momentum (\(E_{\text {T}}^{\text {miss}}\)) is defined as the magnitude of \(\vec {p}_{\text {T}}^{\text {\,miss}}\).

The missing transverse momentum is employed for the definition of the selection criteria of the semileptonic analysis described in Sect. 6. In the fully leptonic analysis, described in Sect. 5, the relative missing transverse momentum (\(E_{\text {T,\,rel}}^{\text {miss}}\)) is used as this improves the signal significance. Its definition is based on the absolute azimuthal separation (\(\Delta \phi \)) of the object closest to \(\vec {p}_{\text {T}}^{\text {\,miss}} \):

$$ E_{\text {T,\,rel}}^{\text {miss}} = {\left\{ \begin{array}{ll} E_{\text {T}}^{\text {miss}} \times \sin (\Delta \phi ),&\, \text {if } \Delta \phi (\vec {p}_{\text {T}}^{\text {\,miss}},\text {closest object}) < \frac{\pi }{2}, \\ E_{\text {T}}^{\text {miss}},& \,\text {otherwise}. \end{array}\right. } $$
(1)

The transverse mass (\(m_{\text {T}}\)) is defined using \(E_{\text {T}}^{\text {miss}}\), the transverse momentum (\(p_{\text {T}}^{\ell }\)) of the most energetic lepton in the event and the absolute angular difference between \(\vec {p}_{\text {T}}^{\text {\,miss}}\) and this lepton (\(\Delta \phi (\vec {p}_{\text {T}}^{\text {\,miss}}, \ell )\)):

$$\begin{aligned} m_{\text {T}} = \sqrt{2 p_{\text {T}}^{\ell } E_{\text {T}}^{\text {miss}} [1- \cos (\Delta \phi (\vec {p}_{\text {T}}^{\text {\,miss}}, \ell ))]}. \end{aligned}$$
(2)

5 Analysis of fully leptonic final states

In the fully leptonic analysis, \(WW\gamma \) events are studied solely in the \(e\nu \mu \nu \gamma \) final state. Events where the two \(W\) bosons decay to leptons of the same flavour, i.e. \(e\nu e\nu \gamma \) or \(\mu \nu \mu \nu \gamma \) final states, have large backgrounds from Drell–Yan processes with photon radiation (\(Z\gamma \)) and do not increase the sensitivity of this measurement.

The event selection for the fully leptonic analysis requires the presence of exactly one electron and one muon with opposite electric charge, each with a transverse momentum of at least 20 \(\text {GeV}\), at least one reconstructed photon with \(E_{\text {T}} > {15\,\text {GeV}}\), and relative missing transverse momentum larger than 15 \(\text {GeV}\). Events containing a third reconstructed electron or muon with \(p_{\text {T}} > {7\,\text {GeV}}\) are discarded to suppress backgrounds from WW and WZ diboson production. For the rejection of Drell–Yan background decaying to \(\tau \) leptons, the invariant mass of the electron–muon pair is required to be larger than 50 \(\text {GeV}\). Finally, events containing any reconstructed jet with \(p_{\text {T}} > {25\,\text {GeV}}\) are discarded, thereby reducing background contributions from top-quark production. These selection requirements are optimised to yield the best sensitivity to the signal and define the signal region. The expected number of signal events is \(12.2 \pm 1.1\), as computed with the VBFNLO program and corrected for acceptance and efficiency effects (described in Sect. 7 along with the corresponding uncertainties). A total of 26 events are observed.

Table 1 Expected and observed event yields for the fully leptonic final state in the \(e\nu \mu \nu \gamma \) signal region. For each background process the corresponding estimation method is stated along with the resulting event yield. The quoted uncertainties include statistical and systematic uncertainties. The uncertainty in the total background expectation is symmetrised. The expected signal is computed with the VBFNLO program and corrected for acceptance and efficiency

Several processes are backgrounds to the fully leptonic \(WW\gamma \) signal; their contributions in the signal region are summarised in Table 1. The dominant source of background is the production of \(t\bar{t}\gamma \) events where the top quarks decay to \(W\) bosons and b-quarks with a leptonic decay of the \(W\) boson (\(t \rightarrow Wb \rightarrow \ell \nu b\)). This process mimics the signal when the jets have low energy or are produced in the forward direction (\(|y| \ge 4.4\)) and hence the jets are not reconstructed. Other subdominant backgrounds are \(Z\gamma \) events, which contribute when the \(Z\) boson decays to a pair of leptonically decaying \(\tau \) leptons, and \(WZ\gamma \) production, which can mimic the signal when one of the final state leptons does not fulfil the identification criteria or is not reconstructed due to the limited geometrical acceptance. Other backgrounds arise from \(WW\gamma \) production including \(\tau \) leptons and the production of Wt and ZZ events. The event yields of all these processes are estimated using MC simulation. The corresponding uncertainties include statistical and systematic uncertainties that are of similar size. The systematic uncertainties can be subdivided into experimental uncertainties and uncertainties from the theoretical calculation. The two components contribute equally to the uncertainty for most processes. The relative uncertainties from the theoretical calculation range from 5 to 22% [6, 40, 63,64,65,66]; the uncertainties associated with the computation of the \(WV \gamma \) process are described in Sect. 7. The experimental uncertainties include the energy scale and energy resolution uncertainties of the reconstructed objects [52, 55, 60, 67, 68], the uncertainties associated with the efficiencies of their reconstruction and identification [50, 52, 54], as well as uncertainties attributed to the simulation of the event pile-up [61]. The relative experimental uncertainties range from 5 to 32% with the largest contribution arising from the jet energy scale uncertainty which mainly contributes due to the requirement that the signal events should not contain reconstructed jets.

Events containing misidentified objects also constitute an important source of background. The background from WZ production where an electron is reconstructed as a photon (fake \(\gamma \) from e) is estimated by using MC simulation, where the rate of electrons being reconstructed as photons is corrected to better describe the data. This rate is determined by studying the decays of \(Z\) bosons to two electrons where one of the electrons is reconstructed as a photon and is below 6% for most of the pseudorapidity region. The uncertainty of this correction is small compared to the total uncertainty, which also includes the statistical uncertainty, uncertainties from the theoretical calculation, and experimental uncertainties as discussed in the previous paragraph.

The production of WW and \(t\bar{t}\) pairs in association with jets can mimic the signal if jets are misidentified as photons (fake \(\gamma \) from jets). Jets can also be misidentified as muons (fake \(\mu \) from jets) or electrons (fake e from jets) in which case \(W\gamma \) + jets events can fulfil the signal selection criteria. The contribution from events containing fake \(\mu \) from jets is determined from MC simulations and found to be very small. Events including fake \(\gamma \) from jets or fake e from jets are removed from the MC simulation, as their contribution is estimated with data. These contributions are estimated by combining two two-dimensional (2D) sideband methods [69] (one per background component). A schematical drawing of the interplay between the methods is given in Fig. 2. It shows the three background-enriched sideband regions (B\(_x\), C\(_x\), D\(_x\)) per fake-object category x (with \(x \in \{\gamma , e\}\)) along with the signal region (A) that is common to the two fake-object categories. In the sideband regions, the contribution from signal and other SM processes containing prompt photons is accounted for using MC estimates. The method relies on the assumption that the definition of the sideband regions uses uncorrelated observables. Then, the ratio \(\tau _{x}\) of the number of events in region C\(_x\) (\(N^{\text {fake}\,x}_{\text {C}_{x}}\)) to the number of events in region D\(_x\) (\(N^{\text {fake}\,x}_{\text {D}_{x}}\)) multiplied by the number of events in region B\(_x\) (\(N^{\text {fake}\,x}_{\text {B}_{x}}\)) can be used to estimate the number of events containing fake objects of category x in region A (\(N^{\text {fake}\,x}_{\text {A}}\)). A possible correlation of the observables is accounted for by introducing the correlation factor \(\rho _{x}\), which is set to one, representing no correlation, for the computation of the background contributions and varied to estimate the corresponding uncertainty.

The sideband regions B\(_{\gamma }\), C\(_{\gamma }\) and D\(_{\gamma }\) are defined using the photon isolation, \(E_{\text {T}}^{\text {iso,}\,\gamma }\), and a set of photon identification criteria related to the energy deposits in the first layer of the LAr calorimeter. The sideband regions B\(_{e}\), C\(_{e}\) and D\(_{e}\) are defined using the electron isolation, \(E_{\text {T}}^{\text {iso},\,e}\), and a set of electron–jet event selection criteria. The latter require the presence of at least one candidate electron and one jet with an absolute azimuthal separation of at least 0.7 in the event as well as \(m_{\text {T}} \le {30\,\text {GeV}}\) and, if there is a second lepton in the event, the invariant mass of the lepton pair, \(m_{\ell \ell }\), has to fulfilFootnote 2 \(|m_{\ell \ell }- m_{Z}| > {7\,\text {GeV}}\). The latter two criteria suppress the contribution of electrons originating from the decay of \(W\) and \(Z\) bosons, respectively.

Fig. 2
figure 2

Schematic drawing of the combination of the two 2D sideband methods to estimate the background from events containing fake \(\gamma \) (triangles) and fake e (squares) from jets. The \(WW\gamma \) events are indicated with filled circles. The figure shows the signal region (region A) along with the six sideband regions. In regions C\(_{\gamma }\) and D\(_{\gamma }\) the requirement on the electron isolation stays unchanged as does the requirement on the photon isolation in regions C\(_{e}\) and D\(_{e}\). The factors \(\tau _{\gamma }\) and \(\tau _{e}\) that relate the event count in the isolated and non-isolated fake-object regions are also shown. The contributions of SM background processes to the different regions are omitted for simplicity

As region A is common to the two fake-object categories, the estimation of the fake \(\gamma \) and fake e from jets contributions in the signal region is performed simultaneously using a maximum likelihood approach. The likelihood function is the product of the Poisson probabilities of observing the expected number of events in the seven regions multiplied by Gaussian functions that incorporate the systematic uncertainties as nuisance parameters. This function has seven free parameters: the number of signal events in the signal region (\(N^{e\nu \mu \nu \gamma }_{\text {obs}}\)), the ratios \(\tau _{\gamma }\) and \(\tau _{e}\) as well as \(N^{\text {fake}\,\gamma }_{\text {A}}\), \(N^{\text {fake}\,e\phantom {\gamma }}_{\text {A}}\), \(N^{\text {fake}\,\gamma }_{\text {C}_{\gamma }}\) and \(N^{\text {fake}\,e\phantom {\gamma }}_{\text {C}_{e\phantom {\gamma }}}\). These parameters are determined by maximising the likelihood function that is constrained using the number of observed events in the seven regions defined by the method.

Apart from providing the contribution of fake \(\gamma \) and fake e from jets in the signal region, the likelihood function also yields the most likely value of the number of signal events in the signal region: \(N^{e\nu \mu \nu \gamma }_{\text {obs}} = 9.4 \pm 6.2\). This value is consistent with the difference between the number of observed events and the total background prediction given in Table 1. The former is used for the determination of the fiducial cross-section in Sect. 7. Several sources of systematic uncertainty are taken into account. Varying the correlation factor \(\rho _{\gamma }\) (\(\rho _{e}\)) from one by its uncertainty \(\Delta \rho _{\gamma }^{MC} = \pm \ 0.44\) (\(\Delta \rho _{e}^{MC} = \pm \ 0.69\)) as extracted from the MC simulation expectation, yields a relative uncertainty in \(N^{e\nu \mu \nu \gamma }_{\text {obs}}\) of 10% (0.4%). The uncertainty in the number of events from SM processes in the sideband regions that are estimated from simulation is accounted for by varying the event yield by its total uncertainty and contributes 6% to the total uncertainty in \(N^{e\nu \mu \nu \gamma }_{\text {obs}}\). The uncertainty in estimating the number of signal events in the sideband regions contributes less than 1% to the total uncertainty. The dominant uncertainty in \(N^{e\nu \mu \nu \gamma }_{\text {obs}}\) originates from the limited number of data events and contributes a relative uncertainty of 60%.

Fig. 3
figure 3

Observed and expected transverse energy distribution of the photon with the highest \(E_{\text {T}}\) in the \(e\nu \mu \nu \gamma \) signal region. The data are shown together with the predicted signal and backgrounds. Also indicated is the expected event yield for a reference model describing aQGCs with \(f_{M,0}/\Lambda ^{4} = -1876\,\text {TeV}^{-4}\) (see Sect. 8). The last bin contains all overflow events. The lower panel shows the ratio of the observed number of events to the sum of expected signal and background events as well as the corresponding uncertainties

Figure 3 shows the transverse energy distribution of the photon with the highest \(E_{\text {T}}\) in the signal region. The data are shown together with the expected signal from the MC prediction and the results from the background estimation. Also shown is the predicted event yield for a reference point in the parameter space of aQGCs discussed in Sect. 8. The lower panel of the figure shows the ratio of the number of observed events to the sum of the expected signal and background events.

6 Analysis of semileptonic final states

In the semileptonic analysis, \(WV \gamma \) production with one leptonically decaying \(W\) boson and one hadronically decaying \(W\) or \(Z\) boson is studied. The event selection requires one lepton, at least two jets, at least one photon, and missing transverse momentum. The analysis is performed separately in the electron and the muon channels. The transverse momentum of the reconstructed electron or muon is required to be larger than 25 \(\text {GeV}\). Events containing additional reconstructed electrons or muons with \(p_{\text {T}} > {7\,\text {GeV}}\) are discarded. Photons are required to have \(E_{\text {T}} > {15\,\text {GeV}}\). Jets are required to have \(p_{\text {T}} > {25\,\text {GeV}}\) and to be within the volume of the tracking detector, \(|\eta | < 2.5\), to ensure that jets originating from heavy-flavour quarks can be identified. In addition, the two jets with the highest transverse momenta are required to be close together with \(|\Delta \eta _{jj}| < 1.2\) and \(\Delta R_{jj} < 3.0\) to reject backgrounds from \(W\gamma \) + jets events. The missing transverse momentum and the transverse mass of the event are both required to exceed 30 \(\text {GeV}\). In events containing electrons, the invariant mass of the electron–photon pair is required to differ from the value of the \(Z\) boson mass by at least 10 \(\text {GeV}\) to suppress backgrounds from events containing leptonically decaying \(Z\) bosons. To reduce background contributions from processes including top quarks, mainly \(t\bar{t}\gamma \), events containing jets that are identified as originating from the decay of a b-hadron are rejected. The b-jet identification is performed using the MV1 algorithm [71] based on an artificial neural network with an efficiency of 85% and a light-quark-jet and gluon-jet misidentification rate of 10%. Finally, the invariant mass of the two jets with the highest transverse momenta in the event is required to be close to the mass of the decaying \(W\) or \(Z\) boson, i.e. \({70\,\text {GeV}}< m_{jj} < {100\,\text {GeV}}\). These selection requirements are optimised to yield the best sensitivity to the signal and define the signal region. The expected number of signal events is \(14 \pm 2\) (\(18 \pm 2\)) in the electron (muon) channel, as computed with the VBFNLO program and corrected for acceptance and efficiency effects (described in Sect. 7 along with the corresponding uncertainties). A total of 490 (599) events are observed in the electron (muon) channel.

Table 2 Expected and observed event yields in the signal region of the electron and muon channels of the semileptonic analysis. For each background process the corresponding estimation method is stated. The uncertainties of the \(W\gamma \) + jets, fake \(\gamma \) from jets and fake \(\ell \) from jets are solely the statistical uncertainties from data. The uncertainties of the \(t\bar{t}\gamma \), fake \(\gamma \) from e, \(Z\gamma \) + jets and \(WV \gamma \rightarrow \tau \nu jj \gamma \) backgrounds correspond to the sum in quadrature of the statistical uncertainty of the MC simulation and the uncertainties of the theoretical prediction. The uncertainty in the total background estimate is symmetrised and contains the statistical uncertainty of the data, the uncertainties of the theoretical prediction, and experimental uncertainties. The expected signals are computed with the VBFNLO program and corrected for acceptance and efficiency

The background processes of the semileptonic analysis are listed in Table 2. The dominant contribution arises from \(W\gamma \) + jets production, as it has the same final state as the signal. The contribution from \(t\bar{t}\gamma \), \(Z\gamma \) + jets as well as from \(WV \gamma \) processes containing \(\tau \) leptons (\(WV \gamma \rightarrow \tau \nu jj \gamma \)) processes, is estimated using MC simulation. The uncertainties in these background contributions given in Table 2 solely include statistical uncertainties and the uncertainties of the theoretical prediction, that are of the same size. The relative uncertainties of the theoretical predictions range from 4 to 22% [6, 40]; the uncertainties associated with the computation of the \(WV \gamma \) process are described in Sect. 7. The experimental uncertainties are only included in the uncertainty of the total background estimation in Table 2, as they are correlated for the individual background components.

Events containing misidentified objects constitute an important source of background in this analysis as well. When electrons are misidentified as photons (fake \(\gamma \) from e), \(Z \rightarrow ee\) production in association with jets and \(t\bar{t}\) events can mimic the signal. As in the fully leptonic analysis, this background is estimated using MC simulation which is corrected to match the misidentification rate measured in data. The uncertainty of this correction is small compared to the statistical uncertainty and the uncertainties from the theoretical calculation. The latter uncertainty is estimated to be 5% for the \(Z \rightarrow ee\) and the \(t\bar{t}\) processes in agreement with the corresponding measurements [72, 73]. Mainly events from \(W \ +\) jets production contribute as background when a jet is misidentified as a photon (fake \(\gamma \) from jets). In events containing jets misidentified as leptons (fake \(\ell \) from jets) predominantly \(\gamma \ +\) jets production constitues a background. Events containing fake \(\gamma \) from jets or fake \(\ell \) from jets are removed from the MC simulation, as their contribution is estimated with data.

A simultaneous fit is used to estimate the background contributions from \(W\gamma \) + jets production and from events containing fake \(\gamma \) from jets and fake \(\ell \) from jets (the fake e from jets component also includes the small contribution from fake e from \(\gamma \)). The simultaneous fit consists of three components: a binned extended maximum-likelihood fit of the invariant dijet mass distribution to constrain the \(W\gamma \) + jets contribution, a binned extended maximum-likelihood fit of the \(E_{\text {T}}^{\text {miss}}\) distribution to constrain the fake \(\ell \) backgrounds and a two-dimensional sideband method to constrain the contribution from fake \(\gamma \) from jets. The free parameters of the simultaneous fit are the normalisation of the \(W\gamma \) + jets background, the normalisation of the processes containing fake \(\ell \) from jets and the normalisation of the processes containing fake \(\gamma \) from jets. The normalisation of all other background components is fixed. The fit is performed separately in the electron and muon channels of the analysis. For all three estimation methods the signal region with \({70\,\text {GeV}}< m_{jj} < {100\,\text {GeV}}\) is excluded such that the overall signal contribution to the fiducial region used for the background estimation is negligible. Therefore, the signal contribution in all regions used in the fit is neglected and the result is independent of the signal modelling. The \(m_{jj}\) distribution is fitted in the range 10–70 and 100–505 \(\text {GeV}\); the \(E_{\text {T}}^{\text {miss}}\) distribution is fitted in the range 0–300 \(\text {GeV}\). No minimum \(E_{\text {T}}^{\text {miss}}\) requirement is imposed in the fit of the \(E_{\text {T}}^{\text {miss}}\) distribution, in order to increase the sensitivity to fake \(\ell \) from jets, as these events are expected to have low missing transverse momentum. Apart from neglecting the signal contribution, the two-dimensional sideband method is performed as for the fake photons from jets in the fully leptonic analysis.

The extended likelihood fits employ shape templates for the \(m_{jj}\) and \(E_{\text {T}}^{\text {miss}}\) distributions of the different background components. The shape templates for all backgrounds are derived from simulation apart from the ones associated with fake \(\ell \) from jets and fake \(\gamma \) from jets. The latter shape templates are obtained from data events selected similarly to the fit regions with some requirements modified as follows to enhance the contribution from the respective fake object. To estimate the shape template for fake e from jets, the requirement on \(E_{\text {T}}^{\text {miss}}\) is removed and the requirements on the electron identification and isolation are modified. To this end, the requirements on the calorimeter-based isolation and the origin of the electron track are removed and the track-based isolation requirement is inverted. To estimate the shape template for fake \(\mu \) from jets, the requirement on \(E_{\text {T}}^{\text {miss}}\) is removed and the requirements on the muon isolation and the origin of the track are inverted. To estimate the shape template for fake \(\gamma \) from jets, the requirement on the photon isolation is removed and at least one of the photon identification criteria based on the energy deposits in the first layer of the LAr calorimeter must not be satisfied. The \(m_{jj}\) shape templates are also employed to extrapolate the background estimation results of the different background components to the signal region.

Fig. 4
figure 4

Missing transverse momentum and dijet invariant mass distributions of the electron (upper row) and the muon channels (lower row) of the semileptonic analysis. The different background components are shown together with the data. The signal region (\({70\,\text {GeV}}< m_{jj} < {100\,\text {GeV}}\)) is excluded in (a) and (c) as well as in the simultaneous fit as indicated by the arrows in (b) and (d). The last bin of each figure contains the event overflow. The lower panels show the ratio of the observed number of events to the predicted background as well as the corresponding uncertainties. The red arrows indicate entries that are outside the y-axis range

Figure 4 shows the results of the simultaneous fit, in the upper panel for the electron channel and in the lower panel for the muon channel. In Fig. 4a, c the resulting \(E_{\text {T}}^{\text {miss}}\) distributions are presented; the events are selected using the criteria for the signal region, but the requirement on \(E_{\text {T}}^{\text {miss}}\) is removed and the requirement on \(m_{jj}\) is inverted. The lower panels of the figures show the ratio of the observed number of events to the expected number of events, which agrees with unity within uncertainties. In Fig. 4b, d the resulting \(m_{jj}\) distributions are shown. All signal selection requirements apart from the \(m_{jj}\) requirement are imposed. The distribution observed in data is underestimated by the background estimation in both channels at low \(m_{jj}\) values but agrees within uncertainties. As a cross check, an alternative shape template for the \(W\gamma \) + jets background is obtained from simulated events generated with SHERPA. While the resulting background estimate shows better agreement with the data at low values of \(m_{jj}\), no significant impact on the background estimate in the signal region is found. The event yields of the \(W\gamma \) + jets, fake \(\gamma \) from jets and fake \(\ell \) from jets events in the signal region are given in Table 2. The uncertainties in these components in Table 2 correspond solely to the statistical uncertainty from data.

The uncertainty in the total number of background events has several sources. The uncertainty associated with the shape templates is estimated by performing 10,000 pseudo experiments that use alternative shape templates obtained from sampling the nominal ones bin-wise using a Gaussian distribution. The width of the Gaussian distribution corresponds to the statistical uncertainty of the shape templates determined from data, or to the statistical uncertainty of the MC simulation and the uncertainties from the theoretical calculation if they are determined from simulation. The shape templates are varied simultaneously and yield an uncertainty in the total background of 5% (4%) in the electron (muon) channel. The experimental uncertainties are the uncertainties due to reconstruction and identification efficiencies of the objects [50, 52, 54, 74, 75] including energy scale and energy resolution uncertainties [52, 55, 60, 67, 68] as well as uncertainties arising from the simulation of the event pile-up [61]. These uncertainties are estimated for all background components simultaneously and amount to a total of 4 (3%) in the electron (muon) channel. They are dominated by the uncertainty in the jet energy scale. The uncertainty related to the choice of fit boundaries for the extended maximum-likelihood fits is estimated by varying these boundaries. The lower \(m_{jj}\) (\(E_{\text {T}}^{\text {miss}}\)) boundary is set to 25 (15 \(\text {GeV}\)) and the upper boundary is set to 490 or 520 \(\text {GeV}\) (285 or 315 \(\text {GeV}\)) independently. The uncertainty introduced by the choice of binning for the distributions used for the extended maximum-likelihood fits is estimated by varying the bin sizes by a factor of two. The uncertainty due to the possible correlation of the selection criteria defining the sideband regions of the 2D sideband method is estimated by changing the value of the correlation factor \(\rho \) from one by its uncertainty \(\Delta \rho _{e\nu jj \gamma }^{MC} = \pm \ 0.38\) (\(\Delta \rho _{\mu \nu jj \gamma }^{MC} = \pm \ 0.23\)) as extracted from the MC simulation expectation. The uncertainty associated with any of these fit parameter variations is less than 1% in each channel of the analysis. The statistical uncertainty in the expected total number of background events corresponds to 2.6 (2.5%) in the electron (muon) channel.

Fig. 5
figure 5

Observed and expected transverse energy distributions of the photon with the highest \(E_{\text {T}}\) in the signal region in the a electron and b muon channels of the semileptonic analysis. The data are shown together with the predicted signal and backgrounds. Also indicated is the expected event yield for a reference model describing aQGCs with \(f_{T,0}/\Lambda ^{4} = 1374\,\text {TeV}^{-4}\) (see Sect. 8). The last bin of each figure contains all overflow events. The lower panels show the ratio of the observed number of events to the sum of expected signal and background events as well as the corresponding uncertainties

Figure 5 shows the transverse energy distributions of the photon with the highest \(E_{\text {T}}\) in the signal region in the electron and the muon channels. The data are shown together with the estimated background contributions and the expected signal yield. The expected distribution for a reference point in the parameter space of aQGCs (see Sect. 8) is also indicated. The lower panels of the two figures show the ratios of the number of observed events to the sum of expected signal and background events.

7 Production cross-section

The cross-section for \(WV \gamma \) production is determined in fiducial regions close to the signal regions defined in Sects. 5 and 6. While the signal region definition is based on reconstructed objects, the definition of the fiducial region is based on particle-level MC generator information. The latter corresponds to the MC simulation including the parton shower, hadronisation and underlying event, as opposed to the parton level, which does not account for these effects and solely includes the hard-scattering process of the event.

At particle level, jets are reconstructed from all stable particles (traveling at least 10 mm before decaying) in the final state, except for muons and neutrinos, using the anti-\(k_{t}\) algorithm with \(R = 0.4\). The identification of b-jets at particle level is based on a matching of the jets to b-hadrons within a cone of size \(\Delta R = 0.3\) around the jet axis. The final-state radiation of photons from leptons is accounted for by adding the four-momenta of photons that lie within a cone of size \(\Delta R = 0.1\) around a lepton to the lepton four-momentum. The missing transverse momentum of a particle-level event is obtained from the momenta of the neutrinos in the final state.

The selection criteria defining the fiducial region are summarised in Table 3. They differ from the criteria defining the signal region only for the requirements on the pseudorapidity range and the isolation of the objects. Leptons are required to fulfil \(|\eta | < 2.5\) and photons \(|\eta | < 2.37\). Thus, the transition region (\(1.37< |\eta | < 1.52\)) is included in the fiducial region and the \(\eta \) requirements of the electrons and muons are unified. No isolation requirements are imposed on electrons or muons. The photon isolation requirement is based on the isolation fraction \(\epsilon _{h}^{p}\). The latter is defined as the ratio of the transverse energy of the closest jet that lies within a cone of size \(\Delta R = 0.4\) around the photon to the transverse energy of the photon. Photons are considered isolated when \(\epsilon _{h}^{p} < 0.5\).

Table 3 Definition of the fiducial regions of the fully leptonic and semileptonic \(WV \gamma \) analyses. The objects are defined at particle level and the \(\Delta R\) requirements are employed in the overlap removal. The latter is implemented differently for electrons and muons. For electron–jet pairs failing the \(\Delta R(\text {jet}, \ell )\) requirement, the jet candidate is discarded and for muon–jet pairs failing the requirement, the muon candidate is discarded

7.1 Cross-section predictions

The cross-section predictions are computed at NLO in \(\alpha _{\text {S}}\) using the VBFNLO program. The computations are performed at parton level, while the measurement is performed at particle level. Therefore, the cross-section predictions are corrected to particle level by multiplying them by the parton-to-particle-level correction factors (\(C^{\text {p2p}}\)). Each correction factor is defined as the number of signal events that satisfy the selection criteria for the fiducial region defined at particle level divided by the number of signal events that satisfy the selection criteria for the fiducial region defined at parton level. These factors are evaluated using the SHERPA signal simulation and amount to \(1.10 \pm 0.01\), \(0.64 \pm 0.01\) and \(0.57 \pm 0.02\) for the \(e\nu \mu \nu \gamma \), \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states, respectively. The main difference between these corrections for the fully leptonic and the semileptonic final states arises from the fundamentally different requirements on the presence of jets and partons in the events. The difference between the electron and muon channels in the semileptonic analysis arises from different overlap removal algorithms employed for electrons and muons; while jet candidates are discarded when they are close to electrons, muon candidates are discarded when they are reconstructed close to a jet, to remove contributions from heavy-flavour quark decays. The uncertainties of the parton-to-particle-level correction factors include the statistical uncertainty of the SHERPA sample and a systematic component evaluated as the difference between the corrections estimated with the SHERPA and the MadGraph signal samples. The latter uncertainty accounts for differences in the parton shower modelling and the description of the underlying event between the two generators. The expected cross-section at particle level for the different final states and for the average of the electron and muon channels of the semileptonic analysis (\(\ell \nu jj\gamma \)) are summarised in Table 4. The expected cross-sections for the fully leptonic and semileptonic final states are of similar size despite the larger hadronic branching fraction of the \(W\) and \(Z\) bosons, as the selection criteria for the fiducial regions in the semileptonic analysis are more restrictive. The uncertainty in the expected cross-section is about 5% for all final states. This value accounts for the uncertainty associated with \(C^{\text {p2p}}\), the numerical accuracy of the calculation, variations of the renormalisation and factorisation scales (\(\mu _{\text {R}}\) and \(\mu _{\text {F}}\)) by a factor of two (varied independently with the constraint \(0.5 \le \mu _{\text {F}}/\mu _{\text {R}} \le 2\)), uncertainties due to the choice of PDF set and value of the strong coupling constant \(\alpha _{\text {S}}\) as well as uncertainties due to the choice of isolation fraction requirement evaluated by changing the criterion by \(\pm \,0.25\). No additional uncertainty related to the scale introduced by restricting the jet multiplicity in the fully leptonic analysis is taken into account. This uncertainty has been shown to be of the same order as the already included scale uncertainty by studying \(W\)-boson pair production [76]. Accordingly, no additional uncertainty is considered here as the experimental uncertainties are comparatively large and its inclusion would not change the results of this analysis.

7.2 Cross-section determination

The observed production cross-section is determined from the number of signal events in the signal region, \(N_{\text {obs}}\), and the integrated luminosity of the data set, \(L_{\text {int}}\), according to \(\sigma _{\text {fid}} = N_{\text {obs}}/(\epsilon L_{\text {int}}),\) where the correction factor, \(\epsilon \), accounts for the different geometrical acceptance and selection efficiencies of the signal region defined using reconstructed objects and the fiducial region defined at particle level. The correction factor is evaluated using the SHERPA signal simulation and amounts to \(0.30 \pm 0.02\) for the \(e\nu \mu \nu \gamma \) final state and to \(0.28 \pm 0.02\) (\(0.40 \pm 0.03\)) for the electron (muon) channel of the semileptonic analysis. The larger ranges in pseudorapidity of the leptons and photons in the fiducial region compared to the signal region contribute about 11% to \(\epsilon \). The uncertainties of \(\epsilon \) include the experimental uncertainties associated with the signal, a statistical component, and a systematic component evaluated as the difference between the corrections estimated with the SHERPA and the MadGraph signal sample to account for differences in the parton shower modelling and the description of the underlying event. The latter yields the largest contribution to the total uncertainty with the second largest contribution being the uncertainty associated with the jet energy scale.

For the fully leptonic analysis, the fiducial cross-section computed using \(N^{e\nu \mu \nu \gamma }_{\text {obs}}\) from Sect. 5 is

$$\begin{aligned} \sigma _{\text {fid}}^{e\nu \mu \nu \gamma } = 1.5 \pm 0.9 (\text {stat.}) \pm 0.5 (\text {syst.})\,\text {fb}, \end{aligned}$$

where the uncertainties are symmetrised and the luminosity uncertainty is included as part of the systematic uncertainty. The observed (expected) significance of this cross-section is determined by evaluating the p value of the background-only hypothesis at 95% confidence level, CL, and corresponds to 1.4 \(\sigma \) sigma (1.6 \(\sigma \)). The p value is calculated using a maximum likelihood ratio as the test statistic. This determination of the \(e\nu \mu \nu \gamma \) production cross-section is in agreement with the theory prediction from Table 4 corresponding to 2.0 fb. The cross-section is not determined in the semileptonic final states due to its smaller significance.

Upper limits on the production cross-sections are computed for the \(e\nu \mu \nu \gamma \), \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states and for the average cross-section per lepton flavour (\(\ell \nu jj\gamma \)) in the semileptonic final states. They are determined at 95% CL using the CL\(_{\text {s}}\) technique [77]. For the combination of the semileptonic final states, the product of the likelihood functions of the \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states is used as the \(\ell \nu jj\gamma \) likelihood function in the CL\(_{\text {s}}\) method. The expected limits in the absence of a signal are computed using an Asimov data set [78], which provides an analytical approximation of the distribution of expected limits based on a \(\chi ^2\)-distribution of the test statistics. The observed and expected limits are listed in Table 4. The observed limits are between 1.8 and 4.1 times larger than the SM cross-section. The observed upper limit on the \(\ell \nu jj\gamma \) production cross-section is the most stringent limit reported to date.

Table 4 Observed and expected cross-section upper limits at 95% CL for the different final states using the CL\(_{\text {s}}\) method. The expected cross-section limits are computed assuming no signal is present. The last column shows the theory prediction for the signal cross-section (\(\sigma _{\text {theo}}\)) computed with the VBFNLO program and corrected to particle level. The \(\ell \nu jj\gamma \) cross-section corresponds to the average cross-section per lepton flavour in the semileptonic analysis and all events of the \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states are employed for the determination of this limit

8 Search for new physics beyond the Standard Model

In addition to the results derived in the previous chapter, exclusion limits on the production cross-section and conficence intervals on aQGCs are derived in a fiducial region optimised for a search for new physics beyond the SM. This fiducial region differs from the fiducial region defined in Sect. 7 by an increased photon \(E_{\text {T}}\) requirement.

The aQGCs are introduced by extending the SM Lagrangian density function (\(\mathcal {L}_{\text {SM}}\)) with terms containing operators (\(\mathcal {O}_{x}\)) of energy-dimension eight as this is the lowest dimension that describes quartic gauge boson couplings without exhibiting triple gauge-boson vertices [79]. The operators consist of different combinations of the SM fields and their coefficients are written as the ratio of a coupling parameter (\(f_{x}\)) to the fourth power of the energy scale (\(\Lambda \)) at which the new physics beyond the SM would occur. Thus, the effective Lagrangian density (\(\mathcal {L}_{\text {eff}}\)) for \(WV \gamma \) production can be written as:

$$\begin{aligned} \mathcal {L}_{\text {eff}} = \mathcal {L}_{\text {SM}} + \sum _{j=0}^{7} \frac{f_{M,j}}{\Lambda ^{4}}\mathcal {O}_{M,j} + \sum _{j=0,1,2,5,6,7} \frac{f_{T,j}}{\Lambda ^{4}}\mathcal {O}_{T,j}, \end{aligned}$$
(3)

as there are 14 different operators that describe anomalous \(WWZ\gamma \) and \(WW\gamma \gamma \) couplings. The indices T and M of the coupling parameter indicate two different classes of aQGC operators: operators containing only field strength tensors (T) and operators containing field strength tensors and the covariant derivative of the Higgs field (M). The SM prediction of each of the coupling parameters is zero. The reference models in Figures 3 and 5 depict values that are excluded by previous analyses.

The effective field theory is not a complete model and violates unitarity at sufficiently high energy scales. This violation can be avoided by multiplying the coupling parameters with a dipole form factor of the form:

$$\begin{aligned} \frac{1}{(1 + \hat{s}/\Lambda ^2_{\text {FF}})^{2}}, \end{aligned}$$
(4)

as described in Ref. [80]. Here, \(\hat{s}\) corresponds to the squared invariant mass of the produced bosons and \(\Lambda _{\text {FF}}\) is the energy scale of the form factor. The latter corresponds to the energy regime above which the contributions of the anomalous couplings are largely suppressed. For triboson processes there is no theoretical algorithm to compute the appropriate value for \(\Lambda _{\text {FF}}\) to avoid unitarity violation. Therefore, the confidence intervals in this analysis are derived using three different values of \(\Lambda _{\text {FF}}\): 0.5, 1 \(\text {TeV}\) and infinity. The latter corresponds to the non-unitarised case, which is evaluated to allow for the comparison with other analyses.

For the determination of the confidence intervals, only one coupling parameter is varied at a time and all others are set to zero. The expected number of events as a function of the varied parameter is described by a quadratic function and the predictions of the VBFNLO program corrected to particle level are used for the determination of this function. Confidence intervals at 95% CL are computed using a maximum profile-likelihood ratio test statistic as done in Ref. [69].

Table 5 Numbers of observed events (\(N_{\text {obs}}\)) and predicted background events (\(N_{\text {bg}}\)) for the different final states with the respective photon \(E_{\text {T}}\) threshold optimised for maximal aQGC sensitivity. Also given are the correction factors \(\epsilon \) to correct from reconstruction level to particle level and \(C^{\text {p2p}}\) to correct from parton level to particle level

The aQGCs would modify \(WV \gamma \) production at high values of \(\hat{s}\) such that the sensitivity to aQGCs can be improved by raising the threshold of the transverse energy of the photon. As the event count in the signal region decreases with an increasing \(E_{\text {T}} ^{\gamma }\) threshold, the expected background contribution from the other processes is extrapolated from the results obtained in Sects. 5 and 6 with \(E_{\text {T}} ^{\gamma } > {15\,\text {GeV}}\). To this end, the \(E_{\text {T}} ^{\gamma }\) distribution of the total background prediction is fitted using an exponential function (the sum of two exponential functions) in the fully leptonic (semileptonic) analysis and the total background yield is derived from the fit. The optimal value of the \(E_{\text {T}} ^{\gamma }\) threshold is determined by varying the threshold, computing the expected confidence intervals for all 14 parameters and choosing the threshold that yields the smallest expected intervals for each final state individually. This optimisation yields the best sensitivity for the requirement \(E_{\text {T}} ^{\gamma } > {120\,\text {GeV}}\) in the fully leptonic analysis and for \(E_{\text {T}} ^{\gamma } > {200\,\text {GeV}}\) in both channels of the semileptonic analysis.

The number of observed events and the expected number of background events above the optimised \(E_{\text {T}} ^{\gamma }\) threshold are given in Table 5. The uncertainty in the background estimation includes the uncertainty in the original background estimation and an additional uncertainty due to the extrapolation procedure, which is dominant. The latter is evaluated by varying the fit range as well as evaluating the impact of the uncertainty of the fit parameters on the background estimation. Due to the higher \(E_{\text {T}} ^{\gamma }\) threshold, the factors \(\epsilon \) and \(C^{\text {p2p}}\) are recomputed using the SM signal samples and are also listed in Table 5. As an additional source of systematic uncertainty, \(\epsilon \) and \(C^\mathrm {p2p}\) are evaluated using the aQGC simulated samples, and their maximal deviations from the SM predictions are considered to account for their dependence on the aQGC coupling. This uncertainty is the dominant one for \(C^{\text {p2p}}\) in the fully leptonic analysis.

The upper limits on the \(WV \gamma \) production cross-section in the high-\(E_{\text {T}}\) photon fiducial region are computed using the CL\(_{\text {s}}\) formalism at 95% CL. The results are given in Table 6 together with limits expected in absence of \(WV \gamma \) production. In addition, the theory prediction for the SM signal cross-section computed with the VBFNLO program and corrected to particle level is reported. The cross-section uncertainties are evaluated as described in Sect. 7.1 and range up to 22%.

Table 6 Observed and expected cross-section upper limits at 95% CL using the CL\(_{\text {s}}\) method for the different final states with the photon \(E_{\text {T}}\) threshold optimised for maximal aQGC sensitivity. The expected cross-section limits are computed assuming the absence of \(WV \gamma \) production. The last column shows the theory prediction for the SM signal cross-section computed with the VBFNLO program and corrected to particle level. The \(\ell \nu jj\gamma \) cross-section corresponds to the average cross-section per lepton flavour in the semileptonic analysis and all events of the \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states are employed for the determination of this limit

For the computation of the confidence intervals, the \(e\nu \mu \nu \gamma \), \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states are combined. The test statistic is computed from the product of the likelihood functions of the individual final states. This combination improves the confidence intervals by up to 11% compared to the results obtained with the \(e\nu \mu \nu \gamma \) final state only. The results are given in Table 7. In Fig. 6 the expected and observed confidence intervals using the form factor scale \(\Lambda _{\text {FF}} = 1\,\text {TeV}\) are shown. The non-unitarised couplings have also been studied by other analyses (e.g. [5,6,7,8,9,10,11,12,13, 17]) and found to be consistent with the SM prediction of zero as confirmed by this analysis.

Table 7 Observed and expected confidence intervals at 95% CL on the different anomalous quartic gauge couplings for the combined \(WV \gamma \) analysis for three different values of the form factor scale \(\Lambda _{\text {FF}}\)
Fig. 6
figure 6

Observed and expected confidence intervals at 95% CL on the different anomalous quartic gauge couplings for the combined \(WV \gamma \) analysis. The couplings are unitarised using a dipole form factor with a form factor energy scale of \(\Lambda _{\text {FF}} = {1\,\text {TeV}}\)

9 Conclusion

The production of \(WV \gamma \) events is studied in \(e\nu \mu \nu \gamma \), \(e\nu jj\gamma \) and \(\mu \nu jj\gamma \) final states using 20.2 fb\(^{-1}\) of proton–proton collisions at a centre-of-mass energy of \(\sqrt{s} = {8\,\text {TeV}}\) recorded with the ATLAS detector at the LHC. The fiducial production cross-section of the \(e\nu \mu \nu \gamma \) final state is determined with a significance of 1.4 \(\sigma \) (1.6 \(\sigma \) expected) and good agreement with the SM prediction at NLO in \(\alpha _{\text {S}}\) is observed. Furthermore, upper limits on the production cross-section are derived for the \(e\nu \mu \nu \gamma \), \(e\nu jj\gamma \), \(\mu \nu jj\gamma \) and \(\ell \nu jj\gamma \) final states in two fiducial regions: one optimised for the measurement of the process and one optimised for a search for new physics beyond the SM. No deviation from the SM predictions is observed and the results are interpreted in the framework of an effective field theory. Confidence intervals are derived with and without unitarisation for all 14 parameters of anomalous quartic gauge couplings this analysis is sensitive to.