Automorphism Groups of Certain Enriques Surfaces

Abstract

We calculate the automorphism group of certain Enriques surfaces. The Enriques surfaces that we investigate include very general n-nodal Enriques surfaces and very general cuspidal Enriques surfaces. We also describe the action of the automorphism group on the set of smooth rational curves and on the set of elliptic fibrations.

1 Introduction

A central theme in algebraic geometry is to study varieties using convex geometry. The cone of curves of a variety is the convex hull of the numerical equivalence classes of curves. Its dual is the cone of nef line bundles. Much of the birational geometry of a variety is encoded in these cones and their interplay with the canonical divisor. While for Fano varieties the nef cone is rational polyhedral [15, Theorem 3.7], in general the nef cone is not well understood. For instance it can have infinitely many faces or be round.

The Morrison–Kawamata cone conjecture [12, 20] gives a clear picture of the effective nef cone of a Calabi–Yau variety. It predicts that the action of the automorphism group on the effective nef cone admits a fundamental domain which is a rational polyhedral cone.

The conjecture is wide open in dimension three and beyond [18]. But it has been verified for K3 surfaces by Sterk [33], and for Enriques surfaces by Namikawa [21]. It follows that an Enriques surface admits up to the action of the automorphism group only finitely many smooth rational curves, finitely many elliptic fibrations, finitely many projective models of a given degree and its automorphism group is finitely generated and in fact finitely presented [19, Corollaries 4.15, 4.16].

Naturally, enumerative questions arise:

• Can one explicitly describe a fundamental domain?

• How many smooth rational curves, elliptic fibrations or projective models are there up to the action of the automorphism group?

• Can one give generators for the automorphism group?

Barth and Peters [2] noted that very general Enriques surfaces do not contain smooth rational curves. Hence their nef cone is round—it is the entire positive cone, and they proceed to answer the three questions for very general Enriques surfaces.

Enriques surfaces containing a smooth rational curve are called nodal. They form a subset of codimension one in the moduli space of Enriques surfaces. Very general nodal Enriques surfaces are treated by Cossec–Dolgachev [8] (see also the works of Allcock [1] and Peters and Sterk [25]).

When an Enriques surface is deformed to one containing more rational curves several phenomena working against each other occur. On the one hand the nef cone gets smaller and on the other hand the automorphism group may change drastically. Barth and Peters [2, p. 395] write that they do not know whether one can control these effects. Albeit the behaviour of the nef cone and the automorphism group may be erratic, the cone conjecture promises that the fundamental domain on the nef cone stays of finite volume at least. Our first main result (Theorem 3.4) states that we can control the (change of) volume in a precise way under mild assumptions.

To generalize the aforementioned results of Barth, Peters, Cossec and Dolgachev to Enriques surfaces with more nodes, we introduce the notion of $(\tau ,{\bar{\tau }})$-generic Enriques surfaces, which is closely related to the root invariant introduced by Nikulin [24]. See the next subsection for the precise definition. For instance the very general Enriques surface is (0, 0)-generic, a very general nodal Enriques surface is $(A_1,A_1)$-generic and if Y is an Enriques surface that is very general in the moduli of Enriques surfaces containing n disjoint smooth rational curves, then Y is $(nA_1, nA_1)$-generic. If Y is very general in the moduli of Enriques surfaces containing two smooth rational curves whose dual graph is (that is, Y is a very general cuspidal Enriques surface), then Y is $(A_2, A_2)$-generic.

Next we give algorithms to compute generators for the automorphism group ${\mathrm {Aut}}(Y)$, a fundamental domain for ${\mathrm {Aut}}(Y)$ on the nef and big cone $\mathrm{Nef}(Y)$ and orbit representatives for its action on

$$\begin{aligned} {{\mathcal {R}}}(Y):= & {} \text { the set of smooth rational curves on } Y, \\ {{\mathcal {E}}}(Y):= & {} \text {the set of elliptic fibrations } Y\rightarrow {{\mathbb {P}}}^1. \end{aligned}$$

We apply Theorem 3.4 and the aforementioned algorithms to $(\tau , {\bar{\tau }})$-generic Enriques surfaces. This results in our second, series of main results: Theorem 1.18 expresses the volume of the fundamental domain of ${\mathrm {Aut}}(Y)$ on the nef cone $\mathrm{Nef}(Y)$ in terms of the Weyl group of $\tau$, Theorem 1.19 relates the orbits of ${\mathrm {Aut}}(Y)$ on the set of smooth rational curves ${{\mathcal {R}}}(Y)$ to the connected components of the Dynkin diagram $\tau$ and Theorem 1.21 counts the ${\mathrm {Aut}}(Y)$-orbits of the set of elliptic fibrations ${{\mathcal {E}}}(Y)$ and their fiber types.

Our new idea is the lattice theoretic result obtained in [6] (see also Dolgachev and Kondo [9, Chapter 10]). For a lattice L with the intersection form $\langle -,- \rangle$, let L(m) denote the lattice with the same underlying ${{\mathbb {Z}}}$-module as L and with the intersection form $m\,\langle -,- \rangle$. A lattice L of rank $n>1$ is said to be hyperbolic if the signature is $(1, n-1)$. For a positive integer n with $n\bmod 8=2$, let $L_n$ denote an even unimodular hyperbolic lattice of rank n, which is unique up to isomorphism. Borcherds [4, 5] developed a method to calculate the orthogonal group of an even hyperbolic lattice S by embedding S primitively into $L_{26}$ and using the result of Conway [7]. This method has been applied to the study of automorphism groups of K3 surfaces by many authors. However, the method often requires impractically heavy computation (see, for example, [11, 28]).

On the other hand, in [6], we have classified all primitive embeddings of $L_{10}(2)$ into $L_{26}$ and showed that they have a remarkable property (see Theorems 4.2 and 4.3) which enables us to calculate automorphism groups of Enriques surfaces efficiently and explicitly for the first time. The resulting speed up (roughly by a factor of $10^{20}$ in the best situation see Remark 6.1) over a more direct approach, allows us to calculate the automorphism groups of the 184 families of $(\tau , {\bar{\tau }})$-generic Enriques surfaces.

1.1 Definition of $(\tau , {\bar{\tau }})$-Generic Enriques Surfaces

First, we define $(\tau , {\bar{\tau }})$-generic Enriques surfaces. Let L be a lattice. We let the group ${\mathrm {O}}(L)$ of isometries of L act on L from the right, and write the action as $v\mapsto v^g$ for $v\in L\otimes {{\mathbb {R}}}$ and $g\in {\mathrm {O}}(L)$. We have a natural identification ${\mathrm {O}}(L)={\mathrm {O}}(L(m))$ for any nonzero integer m. A vector v of a lattice is called a k-vector if $\langle v,v \rangle =k$. A $(-2)$-vector is called a root.

Definition 1.1

An $\mathrm{ADE}$-lattice is an even negative definite lattice generated by roots. An $\mathrm{ADE}$-lattice R has a basis consisting of roots whose dual graph is a Dynkin diagram of an $\mathrm{ADE}$-type. This $\mathrm{ADE}$-type is denoted by $\tau (R)$.

A positive half-cone of a hyperbolic lattice L is one of the two connected components of $\{\,{x\in L\otimes {{\mathbb {R}}}}\mid {\langle x, x \rangle >0} \,\}$. Let ${{\mathcal {P}}}$ be a positive half-cone of a hyperbolic lattice L. We put

$$\begin{aligned} {{\mathrm {O}}}^{{{\mathcal {P}}}}(L):=\{\,{g\in {\mathrm {O}}(L)}\mid {{{\mathcal {P}}}^g={{\mathcal {P}}}} \,\}. \end{aligned}$$

In [29], we classified the $\mathrm{ADE}$-sublattices of $L_{10}$ up to the action of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(L_{10})$. Let R be an $\mathrm{ADE}$-sublattice of $L_{10}$, and $\overline{R}$ the primitive closure of R in $L_{10}$. It turned out that $\overline{R}$ is also an $\mathrm{ADE}$-sublattice of $L_{10}$.

Proposition 1.2

[29]

• (1) Let $R^\prime$ be another $\mathrm{ADE}$-sublattice of $L_{10}$ with the primitive closure $\overline{R}^\prime$. Then R and $R^\prime$ are in the same orbit under the action of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(L_{10})$ if and only if $(\tau (R), \tau (\overline{R}))=(\tau (R^\prime ), \tau (\overline{R}^\prime ))$.

• (2) The pair $(\tau , {\bar{\tau }})$ of $\mathrm{ADE}$-types is equal to $(\tau (R), \tau (\overline{R}))$ of an $\mathrm{ADE}$-sublattice R of $L_{10}$ if and only if $(\tau , {\bar{\tau }})$ is one of the 184 pairs in Table 1, where the 3rd column being “–” means $\tau ={\bar{\tau }}$. $\square$

Table 1 $\mathrm{ADE}$-sublattices of $L_{10}$

Let R be an $\mathrm{ADE}$-sublattice of $L_{10}$. We denote by $\iota _R{:}\,R\hookrightarrow L_{10}$ the inclusion. We define $M_R$ to be the ${{\mathbb {Z}}}$-submodule of $(L_{10}(2)\oplus R(2))\otimes {{\mathbb {Q}}}$ generated by $L_{10}(2)$ and $(\iota _R(v), \pm v)/2\in (L_{10}\oplus R)\otimes {{\mathbb {Q}}}$, where v runs through R, and equip $M_R$ with an intersection form by extending the intersection form of $L_{10}(2)\oplus R(2)$. By definition, $M_R$ is an even hyperbolic lattice with a chosen primitive embedding $\varpi _R {:}\,L_{10}(2) \hookrightarrow M_R$. If $R^\prime$ is another $\mathrm{ADE}$-sublattice of $L_{10}$ such that $(\tau (R^\prime ), \tau (\overline{R}^\prime ))=(\tau (R), \tau (\overline{R}))$, then, by Proposition 1.2, we have an isometry $g{:}\,L_{10}\xrightarrow {\sim }L_{10}$ that induces an isometry $g|_R{:}\,R\xrightarrow {\sim }R^\prime$, and hence we obtain an isometry ${\tilde{g}}{:}\,M_R\xrightarrow {\sim }M_{R^\prime }$ induced by $g \oplus g|_R$, which makes the following diagram commutative:

By an explicit calculation, we obtain the following:

Proposition 1.3

Let R be an $\mathrm{ADE}$-sublattice of $L_{10}$. Then the orthogonal complement of $\varpi _R {:}\,L_{10}(2) \hookrightarrow M_R$ is isomorphic to ${\widetilde{R}}(2)$ for some $\mathrm{ADE}$-lattice ${\widetilde{R}}$. In the 4th column of Table 1, we give the $\mathrm{ADE}$-type $\tau ({\widetilde{R}})$ of ${\widetilde{R}}$, where “–” means $\tau (R)=\tau ({\widetilde{R}})$. $\square$

Let Y be an Enriques surface. We denote by $S_{Y}$ the lattice of numerical equivalence classes of divisors of Y. It is well-known that $S_{Y}$ is isomorphic to $L_{10}$. Let $\pi {:}\,X\rightarrow Y$ be the universal covering of Y, and let $S_{X}$ denote the lattice of numerical equivalence classes of divisors of the K3 surface X. Then the étale double covering $\pi$ induces a primitive embedding

$$\begin{aligned} \pi ^*{:}\,S_{Y}(2)\hookrightarrow S_{X}. \end{aligned}$$

Definition 1.4

Let $(\tau , {\bar{\tau }})$ be one of the 184 pairs in Table 1. An Enriques surface Y is said to be $(\tau , {\bar{\tau }})$-generic if the following conditions are satisfied.

• (i) Let $T_X$ be the transcendental lattice of X, and $\omega$ a nonzero holomorphic 2-form of X, so that we have ${\mathbb {C}}\omega =H^{2, 0}(X) \subset T_X\otimes {\mathbb {C}}$. Then the group

  $$\begin{aligned} {\mathrm {O}}(T_X, \omega ):=\{\,{g\in {\mathrm {O}}(T_X)}\mid {g \text { preserves } {\mathbb {C}}\omega \subset T_X\otimes {\mathbb {C}}} \,\} \end{aligned}$$

  is equal to $\{\pm 1\}$.

• (ii) Let R be an $\mathrm{ADE}$-sublattice of $L_{10}$ with $(\tau (R), \tau (\overline{R}))=(\tau , {\bar{\tau }})$. Then there exist isometries $g{:}\,L_{10}\xrightarrow {\sim }S_Y$ and ${\tilde{g}}{:}\,M_R\xrightarrow {\sim }S_X$ that make the following commutative diagram

  

  (1.1)

The numbering of the $\mathrm{ADE}$-types in Table 1 of the present article is the same as the numbering in Table 1.1 of our previous paper [29], and hence the 1st-3rd columns of the two tables are identical. By definition, a $(\tau , {\bar{\tau }})$-generic Enriques surface exists if and only if the 4th column of the corresponding row of Table 1.1 of [29] contains 0. Hence we obtain the following:

Proposition 1.5

[29] A $(\tau , {\bar{\tau }})$-generic Enriques surface exists if and only if the 5th column of the corresponding row in Table 1 is not marked by $\times$. $\square$

Example 1.6

Consider the case where $\tau (R)=8 A_1$ (Nos. 86, 87, 88). By [29], we have no $S_X$ (No. 86), or $S_X/M_R$ is non-trivial ($({{\mathbb {Z}}}/2{{\mathbb {Z}}})^2$ for No. 87 and $({{\mathbb {Z}}}/2{{\mathbb {Z}}})^3$ for No. 88), that is, the inclusion ${\tilde{g}}$ is not an isometry. Hence there exist no $(\tau , {\bar{\tau }})$-generic Enriques surfaces with $\tau =8A_1$, even though there exist surfaces with 8 ordinary nodes birational to Enriques surfaces.

Remark 1.7

The geometry of Enriques surfaces with ${\mathrm {O}}(T_X, \omega )=\{\pm 1\}$ but with $S_X/M_R$ being non-trivial and finite is left for future studies.

Let ${{\mathcal {P}}}_Y$ (resp. ${{\mathcal {P}}}_X$) be the positive half-cone of $S_{Y}$ (resp. $S_{X}$) containing an ample class. We regard ${{\mathcal {P}}}_Y$ as a subspace of ${{\mathcal {P}}}_X$ by the embedding $\pi ^*\otimes {{\mathbb {R}}}$. We put

$$\begin{aligned} \mathrm{Nef}_{X}:= & {} \{\,{x\in {{\mathcal {P}}}_X}\mid {\langle x, [{\widetilde{C}}] \rangle \ge 0 \;\; \text {for all curves } {\widetilde{C}}\text { on } X} \,\}, \\ \mathrm{Nef}_{Y}:= & {} \{\,{y\in {{\mathcal {P}}}_Y}\mid {\langle y, [C] \rangle \ge 0 \;\;\text {for all curves } C \hbox { on } Y} \,\}= {{\mathcal {P}}}_Y\cap \mathrm{Nef}_{X}, \end{aligned}$$

where [D] is the class of a divisor D. The following will be proved in Sect. 3.2.

Proposition 1.8

Let Y and $Y^\prime$ be $(\tau , {\bar{\tau }})$-generic Enriques surfaces with the universal coverings $\pi {:}\,X\rightarrow Y$ and $\pi ^\prime {:}\,X^\prime \rightarrow Y^\prime$, respectively. Then there exist isometries $\psi _X{:}\,S_{X}\xrightarrow {\sim }S_{X^\prime }$ and $\psi _Y{:}\,S_{Y}\xrightarrow {\sim }S_{Y^\prime }$ that make the diagram

(1.2)

commutative and that induce $\mathrm{Nef}_{X}\cong \mathrm{Nef}_{X^\prime }$ and $\mathrm{Nef}_{Y}\cong \mathrm{Nef}_{Y^\prime }$.

We denote by $\mathrm{aut}(Y)$ the image of the natural representation ${\mathrm {Aut}}(Y) \rightarrow {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$. We embed the set ${{\mathcal {R}}}(Y)$ of smooth rational curves C on Y into $S_{Y}$ by $C\mapsto [C]$, and the set ${{\mathcal {E}}}(Y)$ of elliptic fibrations $\phi {:}\,Y\rightarrow {{\mathbb {P}}}^1$ into $S_{Y}$ by $\phi \mapsto [F]/2$, where F is a general fiber of $\phi$. In Sect. 6, we will see that $\mathrm{aut}(Y)$ and its actions on $\mathrm{Nef}_{Y}$, ${{\mathcal {R}}}(Y)$, ${{\mathcal {E}}}(Y)$ depend only on the data $\pi ^* {:}\,S_{Y}(2)\hookrightarrow S_{X}$ and $\mathrm{Nef}_{X}$. Therefore we obtain the following:

Corollary 1.9

Let Y and $Y^\prime$ be as in Proposition 1.8. Then there exist an isomorphism $\mathrm{aut}(Y)\cong \mathrm{aut}(Y^\prime )$ and bijections ${{\mathcal {R}}}(Y)\cong {{\mathcal {R}}}(Y^\prime )$ and ${{\mathcal {E}}}(Y)\cong {{\mathcal {E}}}(Y^\prime )$ that are compatible with $\mathrm{aut}(Y)\cong \mathrm{aut}(Y^\prime )$. $\square$

Remark 1.10

The root invariant of a $(\tau , {\bar{\tau }})$-generic Enriques surface (defined by Nikulin [24]) is equal to $(\tau , {\mathrm {Ker}}\, \xi )$, where $\xi {:}\,R\otimes {{\mathbb {F}}}_2\rightarrow L_{10}\otimes {{\mathbb {F}}}_2$ is the linear homomorphism induced by the inclusion $R\hookrightarrow L_{10}$ of the $\mathrm{ADE}$-sublattice R of $L_{10}$ such that $(\tau , {\bar{\tau }})=(\tau (R), \tau (\overline{R}))$.

1.2 Chambers

Before we state our geometric results, we define the notion of chambers of hyperbolic lattices, and recall the classical result of Vinberg [35].

A root r of an even lattice L defines the reflection $s_r{:}\,x\mapsto x+\langle x, r \rangle r$ of L with respect to r. The Weyl group W(L) of L is the subgroup of ${\mathrm {O}}(L)$ generated by all the reflections $s_r$ with respect to the roots of L. Let L be an even hyperbolic lattice with a positive half-cone ${{\mathcal {P}}}$. For $v\in L\otimes {{\mathbb {R}}}$ with $\langle v,v \rangle <0$, let $(v)^{\perp }$ denote the hyperplane of ${{\mathcal {P}}}$ defined by $\langle x, v \rangle =0$. Then we have $W(L)\subset {{\mathrm {O}}}^{{{\mathcal {P}}}}(L)$, and the action of $s_r$ on ${{\mathcal {P}}}$ is the reflection into the mirror $(r)^{\perp }$. A closed subset D of ${{\mathcal {P}}}$ is called a chamber if D contains a non-empty open subset of ${{\mathcal {P}}}$ and D is defined by inequalities

$$\begin{aligned} \langle x, v_i \rangle \ge 0\quad (i\in I), \end{aligned}$$

where $\{(v_i)^{\perp }\}_{i\in I}$ is a locally finite family of hyperplanes of ${{\mathcal {P}}}$. A wall of a chamber D is a closed subset of D of the form $D\cap (v)^{\perp }$ such that $(v)^{\perp }$ is disjoint from the interior of D and that $D\cap (v)^{\perp }$ contains a non-empty open subset of $(v)^{\perp }$. We say that a vector $v\in L\otimes {{\mathbb {R}}}$ defines a wall $D\cap (v)^{\perp }$ of D if $D\cap (v)^{\perp }$ is a wall of D and $\langle x, v \rangle >0$ holds for one (and hence any) point x in the interior of D. We say that a closed subset A of ${{\mathcal {P}}}$ is tessellated by a set $\{D_j\}_{j\in J}$ of chambers if A is the union of $D_j$ ($j\in J$) and the interiors of two distinct chambers $D_j$ and $D_{j^\prime }$ in the family $\{D_j\}_{j\in J}$ have no common points.

Definition 1.11

Let L be an even hyperbolic lattice with a positive half-cone ${{\mathcal {P}}}$. An L-chamber is the closure in ${{\mathcal {P}}}$ of a connected component of

$$\begin{aligned} {{\mathcal {P}}}{{\setminus }}\bigcup _{r}\; (r)^{\perp }, \end{aligned}$$

where r runs through the set of roots of L. For an L-chamber D, we denote the stabilizer of D by

$$\begin{aligned} {\mathrm {O}}(L, D):=\{\,{g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(L)}\mid {D^g=D} \,\}. \end{aligned}$$

Remark 1.12

In Sect. 2.4, we extend the notion of L-chambers to the notion of L/M-chambers in the positive-half cone ${{\mathcal {P}}}_M$ of a primitive lattice M of L.

The group ${{\mathrm {O}}}^{{{\mathcal {P}}}}(L)$ acts on the set of L-chambers. The action of the subgroup W(L) of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(L)$ on this set is free and transitive. Hence an L-chamber is a standard fundamental domain of the Weyl group W(L). Let D be an L-chamber. Then we have ${{\mathrm {O}}}^{{{\mathcal {P}}}}(L)=W(L)\rtimes {\mathrm {O}}(L, D)$, and moreover, W(L) is generated by the reflections $s_r$ with respect to the roots r that define the walls of D.

Recall that $L_{10}$ is an even unimodular hyperbolic lattice of rank 10. Then $L_{10}$ has a basis $e_1, \dots , e_{10}$ consisting of roots whose dual graph is given in Fig. 1. Let ${{\mathcal {P}}}_{10}$ be the positive half-cone of $L_{10}$ containing $e_1^{\vee }+\cdots +e_{10}^{\vee }$, where $\{e_1^{\vee }, \dots , e_{10}^{\vee }\}$ is the basis of $L_{10}^{\vee }=L_{10}$ dual to $\{e_1, \dots , e_{10}\}$.

Theorem 1.13

(Vinberg [35]) The chamber $D_0$ in ${{\mathcal {P}}}_{10}$ defined by $\langle x, e_i \rangle \ge 0$ for $i=1, \dots , 10$ is an $L_{10}$-chamber, and $\{e_1, \dots , e_{10}\}$ is the set of roots defining walls of $D_0$. $\square$

Definition 1.14

We call an $L_{10}$-chamber a Vinberg chamber.

Let $D_0$ be a Vinberg chamber. Since the dual graph in Fig. 1 has no non-trivial symmetries, we have ${\mathrm {O}}(L_{10}, D_0)=\{1\}$ and hence

$$\begin{aligned} {{\mathrm {O}}}^{{{\mathcal {P}}}}(L_{10})=W(L_{10}). \end{aligned}$$

(1.3)

Fig. 1

The basis $e_1, \dots , e_{10}$ of $L_{10}$

1.3 Main Results

We investigate the geometry of a $(\tau , {\bar{\tau }})$-generic Enriques surface Y. In particular, we calculate a finite generating set of $\mathrm{aut}(Y)$ and the action of $\mathrm{aut}(Y)$ on $\mathrm{Nef}_{Y}$, ${{\mathcal {R}}}(Y)$ and ${{\mathcal {E}}}(Y)$.

Remark 1.15

Since our approach relies on the interplay between lattice theory and hyperbolic geometry, we can, except for the cases Nos. 88 and 146 in Table 1, calculate the geometric data of a hypothetical $(\tau , {\bar{\tau }})$-generic Enriques surface even when it is not realized by an actual Enriques surface. (See Remark 4.7).

Remark 1.16

For the cases Nos. 88 and 146 in Table 1, we cannot construct $S_X$ by means of the method described in Sect. 4.3. Since there do not exist $(\tau , {{\bar{\tau }}})$-generic Enriques surfaces in these cases, we leave them blank.

Let Y be an Enriques surface. Recall that $\mathrm{aut}(Y)\subset {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ is the image of the natural homomorphism ${\mathrm {Aut}}(Y)\rightarrow {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$. Since $S_{Y}$ is isomorphic to $L_{10}$, we have Vinberg chambers in the positive half-cone ${{\mathcal {P}}}_Y$. Since $\mathrm{Nef}_{Y}$ is bounded by $([C])^{\perp }$, where C runs through ${{\mathcal {R}}}(Y)$, and $\langle [C], [C] \rangle =-2$, the cone $\mathrm{Nef}_{Y}$ is tessellated by Vinberg chambers. We put

$$\begin{aligned} {{\mathcal {V}}}(\mathrm{Nef}_{Y}):= \text {the set of Vinberg chambers contained in } \mathrm{Nef}_{Y}, \end{aligned}$$

on which $\mathrm{aut}(Y)$ acts, and define

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y)):= \text {the number of orbits of the action of } \mathrm{aut}(Y) \hbox { on } {{\mathcal {V}}}(\mathrm{Nef}_{Y}). \end{aligned}$$

An Enriques surface that is very general in the sense of Barth and Peters [2] is (0, 0)-generic, and its automorphism group was determined by Barth and Peters [2] and Nikulin [23, Theorem 10.1.2 (c)] independently.

Theorem 1.17

(Barth and Peters [2], Nikulin [23]) Let $Y_0$ be a (0, 0)-generic Enriques surface. Then $\mathrm{aut}({Y_0})\subset {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y_0})$ is equal to the kernel of the reduction homomorphism ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y_0}) \rightarrow {\mathrm {O}}(S_{Y_0}) \otimes {{\mathbb {F}}}_2$. In particular, the index of $\mathrm{aut}(Y_0)$ in ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y_0})$ is equal to

$$\begin{aligned} 2^{21}\cdot 3^5\cdot 5^2\cdot 7 \cdot 17\cdot 31=46998591897600. \end{aligned}$$

Since a (0, 0)-generic Enriques surface $Y_0$ contains no smooth rational curves, we have ${{\mathcal {P}}}_{Y_0}=\mathrm{Nef}_{Y_0}$. Combining this with (1.3), we obtain bijections

$$\begin{aligned} {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y_0})=W(S_{Y_0})\cong {{\mathcal {V}}}(\mathrm{Nef}_{Y_0}). \end{aligned}$$

We define the unit $1_{\mathrm{{BP}}}$ ($\mathrm{{BP}}$ stands for Barth–Peters) of volume to be

$$\begin{aligned} 1_{\mathrm{{BP}}}:=\mathrm{{vol}}(\mathrm{Nef}_{Y_0}/\mathrm{aut}(Y_0))=[{{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y_0}){:}\,\mathrm{aut}(Y_0)] =2^{21}\cdot 3^5\cdot 5^2\cdot 7 \cdot 17\cdot 31. \end{aligned}$$

Our first main result is as follows. For an $\mathrm{ADE}$-type $\tau$, let $W(R_{\tau })$ denote the Weyl group of the $\mathrm{ADE}$-lattice $R_{\tau }$ with $\tau (R_{\tau })=\tau$, that is, the finite Coxeter group defined by the Dynkin diagram of type $\tau$. An automorphism of Y is called numerically trivial if it acts trivially on $S_{Y}$.

Theorem 1.18

Let Y be a $(\tau , {\bar{\tau }})$-generic Enriques surface. Then we have

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/ \mathrm{aut}(Y))=\frac{c_{(\tau , {\bar{\tau }})}}{|W(R_{\tau })|} \cdot 1_{\mathrm{{BP}}}, \end{aligned}$$

where $c_{(\tau , {\bar{\tau }})}\in \{1, 2\}$ is the number of numerically trivial automorphisms of Y and is given in 6th column of Table 1.

In two non-geometric cases Nos. 142 and 170 (Remark 1.15), there exists a contribution to $c_{(\tau ,{\bar{\tau }})}$ not coming from a numerically trivial automorphism. (See Theorem 3.11 and Remark 3.12). Theorem 1.18 is in fact obtained from a more general result Theorem 3.4 on $\mathrm{{vol}}(\mathrm{Nef}_{Y}/ \mathrm{aut}(Y))$. To obtain Theorem 3.4, we prove a result (Proposition 2.1) of the theory of discriminant forms in the spirit of Nikulin [22]. The proof of these theorems is conceptual. Nevertheless the ability to compute examples played a crucial role in finding the correct statements.

Next, we calculate explicitly a finite generating set of $\mathrm{aut}(Y)$ and a complete set of representatives of the orbits of the action of $\mathrm{aut}(Y)$ on $\mathrm{Nef}_{Y}$. The algorithms we use for this purpose are variations of a simple algorithm given in Sect. 4.1, which is an abstraction of the generalized Borcherds’ method described in [28]. By means of these computational data, we analyze the action of $\mathrm{aut}(Y)$ on ${{\mathcal {R}}}(Y)$ and ${{\mathcal {E}}}(Y)$. (Recall that ${{\mathcal {R}}}(Y)$ and ${{\mathcal {E}}}(Y)$ are embedded into $S_{Y}$.)

Our second main result is as follows.

Theorem 1.19

Let Y be a $(\tau , {\bar{\tau }})$-generic Enriques surface.

• (1) There exist smooth rational curves $C_1, \dots , C_m$ on Y whose dual graph $\Gamma$ is a Dynkin diagram of type $\tau$. Under the action of $\mathrm{aut}(Y)$, any smooth rational curve C on Y is in the same orbit as one of $C_1, \dots , C_m$.

• (2) The size of ${{\mathcal {R}}}(Y) /\mathrm{aut}(Y)$ is given in the 7th column rat of Table 1. Except for the cases marked by $\times$ in this column, two curves $C_i$ and $C_j$ are in the same orbit if and only if the vertices of the dual graph $\Gamma$ corresponding to $C_i$ and $C_j$ belong to the same connected component of $\Gamma$, and hence $|{{\mathcal {R}}}(Y)/\mathrm{aut}(Y)|$ is equal to the number of connected components of the Dynkin diagram of type $\tau$.

In [2], Barth and Peters also proved the following.

Theorem 1.20

(Barth and Peters [2]) Let $Y_0$ be a (0, 0)-generic Enriques surface. Then $Y_0$ has exactly $17\cdot 31=527$ elliptic fibrations modulo $\mathrm{aut}(Y_0)$.

We calculate ${{\mathcal {E}}}(Y)/\mathrm{aut}(Y)$ for $(\tau , {\bar{\tau }})$-generic Enriques surfaces. Since the tables span 7 pages, we relegate a part of it to the ancillary files.

Theorem 1.21

Let Y be a $(\tau ,{\bar{\tau }})$-generic Enriques surface. Then the orbits of the action of $\mathrm{aut}(Y)$ on the set ${{\mathcal {E}}}(Y)$ of elliptic fibrations of Y are indicated in Sect. 6.5 for ${\mathrm {rank}}\,\tau \le 7$ and in the ancillary files [32] for ${\mathrm {rank}}\,\tau \ge 8$.

1.4 The Plan of the Paper

This paper is organized as follows. In Sect. 2, we prepare basic notions about finite quadratic forms, discriminant forms, lattices and chambers. Proposition 2.1 in Sect. 2.1 plays a crucial role in the proof of the volume formula in the next section. The notion of L/M-chambers given in Sect. 2.4 is the main tool of our computation. In Sect. 3, we investigate the nef-and-big cone $\mathrm{Nef}_{Y}$ of an Enriques surface Y from the point of view of L/M-chambers, and prove Proposition 1.8. Then, by means of Proposition 2.1, we prove a formula (Theorem 3.4) for the volume of $\mathrm{Nef}_{Y}/\mathrm{aut}(Y)$, and in Sect. 3.4, we deduce Theorem 1.18 from Theorem 3.4.

In Sect. 4, we present a computational procedure on a graph (Procedure 4.1), which is an abstraction of the generalized Borcherds’ method formulated in [28]. Then we recall the classification of primitive embeddings $L_{10}(2)\hookrightarrow L_{26}$ obtained in [6], and construct primitive embeddings $S_{Y}(2)\hookrightarrow S_{X}\hookrightarrow L_{26}$ for $(\tau , {\bar{\tau }})$-generic Enriques surfaces Y. In Sect. 5, we prepare some geometric algorithms used in the application of the generalized Borcherds’ method to $(\tau , {\bar{\tau }})$-generic Enriques surfaces. In Sect. 6, we calculate $\mathrm{aut}(Y)$ and $\mathrm{Nef}_{Y}/\mathrm{aut}(Y)$, and prove Theorems 1.19 and 1.21. The table of elliptic fibrations is given in Sect. 6.5.

In Sect. 7, we exhibit some examples. In particular, we treat an $(E_6, E_6)$-generic Enriques surface (No. 47 of Table 1) in detail, because we investigated this surface in [31]. Section 7.1 contains a correction of a wrong assertion made in [31].

In the second author’s webpage and in the repository “zenodo” [32], we put a detailed computation data made by GAP [34].

Thanks are due to Professor Igor Dolgachev for his comments on the manuscript of this paper. The authors also thank the referees for many valuable comments.

2 Finite Quadratic Forms, Lattices and Chambers

We fix notions and terminologies about finite quadratic forms, discriminant forms, lattices and chambers.

2.1 Finite Quadratic Forms

A finite quadratic form is a finite abelian group A with a quadratic form

$$\begin{aligned} q_A{:}\,A\rightarrow {{\mathbb {Q}}}/2{{\mathbb {Z}}}. \end{aligned}$$

We say that a finite quadratic form is non-degenerate if the bilinear form

$$\begin{aligned} b_A{:}\,A\times A\rightarrow {{\mathbb {Q}}}/{{\mathbb {Z}}}\end{aligned}$$

associated with $q_A$ is non-degenerate. The automorphism group of a finite quadratic form A is denoted by ${\mathrm {O}}(A)$, and we let it act on A from the right. For a subgroup $D\subset A$, let $D^{\perp }$ denote the orthogonal complement of D with respect to $b_A$, and let ${\mathrm {O}}(A, D)$ denote the subgroup $\{\,{g\in {\mathrm {O}}(A)}\mid {D^g=D} \,\}$ of ${\mathrm {O}}(A)$.

The following proposition will play a crucial role in the proof of the volume formula (Theorem 3.4).

Proposition 2.1

Let $(A, q_A)$ and $(B, q_B)$ be non-degenerate finite quadratic forms, and let $D_A\subset A$ and $D_B\subset B$ be subgroups. Suppose that we have an isomorphism $\phi {:}\,D_A\xrightarrow {\sim }D_B$ that induces an isometry $(D_A, -q_A|D_A)\cong (D_B, q_B|D_B)$ of finite quadratic forms. Let $\Gamma \subset A\oplus B$ be the graph of $\phi$, which is an isotropic subgroup with respect to $q_A\oplus q_B$. We put $C:=\Gamma ^{\perp }/\Gamma$. Then $q_A\oplus q_B$ induces a quadratic form $q_C$ on C, and we have a natural homomorphism

$$\begin{aligned} \{\,{(g, h) \in {\mathrm {O}}(A)\times {\mathrm {O}}(B)}\mid {\Gamma ^{(g, h)}=\Gamma } \,\}\rightarrow {\mathrm {O}}(C). \end{aligned}$$

We denote by K the kernel of this homomorphism. Then the homomorphism

$$\begin{aligned} i_A{:}\,K\hookrightarrow {\mathrm {O}}(A)\times {\mathrm {O}}(B) \rightarrow {\mathrm {O}}(A), \quad (g, h)\mapsto g \end{aligned}$$

is injective, and the image of $i_A$ is equal to the kernel of the natural homomorphism

$$\begin{aligned} {\mathrm {O}}(A, D_A)\rightarrow {\mathrm {O}}(D_A^{\perp }). \end{aligned}$$

Proof

First we prove that the natural projection $\Gamma ^{\perp }\rightarrow B$ is surjective. Since $q_A$ and $q_B$ are non-degenerate, we have natural isomorphisms $A\cong {\mathrm {Hom}}(A, {{\mathbb {Q}}}/{{\mathbb {Z}}})$ and $B\cong {\mathrm {Hom}}(B, {{\mathbb {Q}}}/{{\mathbb {Z}}})$ induced by $b_A$ and $b_B$. Hence we have natural isomorphisms ${\mathrm {Hom}}(D_A, {{\mathbb {Q}}}/{{\mathbb {Z}}})\cong A/D_A^{\perp }$ and ${\mathrm {Hom}}(D_B, {{\mathbb {Q}}}/{{\mathbb {Z}}})\cong B/D_B^{\perp }$. We have an isomorphism

$$\begin{aligned} -\phi ^*{:}\, {\mathrm {Hom}}(D_B, {{\mathbb {Q}}}/{{\mathbb {Z}}})\cong {\mathrm {Hom}}(D_A, {{\mathbb {Q}}}/{{\mathbb {Z}}}) \end{aligned}$$

induced by $-\phi {:}\,D_A\xrightarrow {\sim }D_B$. Combining them, we obtain a homomorphism

$$\begin{aligned} \psi {:}\,B\mathbin {\rightarrow \rightarrow }B/D_B^{\perp }\cong {\mathrm {Hom}}(D_B, {{\mathbb {Q}}}/{{\mathbb {Z}}})\cong {\mathrm {Hom}}(D_A, {{\mathbb {Q}}}/{{\mathbb {Z}}}) \cong A/D_A^{\perp }. \end{aligned}$$

(2.1)

For $\alpha \in A$, we put

$$\begin{aligned} {\bar{\alpha }}:=\alpha \bmod D_A^{\perp }\;\;\in \;\; A/D_A^{\perp }. \end{aligned}$$

Then, for $\alpha \in A$ and $\beta \in B$, we have

$$\begin{aligned} {\bar{\alpha }}=\psi (\beta ) \;\Longleftrightarrow \; b_A(\alpha , x)=-b_B(\beta , \phi (x)) \;\text {for all } x\in D_A \; \Longleftrightarrow \; (\alpha , \beta )\in \Gamma ^{\perp }.\nonumber \\ \end{aligned}$$

(2.2)

In particular, for any $\beta \in B$, we have $\alpha \in A$ such that $(\alpha , \beta )\in \Gamma ^{\perp }$.

Next we prove that $i_A{:}\,K\rightarrow {\mathrm {O}}(A)$ is injective. Let $(1, h)\in K$ be an element of ${\mathrm {Ker}}\, i_A$. For $\beta \in B$, we choose $\alpha \in A$ such that $(\alpha , \beta )\in \Gamma ^{\perp }$. Since (1, h) acts on $C=\Gamma ^{\perp }/\Gamma$ trivially, we have $(\alpha , \beta )-(\alpha , \beta ^h)=(0, \beta -\beta ^h)\in \Gamma$. Since $\Gamma \cap B=0$, we have $\beta ^h=\beta$. Since $\beta \in B$ is arbitrary, we have $h=1$.

Now we determine the image of $i_A$. “$\subset$”: Suppose that $(g, h)\in K$. Since (g, h) preserves $\Gamma$, we see that $g=i_A(g, h)$ preserves the image $D_A$ of the projection $\Gamma \rightarrow A$. For any $\alpha \in D_A^{\perp }$, we have $(\alpha , 0)\in \Gamma ^{\perp }$. Since (g, h) acts on $C=\Gamma ^{\perp }/\Gamma$ trivially, we have $\alpha ^g-\alpha \in \Gamma \cap A=0$. Therefore ${\mathrm {Im}}\, i_A$ is contained in ${\mathrm {Ker}}({\mathrm {O}}(A, D_A)\rightarrow {\mathrm {O}}(D_A^{\perp }))$.

“$\supset$”: To show the opposite inclusion, we fix $g\in {\mathrm {Ker}}({\mathrm {O}}(A, D_A)\rightarrow {\mathrm {O}}(D_A^{\perp }))$ and construct $h\in {\mathrm {O}}(B)$ such that $(g, h)\in K$. Since g acts on $D_A^{\perp }$ trivially, the linear map

$$\begin{aligned} l_g{:}\,A/D_A^{\perp }\rightarrow A, \quad {\bar{\alpha }}\mapsto \alpha ^g-\alpha \end{aligned}$$

is well-defined. The image of $l_g$ is contained in $D_A=(D_A^{\perp })^{\perp }$: indeed, for any $\alpha \in A$ and $y\in D_A^{\perp }$, we have

$$\begin{aligned} b_A(l_g({\bar{\alpha }}), y)=b_A(\alpha ^g, y)-b_A(\alpha , y)=b_A(\alpha ^g, y^g)-b_A(\alpha , y)=0. \end{aligned}$$

We define $h: B\rightarrow B$ by

$$\begin{aligned} \beta ^h:=\beta +\phi l_g \psi (\beta ), \end{aligned}$$

where $\psi$ is given in (2.1). We show that $h\in {\mathrm {O}}(B)$. We put ${\bar{\alpha }}=\psi (\beta )$. Then we have

$$\begin{aligned} q_B(\beta ^h)-q_B(\beta )= & {} 2b_B(\beta , \phi l_g ({\bar{\alpha }}))+q_B(\phi l_g({\bar{\alpha }}))\\= & {} -2b_A(\alpha , \alpha ^g-\alpha )-q_A( \alpha ^g-\alpha )=0, \end{aligned}$$

because $g\in {\mathrm {O}}(A)$. It only remains to show that $(g, h)\in {\mathrm {O}}(A)\times {\mathrm {O}}(B)$ preserves $\Gamma$ and acts on $C=\Gamma ^{\perp }/\Gamma$ trivially. Using (2.2) and $\Gamma \subset \Gamma ^\perp$, we see that for any $\alpha \in D_A$, we have ${\bar{\alpha }} =\psi \phi (\alpha )$, and therefore

$$\begin{aligned} \phi (\alpha )^h=\phi (\alpha )+\phi l_g ({\bar{\alpha }})=\phi (\alpha )+\phi (\alpha ^g)-\phi (\alpha )=\phi (\alpha ^g). \end{aligned}$$

Since g preserves $D_A$, we have $(\alpha , \phi (\alpha ))^{(g, h)}=(\alpha ^g, \phi (\alpha ^g))\in \Gamma$ for any $\alpha \in D_A$. Therefore (g, h) preserves $\Gamma$. Suppose that $(\alpha , \beta )\in \Gamma ^{\perp }$. Then we have ${\bar{\alpha }} =\psi (\beta )$ by (2.2), and

$$\begin{aligned} (\alpha ^g, \beta ^h)-(\alpha , \beta )=(l_g({\bar{\alpha }}), \phi l_g({\bar{\alpha }}) )\;\; \in \;\; \Gamma . \end{aligned}$$

Therefore (g, h) acts on $\Gamma ^{\perp }/\Gamma$ trivially. $\square$

Remark 2.2

Proposition 2.1 holds for non-degenerate finite bilinear forms $(A, b_A)$ and $(B, b_B)$ as well.

2.2 Discriminant Forms and Overlattices

Let L be an even lattice. We put

$$\begin{aligned} L^{\vee }:=\{\,{x\in L\otimes {{\mathbb {Q}}}}\mid {\langle x, v \rangle \in {{\mathbb {Z}}}\;\;\text {for all}\;\; v\in L} \,\}, \end{aligned}$$

on which ${\mathrm {O}}(L)$ acts naturally. The finite abelian group $L^{\vee }/L$ is called the discriminant group of L. Then

$$\begin{aligned} q({\bar{x}})=\langle x, x \rangle \bmod 2{{\mathbb {Z}}}\quad \text {for } x\in L^{\vee }\hbox { and } {\bar{x}}=x \bmod L \end{aligned}$$

defines a finite quadratic form $q{:}\,L^{\vee }/L \rightarrow {{\mathbb {Q}}}/2{{\mathbb {Z}}}$, which is called the discriminant form of L. An even lattice $L^\prime$ is an overlattice of L if we have $L\subset L^\prime \subset L^{\vee }$ and the intersection form of $L^\prime$ is the extension of that of L. See Nikulin [22] for the details of the theory of discriminant forms and its application to the enumeration of even overlattices of a given even lattice.

To illustrate Proposition 2.1, we apply it to two known extreme cases.

Example 2.3

Let $M,N \subset L$ be primitive sublattices of an even lattice L such that $M\perp N$ and ${\mathrm {rank}}\, M+{\mathrm {rank}}\, N={\mathrm {rank}}\,L$. Then we have

$$\begin{aligned} M \oplus N \subset L \subset L^{\vee }\subset M^{\vee }\oplus N^{\vee }, \end{aligned}$$

and L is an overlattice of $M\oplus N$. Let $(A,q_A)= (M^{\vee }/M,q_M)$ and $(B,q_B)= (N^{\vee }/N,q_N)$ be the respective discriminant forms. Then $\Gamma = L/(M \oplus N)$ is the graph of an anti-isometry $\phi {:}\,A \supset D_A \rightarrow D_B \subset B$ and $\Gamma ^\perp /\Gamma \cong L^{\vee }/L$.

First suppose that L is unimodular. Then, by a result of Nikulin [22], $D_A = A$ and $D_B=B$. Since $L^{\vee }/L \cong \Gamma ^\perp /\Gamma$ is trivial, we have

$$\begin{aligned} K=\{(g,h) \in {\mathrm {O}}(A) \times {\mathrm {O}}(B){:}\,h \circ \phi = \phi \circ g \}. \end{aligned}$$

We see that $i_A{:}\,K \rightarrow {\mathrm {O}}(A)$ is an isomorphism as predicted by Proposition 2.1. Indeed, since $D_A^\perp = A^\perp =0$, the homomorphism ${\mathrm {O}}(A,D_A) \rightarrow {\mathrm {O}}(D_A^\perp )$ is trivial.

For the other extreme suppose that $M\oplus N = L$. Then $D_A = 0$, $D_B=0$, $K = 1$ and $D_A^\perp = A$.

2.3 Faces of a Chamber

Let L be a hyperbolic lattice with a positive half-cone ${{\mathcal {P}}}$, and D a chamber in ${{\mathcal {P}}}$. A face of D is a closed subset of D that is an intersection of some walls of D. Let f be a face of D. The dimension $\dim f$ of f is the dimension of the minimal linear subspace of $L\otimes {{\mathbb {R}}}$ containing f, and the codimension of f is ${\mathrm {rank}}\, L-\dim f$. The walls of D are exactly the faces of D with codimension 1.

Let $\overline{{{\mathcal {P}}}}$ and $\overline{D}$ be the closures of ${{\mathcal {P}}}$ and D in $L\otimes {{\mathbb {R}}}$, respectively. A half-line contained in $(\overline{{{\mathcal {P}}}}{\setminus } {{\mathcal {P}}}) \cap \overline{D}$ is called an isotropic ray of D.

Suppose that D has only finitely many walls, that they are defined by vectors in $L\otimes {{\mathbb {Q}}}$, and that the list of defining vectors of these walls in $L\otimes {{\mathbb {Q}}}$ is available. Then we can make the list of faces of D by means of linear programming. For each isotropic ray ${{\mathbb {R}}}_{\ge 0}v$, we have a unique primitive vector $v\in L$ that generates ${{\mathbb {R}}}_{\ge 0}v$, which we call a primitive isotropic ray of D. We can also make the list of primitive isotropic rays of D.

2.4 L/M-Chambers

Let $(L, \langle \;,\; \rangle _L)$ and $(M, \langle \;,\; \rangle _M)$ be even hyperbolic lattices with fixed positive half-cones ${{\mathcal {P}}}_L$ and ${{\mathcal {P}}}_M$, respectively. Suppose that we have an embedding $M\hookrightarrow L$ that maps ${{\mathcal {P}}}_M$ into ${{\mathcal {P}}}_L$. We regard ${{\mathcal {P}}}_M$ as a subspace of ${{\mathcal {P}}}_L$ by this embedding. The notion of L-chambers was introduced in Sect. 1.2. The following class of chambers plays an important role in this paper.

Definition 2.4

A chamber $D_M$ in ${{\mathcal {P}}}_M$ is called an L/M-chamber if there exists an L-chamber $D_L \subset {{\mathcal {P}}}_L$ such that $D_M= {{\mathcal {P}}}_M\cap D_L$. In this case, we say that $D_M$ is induced by $D_L$.

In particular, an L-chamber is an L/L-chamber.

Definition 2.5

Let N be a negative definite even lattice. For a root r of N, let $[r]^{\perp }$ denote the hyperplane of $N\otimes {{\mathbb {R}}}$ defined by $\langle x, r \rangle =0$. The connected components of $(N\otimes {{\mathbb {R}}}){\setminus } \bigcup \, [r]^{\perp }$, where r runs through the set of roots of N, are called the Weyl-chambers of N. The Weyl group W(N) acts simply transitively on the set of Weyl-chambers.

Remark 2.6

Let $D_M$ be an L/M-chamber. Then the number of L-chambers that induce $D_M$ is equal to the number of Weyl-chambers of the orthogonal complement $(M\hookrightarrow L)^{\perp }$ of M in L. In particular, if $(M\hookrightarrow L)^{\perp }$ contains no roots, then each L/M-chamber is induced by a unique L-chamber.

Definition 2.7

Two distinct L/M-chambers $D_1$ and $D_2$ are adjacent if there exists a hyperplane $(v)^{\perp }$ of ${{\mathcal {P}}}_M$ such that $D_1\cap (v)^{\perp }$ is a wall of $D_1$, that $D_2\cap (v)^{\perp }$ is a wall of $D_2$, and that $D_1\cap (v)^{\perp }=D_2\cap (v)^{\perp }$ holds. In this case, we say that $D_2$ is adjacent to $D_1$ across the wall $D_1\cap (v)^{\perp }$.

Let ${\mathrm {pr}}{:}\,L\rightarrow M\otimes {{\mathbb {Q}}}$ be the orthogonal projection. Then an L/M-chamber is the closure in ${{\mathcal {P}}}_M$ of a connected component of

$$\begin{aligned} {{\mathcal {P}}}_M{{\setminus }} \bigcup _r \,({\mathrm {pr}}(r))^{\perp }, \end{aligned}$$

where r runs through the set of roots r of L such that $\langle {\mathrm {pr}}(r), {\mathrm {pr}}(r) \rangle _M<0$ holds, and $({\mathrm {pr}}(r))^{\perp }={{\mathcal {P}}}_M\cap (r)^{\perp }$ is the hyperplane of ${{\mathcal {P}}}_M$ defined by ${\mathrm {pr}}(r)$. Hence, for each wall $D_M\cap (v)^{\perp }$ of an L/M-chamber $D_M$, there exists a unique L/M-chamber adjacent to $D_M$ across the wall $D_M\cap (v)^{\perp }$.

Since a root of M is mapped to a root of L by the embedding $M\hookrightarrow L$, an M-chamber is tessellated by L/M-chambers. More generally, we have the following proposition, which is easy to prove:

Proposition 2.8

Suppose that $M_1 \hookrightarrow M_2 \hookrightarrow L$ is a sequence of embeddings of even hyperbolic lattices that induces a sequence of embeddings ${{\mathcal {P}}}_{M_1}\hookrightarrow {{\mathcal {P}}}_{M_2} \hookrightarrow {{\mathcal {P}}}_{L}$ of fixed positive half-cones. Then each $M_2/M_1$-chamber is tessellated by $L/M_1$-chambers. $\square$

If ${\tilde{g}}\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(L)$ satisfies $M^{{\tilde{g}}}=M$, then ${\tilde{g}}|M\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(M)$ preserves the tessellation of ${{\mathcal {P}}}_M$ by L/M-chambers.

In general, two distinct L/M-chambers are not isomorphic to each other. See [11, 28] for examples of K3 surfaces X with a primitive embedding $S_{X}\hookrightarrow L_{26}$ such that ${{\mathcal {P}}}_X$ is tessellated by $L_{26}/S_{X}$-chambers of various shapes.

Definition 2.9

We say that the tessellation of ${{\mathcal {P}}}_M$ by L/M-chambers is reflexively simple if, for each wall $D_M\cap (v)^{\perp }$ of an L/M-chamber $D_M$, there exists an isometry ${\tilde{g}}$ of L preserving M such that the restriction ${\tilde{g}}|M$ of ${\tilde{g}}$ to M is an involution that fixes every point of the hyperplane $(v)^{\perp }$. Note that, if this is the case, the isometry ${\tilde{g}}|M$ of M maps $D_M$ to the L/M-chamber adjacent to $D_M$ across the wall $D_M\cap (v)^{\perp }$.

The tessellation of ${{\mathcal {P}}}_L$ by L/L-chambers is obviously reflexively simple.

3 The Cone $\mathrm{Nef}_{Y}$

Let Y be an Enriques surface with the universal covering $\pi {:}\,X\rightarrow Y$. Let $\varepsilon \in {\mathrm {Aut}}(X)$ be the deck-transformation of $\pi {:}\,X\rightarrow Y$, and we put

$$\begin{aligned} S_{X+}:=\{\,{v\in S_{X}}\mid {v^\varepsilon =v} \,\}, \quad S_{X-}:=\{\,{v\in S_{X}}\mid {v^\varepsilon =-v} \,\}. \end{aligned}$$

Then $S_{X+}$ is equal to the image of $\pi ^*{:}\,S_{Y}(2)\hookrightarrow S_{X}$, and $S_{X-}$ is the orthogonal complement of $S_{X+}$. We regard ${{\mathcal {P}}}_Y$ as a subspace of ${{\mathcal {P}}}_X$ by $\pi ^*\otimes {{\mathbb {R}}}$.

3.1 $S_{X}/S_{Y}(2)$-Chambers

It is well-known that $\mathrm{Nef}_{X}$ is an $S_{X}$-chamber. Therefore the chamber $\mathrm{Nef}_{Y}={{\mathcal {P}}}_Y\cap \mathrm{Nef}_{X}$ is an $S_{X}/S_{Y}(2)$-chamber. Since $\pi$ is étale, the lattice $S_{X-}$ contains no roots, and hence each $S_{X}/S_{Y}(2)$-chamber $D_Y$ is induced by a unique $S_{X}$-chamber $D_X$, that is, $D_Y$ contains an interior point of $D_X$.

Proposition 3.1

The tessellation of ${{\mathcal {P}}}_Y$ by $S_{X}/S_{Y}(2)$-chambers is reflexively simple. More precisely, every wall of an $S_{X}/S_{Y}(2)$-chamber $D_Y$ is defined by a root r of $S_{Y}$, and the reflection $s_r\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ with respect to the root r is the restriction $s_{{\tilde{r}}_+} s_{{\tilde{r}}_-}|S_{Y}(2)$ of the product of two reflections with respect to roots ${\tilde{r}}_+, {\tilde{r}}_-$ of $S_{X}$.

Proof

Let $\langle -,- \rangle _X$ and $\langle -,- \rangle _Y$ be the intersection forms of $S_{X}$ and $S_{Y}$, respectively. We denote by $(u)^{\perp }_X$ the hyperplane of ${{\mathcal {P}}}_X$ defined by $u\in S_{X}\otimes {{\mathbb {R}}}$, and by $(v)^{\perp }_Y$ the hyperplane of ${{\mathcal {P}}}_Y$ defined by $v\in S_{Y}\otimes {{\mathbb {R}}}$. Let $D_Y$ be an $S_{X}/S_{Y}(2)$-chamber, and let $D_Y \cap (v)^{\perp }_Y$ be a wall of $D_Y$.

By the definition of $S_{X}/S_{Y}(2)$-chambers, there exists a root ${\tilde{r}}$ of $S_{X}$ such that $(v)_Y^{\perp }={{\mathcal {P}}}_Y\cap ({\tilde{r}})_X^{\perp }$. We first prove that $\langle {\tilde{r}}, {\tilde{r}}^{\varepsilon } \rangle _X=0$. Let ${\tilde{r}}$ be written as $v_L+v_R$, where $v_L\in S_{Y}(2)^{\vee }$ and $v_R\in S_{X-}^{\vee }$. We have $\langle v_L, v_L \rangle _X+\langle v_R, v_R \rangle _X=-2$. Since ${\tilde{r}}^{\varepsilon }=v_L-v_R$, it is enough to show that $\langle v_L, v_L \rangle _X=-1$. Since

$$\begin{aligned} {{\mathcal {P}}}_Y\cap ({\tilde{r}})_X^{\perp }=(v_L)_Y^{\perp }\end{aligned}$$

is non-empty, we have $\langle v_L, v_L \rangle _Y<0$. Note that $2 v_L\in S_{Y}$ because $2S_{Y}(2)^{\vee }=S_{Y}(2)$. Since $S_{Y}$ is even, $\langle v_L, v_L \rangle _X=2\langle v_L, v_L \rangle _Y$ must be an integer. Since $S_{X-}$ is negative definite, we have $\langle v_R, v_R \rangle _X\le 0$ and hence $\langle v_L, v_L \rangle _X$ is $-2$ or $-1$. If $\langle v_L, v_L \rangle _X=-2$, then $v_R=0$ and ${\tilde{r}}=v_L\in S_{Y}(2)$, which is absurd.

Let s and $s^\prime$ be the reflections with respect to the roots ${\tilde{r}}$ and ${\tilde{r}}^{\varepsilon }$ of $S_{X}$, respectively. By $\langle {\tilde{r}}, {\tilde{r}}^{\varepsilon } \rangle _X=0$, we have $s s^\prime =s^\prime s$. Since $s^\prime =\varepsilon s \varepsilon$, we see that $ss^\prime$ commutes with $\varepsilon$ and hence $ss^\prime$ preserves ${{\mathcal {P}}}_Y$. The vector $r:={\tilde{r}}+{\tilde{r}}^{\varepsilon }$ is contained in $S_{Y}$. Moreover we have $\langle r, r \rangle _Y=-2$ and

$$\begin{aligned} (v)_Y^{\perp }={{\mathcal {P}}}_Y\cap ({\tilde{r}})_X=(v_L)_Y^{\perp }=(r)_Y^{\perp }. \end{aligned}$$

Therefore the wall $D_Y \cap (v)^{\perp }_Y$ of $D_Y$ is defined by a root r or $-r$ of $S_{Y}$. It is easy to confirm that the restriction of $ss^\prime$ to $S_{Y}$ is equal to the reflection with respect to the root r of $S_{Y}$ and therefore maps $D_Y$ to the $S_{X}/S_{Y}(2)$-chamber $D^\prime _Y$ adjacent to $D_Y$ across the wall $D_Y\cap (v)_Y^{\perp }=D_Y\cap (r)_Y^{\perp }$. $\square$

3.2 Proof of Proposition 1.8

We prove Proposition 1.8. By Proposition 1.2, we have isomorphisms $\psi _X$ and $\psi _Y$ that make the diagram (1.2) commutative. By Proposition 3.1, we have ${\tilde{g}}\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$ commuting with $\varepsilon$ such that ${\tilde{g}}|S_{Y}(2)$ maps $\mathrm{Nef}_{Y}$ to the inverse image of $\mathrm{Nef}_{Y^\prime }$ by $\psi _Y$. Then the isometries ${\tilde{g}}\circ \psi _X{:}\,S_{X^\prime }\xrightarrow {\sim }S_{X}$ and ${\tilde{g}}|S_{Y}(2)\circ \psi _Y {:}\,S_{Y^\prime }\xrightarrow {\sim }S_{Y}$ satisfy the required properties. $\square$

3.3 The Volume of $\mathrm{Nef}_{Y}/\mathrm{aut}(Y)$

In this subsection, we give a formula (Theorem 3.4) for $\mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))$ under the assumption that

$$\begin{aligned} \text {the group } {\mathrm {O}}(T_X, \omega ) \hbox { in } \text {Definition}~1.4 \hbox { is } \{\pm 1\}. \end{aligned}$$

(3.1)

We put

$$\begin{aligned} G_X:=\{\,{{\tilde{g}}\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})}\mid {{\tilde{g}}\text { commutes with } \varepsilon \hbox { and acts on } S_{X}^{\vee }/S_{X}\hbox { trivially}} \,\}. \end{aligned}$$

(3.2)

Then ${\tilde{g}}\mapsto ({\tilde{g}}| S_{X+}, {\tilde{g}}| S_{X-})$ embeds $G_X$ into ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})\times {\mathrm {O}}(S_{X-})$. Let $G_{X+}$ and $G_{X-}$ denote the images of the projections $G_X\rightarrow {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})$ and $G_X\rightarrow {\mathrm {O}}(S_{X-})$, respectively. When we regard $G_{X+}$ as a subgroup of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ via the identification $S_{X+}=S_{Y}(2)$ induced by $\pi ^*$, we write $G_Y$ instead of $G_{X+}$. Recall that the set ${{\mathcal {R}}}(Y)$ of smooth rational curves on Y is embedded into $S_{Y}$ by $C\mapsto [C]$. The correspondence

$$\begin{aligned} C \mapsto \mathrm{Nef}_{Y}\cap ([C])^{\perp }\end{aligned}$$

gives a bijection from ${{\mathcal {R}}}(Y)$ to the set of walls of the $S_{X}/S_{Y}(2)$-chamber $\mathrm{Nef}_{Y}$. We denote by $W({{\mathcal {R}}}(Y))$ the subgroup of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ generated by the reflections $s_{[C]}$ with respect to the roots $[C]\in {{\mathcal {R}}}(Y)$. Recall also that $\mathrm{aut}(Y)$ is the image of the natural representation ${\mathrm {Aut}}(Y)\rightarrow {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$.

Proposition 3.2

Suppose that Y satisfies (3.1).

• (1) The action of $G_Y$ on ${{\mathcal {P}}}_Y$ preserves the set of $S_{X}/S_{Y}(2)$-chambers, and $\mathrm{aut}(Y)$ is equal to the stabilizer subgroup of $\mathrm{Nef}_{Y}$ in $G_Y$.

• (2) The group $W({{\mathcal {R}}}(Y))$ is contained in $G_Y$ as a normal subgroup, and we have $G_Y=W({{\mathcal {R}}}(Y))\rtimes \mathrm{aut}(Y)$.

Proof

Since every $g\in G_Y$ lifts to an element ${\tilde{g}}$ of $G_X\subset {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$, the action of $G_Y$ on ${{\mathcal {P}}}_Y$ preserves the tessellation of ${{\mathcal {P}}}_Y$ by $S_{X}/S_{Y}(2)$-chambers.

Let $\mathrm{aut}(X)$ be the image of the natural representation ${\mathrm {Aut}}(X)\rightarrow {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$. By the Torelli theorem for complex K3 surfaces ( [3, Chapter VIII]), we have a natural embedding

$$\begin{aligned} {\mathrm {Aut}}(X)\hookrightarrow {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})\times {\mathrm {O}}(T_X, \omega ), \end{aligned}$$

(3.3)

and an element $({\tilde{g}}, f)$ of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})\times {\mathrm {O}}(T_X, \omega )$ belongs to ${\mathrm {Aut}}(X)$ if and only if $({\tilde{g}}, f)$ preserves the overlattice $H^2(X, {{\mathbb {Z}}})$ of $S_{X}\oplus T_X$ and ${\tilde{g}}$ preserves $\mathrm{Nef}_{X}$. The even unimodular overlattice $H^2(X, {{\mathbb {Z}}})$ of $S_{X}\oplus T_X$ induces an isomorphism

$$\begin{aligned} i_{H(X)} {:}\,S_{X}^{\vee }/S_{X}\;\cong \; T_X^{\vee }/T_X \end{aligned}$$

of discriminant groups, and $({\tilde{g}}, f)$ preserves $H^2(X, {{\mathbb {Z}}})$ if and only if the action of ${\tilde{g}}$ on $S_{X}^{\vee }/S_{X}$ is compatible with the action of f on $T_X^{\vee }/T_X$ via $i_{H(X)}$ (see [22]). Therefore, by assumption (3.1), an isometry ${\tilde{g}}\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$ belongs to $\mathrm{aut}(X)$ if and only if ${\tilde{g}}$ preserves $\mathrm{Nef}_{X}$ and acts on $S_{X}^{\vee }/S_{X}$ as $\pm 1$.

Let ${\mathrm {Aut}}(X, \varepsilon )$ denote the centralizer of $\varepsilon$ in ${\mathrm {Aut}}(X)$. We have a natural identification ${\mathrm {Aut}}(Y)\cong {\mathrm {Aut}}(X, \varepsilon )/\langle {\varepsilon } \rangle$. Suppose that $g\in \mathrm{aut}(Y)$. We will show that g belongs to the stabilizer subgroup of $\mathrm{Nef}_{Y}$ in $G_Y$. It is obvious that g preserves $\mathrm{Nef}_{Y}$. Let ${\tilde{\gamma }}$ be an element of ${\mathrm {Aut}}(X, \varepsilon )$ that induces g on $S_{Y}$. We write ${\tilde{\gamma }}$ as $({\tilde{g}}, f)$ by (3.3). Note that $\varepsilon$ acts on $T_X$ as $-1$. Hence, replacing ${\tilde{\gamma }}$ with ${\tilde{\gamma }}\varepsilon$ if $f=-1$, we can assume $f=1$. Then the action ${\tilde{g}}\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$ of ${\tilde{\gamma }}$ on $S_{X}$ induces the trivial action on $S_{X}^{\vee }/S_{X}$, which means ${\tilde{g}}\in G_X$. Hence $g={\tilde{g}}|S_{Y}$ belongs to $G_Y$.

Conversely, suppose that g is an element of the stabilizer subgroup of $\mathrm{Nef}_{Y}$ in $G_Y$. We will show that $g\in \mathrm{aut}(Y)$. Let ${\tilde{g}}$ be an element of $G_X$ such that $g={\tilde{g}}|S_{Y}$. Since $\mathrm{Nef}_{Y}$ contains an interior point of $\mathrm{Nef}_{X}$, ${\tilde{g}}$ preserves $\mathrm{Nef}_{X}$, and hence ${\tilde{g}}$ belongs to $\mathrm{aut}(X)$. Let ${\tilde{\gamma }} =({\tilde{g}}, f)$ be an element of ${\mathrm {Aut}}(X)$ that induces ${\tilde{g}}$. Since ${\tilde{g}}\in G_X$ commutes with the action of $\varepsilon$ on $S_{X}$, the first factor of the commutator $[{\tilde{\gamma }}, \varepsilon ]\in {\mathrm {Aut}}(X)$ is 1. Since ${\mathrm {O}}(T_X, \omega )=\{\pm 1\}$ is abelian, the second factor of $[{\tilde{\gamma }}, \varepsilon ]$ is also 1. Hence ${\tilde{\gamma }} \in {\mathrm {Aut}}(X, \varepsilon )$, and therefore g is induced by an element of ${\mathrm {Aut}}(Y)$. Thus assertion (1) is proved.

By Proposition 3.1, for each $r\in {{\mathcal {R}}}(Y)$, the reflection $s_r=s_{{\tilde{r}}_+} s_{{\tilde{r}}_-}|S_{Y}(2)$ belongs to $G_Y$, because the reflections $s_{{\tilde{r}}_+}$ and $s_{{\tilde{r}}_-}$ act on $S_{X}^{\vee }/S_{X}$ trivially and hence $s_{{\tilde{r}}_+} s_{{\tilde{r}}_-}\in G_X$. Therefore we have $W({{\mathcal {R}}}(Y))\subset G_Y$. Moreover, by Proposition 3.1 again, we see that $W({{\mathcal {R}}}(Y))$ acts on the set of $S_{X}/S_{Y}(2)$-chambers transitively.

If $C_1, C_2\in {{\mathcal {R}}}(Y)$ satisfy $\langle C_1, C_2 \rangle _Y>1$, then the walls $\mathrm{Nef}_{Y}\cap ([C_1])^{\perp }$ and $\mathrm{Nef}_{Y}\cap ([C_2])^{\perp }$ of $\mathrm{Nef}_{Y}$ do not intersect. Hence each face of $\mathrm{Nef}_{Y}$ with codimension 2 is of the form

$$\begin{aligned} \mathrm{Nef}_{Y}\cap ([C_1])^{\perp }\cap ([C_2])^{\perp }\quad \text{ with } \quad \langle C_1, C_2 \rangle _Y\in \{0, 1\}, \end{aligned}$$

and we have $(s_{[C_1]} s_{[C_2]})^m=1$, where $m=2$ if $\langle C_1, C_2 \rangle _Y=0$ and $m=3$ if $\langle C_1, C_2 \rangle _Y=1$. Therefore, by the standard method of geometric group theory (see, for example, Section 1.5 of [36]), we see that $\mathrm{Nef}_{Y}$ is a standard fundamental domain of the action of $W({{\mathcal {R}}}(Y))$ on ${{\mathcal {P}}}_Y$, and $W({{\mathcal {R}}}(Y))$ acts on the set of $S_{X}/S_{Y}(2)$-chambers simply-transitively. Recalling that $\mathrm{aut}(Y)$ is the stabilizer subgroup of $\mathrm{Nef}_{Y}$ in $G_Y$, we have $W({{\mathcal {R}}}(Y))\cap \mathrm{aut}(Y)=\{1\}$. Moreover $G_Y$ is generated by the union of $W({{\mathcal {R}}}(Y))$ and $\mathrm{aut}(Y)$.

It remains to show that $W({{\mathcal {R}}}(Y))$ is a normal subgroup of $G_Y$. Let r be a root in ${{\mathcal {R}}}(Y)$ and g an arbitrary element of $G_Y$. It is enough to show that $g^{-1}s_r g$ belongs to $W({{\mathcal {R}}}(Y))$. Note that $g^{-1}s_r g=s_{r^g}$ and $r^g$ defines a wall of the $S_{X}/S_{Y}(2)$-chamber $D_Y:={\mathrm{Nef}_{Y}}^g$. We have an element $w\in W({{\mathcal {R}}}(Y))$ such that $D_Y={\mathrm{Nef}_{Y}}^w$. Then $r^\prime :=r^{gw^{-1}}$ defines a wall of $\mathrm{Nef}_{Y}$, and $w s_{r^g} w^{-1}=s_{r^\prime }$ is an element of $W({{\mathcal {R}}}(Y))$. Hence $g^{-1}s_r g=s_{r^g}=w^{-1}s_{r^\prime } w \in W({{\mathcal {R}}}(Y))$. $\square$

Let $(A_+, q_+)$ and $(A_-, q_-)$ be the discriminant forms of $S_{X+}=S_{Y}(2)$ and $S_{X-}$, respectively. We put

$$\begin{aligned} \Gamma _X:=S_{X}/(S_{X+}\oplus S_{X-})\;\; \subset \;\; A_+\oplus A_-, \end{aligned}$$

and let $D_+\subset A_+$ and $D_-\subset A_-$ be the image of the projections of $\Gamma _X$. Then $\Gamma _X$ is the graph of an isometry $(D_+, q_+|D_+)\cong (D_-, -q_-|D_-)$, and the discriminant group of $S_{X}$ is canonically isomorphic to $\Gamma _X^{\perp }/\Gamma _X$. We denote by $\overline{G}_{X+}$ and $\overline{G}_{X-}$ the images of $G_{X+}$ and $G_{X-}$ by the natural homomorphisms ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})\rightarrow {\mathrm {O}}(A_{+})$ and ${\mathrm {O}}(S_{X-})\rightarrow {\mathrm {O}}(A_{-})$, respectively, and by $\overline{G}_{X}$ the image of $G_X$ by the natural homomorphism to ${\mathrm {O}}(A_{+}) \times {\mathrm {O}}(A_{-})$. Note that $\overline{G}_X$ is a subgroup of the kernel K of the natural homomorphism

$$\begin{aligned} \{\,{(g, h) \in {\mathrm {O}}(A_+)\times {\mathrm {O}}(A_-)}\mid {\Gamma _X^{(g, h)}=\Gamma _X} \,\}\rightarrow {\mathrm {O}}(\Gamma _X^{\perp }/\Gamma _X)={\mathrm {O}}(S_{X}^{\vee }/S_{X}). \end{aligned}$$

Then we have a commutative diagram

(3.4)

where the two arrows below are isomorphisms by the first part of Proposition 2.1 applied to $(A, B)=(A_+, A_-)$ and $(A, B)=(A_-, A_+)$.

Lemma 3.3

Suppose that Y satisfies (3.1). Then the group

$$\begin{aligned} {\mathrm {Aut}}_{nt}(Y):={\mathrm {Ker}}({\mathrm {Aut}}(Y) \rightarrow \mathrm{aut}(Y)) \end{aligned}$$

of numerically trivial automorphisms of Y is isomorphic to the kernel of the natural homomorphism $G_{X-} \rightarrow \overline{G}_{X-}$.

Proof

There is an isomorphism of ${\mathrm {Aut}}_{nt}(Y)$ with ${\mathrm {Ker}}\left( G_X \rightarrow G_{X+}\right)$ given by mapping a numerically trivial automorphism g to its lift ${\tilde{g}} \in {\mathrm {Aut}}(X)$ acting trivially on the 2-form $\omega$ and restricting to its action on $S_{X}$. By the diagram (3.4) and $G_X\subset G_{X+}\times G_{X-}$, the kernel ${\mathrm {Ker}}\left( G_X \rightarrow G_{X+}\right)$ injects into ${\mathrm {Ker}}(G_{X-} \rightarrow \overline{G}_{X-})$. Conversely any element of ${\mathrm {Ker}}(G_{X-} \rightarrow \overline{G}_{X-})$ can be extended to an element of ${\mathrm {Ker}}\left( G_X \rightarrow G_{X+}\right)$ by complementing it with the trivial action of $S_{X+}$. $\square$

Theorem 3.4

Suppose that Y satisfies (3.1). Let ${\mathrm {O}}(S_{X-}, D_-)$ be the subgroup of ${\mathrm {O}}(S_{X-})$ consisting of isometries g whose action on $A_-$ preserves $D_-$. Then we have

$$\begin{aligned} G_{X-}={\mathrm {Ker}}({\mathrm {O}}(S_{X-}, D_-) \rightarrow {\mathrm {O}}(D_-^{\perp })). \end{aligned}$$

(3.5)

Moreover we have

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))=\frac{1_{\mathrm{{BP}}}}{|\overline{G}_{X-}|}=\frac{|{\mathrm {Aut}}_{nt}(Y)|}{|G_{X-}|}1_{\mathrm{{BP}}}. \end{aligned}$$

(3.6)

Proof

Recall that we have $|\overline{G}_X|=|\overline{G}_{X+}|=|\overline{G}_{X-}|$. Let $G_{\mathrm{{BP}}}$ be the kernel of the natural homomorphism ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})={{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y}(2))\rightarrow {\mathrm {O}}(A_+)$. Then $G_{\mathrm{{BP}}}$ is equal to $\mathrm{aut}(Y_0)$ by Theorem 1.17 and hence the index of $G_{\mathrm{{BP}}}$ in ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})= {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ is $1_{\mathrm{{BP}}}$. If $g\in G_{\mathrm{{BP}}}$, then $(g, 1)\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})\times {\mathrm {O}}(S_{X-})$ acts trivially on $A_+\oplus A_-$, and hence preserves $\Gamma _X$ and acts on $\Gamma _X^{\perp }/\Gamma _X$ trivially. Therefore the action of (g, 1) on $S_{X+}\oplus S_{X-}$ preserves the overlattice $S_{X}$, and $(g, 1)|S_{X}$ is an element of $G_X$. Thus $G_{\mathrm{{BP}}}$ is contained in $G_{X+}=G_Y$. Since the natural homomorphism ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y}(2))\rightarrow {\mathrm {O}}(A_+)$ is surjective (see [2]), the index of $G_{\mathrm{{BP}}}$ in $G_Y$ is equal to $|\overline{G}_{X+}|=|\overline{G}_{X-}|$.

Applying the second part of Proposition 2.1 to $(A, B)=(A_-, A_+)$, we see that

$$\begin{aligned} \overline{G}_{X-} \subset {\mathrm {Im}}\, i_{A_-}={\mathrm {Ker}}({\mathrm {O}}(A_-, D_-)\rightarrow {\mathrm {O}}(D_-^{\perp })). \end{aligned}$$

Hence the inclusion $\subset$ in (3.5) is proved. Conversely, let f be an element of the right-hand side of (3.5), and denote by ${\bar{f}}\in {\mathrm {O}}(A_-)$ the action of f on $A_-$. By Proposition 2.1, we have ${\bar{f}}\in {\mathrm {Im}}\, i_{A_-}$ and hence there exists a unique element ${\bar{h}}\in K$ such that $i_{A_-}({\bar{h}})={\bar{f}}$. We put ${\bar{g}}:=i_{A_+}({\bar{h}})$. Since the natural homomorphism ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})\rightarrow {\mathrm {O}}(A_+)$ is surjective, we have $g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X+})$ that maps to ${\bar{g}}$. Since ${\bar{h}}=({\bar{g}}, {\bar{f}})\in K$, we have $(g, f)\in G_X$, which implies $f\in G_{X-}$. Thus (3.5) is proved. Moreover we have

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))=\mathrm{{vol}}({{\mathcal {P}}}_Y/G_Y)=\frac{1}{[G_Y: G_{\mathrm{{BP}}}]}\cdot \mathrm{{vol}}({{\mathcal {P}}}_Y/G_{\mathrm{{BP}}})=\frac{1_{\mathrm{{BP}}}}{|\overline{G}_{X-}|}, \end{aligned}$$

where the first equality follows from Proposition 3.2. From Lemma 3.3 we get the second equality of (3.6). $\square$

Since $S_{X-}$ is negative definite, ${\mathrm {O}}(S_{X-})$ is a finite group and can be computed easily. Thus this formula enables us to calculate $\mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))$.

3.4 Proof of Theorem 1.18

In what follows we calculate the finite group $\overline{G}_{X-}$ of a $(\tau , {\bar{\tau }})$-generic Enriques surface. It is closely related to the Weyl group $W(R_{\tau })$.

For a sublattice $L^\prime$ of a lattice L, we denote by ${\mathrm {O}}(L, L^\prime )$ the group of isometries of L preserving $L^\prime$. When L is an overlattice of $L^\prime$, then ${\mathrm {O}}(L, L^\prime )$ is the group of isometries of $L^\prime$ preserving the overlattice L, or equivalently the intersection ${\mathrm {O}}(L)\cap {\mathrm {O}}(L^\prime )$ in ${\mathrm {O}}(L\otimes {{\mathbb {Q}}})={\mathrm {O}}(L^\prime \otimes {{\mathbb {Q}}})$, and hence sometimes is written as ${\mathrm {O}}(L^\prime , L)$.

Lemma 3.5

Let Y be $(\tau ,{\bar{\tau }})$-generic. Recall the commutative diagram (1.1)

Denote by $\pi _-{:}\,S_{X}\rightarrow S_{X-}^{\vee }$ the orthogonal projection. Identify $M_R$ with $S_X$ via ${\tilde{g}}$. Then the following equalities hold:

$$\begin{aligned} \begin{array}{lll} (1) \quad {\widetilde{R}}= S_{X-}, &{}\quad &{}\quad (4)\quad \tfrac{1}{2}R/ {\widetilde{R}}= D_{-},\\ (2) \quad R= \pi _-(2 S_{X}), &{}\quad &{}\quad (5)\quad R^{\vee }/ {\widetilde{R}}=D_{-}^\perp ,\\ (3)\quad \tfrac{1}{2}{\widetilde{R}}^{\vee }/ {\widetilde{R}}=A_-, &{}\quad &{}\quad (6)\quad {\mathrm {O}}({\widetilde{R}},R)={\mathrm {O}}(S_{X-},D_-).\\ \end{array} \end{aligned}$$

Note that we neglect the quadratic forms in (1)–(5) and just consider them as equalities of abelian groups.

Proof

The equality (1) is by the definition.

• (2) Note that $M_R$ is spanned by ${\mathrm {Im}}\, \varpi _R$ and $\{(i_R(v) \pm v)/2 \mid v \in R\}$. Hence $\pi _-(M_R)$ is spanned by 0 and $\tfrac{1}{2}R$.

• (3) As lattices we have ${\widetilde{R}}(2) = S_{X-}$, and $({\widetilde{R}}(2))^{\vee }= \tfrac{1}{2}{\widetilde{R}}^{\vee }$ yields the claim.

• (4) By definition, we have $\pi _{-}(S_{X})/S_{X-}=D_{-}$.

• (5) Let $x \in \tfrac{1}{2}R$ and $y \in R^{\vee }$. Then $\langle x, y \rangle _{M_R} = 2 \langle x, y \rangle _R\equiv 0 \mod {{\mathbb {Z}}}$ and $x +{\widetilde{R}}\in D_-$. This shows that $R^{\vee }/{\widetilde{R}}\subset D_-^\perp$. Conversely let $x +{\widetilde{R}}\in D_-^\perp$. For $y \in R$ we have $\langle x,y \rangle _R = \tfrac{1}{2} \langle x, y \rangle _{M_R} =\langle x,\tfrac{1}{2} y \rangle _{M_R} \equiv 0 \mod {{\mathbb {Z}}}$ because $\tfrac{y}{2}+{\widetilde{R}}\in D_-=\tfrac{1}{2}R/{\widetilde{R}}$. This shows that $x \in R^{\vee }$.

• (6) ${\mathrm {O}}({\widetilde{R}},\tfrac{1}{2}R/{\widetilde{R}}) = {\mathrm {O}}({\widetilde{R}},\tfrac{1}{2}R) = {\mathrm {O}}({\widetilde{R}},R)$. $\square$

Let $R$ be an $\mathrm{ADE}$-lattice and $\Phi$ the set of its roots. We fix a subset $\Phi ^+\subset \Phi$ of positive roots. There exists a unique Weyl-chamber C of $R$ (see Definition 2.5) such that for all $r \in \Phi ^+$ and $c \in C$ we have $\langle r, c\rangle >0$. We call C the fundamental chamber. The positive roots perpendicular to the walls of C are the so-called simple roots. The simple roots form a basis of $R$ whose Dynkin diagram is of $\mathrm{ADE}$-type $\tau (R)$. As before we have ${\mathrm {O}}(R) = W(R) \rtimes {\mathrm {O}}(R,C)$, where ${\mathrm {O}}(R,C)$ is the stabilizer of C in ${\mathrm {O}}(R)$. Via the action of ${\mathrm {O}}(R,C)$ on the vertices of the Dynkin diagram, we identify ${\mathrm {O}}(R,C)$ with the symmetry group ${\mathrm {Aut}}(\tau (R))$ of the Dynkin diagram $\tau (R)$, that is, we have

$$\begin{aligned} {\mathrm {O}}(R)=W(R) \rtimes {\mathrm {Aut}}(\tau (R)). \end{aligned}$$

A lattice is called irreducible if it cannot be written as a non-trivial orthogonal sum of two sublattices. Definite lattices admit an orthogonal decomposition into irreducible sublattices which is unique up to reordering (cf. [14, 27.1]).

Lemma 3.6

Let $R$ be an $\mathrm{ADE}$-lattice, and let ${\mathrm {O}}_0(R)$ be the kernel of the natural homomorphism ${\mathrm {O}}(R) \rightarrow {\mathrm {O}}(R^{\vee }/R)$. Then we have

$$\begin{aligned}{}[{\mathrm {O}}_0(R){:}\,W(R)]=n!, \end{aligned}$$

where n is the number of $E_8$ components of $\tau (R)$.

Proof

Since reflections with respect to roots act trivially on the discriminant group, we have $W(R) \subseteq {\mathrm {O}}(R)_0$. Thus it suffices to compute the kernel of

$$\begin{aligned} \psi {:}\,{\mathrm {Aut}}(\tau (R))\rightarrow {\mathrm {O}}(R^{\vee }/R). \end{aligned}$$

If $\tau (R)$ is irreducible, a case by case analysis shows that this map is injective: indeed for $A_1$, $E_7$ and $E_8$, ${\mathrm {Aut}}(\tau (R))=1$; for $A_k$ with $k \ge 1$, $D_k$ with $k>4$ and $E_6$ the group ${\mathrm {Aut}}(\tau (R))$ is of order two. A direct computation shows that it acts faithfully on the discriminant group.

Suppose that the root system $\tau (R)$ is reducible. The decomposition of $\tau (R)$ into connected components corresponds to a decomposition of $R$ into an orthogonal sum of irreducible $\mathrm{ADE}$-lattices, which in turn induces a corresponding decomposition of the discriminant group $R^{\vee }/R$. The action of ${\mathrm {Aut}}(\tau (R))$ preserves the three decompositions. Hence the elements of ${\mathrm {Ker}}\, \psi$ must preserve the components which have a non-trivial discriminant group, that is, all components which are not of type $E_8$. By the first part, they must act trivially on these components. Finally, since the $E_8$ diagram has no symmetry, the elements in the kernel act as a permutation of the connected components of $\tau (R)$ of type $E_8$. $\square$

Lemma 3.7

Let $R$ be an $\mathrm{ADE}$-lattice of rank at most 10 and ${\widetilde{R}}$ an even overlattice. Consider the homomorphism

$$\begin{aligned} {\mathrm {O}}(R,{\widetilde{R}}) \rightarrow {\mathrm {O}}(R^{\vee }/{\widetilde{R}}). \end{aligned}$$

(3.7)

If there is a component ${\widetilde{R}}_j$ of ${\widetilde{R}}$ with $\tau ({\widetilde{R}}_j)=E_8$ and $\tau ({\widetilde{R}}_j \cap R)=2D_4$, then the kernel of (3.7) is $W(R)\rtimes \langle h \rangle$ where $h \in {\mathrm {Aut}}(\tau (R),{\widetilde{R}})$ is an involution. Otherwise the kernel is just the Weyl group $W(R)$.

Proof

Let ${\mathrm {Aut}}(\tau (R), {\widetilde{R}}) \le {\mathrm {Aut}}(\tau (R))$ be the stabilizer of ${\widetilde{R}}$. Since the elements of $W(R)$ act trivially on $R^{\vee }/R$, they preserve ${\widetilde{R}}$ and

$$\begin{aligned} {\mathrm {O}}({\widetilde{R}}, R)=W(R) \rtimes {\mathrm {Aut}}(\tau (R),{\widetilde{R}}) \le W(R) \rtimes {\mathrm {Aut}}(\tau (R)). \end{aligned}$$

The elements of $W(R)$ act trivially on the domain of $R^{\vee }/R\twoheadrightarrow R^{\vee }/ {\widetilde{R}}$, so they lie in the kernel of (3.7). Thus it suffices to compute the kernel of

$$\begin{aligned} \varphi {:}\,{\mathrm {Aut}}(\tau (R), {\widetilde{R}}) \rightarrow {\mathrm {O}}(R^{\vee }/{\widetilde{R}}). \end{aligned}$$

Indeed, the kernel of (3.7) is given by $W(R) \rtimes {\mathrm {Ker}}\,\varphi$.

First we suppose that $\tau (R)$ is irreducible. If $R= {\widetilde{R}}$, then $W(R)={\mathrm {O}}_0(R)$ by Lemma 3.6, and hence $\varphi$ is injective. Otherwise (as ${{\,\mathrm{rank}\,}}\, R\le 10$) the pair $(\tau (R),\tau ({\widetilde{R}})) \in \{(A_7,E_7),(A_8,E_8), (D_8,E_8)\}$. Suppose we are in the case $(A_7,E_7)$. Then $R^{\vee }/R\cong {{\mathbb {Z}}}/8{{\mathbb {Z}}}$ and ${\widetilde{R}}/R= 4 (R^{\vee }/R)$. Then ${\mathrm {Aut}}(\tau (R))$ is of order two and acts as $\pm 1$ on $R^{\vee }/R$ which is non-trivial in $R^{\vee }/{\widetilde{R}}\cong {{\mathbb {Z}}}/4{{\mathbb {Z}}}$. A similar argument applies to $(A_8,E_8)$. Finally the symmetry of the $D_8$ diagram exchanges the two isotropic vectors of its discriminant. In particular it does not fix any non-trivial even overlattice which implies that ${\mathrm {Aut}}(\tau (R),{\widetilde{R}})=1$ in the $(D_8,E_8)$ case. In any case $\varphi$ is injective.

Now suppose that $R=\bigoplus R_i$ has several irreducible components $R_i$ and let $h \in {\mathrm {Ker}}\, \varphi$. Note that h preserves the decomposition $R^{\vee }=\bigoplus R_i^{\vee }$. Let $x \in R_i^{\vee }$ be a nonzero element.

If $x^h$ lies in the same component $R_i^{\vee }$ as x, then h must preserve it. Hence we may restrict h to this component and the previous paragraph yields $x^h=x$.

If x and $x^h$ lie in different components $R_i^{\vee }$ and $R_j^{\vee }$, then these components are isomorphic and $q(x^h-x)=q(x^h)+q(x) = 2q(x)$. Since $h \in {\mathrm {Ker}}\, \varphi$, we have $x^h - x \in {\widetilde{R}}$ Further ${\widetilde{R}}/R$ is totally isotropic with respect to the discriminant form. Thus $q(x^h-x)=2q(x) \equiv 0 \mod 2{{\mathbb {Z}}}$, i.e. $q(x) \equiv 0 \mod {{\mathbb {Z}}}$. If y is any non-trivial element of $R_i^{\vee }$, then $x^h$ and $y^h$ lie in the same connected component $R_j^{\vee }$ and the same reasoning applies. In particular

$$\begin{aligned} \forall y \in R_i^{\vee }{:}\,q(y) \equiv 0 \mod {{\mathbb {Z}}}\end{aligned}$$

which implies that $R_i$ is 2-elementary and $q_{R_i}$ has values in ${{\mathbb {Z}}}/2{{\mathbb {Z}}}$. Under the constraint ${{\,\mathrm{rank}\,}}\, R\le 10$, this is possible only if $\tau (R_i)=\tau (R_j)=D_4$. To sum up $\varphi$ is injective, except possibly if $\tau (R)$ has two $D_4$ components. We analyse this case in detail.

We may assume that $R=R_1\oplus R_2$ is of type $2D_4$ and ${\tilde{R}}$ an overlattice of R. If ${\widetilde{R}}= R$, then $\varphi$ is injective by Lemma 3.6. Hence we may further assume that $R\subsetneq {\widetilde{R}}$. Suppose there exists a non-trivial element h in the kernel of $\varphi$. By the previous part this implies that $R_1^h=R_2$.

Let $e_1,e_2,e_3,e_4$ be the simple roots of $R_1$ with $e_4$ giving the central vertex of the Dynkin diagram of type $D_4$, i.e. $\langle e_4,e_i \rangle =1$ for $i=1,2,3$. Let $(e_1^{\vee }, \dots , e_4^{\vee }) \in R_1^{\vee }$ be the dual basis. The four elements of $R_1^{\vee }/R_1$ are represented by $e_1^{\vee }, e_2^{\vee }, e_3^{\vee }$ and $e_4^{\vee }$ representing 0. Set $f_i = e_i^h \in R_2$. Then $f_i^h = e_{\sigma (i)}$ for some permutation $\sigma \in S_4$ with $\sigma (4)=4$. Since $h \in {\mathrm {Ker}}\, \varphi$, we have $t_i:=e_i^{\vee }- f_i^{\vee }\in {\widetilde{R}}$ for $i\in \{1,2,3\}$. Now the cosets of 0, $t_1$, $t_2$ and $t_3$ constitute a maximal totally isotropic subspace of $R^{\vee }/R$ contained in ${\widetilde{R}}/R$. Since ${\widetilde{R}}/R$ is totally isotropic as well, the subspaces must be equal. We conclude that $\tau ({\widetilde{R}})=E_8$. By the same reasoning we have $f_i^{\vee }-e_{\sigma (i)}^{\vee }\in {\widetilde{R}}$. As ${\widetilde{R}}/R$ has only four elements, this is possible only if $\sigma =1$. Hence h is an involution and uniquely determined by ${\widetilde{R}}/R$. This shows that the kernel of $\varphi$ is of order 2. $\square$

Lemma 3.8

Let ${\widetilde{R}}$ be an $\mathrm{ADE}$-lattice and $\Phi ^+$ the set of its positive roots. Then the natural map $\Phi ^+ \rightarrow {\widetilde{R}}/2{\widetilde{R}}$ is injective.

Proof

We may assume that ${\widetilde{R}}$ is irreducible. In what follows we explicitly compute $\eta {:}\,\Phi ^+ \rightarrow {\widetilde{R}}/2{\widetilde{R}}$ for each case using classical constructions of the $\mathrm{ADE}$-lattices (see e.g. [10, Theorem 1.2]).

Let $(\epsilon _1, \dots , \epsilon _{n+1})$ be the standard basis of ${{\mathbb {Z}}}^{n+1}$. The $n(n+1)$ roots of the lattice

$$\begin{aligned} A_n = \left\{ (x_i) \in {{\mathbb {Z}}}^{n+1}{:}\,\sum _{i=1}^{n+1} x_i = 0 \right\} \end{aligned}$$

are given by

$$\begin{aligned} \Phi (A_n)=\{\alpha _{ij}=\epsilon _i - \epsilon _j \mid 1\le i \ne j \le n+1\}. \end{aligned}$$

Suppose that $\alpha _{ij}\equiv \alpha _{lk} \mod 2A_n\subseteq 2{{\mathbb {Z}}}^{n+1}$. Then we have that $\epsilon _i - \epsilon _j +\epsilon _k - \epsilon _l \equiv 0 \mod 2{{\mathbb {Z}}}^{n+1}$. This is possible only if each standard basis vector appears twice, i.e. $(i,j)=(k,l)$ or $(i,j)=(l,k)$ which means that $\alpha _{ij} = \pm \alpha _{lk}$. Since either $\alpha _{lk}\in \Phi ^+$ or $-\alpha _{lk} \in \Phi ^+$, the map $\eta$ is injective.

Let $(\epsilon _1, \dots , \epsilon _{n})$ be the standard basis of ${{\mathbb {Z}}}^{n}$, $n\ge 4$. The $2n(n-1)$ roots of the lattice

$$\begin{aligned} D_n = \left\{ (x_i) \in {{\mathbb {Z}}}^{n}{:}\,\sum _{i=1}^n x_i \equiv 0 \mod 2 \right\} \end{aligned}$$

are given by $\pm (\epsilon _i + \epsilon _j)$ and $\pm (\epsilon _i - \epsilon _j)$ for $1\le i < j \le n$. Suppose that $\pm \epsilon _i \pm \epsilon _j \equiv \pm \epsilon _k \pm \epsilon _l \mod 2D_n$. As before this implies that $\{i,j\} = \{k,l\}$. Since

$$\begin{aligned} (\epsilon _i + \epsilon _j)- (\epsilon _i - \epsilon _j)=2\epsilon _j \notin 2D_4, \end{aligned}$$

the map $\eta$ is injective. We leave the exceptional cases $E_6,E_7,E_8$ to the reader. $\square$

Lemma 3.9

Let ${\widetilde{R}}=\bigoplus _{j \in J} {\widetilde{R}}_j$ be an $\mathrm{ADE}$-lattice with ${\widetilde{R}}_j$ irreducible. Then the kernel of the natural homomorphism

$$\begin{aligned} \psi {:}\,{\mathrm {O}}({\widetilde{R}})={\mathrm {O}}({\widetilde{R}}(2)) \rightarrow {\mathrm {O}}(\tfrac{1}{2}{\widetilde{R}}^{\vee }/{\widetilde{R}}), \end{aligned}$$

where $\tfrac{1}{2}{\widetilde{R}}^{\vee }/{\widetilde{R}}$ is the discriminant form of ${\widetilde{R}}(2)$, is generated by the elements $\oplus _{j\in J} g_j$ with $g_j = \pm 1_{{\widetilde{R}}_j}$ if ${\widetilde{R}}_j$ is unimodular and $g_j = 1_{{\widetilde{R}}_j}$ otherwise.

Proof

We identify $\tfrac{1}{2}{\widetilde{R}}^{\vee }/{\widetilde{R}}$ and ${\widetilde{R}}^{\vee }/2{\widetilde{R}}$. Let $g \in {\mathrm {Ker}}\, \psi$. Since ${\widetilde{R}}\subseteq {\widetilde{R}}^{\vee }$, g acts trivially on ${\widetilde{R}}^{\vee }/2{\widetilde{R}}^{\vee }$. The action of ${\mathrm {O}}({\widetilde{R}})$ preserves the decomposition ${\widetilde{R}}= \bigoplus _{j \in J} {\widetilde{R}}_j$. In particular g acts on the set J. As ${\widetilde{R}}^{\vee }/2{\widetilde{R}}^{\vee }= \bigoplus _{j \in J}{\widetilde{R}}_j^{\vee }/2{\widetilde{R}}_j^{\vee }$ and g is in ${\mathrm {Ker}}\, \psi$ we have $j^g=j$. Hence g must fix each connected component of ${\widetilde{R}}$ and we may and will assume that ${\widetilde{R}}$ is irreducible.

We tensor the perfect pairing ${\widetilde{R}}^{\vee }\times {\widetilde{R}}\rightarrow {{\mathbb {Z}}}$ with ${{\mathbb {F}}}_2$, to obtain a perfect pairing ${\widetilde{R}}^{\vee }/2{\widetilde{R}}^{\vee }\times {\widetilde{R}}/2{\widetilde{R}}\rightarrow {{\mathbb {F}}}_2$. Since g acts trivially on the first factor, so does it on the second factor ${\widetilde{R}}/2{\widetilde{R}}$. By Lemma 3.8$\Phi ({\widetilde{R}})/\{\pm 1\} \cong \Phi ^+({\widetilde{R}})$ injects into ${\widetilde{R}}/2{\widetilde{R}}$, which implies that $g(r)= \pm r$ for every root $r\in \Phi ({\widetilde{R}})$. As any simple root system of ${\widetilde{R}}$ is connected, the sign is the same for each simple root. Since the simple roots form a basis, $g=\pm 1$.

Set ${\widetilde{R}}_\pm = {\mathrm {Ker}}(g\mp 1)\subset {\widetilde{R}}$. We apply Proposition 2.1 to the primitive extension

$$\begin{aligned} {\widetilde{R}}_+(2) \oplus {\widetilde{R}}_-(2) \subseteq {\widetilde{R}}(2). \end{aligned}$$

Since g acts trivially on the discriminant group $\tfrac{1}{2}{\widetilde{R}}^{\vee }/{\widetilde{R}}$ of ${\widetilde{R}}(2)$, the implication

$$\begin{aligned} g|_{\tfrac{1}{2}{\widetilde{R}}_{+}^{\vee }/{\widetilde{R}}_+}=1 \implies g|_{\tfrac{1}{2}{\widetilde{R}}_{-}^{\vee }/{\widetilde{R}}_-}=1 \end{aligned}$$

(3.8)

holds. By definition $g|_{{\widetilde{R}}_-}=-1_{{\widetilde{R}}_-}$ and then by the right hand side of (3.8), the lattice ${\widetilde{R}}_{-}(2)$ must be 2-elementary, i.e. ${\widetilde{R}}_-$ is unimodular. In particular ${\widetilde{R}}_-$ is a direct summand of ${\widetilde{R}}$. But we assumed the latter to be irreducible, so that ${\widetilde{R}}\in \{0,{\widetilde{R}}_-\}$. Thus $g=\pm 1$ if ${\widetilde{R}}$ is unimodular and $g = 1$ else. $\square$

Remark 3.10

Let $R$ be an irreducible $\mathrm{ADE}$-lattice. By [10, Proposition 1.5], we have $-1 \in W(R)$ if and only if $R$ contains ${{\,\mathrm{rank}\,}}\, R$ pairwise orthogonal roots, if and only if $\tau (R)$ is one of $A_n$ ($n\ge 1$), $D_n$ ($n\ge 4$, n even), $E_7$, $E_8$.

Theorem 3.11

Let Y be a $(\tau , {\bar{\tau }})$-generic Enriques surface, and let $R$, ${\overline{R}}$, ${\widetilde{R}}$ be as in Table 1. Let ${\widetilde{R}}= \bigoplus _j {\widetilde{R}}_j$ be the decomposition into irreducible components. Then we have

$$\begin{aligned} |\overline{G}_{X-}| = |W(R)|\; \frac{d_{(\tau ,{\bar{\tau }})}}{ e_{(\tau ,{\bar{\tau }})}}, \end{aligned}$$

where $d_{(\tau ,{\bar{\tau }})}, e_{(\tau ,{\bar{\tau }})}$ are given as follows.

$$\begin{aligned} d_{(\tau ,{\bar{\tau }})}:= & {} {\left\{ \begin{array}{ll} 2 &{}\quad \exists j \text { such that } \tau ({\widetilde{R}}_j) = E_8 \hbox { and } \tau ({\widetilde{R}}_j\cap R) = 2D_4, \\ 1 &{}\quad \text {otherwise,} \end{array}\right. } \\ e_{(\tau ,{\bar{\tau }})}:= & {} {\left\{ \begin{array}{ll} 2 &{}\quad \exists j \hbox { such that } \tau ({\widetilde{R}}_j) = E_8 \hbox { and } {\widetilde{R}}_j \cap R\hbox { contains } 8 \hbox { orthogonal roots},\\ 1 &{}\quad \text {otherwise.} \end{array}\right. } \end{aligned}$$

Hence the value of $c_{(\tau ,{\bar{\tau }})}$ in Table 1 is equal to $e_{(\tau ,{\bar{\tau }})}/ d_{(\tau ,{\bar{\tau }})}=|{\mathrm {Aut}}_{nt}(Y)|/d_{(\tau ,{\bar{\tau }})}$.

Proof

By Theorem 3.4 and Lemma 3.5, we have

$$\begin{aligned}G_{X-}= {\mathrm {Ker}}({\mathrm {O}}({\widetilde{R}}, R) \rightarrow {\mathrm {O}}(R^{\vee }/ {\widetilde{R}})), \end{aligned}$$

which, by Lemma 3.7, is given by $W(R)$, or by $W(R)\rtimes \langle h \rangle$ for some involution $h \in {\mathrm {Aut}}(\tau (R),{\widetilde{R}})$ if there is some component ${\widetilde{R}}_j$ with $\tau ({\widetilde{R}}_j)=E_8$ and $\tau ({\widetilde{R}}_j \cap R)=2D_4$. Consider the natural homomorphism $\psi {:}\,{\mathrm {O}}({\widetilde{R}}) \rightarrow {\mathrm {O}}(\tfrac{1}{2}{\widetilde{R}}^{\vee }/{\widetilde{R}})$ in Lemma 3.9. By our dictionary in Lemma 3.5, we have $\overline{G}_{X-} = \psi (G_{X-})$. By Lemma 3.9, the kernel of $\psi$ consists of those $g=\oplus _{j \in J} g_j$ with $g_j = \pm 1_{{\widetilde{R}}_j}$ if ${\widetilde{R}}_j$ is unimodular and $g_j = 1_{{\widetilde{R}}_j}$ else. Further ${\mathrm {Ker}}\, \psi \cap W(R)$ consists of those g with $g_j = \pm 1$ if ${\widetilde{R}}_j$ is unimodular and $-1 \in W(R\cap {\widetilde{R}}_j)$, and $g_j=1$ else. Now Remark 3.10 yields the condition for $e_{(\tau ,{\bar{\tau }})}$. Since the $g_j=\pm 1$ do not preserve any positive root system, the involution h is not in ${\mathrm {Ker}}\, \psi$. This explains the presence of $d_{(\tau ,{\bar{\tau }})}$. Finally, in the geometric situation, we have $G_{X-}=W(R)$ (see Remark 3.12 below), and hence ${\mathrm {Aut}}_{nt}(Y) \cong {\mathrm {Ker}}\left( G_{X-} \rightarrow \overline{G}_{X-}\right) = {\mathrm {Ker}}\, \psi \cap W(R)$ gives $e_{(\tau ,{\bar{\tau }})}=|{\mathrm {Aut}}_{nt}(Y)|$, where the isomorphism follows from Lemma 3.3. $\square$

Remark 3.12

The factor $d_{(\tau ,{\bar{\tau }})}$ is nontrivial only for Nos. 142 and 170 which are not realized geometrically. This is explained by an extra “automorphism” of Y which exchanges two $D_4$ configurations of “smooth rational curves” and acts trivially on their orthogonal complement in $S_{Y}$. This is not visible in the Weyl group. Thus in the geometric cases a nontrivial contribution of $c_{(\tau ,{\bar{\tau }})}=e_{(\tau ,{\bar{\tau }})}$ is indeed explained by the presence of a numerically trivial involution of Y.

4 Borcherds’ Method

4.1 An Algorithm on a Graph

The algorithms to prove our main results are variations of the following computational procedure.

Let (V, E) be a simple non-oriented connected graph, where V is the set of vertices and E is the set of edges, which is a set of non-ordered pairs of distinct elements of V. The set V may be infinite. Suppose that a group G acts on (V, E) from the right. We assume the following.

$(\hbox {VE-}1)$:

For any vertex $v\in V$, the set $\{\,{v^\prime \in V}\mid {\{v, v^\prime \}\in E} \,\}$ of vertices adjacent to v is finite and can be calculated effectively.

$(\hbox {VE-}2)$:

For any vertices $v, v^\prime \in V$, we can determine effectively whether the set

$$\begin{aligned} {T_{G}}(v, v^\prime ):=\{\,{g\in G}\mid {v^g=v^\prime } \,\} \end{aligned}$$

(4.1)

is empty or not, and when it is non-empty, we can calculate an element of ${T_{G}}(v, v^\prime )$.

$(\hbox {VE-}3)$:

For any $v\in V$, the stabilizer subgroup ${T_{G}}(v, v)$ of v in G is finitely generated, and a finite set of generators of ${T_{G}}(v, v)$ can be calculated effectively.

We define the G-equivalence relation $\sim$ on V by

$$\begin{aligned} v\sim v^\prime \;\;\Longleftrightarrow \;\; {T_{G}}(v, v^\prime )\ne \emptyset . \end{aligned}$$

Suppose that $V_0$ is a non-empty finite subset of V with the following properties.

($\hbox {V}_0\hbox {-}1$):

If $v, {v^\prime } \in V_0$ are distinct, then $v\not \sim v^\prime$.

($\hbox {V}_0\hbox {-}2$):

We put ${\widetilde{V}}_0:=\{\,{v\in V}\mid {v \text { is adjacent to a vertex belonging to } V_0} \,\}$. Then, for each ${v}\in {\widetilde{V}_0}$, there exists a vertex ${v^\prime }\in {V_0}$ such that ${v}\sim {v^\prime }$.

For each ${v}\in {\widetilde{V}}_0$, we choose an element $h(v)\in {T_{G}}(v, {v^\prime })$, where ${v^\prime }$ is the unique vertex in $V_0$ such that $v\sim {v^\prime }$, and put

$$\begin{aligned} {{\mathcal {H}}}:=\{\,{h(v)}\mid {v\in {\widetilde{V}}_0} \,\}. \end{aligned}$$

We fix an element $v_0\in V_0$.

Proposition 4.1

The natural mapping

(4.2)

is a bijection, and the group G is generated by the union of ${T_{G}}(v_0, v_0)$ and ${{\mathcal {H}}}$.

Proof

Let $\langle {{{\mathcal {H}}}} \rangle$ be the subgroup of G generated by ${{\mathcal {H}}}$. First we prove that, for any $v\in V$, there exists an element $h\in \langle {{{\mathcal {H}}}} \rangle$ such that $v^h\in V_0$. Let an element $v\in V$ be fixed. A sequence

$$\begin{aligned} v_{(0)}, v_{(1)}, \dots , v_{(l)} \end{aligned}$$

(4.3)

of vertices is said to be a path from $V_0$ to $v^{\langle {{{\mathcal {H}}}} \rangle }$ if $v_{(i-1)}$ and $v_{(i)}$ are adjacent for $i=1, \dots , l$, the starting vertex $v_{(0)}$ is in $V_0$, and the ending vertex $v_{(l)}$ belongs to the orbit $v^{\langle {{{\mathcal {H}}}} \rangle }$ of the fixed vertex v under the action of $\langle {{{\mathcal {H}}}} \rangle$. Since (V, E) is connected and $V_0$ is non-empty, there exists at least one path from $V_0$ to $v^{\langle {{{\mathcal {H}}}} \rangle }$. Suppose that the sequence (4.3) is a path from $V_0$ to $v^{\langle {{{\mathcal {H}}}} \rangle }$ of length $l>0$. Since $v_{(1)}$ is adjacent to the vertex $v_{(0)}$ in $V_0$, we have $v_{(1)}\in {\widetilde{V}}_0$ and there exists an element $h_1:=h(v_{(1)})\in {{\mathcal {H}}}$ that maps $v_{(1)}$ to an element of $V_0$. Then

$$\begin{aligned} v_{(1)} ^{h_1}, \dots , v_{(l)} ^{h_1} \end{aligned}$$

is a path from $V_0$ to $v^{\langle {{{\mathcal {H}}}} \rangle }$ of length $l-1$. Thus we obtain a path from $V_0$ to $v^{\langle {{{\mathcal {H}}}} \rangle }$ of length 0, which implies the claim.

The injectivity of (4.2) follows from property ($\hbox {V}_0$-1) of $V_0$. The surjectivity follows from the claim above. Suppose that $g\in G$. By the claim, there exists an element $h\in \langle {{{\mathcal {H}}}} \rangle$ such that $v_0^{gh}\in V_0$. By property ($\hbox {V}_0$-1) of $V_0$, we have $v_0=v_0^{gh}$ and hence $gh\in {T_{G}}(v_0,v_0)$. Therefore G is generated by the union of ${{\mathcal {H}}}$ and ${T_{G}}(v_0, v_0)$. $\square$

To obtain $V_0$ and ${{\mathcal {H}}}$, we employ Procedure 4.1. This procedure terminates if and only if $|V/G|<\infty$.

4.2 17 Primitive Embeddings

Recall that $L_{26}$ is an even unimodular hyperbolic lattice of rank 26. The $L_{26}$-chamber (that is, the standard fundamental domain of $W(L_{26})$) was studied by Conway [7]. He constructed a bijection between the set of walls of an $L_{26}$-chamber D and the set of vectors of the Leech lattice, and showed that the automorphism group ${\mathrm {O}}(L_{26}, D)$ of D is isomorphic to the group of affine isometries of the Leech lattice. Using this result, Borcherds [4, 5] developed a method to calculate the orthogonal group of an even hyperbolic lattice S by embedding S primitively into $L_{26}$ and investigating the tessellation of an S-chamber (that is, a standard fundamental domain of W(S)) by $L_{26}/S$-chambers.

In [6], we apply this method to $S=L_{10}(2)$. We fix positive half-cones ${{\mathcal {P}}}_{10}$ of $L_{10}$ and ${{\mathcal {P}}}_{26}$ of $L_{26}$. In [6], we have proved the following.

Theorem 4.2

[6] Up to the action of ${\mathrm {O}}(L_{10})$ and ${\mathrm {O}}(L_{26})$, there exist exactly 17 primitive embeddings of $L_{10}(2)$ into $L_{26}$. $\square$

These 17 primitive embeddings of $L_{10}(2)$ into $L_{26}$ are named as

$$\begin{aligned} \mathtt{12A},\; \mathtt{12B},\; \mathtt{20A},\; \dots \;, \mathtt{20F},\; \mathtt{40A},\; \dots \;, \mathtt{40E},\; \mathtt{96A},\; \mathtt{96B},\; \mathtt{96C},\; \mathtt{infty}. \end{aligned}$$

Recall the notion of being reflexively simple from Definition 2.9.

Theorem 4.3

[6] Suppose that a primitive embedding $L_{10}(2)\hookrightarrow L_{26}$ is not of type infty, Then each $L_{26}/L_{10}(2)$-chamber has only finitely many walls, and they are defined by roots of $L_{10}$. Moreover the tessellation of ${{\mathcal {P}}}_{10}$ by $L_{26}/L_{10}(2)$-chambers is reflexively simple. $\square$

The explicit description of the 17 primitive embeddings and $L_{26}/L_{10}(2)$-chambers is given in [6, 30]. From these data, we see the following. Let $L_{10}(2) \hookrightarrow L_{26}$ be a primitive embedding whose type is not infty, and D an $L_{26}/L_{10}(2)$-chamber. The automorphism group of D is denoted by

$$\begin{aligned} {\mathrm {O}}(L_{10}, D):=\{\,{g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(L_{10})}\mid {D^g=D} \,\}. \end{aligned}$$

Since the walls of D are defined by roots of $L_{10}$, the chamber D is tessellated by Vinberg chambers. The volume of D is defined by

$$\begin{aligned} \mathrm{{vol}}(D):=\text {the number of Vinberg chambers contained in } D. \end{aligned}$$

Let f be a face of D with codimension k. Then the defining roots of the walls of D containing f form a configuration whose dual graph is a Dynkin diagram of an $\mathrm{ADE}$-type. The $\mathrm{ADE}$-type of f is the $\mathrm{ADE}$-type of this Dynkin diagram. The closure $\overline{D}$ of D in $S_{X}\otimes {{\mathbb {R}}}$ contains only a finite number of isotropic rays. Let $v\in S_{X}\cap \overline{D}$ be a primitive isotropic ray (see Sect. 2.3). Then the defining roots r of walls of D such that $\langle r, v \rangle =0$ form a configuration whose dual graph is a Dynkin diagram of an affine $\mathrm{ADE}$-type. The affine $\mathrm{ADE}$-type of the isotropic ray ${{\mathbb {R}}}_{>0}v$ is the affine $\mathrm{ADE}$-type of this Dynkin diagram.

Example 4.4

Let $L_{10}(2) \hookrightarrow L_{26}$ be the primitive embedding of type $\mathtt{96C}$, and $D_0$ an $L_{26}/L_{10}(2)$-chamber. Then $D_0$ has exactly 96 walls. The group ${\mathrm {O}}(L_{10}, D_0)$ is of order $110592=2^{12}\cdot 3^3$, and this group acts on the set of walls of $D_0$ transitively. We have

$$\begin{aligned} \mathrm{{vol}}(D_0)=\frac{1_{\mathrm{{BP}}}}{72}=652758220800. \end{aligned}$$

The $L_{26}/L_{10}(2)$-chamber $D_0$ has $1728+768+144$ faces of codimension 2, which are decomposed into orbits of size 1728, 768, 144 under the action of ${\mathrm {O}}(L_{10}, D_0)$. Hence each wall of $D_0$ is bounded by $36+16+3=55$ faces of codimension 2 of $D_0$. The $\mathrm{ADE}$-types of faces in these orbits are $2A_1$, $2A_1$, $A_2$, respectively. The $L_{26}/L_{10}(2)$-chamber $D_0$ has $18+256+256+864$ isotropic rays, which are decomposed into orbits of size 18, 256, 256, 864 by the action of ${\mathrm {O}}(L_{10}, D_0)$. The affine $\mathrm{ADE}$-types of isotropic rays of these orbits are $8{A}_1$, $4{A}_2$, $4{A}_2$, $2{A}_1+2{A}_3$, respectively.

4.3 Constructing $S_{X}$

Let Y be an Enriques surface with the universal covering $\pi {:}\,X\rightarrow Y$. We consider the following assumption:

$$\begin{aligned}&\hbox {we have a primitive embedding } S_{X}\hookrightarrow L_{26} \hbox { such that the composite} \nonumber \\&S_{Y}(2)\cong L_{10}(2) \hookrightarrow L_{26} \hbox { of } \pi ^*{:}\,S_{Y}(2)\hookrightarrow S_{X}\hbox { and } S_{X}\hookrightarrow L_{26} \hbox { is not of} \nonumber \\&\hbox { type } \mathtt{infty}, \hbox { and we have the list of walls of an } L_{26}/S_Y(2)\hbox {-chamber } \nonumber \\&D_0 \hbox { that is contained in } \mathrm{Nef}_{Y}. \end{aligned}$$

(4.4)

Suppose that (4.4) holds. Then ${{\mathcal {P}}}_Y$ has the following three tessellations, each of which is a refinement of the one below.

• by Vinberg chambers,

• by $L_{26}/S_Y(2)$-chambers, each of which has only finite number of walls, and

• by $S_{X}/S_{Y}(2)$-chambers, one of which is $\mathrm{Nef}_{Y}$.

The tessellation of $\mathrm{Nef}_{Y}$ by $L_{26}/S_Y(2)$-chambers is very useful in analyzing $\mathrm{Nef}_{Y}$. Recall that $G_Y\subset {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ is the image of the projection of $G_X\subset {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$ defined by (3.2).

Proposition 4.5

Suppose that Y satisfies (3.1) and (4.4). Then the action of $G_Y$ on ${{\mathcal {P}}}_Y$ preserves the tessellation of ${{\mathcal {P}}}_Y$ by $L_{26}/S_Y(2)$-chambers. In particular, the action of $\mathrm{aut}(Y)$ on $\mathrm{Nef}_{Y}$ preserves the tessellation of $\mathrm{Nef}_{Y}$ by $L_{26}/S_Y(2)$-chambers.

Proof

It is enough to prove that the action of ${\tilde{g}}\in G_X$ on ${{\mathcal {P}}}_X$ preserves the tessellation of ${{\mathcal {P}}}_X$ by $L_{26}/S_{X}$-chambers. Let ${\mathrm {id}}_P$ be the identity of the orthogonal complement P of $S_{X}$ in $L_{26}$. Since the action of ${\tilde{g}}$ on $S_{X}^{\vee }/S_{X}$ is 1, the action of $({\tilde{g}}, {\mathrm {id}}_P)$ on $S_{X}\oplus P$ preserves the even unimodular overlattice $L_{26}$ of $S_{X}\oplus P$. Thus ${\tilde{g}}$ extends to an isometry of $L_{26}$, and hence its action on ${{\mathcal {P}}}_X$ preserves the $L_{26}/S_{X}$-chambers. The second assertion follows from the fact that $\mathrm{aut}(Y)$ is the stabilizer subgroup of $\mathrm{Nef}_{Y}$ in $G_Y$. $\square$

The purpose of this section is to construct a primitive embedding $S_{X}\hookrightarrow L_{26}$ for a $(\tau , {\bar{\tau }})$-generic Enriques surface Y, so that we can assume (4.4). We start from a primitive embedding $\iota {:}\,L_{10}(2)\hookrightarrow L_{26}$ whose type is not infty and which has a fixed $L_{26}/L_{10}(2)$-chamber $D_0$, and then proceed to the construction of $S_{X}$ between $L_{10}(2)\cong S_{Y}(2)$ and $L_{26}$ such that the inclusion of $L_{10}(2)\cong S_{Y}(2)$ into $S_{X}$ is the embedding $\pi ^*$, and that the fixed $L_{26}/L_{10}(2)$-chamber $D_0$ is contained in $\mathrm{Nef}_{Y}$.

Recall that, for a $(\tau , {\bar{\tau }})$-generic Enriques surface Y, the lattice $S_{X}$ is obtained from $S_{Y}(2)$ by adding roots of the form $(r+v)/2$, where r is a root of $S_{Y}$ and v is a $(-4)$-vector in $S_{X-}$. To find roots in $L_{26}$ that yield an appropriate extension from $S_{Y}(2)$ to $S_{X}$, we search for pairs $\alpha =(r, v)$ of a root r of $L_{10}$ defining a wall of $D_0$ and a $(-4)$-vector v of $Q_{\iota }$ such that $(r+v)/2$ is in $L_{26}$, where $Q_{\iota }$ is the orthogonal complement of $L_{10}(2)$ in $L_{26}$. For a finite set $p=\{\alpha _1, \dots , \alpha _m\}$ of such pairs, we consider the sublattice $M_p$ of $L_{26}$ generated by $L_{10}(2)$ and the roots $(r_1+v_1)/2, \dots , (r_m+v_m)/2$ of $L_{26}$, where $\alpha _i=(r_i, v_i)$. Suppose that $p=\{\alpha _1, \dots , \alpha _m\}$ satisfies the following:

• (i) The dual graph of $r_1, \dots , r_m$ is a Dynkin diagram of some $\mathrm{ADE}$-type $\tau$. By Proposition 1.2, the primitive closure $\overline{R}$ of the $\mathrm{ADE}$-sublattice R of $L_{10}$ generated by $r_1, \dots , r_m$ is also an $\mathrm{ADE}$-sublattice of $L_{10}$. Let ${\bar{\tau }}$ denote the $\mathrm{ADE}$-type of $\overline{R}$.

By Proposition 1.2, the embedding $L_{10}(2) \hookrightarrow M_p$ is isomorphic to $L_{10}(2) \hookrightarrow M_R$, and hence, by Proposition 1.3, we see that $L_{10}(2)$ is a primitive sublattice of $M_p$, and the orthogonal complement of $L_{10}(2)$ in $M_p$ contains no roots. We consider the following condition:

• (ii) $M_p$ can be embedded primitively into the K3 lattice (an even unimodular lattice of rank 22 with signature (3, 19)). This condition is checked by calculating the discriminant form of $M_p$ and applying the theory of genera (see [22]).

Suppose that $M_p$ satisfies condition (ii). Since $22-{\mathrm {rank}}\, M_{p}=12-m>2$, the surjectivity of the period mapping of complex K3 surfaces [3, Chapter VIII] implies that there exists a K3 surface X with $M_p \cong S_{X}$ such that ${\mathrm {O}}(T_X, \omega )=\{\pm 1\}$. Moreover, by [13], the K3 surface X has a fixed point free involution $\varepsilon$ with the quotient morphism $\pi {:}\,X\rightarrow Y=X/\langle \varepsilon \rangle$ to the Enriques surface Y such that, under suitable choices of isometries $M_p \cong S_{X}$, the embedding $L_{10}(2)\hookrightarrow M_p$ is identified with $\pi ^*{:}\,S_{Y}(2)\hookrightarrow S_{X}$. By the construction of $M_p$, this Enriques surface Y is $(\tau , {\bar{\tau }})$-generic. Thanks to Proposition 3.1, we can further assume that $D_0$ is contained in $\mathrm{Nef}_{Y}$ by changing the isometry $M_p \cong S_{X}$.

Except for the type $(\tau , {\bar{\tau }})$ of Nos 88 and 146, we can find a set $p=\{\alpha _1, \dots , \alpha _m\}$ satisfying condition (i) above using the primitive embedding $\iota {:}\,L_{10}(2)\hookrightarrow L_{26}$ given in the 8th column (irec) of Table 1. If the 5th column (exist) is not marked by $\times$, then $M_p$ satisfies condition (ii).

Example 4.6

Let $\iota {:}\,L_{10}(2)\hookrightarrow L_{26}$ be the primitive embedding of type 96C (see Example 4.4). Then the even negative definite lattice $Q_{\iota }$ contains 2208 vectors v of square-norm $-4$, and we have 192 pairs $\alpha =(r, v)$ such that $(r+v)/2\in L_{26}$. Choosing appropriate subsets from these 192 pairs, we can construct $S_{X}$ for many types $(\tau , {\bar{\tau }})$ (Nos. 1, 2, ...).

Remark 4.7

Even when $M_p$ does not satisfy condition (ii), we can use $M_p$ as the Néron–Severi lattice $S_{X}$ of a “non-existing K3 surface” X and run the geometric algorithms below.

5 Geometric Algorithms

We prepare some algorithms that will be used in the application of the generalized Borcherds’ method to geometric situations.

Let Y be an Enriques surface with the universal covering $\pi {:}\,X\rightarrow Y$. We assume (3.1) and (4.4). First we prepare the following computational data:

• (i) an integral interior point $a_{Y0}\in S_{Y}$ of $D_0$, which is an ample class of Y,

• (ii) the list of roots defining the walls of $D_0$,

• (iii) the finite group ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y}, D_0)=\{\,{g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})}\mid {D_0^g=D_0} \,\}$,

• (iv) the finite group ${\mathrm {O}}(S_{X-})$, and

• (v) the list of $(-4)$-vectors of $S_{X-}$.

5.1 Separating Roots

Definition 5.1

Let L be an even hyperbolic lattice with a positive half-cone ${{\mathcal {P}}}$, and let $a_1, a_2$ be elements of ${{\mathcal {P}}}\cap L$. We say that a hyperplane $(v)^{\perp }$ of ${{\mathcal {P}}}$ separates $a_1$ and $a_2$ if $\langle v, a_1 \rangle$ and $\langle v, a_2 \rangle$ are nonzero and have different signs. We say that a vector $v\in L\otimes {{\mathbb {Q}}}$ with $\langle v, v \rangle <0$ separates $a_1$ and $a_2$ if $(v)^{\perp }$ separates $a_1$ and $a_2$.

By an algorithm given in [27], we can calculate, for any $a_1, a_2\in {{\mathcal {P}}}\cap L$, the set of roots of L that separate $a_1$ and $a_2$.

5.2 Splitting Roots

Definition 5.2

We say that a root r of $S_{Y}$ splits in $S_{X}$ if there exists a root ${\tilde{r}}$ of $S_{X}$ such that $\pi ^*(r)={\tilde{r}}+{\tilde{r}}^\varepsilon$.

A root r of $S_{Y}$ splits in $S_{X}$ if and only if there exists a $(-4)$-vector v of $S_{X-}$ such that $(\pi ^*(r)+v)/2\in S_{X}$. Hence we can effectively determine whether a given root r of $S_{Y}$ splits in $S_{X}$ or not. Moreover, when r splits, we can calculate the roots ${\tilde{r}}=(\pi ^*(r)+v)/2$ and ${\tilde{r}}^\varepsilon =(\pi ^*(r)-v)/2$ of $S_{X}$ such that $\pi ^*(r)={\tilde{r}}+{\tilde{r}}^\varepsilon$.

Suppose that a root r of $S_{Y}$ satisfies that $\mathrm{Nef}_{Y}\cap (r)^{\perp }$ contains a non-empty open subset of $(r)^{\perp }$ and that $\langle r, a_Y \rangle >0$ for an ample class $a_Y$ of Y. Then the following are equivalent:

• $\mathrm{Nef}_{Y}\cap (r)^{\perp }$ is a wall of $\mathrm{Nef}_{Y}$ (that is, the hyperplane $(r)^{\perp }$ is disjoint from the interior of $\mathrm{Nef}_{Y}$),

• r splits in $S_{X}$, and

• r is the class of a smooth rational curve C on Y.

In this case, the roots ${\tilde{r}}$ and ${\tilde{r}}^\varepsilon$ of $S_{X}$ are the classes of the smooth rational curves ${\widetilde{C}}$ and ${\widetilde{C}}^\varepsilon$ on X such that $\pi ^{-1}(C)={\widetilde{C}}+{\widetilde{C}}^\varepsilon$.

5.3 Membership Criterion of $G_Y$ in ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$

An element g of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ belongs to $G_Y$ if and only if there exists an isometry $h\in {\mathrm {O}}(S_{X-})$ such that the action of (g, h) on $S_{X+}\oplus S_{X-}$ preserves the overlattice $S_{X}$ and that ${\tilde{g}}:=(g, h)|S_{X}$ acts on $S_{X}^{\vee }/S_{X}$ trivially. Since we have the list of elements of the finite group ${\mathrm {O}}(S_{X-})$, we can determine whether an element $g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ belongs to $G_Y$ or not, and if $g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$, we can calculate a lift ${\tilde{g}}\in G_X$ of g.

5.4 Membership Criterion of $\mathrm{aut}(Y)$ in $G_Y$

Suppose that $g\in G_Y$, and let ${\tilde{g}}\in G_X$ be a lift of g. Recall from Proposition 3.2 that g belongs to $\mathrm{aut}(Y)$ if and only if g preserves $\mathrm{Nef}_{Y}$, or equivalently ${\tilde{g}}$ preserves $\mathrm{Nef}_{X}$. Hence $g\in \mathrm{aut}(Y)$ holds if and only if one of the following conditions that are mutually equivalent is satisfied:

• For any ample classes $a_X$ and $a_X^\prime$ of X, there exist no root of $S_{X}$ separating $a_X^{{\tilde{g}}}$ and $a_X^\prime$.

• For any ample classes $a_Y$ and $a_Y^\prime$ of Y, any roots of $S_{Y}$ separating $a_Y^{g}$ and $a_Y^\prime$ does not split in $S_{X}$.

• There exist ample classes $a_X$ and $a_X^\prime$ of X such that there exist no roots of $S_{X}$ separating $a_X^{{\tilde{g}}}$ and $a_X^\prime$.

• There exist ample classes $a_Y$ and $a_Y^\prime$ of Y such that any root of $S_{Y}$ separating $a_Y^{g}$ and $a_Y^\prime$ does not split in $S_{X}$.

Thus we can determine effectively whether a given isometry $g\in G_Y$ belongs to $\mathrm{aut}(Y)$ or not, because we have at least one ample class $a_{Y0}$ of Y.

5.5 Criterion for $\mathrm{aut}(Y)$-Equivalence

Recall from Theorem 4.3 that, for every $L_{26}/S_Y(2)$-chamber D, we have an isometry $g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ such that $D=D_0^g$. Let $D_1$ and $D_2$ be $L_{26}/S_Y(2)$-chambers. Suppose that we have isometries $g_1, g_2\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ such that $D_1=D_0^{g_1}$ and $D_2=D_0^{g_2}$. Then the set

$$\begin{aligned} \mathrm{isoms}(D_1, D_2):=\{\,{g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})}\mid {D_1^g=D_2} \,\}=g_1^{-1}\cdot {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y}, D_0) \cdot g_2 \end{aligned}$$

is finite, and can be explicitly calculated. Therefore we can calculate the set

$$\begin{aligned} \mathrm{isoms}(Y, D_1, D_2):=\mathrm{aut}(Y)\cap \mathrm{isoms}(D_1, D_2) \end{aligned}$$

explicitly, and in particular, we can calculate the group $\mathrm{aut}(Y, D):=\mathrm{isoms}(Y, D, D)$ for an $L_{26}/S_Y(2)$-chamber D.

6 Proofs of Main Theorems

We present algorithms that prove Theorems 1.19 and 1.21. Let Y be an Enriques surface with the universal covering $\pi {:}\,X\rightarrow Y$. Suppose that Y is $(\tau , {\bar{\tau }})$-generic, where $(\tau , {\bar{\tau }})$ is not equal to No. 88 nor No. 146 in Table 1, so that we can assume (3.1) and (4.4).

6.1 Generators of $\mathrm{aut}(Y)$ and Representatives of $\mathrm{Nef}_{Y}/\mathrm{aut}(Y)$

We calculate a finite generating set of $\mathrm{aut}(Y)$ and a complete set of representatives of $\mathrm{Nef}_{Y}/\mathrm{aut}(Y)$. This calculation affirms Theorem 1.18 computationally. Moreover the results will be used in the proofs of Theorems 1.19 and 1.21 below.

Let (V, E) be the graph where V is the set of $L_{26}/S_Y(2)$-chambers contained in $\mathrm{Nef}_{Y}$ and E is defined by the adjacency relation of $L_{26}/S_Y(2)$-chambers. Let G be the group $\mathrm{aut}(Y)$, and let $v_0\in V$ be the $L_{26}/S_Y(2)$-chamber $D_0$ in $\mathrm{Nef}_{Y}$. Let $D=D_0^g$ be an $L_{26}/S_Y(2)$-chamber contained in $\mathrm{Nef}_{Y}$, where $g\in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$. Then we can calculate the set of roots defining the walls of D by mapping the set of roots defining the walls of $D_0$ by the isometry g. For each root r defining a wall of D, the chamber $D^{s_r}=D_0^{gs_r}$ adjacent to D across the wall $D\cap (r)^{\perp }$ of D is contained in $\mathrm{Nef}_{Y}$ if and only if r does not split in $S_{X}$. Therefore we can determine $D^{s_r}\subset \mathrm{Nef}_{Y}$ or not by the method in Sect. 5.2. Therefore condition ($\hbox {VE-1}$) in Sect. 4.1 is satisfied. Since we can calculate $\mathrm{isom}(Y, D_0^g, D_0^{g^\prime })$ for any $g, g^\prime \in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ by Sect. 5.5, conditions ($\hbox {VE-}2$) and ($\hbox {VE-}3$) are also satisfied. Therefore we can apply Procedure 4.1 to the graph (V, E) and the group G, and obtain a complete set $V_0$ of representatives of orbits of the action of G on V, the stabilizer subgroups $\mathrm{isom}(Y, D, D)=\mathrm{aut}(Y, D)$ of these representatives $D\in V_0$, and a generating set

$$\begin{aligned} {{\mathcal {G}}}:={{\mathcal {H}}}\cup \mathrm{aut}(Y, D_0) \end{aligned}$$

of $\mathrm{aut}(Y)$. Then we have

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))=\mathrm{{vol}}(D_0) \sum _{D\in V_0} \frac{1}{| \mathrm{aut}(Y, D)|}. \end{aligned}$$

(6.1)

Thus Theorem 1.18 is computationally affirmed.

Remark 6.1

The amount of computation of Procedure 4.1 grows quadratically as |V/G| becomes large, because we have to check $T_G(v, {v^\prime })=\emptyset$ for all pairs of distinct $v, {v^\prime } \in V_0$. We could calculate a finite generating set of $\mathrm{aut}(Y)$ by using, naively, the graph $({V^\prime }, {E^\prime })$, where ${V^\prime }$ is the set of Vinberg chambers contained in $\mathrm{Nef}_{Y}$ and $E^\prime$ is the adjacency relation of Vinberg chambers. However, the size of ${V^\prime }/\mathrm{aut}(Y)$ is approximately $\mathrm{{vol}}(D_0)$ times the size of $V/\mathrm{aut}(Y)$. Thus, very roughly speaking, using the primitive embedding $S_{Y}(2)\hookrightarrow L_{26}$ of type 96C gives us computational advantage of multiplicative factor the square of $\mathrm{{vol}}(D_0)=652758220800$.

6.2 Calculating ${{\mathcal {R}}}_{\mathrm{temp}}$, ${{\mathcal {E}}}_{\mathrm{temp}}$ and ${{\mathcal {G}}}_X$

From $V_0$ and ${{\mathcal {G}}}$ calculated above, we compute the following data, which will be used in Sects. 6.3 and 6.4.

Recall that ${{\mathcal {R}}}(Y)$ is embedded in $S_{Y}$ by $C\mapsto [C]$. For each $D\in V_0$, let ${{\mathcal {R}}}(Y, D)$ be the set of roots $r=[C]$ in ${{\mathcal {R}}}(Y)$ such that $D\cap (r)^{\perp }$ is a wall of D. Since $D\subset \mathrm{Nef}_{Y}$, a root r defining a wall of D belongs to ${{\mathcal {R}}}(Y)$ if and only if r splits in $S_{X}$. Therefore we can calculate ${{\mathcal {R}}}(Y, D)$ by the method in Sect. 5.2. We put

$$\begin{aligned} {{\mathcal {R}}}_{\mathrm{temp}}:=\bigcup _{D\in V_0} {{\mathcal {R}}}(Y, D). \end{aligned}$$

Then the mapping

$$\begin{aligned} {{\mathcal {R}}}_{\mathrm{temp}}\hookrightarrow {{\mathcal {R}}}(Y)\mathbin {\rightarrow \rightarrow }{{\mathcal {R}}}(Y)/\mathrm{aut}(Y) \end{aligned}$$

is surjective. Via the generating set ${{\mathcal {G}}}$, we can generate (pseudo-)random elements of $\mathrm{aut}(Y)=\langle {{{\mathcal {G}}}} \rangle$. For $[C], [C^\prime ]\in {{\mathcal {R}}}_{\mathrm{temp}}$, if we find $g\in \mathrm{aut}(Y)$ such that $[C]^g=[C^\prime ]$, then we remove $[C^\prime ]$ from ${{\mathcal {R}}}_{\mathrm{temp}}$. Repeating this process many times, we obtain a smaller subset ${{\mathcal {R}}}_{\mathrm{temp}}^\prime$ of ${{\mathcal {R}}}(Y)$ that is mapped to ${{\mathcal {R}}}(Y)/\mathrm{aut}(Y)$ surjectively.

Let $\phi {:}\,Y\rightarrow {{\mathbb {P}}}^1$ be an elliptic fibration of Y, and F a general fiber of $\phi$. Then $f_{\phi }:=[F]/2 \in S_{Y}$ is a primitive isotropic ray (see Sect. 2.3 for the definition) contained in the closure of $\mathrm{Nef}_{Y}$ in $\overline{{{\mathcal {P}}}}_Y$. For each $D\in V_0$, let ${{\mathcal {E}}}(Y, D)$ be the set of primitive isotropic rays contained in the closure $\overline{D}$ of D in $\overline{{{\mathcal {P}}}}_Y$. We put

$$\begin{aligned} {{\mathcal {E}}}_{\mathrm{temp}}:=\bigcup _{D\in V_0} {{\mathcal {E}}}(Y, D). \end{aligned}$$

Then the mapping

$$\begin{aligned} {{\mathcal {E}}}_{\mathrm{temp}}\hookrightarrow {{\mathcal {E}}}(Y)\mathbin {\rightarrow \rightarrow }{{\mathcal {E}}}(Y)/\mathrm{aut}(Y) \end{aligned}$$

is surjective. As above, from ${{\mathcal {E}}}_{\mathrm{temp}}$ and using ${{\mathcal {G}}}$, we obtain a smaller subset ${{\mathcal {E}}}_{\mathrm{temp}}^\prime$ of ${{\mathcal {E}}}(Y)$ that is mapped to ${{\mathcal {E}}}(Y)/\mathrm{aut}(Y)$ surjectively.

Let ${\mathrm {Aut}}(X, \varepsilon )$ be the centralizer of $\varepsilon \in {\mathrm {Aut}}(X)$ in ${\mathrm {Aut}}(X)$, and let $\mathrm{aut}(X, \varepsilon )$ be the image of ${\mathrm {Aut}}(X, \varepsilon )$ in $\mathrm{aut}(X)$. We write an element ${\tilde{\gamma }}\in {\mathrm {Aut}}(X)$ as $({\tilde{g}}, f)$ by (3.3). Since ${\mathrm {O}}(T_X, \omega )=\{\pm 1\}$ is abelian, we see that ${\tilde{\gamma }}$ commutes with $\varepsilon \in {\mathrm {Aut}}(X)$ if and only if ${\tilde{g}}$ commutes with $\varepsilon \in \mathrm{aut}(X)$. Hence $\mathrm{aut}(X, \varepsilon )$ is equal to the centralizer of $\varepsilon \in \mathrm{aut}(X)$ in $\mathrm{aut}(X)$. By the Torelli theorem (see the proof of Proposition 3.2), an element ${\tilde{g}}$ of ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$ belongs to $\mathrm{aut}(X, \varepsilon )$ if and only if ${\tilde{g}}$ acts on $S_{X}^{\vee }/S_{X}$ as $\pm 1$, preserves $\mathrm{Nef}_{X}$, and commutes with $\varepsilon \in {{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{X})$. Let $\mathrm{aut}(X,\varepsilon )_0$ be the group consisting of elements ${\tilde{g}} \in \mathrm{aut}(X, \varepsilon )$ that act on $S_{X}^{\vee }/S_{X}$ as 1. We have $\mathrm{aut}(X,\varepsilon )_0=\mathrm{aut}(X,\varepsilon )\cap G_X$.

The restriction homomorphism ${\tilde{g}}\mapsto {\tilde{g}}|S_{Y}$ gives a surjective homomorphism $\mathrm{aut}(X, \varepsilon ) \rightarrow \mathrm{aut}(Y)$. We calculate the kernel

$$\begin{aligned} K:={\mathrm {Ker}}(\mathrm{aut}(X, \varepsilon ) \rightarrow \mathrm{aut}(Y)). \end{aligned}$$

The kernel K is naturally embedded into ${\mathrm {O}}(S_{X-})$ by ${\tilde{g}}\mapsto {\tilde{g}}|S_{X-}$. We put

$$\begin{aligned} K_0:={\mathrm {Ker}}(\mathrm{aut}(X,\varepsilon )_0 \rightarrow \mathrm{aut}(Y))\subset G_X. \end{aligned}$$

By definition $K_0$ acts trivially on $S_{X+}^{\vee }/ S_{X+}$ and by Proposition 2.1 it must act trivially on $S_{X-}^{\vee }/S_{X-}$ as well. Hence, regarded as a subgroup of $G_{X-}\subset {\mathrm {O}}(S_{X-})$, $K_0$ is contained in the kernel of

$$\begin{aligned} \psi {:}\,G_{X-}\rightarrow {\mathrm {O}}(S_{X-}^{\vee }/S_{X-}). \end{aligned}$$

Conversely the elements of ${\mathrm {Ker}}\, \psi$ can be extended by the identity on $S_{X+}$ to elements of $G_X$ which trivially preserve $\mathrm{Nef}_Y$. Hence they are induced by automorphisms of Y and we have $K_0 = {\mathrm {Ker}}\, \psi$. The kernel of $\psi$ is explicitly computed in the proof of Theorem 3.11. Its order is given by $e_{\tau ,{\bar{\tau }}} \in \{1,2\}$. Suppose that $e_{\tau ,{\bar{\tau }}} =2$. If $\varepsilon \in K_0$, then $K=K_0=\langle \varepsilon \rangle$. This is the case if in addition $\tau ({\widetilde{R}})=E_8$. Otherwise $K=K_0 \times \langle \varepsilon \rangle$ is of order 4.

For each g in the generating set ${{\mathcal {G}}}$ of $\mathrm{aut}(Y)$, we calculate a lift ${\tilde{g}}\in \mathrm{aut}(X, \varepsilon )$ of g, and put

$$\begin{aligned} {{\mathcal {G}}}_X:=\{\,{{\tilde{g}}}\mid {g\in {{\mathcal {G}}}} \,\} \; \cup \;K. \end{aligned}$$

Then $\mathrm{aut}(X, \varepsilon )$ is generated by ${{\mathcal {G}}}_X$.

6.3 Rational Curves on Y

We prove Theorem 1.19. By the construction of $S_{X}$ given in Sect. 4.3, we have a set of splitting roots that define some walls of $D_0\subset \mathrm{Nef}_{Y}$ and form the dual graph of $\mathrm{ADE}$-type $\tau$. Therefore the existence of $C_1, \dots , C_m$ in assertion (1) is proved.

Let C be a smooth rational curve on Y, and $r:=[C]$ the class of C. Let ${\widetilde{V}}_C$ be the set of $L_{26}/S_Y(2)$-chambers D such that $D\cap (r)^{\perp }$ is a wall of D and that D is located on the same side of $(r)^{\perp }$ as $\mathrm{Nef}_{Y}$. Let D be an element of ${\widetilde{V}}_C$, and suppose that $F:=D\cap (r)^{\perp }\cap (r^\prime )^{\perp }$ is a face of codimension 2 of D that is a boundary of the wall $D\cap (r)^{\perp }$, where $r^\prime$ is a root of $S_{Y}$ defining a wall of D. Then there exists a unique element $D^\prime$ of ${\widetilde{V}}_C$ such that $D\cap D^\prime =F$ holds. We say that this chamber $D^\prime$ is adjacent in ${\widetilde{V}}_C$ to D across F. This $L_{26}/S_Y(2)$-chamber $D^\prime$ is calculated as follows. As is seen from the set of faces of $L_{26}/S_Y(2)$-chambers (see [30]), we have $\langle r, r^\prime \rangle =0$ or $\langle r, r^\prime \rangle =1$. Let s and $s^\prime$ be the reflections with respect to the roots $r=[C]$ and $r^\prime$, respectively. Then

$$\begin{aligned} D^\prime = {\left\{ \begin{array}{ll} D^{s^\prime } &{}\quad \text {if } \langle r, r^\prime \rangle =0, \\ D^{ss^\prime } &{}\quad \text {if } \langle r, r^\prime \rangle =1. \end{array}\right. } \end{aligned}$$

Suppose that D is contained in $\mathrm{Nef}_{Y}$. Then $D^\prime$ is contained in $\mathrm{Nef}_{Y}$ if and only if $r^\prime$ is not the class of a smooth rational curve on Y, or equivalently, $r^\prime$ does not split in $S_{X}$. We consider the graph $(V_C, E_C)$, where $V_C$ is the set of $L_{26}/S_Y(2)$-chambers $D\in {\widetilde{V}}_C$ contained in $\mathrm{Nef}_{Y}$, and $E_C$ is the restriction to $V_C\subset {\widetilde{V}}_C$ of the adjacency relation on ${\widetilde{V}}_C$ defined above. Then the stabilizer subgroup

$$\begin{aligned} G_C:=\mathrm{aut}(Y, C)=\{\,{g\in \mathrm{aut}(Y)}\mid {r^g=r} \,\} \end{aligned}$$

of C in $\mathrm{aut}(Y)$ acts on $(V_C, E_C)$. For $D, D^\prime \in V_C$, we have

$$\begin{aligned} {T_{G}}(D, D^\prime )=\{\,{g\in \mathrm{isoms}(Y, D, D^\prime )}\mid { r^g=r} \,\}, \end{aligned}$$

where ${T_{G}}(D, D^\prime )\subset G_C$ is defined by (4.1), and $\mathrm{isoms}(Y, D, D^\prime )$ is defined in Sect. 5.5. Therefore $(V_C, E_C)$ and $G_C$ satisfy conditions ($\hbox {VE-}1$), ..., ($\hbox {VE-}3$) in Sect. 4.1. We apply Procedure 4.1 to every $C\in {{\mathcal {R}}}_{\mathrm{temp}}^\prime$ and obtain a complete set $V_{C,0}$ of representatives of orbits of the action of $G_C$ on $V_C$.

Two elements C and $C^\prime$ of ${{\mathcal {R}}}_{\mathrm{temp}}^\prime$ are contained in the same orbit under the action of $\mathrm{aut}(Y)$ on ${{\mathcal {R}}}(Y)$ if and only if we have one of the following conditions that are mutually equivalent.

• Let D be an arbitrary element of $V_{C, 0}$. Then there exists an $L_{26}/S_Y(2)$-chamber $D^\prime$ in $V_{C^\prime , 0}$ such that $\mathrm{isoms}(Y, D, D^\prime )$ contains an isometry g such that $[C]^g=[C^\prime ]$.

• There exist a pair of $L_{26}/S_Y(2)$-chambers $D\in V_{C, 0}$ and $D^\prime \in V_{C^\prime , 0}$ and an isometry $g\in \mathrm{isoms}(Y, D, D^\prime )$ such that $[C]^g=[C^\prime ]$.

Applying this method to all pairs $C, C^\prime$ of distinct elements of ${{\mathcal {R}}}_{\mathrm{temp}}^\prime$, we obtain a complete set of representatives $C_1^\prime , \dots , C_k^\prime$ of orbits of the action of $\mathrm{aut}(Y)$ on ${{\mathcal {R}}}(Y)$. We then apply this method to the representatives $C_1^\prime , \dots , C_k^\prime$ and the smooth rational curves $C_1, \dots , C_m$ in assertion (1), and complete the proof of Theorem 1.19.

The algorithm given above is a priori guaranteed to work. A posteriori, Theorem 1.19 can be verified by the following simple strategy. Let $\mathrm{aut}(X, \varepsilon )|S_{X-}$ be the image of the homomorphism

$$\begin{aligned} \mathrm{aut}(X, \varepsilon )\rightarrow {\mathrm {O}}(S_{X-}) \end{aligned}$$

given by ${\tilde{g}}\mapsto {\tilde{g}}|S_{X-}$. Since we have calculated a finite generating set ${{\mathcal {G}}}_X$ of $\mathrm{aut}(X, \varepsilon )$, we can calculate the elements of the finite group $\mathrm{aut}(X, \varepsilon )|S_{X-}$. Let $C, C^\prime$ be elements of ${{\mathcal {R}}}(Y)$. If the orbit of $\{\pm v_C\}\subset S_{X-}$ by $\mathrm{aut}(X, \varepsilon )|S_{X-}$ and that of $\{\pm v_{C^\prime }\}$ are disjoint, then the orbits of C and $C^\prime$ by $\mathrm{aut}(Y)$ are disjoint. Even though the converse does not necessarily hold, we know a posteriori that once the size of ${{\mathcal {R}}}_{\mathrm{temp}}^\prime$ is small enough, this separates the orbits of ${{\mathcal {R}}}_{\mathrm{temp}}^\prime$.

6.4 Elliptic Fibrations of Y

Let $\phi {:}\,Y\rightarrow {{\mathbb {P}}}^1$ be an elliptic fibration of Y. We consider the following graph $(V_{\phi }, E_{\phi })$. We define $V_{\phi }$ to be the set of $L_{26}/S_Y(2)$-chambers D contained in $\mathrm{Nef}_{Y}$ such that the closure $\overline{D}$ of D in $S_{Y}\otimes {{\mathbb {R}}}$ contains the primitive isotropic ray $f_{\phi }=[F]/2$, where F is a general fiber of $\phi$, and $E_{\phi }$ to be the set of pairs of adjacent $L_{26}/S_Y(2)$-chambers in $V_{\phi }$. The stabilizer subgroup

$$\begin{aligned} G_{\phi }:=\mathrm{aut}(Y, \phi ):=\{\,{g\in \mathrm{aut}(Y)}\mid {f_{\phi }^g=f_{\phi }} \,\} \end{aligned}$$

of $\phi$ in $\mathrm{aut}(Y)$ acts on $(V_{\phi }, E_{\phi })$. Then condition ($\hbox {VE-}1$) is satisfied. Indeed, the set of $L_{26}/S_Y(2)$-chambers in $V_{\phi }$ adjacent to $D\in V_{\phi }$ is the set of all $D^{s_r}$, where r runs through the set of non-splitting roots of $S_{Y}$ defining walls of D such that $\langle r, f_{\phi } \rangle =0$. For $D, D^\prime \in V_{\phi }$, the subset ${T_{G}}(D, D^\prime )$ of $G_{\phi }$ is the set of isometries belonging to $\mathrm{isoms}(Y, D, D^\prime )$ that fixes $f_{\phi }$. Therefore ($\hbox {VE-}2$) and ($\hbox {VE-}3$) are also satisfied.

We apply Procedure 4.1 to every $\phi \in {{\mathcal {E}}}_{\mathrm{temp}}^\prime$ and obtain a complete set $V_{\phi , 0}$ of representatives of orbits of the action of $G_{\phi }$ on $V_{\phi }$. We also obtain a finite generating set ${{\mathcal {G}}}_{\phi }$ of the stabilizer subgroup $\mathrm{aut}(Y, \phi )$.

The set $\Sigma _{\phi }$ of classes of smooth rational curves C contained in some fiber of $\phi$ is calculated as follows. Let $a_Y$ be an ample class of Y. Every class $[C]\in \Sigma _{\phi }$ satisfies $\langle [C], f_{\phi } \rangle =0$ and $0<\langle [C], a_Y \rangle < 2\langle f_{\phi }, a_Y \rangle$. We calculate the set $\Sigma ^\prime$ of all roots r of $S_{Y}$ satisfying $\langle r, f_{\phi } \rangle =0$ and $0<\langle r, a_Y \rangle <2\langle f_{\phi }, a_Y \rangle$. Then $r\in \Sigma ^\prime$ belongs to $\Sigma _{\phi }$ if and only if r splits in $S_{X}$ (see Sect. 5.2) and there exist no roots $r^\prime \in \Sigma _{\phi }$ such that $\langle r^\prime , a_Y \rangle <\langle r, a_Y \rangle$ and $\langle r, r^\prime \rangle <0$. Therefore we can calculate $\Sigma _{\phi }$ by sorting the elements r of $\Sigma ^\prime$ according to $\langle r, a_Y \rangle$ and applying the above criterion to $r\in \Sigma ^\prime$ in this order.

Each connected component of the dual graph of roots in $\Sigma _{\phi }$ corresponds to a reducible fiber of $\phi$, and is the Dynkin diagram of an affine $\mathrm{ADE}$-type. Let $\Gamma$ be a connected component. The weighted sum of roots in $\Gamma$ with appropriate weights according to the $\mathrm{ADE}$-type of $\Gamma$ (see, for example, [26, Theorem 5.12]) is either $f_{\phi }$ or $2f_{\phi }$. The former case occurs when the corresponding reducible fiber is a multiple fiber, while the latter occurs when the fiber is non-multiple.

Let $\phi ^\prime {:}\,Y\rightarrow {{\mathbb {P}}}^1$ be another element of ${{\mathcal {E}}}_{\mathrm{temp}}^\prime$. Then $\phi$ and $\phi ^\prime$ are contained in the same orbit under the action of $\mathrm{aut}(Y)$ on ${{\mathcal {E}}}(Y)$ if and only if the following holds. Let D be an element of $V_{\phi , 0}$. Then there exists $D^\prime \in V_{\phi ^\prime , 0}$ such that $\mathrm{isoms}(Y, D, D^\prime )$ contains an isometry that maps $f_{\phi }$ to $f_{\phi ^\prime }$. Note that $D'$ can be computed explicitly. Applying this method to all pairs $\phi , \phi ^\prime$ of distinct elements of ${{\mathcal {E}}}_{\mathrm{temp}}^\prime$, we obtain a complete set of representatives of the action of $\mathrm{aut}(Y)$ on ${{\mathcal {E}}}(Y)$.

6.5 Table of Elliptic Fibrations

Let $\phi {:}\,Y\rightarrow {{\mathbb {P}}}^1$ be an elliptic fibration of an Enriques surface Y. Then $\phi$ has exactly two multiple fibers, and both of them are of multiplicity 2. In the table below, the first column shows the $\mathrm{ADE}$-types of non-multiple reducible fibers, and the second column shows the $\mathrm{ADE}$-types of multiple reducible fibers. The third column gives the number of elliptic fibrations modulo $\mathrm{aut}(Y)$. See [32] for the cases with ${\mathrm {rank}}\, \tau \ge 8$.

7 Examples

7.1 An $(E_6, E_6)$-Generic Enriques Surface

In [31], we investigated an $(E_6, E_6)$-generic Enriques surface (No. 47 of Table 1). We briefly review the result of [31].

Let $\overline{X}\subset {{\mathbb {P}}}^3$ be a quartic Hessian surface associated with a very general cubic homogeneous polynomial, and X the minimal resolution of $\overline{X}$. Then $\overline{X}$ contains 10 lines and has 10 ordinary nodes, and the K3 surface X has a fixed-point free involution $\varepsilon$ that interchanges the strict transforms of the 10 lines and the exceptional curves over the 10 ordinary nodes. Let $\pi {:}\,X\rightarrow Y$ be the quotient morphism by $\varepsilon$. Then the Enriques surface Y is $(E_6, E_6)$-generic (see Kondo [17]).

We can construct a sequence of primitive embeddings $S_{Y}(2)\hookrightarrow S_{X}\hookrightarrow L_{26}$ from the primitive embeddings $L_{10}(2) \hookrightarrow L_{26}$ of type 20E. We see that $D_0$ is a fundamental domain of the action of $\mathrm{aut}(Y)$ on $\mathrm{Nef}_Y$, and hence

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))=\mathrm{{vol}}(D_0)=\frac{1_{\mathrm{{BP}}}}{51840}= \frac{1_{\mathrm{{BP}}}}{|W(R_{E_6})|}. \end{aligned}$$

In fact, the $L_{26}/S_Y(2)$-chamber $D_0$ is equal to the chamber $D_Y$ in [31]. We then obtain the same result as Table 1.1 of [31] for ${{\mathcal {E}}}(Y)/\mathrm{aut}(Y)$. We also prove that $\mathrm{aut}(Y)$ acts on ${{\mathcal {R}}}(Y)$ transitively.

The last result contradicts Theorem 1.5 of [31], because Table 1.2 of [31] says that there exist 10 orbits of the action of $\mathrm{aut}(Y)$ on ${{\mathcal {R}}}(Y)$. In fact, the argument in Section 7.6 of [31] for the calculation of the number of $\mathrm{aut}(Y)$-orbits of $\mathrm{RDP}$-configurations is wrong, and Table 1.2 of [31] should be replaced by Table 2.

Table 2 $\mathrm{RDP}$-configurations on Y

Here we present a correct method for the calculation of $\mathrm{aut}(Y)$-orbits of $\mathrm{RDP}$-configurations. Let $\psi {:}\,Y\rightarrow \overline{Y}$ be a birational morphism to a surface $\overline{Y}$ that has only rational double points as its singularities, and let $h_{\psi }$ be an ample class of $\overline{Y}$. Since the $L_{26}/S_Y(2)$-chamber $D_0$ is a fundamental domain of the action of $\mathrm{aut}(Y)$ on $\mathrm{Nef}_{Y}$, we can assume that $\psi ^*(h_{\psi }) \in S_{Y}$ belongs to $D_0$ by composing $\psi$ with an automorphism of Y. Let f be the minimal face of $D_0$ containing $\psi ^*(h_{\psi })$. Then the set of the classes of smooth rational curves C contracted by $\psi$ is equal to

$$\begin{aligned} \Gamma (f):=\{\,{[C]}\mid {C \text { is a smooth rational curve on } Y \text { such that } f\subset ([C])^{\perp }} \,\}. \end{aligned}$$

For a given face f of $D_0$, we calculate the set of roots r of $S_{Y}$ such that $f\subset (r)^{\perp }$. From this set, we can calculate $\Gamma (f)$ by using the ample class $a_Y$ and the set of $(-4)$-vectors in $S_{X-}$. We calculate $\Gamma (f)$ for all faces f of $D_0$, and obtain 750 $\mathrm{RDP}$-configurations of smooth rational curves. Every $\mathrm{RDP}$-configuration on Y is equal to one of them modulo the action of $\mathrm{aut}(Y)$.

Let $\Gamma$ be one of the 750 $\mathrm{RDP}$-configurations. We put $\mu :=|\Gamma |$, that is, $\mu$ is the total Milnor number of the singularities of the surface $\overline{Y}$ corresponding to $\Gamma$. The sublattice $\langle {\Gamma } \rangle$ of $S_{Y}$ generated by the classes in $\Gamma$ is negative definite of rank $\mu$, and its orthogonal complement $\langle {\Gamma } \rangle ^{\perp }$ is hyperbolic of rank $10-\mu$. Let ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}$ be the positive half-cone of $\langle {\Gamma } \rangle ^{\perp }$ contained in ${{\mathcal {P}}}_Y$. Composing the primitive embedding $\langle {\Gamma } \rangle ^{\perp }\hookrightarrow S_{Y}$ with the primitive embedding $S_{Y}(2)\hookrightarrow L_{26}$ of type 20E, we have $L_{26}/\langle {\Gamma } \rangle ^{\perp }(2)$-chambers of ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}$. The intersection $f_0:={{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}\cap D_0$ is one of the $L_{26}/\langle {\Gamma } \rangle ^{\perp }(2)$-chambers, and it is the maximal face of $D_0$ among all the faces f of $D_0$ such that $\Gamma (f)=\Gamma$. Let $(V_{\Gamma }, E_{\Gamma })$ be the graph where $V_{\Gamma }$ is the set of $L_{26}/\langle {\Gamma } \rangle ^{\perp }(2)$-chambers on ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}$ contained in ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}\cap \mathrm{Nef}_{Y}$ and $E_{\Gamma }$ is the usual adjacency relation of chambers. Then $D\mapsto {{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}\cap D$ gives a bijection to the set $V_{\Gamma }$ of vertices from the set of $L_{26}/S_Y(2)$-chambers D contained in $\mathrm{Nef}_{Y}$ such that ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}\cap D$ is a face of D of dimension $10-\mu$, or equivalently, such that ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}\cap D$ contains a non-empty open subset of ${{\mathcal {P}}}_{\langle {\Gamma } \rangle ^{\perp }}$. The group

$$\begin{aligned} G_{\Gamma }:=\{\,{g\in \mathrm{aut}(Y)}\mid {\Gamma ^g=\Gamma } \,\} \end{aligned}$$

acts on the graph $(V_{\Gamma }, E_{\Gamma })$. We apply Procedure 4.1 to $(V_{\Gamma }, E_{\Gamma })$ and $G_{\Gamma }$, and obtain a complete set $V_{\Gamma , 0}$ of representatives of $V_{\Gamma }/G_{\Gamma }$. Let $\Gamma ^\prime$ be one of the 750 $\mathrm{RDP}$-configurations with the same $\mathrm{ADE}$-type as $\Gamma$. Let $V_{\Gamma ^\prime , 0}$ be a complete set of representatives of $V_{\Gamma ^\prime }/G_{\Gamma ^\prime }$. Then the $\mathrm{RDP}$-configurations $\Gamma$ and $\Gamma ^\prime$ are in the same orbit under the action of $\mathrm{aut}(Y)$ if and only if there exists an $L_{26}/\langle {\Gamma ^\prime } \rangle ^{\perp }(2)$-chamber $f^\prime ={{\mathcal {P}}}_{\langle {\Gamma ^\prime } \rangle ^{\perp }}\cap D^\prime \in V_{\Gamma ^\prime ,0}$ with $D^\prime \subset \mathrm{Nef}_{Y}$ such that $\mathrm{isoms}(Y, D_0, D^\prime )$ contains an element g satisfying $\Gamma ^g=\Gamma ^\prime$. Since $|V_{\Gamma ^\prime ,0}|$ is finite, we can determine whether $\Gamma$ and $\Gamma ^\prime$ are in the same orbit or not. Applying this method to all pairs $\Gamma$ and $\Gamma ^\prime$ with the same $\mathrm{ADE}$-type, we obtain a complete set of representatives of $\mathrm{RDP}$-configurations modulo $\mathrm{aut}(Y)$.

7.2 $(4A_1, 4A_1)$-Generic and $(4A_1, D_4)$-Generic Enriques Surfaces

Let Y be a $(4A_1, 4A_1)$-generic Enriques surface (No. 7 of Table 1). We construct a sequence $S_{Y}(2)\hookrightarrow S_{X}\hookrightarrow L_{26}$ from the primitive embedding $L_{10}(2)\hookrightarrow L_{26}$ of type 96C. The complete set $V_0$ of representatives of orbits of the action of $\mathrm{aut}(Y)$ on the set of $L_{26}/S_Y(2)$-chambers contained in $\mathrm{Nef}_{Y}$ consists of 5 elements with the orders of stabilizer subgroups 1, 1, 1, 2, 1. Since $\mathrm{{vol}}(D_0)=1_{\mathrm{{BP}}}/72$, we have

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))= \mathrm{{vol}}(D_0)\left( \frac{1}{1}+\frac{1}{1}+\frac{1}{1}+\frac{1}{2}+\frac{1}{1} \right) =\frac{1_{\mathrm{{BP}}}}{16}=\frac{1_{\mathrm{{BP}}}}{|W(R_{4A_1})|}. \end{aligned}$$

The set ${{\mathcal {R}}}_{\mathrm{temp}}$ is of size 56 and the set ${{\mathcal {E}}}_{\mathrm{temp}}$ is of size 6270.

We also construct $S_{Y}(2)\hookrightarrow S_{X}\hookrightarrow L_{26}$ for a $(4A_1, D_4)$-generic Enriques surface (No. 8 of Table 1) from the primitive embedding of type 96C. The set $V_0$ consists of 18 elements with the orders of stabilizer subgroups $4, \dots , 4$. We have $|{{\mathcal {R}}}_{\mathrm{temp}}|=154$ and $|{{\mathcal {E}}}_{\mathrm{temp}}|=21452$.

7.3 A $(D_5, D_5)$-Generic Enriques Surface

We have to use the primitive embedding of type 40A to construct $S_{Y}(2)\hookrightarrow S_{X}\hookrightarrow L_{26}$ for a $(D_5, D_5)$-generic Enriques surface (No. 24 of Table 1). The set $V_0$ consists of 6 elements with the orders of stabilizer subgroups $2, \dots , 2$. In this case, we have $\mathrm{{vol}}(D_0)=1_{\mathrm{{BP}}}/5760$ and

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))= \mathrm{{vol}}(D_0)\left( \frac{1}{2}+\frac{1}{2}+\frac{1}{2}+\frac{1}{2}+\frac{1}{2}+\frac{1}{2} \right) =\frac{1_{\mathrm{{BP}}}}{1920}=\frac{1_{\mathrm{{BP}}}}{|W(R_{D_5})|}. \end{aligned}$$

We have $|{{\mathcal {R}}}_{\mathrm{temp}}|=15$ and $|{{\mathcal {E}}}_{\mathrm{temp}}|=758$.

7.4 Enriques Surfaces with Finite Automorphism Group

Let Y be an Enriques surface with finite automorphism group of type I in Kondo’s classification [16]. We assume that Y is chosen very general so that the covering K3 surface X is of Picard number 19 and satisfies ${\mathrm {O}}(T_X, \omega )=\{\pm 1\}$. Then Y is $(E_8+A_1, E_8+A_1)$-generic (No. 172 of Table 1). The automorphism group ${\mathrm {Aut}}(Y)$ is a dihedral group of order 8, and its image $\mathrm{aut}(Y)$ in ${{\mathrm {O}}}^{{{\mathcal {P}}}}(S_{Y})$ is order 4. The Enriques surface Y has exactly 12 smooth rational curves, and their dual graph is given in [16, Fig. 1.4]. The chamber $\mathrm{Nef}_{Y}$ is isomorphic to an $L_{26}/L_{10}(2)$-chamber $D_0$ of the primitive embedding $L_{10}(2)\hookrightarrow L_{26}$ of type 12A, and hence $\mathrm{{vol}}(D_0)=1_{\mathrm{{BP}}}/174182400$ (see [6]). Therefore

$$\begin{aligned} \mathrm{{vol}}(\mathrm{Nef}_{Y}/\mathrm{aut}(Y))=\frac{\mathrm{{vol}}(D_0)}{4}=\frac{1_{\mathrm{{BP}}}}{2^{14} 3^5 5^2 7} =\frac{2_{\mathrm{{BP}}}}{ |W(R_{E_8+A_1})|}. \end{aligned}$$

The group $\mathrm{aut}(Y)$ decomposes ${{\mathcal {R}}}(Y)$ as $2+2+2+2+4$.

For a very general Enriques surface Y with finite automorphism group of type II, the chamber $\mathrm{Nef}_{Y}$ is isomorphic to an $L_{26}/L_{10}(2)$-chamber $D_0$ of the primitive embedding $L_{10}(2)\hookrightarrow L_{26}$ of type 12B. We have $\mathrm{{vol}}(D_0)=1_{\mathrm{{BP}}}/3870720$. Note that $3870720\cdot |{{\mathfrak {S}}}_4|=|W(R_{D_9})|$. The Enriques surface Y is $(D_9, D_9)$-generic (No. 184 of Table 1), and we have ${\mathrm {Aut}}(Y)\cong \mathrm{aut}(Y)\cong {{\mathfrak {S}}}_4$. The group $\mathrm{aut}(Y)$ decomposes ${{\mathcal {R}}}(Y)$ as $6+6$.