We investigate properties of a spin-1 Heisenberg model with extended and biquadratic interactions, which captures crucial aspects of the low energy physics in FeSe. While we show that the model exhibits a rich phase diagram with four different magnetic ordering tendencies, we identify a parameter regime with strong competition between Néel, staggered dimer, and stripelike magnetic fluctuations, accounting for the physical properties of FeSe. We evaluate the spin and Raman responses using exact diagonalization. Through comparison with experiments we find enhanced magnetic frustration between Néel and collinear stripe ordering tendencies, which increases with increasing temperature. The explanation of these spectral behaviors with this frustrated spin model supports the idea of local spin interactions in FeSe.

• Received 11 December 2018

DOI:https://doi.org/10.1103/PhysRevB.99.125130