It is well known that the ground state of homogeneous superconducting systems with spin-orbit coupling in the presence of the Zeeman field is the so-called helical state, which is characterized by the phase modulation of the order parameter, but zero supercurrent density. In this paper we investigate the realization of the helical state in a hybrid system with spatially separated superconductivity and exchange field by considering a superconductor/ferromagnet ($S/F$) bilayer on top of a three-dimensional topological insulator. This system is characterized by strong spin-momentum locking and, consequently, provides the most favorable conditions for the helical state generation. The analysis is based on the microscopic theory in terms of the quasiclassical Green's functions. We demonstrate that in the bilayer the helical state survives if the exchange field has a nonzero component perpendicular to the $S/F$ interface even in spite of the fact that the superconducting order parameter and the exchange field are spatially separated. At the same time, in this spatially inhomogeneous situation the helical state is accompanied by the spontaneous currents distributed over the bilayer in such a way that the net current vanishes. Further, we show that this hybrid helical state gives rise to nonreciprocity in the system. We demonstrate the realization of the nonreciprocity in the form of the superconducting diode effect and investigate its dependence on the parameters of the bilayer.

• Received 30 March 2022
• Revised 3 November 2022
• Accepted 21 November 2022

DOI:https://doi.org/10.1103/PhysRevB.106.224509