We investigate an integrable Hamiltonian modeling a heterotriatomic molecular Bose-Einstein condensate. This model describes a mixture of two species of atoms in different proportions, which can combine to form a triatomic molecule. Beginning with a classical analysis, we determine the fixed points of the system. Bifurcations of these points separate the parameter space into different regions. Three distinct scenarios are found, varying with the atomic population imbalance. This result suggests the ground-state properties of the quantum model exhibit a sensitivity on the atomic population imbalance, which is confirmed by a quantum analysis using different approaches, such as the ground-state expectation values, the behavior of the quantum dynamics, the energy gap, and the ground-state fidelity.

• Received 5 November 2008

DOI:https://doi.org/10.1103/PhysRevA.79.013624