We study the system of coupled atomic and molecular condensates within the two-mode model and beyond mean-field theory. Large-amplitude atom-molecule coherent oscillations are shown to be damped by the rapid growth of fluctuations near the dynamically unstable molecular mode. This result contradicts earlier predictions about the recovery of atom-molecule oscillations in the two-mode limit. The frequency of the damped oscillation is also shown to scale as $\sqrt{N}/\ln N$ with the total number of atoms N, rather than the expected pure $\sqrt{N}$ scaling. Using a linearized model, we obtain analytical expressions for the initial depletion of the molecular condensate in the vicinity of the instability, and show that the important effect neglected by mean-field theory is an initially nonexponential “spontaneous” dissociation into the atomic vacuum. Starting with a small population in the atomic mode, the initial dissociation rate is sensitive to the exact atomic amplitudes, with the fastest (superexponential) rate observed for the entangled state formed by spontaneous dissociation.

• Received 21 May 2001

DOI:https://doi.org/10.1103/PhysRevA.64.063611