We present the results of a study of the charmless vector-vector decay $B^0\to \omega K^{*0}$ with $657\times {10}^6$ $B\overline{B}$ pairs collected with the Belle detector at the KEKB $e^{+}{e}^{-}$ collider. We measure the branching fraction to be $\mathcal{B}(B^0\to \omega K^{*0})=[1.8\pm 0.7(\mathrm{stat})\pm 0.3(\mathrm{syst})]\times {10}^{-6}$ with $3.0\sigma$ significance. We also perform a helicity analysis of the $\omega$ and $K^{*0}$ vector mesons, and obtain the longitudinal polarization fraction $f_L(B^0\to \omega K^{*0})=0.56\pm 0.29(\mathrm{stat}{)}_{-0.08}^{+0.18}(\mathrm{syst})$. Finally, we measure a large nonresonant branching fraction $\mathcal{B}[B^0\to \omega K^{+}{\pi }^{-};M_{K\pi }\in (0.755,1.250)\mathrm{GeV}/c^2]=[5.1\pm 0.7(\mathrm{stat})\pm 0.7(\mathrm{syst})]\times {10}^{-6}$ with a significance of $9.5\sigma$.

• Received 27 July 2008

DOI:https://doi.org/10.1103/PhysRevLett.101.231801