Abstract
In recent years it has been shown that clustering and segmentation methods can greatly benefit from the integration of prior information in terms of must-link constraints. Very recently the use of such constraints has been integrated in a rigorous manner also in graph-based methods such as normalized cut. On the other hand spectral clustering as relaxation of the normalized cut has been shown to be among the best methods for video segmentation. In this paper we merge these two developments and propose to learn must-link constraints for video segmentation with spectral clustering. We show that the integration of learned must-link constraints not only improves the segmentation result but also significantly reduces the required runtime, making the use of costly spectral methods possible for today’s high quality video.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
correct refers to the desired ground truth segmentation, which ideally corresponds with the optimal segmentation \(S^*\).
- 2.
certain groupings are the conservative grouping decisions which we propose to learn.
References
Alpert, S., Galun, M., Brandt, A., Basri, R.: Image segmentation by probabilistic bottom-up aggregation and cue integration. TPAMI 34(2), 315–326 (2012)
Andres, B., Kappes, J.H., Beier, T., Köthe, U., Hamprecht, F.A.: Probabilistic image segmentation with closedness constraints. In: ICCV (2011)
Arbeláez, P., Maire, M., Fowlkes, C.C., Malik, J.: Contour detection and hierarchical image segmentation. TPAMI 33(5), 898–916 (2011)
Banica, D., Agape, A., Ion, A., Sminchisescu, C.: Video object segmentation by salient segment chain composition. In: ICCV, IPGM Workshop (2013)
Basu, S., Davidson, I., Wagstaff, K.: Constrained Clustering: Advances in Algorithms, Theory, and Applications. Chapman & Hall, Boca Raton (2008)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Brendel, W., Todorovic, S.: Video object segmentation by tracking regions. In: ICCV (2009)
Brox, T., Malik, J.: Object segmentation by long term analysis of point trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)
Bühler, T., Hein, M.: Spectral clustering based on the graph p-Laplacian. In: ICML (2009)
Chang, J., Wei, D., Fisher, J.W.: A video representation using temporal superpixels. In: CVPR (2013)
Cheng, H.T., Ahuja, N.: Exploiting nonlocal spatiotemporal structure for video segmentation. In: CVPR (2012)
Criminisi, A., Shotton, J., Konukoglu, E.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. In: Foundations and Trends in Computer Graphics and Vision (2012)
Eriksson, A.P., Olsson, C., Kahl, F.: Normalized cuts revisited: a reformulation for segmentation with linear grouping constraints. In: ICCV (2007)
Fowlkes, C., Malik, J.: How much does globalization help segmentation?. Technical report, EECS - UC Berkeley (2004)
Fragkiadaki, K., Shi, J.: Video segmentation by tracing discontinuities in a trajectory embedding. In: CVPR (2012)
Galasso, F., Keuper, M., Brox, T., Schiele, B.: Spectral graph reduction for efficient image and streaming video segmentation. In: CVPR (2014)
Galasso, F., Nagaraja, N.S., Cardenas, T.Z., Brox, T., Schiele, B.: A unified video segmentation benchmark: annotation, metrics and analysis. In: ICCV (2013)
Galasso, F., Cipolla, R., Schiele, B.: Video segmentation with superpixels. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part I. LNCS, vol. 7724, pp. 760–774. Springer, Heidelberg (2013)
Galasso, F., Iwasaki, M., Nobori, K., Cipolla, R.: Spatio-temporal clustering of probabilistic region trajectories. In: ICCV (2011)
Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: CVPR (2010)
Hein, M., Bühler, T.: An inverse power method for nonlinear eigenproblems with applications in 1spectral clustering and sparse pca. In: NIPS (2010)
Hein, M., Setzer, S.: Beyond spectral clustering - tight relaxations of balanced graph cuts. In: NIPS (2011)
Jain, V., Turaga, S.C., Briggman, K.L., Helmstaedter, M., Denk, W., Seung, H.S.: Learning to agglomerate superpixel hierarchies. In: NIPS (2011)
Kamvar, S.D., Klein, D., Manning, C.D.: Spectral learning. In: IJCAI (2003)
Kannan, A., Jojic, N., Frey, B.J.: Generative model for layers of appearance and deformation. In: AISTATS (2005)
Kumar, M.P., Torr, P., Zisserman, A.: Learning layered motion segmentations of video. IJCV 76, 301–319 (2008)
Lee, Y.J., Kim, J., Grauman, K.: Key-segments for video object segmentation. In: ICCV (2011)
Lezama, J., Alahari, K., Sivic, J., Laptev, I.: Track to the future: Spatio-temporal video segmentation with long-range motion cues. In: CVPR (2011)
Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: ICCV (2013)
Li, Z., Liu, J., Tang, X.: Constrained clustering via spectral regularization. In: CVPR (2009)
von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)
Maire, M., Yu, S.X.: Progressive multigrid eigensolvers for multiscale spectral segmentation. In: ICCV (2013)
Maji, S., Vishnoi, N.K., Malik, J.: Biased normalized cuts. In: CVPR (2011)
Ng, A.Y., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS (2001)
Ochs, P., Malik, J., Brox, T.: Segmentation of moving objects by long term video analysis. TPAMI 36, 1187–1200 (2013)
Palou, G., Salembier, P.: Hierarchical video representation with trajectory binary partition tree. In: CVPR (2013)
Paris, S.: Edge-preserving smoothing and mean-shift segmentation of video streams. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 460–473. Springer, Heidelberg (2008)
Rangapuram, S., Hein, M.: Constrained 1-spectral clustering. In: AISTATS (2012)
Shi, J., Malik, J.: Normalized cuts and image segmentation. TPAMI 22, 1–8 (2000)
Sundaram, N., Keutzer, K.: Long term video segmentation through pixel level spectral clustering on gpus. In: ICCV Workshops (2011)
Sundberg, P., T.Brox, Maire, M., Arbelaez, P., Malik, J.: Occlusion boundary detection and figure/ground assignment from optical flow. In: CVPR (2011)
Taylor, C.: Towards fast and accurate segmentation. In: CVPR (2013)
Vazquez-Reina, A., Avidan, S., Pfister, H., Miller, E.: Multiple hypothesis video segmentation from superpixel flows. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 268–281. Springer, Heidelberg (2010)
Wagstaff, K., Cardie, C., Rogers, S., Schrödl, S.: Constrained k-means clustering with background knowledge. In: ICML (2001)
Wang, X., Davidson, I.: Flexible constrained spectral clustering. In: KDD (2010)
Xu, C., Corso, J.J.: Evaluation of super-voxel methods for early video processing. In: CVPR (2012)
Xu, C., Xiong, C., Corso, J.J.: Streaming hierarchical video segmentation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 626–639. Springer, Heidelberg (2012)
Xu, L., Li, W., Schuurmans, D.: Fast normalized cut with linear constraints. In: CVPR (2009)
Yu, S.X., Shi, J.: Grouping with bias. In: NIPS (2001)
Acknowledgments
The authors would like to thank Syama Sundar Rangapuram for his support on the use of the 1-spectral clustering code.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Khoreva, A., Galasso, F., Hein, M., Schiele, B. (2014). Learning Must-Link Constraints for Video Segmentation Based on Spectral Clustering. In: Jiang, X., Hornegger, J., Koch, R. (eds) Pattern Recognition. GCPR 2014. Lecture Notes in Computer Science(), vol 8753. Springer, Cham. https://doi.org/10.1007/978-3-319-11752-2_58
Download citation
DOI: https://doi.org/10.1007/978-3-319-11752-2_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11751-5
Online ISBN: 978-3-319-11752-2
eBook Packages: Computer ScienceComputer Science (R0)