Skip to main content
SpringerLink
Log in

A Proximal Interior Point Algorithm with Applications to Image Processing

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

In this article, we introduce a new proximal interior point algorithm (PIPA). This algorithm is able to handle convex optimization problems involving various constraints where the objective function is the sum of a Lipschitz differentiable term and a possibly nonsmooth one. Each iteration of PIPA involves the minimization of a merit function evaluated for decaying values of a logarithmic barrier parameter. This inner minimization is performed thanks to a finite number of subiterations of a variable metric forward-backward method employing a line search strategy. The convergence of this latter step as well as the convergence the global method itself is analyzed. The numerical efficiency of the proposed approach is demonstrated in two image processing applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. See http://proximity-operator.net/.

  2. Preliminary results regarding the use of proximal interior point methods in these applicative contexts can be found in our previously published communications [65, 66].

  3. www.escience.cn/people/feiyunZHU/Dataset_GT.html.

  4. https://web.stanford.edu/~boyd/cvxbook/cvxbook_examples/chap11/.

References

  1. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    MATH  Google Scholar 

  2. Briceño-Arias, L.M., Chierchia, G., Chouzenoux, E., Pesquet, J.-C.: A random block-coordinate Douglas–Rachford splitting method with low computational complexity for binary logistic regression. Comput. Optim. Appl. 1–20 (2017)

  3. Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1–2), 99–120 (2004)

    MathSciNet  MATH  Google Scholar 

  4. Wright, M.H.: Interior methods for constrained optimization. Acta Numer., pp. 341–407 (1991)

  5. Forsgren, A., Gill, P.E., Wright, M.H.: Interior methods for nonlinear optimization. SIAM Rev. 44(4), 525–597 (2002)

    MathSciNet  MATH  Google Scholar 

  6. Gondzio, J.: Interior point methods 25 years later. Eur. J. Oper. Res. 218(3), 587–601 (2012)

    MathSciNet  MATH  Google Scholar 

  7. Gould, N.I.M., Orban, D., Sartenaer, A., Toint, P.L.: Superlinear convergence of primal-dual interior point algorithms for nonlinear programming. SIAM J. Optim. 11(4), 974–1002 (2001)

    MathSciNet  MATH  Google Scholar 

  8. Johnson, C.A., Seidel, J., Sofer, A.: Interior-point methodology for 3-D PET reconstruction. IEEE Trans. Med. Imaging 19(4), 271–285 (2000)

    Google Scholar 

  9. Chouzenoux, E., Legendre, M., Moussaoui, S., Idier, J.: Fast constrained least squares spectral unmixing using primal-dual interior-point optimization. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 7(1), 59–69 (2014a)

    Google Scholar 

  10. Armand, P., Gilbert, J.-C., Jan-Jégou, S.: A feasible BFGS interior point algorithm for solving convex minimization problems. SIAM J. Optim. 11(1), 199–222 (2000)

    MathSciNet  MATH  Google Scholar 

  11. Bonettini, S., Serafini, T.: Non-negatively constrained image deblurring with an inexact interior point method. J. Comput. Appl. Math. 231(1), 236–248 (2009)

    MathSciNet  MATH  Google Scholar 

  12. Ahmad, R., Schniter, P.: Iteratively reweighted \(\ell _1\) approaches to sparse composite regularization. IEEE Trans. Comput. Imaging 1(4), 220–235 (2015)

    MathSciNet  Google Scholar 

  13. Osher, S., Solé, A., Vese, L.: Image decomposition and restoration using total variation minimization and the \({H}^{-1}\) norm. Multiscale Model. Simul. 1(3), 349–370 (2003)

    MathSciNet  MATH  Google Scholar 

  14. Fu, H., Ng, M.K., Nikolova, M., Barlow, J.L.: Efficient minimization methods of mixed \(\ell \)2-\(\ell \)1 and \(\ell \)1-\(\ell \)1 norms for image restoration. SIAM J. Sci. Comput. 27(6), 1881–1902 (2006)

    MathSciNet  Google Scholar 

  15. Kim, S.-J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point method for large-scale \(\ell _1 \)-regularized least squares. IEEE J. Sel. Top. Signal Process. 1(4), 606–617 (2007)

    Google Scholar 

  16. Fountoulakis, K., Gondzio, J.: Performance of first- and second-order methods for \(\ell _1\)-regularized least squares problems. Comput. Optim. Appl. 65(3), 605–635 (2016)

    MathSciNet  MATH  Google Scholar 

  17. Combettes, P.-L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, Berlin (2011)

  18. Combettes, P.-L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    MathSciNet  MATH  Google Scholar 

  19. Chen, G.H.G., Rockafellar, R.T.: Convergence rates in forward-backward splitting. SIAM J. Optim. 7(2), 421–444 (1997)

    MathSciNet  MATH  Google Scholar 

  20. Combettes, P.L., Vũ, B.C.: Variable metric forward-backward splitting with applications to monotone inclusions in duality. Optimization 63(9), 1289–1318 (2014)

    MathSciNet  MATH  Google Scholar 

  21. Chouzenoux, E., Pesquet, J.-C., Repetti, A.: Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function. J. Optim. Theory Appl. 162(1), 107–132 (2014b)

    MathSciNet  MATH  Google Scholar 

  22. Frankel, P., Garrigos, G., Peypouquet, J.: Splitting methods with variable metric for Kurdyka–Łojasiewicz functions and general convergence rates. J. Optim. Theory Appl. 165(3), 874–900 (2015)

    MathSciNet  MATH  Google Scholar 

  23. Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les équations aux dérivées partielles 117, 87–89 (1963)

    MATH  Google Scholar 

  24. Kurdyka, K.: On gradients of functions definable in o-minimal structures. Ann. de l’institut Fourier 48, 769–783 (1998)

    MathSciNet  MATH  Google Scholar 

  25. Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4), 1205–1223 (2007)

    MATH  Google Scholar 

  26. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. B 116(1), 5–16 (2009)

    MathSciNet  MATH  Google Scholar 

  27. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137(1–2), 91–129 (2013)

    MathSciNet  MATH  Google Scholar 

  28. Kaplan, A., Tichatschke, R.: Proximal methods in view of interior-point strategies. J. Optim. Theory Appl. 98(2), 399–429 (1998)

    MathSciNet  MATH  Google Scholar 

  29. Valkonen, T.: Interior-proximal primal-dual methods. arXiv preprint arXiv:1706.07067 (2017)

  30. Chouzenoux, E., Moussaoui, S., Idier, J.: Majorize-minimize linesearch for inversion methods involving barrier function optimization. Inverse Probl. 28(6), 065011 (2012)

    MathSciNet  MATH  Google Scholar 

  31. Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117(1–2), 387–423 (2009)

    MathSciNet  MATH  Google Scholar 

  32. Bello Cruz, J.Y., Nghia, T.T.A.: On the convergence of the forward-backward splitting method with linesearches. Optim. Methods Softw. 31(6), 1209–1238 (2016)

    MathSciNet  MATH  Google Scholar 

  33. Bonettini, S., Loris, I., Porta, F., Prato, M.: Variable metric inexact line-search-based methods for nonsmooth optimization. SIAM J. Optim. 26(2), 891–921 (2016)

    MathSciNet  MATH  Google Scholar 

  34. Salzo, S.: The variable metric forward-backward splitting algorithm under mild differentiability assumptions. SIAM J. Optim. 27(4), 2153–2181 (2017)

    MathSciNet  MATH  Google Scholar 

  35. Bonettini, S., Prato, M.: New convergence results for the scaled gradient projection method. Inverse Probl. 31(9), 095008 (2015)

    MathSciNet  MATH  Google Scholar 

  36. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis, vol. 317. Springer, Berlin (2009)

    MATH  Google Scholar 

  37. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Berlin (2017). https://doi.org/10.1007/978-3-319-48311-5

    Book  MATH  Google Scholar 

  38. Pustelnik, N., Chaux, C., Pesquet, J.-C.: Parallel proximal algorithm for image restoration using hybrid regularization. IEEE Trans. Image Process. 20(9), 2450–2462 (2011)

    MathSciNet  MATH  Google Scholar 

  39. Lee, J.D., Recht, B., Srebro, N., Tropp, J., Salakhutdinov, R.R.: Practical large-scale optimization for max-norm regularization. In: 23rd Advances in Neural Information Processing Systems (NIPS), pp. 1297–1305, Vancouver, Canada (2010)

  40. Combettes, P.-L., Dũng, D., Vũ, B.C.: Proximity for sums of composite functions. J. Math. Anal. Appl. 380(2), 680–688 (2011)

    MathSciNet  MATH  Google Scholar 

  41. Abboud, F., Chouzenoux, E., Pesquet, J.-C., Chenot, J.-H., Laborelli, L.: Dual block-coordinate forward-backward algorithm with application to deconvolution and deinterlacing of video sequences. J. Math. Imaging Vis. 59(3), 415–431 (2017)

    MathSciNet  MATH  Google Scholar 

  42. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)

    MathSciNet  MATH  Google Scholar 

  43. Li, G., Pong, T.K.: Calculus of the exponent of Kurdyka–Łojasiewicz inequality and its applications to linear convergence of first-order methods. Found. Comput. Math. 18(5), 1199–1232 (2018)

    MathSciNet  MATH  Google Scholar 

  44. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)

    MathSciNet  MATH  Google Scholar 

  45. Harizanov, S., Pesquet, J.-C., Steidl, G.: Epigraphical projection for solving least squares Anscombe transformed constrained optimization problems. In: 4th International Conference on Scale Space and Variational Methods in Computer Vision (SSVM), pp. 125–136, Schloss Seggau, Graz, Austria (2013). Springer

  46. Musse, O., Heitz, F., Armspach, J.-P.: Topology preserving deformable image matching using constrained hierarchical parametric models. IEEE Trans. Image Process. 10(7), 1081–1093 (2001)

    MATH  Google Scholar 

  47. Klodt, M., Cremers, D.: A convex framework for image segmentation with moment constraints. In: 13th IEEE International Conference on Computer Vision (ICCV), pp. 2236–2243. Sydney, Australia (2011)

  48. Chouzenoux, E., Pesquet, J.-C., Repetti, A.: A block coordinate variable metric forward-backward algorithm. J. Glob. Optim. 66(3), 457–485 (2016)

    MathSciNet  MATH  Google Scholar 

  49. Attouch, H., Czarnecki, M.-O., Peypouquet, J.: Prox-penalization and splitting methods for constrained variational problems. SIAM J. Optim. 21(1), 149–173 (2011a)

    MathSciNet  MATH  Google Scholar 

  50. Garrigos, G., Rosasco, L., Villa, S.: Iterative regularization via dual diagonal descent. J. Math. Imaging Vis. 60(2), 189–215 (2018)

    MathSciNet  MATH  Google Scholar 

  51. Attouch, H., Cabot, A., Czarnecki, M.-O.: Asymptotic behavior of nonautonomous monotone and subgradient evolution equations. Trans. Am. Math. Soc. 370(2), 755–790 (2018)

    MathSciNet  MATH  Google Scholar 

  52. Attouch, H., Czarnecki, M.-O., Peypouquet, J.: Coupling forward-backward with penalty schemes and parallel splitting for constrained variational inequalities. SIAM J. Optim. 21(4), 1251–1274 (2011b)

    MathSciNet  MATH  Google Scholar 

  53. Alvarez, F., Cabot, A.: Asymptotic selection of viscosity equilibria of semilinear evolution equations by the introduction of a slowly vanishing term. Discrete Contin. Dyn. Syst. 15(3), 921 (2006)

    MathSciNet  MATH  Google Scholar 

  54. Cabot, A.: Proximal point algorithm controlled by a slowly vanishing term: applications to hierarchical minimization. SIAM J. Optim. 15(2), 555–572 (2005)

    MathSciNet  MATH  Google Scholar 

  55. Iusem, A.N., Svaiter, B.F., Teboulle, M.: Entropy-like proximal methods in convex programming. Math. Oper. Res. 19(4), 790–814 (1994)

    MathSciNet  MATH  Google Scholar 

  56. Brito, A.S., da Cruz Neto, J.X., Lopes, J.O., Oliveira, P.R.: Interior proximal algorithm for quasiconvex programming problems and variational inequalities with linear constraints. J. Optim. Theory Appl. 154(1), 217–234 (2012)

    MathSciNet  MATH  Google Scholar 

  57. Quiroz, E.A.P., Ramirez, L.M., Oliveira, P.R.: An inexact proximal method for quasiconvex minimization. Eur. J. Oper. Res. 246(3), 721–729 (2015)

    MathSciNet  MATH  Google Scholar 

  58. Pustelnik, N., Benazza-Benhayia, A., Zheng, Y., Pesquet, J.-C.: Wavelet-based image deconvolution and reconstruction. In: Wiley Encyclopedia of Electrical and Electronics Engineering, pp. 1–34 (1999)

  59. Chaux, C., Benazza-Benyahia, A., Pesquet, J.-C., Duval, L.: Wavelet transform for the denoising of multivariate images. In: Collet, C., Chanussot, J., Chehdi, K. (eds.) Multivariate Image Processing, pp. 203–237. ISTE Ltd and Wiley (2010)

  60. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    MathSciNet  MATH  Google Scholar 

  61. Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Berlin (2009)

    MATH  Google Scholar 

  62. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    MATH  Google Scholar 

  63. Huang, Y., Dong, Y.: New properties of forward-backward splitting and a practical proximal-descent algorithm. Appl. Math. Comput. 237, 60–68 (2014)

    MathSciNet  MATH  Google Scholar 

  64. Bonnans, J.-F., Gilbert, J.-C., Lemaréchal, C., Sagastizábal, C.A.: Numerical Optimization: Theoretical and Practical Aspects. Springer, Berlin (2006)

    MATH  Google Scholar 

  65. Corbineau, M.-C., Chouzenoux, E., Pesquet, J.-C.: PIPA: a new proximal interior point algorithm for large-scale convex optimization. In: 43rd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1343–1347. Calgary, Canada (2018a)

  66. Corbineau, M.-C., Chouzenoux, E., Pesquet, J.-C.: Geometry-texture decomposition/reconstruction using a proximal interior point algorithm. In: 10th IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM), pp. 435–439, Sheffield, UK (2018b)

  67. Bioucas-Dias, J.M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., Chanussot, J.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 5(2), 354–379 (2012)

    Google Scholar 

  68. Chan, R.H., Kan, K.K., Nikolova, M., Plemmons, R.J.: A two-stage method for spectral-spatial classification of hyperspectral images. arXiv preprint arXiv:1806.00836 (2018)

  69. Iordache, M.-D., Bioucas-Dias, J.M., Plaza, A.: Total variation spatial regularization for sparse hyperspectral unmixing. IEEE Trans. Geosci. Remote Sens. 50(11), 4484–4502 (2012)

    Google Scholar 

  70. Keshava, N., Mustard, J.F.: Spectral unmixing. IEEE Signal Process. Mag. 19(1), 44–57 (2002)

    Google Scholar 

  71. Becker, S., Fadili, J.: A quasi-Newton proximal splitting method. In: 25th Advances in Neural Information Processing Systems (NIPS), pp. 2618–2626, Lake Tahoe, USA (2012)

  72. Setzer, S., Steidl, G., Teuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21(3), 193–199 (2010)

    Google Scholar 

  73. Komodakis, N., Pesquet, J.-C.: Playing with duality: an overview of recent primal-dual approaches for solving large-scale optimization problems. IEEE Signal Process. Mag. 32(6), 31–54 (2015)

    Google Scholar 

  74. Combettes, P.L., Condat, L., Pesquet, J.-C., Vũ, B.C.: A forward-backward view of some primal-dual optimization methods in image recovery. In: 21st IEEE International Conference on Image Processing (ICIP), pp. 4141–4145, Paris, France (2014)

  75. Raguet, H., Fadili, J., Peyré, G.: A generalized forward-backward splitting. SIAM J. Imaging Sci. 6(3), 1199–1226 (2013)

    MathSciNet  MATH  Google Scholar 

  76. Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)

    MathSciNet  MATH  Google Scholar 

  77. Raguet, H., Landrieu, L.: Preconditioning of a generalized forward-backward splitting and application to optimization on graphs. SIAM J. Imaging Sci. 8(4), 2706–2739 (2015)

    MathSciNet  MATH  Google Scholar 

  78. Frecon, J., Pustelnik, N., Wendt, H., Condat, L., Abry, P.: Multifractal-based texture segmentation using variational procedure. In: 12th IEEE Image, Video, and Multidimensional Signal Processing Workshop (IVMSP), pp. 1–5 (2016)

  79. Aujol, J.-F., Chan, T.F.: Combining geometrical and textured information to perform image classification. J. Vis. Commun. Image Represent. 17(5), 1004–1023 (2006)

    Google Scholar 

  80. Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. 12(8), 882–889 (2003)

    Google Scholar 

  81. Briceño-Arias, L.M., Combettes, P.L., Pesquet, J.-C., Pustelnik, N.: Proximal algorithms for multicomponent image recovery problems. J. Math. Imaging Vis. 41(1–2), 3–22 (2011)

    MathSciNet  MATH  Google Scholar 

  82. Pustelnik, N., Wendt, H., Abry, P.: Local regularity for texture segmentation: combining wavelet leaders and proximal minimization. In: 38th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5348–5352. Vancouver, Canada (2013)

  83. Haralick, R.M.: Statistical and structural approaches to texture. Proc. IEEE 67(5), 768–804 (1979)

    Google Scholar 

  84. Kak, A.C., Stanley, M.: Principles of Computerized Tomographic Imaging. SIAM, Philadelphia (2001)

    Google Scholar 

  85. Chouzenoux, E., Zolyniak, F., Gouillart, E., Talbot, H.: A majorize-minimize memory gradient algorithm applied to X-ray tomography. In: 20th IEEE International Conference on Image Processing (ICIP), pp. 1011–1015, Melbourne, Australia (2013)

  86. Gouillart, E., Krzakala, F., Mézard, M., Zdeborová, L.: Belief-propagation reconstruction for discrete tomography. Inverse Prob. 29(3), 035003 (2013)

    MathSciNet  MATH  Google Scholar 

  87. Chouzenoux, E., Pesquet, J.-C., Repetti, A.: Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function. J. Optim. Theory Appl. 162(1), 107–132 (2014c)

    MathSciNet  MATH  Google Scholar 

  88. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CVN, CentraleSupélec, Inria Saclay, University Paris-Saclay, 9 Rue Joliot Curie, 91190, Gif-sur-Yvette, France

    Emilie Chouzenoux, Marie-Caroline Corbineau & Jean-Christophe Pesquet

Authors
  1. Emilie Chouzenoux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marie-Caroline Corbineau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Christophe Pesquet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marie-Caroline Corbineau.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chouzenoux, E., Corbineau, MC. & Pesquet, JC. A Proximal Interior Point Algorithm with Applications to Image Processing. J Math Imaging Vis 62, 919–940 (2020). https://doi.org/10.1007/s10851-019-00916-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-019-00916-w

Keywords

Search

Navigation