Skip to main content
Log in

A Discrete Lagrangian-Based Global-Search Method for Solving Satisfiability Problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Satisfiability is a class of NP-complete problems that model a wide range of real-world applications. These problems are difficult to solve because they have many local minima in their search space, often trapping greedy search methods that utilize some form of descent. In this paper, we propose a new discrete Lagrange-multiplier-based global-search method (DLM) for solving satisfiability problems. We derive new approaches for applying Lagrangian methods in discrete space, we show that an equilibrium is reached when a feasible assignment to the original problem is found and present heuristic algorithms to look for equilibrium points. Our method and analysis provides a theoretical foundation and generalization of local search schemes that optimize the objective alone and penalty-based schemes that optimize the constraints alone. In contrast to local search methods that restart from a new starting point when a search reaches a local trap, the Lagrange multipliers in DLM provide a force to lead the search out of a local minimum and move it in the direction provided by the Lagrange multipliers. In contrast to penalty-based schemes that rely only on the weights of violated constraints to escape from local minima, DLM also uses the value of an objective function (in this case the number of violated constraints) to provide further guidance. The dynamic shift in emphasis between the objective and the constraints, depending on their relative values, is the key of Lagrangian methods. One of the major advantages of DLM is that it has very few algorithmic parameters to be tuned by users. Besides the search procedure can be made deterministic and the results reproducible. We demonstrate our method by applying it to solve an extensive set of benchmark problems archived in DIMACS of Rutgers University. DLM often performs better than the best existing methods and can achieve an order-of-magnitude speed-up for some problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. K. J. Arrow and L. Hurwicz. Gradient method for concave programming, I: Local results. In K. J. Arrow, L. Hurwica, and H. Uzawa, editors, Studies in Linear and Nonlinear Programming. Stanford University Press, Stanford, CA, 1958.

    Google Scholar 

  2. V. Cerny. Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45: 41–51, 1985.

    Google Scholar 

  3. Y.-J. Chang and B. W. Wah. Lagrangian techniques for solving a class of zero-one integer linear programs. In Proc. Computer Software and Applications Conference, pages 156–161, Dallas, TX, August 1995. IEEE.

    Google Scholar 

  4. A. Cichocki and R. Unbehauen. Switched-capacitor artificial neural networks for nonlinear optimization with constraints. In Proc. of 1990 IEEE Int'l Symposium on Circuits and Systems, pages 2809–2812, 1990.

  5. A. Davenport, E. Tsang, C. Wang, and K. Zhu. Genet: A connectionist architecture for solving constraint satisfaction problems by iterative improvement. In Proc. of the 12th National Conf. on Artificial Intelligence, pages 325–330, Seattle, WA, 1994.

  6. M. Davis and H. Putnam. A computing procedure for quantification theory. J. Assoc. Comput. Mach., 7: 201–215, 1960.

    Google Scholar 

  7. T. Feo and M. Resende. A probabilistic heuristic for a computationally difficult set covering problem. Operations Research Letters, (8): 67–71, 1989.

    Google Scholar 

  8. T. Feo and M. Resende. Greedy randomized adaptive search procedures. Technical report, AT&T Bell Laboratories, Murray Hill, NJ 07974, February 1994.

    Google Scholar 

  9. B. Gavish and S. L. Hantler. An algorithm for optimal route selection in SNA networks. IEEE Transactions on Communications, pages 1154–1161, 1983.

  10. M. R. Genesereth and N. J. Nilsson. Logical Foundation of Artificial Intelligence. Morgan Kaufmann, 1987.

  11. I. Gent and T. Walsh. Towards an understanding of hill-climbing procedures for SAT. In Proc. of the 11th National Conf. on Artificial Intelligence, pages 28–33, Washington, DC, 1993.

  12. A. M. Geoffrion. Lagrangian relaxation and its uses in integer programming. Mathematical Programming Study, 2: 82–114, 1974.

    Google Scholar 

  13. F. Glover. Tabu search — Part I. ORSA J. Computing, 1(3): 190–206, 1989.

    Google Scholar 

  14. J. Gu. Parallel Algorithms and Architectures for Very Fast AI Search. PhD thesis, Dept. of Computer Science, University of Utah, August 1989.

  15. J. Gu. How to solve very large-scale satisfiability (VLSS) problems. Technical Report UCECETR-90-002, Univ. of Calgary, Canada, October 1990.

    Google Scholar 

  16. J. Gu. Efficient local search for very large-scale satisfiability problems. SIGART Bulletin, 3(1): 8–12, January 1992.

    Google Scholar 

  17. J. Gu. On optimizing a search problem. In N. G. Bourbakis, editor, Artificial Intelligence Methods and Applications. World Scientific Publishers, 1992.

  18. J. Gu. The UniSAT problem models (appendix). IEEE Trans. on Pattern Analysis and Machine Intelligence, 14(8): 865, Aug 1992.

    Google Scholar 

  19. J. Gu. Local search for satisfiability (SAT) problems. IEEE Trans. on Systems, Man, and Cybernetics, 23(4): 1108–1129, 1993.

    Google Scholar 

  20. J. Gu. Global optimization for satisfiability (SAT) problems. IEEE Trans. on Knowledge and Data Engineering, 6(3): 361–381, Jun 1994.

    Google Scholar 

  21. J. Gu. Constraint-Based Search. Cambridge University Press, New York, to appear.

  22. J. Gu and Q.-P. Gu. Average time complexities of several local search algorithms for the satisfiability problem (sat). Technical Report UCECE-TR-91-004, Univ. of Calgary, Canada, 1991.

    Google Scholar 

  23. J. Gu and Q.-P. Gu. Average time complexity of the sat1.3 algorithm. Technical report, Tech. Rep., Univ. of Calgary, Canada, 1992.

    Google Scholar 

  24. J. Gu and W. Wang. A novel discrete relaxation architecture. IEEE Trans. on Pattern Analysis and Machine Intelligence, 14(8): 857–865, August 1992.

    Google Scholar 

  25. P. Hansen and R. Jaumard. Algorithms for the maximum satisfiability problem. Computing, 44: 279–303, 1990.

    Google Scholar 

  26. M. Held and R. M. Karp. The traveling salesman problem and minimum spanning trees: Part II. {tiMathematical Programming}, 6: 62–88, 1971.

  27. J. H. Holland. Adaption in Natural and Adaptive Systems. University of Michigan Press, Ann Arbor, 1975.

    Google Scholar 

  28. J. N. Hooker. Resolution vs. cutting plane solution of inference problems: some computational results. Operations Research Letters, 7: 1–7, 1988.

    Google Scholar 

  29. A. P. Kamath, N. K. Karmarkar, K. G. Ramakrishnan, and M. G. C. Resende. Computational experience with an interior point algorithm on the satisfiability problem. Annals of Operations Research, 25: 43–58, 1990.

    Google Scholar 

  30. A. P. Kamath, N. K. Karmarkar, K. G. Ramakrishnan, and M. G. C. Resende. A continuous approach to inductive inference. Mathematical Programming, 57: 215–238, 1992.

    Google Scholar 

  31. S. Kirkpatrick, Jr. C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220(4598): 671–680, 1983.

    Google Scholar 

  32. J. Klincewicz. Avoiding local optima in the p-hub location problem using tabu search and GRASP. Annals of Operations Research, (40): 283–302, 1992.

    Google Scholar 

  33. D. G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley Publishing Company, 1984.

  34. Z. Michalewicz. Genetic Algorithms + Data Structure = Evolution Programs. Springer-Verlag, 1994.

  35. S. Minton, M. D. Johnson, A. B. Philips, and P. Laird. Minimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems. Artificial Intelligence, 58: 161–205, Dec 1992.

    Google Scholar 

  36. D. Mitchell, B. Selman, and H. Levesque. Hard and easy distributions of SAT problems. In {tiProc. of the 10th National Conf. on Artificial Intelligence}, pages 459–465, 1992.

  37. P. Morris. The breakout method for escaping from local minima. In Proc. of the 11th National Conf. on Artificial Intelligence, pages 40–45, Washington, DC, 1993.

  38. P. W. Purdom. Search rearrangement backtracking and polynomial average time. Artificial Intelligence, 21: 117–133, 1983.

    Google Scholar 

  39. M. G. C. Resende and T. Feo. AGRASP for satisfiability. In D. S. Johnson and M. A. Trick, editors, {tiCliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge}, volume 26, pages 499–520. DIMACS Series on Discrete Mathematics and Theoretical Computer Science, American Mathematical Society, 1996.

  40. M. G. C. Resende, L. S. Pitsoulis, and P. M. Pardalos. Approximate solution of weighted MAXSAT problems using GRASP. In Satisfiability Problem: Theory and Applications. DIMACS Series on Discrete Mathematics and Theoretical Computer Science. American Mathematical Society, 1997.

  41. J. A. Robinson. A machine-oriented logic based on the resolution principle. J. Assoc. Comput. Mach., pages 23–41, 1965.

  42. R. Sebastiani. Applying GSAT to non-clausal formulas. Journal of Artificial Intelligence Research, 1: 309–314, 1994.

    Google Scholar 

  43. B. Selman, 1995. private communication.

  44. B. Selman and H. Kautz. Domain-independent extensions to GSAT: Solving large structured satisfiability problems. In Proc. of the 13th Int'l Joint Conf. on Artificial Intelligence, pages 290–295, 1993.

  45. B. Selman, H. Kautz, and B. Cohen. Local search strategies for satisfiability testing. In Proc. of the Second DIMACS Challenge Workshop on Cliques, Coloring, and Satisfiability, Rutgers University, pages 290–295, oct 1993.

  46. B. Selman, H. Kautz, and B. Cohen. Noise strategies for improving local search. In Proc. of the 12th National Conf. on Artificial Intelligence, pages 337–343, Seattle, WA, 1994.

  47. B. Selman and H. A. Kautz. An empirical study of greedy local search for satisfiability testing. In {tiProc. of the 11th National Conf. on Artificial Intelligence}, pages 46–51, Washington, DC, 1993.

  48. B. Selman, H. J. Levesque, and D. G. Mitchell. A new method for solving hard satisfiability problems. In Proc. of AAAI-92, pages 440–446, San Jose, CA, 1992.

  49. Y. Shang and B. W. Wah. Discrete lagrangian-based search for solving MAX-SAT problems. In {tiProc. Int'l Joint Conf. on Artificial Intelligence}. IJCAI, (accepted to appear) Aug. 1997.

  50. D. M. Simmons. Nonlinear Programming for Operations Research. Prentice-Hall, Englewood Cliffs, NJ, 1975.

    Google Scholar 

  51. R. Socic and J. Gu. Fast search algorithms for the N-queen problem. IEEE Trans. on Systems, Man, and Cybernetics, 21(6): 1572–1576, November 1991.

    Google Scholar 

  52. R. Sosič and J. Gu. A polynomial time algorithm for the n-queens problem. SIGART Bulletin, 1(3): 7–11, October 1990.

    Google Scholar 

  53. R. Sosič and J. Gu. 3,000,000 queens in less than one minute. SIGART Bulletin, 2(2): 22–24, April 1991.

    Google Scholar 

  54. R. Sosič and J. Gu. Efficient local search with conflict minimization: A case study of the n-queens problem. IEEE Trans. on Knowledge and Data Engineering, 6(5): 661–668, 1994.

    Google Scholar 

  55. B. W. Wah, Y. Shang, and Z. Wu. Discrete lagrangian method for optimizing the design of multiplierless QMF filter banks. In Proc. Int'l Conf. on Application Specific Array Processors. IEEE, (accepted to appear) July 1997.

  56. G. R. Walsh. Methods of Optimization. John Wiley and Sons, 1975.

  57. S. Zhang and A. G. Constantinides. Lagrange programming neural networks. IEEE Transactions on Circuits and Systems-II: Analog and Digital Signal Processing, 39(7): 441–452, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shang, Y., Wah, B.W. A Discrete Lagrangian-Based Global-Search Method for Solving Satisfiability Problems. Journal of Global Optimization 12, 61–99 (1998). https://doi.org/10.1023/A:1008287028851

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008287028851

Navigation