Abstract
We study a prediction + optimisation formulation of the knapsack problem. The goal is to predict the profits of knapsack items based on historical data, and afterwards use these predictions to solve the knapsack. The key is that the item profits are not known beforehand and thus must be estimated, but the quality of the solution is evaluated with respect to the true profits. We formalise the problem, the goal of minimising expected regret and the learning problem, and investigate different machine learning approaches that are suitable for the optimisation problem. Recent methods for linear programs have incorporated the linear relaxation directly into the loss function. In contrast, we consider less intrusive techniques of changing the loss function, such as standard and multi-output regression, and learning-to-rank methods. We empirically compare the approaches on real-life energy price data and synthetic benchmarks, and investigate the merits of the different approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note the different problem where profits are known and the weights are learned is more complicated, since we may need some form of recourse mechanism to repair inconsistent decisions \(X^{f}\).
References
Bessiere, C., Koriche, F., Lazaar, N., O’Sullivan, B.: Constraint acquisition. Artif. Intell. 244, 315–342 (2017). https://doi.org/10.1016/j.artint.2015.08.001, http://www.sciencedirect.com/science/article/pii/S0004370215001162, combining Constraint Solving with Mining and Learning
Borchani, H., Varando, G., Bielza, C., Larrañaga, P.: A survey on multi-output regression. Wiley Interdisc. Rev. Data Min. Knowl. Dis. 5(5), 216–233 (2015)
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 13(1), 21–27 (1967)
Di Liberto, G., Kadioglu, S., Leo, K., Malitsky, Y.: Dash: dynamic approach for switching heuristics. Eur. J. Oper. Res. 248(3), 943–953 (2016)
Donti, P.L., Amos, B., Kolter, J.Z.: Task-based end-to-end model learning in stochastic optimization. In: Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), pp. 5484–5494 (2017)
Dooren, D.V.D., Sys, T., Toffolo, T.A.M., Wauters, T., Berghe, V.: Multi-machine energy-aware scheduling. EURO J. Comput. Optim. 5(1–2), 285–307 (2017). https://doi.org/10.1007/s13675-016-0072-0
Dragone, P., Teso, S., Passerini, A.: Pyconstruct: constraint programming meets structured prediction. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, pp. 5823–5825. International Joint Conferences on Artificial Intelligence Organization, July 2018
Elmachtoub, A.N., Grigas, P.: Smart “predict, then optimize”. Technical report (2017). https://arxiv.org/pdf/1710.08005.pdf
Gilmore, P., Gomory, R.E.: The theory and computation of knapsack functions. Oper. Res. 14(6), 1045–1074 (1966)
Grimes, D., Ifrim, G., O’Sullivan, B., Simonis, H.: Analyzing the impact of electricity price forecasting on energy cost-aware scheduling. Sustain. Comput. Inform. Syst. 4(4), 276–291 (2014). https://doi.org/10.1016/j.suscom.2014.08.009,http://www.sciencedirect.com/science/article/pii/S221053791400050X, special Issue on Energy Aware Resource Management and Scheduling (EARMS)
Joachims, T.: Making large-scale svm learning practical. Technical report, SFB 475: Komplexitätsreduktion in Multivariaten Datenstrukturen, Universität Dortmund (1998)
Joachims, T.: Optimizing search engines using click through data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2002, pp. 133–142. ACM, New York (2002). https://doi.org/10.1145/775047.775067, http://doi.acm.org/10.1145/775047.775067
Joachims, T.: Training linear SVMs in linear time. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 217–226. ACM (2006)
Kotthoff, L.: Algorithm selection for combinatorial search problems: a survey. AI Mag. 35(3), 48–60 (2014). http://www.aaai.org/ojs/index.php/aimagazine/article/view/2460
Liu, T.Y., et al.: Learning to rank for information retrieval. Found. Trends Inf. Retrieval 3(3), 225–331 (2009)
Mathaba, T., Xia, X., Zhang, J.: Analysing the economic benefit of electricity price forecast in industrial load scheduling. Electric Power Syst. Res. 116, 158–165 (2014).https://doi.org/10.1016/j.epsr.2014.05.008, http://www.sciencedirect.com/science/article/pii/S0378779614001886
Matthews, G.: On the partition of numbers. Proc. Lond. Math. Soc. 28, 486–490 (1897)
Passerini, A., Tack, G., Guns, T.: Introduction to the special issue on combining constraint solving with mining and learning. Artif. Intell. 244, 1–5 (2017). https://doi.org/10.1016/j.artint.2017.01.002
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Picard-Cantin, É., Bouchard, M., Quimper, C.-G., Sweeney, J.: Learning the parameters of global constraints using branch-and-bound. In: Beck, J.C. (ed.) CP 2017. LNCS, vol. 10416, pp. 512–528. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66158-2_33
Spall, J.: Introduction to Stochastic Search and Optimization. Wiley, New York (2003)
Teso, S., Passerini, A., Viappiani, P.: Constructive preference elicitation by setwise max-margin learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2067–2073 (2016)
Vapnik, V.: Principles of risk minimization for learning theory. In: Advances in Neural Information Processing Systems, pp. 831–838 (1992)
Wilder, B., Dilkina, B., Tambe, M.: Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI 2019) (2019, to appear). https://arxiv.org/pdf/1809.05504.pdf
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Demirović, E. et al. (2019). An Investigation into Prediction + Optimisation for the Knapsack Problem. In: Rousseau, LM., Stergiou, K. (eds) Integration of Constraint Programming, Artificial Intelligence, and Operations Research. CPAIOR 2019. Lecture Notes in Computer Science(), vol 11494. Springer, Cham. https://doi.org/10.1007/978-3-030-19212-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-19212-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19211-2
Online ISBN: 978-3-030-19212-9
eBook Packages: Computer ScienceComputer Science (R0)