Dhanushka Jayasuriya
Jens Dietrich
Kelly Blincoe
ABSTRACT
Modern software applications rely heavily on the usage of libraries, which provide reusable functionality, to accelerate the development process. As libraries evolve and release new versions, the software systems that depend on those libraries (the clients) should update their dependencies to use these new versions as the new release could, for example, include critical fixes for security vulnerabilities. However, updating is not always a smooth process, as it can result in software failures in the clients if the new version includes breaking changes. Yet, there is little research on how these breaking changes impact the client projects in the wild. To identify if changes between two library versions cause breaking changes at the client end, we perform an empirical study on Java projects built using Maven. For the analysis, we used 18,415 Maven artifacts, which declared 142,355 direct dependencies, of which 71.60% were not up-to-date. We updated these dependencies and found that 11.58% of the dependency updates contain breaking changes that impact the client. We further analyzed these changes in the library which impact the client projects and examine if libraries have adhered to the semantic versioning scheme when introducing breaking changes in their releases. Our results show that changes in transitive dependencies were a major factor in introducing breaking changes during dependency updates and almost half of the detected client impacting breaking changes violate the semantic versioning scheme by introducing breaking changes in non-Major updates.
- Mahmoud Alfadel, Diego Elias Costa, and Emad Shihab. 2021. Empirical Analysis of Security Vulnerabilities in Python Packages. In International Conference on Software Analysis, Evolution and Reengineering (SANER ’21). IEEE, 446–457. https://doi.org/10.1109/SANER50967.2021.00048
Google Scholar
Cross Ref
- Gabriele Bavota, Gerardo Canfora, Massimiliano Di Penta, Rocco Oliveto, and Sebastiano Panichella. 2013. The Evolution of Project Inter-Dependencies in a Software Ecosystem: The Case of Apache. ICSM ’13. IEEE, 280–289. isbn:9780769549811 https://doi.org/10.1109/ICSM.2013.39
Google ScholarDigital Library
- Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. 2010. A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World. Commun. ACM, 53, 2 (2010), feb, 66–75. issn:0001-0782 https://doi.org/10.1145/1646353.1646374
Google ScholarDigital Library
- Aline Brito, Marco Tulio Valente, Laerte Xavier, and Andre Hora. 2020. You broke my code: understanding the motivations for breaking changes in APIs. Empirical Software Engineering, 25, 2 (2020), 1458–1492. https://doi.org/10.1007/s10664-019-09756-z
Google ScholarCross Ref
- Aline Brito, Laerte Xavier, Andre Hora, and Marco Tulio Valente. 2018. APIDiff: Detecting API breaking changes. In 25th International Conference on Software Analysis, Evolution and Reengineering (SANER ’18). IEEE, 507–511. https://doi.org/10.1109/SANER.2018.8330249
Google ScholarCross Ref
- Aline Brito, Laerte Xavier, Andre Hora, and Marco Tulio Valente. 2018. Why and how Java developers break APIs. In 25th International Conference on Software Analysis, Evolution and Reengineering (SANER ’18). 255–265. https://doi.org/10.1109/SANER.2018.8330214
Google ScholarCross Ref
- Eric Bruneton, Eugene Kuleshov, Andrei Loskutov, and Rémi Forax. 2022. ASM. https://asm.ow2.io/
Google Scholar
- Joel Cox, Eric Bouwers, Marko van Eekelen, and Joost Visser. 2015. Measuring Dependency Freshness in Software Systems. In International Conference on Mobile Software Engineering and Systems (MOBILESoft ’15). IEEE, 109–118. https://doi.org/10.1109/ICSE.2015.140
Google ScholarCross Ref
- Daniela S. Cruzes and Tore Dyba. 2011. Recommended Steps for Thematic Synthesis in Software Engineering. In 2011 International Symposium on Empirical Software Engineering and Measurement(ESEM). 275–284. https://doi.org/10.1109/ESEM.2011.36
Google ScholarDigital Library
- Joe Darcy. 2021. Kinds of Compatibility. https://wiki.openjdk.org/display/csr/Kinds+of+Compatibility
Google Scholar
- Alexandre Decan and Tom Mens. 2021. What Do Package Dependencies Tell Us about Semantic Versioning? IEEE Transactions on Software Engineering, 47, 6 (2021), 6, 1226–1240. issn:19393520 https://doi.org/10.1109/TSE.2019.2918315
Google ScholarCross Ref
- Jim des Rivières. 2017. Evolving Java-based APIs 2. https://wiki.eclipse.org/Evolving_Java-based_APIs_2
Google Scholar
- Jens Dietrich, Kamil Jezek, and Premek Brada. 2014. Broken promises: An empirical study into evolution problems in Java programs caused by library upgrades. In Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE 14). 64–73. https://doi.org/10.1109/CSMR-WCRE.2014.6747226
Google ScholarCross Ref
- Jens Dietrich, David Pearce, Jacob Stringer, Amjed Tahir, and Kelly Blincoe. 2019. Dependency Versioning in the Wild. In 16th International Conference on Mining Software Repositories (MSR ’19). 349–359. https://doi.org/10.1109/MSR.2019.00061
Google ScholarDigital Library
- Danny Dig and Ralph Johnson. 2006. How Do APIs Evolve? A Story of Refactoring: Research Articles. Journal of software maintenance and evolution: Research and Practice, 18, 2 (2006), 3, 83–107. issn:1532-060X https://doi.org/10.1002/smr.328
Google ScholarCross Ref
- Dino Distefano, Manuel Fähndrich, Francesco Logozzo, and Peter W. O’Hearn. 2019. Scaling Static Analyses at Facebook. Commun. ACM, 62, 8 (2019), jul, 62–70. issn:0001-0782 https://doi.org/10.1145/3338112
Google ScholarDigital Library
- Khaled El Emam. 1999. Benchmarking Kappa: Interrater agreement in software process assessments. Empirical Software Engineering, 4 (1999), 113–133.
Google ScholarDigital Library
- Darius Foo, Hendy Chua, Jason Yeo, Ming Yi Ang, and Asankhaya Sharma. 2018. Efficient Static Checking of Library Updates. In 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018). ACM, 791–796. isbn:9781450355735 https://doi.org/10.1145/3236024.3275535
Google ScholarDigital Library
- The Apache Software Foundation. 2023. Apache Maven Project. https://maven.apache.org/
Google Scholar
- James Gosling, Bill Joy, Guy Steele, Gilad Bracha, Alex Buckley, Daniel Smith, and Gavin Bierman. 2021. The Java language specification. Oracle America, Inc.
Google Scholar
- Nicolas Harrand, Amine Benelallam, César Soto-Valero, François Bettega, Olivier Barais, and Benoit Baudry. 2022. API beauty is in the eye of the clients: 2.2 million Maven dependencies reveal the spectrum of client–API usages. Journal of Systems and Software, 184 (2022), 111134. issn:0164-1212 https://doi.org/10.1016/j.jss.2021.111134
Google ScholarDigital Library
- Hao He, Runzhi He, Haiqiao Gu, and Minghui Zhou. 2021. A Large-Scale Empirical Study on Java Library Migrations: Prevalence, Trends, and Rationales. ESEC/FSE ’21. ACM, 478–490. isbn:9781450385626 https://doi.org/10.1145/3468264.3468571
Google ScholarDigital Library
- Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, Samuel Ou, and Kelly Blincoe. 2023. Replication Package for Understanding Breaking Changes in the Wild. https://doi.org/10.5281/zenodo.7978507
Google ScholarDigital Library
- Kamil Jezek, Jens Dietrich, and Premek Brada. 2015. How Java APIs Break - An Empirical Study. 65, C (2015), sep, 129–146. issn:0950-5849 https://doi.org/10.1016/j.infsof.2015.02.014
Google ScholarDigital Library
- Riivo Kikas, Georgios Gousios, Marlon Dumas, and Dietmar Pfahl. 2017. Structure and Evolution of Package Dependency Networks. In 14th International Conference on Mining Software Repositories (MSR ’17). IEEE, 102–112. isbn:9781538615447 https://doi.org/10.1109/MSR.2017.55
Google ScholarDigital Library
- Rediana Koçi, Xavier Franch, Petar Jovanovic, and Alberto Abelló. 2019. Classification of Changes in API Evolution. In 23rd International Enterprise Distributed Object Computing Conference (EDOC ’19). IEEE, 243–249. https://doi.org/10.1109/EDOC.2019.00037
Google ScholarCross Ref
- Raula Gaikovina Kula, Daniel M. German, Ali Ouni, Takashi Ishio, and Katsuro Inoue. 2018. Do Developers Update Their Library Dependencies? Empirical Software Engineering, 23, 1 (2018), 2, 384–417. issn:1382-3256 https://doi.org/10.1007/s10664-017-9521-5
Google ScholarDigital Library
- Anders Møller, Benjamin Barslev Nielsen, and Martin Toldam Torp. 2020. Detecting Locations in JavaScript Programs Affected by Breaking Library Changes. Proc. ACM Program. Lang., 4 (2020), 11, 1–25. https://doi.org/10.1145/3428255
Google ScholarDigital Library
- Anders Møller and Martin Toldam Torp. 2019. Model-Based Testing of Breaking Changes in Node.Js Libraries. In 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2019). ACM, 409–419. isbn:9781450355728 https://doi.org/10.1145/3338906.3338940
Google ScholarDigital Library
- Suhaib Mujahid, Rabe Abdalkareem, Emad Shihab, and Shane McIntosh. 2020. Using Others’ Tests to Identify Breaking Updates. In 17th International Conference on Mining Software Repositories (MSR ’20). ACM, 466–476. isbn:9781450375177 https://doi.org/10.1145/3379597.3387476
Google ScholarDigital Library
- Lina Ochoa, Thomas Degueule, and Jean-Rémy Falleri. 2022. BreakBot. In ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER ’22). ACM. https://doi.org/10.1145/3510455.3512783
Google ScholarDigital Library
- Lina Ochoa, Thomas Degueule, Jean-Rémy Falleri, and Jurgen Vinju. 2022. Breaking Bad? Semantic Versioning and Impact of Breaking Changes in Maven Central: An External and Differentiated Replication Study. Empirical Softw. Engg., 27, 3 (2022), may, 42 pages. issn:1382-3256 https://doi.org/10.1007/s10664-021-10052-y
Google ScholarDigital Library
- Fernando Rodriguez Olivera. 2022. MVN Repository: repository stats. https://mvnrepository.com/repos
Google Scholar
- Oracle. n.d.. Java Virtual Machine Specification: Chapter 5. Loading, Linking, and Initializing. https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-5.html
Google Scholar
- Cliodhna O’Connor and Helene Joffe. 2020. Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines. International Journal of Qualitative Methods, 160906919899220. https://doi.org/10.1177/1609406919899220
Google ScholarCross Ref
- Ivan Pashchenko, Henrik Plate, Serena Elisa Ponta, Antonino Sabetta, and Fabio Massacci. 2018. Vulnerable Open Source Dependencies: Counting Those That Matter. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM ’18). ACM. isbn:9781450358231 https://doi.org/10.1145/3239235.3268920
Google ScholarDigital Library
- Tom Preston-Werner. n.d. Semantic Versioning 2.0.0. https://semver.org/
Google Scholar
- Steven Raemaekers, Arie van Deursen, and Joost Visser. 2014. Semantic Versioning versus Breaking Changes: A Study of the Maven Repository. In 14th International Working Conference on Source Code Analysis and Manipulation (SCAM ’14). IEEE, 215–224. https://doi.org/10.1109/SCAM.2014.30
Google ScholarDigital Library
- S. Raemaekers, A. van Deursen, and J. Visser. 2017. Semantic versioning and impact of breaking changes in the Maven repository. Journal of Systems and Software, 129 (2017), 140–158. issn:0164-1212 https://doi.org/10.1016/j.jss.2016.04.008
Google ScholarDigital Library
- Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from Building Static Analysis Tools at Google. Commun. ACM, 61, 4 (2018), mar, 58–66. issn:0001-0782 https://doi.org/10.1145/3188720
Google ScholarDigital Library
- Pasquale Salza, Fabio Palomba, Dario Di Nucci, Cosmo D’Uva, Andrea De Lucia, and Filomena Ferrucci. 2018. Do Developers Update Third-Party Libraries in Mobile Apps? ICPC ’18. Association for Computing Machinery, New York, NY, USA. 255–265. isbn:9781450357142 https://doi.org/10.1145/3196321.3196341
Google ScholarDigital Library
- Danilo Silva and Marco Tulio Valente. 2017. RefDiff: Detecting Refactorings in Version Histories. In IEEE/ACM 14th International Conference on Mining Software Repositories (MSR ’17). 269–279. https://doi.org/10.1109/MSR.2017.14
Google ScholarDigital Library
- Inc Tidelift. 2022. Libraries.io - The Open Source Discovery Service. https://libraries.io/data
Google Scholar
- Ying Wang, Bihuan Chen, Kaifeng Huang, Bowen Shi, Congying Xu, Xin Peng, Yijian Wu, and Yang Liu. 2020. An Empirical Study of Usages, Updates and Risks of Third-Party Libraries in Java Projects. In International Conference on Software Maintenance and Evolution (ICSME ’20). IEEE, 35–45. https://doi.org/10.1109/ICSME46990.2020.00014
Google ScholarCross Ref
- Thomas H. Wonnacott and Ronald J. Wonnacott. 1991. Introductory Statistics.
Google Scholar
- Laerte Xavier, Aline Brito, Andre Hora, and Marco Tulio Valente. 2017. Historical and impact analysis of API breaking changes: A large-scale study. In 24th International Conference on Software Analysis, Evolution and Reengineering (SANER ’17). IEEE, 138–147. https://doi.org/10.1109/SANER.2017.7884616
Google ScholarCross Ref
- Zach. 2021. What is Inter-rater Reliability. https://www.statology.org/inter-rater-reliability/
Google Scholar
- Lyuye Zhang, Chengwei Liu, Zhengzi Xu, Sen Chen, Lingling Fan, Bihuan Chen, and Yang Liu. 2022. Has My Release Disobeyed Semantic Versioning? Static Detection Based on Semantic Differencing. In EEE/ACM International Conference on Automated Software Engineering (ASE ’22). ACM. isbn:9781450394758 https://doi.org/10.1145/3551349.3556956
Google ScholarDigital Library
Index Terms
- Understanding Breaking Changes in the Wild
Recommendations
I Depended on You and You Broke Me: An Empirical Study of Manifesting Breaking Changes in Client Packages
Complex software systems have a network of dependencies. Developers often configure package managers (e.g., npm) to automatically update dependencies with each publication of new releases containing bug fixes and new features. When a dependency release ...
Semantic versioning and impact of breaking changes in the Maven repository
Backward-incompatible interface changes are widespread in software libraries.Breaking changes have impact on client systems using these software libraries.Software developers do not follow proposed versioning guidelines. Systems that depend on third-...
Breaking bad? Semantic versioning and impact of breaking changes in Maven Central: An external and differentiated replication study
AbstractJust like any software, libraries evolve to incorporate new features, bug fixes, security patches, and refactorings. However, when a library evolves, it may break the contract previously established with its clients by introducing Breaking Changes ...
Comments