Nicolas Palix
Gaël Thomas
Gilles Muller
ABSTRACT
In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other directories. This result inspired a number of development and research efforts on improving the reliability of driver code. Today Linux is used in a much wider range of environments, provides a much wider range of services, and has adopted a new development and release model. What has been the impact of these changes on code quality? Are drivers still a major problem?
To answer these questions, we have transported the experiments of Chou et al. to Linux versions 2.6.0 to 2.6.33, released between late 2003 and early 2010. We find that Linux has more than doubled in size during this period, but that the number of faults per line of code has been decreasing. And, even though drivers still accounts for a large part of the kernel code and contains the most faults, its fault rate is now below that of other directories, such as arch (HAL) and fs (file systems). These results can guide further development and research efforts. To enable others to continually update these results as Linux evolves, we define our experimental protocol and make our checkers and results available in a public archive.
- A. Aiken, S. Bugrara, I. Dillig, T. Dillig, B. Hackett, and P. Hawkins. An overview of the Saturn project. In Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE'07, pages 43--48, San Diego, CA, June 2007. Google Scholar
Digital Library
- J. Brunel, D. Doligez, R. R. Hansen, J. Lawall, and G. Muller. A foundation for flow-based program matching using temporal logic and model checking. In The 36th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 114--126, Savannah, GA, USA, Jan. 2009. Google ScholarDigital Library
- How is the department of homeland security involved?, 2009. http://scan.coverity.com/faq.html#how-department-homeland-security-in%volved.Google Scholar
- Checkpatch. http://www.codemonkey.org.uk/projects/checkpatch/.Google Scholar
- A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An empirical study of operating systems errors. In Proceedings of the 18th ACM Symposium on Operating System Principles, pages 73--88, Banff, Canada, Oct. 2001. Google ScholarDigital Library
- Comedi: Linux Control and Mesurement Device Interface. http://www.comedi.org/.Google Scholar
- J. Corbet. The age of kernel code in various subsystems, Feb. 2010. http://lwn.net/Articles/374622/.Google Scholar
- J. Corbet. How old is our kernel?, Feb. 2010. http://lwn.net/Articles/374574/.Google Scholar
- Static source code analysis, static analysis, software quality tools by Coverity Inc. http://www.coverity.com/, 2008.Google Scholar
- A. Depoutovitch and M. Stumm. Otherworld -- giving applications a chance to survive OS kernel crashes. In ACM EuroSys, pages 181--194, Paris, France, Apr. 2010. Google ScholarDigital Library
- Fedora project, 2010. http://fedoraproject.org/.Google Scholar
- J. N. Herder, H. Bos, B. Gras, P. Homburg, and A. S. Tanenbaum. Fault isolation for device drivers. In 2009 IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pages 33--42, Estoril, Portugal, June 2009.Google ScholarCross Ref
- IEEE std 982.2--1988 IEEE guide for the use of IEEE standard dictionary of measures to produce reliable software, 1988.Google Scholar
- A. Israeli and D. G. Feitelson. The Linux kernel as a case study in software evolution. Journal of Systems and Software, 83(3):485--501, 2010. Google ScholarDigital Library
- J. L. Lawall, J. Brunel, R. R. Hansen, H. Stuart, G. Muller, and N. Palix. WYSIWIB: A declarative approach to finding protocols and bugs in Linux code. In The 39th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, pages 43--52, Estoril, Portugal, June 2009.Google ScholarCross Ref
- Z. Li and Y. Zhou. PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code. In Proceedings of the 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 306--315, Lisbon, Portugal, Sept. 2005. Google ScholarDigital Library
- Lkml: The Linux kernel mailing list. http://lkml.org/.Google Scholar
- S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 329--339, Seattle, WA, USA, Mar. 2008. Google ScholarDigital Library
- T. J. McCabe. A complexity measure. IEEE Transactions on Software Engineering, 2(4):308--320, July 1976. Google ScholarDigital Library
- P. E. McKenney and J. Walpole. Introducing technology into the Linux kernel: a case study. ACM SIGOPS Operating Systems Review, 42(5):4--17, 2008. Google ScholarDigital Library
- J. C. Munson and S. G. Elbaum. Code churn: A measure for estimating the impact of code change. In International Conference Software Maintenance (ICSM), pages 24--31, 1998. Google ScholarDigital Library
- N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In 27th International Conference on Software Engineering (ICSE), pages 284--292, St. Louis, Missouri, USA, May 2005. Google ScholarDigital Library
- P. Oman and J. Hagemeister. Construction and testing of polynomials predicting software maintainability. Journal of Systems and Software, 24(3):251--266, 1994. Google ScholarDigital Library
- Y. Padioleau, J. Lawall, R. R. Hansen, and G. Muller. Documenting and automating collateral evolutions in Linux device drivers. In EuroSys 2008, pages 247--260, Glasgow, Scotland, Mar. 2008. Google ScholarDigital Library
- N. Palix, J. Lawall, and G. Muller. Tracking code patterns over multiple software versions with Herodotos. In Proc. of the ACM International Conference on Aspect-Oriented Software Development, AOSD'10, pages 169--180, Rennes and Saint Malo, France, Mar. 2010. Google ScholarDigital Library
- N. Palix, J. L. Lawall, and G. Muller. Herodotos: A tool to expose bugs' lives. Research report RR-6984, INRIA, July 2009.Google Scholar
- N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Database of Faults in Linux: Ten Years Later, Aug. 2010. http://hal.inria.fr/docs/00/50/92/56/ANNEX/10years.sql.pg_dump.Google Scholar
- N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Faults in Linux: Ten years later. Research report RR-7357, INRIA, Aug. 2010.Google Scholar
- N. Palix, S. Saha, G. Thomas, C. Calvès, J. Lawall, and G. Muller. Website of Faults in Linux: Ten Years Later, Dec. 2010. http://faultlinux.lip6.fr/.Google Scholar
- E. S. Raymond. The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. O'Reilly & Associates, Inc., 2001. Google ScholarDigital Library
- D. Searls. Sparse, Linus & the Lunatics, Nov. 2004. Available at http://www.linuxjournal.com/article/7272.Google Scholar
- X. Song, H. Chen, and B. Zang. Why software hangs and what can be done with it. In International Conference on Dependable Systems and Networks (DSN 2010), Chicago, IL, USA, June 2010.Google ScholarCross Ref
- Sparse. https://sparse.wiki.kernel.org/index.php/Main_Page.Google Scholar
- B. Spencer. Local kernel exploit in /dev/net/tun. http://grsecurity.net/ spender/cheddar_bay.tgz.Google Scholar
- M. M. Swift, M. Annamalai, B. N. Bershad, and H. M. Levy. Recovering device drivers. ACM Transactions on Computer Systems, 24(4):333--360, 2006. Google ScholarDigital Library
- The Kernel Janitors. Smatch, the source matcher, 2010. Available at http://smatch.sourceforge.net.Google Scholar
- Ubuntu, 2010. http://www.ubuntu.com/.Google Scholar
- D. Wheeler. Flawfinder home page. Web page: http://www.dwheeler.com/flawfinder/, Oct. 2006.Google Scholar
- D. A. Wheeler. SLOCCount. http://www.dwheeler.com/sloccount/.Google Scholar
Index Terms
- Faults in linux: ten years later
Recommendations
Faults in Linux 2.6
In August 2011, Linux entered its third decade. Ten years before, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up ...
Faults in linux: ten years later
ASPLOS '11In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other ...
Faults in linux: ten years later
ASPLOS '11In 2001, Chou et al. published a study of faults found by applying a static analyzer to Linux versions 1.0 through 2.4.1. A major result of their work was that the drivers directory contained up to 7 times more of certain kinds of faults than other ...
Comments