Abstract
A complete and detailed (full) Design Rationale Documentation (DRD) could support many software development activities, such as an impact analysis or a major redesign. However, this is typically too onerous for systematic industrial use as it is not cost effective to write, maintain, or read. The key idea investigated in this article is that DRD should be developed only to the extent required to support activities particularly difficult to execute or in need of significant improvement in a particular context. The aim of this article is to empirically investigate the customization of the DRD by documenting only the information items that will probably be required for executing an activity. This customization strategy relies on the hypothesis that the value of a specific DRD information item depends on its category (e.g., assumptions, related requirements, etc.) and on the activity it is meant to support. We investigate this hypothesis through two controlled experiments involving a total of 75 master students as experimental subjects. Results show that the value of a DRD information item significantly depends on its category and, within a given category, on the activity it supports. Furthermore, on average among activities, documenting only the information items that have been required at least half of the time (i.e., the information that will probably be required in the future) leads to a customized DRD containing about half the information items of a full documentation. We expect that such a significant reduction in DRD information should mitigate the effects of some inhibitors that currently prevent practitioners from documenting design decision rationale.
- Ambler, S. W. 2007. The tagri (they aren't gonna read it) principle of software development. http://www.agilemodeling.com/essays/tagri.htm.Google Scholar
- Ambler, S. W. and Jeffries, R. 2002. Agile Modeling: Effective Practices for Extreme Programming and the Unified Process. Wiley. Google ScholarDigital Library
- Arkley, P., Riddle, S., and Brookes, T. 2006. Tailoring traceability information to business needs. In Proceedings of the 14th IEEE International Requirements Engineering Conference (RE'06). IEEE Computer Society. Google ScholarDigital Library
- Basili, V. R. and Rombach, H. D. 1991. Support for comprehensive reuse. Softw. Engin. J. 6, 5. 303--316. Google ScholarDigital Library
- Basili, V. R., Shull, F., and Lanubile, F. 1999. Building knowledge through families of experiments. IEEE Trans. Softw. Engin. 25, 4. 456--473. Google ScholarDigital Library
- Biffl, S., Aurum, A., Bohem, B., Erdogmus, H., and Grünbacher, P. 2005. Value-Based Software Engineering. Springer. Google ScholarDigital Library
- Biffl, S., Aurum, A., Bohem, B., Erdogmus, H., and Grünbacher, P. 2006. Value-Based Software Engineering. Springer. Google ScholarDigital Library
- Boehm, B., Rombach, D., and Zelkowitz, M. 2010. Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili. Springer. Google ScholarDigital Library
- Boehm, B. W. 1981. Software Engineering Economics. Prentice Hall PTR. Google ScholarDigital Library
- Boehm, B. W. and Sullivan, K. J. 2000. Software economics: A roadmap. In Proceedings of the Conference on The Future of Software Engineering. ACM Press, New York. Google ScholarDigital Library
- Bosch, J. 2004. Software architecture: The next step. In Proceedings of the 1st European Workshop on Software Architecture (EWSA'04). Springer.Google ScholarCross Ref
- Bratthall, L., Johansson, E., and Regnell, B. 2000. Is a design rationale vital when predicting change impact? A controlled experiment on software architecture evolution. In Proceedings of the International Conference on Product Focused Software Process Improvement. Google ScholarDigital Library
- Burge, J. and Brown, D. 1998. Design rationale types and tools, ai in design group. http://web.cs.wpi.edu/Research/aidg/DR-Rpt98.html.Google Scholar
- Burge, J. E., Carroll, J. M., Mccall, R., and Mistrk, I. 2008. Rationale-Based Software Engineering. Springer. Google ScholarDigital Library
- Cantone, G., Colasanti, L., Abdulnabi, Z., Lomartire, A., and Calavaro, G. 2003. Evaluating checklist-based and use-case-driven reading techniques as applied to software analysis and design uml artifacts. In Experiences from ESERNET: Empirical Methods and Studies on Software Engineering.Google Scholar
- Capilla, R., Nava, F., and Dueñas, J. C. 2007. Modeling and documenting the evolution of architectural design decisions. In Proceedings of the 2nd Workshop on Sharing and Reusing Architectural Knowledge, (ICSE Workshops). IEEE. Google ScholarDigital Library
- Capilla, R., Nava, F., and Carrillo, C. 2008. Effort estimation in capturing architectural knowledge. In Proceedings of the 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE'08). Google ScholarDigital Library
- Cleland-Huang, J. 2006a. Just enough requirements traceability. In Proceedings of the 30th Annual International Computer Software and Applications Conference (COMPSAC'06). Vol. 1, IEEE Computer Society. Google ScholarDigital Library
- Cleland-Huang, J. 2006b. Requirements traceability - When and how does it deliver more than it costs? In Proceedings of the 14th IEEE International Requirements Engineering Conference (RE'06). IEEE Computer Society. Google ScholarDigital Library
- Clements, P., Mcgregor, J. D., and Cohen, S. G. 2005. The structured intuitive model for product line economics (simple). Tech. rep. CMU/SEI-2005-TR-003, Carnegie Mellon University.Google Scholar
- Conklin, J. and Begeman, L. M. 1988. gIBIS: A hypertext tool for exploratory policy discussion. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work. Google ScholarDigital Library
- Cox, J. 1997. The Concise Guide to Economics. Laissez Faire Books.Google Scholar
- De Boer, R. C. and Farenhorst, R. 2008. In search for architectural knowledge. In Proceedings of the 3rd Workshop on Sharing and Reusing Architectural Knowledge (SHARK'08), co-located with the 30th International Conference on Software Engineering (ICSE'08). Google ScholarDigital Library
- Dzidek, W., Arisholm, E., and Briand, L. 2008. A realistic empirical evaluation of the costs and benefits of uml in software maintenance. IEEE Trans. Softw. Engin. 34, 3. 407--432. Google ScholarDigital Library
- Egyed, A., Biffl, S., Heindl, M., and Grunbacher, P. 2005. Determining the cost-quality trade-off for automated software traceability. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering. ACM Press, New York. Google ScholarDigital Library
- Egyed, A., Grunbacher, P., Heindl, M., and Biffl, S. 2007. Value-based requirements traceability: Lessons learned. In Proceedings of the 15th IEEE International Requirements Engineering Conference (RE'07).Google Scholar
- Falessi, D., Ali Babar, M., Cantone, G., and Kruchten, P. 2010a. Applying empirical software engineering to software architecture: Challenges and lessons learned. Empirical Softw. Engin. 15, 3. 250--276. Google ScholarDigital Library
- Falessi, D. and Cantone, G. 2006. Exploring feasibility of software defects orthogonal classification. In Software and Data Technologies, Springer.Google Scholar
- Falessi, D., Cantone, G., and Becker, M. 2006. Documenting design decision rationale to improve individual and team design decision making: An experimental evaluation. In Proceedings of the 5th ACM/IEEE International Symposium on Empirical Software Engineering. Google ScholarDigital Library
- Falessi, D., Cantone, G., and Canfora, G. 2011a. Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Trans. Softw. Engin. 39, 1, 18--44. Google ScholarDigital Library
- Falessi, D., Cantone, G., and Grande, C. 2007. A comparison of structured analysis and object oriented analysis: An experimental study. In Proceedings of the International Conference on Software and Data Technologies.Google Scholar
- Falessi, D., Cantone, G., Kazman, R., and Kruchten, P. 2011b. Decision-making techniques for software architecture design: A comparative survey. ACM Comput. Surv. 43, 4. Google ScholarDigital Library
- Falessi, D., Cantone, G., Sarcià, S. A., Calavaro, G., Subiaco, P., and D'amore, C. 2010b. Peaceful coexistence: Agile developer perspectives on software architecture. IEEE Softw. 27, 2. Google ScholarDigital Library
- Farenhorst, R., Hoorn, J. F., Lago, P., and Van Vliet, H. 2009. What architects do and what they need to share knowledge. Tech. rep. IR-IMSE-003, VU University Amsterdam.Google Scholar
- Favaro, J., Favaro, K., and Favaro, P. 1998. Value based software reuse investment. Ann. Softw. Enin. 5, 5--52. Google ScholarDigital Library
- Fowler, M. 2005. The new methodology. http://martinfowler.com/articles/newMethodology.html.Google Scholar
- Gorton, I. 2006. Essential Software Architecture. Springer. Google ScholarDigital Library
- Greenacre, M. 2007. Correspondence Analysis in Practice. Chapman and Hall/CRC.Google Scholar
- Hannay, J. and Jørgensen, M. 2008. The role of deliberate artificial design elements in software engineering experiments. IEEE Trans. Softw. Engin. 34, 2, 242--259. Google ScholarDigital Library
- Heindl, M. and Biffl, S. 2005. A case study on value-based requirements tracing. In Proceedings of the 10th European Software Engineering Conference. ACM Press, New York. Google ScholarDigital Library
- Holt, R., Boehm-Davis, D., and Shultz, A. 1987. Mental representations of programs for student and professional programmers. In Proceedings of the Empirical Studies of Programmers, 2nd Workshop. Ablex Publishing. Google ScholarDigital Library
- Hosmer, D. and Lemeshow, S. 2000. Applied Logistic Regression. Wiley-Interscience.Google Scholar
- Host, M., Regnell, B., and Wohlin, C. 2000. Using students as subjects: A comparative study of students and professionals in lead-time impact assessment. Empirical Softw. Engin. J. 5, 3, 201--214. Google ScholarDigital Library
- Jansen, A. and Bosch, J. 2005. Software architecture as a set of architectural design decisions. In Proceedings of the 5th Working IEEE/IFIP Conference on Software Architecture (WICSA'05). IEEE Computer Society. Google ScholarDigital Library
- Jansen, A., Bosch, J., and Avgeriou, P. 2008. Documenting after the fact: Recovering architectural design decisions. J. Syst. Softw. 81, 4. 536--557. Google ScholarDigital Library
- Jedlitschka, A., Ciolkowski, M., and Pfahl, D. 2008. Reporting experiments in software engineering. In Guide to Advanced Empirical Software Engineering, Springer.Google Scholar
- John, I., Knodel, J., Lehner, T., and Muthig, D. 2006. A practical guide to product line scoping. In Proceedings of the 10th International Software Product Line Conference. Google ScholarDigital Library
- Juristo, N. and Moreno, A. M. 2006. Basics of Software Engineering Experimentation. Springer. Google ScholarDigital Library
- Karsenty, L. 1996. An empirical evaluation of design rationale documents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Common Ground. ACM Press, New York. Google ScholarDigital Library
- Kruchten, P. 2003. The Rational Unified Process: An Introduction 3rd Ed. Addison-Wesley Professional. Google ScholarDigital Library
- Kruchten, P., Capilla, R., and Duenas, J. 2009. The decision view's role in software architecture practice. IEEE Softw. 26, 2. 36--42. Google ScholarDigital Library
- Kruchten, P., Lago, P., and Van Vliet, H. 2006. Building up and reasoning about architectural knowledge. In Proceedings of the 2nd International Conference on the Quality of Software Architectures. Lecture Notes in Computer Science, vol. 4214, Springer, 43--58. Google ScholarDigital Library
- Lago, P., Avgeriou, P., Capilla, R., and Kruchten, P. 2008. Wishes and boundaries for a software architecture knowledge community. In Proceedings of the 7th Working IEEE/IFIP Conference on Software Architecture (WICSA'08). Google ScholarDigital Library
- Lee, J. 1997. Design rationale systems: Understanding the issues. IEEE Expert: Intell. Syst. Their Appl. 12, 3. 78--85. Google ScholarDigital Library
- Lee, L. and Kruchten, P. 2007. Capturing software architectural design decisions. In Proceedings of the Canadian Conference on Electrical and Computer Engineering (CCECE'07).Google Scholar
- Maclean, A., Young, R. M., Bellotti, V. M. E., and Moran, T. P. 1996. Questions, options, and criteria: Elements of design space analysis. In Design Rationale: Concepts, Techniques, and Use. Lawrence Erlbaum Associates, 53--105. Google ScholarDigital Library
- Nehmer, J., Becker, M., Karshmer, A., and Lamm, R. 2006. Living assistance systems: An ambient intelligence approach. In Proceedings of the 28th International Conference on Software Engineering. ACM Press, New York. Google ScholarDigital Library
- Oates, B. J. 2003. Widening the scope of evidence gathering in software engineering. In Proceedings of the 11th Annual International Workshop on Software Technology and Engineering Practice. IEEE Computer Society. Google ScholarDigital Library
- Ragin, C. C. 1989. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. University of California Press.Google Scholar
- Shull, F., Carver, J., Vegas, S., and Juristo, N. 2008. The role of replications in empirical software engineering. J. Empirical Softw. Engin. 13, 2. 211--218. Google ScholarDigital Library
- Shull, F., Seaman, C., and Zelkowitz, M. 2006. Victor R. Basili's contributions to software quality. IEEE Softw. 23, 1. Google ScholarDigital Library
- Shum, B. and Hammond, N. 1994. Argumentation-based design rationale: What use at what cost? Int. J. Hum.-Comput. Stud. 40, 4, 603--652. Google ScholarDigital Library
- Sjøberg, D., Anda, B., Arisholm, E., Tore, D., Jørgensen, M., Karahasanovic, A., Koren, E., and Marek, V. 2002. Conducting realistic experiments in software engineering. In Proceedings of the International Symposium on Empirical Software Engineering. IEEE Computer Society. Google ScholarDigital Library
- Sjoberg, D., Arisholm, E., and Jorgensen, M. 2001. Conducting experiments on software evolution. In Proceedings of the 4th International Workshop on Principles of Software Evolution. ACM Press, New York. Google ScholarDigital Library
- Svahnberg, M., Aurum, A., and Wohlin, C. 2008. Using students as subjects - An empirical evaluation. In Proceedings of the 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. ACM Press, New York. Google ScholarDigital Library
- Tang, A. 2011. Software designers are you biased? In Proceedings of the 6th Workshop on Sharing and Reusing Architectural Knowledge (SHARK'11). Google ScholarDigital Library
- Tang, A., Ali Babar, M., Gorton, I., and Han, J. 2007. A survey of architecture design rationale. J. Syst. Softw. 79, 12. 1792--1804. Google ScholarDigital Library
- Tyree, J. and Akerman, A. 2005. Architecture decisions: Demystifying architecture. IEEE Softw. 22, 2, 19--27. Google ScholarDigital Library
- Van Der Ven, J. S., Jansen, A., Avgeriou, P., and Hammer, D. K. 2006. Using architectural decisions. In Proceedings of the 2nd International Conference on the Quality of Software Architectures.Google Scholar
- Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. 2000. Experimentation in Software Engineering: An Introduction, Springer. Google ScholarDigital Library
- Zimmermann, O., Gschwind, T., Küster, J., Leymann, F., and Schuster, N. 2007. Reusable architectural decision models for enterprise application development. In Proceedings of the 3rd Quality of Software Architectures International Conference on Software Architectures, Components, and Applications (QoSA'07). Lecture Notes in Computer Science, vol. 4880. Springer, 15--32. Google ScholarDigital Library
Index Terms
- The value of design rationale information
Recommendations
Design rationale capture in software architecture: what has to be captured?
WCOP '14: Proceedings of the 19th international doctoral symposium on Components and architectureMore than thirty years of research on the topic of design rationale (DR) have passed. However, as of today there are just some exceptional cases where DR are applied in industrial practise. Researchers analysed them for its nature, its structure, its ...
A value-based approach for documenting design decisions rationale: a replicated experiment
SHARK '08: Proceedings of the 3rd international workshop on Sharing and reusing architectural knowledgeThe explicit documentation of the rationale of design decisions is a practice generally encouraged but rarely implemented in industry because of a variety of inhibitors. Known methods for Design Decisions Rationale Documentation (DDRD) are aimed to ...
Value-Based Design Decision Rationale Documentation: Principles and Empirical Feasibility Study
WICSA '08: Proceedings of the Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008)The explicit documentation of the rationale of design decisions is a practice generally encouraged, but rarely implemented in industry because of a variety of inhibitors. Methods proposed in the past for Design Decisions Rationale Documentation (DDRD) ...
Comments