Exploring the benefits of the combination of a software architecture analysis and a usability evaluation of a mobile application

https://doi.org/10.1016/j.jss.2010.03.079Get rights and content

Abstract

Designing easy to use mobile applications is a difficult task. In order to optimize the development of a usable mobile application, it is necessary to consider the mobile usage context for the design and the evaluation of the user–system interaction of a mobile application. In our research we designed a method that aligns the inspection method “Software ArchitecTure analysis of Usability Requirements realizatioN” SATURN and a mobile usability evaluation in the form of a user test. We propose to use mobile context factors and thus requirements as a common basis for both inspection and user test. After conducting both analysis and user test, the results described as usability problems are mapped and discussed. The mobile context factors identified define and describe the usage context of a mobile application. We exemplify and apply our approach in a case study. This allows us to show how our method can be used to identify more usability problems than with each method separately. Additionally, we could confirm the validity and identified the severity of usability problems found by both methods. Our work presents how a combination of both methods allows to address usability issues in a more holistic way. We argue that the increased quantity and quality of results can lead to a reduction of the number of iterations required in early stages of an iterative software development process.

Introduction

Mobile applications that run on mobile devices and are used in a mobile environment (Kjeldskov and Graham, 2003) become more and more important. A recent United Nations report shows that more than half of the global population use a mobile phone. Despite the economic downturn, the prognosis given in the report is positive due to the “pervasive nature of information and communication technologies” (International Telecommunications Union (ITU), 2009). Although technological progress has been made – mobile devices are enhanced with more computing power, connectivity, and interaction capability – many limitations and challenges still remain, for example regarding usability.

Usability is defined by ISO 9241-11 as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (International Standard Organization, 1998). It is therefore a quality attribute of the usage of a software product. A product has no intrinsic usability, it has a capability to be used in a particular context (Bevan, 2001). “Design for usability” incorporates product characteristics that benefit users in a particular usage context.

Considering the usage context of a mobile application, in literature also found as the mobile usage context, limitations and challenges become apparent. Parameters like a changing and distracting environment, mobile devices’ small screens and non-traditional input capabilities as well as task limitations (Hassanein and Head, 2003) increase the complexity of designing usable mobile applications. The mobile usage context is analyzed and described based on mobile context factors (see Section 3) which describe the characteristics of the particular usage context. In the research area of mobile human computer interaction (mobile HCI) the mobile usage context is addressed, and usability evaluations (UE) in the form of user tests are applied to find problems appearing during the interaction between a human and an application's interface. But there is a lack in mobile usability research related to the technology beyond the interface (Coursaris and Kim, 2007), the technology that realizes interaction. Research shows that software architecture plays an important role in designing usable systems by separating the user interface (UI) from the core functionality and by addressing user interactions that demand architectural support (Clements et al., 2002, Bass et al., 2003, Folmer and Bosch, 2004, Juristo et al., 2007b).

The software architecture (SA) defines the scope and constraints for the quality of a software product. Kruchten defines SA as “[...] the set of significant decisions about the organization of a software system, the selection of the structural elements and their interfaces by which the system is composed, together with their behavior as specified in the collaborations among those elements, the composition of these structural and behavioral elements into progressively larger subsystems, and the architectural style that guides this organization—these elements and their interfaces, their collaborations, and their composition” (Kruchten, 1998). According to this definition, software architecture has an effect on the behavior of a system. The behavior of a system towards users and vice versa is also known as user–system interaction. Interaction and user interface influence usability. Thus usability is a quality attribute that does not only depend on the interface and its design but also on the software architecture (Bass et al., 2003). It is closely related to other quality attributes such as performance, reliability, security, trust, and others that equally need to be regarded when designing an application and its interface.

Our approach combines the methods of software architecture analysis and usability evaluation to exploit the benefits of both methods. We designed a method that analyzes the software architecture of a mobile application for the design of usability and validates its results with a usability evaluation. The usability evaluation finds usage problems of a software application by observing and monitoring users to identify user performance issues and usage problems. It can use the SA analysis to locate problems identified. We use the method “Software ArchitecTure analysis of Usability Requirements realizatioN” SATURN (see Section 4.1) that analyzes the SA of an application. It finds architectural risks that are based on how usability requirements are realized in the software architecture. Further, we conduct a laboratory user test that evaluates a software. Afterwards we combine, compare, and evaluate the results of both methods.

The common basis for both evaluations is the usage context. Focusing on mobile applications, we identify mobile context factors (MCF) representing the parameters that have an effect on the interface and the interaction design and thus the usability of a mobile application.

Contributions of this paper are: (1) in the method, the results of a SA Analysis and a UE (see Section 4) are compared. It shows how SA analysis and a UE method can be combined and how results relate to each other. We evaluate this method in a case study in which we analyze and evaluate a mobile application. (2) We define the parameters determining the usage context of a mobile application (see Section 3). (3) In the case study, we identified specific usability problems and their architectural reasons during a SA analysis. We were able to confirm these problems and define their severity through the usability evaluation. By this the importance of considering usability in the SA is highlighted (see Section 5). (4) By combining the evaluation and the analysis, we identified more usability problems than with each method separately. We could confirm the validity of problems by using the combination of both methods. This enhances our results by the possibility of verifying the severeness of the particular problems found.

Compared to the methods currently used (see Section 2), which do not combine software architecture and usability evaluation methods, we can regard more results earlier in an iterative software development process. Thus, we argue that the improved quality and quantity of our evaluation results lead to a reduction of the number of iterations required. We conclude that the combination of a SA analysis and a usability evaluation improves the current state of the art of evaluation methods applied in software architecture as well as in usability evaluations.

This paper is organized as follows: In Section 2 related work regarding the relationship between usability and software architecture, software architecture analysis methods, and usability evaluation methods is covered. Afterwards (in Section 3), we define mobile context factors and the method. The evaluation setup, evaluation methods used and how their results can be compared will be shown in Section 4. In Section 5, the evaluation of the software “Shake Your Mac” (SYM) is described and the results of the SATURN method applied to SYM and the usability evaluation are compared. We conclude the paper discussing the results and giving an outlook for future work in Section 6.

Section snippets

Software architecture and usability

Regarding the architectural support of user interfaces and user–system interaction, two approaches are followed. The first approach separates the user interface (UI) from the core of the system. Examples illustrating this strategy are architecture patterns such as Model View Controller and Presentation Abstraction Control (Buschmann et al., 1996). Yet, by the separation of the UI and the system alone, not all necessary aspects of user–system interaction can be realized.

The second approach

Mobile context factors

The usage context of a mobile application is defined by mobile context factors (MCFs). It shows the complex dependencies of a mobile interactive system that software and usability engineers are confronted with when designing an interactive mobile application.

For users being mobile “[...] means that user location, physical, and social context may change, that physical resources cannot be relied upon, and that physical world navigation may have to be accomplished” (Ballard, 2007). From a

Combining the methods

The method proposed (see Fig. 1) in this paper is based on the combination of a software architecture analysis and a usability evaluation in the form of a user test.

Usability problems occur because of missing or wrong definitions or violations of usability requirements which depend on the usage context.

To be able to combine both methods the evaluation parameters of both methods are based on the same requirements, especially identified through the MCFs defined in Section 3.

While the usability

Case study—Shake Your Mac evaluation

The case study “Shake Your Mac” evaluates a system used for interacting within a mobile environment. In order to evaluate and improve the system in an iterative process we apply our approach by combining a usability evaluation with a software architecture analysis. Finally we discuss the results and show how it is possible to improve a software engineering process by considering findings that would not have been found by conducting one evaluation method only.

The SYM system was elicited for

Conclusion and future work

In this paper we present a method that aligns a software architecture (SA) analysis and a usability evaluation (UE), and thus addresses usability holistically. We exemplified how these methods can be combined and how their results relate to each other. In particular, we proposed to use requirements as a common basis for both the SA analysis (an inspection method) and a usability evaluation UE (a user test), as well as to map and discuss results described as usability problems. Also we

Acknowledgements

We would like to thank the reviewers for their comments. They were a valuable input which helped us to improve our paper.

Bettina Biel is research assistant and PhD student at the Applied Telematics/e-Business group at the University of Leipzig. Her research interests are mobile interface and interaction design, software architecture, design patterns, and trust in ubiquitous computing. In her thesis, she designs a method for the analysis of architecture-level support of mobile usability.

References (52)

  • F. Buschmann et al.
    (1996)
  • Carter, S., Mankoff, J., 2004. Challenges for ubicomp evaluation. Tech. Rep. UCB/CSD-04-1331, EECS Department,...
  • A. Chang et al.

    Sensorial interfaces

  • P. Clements et al.

    Evaluating Software Architectures

    (2002)
  • A. Cockburn

    Writing Effective Use Cases

    (2001)
  • A. Cooper

    About Face: The Essentials of User Interface Design

    (1995)
  • A. Cooper et al.

    About Face 3.0: The Essentials of Interaction Design

    (2007)
  • C.K. Coursaris et al.

    A qualitative review of empirical mobile usability studies

  • C.K. Coursaris et al.

    A research agenda for mobile usability

  • C. Coutrix et al.

    Balancing physical and digital properties in mixed objects

  • E. Folmer et al.

    A framework for capturing the relationship between usability and software architecture

    Software Process: Improvement and Practice

    (2003)
  • Folmer, E., van Gurp, J., Bosch, J., 2004. Software Architecture Analysis of Usability. In: Proceedings of the Joint...
  • Grill, T., 2009. Designing interactions in mixed interaction environments. Ph.D. Thesis. Johannes Kepler University...
  • T. Grill et al.

    A pattern approach to mobile interaction design

    Information Technology

    (2009)
  • J. Grudin et al.

    Personas, participatory design and product development: an infrastructure for engagement

  • J. Haikara

    Usability in agile software development: extending the interaction design process with personas approach

    Agile Processes in Software Engineering and Extreme Programming

    (2007)
  • Cited by (0)

    Bettina Biel is research assistant and PhD student at the Applied Telematics/e-Business group at the University of Leipzig. Her research interests are mobile interface and interaction design, software architecture, design patterns, and trust in ubiquitous computing. In her thesis, she designs a method for the analysis of architecture-level support of mobile usability.

    Thomas Grill is a researcher in the in the HCI & Usability Unit of the ICT&S Center at the University of Salzburg. He received his doctoral degree in business informatics from the Johannes Kepler University Linz in the area of interaction design with a focus on alternative interaction styles and techniques. His research includes tools and methods in the area of human computer interaction with a special focus on ubiquitous computing appliances. He is also working on alternative interaction methods with the goal of improving interaction and performance by using external devices and interaction methods like motion tracking and eye-tracking. Current research also tries to enhance usability evaluation methods through the measurement and interpretation of context information in order to gain more advanced information about the user and the environment he resides in.

    Prof. Dr. Volker Gruhn is leading the Software Engineering Group at University of Essen and is a member of the board of Paluno - The Research Institute for Software Technology at the University of Duisburg-Essen. His research interests are software processes, processes in general, software architecture, usability as well as component-based and generative software development of web and mobile systems. He was the general and program chair of the Software Engineering SE 2006, and program co-chair for the International Conference on Software Engineering ICSE 2008. He is author and co-author of more than 120 national and international publications.

    View full text