An analysis of attributes that impact information technology audit quality: A study of IT and financial audit practitioners

Abstract

The importance of information technology (IT) auditing has grown with increased reliance on IT for business operations and new regulations regarding the assurance of IT for these operations. Prior work on IT and financial auditing has suggested several general frameworks that may affect IT audit quality; however, the prior work has not provided measurable constructs nor has it considered whether these proposed constructs are the same or different. Building on prior work that has proposed frameworks of IT audit quality, we identify and evaluate potential constructs suggested by these frameworks as well as financial auditing literature. We develop a survey tool and ask IT and financial accounting practitioners to assess the impact of these items on IT audit quality. A factor analysis is used to refine the set of IT audit quality factors identified, and we are able to provide insight into the prioritized impact of each factor on IT audit quality. In comparison to prior research, we find that additional factors are significant for IT audit quality and that the relative importance of the factors for IT audit quality differs for IT versus financial auditors.

Introduction

The purpose of this study is to analyze attributes identified in prior research that are thought to impact the quality of the information technology (IT) audit process. Prior research has identified several attributes that are argued to impact IT audit quality, both positively and negatively. These attributes include various characteristics of the process or system being audited, the procedures or techniques used to perform the audit, traits of the audit personnel themselves, organizational and environmental conditions, as well as many others. The natural extension of this work is to develop a structural model of IT audit quality and its antecedents; however, to date, there has not been an examination of these attributes to identify the underlying measurable components of IT audit quality. Additionally, the financial audit literature suggests other audit quality attributes which may also provide insight into IT audit quality. Therefore, the purpose of this research is to rationalize the potential constructs and develop potential instruments that allow for measurement of these constructs.

Recent research has identified the importance of IT audit to organizations and has called for additional research in this area (Weidenmier and Ramamoorti, 2006, Curtis et al., 2009). This attention to IT audit has been driven by two primary reasons, 1) increased spending and dependence on IT for business operations, and 2) new legislation and professional requirements related to the audit of these operations. Demand for IT services to support key business activities has driven the level of global IT spending to over $3.6 trillion for 2011 (Gartner Group, 2011). This reliance on IT and the investment it entails require an increased level of assurance that these systems deliver what they promise. IT audits are widely used internally to examine the operations, effectiveness, controls, and security of critical systems to identify opportunities for improvement and areas of weakness.

Discussions by the Public Company Accounting Oversight Board (PCAOB) Standing Advisory Group (SAG) regarding auditor's knowledge of information systems (IS) have emphasized the importance of information technology (IT) in general, and IT auditing specifically, to the external financial audits of public companies. In addition to the United States' Sarbanes–Oxley Act (SOX), a plethora of other laws, regulations, and standards have all necessitated additional IT assurance related to information security and privacy. These include, but are not limited to, the Health Insurance Portability and Accountability Act (HIPAA), Canada's Personal Information Protection and Electronic Documents Act (PIPEDA), the Payment Card Industry Data Security Standard (PCI DSS) and the European Union's Data Privacy Directive. In addition to these regulatory compliance requirements, auditors of public companies in the U.S. are bound by auditing standards that require adequate IT expertise to assess controls; for an insightful discussion of these standards please see Curtis et al. (2009).

IT audits may serve various objectives and multiple parties within an organization, and therefore there may be different definitions of IT audit quality. These definitions may include ideas such as impact or effectiveness, completeness as related to different standards, and efficiency or cost. One purpose of IT audits is to provide management with assurance that a system or automated process is meeting its objectives. Specifically, the focus may be on managements' control responsibilities over computer-based information assets and processes. In these cases, specific standards developed by groups such as ISO, PCAOB, or AICPA may assist in defining certain IT audit quality. For internal operational audits, the focus is usually on performance, i.e. cost reduction or improvements in productivity. Therefore the focus may be on the overall impact of the audit findings and the cost of performing the audit. Regardless, to perform an IT audit efficiently and effectively, firms must make appropriate decisions regarding the scope, resources (e.g., personnel or computer-automated audit tools), tasks or activities to be performed, methods, techniques, and other “inputs” to the IT audit process. Management's decisions regarding specific resources to deploy for a specific IT audit should attempt to maximize the overall audit quality and minimize the cost as related to their specific IT audit objectives. This also requires a consideration of other attributes that might impact the performance and outcome of the IT audit, but over which they have no or little control. These attributes might include the availability of key auditee personnel, the infrastructure or architecture on which a system is running, or the organizational structure of a business unit being audited.

One objective of this research is to assist these decision-makers by providing additional information regarding the relative importance of the attributes previously identified in the research and the underlying components that these attributes comprise. We believe that these could be used to help guide audit management's planning by making tradeoffs among the attributes. We also analyze whether differences exist with regard to perceptions of these components between key constituents.

We believe that there are four primary contributions attributable to this research activity. First, we identify and rationalize specific attributes associated with the IT audit quality domain, including attributes from the general audit quality domain that are relevant to IT audit quality. Specifically, we begin with the general framework proposed by Merhout and Havelka (2008) and expand this work by integrating other attributes from the IT audit domain and then relevant items identified in the financial audit quality domain.

Second, we evaluate the relative importance of these attributes to IT audit quality. By evaluating the individual attributes, we can compare our results to the work of prior researchers that have done similar work in the general audit quality domain to determine if IT audits have different requirements or peculiarities. Our results indicate that there is a different priority in the skills and knowledge required for IT audit quality as compared to attributes identified in prior financial audit quality literature.

Third, we perform a factor analysis to determine the underlying components of the attributes identified. In general, factor analysis allows us to reduce a large set of items into a smaller set of composite components that are more easily manageable. The long-term goal of this stream of research related to IT audit quality is to develop a testable model of constructs that may impact IT audit quality. The research presented in this paper is a critical, initial step in rationalizing and developing measurable constructs related to IT audit quality. This paper is similar to Carcello et al. (1992) who focus on identifying the critical factors related to financial audit quality.

Fourth, we analyze differences in perceived importance of the IT audit quality factors generated by the factor analysis between distinct groups involved in the IT audit process. We identify differences in perceptions between IT and financial auditing participants. This may aid in developing a better understanding of expectations and perceptions of the IT audit process and quality for these groups, and, hopefully allow management to improve audit planning and execution so all parties perceive similar (higher) quality.

The remainder of this paper proceeds as follows. Section 2 provides a review of the existing IT and financial audit quality literature and identifies potential IT audit quality attributes. Section 3 describes the research method and approach for analyzing the IT audit quality attributes. In Section 4, we describe the results of the analysis including explanation and interpretation of supported factors, perceived importance of each factor, and differences in evaluation of the factors between IT audit participants. Section 5 discusses specific observations from our data, limitations of our study, and implications for future research and for practitioners.