Performance evaluation of the recommendation mechanism of information security risk identification

Abstract

In recent decades, information security has become crucial for protecting the benefits of a business operation. Many organizations perform information security risk management in order to analyze their weaknesses, and enforce the security of the business processes. However, identifying the threat–vulnerability pairs for each information asset during the processes of risk assessment is not easy and time-consuming for the risk assessor. Furthermore, if the identified risk diverges from the real situation, the organization may put emphasis on the unnecessary controls to prevent the non-existing risk. In order to resolve the problem mentioned above, we utilize the data mining approach to discover the relationship between assets and threat–vulnerability pairs. In this paper, we propose a risk recommendation mechanism for assisting user in identifying threats and vulnerabilities. In addition, we also implement a risk assessment system to collect the historical selection records and measure the elapsed time. The result shows that with the assistance of risk recommendations, the mean elapsed time is shorter than with the traditional method by more than 21%. The experimental results show that the risk recommendation system can improve both the performance of efficiency and accuracy of risk identification.

Introduction

More and more organizations rely on information technology to assist them in achieving their business goals such as faster service response or better quality. However, focusing on ease of use in terms of system configuration and operation makes systems more vulnerable and easily compromised. This is why information security is of paramount importance to organizations. A systematic approach for information security risk management is necessary to help identify information security requirements and to create an effective management system. Risk is the effect of uncertainty on objectives, and information security risk is often expressed in terms of a combination of the consequences of an information security event and the associated likelihood of occurrence [1]. The object for assessment also called information asset, which means anything that has value to organization. It is noting that information asset of an information system consists of more than hardware and software [1]. In this paper, we classify the information asset into five categories: hardware, software, people, information and service. Risk assessment, both the process and associated techniques, offers an analytical and structured walk-through of the organization’s security state [2]. Risk identification is an important step in risk assessment, to determine what could cause a potential loss, and to gain insight into how and why the loss might happen. Thus, if a corporation expects to perform risk assessment successfully, finding the appropriate threat–vulnerability pair of each asset is a crucial step. However, in the process of identifying threat–vulnerability pairs, it is difficult for the risk assessor, especially one who lacks information security competence, to recognize the feasible combinations.

Without the support of a recommendation system, a risk assessor may encounter at least three challenges: First, in spite of the threat and vulnerability list being provided as a candidate list for risk assessors, it is still time-consuming to choose the appropriate one from more than a hundred combinations. Second, the threat–vulnerability pairs may be irrational if the root cause is not considered discreetly. For example, a physical server appliance may have some vulnerability due to the lack of physical protection. Theoretically, environmental damage and physical breakage are reasonable threats. However, a mistake may be made when people choose another irrational threat such as “insufficient software testing”. Third, not all the users have the ability to find the security issue for the information asset, and may choose non-existing risks. Non-existing threat–vulnerability pairs may make organizations spend unnecessary time and money to prevent a risk that may not happen, which may lead the manager to neglect the real weaknesses, or invest in improper security measures.

There are a number of information risk assessment approaches that have been proposed. These methods of identifying threats and vulnerabilities are based on the International Organization for Standardization (ISO) stands, such as ISO 31000:2009 [3], ISO/IEC 27001:2013 [4] and ISO/IEC 27005:2011 [1]. Stølen presented a risk assessment model called CORAS [5], which uses a threat diagram and structured brainstorming to analyze risks. These methods are always performed with expert guidance and may take too much time to complete. Some researchers have identified threats and vulnerabilities according to the security requirements, such as OCTAVE [6], which only addresses the security requirements of information asset onlys but lacks comprehensive consideration. Another mechanism uses business processes to complete the risk assessment [7]. However, in their work, it is hard for common users to determine each asset’s risk on their lifecycle. In addition, other researchers [8] identify risk by building a security ontology. However, it is complicated for users who lack of security knowledge and also impossible to build on their own. There are still some researchers who recommend threats or vulnerabilities for users, but this only suitable for specific domains, such as cloud computing [9].

Due to the deficiencies mentioned above, we propose a recommendation approach for risk identification iterations to resolve the problem. In this paper, the asset category is classified after the asset identification step. By the use of a data mining technique, the threat–vulnerability pairs for each asset category were identified by the predictive aprori algorithm and provided as a recommendation list. The risk assessor can choose the appropriate pairs that correspond to the real encountered problems from the recommendation list. The main contribution of our proposed approach is to improve the efficiency and accuracy of identifying the threat–vulnerability pairs. In order to evaluate the performance of the efficiency improvement, we first invited information experts to evaluate the accuracy of the threat–vulnerability pairs on the recommendation list. In addition, we designed a risk assessment system that can provide the recommendations of threat–vulnerability pairs for the risk assessor, and can measure the elapsed time of the risk assessor’s selections.

The remainder of the paper is organized as follows: Section 2 describes relevant research on risk assessment and the problems in the past. Section 3 presents our research model, which recommends threat–vulnerability pairs for different categories of asset. Section 4 contains the experimental design and results. Conclusions and future directions are given in Section 5.