https://orcid.org/0000-0001-7668-8524
Figures
Abstract
Since the pandemic organizations have been required to build agility to manage risks, stakeholder engagement, improve capabilities and maturity levels to deliver on strategy. Not only is there a requirement to improve performance, a focus on employee engagement and increased use of technology have surfaced as important factors to remain competitive in the new world. Consideration of the strategic horizon, strategic foresight and support structures is required to manage critical factors for the formulation, execution and transformation of strategy. Strategic foresight and Artificial Intelligence modelling are ways to predict an organizations future agility and potential through modelling of attributes, characteristics, practices, support structures, maturity levels and other aspects of future change. The application of this can support the development of required new competencies, skills and capabilities, use of tools and develop a culture of adaptation to improve engagement and performance to successfully deliver on strategy. In this paper we apply an Artificial Intelligence model to predict an organizations level of future agility that can be used to proactively make changes to support improving the level of agility. We also explore the barriers and benefits of improved organizational agility. The research data was collected from 44 respondents in public and private Australian industry sectors. These research findings together with findings from previous studies identify practices and characteristics that contribute to organizational agility for success. This paper contributes to the ongoing discourse of these principles, practices, attributes and characteristics that will help overcome some of the barriers for organizations with limited resources to build a framework and culture of agility to deliver on strategy in a changing world.
Citation: Shafiabady N, Hadjinicolaou N, Din FU, Bhandari B, Wu RMX, Vakilian J (2023) Using Artificial Intelligence (AI) to predict organizational agility. PLoS ONE 18(5): e0283066. https://doi.org/10.1371/journal.pone.0283066
Editor: Sathishkumar V. E., Jeonbuk National University, REPUBLIC OF KOREA
Received: October 27, 2022; Accepted: March 1, 2023; Published: May 10, 2023
Copyright: © 2023 Shafiabady et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Sharing the data is not allowed because it potentially contains identifying information about the organisations where participants were associated to and sharing the data would violate the confidentiality agreement signed with the participants. These restrictions were imposed by the Human Research Ethics Committee of Torrens University Australia. Contact information for data access inquiries is as follows: Human Research Ethics Committee of Torrens University Australia (Ethics@torrens.edu.au).
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
The concepts of organizational agility have become increasingly important since the pandemic as organisations require new ways to improve employee engagement, build capabilities and organizational performance to support competitiveness and deliver on their strategy. Other pressures include the increased use and merging of technology, use of Artificial Intelligence, greater investment in technology, greater customer needs and levels of customer satisfaction demanded. Market competition, legislative, political and economic pressures [1,2] have also played a role in the need for increased organizational and strategic agility. Organizational agility is defined as “the capability to quickly sense and adapt to external and internal changes to deliver relevant results in a productive and cost-effective manner” [3]. It is closely linked to the execution of an organisation’s strategy. Benefits include the acceleration of organizational learning to meet the pace of rapid environmental change through flexibility in assembling resources, knowledge, processes and capabilities. Agility also enables organisations to sustain their competitive advantage by increasing responsiveness when making business decisions [4,5]. New policies, process and procedures together with changes to organizational structures, systems and management practices are also required [6], which also support improving levels of maturity in these areas.
In this research, characteristics and practices of organisational agility are examined together with organizational size and type, industry sector, support structures, maturity levels and other aspects of future change are applied into an artificial intelligence model to predict an organizations future agility[6,7]. Barriers and benefits of improved organizational agility are also explored through this data collection to support building an organizational knowledge and framework to support improving agility and associated attributes of maturity [8,9].
Artificial Intelligence (AI) is applicable to a vast variety of interdisciplinary business systems, with the main objective of improving standards, reducing the dependable product quality control methods and creating new possibilities to meet customer satisfaction while maintaining low operational costs and robust business management [10]. For example, AI has been implemented to improve supply chain management [11], smart factory-based operational efficiency and warehouse management [12]. AI has helped improve a range of other domains as well such as health and medicine, cancer care [13] among other avenues such as pandemic management [14,15]. The most effective AI methodologies are Machine Learning (ML) methods e.g., Support Vector Machine (SVM), Decision Trees (DT), and K-Nearest Neighbours (KNN), as detailed in the later sections, which have been utilised in this research article to predict organizational agility. Applications of AI and ML methods include commercial use of SVM [16–19], energy load management [20], finance and portfolio management, inventory management [21,22], sales forecasting, profit maximisation, and sales growth [23], among other domains e.g., dynamic cognitive networking approach [18], modelling and identifying frailty [24,25]. In order to maximise the potential of revenue generation, predicting Organizational Agility (OA) is an important factor. Hence this paper focuses on highlighting the use of AI to predict organizational agility together with barriers and benefits of improved organizational agility by incorporating a comprehensive data collection, prediction and analysis methodology.
A base collection of data was collected from 44 respondents in private and public Australian sectors. The following attributes of an organization and characteristics of organizational agility (1–17) were identified by PMI [3,26–30] and explored within the model together with components of maturity:
1. Flexible and adaptable, 2. Open communications, 3. Transparency in decision making, 4. Rapid decision making, 5. Decentralised decision making, 6. Open to change, 7. Self-aware and honest, 8. Customer orientated, 9. Focused on talent development, 10. Committed to Agility, 11. Empowered team members, 12. Action based, 13. Agility recognised as a team competence, 14. Catalyst Leadership, 15. Effective methods of rapid knowledge transfer, 16. Continuous learning from experience, 17. Clear guidelines for tailoring standardised processes to suit the size and type of project, 18. Effective environment scanning, 19. Appetite for risk, 20. Active Governance
Building on a previous study, further characteristics (18–20) were identified by Hadjinicolaou et al [6], which included effective environmental scanning, having an appetite for risk and active governance [6]. In addition to the organisational characteristics’ successful implementation of organizational agility generally requires a number of ongoing practices. A PMI report (p. 6) [29] identifies the six foundational practices of organizational agility including:
1. Responding quickly to strategic opportunities, 2. Shortening production/review/decision cycles, 3. Eliminating organizational silos, 4. Aligning new business capabilities to strategy, 5. Integrating the voice of the customer and 6. Focusing on change management
These practices and others have been taken into consideration in this study where respondents were asked about the following 27 practices that have been found by PMI [30] that support organizational agility:
1. Focus on change management, 2. Application of iterative project management concepts to portfolio management, 3. Use of program management practices, 4. Focus on resource management, 5. Focus on risk management, 6. Focus on lean practices and value, 7. Project task simplification, 8. Quick response to strategic opportunities, 9. Shorter production/review/decision cycles, 10. Elimination of organisational silos, 11. Use of Project Portfolio Management practices, 12. Integrates voice of the customer, 13. Use of iterative or incremental project management practices, 14. Interdisciplinary project teams, 15. Contingency planning, 16. Leverages technology, 17. Empirical (real-time) project management, 18. Matrix management, 19. Use of models, pilots and simulation, 20. Question assumptions, 21. Assessment of disruptive technological or other changes, 22. Increased environmental scanning, 23. Growth by acquisition, 24. Focus on innovation, 25. Standardisation of project management practices, 26. Progressively elaborated and active use of Project Business Cases, 27. Use of Business Cases for post implementation reviews and benefits management.
Some of these practices are more complex than others and required at the individual, team and organizational level. Consideration is also required on how an organisations strategy is formulated, the planning horizon, support structures required for execution and other strategic practices. Changes to an organizations culture rely heavily on strong leadership and ongoing commitment. While factors, characteristics and practices have been studied in the existing literature, little has been covered to apply Artificial Intelligence to predict organizational agility. The questionnaire survey explored respondent roles, respondent industry sector, organizational size, characteristics and practices of organizational agility, aspects of future change, maturity levels of practices, benefits and barriers to improve organizational agility and aspects of maturity to support organizational agility.
The objectives and primary research questions for this study were:
- What are the leading characteristics and foundational practices to support organisational agility?
- How can Artificial Intelligence be used to predict organizational agility?
Data was analysed and an artificial intelligence model was developed and using this data the organizations agility was predicted. We begin by examining the literature that is relevant to our study. In Section 2, we describe our methods, including data, measures and research methodology. The research results and discussion are presented in Section 3 our overall conclusions are presented in Section 4.
2. Research methodology
The data that was collected for this research is sourced from 44 industry practitioners from differing industry sectors in Australia (with 42% in private sector organizations, 17% public sector, 22% in services organizations, 11% Universities and 8% not for profit and others) using an online questionnaire survey with a semi-structured questionnaire. Definite (yes/no) and concrete answers were managed by the use of closed questions. On the other hand, any further information or opinions from the respondents that did not appear in the closed questions could be added to ‘Others (please specify).’ Most fields in the data comprised of categorical variables. Our analysis provides original evidence that the process of data analysis followed the four steps modified from Creswell and Plano [4] for channel analysis based on 1) Preparing the data for analysis, 2) Exploring the data, 3) Analysing the collected data, and 4) Representing the data analysis.
The questionnaire survey asked explored respondents for their role within the organisations, country, the number of employees within the organisation, the organisational type, respondent industry sector, whether the organisation had a PMO, organizational size, agile characteristics within organisation, agile and practices of the organizational agility, aspects of future change, level of maturity levels of practices within the organisation, changes that occurred within the last two years, external factors that changed, perceived benefits and barriers to improve organizational agility and aspects of maturity to support organizational agility. Responses are covered in the results and discussion section of the paper.
2.1 Data collection
There have been a diverse range of organisations types and sizes in the sample size covered in the results and discussion section 2.1.2 and 2.1.3 with challenges for organisational agility greater in larger organisations in the public sector.
Greater effort is required in larger organisations to improve organisational agility.
2.1.1 Respondent roles.
The results are based on the primary roles of the respondents (refer Fig 1) were Project Manager (19%), Team Member (16%), Manager (13%), Executive General Manager (13%) and business trainers or lecturer roles within their organisation (19%).
2.1.2 Industry sector.
The predominate industry sector was education (15%) followed by Professional Services, Hospitality and Tourism (13%) (refer Fig 2).
2.1.3 Organisational size.
In Fig 3, a significant portion (33%) of respondents were from organisations with a size in the 300 to 2000 range with 17% less than 19 and from 100 to 299 employees (refer Fig 3).
The state-of-the-art Artificial Intelligence (AI) methods have been incorporated in this research to model and predict the organizational agility. The AI methods utilised in this paper are discussed in the following section.
2.2 Artificial Intelligence (AI) methods and data analysis
Artificial Intelligence has been used to solve different problems in different industries [31]. Machine learning is an evolving field of computational science that aims to imitate intelligence by incorporating learning ability and adaptability as per the environment [32]. It falls under the umbrella of Artificial Intelligence. With the emergence of big data technologies and high-performance computing, machine learning has created new opportunities for data-intensive studies in a variety of multi-disciplinary technological disciplines [33]. Machine learning generally uses two different applied techniques: unsupervised learning and supervised learning. Unsupervised Learning utilises underlying internal structures and patterns in the data to analyse and cluster unlabelled datasets, for example, there are a number of standard clustering techniques available i.e., k-means, Gaussian mixture models, hidden Markov models, fuzzy C-means clustering, hierarchical clustering, self-organizing maps etc. In contrast, Supervised Learning uses labelled datasets to train algorithms to classify data or predict outcomes accurately e.g., via classification and regression techniques to construct machine learning models that train a model on known input and output data to predict future outcomes. There exists a range of algorithms for performing classification including Support Vector Machines (SVMs), Decision Trees (DT), k-Nearest Neighbours (KNN), Naive Bayes, Neural Networks and Logistic Regression. This paper utilises a hybrid approach comparing three effective machine learning algorithms: SVM, DT and KNN to predict organizational agility and includes managing risks, stakeholder engagement and delivering strategic outcomes, which can support the development of required new competencies, apply new skills and develop a culture of adaptation to improve performance and successfully deliver on strategy.
Data classification is a technique that is used to organize data into categories/classes that also involves tagging the data to make it easily searchable and trackable in order to make it is easy to retrieve, sort and store for future use i.e., for making predictions based of factual trends. Support Vector Machine (SVM) is a strong classification method in ML techniques that incorporate maximisation i.e., (support) of the separating margin (i.e., vector) and is commonly applied in classification and nonlinear function estimation [34]. SVM has a vast applicability in research related to text and data mining, e.g., in order to identify effective marketing tactics, where very little client information is usually superseded with the preferences of a broader population. Similarly, credit rating analysis and power price forecasts are among many other corporate applications where SVM have been used as a successful methodology [35]. SVM Hyperplane generates a decision border that distinguishes between the two data classes. Depending on which side of the hyperplane a data point falls, it links to that respective class. The goal of SVM Hyperplane algorithms for binary classification problems is to locate a hyperplane for each data class that is both distant from and proximate to the data points of the other class. SVM hyperplane algorithms perform better and have lesser computing cost when compared to the traditional SVM. However, when the dataset includes considerable noise, SVM Hyperplane does not perform very well, which calls for a hybrid approach in ML algorithms to make accurate predictions [36].
Similarly, just like SVM, k-Nearest Neighbours (KNN) is a simple but effective method for classification, which also has successfully been implemented in real-time applications. The scalability of the KNN methods to large-scale datasets makes it more adaptable for a vast range of applications [37]. The KNN has been applied and several different domains including the prediction of financial straits in commercial entities, with the goal of limiting societal losses. In some research problems, combining-classifiers system provides high performance, and KNN along with SVM has been used in recent research literature as an outperforming combining-classifiers system, especially for predicting business failures [24].
Other than SVM and KNN, another outperforming classification method is Decision Tree (DT) [38], which is commonly utilised to design prediction algorithms for a target variable or to create classification systems based on multiple covariates. By splitting the large datasets into training and validation datasets, DT can easily deal with huge, intricate datasets without imposing a sophisticated parametric structure, which makes it an effective classification technique [37]. Decision Tree has proven to be a successful methodology, especially when it comes to deciding whether or not to invest in business i.e., venture capital firms, who most of time have limitation of available information and time. Decision tree algorithm provides detailed analysis which can assist investors while performing initial screening for the suitable projects [35].
A classical machine learning approach for data classification is Naïve Bayes [39], which is usually considered the preferred methodology for binary or categorical data. Naïve Bayes is a simple machine learning methodology and doesn’t require huge amounts of training data and can handle continuous as well as discrete data. A range of different studies have utilised Naïve Bayes methodology to make real-time predictions as it is highly scalable with the number of predictors and data points [32,40]. However, some recent studies have highlighted the limitations of Naïve Bayes, especially in cases that there are insufficient occurrences of some data class labels where certain attributes have similar values [39].
Another machine learning method is called gradient boosting machine (GBM), which is used for data classification and for regression problems. GBM [41] is based on the hunch of prior models coupled with the best feasible training data model to minimise the overall prediction error. Some of the benefits of GBM is being faster in training speed and providing higher accuracy. GBM has some limitations as well—for example, it will continue improving to minimize all errors, which makes it computationally expensive, time-consuming and memory exhaustive [41].
One of the frequently used supervised machine learning algorithms is called random forest (RF), which is used for data classification. On various samples, RF constructs decision trees and uses their average for classification and majority vote for regression [42]. RF offers quite a good accuracy among the categorization machine learning techniques. One of its advantages is its ability to handle large amounts of data with hundreds of different variables, with automatic balancing mechanisms where there are imbalanced classes within the dataset [31].
Logistic Regression [31,43] is another machine learning technique that is widely used to conduct supervised learning to predict different phenomena. Logistic Regression is easier to set up and train than other ML techniques. Several studies have suggested implementing Logistic Regression because of its efficiency especially when different data outcomes exist in a linearly separable manner. Some recent studies outline issues of Logistic Regression [31,43], especially when it fails to predict a continuous outcome, and its tendency to assume linearity between different variables. In some cases, when the data sample is small, the results generated by Logistic Regression may not be accurate [32,40].
Considering the benefits and applicability of these ML algorithms, in this paper, we have used an analytical method to generate synthetic data from the actual responses provided by the survey’s respondents.
3. Results and discussion
3.1 Characteristics of organisational agility
The ability to adapt quickly in response to changes to sustain competitive advantage is a critical factor for success [3]. Organizational agility (OA) is the ability to adapt and accommodate changes required to respond with the required actions and requires the development of a number of characteristics and practices.
The following characteristics of organizational agility (1–17) were identified by PMI [3] and explored within this study:
1. Flexible and adaptable, 2. Open communications, 3. Transparency in decision making, 4. Rapid decision making, 5. Decentralised decision making, 6. Open to change, 7. Self-aware and honest, 8. Customer orientated, 9. Focused on talent development, 10. Committed to Agility, 11. Empowered team members, 12. Action based, 13. Agility recognised as a team competence, 14. Catalyst Leadership, 15. Effective methods of rapid knowledge transfer, 16. Continuous learning from experience, 17. Clear guidelines for tailoring standardised processes to suit the size and type of project, 18. Effective environment scanning, 19. Appetite for risk, 20. Active Governance
Further characteristics (18–20) were identified by Hadjinicolaou et al [6] which included having an appetite for risk, effective environmental screening and active governance [6]. The most important characteristics of organizational agility as identified by PMI [26] were:
Flexibility and adaptability, 2. Open communication, 3. Openness to change, 4. Empowered team members, 5. Experiential learning, 6. Rapid decision making, 7. A strong customer focus
The results of this study to support the objective 1 of this study found the leading top eight characteristics of organizational agility were:
1. Open communications, 2. Flexible and adaptable, 3. Transparency in decision making, 4. Empowered team members, 5. Openness to change, 6. Committed to Agility, 7. Continuous learning from experience, 8. Self-aware and honest
3.2 Practices of organisational agility
Improving organizational agility requires a number of ongoing practices including continuous communication, collaboration, engagement and providing support within organisations.
A PMI’s report (p. 6) [27] identifies the six foundational practices of organizational agility including
1. Responding quickly to strategic opportunities, 2. Shortening production/review/decision cycles, 3. Eliminating organizational silos, 4. Aligning new business capabilities to strategy, 5. Integrating the voice of the customer and 6. Focusing on change management.
In this study respondents were asked about the following 27 practices that have been found by PMI [28] that support organizational agility:
1. Focus on change management, 2. Application of iterative project management concepts to portfolio management, 3. Use of program management practices, 4. Focus on resource management, 5. Focus on risk management, 6. Focus on lean practices and value, 7. Project task simplification, 8. Quick response to strategic opportunities, 9. Shorter production/review/decision cycles, 10. Elimination of organisational silos, 11. Use of Project Portfolio Management practices, 12. Integrates voice of the customer, 13. Use of iterative or incremental project management practices, 14. Interdisciplinary project teams, 15. Contingency planning, 16. Leverages technology, 17, Empirical (real-time) project management, 18. Matrix management, 19. Use of models, pilots and simulation, 20. Question assumptions, 21. Assessment of disruptive technological or other changes, 22. Increased environmental scanning, 23. Growth by acquisition, 24. Focus on innovation, 25. Standardisation of project management practices, 26. Progressively elaborated and active use of Project Business Cases, 27. Use of Business Cases for post implementation reviews and benefits management
The results of this study to support objective 1 of this study found the leading top eight practices needed for organizational agility were:
1. Quick response to strategic opportunities, 2. Elimination of organisational silos, 2. Focus on change management , 3. Shorter production/review/decision cycles, 4. Integrate voice of the customer, 5. Interdisciplinary project teams, 5. Leverages technology, 6. Assessment of disruptive technological or other changes
3.3 Predicting agility through artificial intelligence
The data collection for this research included a comprehensive inclusivity from 44 different industry practitioners from a range of industrial sectors in Australia (including private sector organizations, public sector, services organizations, Universities and not for profit organizations among others). Nevertheless, considering human bias and limitation in providing real-time data responses, however, it was still not enough to train Artificial Intelligence (AI) or Machine Learning (ML) models. Having high-quality, precise though comprehensive, and large dataset is important to train a machine learning algorithm effectively, as without it even the most effective ML model may fail. Hence, it is vital to have a comprehensive dataset to provide accurate and precise predictions.
Organizational success depends on its capacity to mould according to market dynamics in order to maintain a competitive edge. Organizational Agility (OA) is the capacity to swiftly adapt to changes and carry out necessary activities to keep up the growth pace. It entails a variety of traits and behaviours for example, rapid and decentralised decision making; self-aware, honest, and customer-oriented talent development; active, competitive and agile team members; catalyst leadership and continuous learning from past experience. All these traits are measurable (as detailed in Section 2) from real-time responses sought from the selected/targeted audience; however, comprehensive and consistent information is important to formulate trends and eventually predict the most probable outcome to remain vigilant and competitive in the market. Hence, this research gathers required data, analyses the necessary data parameters, uses synthetic data extension, and eventually applies machine learning models to predict the characteristics and practices that are required to adopt in order to maintain organizational agility.
A recent article in The Economist [44] has shown that data is a new and expensive asset. Companies and researchers try to gather a large volume of quality data for data analytics to provide insights and trends. Machine Learning (ML) techniques rely on extensive training and validation datasets [40]. ML maps the input data to output data, for which enough data is needed to capture the relationships between input and output features. Even with the knowledge of the importance of data, data analysts sometimes encounter limited data. The difficulty of data collection, time and privacy, and legal restrictions on data might be reasons for freely collecting data. In such cases, techniques such as data augmentation or synthetic data can assist in generating additional data. Data augmentations are generally used in images with minor transformations such as zooms, flips, cropping, shifts, blur, rotation, translation etc. In comparison, data synthesis is a technique to generate artificial data that mimics observations to train ML models when actual data is difficult or expensive to get [45].
Synthetic Data Vault (SDV) Python package [46] was applied to the existing data to generate additional synthetic data in this study. SDV is a popular synthetic data generator and validation framework using multivariate cumulative distribution functions or Generative Adversarial Networks (GANs). In machine learning (ML) paradigm, the GANs are utilised to increase accuracy of data points, as they are based on two concurrent neural networks, which compete with one another to generate more accurate predictions. The data was first organised according to the scenarios (scenario-1, scenario-2). Each scenario comprises of three categories of outputs with respect to the degree of organisational agility (1, 2, or 3). Two hundred synthetic data points were created by initiating the Gaussian Copula model and fitting it with the data. To test the goodness of fit of the synthetic data, evaluate (metrics = [KSTest]) method with metrics Inverted Kolmogorov-Smirnov D statistic (KSTest) from SDV class was used to compare the real dataset with the sample dataset.
The KSTest score is normalised between 0 and 1 with the target to maximise the score. From the result above, the generated samples are good and suitable for the purpose of this study.
Scenario-1 Results: Hadjinicolaou et al [6] explored diverse characteristics of organizational agility and how diverse combinations existed within organizations at different levels of strategic management maturity. The dataset utilised in this study, as detailed in Sections 1 and 2, is based on 21 factors related to organizational agility and project portfolio management, which have been categorised into three distinct data groups (i.e., Group 1, Group 2, and Group 3) by utilising Canonical analysis after a thorough synthetic data review. The findings, [6] reveal a considerable positive link between organizational agility and SPM maturity within the groups studied.
In order to provide a comprehensive analysis and to increase the readability, the experimental process has been categorised into two distinct scenarios as detailed above in section 3.3 and both of these scenarios have been processed through effective Machine Learning (ML) techniques: Support Vector Machine (SVM), Decision Tree (DT), k-Nearest Neighbours (KNN), Gradient Boosting Machine (GBM), Random Forest (RF), Naïve Bayes (NB) and Logistic Regression (LR). In Scenario 1, only the most relatable 92 responses are utilised including organizational type, organizational size, industry sectors, organizational support structures, level of internal and external change, organizational characteristics, traits, and practices of Organizational Agility (OA). The experimental results reflect the efficiency of the aforementioned ML techniques SVM, DT, KNN, GBM, RF, NB and LR as shown in Table 1, which also presents the group-wise accuracy percentages in each respective group (e.g., Group 1, Group 2, and Group 3).
The overall high test accuracy percentages confirm the effectiveness of the ML algorithms utilised in this study, which reflects that RF provides the best possible overall outcome with the accuracy of 97.674%, that is slightly better than SVM, GBM and KNN. Nonetheless, the results vary in different groups, however, there is no test accuracy that is less than 81.818%.
To demonstrate the performance of the ML models in more details, the related test confusion matrices are depicted in Table 2. A confusion matrix provides a comparison of the actual target values with the predicted values to illustrate the performance of machine learning models. The confusion matrices used for the ML models employed in this study as highlighted in Table 2, show the numbers of the correct and incorrect decisions made in each group.
Furthermore, test models’ sensitivity, specificity and F1 scores for the three groups have been reported in Table 3 below.
The results show that RF has had the highest overall sensitivity, specificity and F1 score and KNN, GBM and SVM have had scores which are relatively comparable with RF.
The results of these test models yield a substantial output as depicted in Fig 4. A closer look at Fig 4 explains that in most of the cases the model has predicted the values quite accurately, where red circles show the model’s test output and blue dots represent the desired test output. As it is depicted in Fig 4, there are just a few cases where the test outcome is different from the desired values (such as (9,2), (3,2), (17,2) and (18,3)).
The overall accuracy percentage for SVM (as presented in Table 3) is also depicted in Fig 5A, which reflects the performance of the test models to suffice the experiments.
The accuracy percentages in each group are demonstrated in Fig 5B. Colour distribution distinguishes between the test mistakes and the correct predictions for the respective groups e.g., the bold red segment reflects the test mistakes in Group1, and the light green segment shows the correct decisions in the same group. Similarly, the small purple segment in the second inner circle reflects the test mistakes, and the bigger dark green segment shows the correct predictions for Group2 and the same goes for Group3.
Decision Tree (DT): For the test scenario as detailed above, Decision Tree (DT) method has also been applied and results have been generated as per the following Test Confusion Matrix highlighted in Table 4. Fig 6 represents the results of the overall test outcomes using DT, which reflects that in most of the cases the model has predicted the values quite accurately. Red circles in Fig 6 show the model’s test outputs and blue dots represent the desired test outputs. It’s just a few cases the developed model has made a mistake in identifying the correct group.
The overall accuracy percentage for Decision Tree (DT) (as presented in Table 1) is depicted in Fig 7A which reflects the performance of the test models.
The accuracy percentages in each group are demonstrated in Fig 7B. Colour distribution distinguishes between the test mistakes and the correct predictions for the respective groups e.g., the bold red segment reflects the test mistakes in Group1, and the light green segment shows the correct decisions in the same group. Similarly, the small purple segment in the second inner circle reflects the test mistakes, and the bigger dark green segment shows the correct predictions for Group2 and the same goes for Group3.
k-Nearest Neighbours (KNN): For the test scenario as detailed above, k-Nearest Neighbours (KNN) along with the Euclidean distance have been applied. The Test Confusion Matrix is given in Table 1. Fig 8 represents the results of the overall test outcome applying KNN, which reflects that in most of the cases the model has predicted the values quite accurately. As mentioned above, red circles show the model’s test outputs and blue dots represent the desired test outputs. It is just a few cases the model has made mistakes, e.g., points (7,1) and (18,2).
Furthermore, the overall accuracy percentage for KNN (as presented in Table 1) is depicted in Fig 9A that reflects the good performance of the test model.
The accuracy percentages in each group are demonstrated in Fig 9B. Colour distribution distinguishes between the test mistakes and the correct predictions for the respective groups e.g., the bold red segment reflects the test mistakes in Group1, and the light green segment shows the correct decisions in the same group. Similarly, the small purple segment in the second inner circle reflects the test mistakes, and the bigger dark green segment shows the correct predictions for Group2 and the same goes for Group3.
The GBM test outcomes for Group1, Group2 and Group3 have been demonstrated in Fig 10 below.
As Fig 10 shows, there have been only 7 mistakes and the rest of the outcomes have been categorised correctly.
Fig 11A shows the GBM’s overall test accuracy percentage whereas the test accuracy percentages within each group for GBM have been demonstrated in Fig 11B.
The test outcomes for RF are shown in Fig 12. There have been only 3 test mistakes and the rest of the decisions have for the test dataset have been made correctly.
The overall accuracy percentage for RF and the accuracies for each groups are demonstrated in Fig 13A and 13B respectively. RF has had a very good performance compared to the other methods applied in the dataset for Scenario1.
Fig 14 outlines the test outcomes produced by NB. There have been 8 mistakes and the rest of the decisions have been made correctly.
The test accuracy for each group and the overall test accuracy percentage for NB have been demonstrated in Fig 15. It is understandable that NB hasn’t been able to provide the results as good as RF or GBM.
LR’s test performance is shown in Fig 16. LR has made more test mistakes in compare to the other methods applied on the Scenario1 dataset.
Fig 17A and 17B show the overall accuracy and the accuracy percentage in each group related to LR for Scenario1.
Scenario-2 Results: In Scenario 2, all the 142 data responses are utilised including all the identified characteristics, traits and practices of Organizational Agility (OA). SVM, DT, KNN, GBM, RF, NB and LR have been applied to Scenario-2 as well and the simulation results reflect the efficiency of the aforementioned ML techniques as shown in Table 5. In addition, Table 5 also presents the accuracy percentages in each respective group (e.g., Group 1, Group 2, and Group 3).
The overall 91+ test accuracy percentages confirm the effectiveness of the ML algorithms utilised in this study. KNN provides the best possible overall outcome with the accuracy of 99.224%, that is better than SVM, RF and other methods in this scenario.
Confusion matrices are used to evaluate the performance of the ML models since they provide a clear comparison between the actual target values with the predicted values. The applied ML models’ confusion matrices are shown in Table 6:
Furthermore, test models’ sensitivity, specificity and F1 scores for the three groups have been reported in Table 7 below.
The results of these test models are depicted in Fig 18. The results in Fig 18 explain that in most of the cases the model has predicted the values quite accurately, where red circles show the model’s test output and blue dots represent the desired test output.
The overall accuracy percentage for SVM (as presented in Table 5) is shown in Fig 19A which reflects the performance of the test models.
The accuracy percentages in each group are demonstrated in Fig 19B. Colour distribution distinguishes between the test mistakes and the correct predictions for the respective groups e.g., the bold red segment reflects the test mistakes in Group1, and the light green segment shows the correct decisions in the same group. Similarly, the small purple segment in the second inner circle reflects the test mistakes, and the bigger dark green segment shows the correct predictions for Group2 and the same goes for Group3.
Decision Tree (DT): Additionally, for the test scenario as detailed above, Decision Tree (DT) has been applied and results have been generated as per the following Test Confusion Matrix highlighted in Table 6. Fig 20 represents the results of the overall test outcomes using DT which reflects that in most of the cases the model has predicted the values quite accurately. Red circles in Fig 20 show the model’s test outputs and blue dots represent the desired test outputs. It’s just a few cases the developed model has made a mistake in identifying the correct group.
The overall accuracy percentage for Decision Tree (DT) (as presented in Table 3) is depicted in Fig 13A to illustrate the performance of the test models.
The accuracy percentages in each group are demonstrated in Fig 21B. Colour distribution distinguishes between the test mistakes and the correct predictions for the respective groups e.g., the bold red segment reflects the test mistakes in Group1, and the light green segment shows the correct decisions in the same group. Similarly, the small purple segment in the second inner circle reflects the test mistakes, and the bigger dark green segment shows the correct predictions for Group2 and the same goes for Group3.
k-Nearest Neighbours (KNN): For the test scenario as detailed above, k-Nearest Neighbours (KNN) along with the Euclidean distance have been applied. The Test Confusion Matrix is given in Table 6. Fig 22 represents the results of the overall test outcome after applying KNN, which reflects that in most of the cases the model has predicted the values quite accurately. As mentioned above, red circles show the model’s test outputs and blue dots represent the desired test outputs. It is just a few cases the model has made mistakes, e.g., points (47,1) and (47,2).
Furthermore, the overall accuracy percentage for KNN (as presented in Table 5) is depicted in Fig 23A that reflects the good performance of the test model.
The accuracy percentages in each group are demonstrated in Fig 23B. Colour distribution distinguishes between the test mistakes and the correct predictions for the respective groups e.g., the bold red segment reflects the test mistakes in Group1, and the light green segment shows the correct decisions in the same group. Similarly, the small purple segment in the second inner circle reflects the test mistakes, and the bigger dark green segment shows the correct predictions for Group2 and the same goes for Group3.
The test outcomes using GBM have been demonstrated in Fig 24. For Scenario 2, GBM has had the third best performance with respect to test accuracy after KNN, and RF (similar to SVM) among the other methods.
Similarly, the overall accuracy and the accuracies for each group are shown in Fig 25A and 25B below for GBM related to Scenario2.
The test performance for RF is demonstrated in Fig 26 for Scenario2. RF has been the second best performing method with respect to accuracy among all the other methods used. The overall test accuracy and the test accuracies for each group using RF for the second scenario are shown in Fig 27A and 27B below.
Similarly, NB’s test outcome is given in Fig 28. NB hasn’t performed as well as the other methods for Scenario2.
Fig 29A and 29B demonstrate the overall test accuracy and the accuracies for each group for NB.
The test outcome for LR for Scenario2 is shown in Fig 30. LR has had the lowest performance accuracy-wise among all the other methods. This could be due to the complex nature of the patterns in the data.
The test accuracy and the accuracy in each group has been demonstrated in Fig 31A and 31B for Scenario2 for LR. The test accuracy for each group is not as well as the other methods either which could be due to the complex nature of the data.
All the applied machine learning methods have provided relatively good results. However, for Scenario1, the best test accuracy was provided by RF and for Scenario2, the best test accuracy was given by KNN. GBM, SVM and RF ranked next after the best performing methods with regards to test accuracies.
4. Discussion
4.1 Applying strategic portfolio management to improve organizational agility
According to PMI [3,26–30], Portfolio Management also referred to as Strategic Portfolio Management (SPM) [47] aims to achieve strategic objectives through projects, programs and operations. SPM is a means to support agility by selecting the right projects, maximizing resource allocation, measuring and evaluating strategic alignment and balance to maximize the use of available resources [5,48–50].
SPM is required to support engagement and maintain organizational competitiveness [30] to support organizational agility. SPM implementation requires a supportive environment and consideration of risks, strategic foresight, constraints and barriers together with other impacting factors for SPM implementation must be identified [5].
These may include, but not limited to, changes to work practices due to the pandemic, increased use of technology, law limitations, market conditions, government or industry standards, infrastructure, human resource management and organizational process assets. In addition to this it will include project management information systems, methods for knowledge sharing and agile project delivery approaches.
Benefits of SPM include organizations becoming more flexible, dynamic, innovative, creative, communicative, strategically oriented, efficient and motivated [51]. Janssen [52] highlights the important lessons learnt for organisational agility, in particular post-pandemic. Incorporating organisational crisis/disaster management is one of the most important elements in bringing self-sustainability in businesses via keeping agility on the top of strategic vision [14,53,54]. Organisational agility and flexibility in decision marking [55–57] become more important to empower individuals, and businesses to innovate, satisfy the changing needs of the customers and swiftly adapt to business fluctuation [58–61].
Heising [9] views SPM as a support structure for organizational agility and a strategic role that enables organizations to respond and adapt to changing environmental conditions by monitoring and managing the project portfolio.
Killen & Hunt [48] stated that organizational agility supports strategic flexibility by enabling organisations to anticipate, identify, develop and implement change and investment strategies that are aligned with strategic objectives.
Hadjinicolaou et al [6] explored diverse characteristics of organizational agility and how diverse combinations existed within organisations at different levels of strategic management maturity.
Three sets of data containing 21 variables of Project Portfolio Management and organizational agility were analysed using canonical analysis. The results showed a significant positive strong correlation between SPM maturity and organizational agility characteristics within the studied groups. The research identified various combinations of characteristics constructed at different levels of maturity, and specific characteristics that highly contribute to the highest levels of maturity [6].
4.2 Aspects of future change
In respect to aspects of future change respondents were asked about the degree of expected change in the next two years whether a Major Decrease, Minor Decrease, Stay about the same, Minor Increase or Major Increase of the following:
- Revenue/Budget
- Profit/Surplus
- Market Share
- Market Size for our products or services
- Number of employees
- Competition
- Impacts of Legislative Changes
- Impacts from Globalisation
- Impacts from technology and digital disruption
- Impacts from acquisitions, management restructures
- A greater number of small projects
- A greater number of large projects
The response to the survey questions is presented in Fig 32 regarding the aforementioned aspects, which represents that some parameters mostly remain the same e.g., the impact of globalisation, however, other parameters i.e., the competition and revenue/profits related changes are imminent.
4.3 Interpretation of results
4.3.1 Benefits of organisational agility.
The results of this study found the top five benefits of organizational agility of statistical significance were:
- More efficient ability to move from strategy formulation to execution
- Quickly adapt practices based on learnings from successes and failures
- Faster response to changing market conditions
- Improved organisational efficiency overall
- Improved customer satisfaction
These were in alignment with a previous study exploring the relationships between 20 agile characteristics and 15 perceived benefits which included improved responsiveness, efficiency, sustainability or business decision making. The top three perceived benefits for organizational agility found in that study were:
1) “Faster response to changing market conditions"; 2) "Greater cost-saving" and 3) "More effective targeting of innovation opportunities" [7].
Each of these are fundamental to improving organisational performance with a focus on innovation. These benefits also highlight the need for organizational agility and the benefits that it brings however attention is required to overcome the barriers of organizational agility.
4.3.2 Barriers of organizational agility.
The top six barriers of Organizational Agility found in this study were:
- Lack of executive engagement/leadership
- Cultural mindset does not support agility
- Lack of effective executive sponsorship
- Lack of communication between departments
- Slow decision making
- Unclear organisational vision or strategy
Each of these involve action required at the executive leadership levels to improve engagement, focus on the culture, provide support and improved communication, reduce the time taken to make decisions and also define an organizations vision and strategy. A component of this will be the adoption of change management.
Change management practices adopted within organizations include:
- Use of a PMO for change management
- Work across organizational silos
- Implement a formal change management process
- Assign change managers
Each of these play an important part to improve organizational agility and implement practices required to improve maturity levels to support agility.
4.3.3 Aspects of organisational maturity.
In line with the barriers for organizational agility respondents were asked about aspects of organizational maturity levels in the following areas:
- Organizational process and procedure maturity
- Product Development Maturity
- Project Management Maturity
- Project Portfolio Management Maturity
- Organizational Resilience
- Organizational Agility
- Benefits Lifecycle Management
- A focus on risk management
- Value Management
The top five areas that received the lowest levels of maturity were:
- Project Portfolio Management Maturity
- Benefits Lifecycle Management
- Organizational Agility
- Product Development Maturity
- Organizational process and procedure maturity
This indicates areas that work is required to improve these levels and practices to support future organizational agility to improve performance and deliver on strategy. Project Portfolio Management supports organizational agility as organisations develop agility practices and improve performance to adapt to external pressures [6].
Heising [9] views Project Portfolio Management as a strategic role that supports organizational agility by enabling organizations to respond and adapt to changing environmental conditions by monitoring and altering the project portfolio. Killen & Hunt [48] discovered connections between PPM and organizational agility in supporting organizations with responsive decision making and resource allocation as well as building dynamic capability at organizational level.
In terms of key areas of changes for an organisation, respondents reported external factors causing change were: 1) customers; 2) funding or funding sources; 3) Socio Political and 4) Legislation. These factors require adequate risk management and strategic foresight with comprehensive scanning and monitoring of external environments to ensure minimal impact on the organisation and associated development teams. The monitoring and scanning for external changes play a key role in the development of organizational agility.
5. Conclusion
The objective of this study was to examine the extent to which artificial intelligence could be used to predict an organization’s future agility to mitigate risks and changes that may be required to support improving organisational agility. One of the limitations of the study was the insufficient sample size of the data collected requiring the Since the numbers of the respondents in the survey were not sufficient, firstly we have increased the sample size by producing synthetic data modelled on the data collected. After that the three efficient Machine Learning Models (Support Vector Machine (SVM), Decision Tree (DT) and k-Nearest Neighbours (KNN), Gradient Boosting Machine (GBM), Random Forest (RF), Naïve Bayes (NB) and Logistic Regression (LR)) have been used to predict agility among the participants. The overall test accuracy for most of the models is 90+% where GBM, RF, SVM and KNN are ranked higher among the other methods. The best-performing model is RF for Scenario 1 with the test accuracy of 97.674% and KNN with the test accuracy of 99.224 in Scenario 2. This means that using the above-mentioned model, the proposed system would be able to predict the Organizational Agility using the responses from the survey questions. Considering the importance of using technology to enhance informed decision-making, Artificial Intelligence methods proposed in the current study have achieved an accurate prediction outcome and performed the prediction task efficiently.
Modelling was developed using organizational attributes, principles and practices that are being utilized by industry to develop a culture of agility as applied to an organizations projects and operational work. The research data was collected from 44 respondents in public and private Australian industry sectors with varying sizes and industry sectors. These research findings were used to develop an Artificial Intelligence model to predict and support future organizational agility. The model included predominate agile principles and practices, aspects of size and type of organization, support structures being adopted critical to success, maturity levels and aspects of future change.
The findings from the research data collected (which addresses objective 1 of the study) supported the requirement of organisations to focus on found amongst the leading characteristics such as of organizational agility were having open communications; being flexible and adaptable; having transparency in decision making with continuous learning from experience; self-awareness and honesty; empowering team members with openness to change and having a commitment to agility.
The leading practices from the research data collected for organizational agility were being responsive to strategic opportunities; eliminating organisational silos; focusing on change management; having shorter production/review/decision cycles; integrating the voice of the customer; using interdisciplinary project teams; leveraging technology and an ongoing assessment of disruptive technological or other changes. Many of these practices to support agility and crisis management require commitment to an ongoing cycle of people, process and technology changes together with a culture of continuous improvement and responsiveness to internal and external change.
The findings from the research data collected also identified challenges and barriers to improving organizational agility together with the benefits of improved organizational agility. The barriers included the lack of executive engagement and leadership; the cultural mindset not supporting agility; lack of effective executive sponsorship; lack of communication between departments; slow decision making and unclear organisational vision or strategy. All of which require attention and mitigation of risks that reduce agility. The benefits and reasons to focus on organisational agility included improving the ability to move from strategy formulation to execution more efficiently; quickly adapting practices based on learnings from successes and failures; a faster response to changing market conditions; improved organisational efficiency overall and improved customer satisfaction.
A further limitation of this study is the data collected was from Australian organisations and prior to the Covid-19 pandemic. A further and broader collection would further test the model and potentially uncover changes as a result of new ways of working and working from home in a future study.
As a result of this study and the application of machine learning to predict agility organizations are able to apply to identify gaps and apply the framework to build new skills and competencies required, apply new strategic practices and improving the culture through training and education to focus required on innovation, change management and maturity levels to support engagement and continuous improvement.
The focus on cultural change and change management requires consideration of PMOs and other support structures, working across organizational silos, assignment of change champions and implementation of change processes to mitigate risks. Monitoring of internal and external changes also becomes critical to build agility to manage risks, apply strategic foresight, stakeholder engagement and deliver on strategy.
Our study makes a valuable contribution to emerging literature and an understanding of the principles, practices and characteristics that will help organizations with limited resources overcome barriers to build a framework and culture of agility to delivery on strategy. This will also support the building of organizational maturity levels for ongoing processes, product and project management, portfolio management, benefits lifecycle management, risk and value management.
Further work could be undertaken to apply the model to create a baseline within an organisation, apply aspects of change and resurvey after twelve months to analyse findings and create a case study.
References
- 1. Drew S. and Coulson-Thomas C. (1997). Transformation through teamwork: the path to the new organization?, Team Performance Management: An International Journal, vol. 3, no. 2, pp. 162–78.
- 2. Gerald E., Obianuju A., & Chukwunonso N. (2020). Strategic agility and performance of small and medium enterprises in the phase of Covid-19 pandemic. International Journal of Financial, Accounting, and Management, 2(1), 41–50.
- 3.
Project Management Institute (PMI). (2015). Pulse of the profession®: capturing the value of project management through organizational agility, PMI, Newtown Square, PA.
- 4.
Creswell J.W. & Plano V.L. (2011). Designing and conducting mixed methods research (2nd ed.), Sage, Thousand Oaks, California.
- 5. Guo N., & Yu S. W. (2013). The necessity of project portfolio management in the construction industry of China mainland. In Applied Mechanics and Materials (Vol. 357, pp. 2238–2241). Trans Tech Publications Ltd.
- 6.
Hadjinicolaou, N., Dumrak, J., & Mostafa, S. (2020). Relationship Investigation of Organizational Agility Characteristics and Portfolio Management Maturity. In MATEC Web of Conferences (Vol. 312, p. 02011). EDP Sciences.
- 7. Hadjinicolaou N., Dumrak J., & Mostafa S. (2020b). Organizational agility and its perceived benefits from Australian perspectives. Project Management Research and Practice, 6(Oct-Dec).
- 8. Hamel G. (2009). ’Moon shots for management (reinventing management to make it more relevant to a volatile business environment)’, Harvard Business Review, vol. 87, no. 2, pp. 91–99.
- 9. Heising W, (2012). The integration of ideation and project portfolio management-A key factor for sustainable success, International Journal of Project Management 30(5):582–595.
- 10.
Pallathadka, H. R.A.P. (2021). Applications of artificial intelligence in business management, ecommerce and finance. Materials Today: Proceedings., https://doi.org/10.1016/j.matpr.2021.06.419.
- 11. Din F. U., Paul D., Ryan J., Henskens F., & Wallis M. (2020). Revitalising and validating the novel approach of xAOSF framework under industry 4.0 in comparison with linear SC. In Agents and Multi-Agent Systems: Technologies and Applications 2020 (pp. 3–16). Springer, Singapore.
- 12. Ud Din F., Henskens F., Paul D., & Wallis M. (2018). Agent-Oriented Smart Factory (AOSF): an mas based framework for SMEs under industry 4.0. In KES International Symposium on Agent and Multi-Agent Systems: Technologies and Applications (pp. 44–54). Springer, Cham.
- 13. Coccia M. 2020. Deep Learning Technology for Improving Cancer Care In Society: New Directions In Cancer Imaging Driven By Artificial Intelligence. Technology In Society, Vol. 60, February, Pp. 1–11, Art. N. 101198,
- 14. Ardito L., Coccia M., Messeni Petruzzelli A. 2021. Technological Exaptation and Crisis Management: Evidence From COVID-19 Outbreaks. R&D Management, Vol. 51, N. 4, Pp. 381–392,
- 15. Crow D.A.; Albright E.A.; Ely T.; Koebel E.; Lawhon L. Do Disasters Lead to Learning? Financial Policy Change in Local Government. Rev. Policy Res. 2018, 35, 564–589.
- 16. Chia Y. Y., Lee L. H., Shafiabady N., & Isa D. (2015). A load predictive energy management system for supercapacitor-battery hybrid energy storage system in solar application using the Support Vector Machine. Applied Energy, 137, 588–602.
- 17. Kheirandish A., Motlagh F., Shafiabady N., & Dahari M. (2016). Dynamic modelling of PEM fuel cell of power electric bicycle system. International Journal of Hydrogen Energy, 41(22), 9585–9594.
- 18. Kheirandish A., Motlagh F., Shafiabady N., Dahari M., & Wahab A. K. A. (2017). Dynamic fuzzy cognitive network approach for modelling and control of PEM fuel cell for power electric bicycle system. Applied energy, 202, 20–31.
- 19. Shafiabady N. L. (2016). Using unsupervised clustering approach to train the Support Vector Machine for text classification. Neurocomputing, 211, 4–10.
- 20. Kheirandish A., Shafiabady N., Dahari M., Kazemi M. S., & Isa D. (2016). Modeling of commercial proton exchange membrane fuel cell using support vector machine. international journal of hydrogen energy, 41(26), 11351–11358.
- 21. Ud Din F., Paul D., Henskens F., & Wallis M. (2021). Conceptualised Visualisation of Extended Agent Oriented Smart Factory (xAOSF) Framework with Associated AOSR-WMS System. Journal of Software,
- 22. Ud Din F.; Paul D.; Ryan J.; Henskens F.; Wallis M. AOSR 2.0: A Novel Approach and Thorough Validation of an Agent-Oriented Storage and Retrieval WMS Planner for SMEs, under Industry 4.0. Future Internet 2021, 13(6), 155. https://doi.org/10.3390/fi13060155.
- 23. Klepáč V. &. (2016). Prediction of bankruptcy with SVM classifiers among retail business companies in EU. Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis, 64(2), 627–634.
- 24. Li H. &. (2019). Predicting business failure using multiple case-based reasoning combined with support vector machine. Expert Systems with Applications, 36(6), 10085–10096.
- 25. Ambagtsheer R. C. (2020). The application of artificial intelligence (AI) techniques to identify frailty within a residential aged care administrative data set. International journal of medical informatics, 136, 104094. pmid:32058264
- 26. Project Management Institute (PMI). (2017). Pulse of the profession®: achieving the greater agility-the people and process drivers that accelerate the results, PMI, Newtown Square, PA.
- 27.
Project Management Institute (PMI) (2021). A guide to the project management body of knowledge (PMBOK® Guide) (7th ed). Newtown Square, PA: Project Management Institute.
- 28.
Project Management Institute (PMI) (2017b). The standard for portfolio management (4th ed). Newtown Square, PA: Project Management Institute.
- 29.
Project Management Institute (PMI) (2017c). A guide to the project management body of knowledge (PMBOK® Guide) (6th ed). Newtown Square, PA: Project Management Institute.
- 30.
Project Management Institute (PMI) (2017d). Agile Practice Guide. Newtown Square, PA: Project Management Institute.
- 31. Oates J., Shafiabady N., Ambagtsheer R., Beilby J., Seiboth C., & Dent E. (2022). Evolving hybrid partial genetic algorithm classification model for cost-effective frailty screening: Investigative study. JMIR Aging, 5(4), e38464. pmid:36206042
- 32.
El Naqa I. &. (2015). What is machine learning? Machine Learning in Radiation Oncology, Springer, Cham, 3–11.
- 33. Liakos K. G. (2018). Machine Learning in Agriculture: A Review. Sensors, 18(8), 2674.
- 34. Huang S. C. (2018). Applications of Support Vector Machine (SVM) Learning in Cancer Genomics. Cancer Genomics & Proteomics, 15(1), 41–51. pmid:29275361
- 35. Deng Z. Z. (2016). Efficient kNN Classification Algorithm for Big Data. Neurocomputing, 195, 143–148.
- 36. Ding S., Hua X., & Yu J. (2014). An overview of nonparallel hyperplane support vector machine algorithms. Neural computing and applications, 25(5), 975–982.
- 37. Wang Y. M. (2022). Application of Decision Tree Algorithm in Early Entrepreneurial Project Screening. Scientific Programming, https://doi.org/10.1155/2022/3584196.
- 38. Song Y. Y. (2015). Decision Tree Methods: Applications for Classification and Prediction. Shanghai Archives of Psychiatry, 27(2), 130. pmid:26120265
- 39. Chen S., Webb G. I., Liu L., & Ma X. (2020). A novel selective naïve Bayes algorithm. Knowledge-Based Systems, 192, 105361.
- 40. Dahmen J. a. C. D., (2019). SynSys: A Synthetic Data Generation System for Healthcare Applications, Sensors, 2019. pmid:30857130
- 41.
Ayyadevara V. K. (2018). Gradient boosting machine. In Pro machine learning algorithms (pp. 117–134). Apress, Berkeley, CA.
- 42. Rigatti S. J. (2017). Random forest. Journal of Insurance Medicine, 47(1), 31–39. pmid:28836909
- 43. Shipe M. E., Deppen S. A., Farjah F., & Grogan E. L. (2019). Developing prediction models for clinical use using logistic regression: an overview. Journal of thoracic disease, 11(Suppl 4), S574. pmid:31032076
- 44. The Economist, (2022). The world’s most valuable resource is no longer oil, but data. [Online]. Available: https://www.economist.com/leaders/2017/05/06/the-worlds-most-valuable-resource-is-no-longer-oil-but-data. [Accessed 31 05 2022].
- 45. Altexsoft, (2022). Synthetic Data for Machine Learning: Its Nature, Types, and Means of Generation. [Online]. Available: https://www.altexsoft.com/blog/synthetic-data-generation/. [Accessed 31 05 2022].
- 46.
N. a. W. R. a. V. K. Patki, (2016). The Synthetic Data Vault, in 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2016.
- 47. Leavy B. (2014). ’Strategy, organization, and leadership in a new ’transient-advantage’ world’, Strategy & Leadership, vol. 42, no. 4, pp. 3–13.
- 48. Killen C., Hunt R., (2013). Robust project portfolio management: capability evolution and maturity, International Journal of Managing Projects in Business 6(1):131–151.
- 49. Martinsuo M, Killen C (2014). Value Management in Project Portfolios: Identifying and Assessing Strategic Value, Project Management Journal 45(5):56–70.
- 50. Martinsuo M, Lehtonen P (2007). Role of single-project management in achieving portfolio management efficiency, International Journal of Project Management 25:56–65.
- 51. Madic B Trujic V, Mihajlovic I (2011). African Project portfolio management implementation review, Journal of Business Management 5(2):240–248.
- 52. Janssen Marijn, and Haiko Van der Voort. Agile and Adaptive Governance in Crisis Response: Lessons from the COVID-19 Pandemic. International Journal of Information Management, 55 (2020): 102180.
- 53.
Solheim M.C.W., Aadland T., Eide A.E., Haneberg D.H. 2022 Drivers for Agility in Times Of Crisis. European Business Review, Article in Press.
- 54. El Idrissi M., El Manzani Y., Ahl Maatalah W., Lissaneddine Z. 2022.Organizational Crisis Preparedness During The COVID-19 Pandemic: An Investigation of Dynamic Capabilities and Organizational Agility Roles. International Journal of Organizational Analysis.
- 55. Evans S.; Bahrami H. Super-Flexibility in Practice: Insights From A Crisis. Glob. J. Flex. Syst. Management. 2020, 21, 207–214.
- 56.
Coccia M. 2021. Comparative Critical Decisions in Management. In: Farazmand A. (Eds), Global Encyclopedia of Public Administration, Public Policy, And Governance. Springer Nature, Cham. https://doi.org/10.1007/978-3-319-31816-5_3969-1
- 57. Coccia M. 2021e. Pandemic Prevention: Lessons From COVID-19. Encyclopedia, Vol. 1, N. 2, Pp. 433–444.
- 58. Mero J., Haapio H. 2022 An Effectual Approach to Executing Dynamic Capabilities Under Unexpected Uncertainty. Industrial Marketing Management, 107, Pp. 82–91.
- 59. Coccia M. 2019. Intrinsic And Extrinsic Incentives to Support Motivation and Performance of Public Organizations, Journal of Economics Bibliography, Vol. 6, No. 1, Pp. 20–29,
- 60. Akkaya B., Mert G. 2022 Organizational Agility, Competitive Capabilities, And the Performance of Health Care Organizations During the Covid-19 Pandemic. Central European Management Journal, 30(1), Pp. 2–25.
- 61. Anton P. 2021 Organisational Agility in A Context of Crisis, Soins, 66 (856), Pp. 61–65.