Towards ubiquitous requirements engineering through recommendations based on context histories

Principles of Nhatos: survey with 56 professionals

A survey involved 56 professionals working in the software development industry, including project managers, analysts, project teams, and teachers. Participants answered an electronic questionnaire with multiple choice and transcribed questions. About 71% of the interviewees had more than five years of experience in projects. More than 70% of respondents worked in companies with more than 100 employees. The main objective of the research was to capture the perception of professionals regarding the support tools in project management currently used in their work environment. This research allowed to identify gaps and possible improvements in the RE area guiding the specification of Nhatos. The following are the research questions and results:

• Which areas do you consider most critical to the success of the project? 50% of respondents selected the scope as the most critical area. Participants also mentioned the areas of Time, Communications, and Integration, with 44.6%, 42.9%, and 39.3% of responses, respectively;

• In the projects where problems occurred, what were the areas in which the problems were identified? A total of 39.29% of the participants answered that problems in project management are due to incorrect Scope Management (SM). Other project areas, such as Time Management and Communications, obtained 25 (44.6%) and 21 (37.5%) responses, respectively;

• What types of suggestions would you like to receive from a proactive project management tool? According to the interviewees’ perception, 32.1% answered that a tool should suggest new requirements for projects;

• Do you believe that information from other projects already completed could assist in project management? 85.7% of the members confirmed that history contributes to the management of the new projects.

The perception of the teams collected in the survey allowed to conclude that there is interest from the professionals regarding the use of an intelligent tool to support the project teams during the RE processes. This opportunity stimulated the development of the Nhatos, which aims to assist teams during the RE process life cycle.

Model architecture

Figure 1 shows Nhatos architecture using the Technical Architecture Module (TAM) modeling specification (FMC, 2021). The following components are part of the model and they are seen in the figure with their respective numbering:

1. Mobile App: Application in a hybrid structure based on Javascript language, which operates both in mobile applications (Android/IOS), as well as in browsers web. Stakeholders use this application during interactions in RE processes. The stakeholders can: (a) add and manage projects and their information; (b) add requirements and modify them; (c) view the recommendations generated by the model; (d) provide feedback about the recommendations received, accepting or rejecting them;

2. Integration Interface: It enables the bulk import of data from ongoing projects. When using this system web interface, users may import projects and requirements exported from third-party project management tools, such as MS Project, for example;

3. Configuration: Project team preferences are entered through an assistant agent, accessed via mobile devices. Each project variable receives a weight (area of knowledge, size, methodology, and level of completeness of the schedule). Each recommendation made by the model considers the weight configured by a specialist.

4. Views and API: These applications operate in an integrated manner in a server environment.

   1. Views: They are characterized by controllers responsible for the business rules of the model, obtaining information already stored in the database to provide information to those involved;

   2. API: A data access interface which uses the RESTFul protocol to interconnect the application Mobile App and Database.

5. Agents: Multi-Agent System (MAS) that captures the events related to project’s evolution or modification. The capture is triggered when some of these events occur: (a) addition of a new requirement; (b) termination of an activity; or (c) evolution in the percentage of completion of the project. Figure 2 shows the proposed MAS using the the Prometheus methodology (Larioui, 2020). The MAS has six agents. The Translate agent converts to English the texts from native languages used in the projects. The NLP uses English as the language, so this translation is necessary to Nhatos. Projects Similarity analyzes the similarity of the projects using project size, methodology applied and area of expertise. NLP techniques allow to group projects according to their expertise. Context Storage stores each event occurrence in the project‘s history. Recommendation Engine permanently monitors the project’s events to orchestrate the execution of the other agents when one event occurs. Requirements Similarity uses semantic analysis to determine the requirements similarity based on texts written in natural language. This analysis is detailed in the Similarity Analysis subsection. The agent also compares requirements to determine if requirements have the same number of actors. Context Similarity performs the similarity analysis in the context histories of the projects.

6. Projects Database: It saves application settings, such as (a) Project data; (b) Recommendations made by the model; (c) Feedback from stakeholders regarding the recommendations; and (d) Context histories that occurred throughout the life cycle of the projects.

Similarity analysis

The similarity analysis occurs in two moments: (1) similarity analysis based on project characteristics, and (2) similarity analysis based on context histories of projects. The first analysis occurs at each insertion of a new project, while the second analysis occurs during a new evolution of the projects’ life cycle. Figure 2 shows the representation of the multi-agent system with the three agents that conduct the similarity analysis (Context, Projects and Requirements).

The model captures the characteristics of the new project during its creation. At this time, the project teams have not yet implemented it. The project is in the creation or planning phase. The Project Management Institute (2017a) has characteristics that are considered properties of a project. Stakeholders determine these characteristics in the early stages of the life cycle, which are: (a) Area of knowledge; (b) Development methodology (agile, traditional, or hybrid); (c) Level of completeness of the schedule; and (d) Size. Figure 3 shows how the recommendation flow occurs in this step.

After capturing the fundamental characteristics of the project, an expert defines the weights for each one. The specialist defines the importance of each characteristic, based on its degree of relevance to the project and organization. This definition allows to measure the similarity between the projects before the start of their execution.

After the insertion of a new project, the multi-agent system identifies this event and initiates the recommendation process. The multi-agent system goes through the stored histories and compares the variables with characteristic of each project in the history with the same variables as the original project. The model considers the configurations previously informed by the specialist - these configurations make up a weight system, which will be applied during the calculation.

After the model groups similar projects, the context histories of those projects are analyzed to identify reusable requirements among them. Therefore, for projects in the same group, Nhatos calculates similarity considering the semantic distance between their requirements.

Nhatos defines the semantic distance by analyzing the distance between text documents proposed by Kusner et al. (2015). This approach takes advantage of the results of Mikolov et al. (2013), whose model word2vec generates combinations of words, on a large-scale ontology, for extensive data sets (for example, we use the training of approximately 100 billion words in this model). In this way, Nhatos compares the texts that describe the objectives of the requirements. Afterward, the results are stored in history, enabling the recommendation of requirements related to the same theme (similar purposes) and similar projects (same area of knowledge).

The recommendation of requirements in the initial phases of the projects aims to bring historical information to the project teams, mainly to the requirements engineers and stakeholders. Then, these users will be able to accept or reject the requirements recommended by the model. Nhatos thus ensures that no requirements of the historical basis are disregarded by those involved during management.

During the life cycle of projects, Nhatos saves the events in context histories. The model identifies information that is susceptible to changes in state over the life cycle of the projects, which are: (a) purpose of the requirements; (b) actors involved in the requirements.

Whenever a user inserts a new requirement into the project or at least one of this context information is modified, the similarity analysis of projects by context histories begins. Nhatos uses the stored histories to complement the similarity analysis based on the characteristics, updating recommendations based on the new information.

Figure 4 shows how the recommendation flow occurs in this step. This flow seeks similar contexts by analyzing the context histories of the project and comparing this information with other stored contexts. Each step presented in the projects’ timeline represents information about event occurrences saved in context histories. In this way, this information can later be used by the model to generate a new recommendation. Whenever one of the defined events occurs, Nhatos creates a record in the context history of the project.

Nhatos compares each project context with contexts from similar projects by using the semantic distance between the requirements from the previously-stored histories. The recommendation of the next occurrence of the context history occurs for the project in execution when the distance between the requirements is acceptable (according to the specialist’s settings), and the number of actors is equal between the requirements compared.

Figure 5 shows an example of analyzing the context histories of an ongoing project with histories from similar projects. In this example, the entire recommendation flow is elucidated step by step.

We considered two projects to exemplify the similarity analysis of context histories. Both projects (Pa and Pb) have five requirements. The multi-agent system compares requirements between projects in chronological order. In this scenario, the specialist configured the minimum semantic distance of 0.3, training sample 70%, and three similar steps to generate the recommendations. Thus, the comparison of the first requirement of Project A with the first requirement of Project B generated a semantic distance of 0.41 (distance not acceptable according to the configured parameters). Another excluding factor is that these requirements have a different number of actors. Thus, step 1 is not similar. On the other hand, steps 2, 3, and 4 obey both the minimum acceptable semantic distance and the same number of actors involved in the compared requirements. Since the expert has configured recommendations that require at least three similar consecutive steps, Nhatos recommends the fifth requirement of project B for project A.

Assuming that the configured training sample was 70%, Nhatos uses the remaining percentage of the project (30%) to verify that at least one requirement with the same semantic distance and the same number of actors as the recommended requirement occurred throughout the life cycle. Once it occurs, the recommendation made is considered assertive.

The analysis of more than one chronological context, which occurred during the project, aims to identify projects that have a similar execution sequence. This analysis contributes to a higher degree of precision in the recommendations made. The most significant number of similar consecutive contexts indicates the proximity between project implementation. However, when Nhatos considers more contexts, the fewer projects must be identified as similar and, therefore, the fewer recommendations made, since each project is unique (Project Management Institute, 2017a).

Ontology of requirements recommendation

Figure 6 shows the Ontology of Requirements Recommendation proposed by Nhatos. The domain ontology contains Projects, Requirements, and Specification. This representation is an extension of the work of Silver (2014), with the addition of the Projects Ontology. Three ontologies covered the domain considered by Nhatos: (a) Requirements Specification Ontology which makes up the requirements specification; (b) Projects Ontology which is characterized by the domain of the project and its contextual information; and (c) Requirements Ontology that represents the requirements.

The project ontology has the Project class composed of the Area, Completion Level, Methodology and Size. These subclasses define the contextual information of the projects. The ontology that comprises the requirements specification has the classes Scenario, Goal, and Actor. These classes represent the elicitation process, identifying new requirements and relating them to projects. The ontology that determines the requirements has the class Requirement and its subclasses Functional Requirement and Non Functional Requirement, which define the types of requirements.

We added a specification document (Requirement Specification Document) to exemplify how the requirement is instantiated when using ontologies. Each document has a scenario, where the actors through the defined objectives identify requirements for a given project.

Hybrid application

The Hybrid Application runs both on mobile devices and on conventional computers operating in a browser web. In this way, the application has two interfaces for communication with system users: (1) Web Application; and (2) Mobile App.

This application allows the interaction of project teams with Nhatos, allowing the management of projects, requirements, activities, resources, registration of interested parties, and evaluation of recommendations. The prototype allows to monitor projects, capturing context information to compose the context histories. The software can be used throughout the life cycle of projects. Further, this application presents the recommendations to the user, thus allowing the collection of feedback from interested parties.

Figure 8A shows the interface for the project presentation with characteristic information about the project such as size, methodology, percentage of evolution, and area of knowledge. Figure 8B presents the list of project requirements, as well as their respective percentage of evolution.

Figure 9A shows the settings of the weight variables, which are defined by an expert. These variables define the importance of each aspect of the project and its requirements during the recommendation process. The user can define weights related to project area, size, methodology, and level of completion, as well as the acceptable semantic proximity between the requirements that will be considered for a possible recommendation.

Figure 9B presents examples of recommendations. The interface contains the recommended requirement, as well as the requirement that raised this recommendation and the semantic distance between these requirements. The interaction area also enables users to provide their feedback, selected from the options to accept or reject the recommendation. This user decision is registered in order to evaluate the acceptability of the recommendations by stakeholders in the future.

Console application

This application developed in Python is an encapsulated software that acts in the form of service. The software uses the concept of multi-agent systems, proposed by Padgham & Winikoff (2004) to implement the Agents layer. The application does not depend on direct interaction with the users, being triggered by modifications detected in the environment. The Agents layer implements the following 6 agents: (1) Recommendation Engine; (2) Translate;(3) Projects Similarity; (4) Requirements Similarity;(5) Context Similarity; and (6) Context Storage.

The Recommendation Engine runs systematically through the detection of changes occurred in the environment. The agent always activates when a user inserts a new project or requirement. Once performed, this agent is responsible for starting the execution of all other agents.

The Translate agent translates all the contents entered by the user into the English language, enabling the execution of subsequent agents. Since, as a premise, all contents must be registered in English. Nhatos uses NLP and the corpus of texts obtained for the application of the study is written in this language. The software loads the projects and their requirements from the database, translating them using the API Google Cloud Translate (Google, 2021b). We considered the usage of this API does not generate discrepancies in translations since this API has an estimated accuracy of 85% (Aiken, 2019) and the texts translated are technical, having a technical writing pattern.

The Projects Similarity agent activates after the translation of the content of the projects and their requirements. This agent analyzes the similarity between all projects in the database and groups the characteristics of the projects separately. The software considers the information in the ontology (Fig. 6), considering all projects in the database according to size, area of knowledge, management methodology, and level of completeness (schedule). After consulting all projects, the agent classifies and labels each project, so that the next step of the algorithm starts.

The Requirements Similarity employs the use of NLP to find requirements that contain equivalent objectives, as well as the same number of actors involved. This agent also considers the similarity analysis between the projects, performed previously. In this way, the algorithm analyzes the similarity between the requirements of projects considered similar.

First, the agent appropriates the new settings for distance, steps, and sampling, thus starting new processing. Then, it removes the previous recommendations (if any), to start a new recommendation process. This method removes all recommendations that originated from the same distance, number of steps, and sampling setting. Because, it is considered that throughout the life cycle, the requirements may have changed regarding their objectives or actors involved (Project Management Institute, 2017a).

Then, the agent retrieves all projects from the database. The requirements of each project are obtained. Each requirement is compared with the requirements of similar projects. In this step, the agent checks whether the objective of both requirements meets the established distance parameters, as well as the number of actors. Finally, once the number of requirements found sequentially is equal to the number of configured steps, this requirement is considered recommendable to the project.

The Context Similarity agent analyzes whether the recommendations carried out within the sample selected by the specialist occurred in the project. The software loads all recommendations made and it selects the project requirements after the recommendation’s record date. If among the requirements, a requirement is found with the same semantic distance to the compared requirement and, containing the same number of actors, this recommendation is considered assertive. Otherwise, the recommendation becomes non-assertive. Finally, this decision is stored in the database.

The Context Storage agent keeps the information in a database with four main entities: (1) projects; (2) requirements; (3) requirements distance; and (4) recommendations.

Figure 10 shows the relational entity diagram of the model database. The projects entity saves information related to projects. This entity registers basic project information and its characteristic information, such as: opening statement (description), knowledge area (domain), size (size), and methodology applied during its development (methodology).

The requirements entity keeps model requirements. The same project can contain several requirements, according to the relationship in the diagram (1..n). The actors of each requirement are stored in the entity actors and an actor can be linked to several requirements and vice versa, as shown in the entity requirements_actors (n..n).

The requirements_distance entity saves the distance information between the processed requirements. The entity stores the original requirement (req_a_id) and the compared requirement (req_b_id), as well as the respective semantic distance (distance) between them.

Finally, the recommendations entity keeps the recommendations inferred by Nhatos. Each recommendation is directed to a project (project_id) and the requirement that generated such a recommendation (requirement_id). The assessment of the assertiveness of each recommendation is stored in the is_assertive property.

RESTFull API application or web service

The RESTFull API Application provides a communication channel between the Hybrid Application and the Context Storage application. Using the RESTFull protocol, the application allows data traffic in the JSON format between applications. It enables the exchange of data between the hybrid application, used by users, and the information already processed by Console Application, which stores its information in the Nhatos model database.

Team evaluation during project execution

The first case involved two teams with a total of 12 professionals. These professionals were asked to validate the recommendations made by the model. Table 2 shows the profile of the teams that participated in this experiment.

Initially, project teams inserted information related to the projects in the database (scope and descriptive information, terms of reference, resources, schedule, and tasks/activities). The professionals used the integration interface to insert information into the database. After, the project teams identified and registered the requirements using the prototype running on mobile devices.

The similarity analysis of the projects used NLP since approximately 79% of the requirements and project documents are written in natural language (Luisa, Mariangela & Pierluigi, 2004). The algorithm classified each project through the use of NLP according to its respective area of knowledge. The identification of these areas involved the use of the opening statement. In addition, the project charter contains a high-level description of the project. The classification used the Google Natural Language API (GNL), which provides resources for the analysis of unstructured texts, such as content classification and entity identification. The content rating analyzes a document and results in a list of categories that apply to the found text. The classification can still contain several levels, specifying the greater depth of details about the area of knowledge in question (Google, 2021a).

Currently, GNL processes English sentences only. The projects had information in Portuguese, so it was necessary to previously translate the descriptive content of the projects before carrying out the classification process. The prototype performed the translation automatically using the Google Cloud Translation API (GCT) (Google, 2021b). The GCT receives a phrase as input, identifies its language and translates it into the language selected by the user. During the study, the agent translated all sentences into English. After translation, Nhatos categorized the projects using GNL based on the 153 projects. This step classified 23 projects as Finance (15.03%), 20 as Business and Industry (13.07%), 10 as Computers and Electronics (6.53%), 6 as Credit and Lending (3.92%), and 4 as Accounting and Auditing (2.61%), having these categories the highest number of classified projects.

In addition, the variables registered in the Configuration module allowed to defining the knowledge area of each project. The similarity analysis used this definition. These variables received weights, which reflect the uniqueness of each project. The weight setting allows the algorithm to generate different recommendations according to the characteristics recorded by the specialist. Table 3 presents the recommended requirement values for each project and the requirement values added to new projects through the recommendations.

During the case study, users included in the projects twenty requirements, which were used to evaluate the recommendations made in the execution of the project. Whenever a user added a new requirement, the Requirements Similarity Agent identifies the event and performs the semantic proximity analysis. The Agent compares the new requirement description with the requirements stored in the context history of the project.

The GNL algorithm analyzes texts in English, performing a semantic analysis. Thus, the model translates the description of the project or requirements which are in another language. Table 4 shows an example of a requirement added to a project and the recommended requirements based on this insertion. Nhatos considers the objective of each requirement and also the number of related actors.

At this stage, Nhatos applies the analysis to the entire database, regardless of whether projects were compared in the first stage of the recommendation. The semantics of the included text and the objectives of the requirements may change throughout the project (Dick, Hull & Jackson, 2017). This step also allows project requirements that were not originally recommended to be analyzed and considered. The analysis is carried out at this point on the requirements, considering the grouping of projects based on their characteristics. The semantic distance represents the comparison of the requirement objectives, having a floating-point value between 0.0 and 1.0. The distance closer to 0 indicates that the recommended requirement is semantically closer to the original.

Consequently, the closer the semantic distance to 1, the less similar the recommended requirement is considered when compared to the original. The actors of each requirement are also considered during this stage of the analysis. This context information considers the number of actors to which a requirement is related.

During the case study, for each of the five projects, the model analyzed the similarity, recommending the requirements between similar projects. Soon after, the team analyzed the recommended requirements for the five registered projects. Table 3 shows that the approval rate of the requirements had an average of 71.0%. The average appropriation of the recommendations presented for new projects shows the acceptance of the requirement recommendation model evaluated by the teams, in order to provide more information to the managers since the beginning of the project.

Evaluation of recommendations through analysis of context histories

The second case compared the requirements registered in 147 completed projects with the requirements recommendations made by Nhatos. This study allowed to infer whether the recommendations made, considering a sample of 70% of the progress of the projects, were in fact inserted into the remaining percentage of the project.

This scenario evaluated the recommendations made in many situations considering projects with different characteristics. Most of the projects (81) used the agile methodology based on the framework SCRUM (Sutherland & Coplien, 2019). The others (66) employed traditional methodology based on the good practices proposed by Project Management Institute (2017a). We classified the projects into one of three size categories, considering the execution time of each one: small (up to 500 h), medium (up to 3,000 h), and large (over 3,000 h).

The learning phase used a sample of 70% referring to the execution of each project. The Nhatos Learning consists of collecting the project’s evolutionary events. Figure 4 shows this process in the Recommendation Engine, Context Storage, and Context History Similarity steps. With this learning, Nhatos generated requirement recommendations for the same projects. The remaining 30% of data from the executed process was the base for assessing the recommendations made by the model.

The similarity analysis between the projects taking into account the context histories considers that a consecutive sequence of contexts must be similar. The model then recommends the next requirements for the project being implemented based on the sequence of events that generated context histories.

In all, nine test scenarios were configured using different parameterizations to find the best scenario of recommendations, where each test considered the entire historical database. Table 5 shows the results of the different settings applied. A test scenario addresses different combinations of the variables Distance, Steps and Sample.

All test scenarios considered a 0.7 sample value. Therefore, the training sample to generate the recommendations contained 70% of the evolution of the projects’ schedule (life cycle). Tests 1, 2 and 3 considered configurations with a minimum semantic distance between the requirements of 0.25 (75%). All 3 step configurations for these tests resulted in an assertiveness rate equal to or less than 50%. Scenario 2 and 3 generated a few recommendations, 27 and 2, respectively. Test 1 generated a total of 481 recommendations. However, as mentioned, the assertiveness did not reach 50%.

On the other hand, tests 7, 8, and 9 generated a significant amount of recommendations. The assertiveness rate was high, above 80% in the three cases. However, the semantic distance proved to be relatively comprehensive, considering requirements only 65% similar. Therefore, the tests allowed the inference of a large number of requirements recommendations (a total of 9,214). In these scenarios, Nhatos provided a high number of recommendations, being more than the teams could assess. Therefore, the model did not attend to a requirement of ubiquitous applications in these cases since these applications must be minimally intrusive (Satyanarayanan, 2001).

Scenarios 4, 5, and 6 were more promising than the first ones. All three achieved a percentage rate of assertiveness close to or higher than 70%. However, only scenarios 4 and 5 had an adequate number of recommendations for analysis in the use case.

The configuration of the Scenario 4 considered a minimum distance of 0.3 (70%) and three steps, obtaining a hit rate of 70.35% for assertive recommendations. Of the 1,872 inferred recommendations, 1,317 were correct, while 555 were incorrect. Scenario 5 considered a configuration of a minimum distance of 0.3 (70%) and four steps. This scenario obtained a hit rate of 69.97% of assertive recommendations. A total of 452 of the 646 recommendations were correct, while 194 were unsuccessful. The different scenarios allowed a new round of tests to be carried out, obtaining the most certain configuration for the database.