Keywords

1 Introduction

To meet the first commitment, one of the Eight Commitments School Management given by the government [1], the school (IE: InstituciĂłn Educativa in Spanish) organize and propose periodically assessments for students; it means, they implement an evaluation culture for students and propose corrective measures for poor academic performance in the evaluated areas. According to Gonzales et al. [2], an evaluation culture could be defined as the set of values, agreements, traditions, beliefs and thoughts that an educational community attaches to the action of evaluation. BolseguĂ­ and Fuguet [3], point out that the evaluation culture is an evolving concept that refers to the need to evaluate on an ongoing basis; for them, the assessment is a complex and multidimensional process that includes different components: vision, values, behaviours, routines, organizational and social context, past and present experiences, epistemological, theoretical and methodological.

Organisational complexity is very high because the evaluation processes and functions require multi-systems collaborations, and has a high reliance on effective information communication among DRE (from spanish DirecciĂłn Regional de EducaciĂłn), UGEL (from spanish Unidad de GestiĂłn Educativa), IE, Director, Teachers and therefore faces many challenges regarding pragmatic interoperability such as information collision, policy obstacles, and procedure mismanagement.

Once evaluated, it is necessary to analyze and interpret the results, for which it is important that the data is properly displayed; this topic has been widely discussed by the authors in the area of data visualization [4, 5]. Most frequently, a key feature of such an approach is showing relationships between different data groups of a provided statistical selection: in order to compare relative proportions between various indicators [6, 7].

This article presents a strategy to qualify student test primary and secondary level in Math and Communication areas in Educational Institutions of Apurimac-Peru, showing the obtained results by statistical charts sharing achievement levels for each student, this information is useful to teachers who can use improvement strategies for underperforming students. In addition, the tool also allows you to display statistical graphs of the results by district, by level, by grade, by area; so that the Directors of DRE, UGEL and IE can make decisions to improve student learning.

Section 2 of this article presents and discusses the work related to the subject of semiotics of educational organizations and data visualization; Sect. 3 explains the design and implementation of the proposed strategy; Sect. 4 explains the evaluation methodology used to validate the proposed strategy; finally, Sect. 5 describes the conclusions and future work of this research.

2 Related Works

Semiotics is the study of meaning of sign processes and meaningful communication. This includes the study of signs and sign processes (semiosis), analogy, metaphor, symbolism, signification, and communication. Within the organisational semiotics literature, three forms of activities are discussed, namely substantive, communication and control [8]. While much attention has been paid to substantive norms in business process modelling [9], relatively less research has focused on control norms, especially in virtual environments. Process controls are embedded constraints that serve as conditions for performing related activities [10].

Effah and Liu [9] publishes an article titled “Virtual Process Control Modelling in Organisational Semiotics: A Case of Higher Education Admission”, this study explores Web-based virtual process control modelling based on organisational semiotics, Web modelling language (WebML), and higher education admission process. The study contributes to organisational semiotics research, which has so far focused more on substantive norm and less on control norm modelling.

On the other hand, Data Visualization is a general term that describes any effort to help people understand the significance of data by placing it in a visual context. Patterns, trends and correlations that might go undetected in text-based data can be exposed and recognized easier with data visualization software. There are techniques for facilitating data selection in the data transformation process [11, 12], techniques for selecting chart type and visual components (e.g., line style, point face, axis range) automatically in the visual mapping process [13, 14]; and techniques for changing visual effects to clarify the user’s viewpoint and assertion easily [15] in the view transformation process.

Matsushita and Kato [16] make a research titled “Interactive Visualization Method for Exploratory Data Analysis”. They propose an interactive visualization method suitable for exploratory data analysis. The method extracts parameters for drawing from a series of user requirements written in a natural language and redraws the drawn chart interactively according to the change in the user’s viewpoint. In this type of data analysis, statistical charts are employed to help the user understand the target data, and displayed chart is redrawn interactively according to the user’s input inspired by looking at the chart.

3 Design and Implementation

For this work we used the sign-based Semiotic Communication and the System Oriented Approach of Semiotic Organisation which includes Sign System Oriented Approach. Sign System Oriented Approach studies media (spoken language, texts, instruments, computer interfaces) as sign systems, and see the use of these media by people as based on systems of narration and interpretation. User interaction with media (texts, computer interfaces, instruments) is observed, as well as communication between people at work.

We also used a Visual Communication approach to represent the obtained results, which has various types of codes that help determine the message. The argument can be made that visual communication operates similar to a language sign system.

3.1 Educational Business Process Management for Assessment

The process starts when DRE proposes a schedule for evaluations, then Specialists develop indicators and questions for the test. Then each UGEL’s Director, print the tests and distributes to each IE. Then The Director of each IE, assign evaluators (students) for each grade and section. The evaluation day, students take the exam, then teachers fill answers in Excel sheet to qualify each test using Excel Macro (just clicking a button). Then, Director and teachers of IE can see the graphical results generated by the system. Then, Excel file is very heavy in size (about 10 Mb), so we convert the data to a lightweight format file “CSV” (about 5 k), so that file is easier to transfer by Internet. Finally, the System processes updated data and generate new reports for users. The process is shown in Fig. 1.

Fig. 1.
figure 1

Business process management for assessment application

Full size image

3.2 Input and Output of Data Visualization

Designing Excel File.

An Excel File was designed to fill the answers marked by students, as shown in Fig. 2. Every question can be market with “A”, “B”, “C”, “X” (when students mark two or more answers) or “ ” (blank, when a student did not mark any answer). Mathematics and Communication have 23 questions, every section has from one to forty five students approximately, and in the primary level there are six grades named “First”, “Second”, “Third”, “Forth”, “Fifth” and “Sixth”; and in the secondary level there are five grades named “First”, “Second”, “Third”, “Forth”, “Fifth”.

Fig. 2.
figure 2

Sheet to fill answers

Full size image

Designing Reports.

After the teacher fill out the answers, the system use functions in Excel Macro to rate the assessment using values assigned in Table 1. An example of the Report is shown in Fig. 3.

Table 1. Distribution of reached level according to punctuation and colour assigned
Full size table
Fig. 3.
figure 3

Report using semaphore indicators (Color figure online)

Full size image

Assigning Reached Level.

Each test has 23 questions for each area. To determine the level a student has reached, specialists produce distribution of scores, as shown in Table 1. For example, if a student gets a score of 5, then, the level acquired is “Initiation”; if a student gets a score of 12, then, the level acquired is “In Process”; and if a student gets a score of 20, then, the level acquired is “Achieved”.

4 Evaluation of the Proposed Strategy

The validation of the tool was conducted with Education Specialist (workers) of the DRE-ApurĂ­mac through a focus group. The two events took place on: May 15 and September 14, 2015; in a meeting room of the Pedagogical Management Area of DRE. The participants were five specialists in evaluation. All participants had significant years of experience in monitoring and supervising educational schools.

Before starting the activity, the developed system was briefly shown to each specialist in evaluation. A simulation of the software functionality was then done. An operator, using the SIERA system, entered the “modular code” of the IE, then, the System generates the list of students of each grade and sections. The specialist fill out the answers for each student and then the system show the results graphically, then, he exports the result to CSV file and finally sends the result to the datacentre.

After this simulated process, the Education Specialists provided feedback, suggestions and opinions, and agreed that the system can definitely help improve decision making and perception (awareness) of the assessment results. When asked: “will the use of the designed software help you to make it easier the data visualization of evaluation assessment results?”, they all answered that in their opinion that assumption was valid. Then they were asked: “would the decision-making speed and quality of decisions have been better if the Director would have had a support tool that is visual and provides the appropriate suggestions?”, they all replied that a positive answer would be valid.

There were two evaluations until August 31 2015 using SIERA system; 614 schools and 38338 students were evaluated in the first assessment; 961 schools and 68075 students in the second assessment.

5 Conclusion and Future Work

Education Specialists agreed that having this type of information in the SIERA system would help to know the levels reached (Beginning, In Process, and Achieved) by students in every area, level, grade and district; also, Director could have accurate information when making decisions based on the learning achievement indicators. The time required to have results of the assessment is reduced significantly and the information is available at every time and everywhere whenever Internet connection is available. Likewise the participant Specialists agreed that, a priori, they can expect a significant improvement in the process of student learning and the quality of decisions, because Directors would have accurate information when making decisions.

For the future work, we need to define some key performance indicator to measure the improvement speed and then compare current way of working with the improved way of working. We hope to continue the experimentation process of using the other Departments to improve the application.