Abstract
Conformance checking techniques compare observed behavior (i.e., event logs) with modeled behavior for a variety of reasons. For example, discrepancies between a normative process model and recorded behavior may point to fraud or inefficiencies. The resulting diagnostics can be used for auditing and compliance management. Conformance checking can also be used to judge a process model automatically discovered from an event log. Models discovered using different process discovery techniques need to be compared objectively. These examples illustrate just a few of the many use cases for aligning observed and modeled behavior. Thus far, most conformance checking techniques focused on replay fitness, i.e., the ability to reproduce the event log. However, it is easy to construct models that allow for lots of behavior (including the observed behavior) without being precise. In this paper, we propose a method to measure precision of process models, given their event logs by first aligning the logs to the models. This way, the measurement is not sensitive to non-fitting executions and more accurate values can be obtained for non-fitting logs. Furthermore, we introduce several variants of the technique to deal better with incomplete logs and reduce possible bias due to behavioral property of process models. The approach has been implemented in the ProM 6 framework and tested against both artificial and real-life cases. Experiments show that the approach is robust to noise and applicable to handle logs and models of real-life complexity.
Similar content being viewed by others
Notes
For the reader not familiar with Petri nets, a Petri net is a bipartite graph that contains two types of nodes: places (circles) and transitions (boxes). A place may contain tokens (black dots), and a transition can fire if its predecessor places contain a token. When fired, a transition removes a token from each input place and adds a token to each successor place.
The distance function can be user-defined, but for simplicity we use a default distance function that assigns unit costs to moves in log/model only.
For the sake of readability, in the figures, we use the label abc as an abuse of notation for referring to the sequence \(\langle a,b,c \rangle.\)
Notice that, for the case of Petri nets with one unique initial and final markings, the set of all reversed complete activity sequences can be generated by simulating the behavior of a net obtained from the original net by reversing its arcs and swapping their initial with final marking.
References
Adriansyah A, van Dongen B, van der Aalst W (2011a) Conformance checking using cost-based fitness analysis. In: Proceedings of the 2011 IEEE 15th international enterprise distributed object computing conference (EDOC ’11), Helsinki, Finland. IEEE Computer Society, pp 55–64
Adriansyah A, Sidorova N, van Dongen B (2011b) Cost-based fitness in conformance checking. In: 11th International conference on application of concurrency to system design (ACSD), pp 57–66
Adriansyah A, van Dongen B, van der Aalst W (2013a) Memory-efficient alignment of observed and modeled behavior. Technical report BPM-03-03, BPMcenter.org
Adriansyah A, Munoz-Gama J, Carmona J, van Dongen B, van der Aalst W (2013b) Alignment based precision checking. In: Rosa ML, Soffer P (eds) Business process management workshops. Lecture notes in business information processing, vol 132. Springer, Berlin, pp 137–149
Banescu S, Zannone N (2011) Measuring privacy compliance with process specifications. In: Proceedings of the 2011 third international workshop on security measurements and metrics (METRISEC ’11). IEEE Computer Society, pp 41–50
Buijs J, van Dongen B, van der Aalst W (2012a) On the role of fitness, precision, generalization and simplicity in process discovery. In: Meersman R, Panetto H, Dillon T, Rinderle-Ma S, Dadam P, Zhou X, Pearson S, Ferscha A, Bergamaschi S, Cruz I (eds) On the move to meaningful internet systems (OTM 2012). Lecture notes in computer science, vol 7565. Springer, Berlin, pp 305–322
Buijs J, van Dongen B, van der Aalst W (2012b) Towards cross-organizational process mining in collections of process models and their executions. In: Daniel F, Barkaoui K, Dustdar S (eds) Business process management workshops. Lecture notes in business information processing, vol 100. Springer, Berlin, pp 2–13
Cook J, Wolf A (1999) Software process validation: quantitatively measuring the correspondence of a process to a model. ACM Trans Softw Eng Methodol (TOSEM) 8:147–176
de Weerdt J, de Backer M, Vanthienen J, Baesens B (2011a) A critical evaluation study of model-log metrics in process discovery. In: Muehlen M, Su J (eds) Business process management workshops. Lecture notes in business information processing, vol 66. Springer, Berlin, pp 158–169
de Weerdt J, de Backer M, Vanthienen J, Baesens B (2011b) A robust F-measure for evaluating discovered process models. In: IEEE symposium series on computational intelligence. IEEE, pp 148–155
Dijkman R, Dumas M, Ouyang C (2008) Semantics and analysis of business process models in BPMN. Inf Softw Technol 50(12):1281–1294
Dumas M, Rosa ML, Mendling J, Reijers H (2013) Fundamentals of business process management. Springer, Berlin
Gerke K, Cardoso J, Claus A (2009) Measuring the compliance of processes with reference models. In: Proceedings of the confederated international conferences, CoopIS, DOA, IS, and ODBASE 2009 on. On the move to meaningful internet systems: part I (OTM ’09). Springer, Berlin, pp 76–93
Greco G, Guzzo A, Pontieri L, Sacca D (2006) Discovering expressive process models by clustering log traces. IEEE Trans Knowl Data Eng 18:1010–1027
Hofstede A, van der Aalst W, Adams M, Russell N (2010) Modern business process automation. Springer, Berlin
Kristensen L, Schmidt K, Valmari A (2006) Question-guided stubborn set methods for state properties. Formal Methods Syst Des 29(3):215–251
Munoz-Gama J, Carmona J (2010) A fresh look at precision in process conformance. In: Proceedings of the 8th international conference on business process management (BPM ’10). Springer, Berlin, pp 211–226
Munoz-Gama J, Carmona J (2011) Enhancing precision in process conformance: stability, confidence and severity. In: Proceedings of the IEEE symposium on computational intelligence and data mining (CIDM 2011), April 11–15, Paris, France. IEEE, pp 184–191
Munoz-Gama J, Carmona J (2012) A general framework for precision checking. Int J Innov Comput Inf Control (IJICIC) 8(7B):5317–5339
Murata T (1989) Petri nets: properties, analysis and applications. Proc IEEE 77(4):541–580
Pande P, Neuman R, Cavanagh R (2000) The six sigma way: how GE, Motorola, and other top companies are honing their performance. McGraw-Hill, New York
Petkovic M, Prandi D, Zannone N (2011) Purpose control: did you process the data for the intended purpose? In: Jonker W, Petkovic M (eds) Secure data management. Lecture notes in computer science, vol 6933. Springer, Berlin, pp 145–168
Porter L, Parker A (1993) Total quality managementthe critical success factors. Total Qual Manag 4(1):13–22
Rozinat A, van der Aalst W (2008) Conformance checking of processes based on monitoring real behavior. Inf Syst 33:64–95
Rozinat A, de Jong I, Günther C, van der Aalst W (2009) Process Mining applied to the test process of wafer steppers in ASML. IEEE Trans Syst Man Cybern Part C 39:474–479
Schmidt K (1999) Stubborn sets for standard properties. In: Proceedings of the 20th international conference on application and theory of petri nets. Springer, London, pp 46–65
van der Aalst W (2011a) Business process configuration in the cloud: how to support and analyze multi-tenant processes? In: Zavattaro G, Schreier U, Pautasso C (eds) Proceedings of the 9th IEEE European conference on web services (ECOWS 2011). IEEE Computer Society Press, pp 3–10
van der Aalst W (2011b) Process mining: discovery, conformance and enhancement of business processes. Springer, New York
van der Aalst W, van Hee K (2004) Workflow management: models, methods, and systems. MIT Press, Cambridge, MA
van der Aalst W, Rubin V, Verbeek H, van Dongen B, Kindler E, Gunther C (2010) Process mining: a two-step approach to balance between underfitting and overfitting. Softw Syst Model 9(1):87–111
van der Aalst W, Adriansyah A, van Dongen B (2011) Causal nets: a modeling language tailored towards process discovery. In: Proceedings of the 22nd international conference on concurrency theory (CONCUR ’11). Springer, Berlin, pp 28–42
van der Aalst W, Adriansyah A, van Dongen B (2012) Replaying history on process models for conformance checking and performance analysis. Wiley Interdiscip Rev Data Min Knowl Discov 2(2):182–192
vanden Broucke S, de Weerdt J, Baesens B, Vanthienen J (2012) Improved artificial negative event generation to enhance process event logs. In: Ralyt J, Franch X, Brinkkemper S, Wrycza S (eds) Advanced information systems engineering. Lecture notes in computer science, vol 7328. Springer, Heidelberg, pp 254–269
vanden Broucke S, de Weerdt J, Vanthienen J, Baesens B (2013) A comprehensive benchmarking framework (CoBeFra) for conformance analysis between procedural process models and event logs in ProM. In: Proceedings of the IEEE symposium on computational intelligence and data mining (CIDM 2013), part of the IEEE symposium series on computational intelligence 2013 (SSCI 2013)
Weidlich M, Polyvyanyy A, Desai N, Mendling J (2010) Process compliance measurement based on behavioural profiles. In: Proceedings of the 22nd international conference on Advanced information systems engineering (CAiSE ’10). Springer, Berlin, pp 499–514
Weijters A, van der Aalst W, de Medeiros AA (2006) Process mining with the heuristics miner-algorithm. BETA working paper series, WP 166. Technical report, Eindhoven University of Technology, Eindhoven
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Adriansyah, A., Munoz-Gama, J., Carmona, J. et al. Measuring precision of modeled behavior. Inf Syst E-Bus Manage 13, 37–67 (2015). https://doi.org/10.1007/s10257-014-0234-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10257-014-0234-7