A multiplatform energy-aware OWL reasoner benchmarking framework

https://doi.org/10.1016/j.websem.2021.100694Get rights and content

Abstract

Performance evaluation is increasingly relevant for Web Ontology Language (OWL) reasoners, due to the expanding availability of knowledge corpuses on the Web, the growing variety of applications, and the rise to prominence of mobile and pervasive computing. Motivated mainly by the difficulty of comparing reasoning engines in the Semantic Web of Things (SWoT), this paper introduces evOWLuator, a novel approach and a multiplatform framework devised to be both flexible and expandable. It features integration of traditional and mobile/embedded engines as well as ontology dataset management, reasoning test execution, and report generation. A case study consisting of an experimental setting for time, memory peak and energy footprint evaluation with eight reasoners and four different platforms allows showcasing usage and validating features and usability of the tool.

Section snippets

Overview and motivation

One of the fundamental standards underpinning the Semantic Web is the Web Ontology Language (OWL), currently at version 2 [1]. It is used to create ontologies, i.e., vocabularies endowed with a formal meaning which grounds terminological characterizations. OWL 2 semantics are based on the SROIQ Description Logic (DL), a fragment of First Order Logic (FOL).

Automated reasoning is a process to infer implicit knowledge from what has been explicitly declared in an ontology, and to answer specific

Preliminaries

In the semantics of OWL, basic elements include: classes (14concepts in DL jargon), representing sets of objects; object properties (14roles), linking pairs of objects; data properties (14functional roles on concrete domains), linking objects with data values (14 literals); individuals (14instances), representing specific objects. These elements can be combined using constructors to form class expressions, which can be used in sets of inclusion assertions and definitions called TBoxes

Case study: benchmarking ontology classification and consistency

evOWLuator is a multiplatform software framework devised to assess the correctness, performance and energy footprint of inference services exposed by OWL reasoners. In order to promote its adoption in both academic and industrial contexts, it is released under the Eclipse Public License (EPL) version 2.0.3

A small experimental campaign has been carried out to validate evOWLuator’s effectiveness and features as well as to

OWL reasoner evaluation framework

evOWLuator has been designed with flexibility in mind, particularly concerning the ability to test multiple reasoning engines and the capability to run inference services on mobile and embedded devices.

To achieve these goals, evOWLuator follows the object-oriented paradigm: user configuration involves extending Python abstract base classes in the framework with concrete subclasses implementing their parents’ interfaces. Compared to a declarative solution, e.g., structured configuration files,

Related work

Approximately until 2010, reasoner evaluation has been dominated by benchmarks based on a relatively low number of ontologies and small sets of hand-crafted queries [28]. The Lehigh University Benchmark (LUBM) [29] is one of the most representative and popular specimens: it consists of one ontology on the domain of universities, fourteen extensional queries testing several properties, and a synthetic data generator to create scalable ABoxes. In [30] LUBM was used together with other three

Conclusion

This paper has proposed a novel multi-platform and energy-aware framework for OWL reasoner benchmarking. The first release allows evaluating correctness, performance (time and memory peak) and energy footprint of a set of standard and non-standard reasoning tasks. Support for both desktop and mobile platforms, scalability and flexibility are some of the most relevant core features. Automatic report generation in tabular and plot forms facilitates the presentation of experimental outcomes.

Future

CRediT authorship contribution statement

Floriano Scioscia: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Writing – original draft, Writing – review & editing. Ivano Bilenchi: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Writing – original draft, Writing – review & editing. Michele Ruta: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Writing – original draft, Writing – review & editing. Filippo

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

All authors approved the version of the manuscript to be published.

Funding

This work has been supported by Italian Ministry of Economic Development R&D project BARIUM5G (Blockchain and ARtificial Intelligence for Ubiquitous coMputing via 5G) and by Italian PON project RPASInAir (Remotely Piloted Aircraft Systems Integration in non-segregated Air space for services).

Intellectual property

We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property.

References (39)

  • JaraA.J. et al.

    Semantic Web of Things: an analysis of the application semantics for the IoT moving towards the IoT convergence

    Int. J. Web Grid Serv.

    (2014)
  • NouraM. et al.

    Automatic knowledge extraction to build Semantic Web of Things applications

    IEEE Internet Things J.

    (2019)
  • AbbasN. et al.

    Mobile Edge Computing: A survey

    IEEE Internet Things J.

    (2017)
  • GrimmS. et al.

    Embedded EL+ reasoning on programmable logic controllers

  • Bermúdez-EdoM. et al.

    Semantic challenges for the variety and velocity dimensions of Big Data

    Int. J. Semant. Web Inf. Syst.

    (2016)
  • RutaM. et al.

    Mini-ME Swift: the first OWL reasoner for iOS

  • SciosciaF. et al.

    Mini-ME matchmaker and reasoner for the Semantic Web of Things

  • GuarinoN. et al.

    What is an ontology?

  • BaaderF. et al.

    The Description Logic Handbook

    (2002)
  • Cited by (8)

    • Future data center energy-conservation and emission-reduction technologies in the context of smart and low-carbon city construction

      2023, Sustainable Cities and Society
      Citation Excerpt :

      The energy consumption of switches accounts for approximately 15% of the IT equipment (Moazamigoodarzi, Gupta, & Pal, 2020). A 48-port switch consumes approximately 116–190 watts when fully loaded (Scioscia, Bilenchi, & Ruta, 2022). As can be seen, reducing energy consumption in DC by adjusting the transfer rate of the switch ports is inefficient.

    • A multiplatform reasoning engine for the Semantic Web of Everything

      2022, Journal of Web Semantics
      Citation Excerpt :

      An early experimental campaign has been carried out to evaluate the computational performance of Tiny-ME in Ontology Classification and non-standard inference tasks. Tests have been executed on desktop and mobile platforms by means of the evOWLuator [48] framework.24 The desktop testbed is an Apple Mac Mini (2014)25, while mobile tests have been carried out on an Apple iPhone 726, and a HTC/Google Nexus 9 tablet.27

    • Tiny-ME Wasm: Description Logics Reasoning in Your Browser

      2024, Communications in Computer and Information Science
    • Construction of Learning Process Ontology and Application of SWRL Rules

      2023, 16th International Conference on Advanced Computer Theory and Engineering, ICACTE 2023
    View all citing articles on Scopus
    View full text