Measuring with Timed Patterns

Abstract

We propose a declarative measurement specification language for quantitative performance evaluation of hybrid (discrete-continuous) systems based on simulation traces. We use timed regular expressions with events to specify patterns that define segments of simulation traces over which measurements are to be taken. In addition, we associate measure specifications over these patterns to describe a particular type of performance evaluation (maximization, average, etc.) to be done over the matched signal segments. The resulting language enables expressive and versatile specification of measurement objectives. We develop an algorithm for our measurement framework, implement it in a prototype tool, and apply it in a case study of an automotive communication protocol. Our experiments demonstrate that the proposed technique is usable with very low overhead to a typical (computationally intensive) simulation.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Verification consists in checking whether system behaviors, sequences of states and events, satisfy some specifications. These specifications are expressed in a formalism, for example temporal logic, with well-defined semantics such that the satisfaction or violation of a property $\varphi$ by a behavior w can be computed based on $\varphi$ and w. To perform exhaustive formal verification, property $\varphi$ is typically converted into an automaton $\mathcal A_{\lnot \varphi }$ that accepts only violating sequences which is later composed with the system model and checked for emptiness. Such specifications are also used in a more lightweight and scalable form of verification (known as runtime verification in software and assertion checking in hardware) where individual behaviors are checked for property satisfaction. In this context, the formal specification language can be used to automatically derive property monitors rather than inspect execution traces manually or program monitors by hand. The specification formalism allows us to focus on the observable properties of the system we are interested in and write them in a declarative way, separated from their implementation. It is this concept that we export from the qualitative to the quantitative world.

Properties offer a purely qualitative way to evaluate systems and their behaviors: correct or incorrect. There are many contexts, however, where we want also to associate quantitative measures with systems and their executions. Consider for example a real-time system with both safety-critical and non-critical aspects, evaluated according to the temporal distance between pairs of request and service events. Its safety-critical part will be evaluated according to whether some distance goes beyond a hard deadline. In contrast, its non-critical part is typically evaluated based on quality-of-service performance measures which are numerical in nature, such as the average response time or throughput.

Quantitative measures are used heavily in the design of cyber-physical systems involving heterogeneous components of computational and physical natures. Such systems exhibit continuous and hybrid behaviors and are often designed using modeling languages such as Simulink, Modelica or hardware description languages. These models are analyzed using a combination of numerical and discrete-event simulation, producing traces from which performance measures are extracted to evaluate design quality. Measures are computed by applying various operations such as summation/integration, arithmetical operations, max-min, etc. to certain segments of the simulation trace. The boundaries of these segments are defined according to the occurrence of certain events and patterns in the trace. When the measures are simple they are realized by inserting additional observer blocks to the system model but when they are more complex, they are extracted using manually-written (and error prone) procedural scripts that perform computations over the traces.

Fig. 1.

Stopping distance measurement for anti-lock brake systems.

We illustrate how measurements can be used to compare two correct implementations of an anti-lock brake system (ABS), which prevents wheels from locking during heavy braking or on slippery roads. Figure 1 depicts braking control signals $b_{1}$ and $b_{2}$ and velocity signals $v_{1}$ and $v_{2}$ for two controller models $C_{1}$ and $C_{2}$. The driver starts to brake fully at $t = r$ and then the ABS takes control at $t = s$ and applies rapid pulsation to prevent locking. Both controllers $C_{1}$ and $C_{2}$ satisfy the anti-lock property but we also want to compare the distance covered during their respective breaking periods. These periods are identified as those where signal b matches some braking pattern, and are the intervals $(r, t_1)$ for $C_1$ and $(r, t_2)$ for $C_2$. Integrating $v_i$ over respective intervals $(r,t_i)$ for $i=1..2$ we get a numerical measure and conclude that $C_1$ performs better.

In this paper we propose a declarative and formal measure specification language for automatically extracting measures from hybrid discrete-continuous traces. The patterns that define the scope of measurements are expressed using a variant of the timed regular expressions (tre) of [2, 3], specially adapted for this purpose by adding preconditions, postconditions and events. An additional language layer is used to define the particular measures applied to the matching segments. The actual extraction of the measures takes advantage of the recent pattern matching procedure introduced in [19] for computing the set of segments of a Boolean signal that match a timed regular expression. In the general case, the number of such matches can be uncountable and the procedure of [19] represents them as a finite union of zones. In our language, where pattern boundaries are punctual events, we obtain a finite number of matches.

The resulting framework provides a step toward making the common practice of quantitative measurement extraction more rigorous, bridging the gap between qualitative verification and quantitative performance evaluation. We demonstrate the applicability of our approach using the Distributed System Interface (DSI3) standard protocol [15] developed by the automotive industry. We formalize in our language measurements of some features described in the standard, extract them from simulation traces and report the performance of our prototype implementation.

Related Work

The approach proposed in this paper builds upon the timed regular expressions introduced in [2, 3] and shown there to be equivalent in expressive power to timed automata. We omit the renaming operator used for this expressivity theoretical result and enrich the formalism with other features that lead to a pattern language dedicated to measurements, which we call conditional tre. Precondition and postcondition constraints allow us to express zero-duration events such as rising and falling edges of dense-time Boolean signals. Focusing on patterns that start and end with an event, the pattern matching algorithm of [19] returns a finite number of matching segments.

Our approach differs in several respects from monitoring procedures based on real-time temporal logics and their extensions to real-valued signals such as STL [16]. In a nutshell here is the difference between satisfaction in temporal logic and matching in regular expression. For any temporal logic with future operators, satisfaction of $\varphi$ by a behavior w is defined as $(w,0)\models \varphi$. To compute this satisfaction value of $\varphi$ at 0 we need to compute $(w,t)\models \psi$ for some sub-formulas $\psi$ of $\varphi$ and some time $t\ge 0$, in other words determine whether some suffix of w satisfies $\psi$. This can be achieved by associating with every formula $\varphi$ a satisfaction signal relative to w which is true for every t such that $(w,t)\models \varphi$. On the other hand, the matching of a regular expression $\varphi$ in w is not defined relative to a single time point but to a pair of points $(t,t')$ such that the segment of w between t and $t'$ satisfies the expression. This property of regular expressions makes them ideal for defining intervals that match patterns.

The recent work on assertion-based features [7] is similar in spirit to ours. The authors propose an approach for quantitative evaluation of mixed-signal design properties expressed as regular expressions. In contrast to our work, the regular expressions are extended with local variables, which are used to explicitly store values of interest, such as the beginning and the end time of a matched pattern. This work addresses the problem of measuring properties (features) of hybrid automata models using formal methods. We also mention the extension to tre proposed in [13] that combines specification of real-time events and states occurring in continuous-time signals. Their syntax and primitive constructs are inspired by and extend industrial standards PSL [10] and SVA [20]. This work focuses on a translation from tre to timed automata acceptors, but does not address the problem of pattern matching an expression on a concrete trace.

In the context of modeling resource-constrained computations, quantitative languages [6] were studied as generalizations of formal languages in which traces are associated with a real number rather than a Boolean value. The authors use weighted automata to define several classes of quantitative languages and determine the trace values by computing maximum, limsup, liminf, limit average and discounted sum over a (possibly infinite) trace. The ideas of quantitative languages are further extended in [14], by defining the model measuring problem. The model checking problems of TCTL and LTL are extended in [1, 11, 21] to a model measuring paradigm by parameterizing the bounds of the temporal operators. The authors propose algorithms for identifying minimum and maximum parameter values for which the model satisfies the temporal formula. A similar extension is proposed in [4] for signal temporal logic (STL), where both the temporal bounds and real-valued thresholds are written as parameters and inferred from signals. Robust interpretation of temporal logic specifications [8, 9, 12] is another way to associate numbers with traces according to how strongly they satisfy or violate a property.

Hardware designers and others who use block diagrams for control and signal processing often realize measurement using additional observer blocks, but these are restricted to online measurements. As a result commercial circuit simulation suites offer scripting languages or built-in functions dedicated to measurement extraction, such as the .measure (Synopsys) and .extract (Mentor Graphics) libraries. The former is structured according to the notion of trigger and target events, the measurement being performed on the segment(s) of the trace in between. This is particularly suited for timing analysis such as rise-time or propagation time. The latter is more general but relies mostly on functional composition. Absolute time of events in the trace can be found by threshold crossing functions, and then passed on as parameters to other measurement primitives to apply an aggregating function over suitable time intervals. In the approach we propose, one gains the expressiveness of the language of timed regular expression, that allow to detect complex sequences of events and states in the trace. This facilitates repeated measurements over a sequence of specified patterns, by clearly separating the behavior description from the measure itself.

2 Timed Regular Expression Patterns

In this section, we first recall the definition of the timed regular expressions (tre) from [19]. Such expressions were defined over Boolean signals and in order to use them for real-valued signals we add predicates on real values to derive Boolean signals. This straightforward extension is still not entirely suitable for defining measurement segments, for the simple reason that an arbitrary regular expression may have infinitely many matches. For example an atomic proposition p is matched by all sub-segments of a dense-time Boolean signal where p continuously holds. Consequently in the second part of this section, we propose a novel extension that we call conditional timed regular expressions (ctre). This extension enables to condition the match of a tre to a prefix and suffix, and allows defining events of zero duration. We define a restriction to ctre, that we call event-bounded timed regular expressions (e-tre), which guarantees that the set of patterns matching a e-tre is always finite. Thanks to this finiteness property, we will use e-tre as the main building block in defining our measurement specification language.

2.1 Timed Regular Expressions

Let X and B be sets of real and propositional variables and $w~:~[0,d] \rightarrow \mathbb {R}^m \times \mathbb {B}^n$, where $m = |X|$ and $n=|B|$, a multi-dimensional signal of length d. For a variable $v \in X \cup B$ we denote by $\pi _{v}(w)$ the projection of w on its component v.

A propositional variable $b\in B$ admits a negation $\lnot b$, which value at time t is the opposite of that of b. For $\theta$ a concrete predicate $\mathbb {R}\rightarrow \mathbb {B}$ we may create a propositional symbol $\theta (x)$ which interpretation at time t will be given by the evaluation of $\theta$ on the value of real variable x at time t. We define the projection of w on $\lnot b$ by letting $\pi _{\lnot b}(w)[t] = 1-\pi _{b}(w)[t]$, and the projection of w on $\theta (x)$ by letting $\pi _{\theta (x)}(w)[t]=\theta (\pi _{x}(w)[t])$. A proposition p is taken to be either a variable $b \in B$, a predicate $\theta (x)$ over some real variable x, or their negation $\lnot b$ and $\lnot \theta (x)$ respectively. We assume a given set of real predicates and take P the set of propositions derived from real and propositional variables as described. A signal is said to have finite variability if for every proposition $p \in P$ the set of discontinuities of $\pi _{p}(w)$ is finite.

We now define the syntax of timed regular expressions according to the following grammar:

$$\begin{aligned} \varphi := \epsilon ~|~p~|~\varphi _{1} \cdot \varphi _{2}~|~\varphi _{1} \cup \varphi _{2}~|~\varphi _{1} \cap \varphi _{2}~|~\varphi ^{*}~|~\langle \varphi \rangle _I \end{aligned}$$

where p is a proposition of P, and I is an interval of $\mathbb R_+$.

The semantics of a timed regular expression $\varphi$ with respect to a signal w and times $t \le t'$ in [0, d] is given in terms of a satisfaction relation $(w,t,t') \models \varphi$ inductively defined as follows:

Following the definitions in [19], we characterize the set of segments of w that match an expression $\varphi$ by their match set. The match set of expression $\varphi$ over w is the set of all pairs $(t,t')$ such that the segment of w between t and $t'$ matches $\varphi$.

Definition 1

(Match Set). For any signal w and expression $\varphi$, we define their match set as

$$\begin{aligned} \mathcal {M}(\varphi ,w) := \{ (t,t') \in \mathbb {R}^2~|~(w,t,t') \models \varphi \} \end{aligned}$$

We recall that a match set is a subset of $[0,d] \times [0,d]$ confined to the upper triangle defined by $t \le t'$ taking $t,t'$ the first and second coordinates of $\mathbb {R}^2$. It has been established that such a set can always be represented as a finite union of zones. In $\mathbb {R}^n$, zones are a special class of convex polytopes definable by intersections of inequalities of the form $x_i \ge a_i$, $x_i \le b_i$ and $x_i - x_j \le c_{i,j}$ or corresponding strict inequalities. We say that a zone is punctual when the value of each variable is uniquely defined, with for instance $a_i = b_i$ for all $i =1..n$. We use zones in $\mathbb {R}^2$ to describe the relation between t and $t'$ in a match set.

Theorem 1

([19]). For any finite variability signal w and tre $\varphi$, the set $\mathcal {M}(\varphi ,w)$ is a finite union of zones.

2.2 Conditional TRE

We propose in the sequel conditional timed regular expressions (ctre) that extend tre. This extension enables to condition the match of a tre to a prefix or a suffix. We introduce in the syntax of ctre two new binary operators, “${{\mathrm{?}}}$” for preconditions, and “${{\mathrm{!}}}$” for postconditions. For some expressions $\varphi _1$ and $\varphi _2$ a trace w matches the expression $\varphi _1 {{\mathrm{?}}}\varphi _2$ at $(t,t')$ if it matches $\varphi _2$ and there is an interval ending at t where w matches $\varphi _1$. Symmetrically w matches the expression $\varphi _1 {{\mathrm{!}}}\varphi _2$ at $(t,t')$ if it matches $\varphi _1$ and there is an interval beginning at $t'$ where w matches $\varphi _2$. We define formally the semantics of these operators for $\varphi _1$, $\varphi _2$ arbitrary ctre and w an arbitrary signal as follows:

$$\begin{aligned} \begin{array}{rlrl} (w,t,t') &{}\models \varphi _1 {{\mathrm{?}}}\varphi _2 &{}\ \leftrightarrow \ (w,t,t') &{}\models \varphi _2 \text { and } \exists t'' \le t,\, (w,t'',t) \models \varphi _1\\ (w,t,t') &{}\models \varphi _1 {{\mathrm{!}}}\varphi _2 &{}\ \leftrightarrow \ (w,t,t') &{}\models \varphi _1 \text { and } \exists t'' \ge t',\,(w,t',t'') \models \varphi _2 \end{array} \end{aligned}$$

A precondition $\varphi _1$ and a postcondition $\varphi _3$ can be associated to an expression $\varphi _2$ independently as we have $\varphi _1 {{\mathrm{?}}}(\varphi _2 {{\mathrm{!}}}\varphi _3) \equiv (\varphi _1 {{\mathrm{?}}}\varphi _2) {{\mathrm{!}}}\varphi _3$ so that such expressions may be noted $\varphi _1 {{\mathrm{?}}}\varphi _2 {{\mathrm{!}}}\varphi _3$ without ambiguity. Associating several conditions can form a sequential condition as with $(\varphi _1 {{\mathrm{?}}}\varphi _2) {{\mathrm{?}}}\varphi _3 \equiv (\varphi _1 \cdot \varphi _2) {{\mathrm{?}}}\varphi _3$, or conjoint conditions as with $\varphi _1 {{\mathrm{?}}}(\varphi _2 {{\mathrm{?}}}\varphi _3) \equiv (\varphi _1 {{\mathrm{?}}}\varphi _3) \cap (\varphi _2 {{\mathrm{?}}}\varphi _3)$. There are further relationships with respect to other tre operators, which we will not detail.

2.3 TRE with Events

Another important aspect of ctre is that they enable defining rise and fall events of zero duration associated to propositional terms. The rise edge ${{\mathrm{\uparrow }}}p$ associated to the propositional term p is obtained by syntactic sugar as ${{\mathrm{\uparrow }}}p := \lnot p {{\mathrm{?}}}\epsilon {{\mathrm{!}}}p$, while the fall edge ${{\mathrm{\downarrow }}}p$ corresponds to ${{\mathrm{\downarrow }}}p := {{\mathrm{\uparrow }}}\lnot p$. We now define a restriction of ctre that we call tre with events. This sub-class of ctre consists of restricting the use of conditional operators to the definition of events. The introduction of events in tre still guarantees the finite representation of their match set.

Corollary 1

(of Theorem 1 ). For any finite variability signal w and tre with events $\varphi$, the set $\mathcal {M}(\varphi ,w)$ is a finite union of zones.

Proof

By induction on the expression structure. For expressions of the form $\varphi = {{\mathrm{\uparrow }}}p$, the match set $\mathcal M(\varphi ,w)$ is of the form $\{ (t,t) \, : \, t \in R\}$. By finite variability hypothesis R is finite as contained in the set of discontinuities of p, and in particular $\mathcal M(\varphi ,w)$ is a finite union of punctual zones. All other operators are part of the grammar of timed regular expressions, and the proof of Theorem 1 grants us the property.

In what follows we consider events to be part of the syntax of timed regular expressions, and will just write tre instead of tre with events.

Remark.

Our support for events is minimal as compared to the real-time regular expressions of [13] where the authors use special operators $\#\#$ 0 and $\#\#$ 1 for event concatenation. Their work extends discrete-time specification languages, which have the supplementary notion of clocks noted $@({{\mathrm{\uparrow }}}c)$ with c a Boolean variable, and the implicit notion of clock context. A clock $@({{\mathrm{\uparrow }}}c)$ can then be used in conjunction with a proposition p to form a clocked event noted $@({{\mathrm{\uparrow }}}c)\, p$. Such an event allows to probe the value of p at the exact times where ${{\mathrm{\uparrow }}}c$ occurs, which we did not consider. Assuming we dispose of atomic expressions $@({{\mathrm{\uparrow }}}c) \, p$ holding punctually at times such that ${{\mathrm{\uparrow }}}c$ occurs and p is true, the event concatenation $\#\#$ 1 can be emulated by $@({{\mathrm{\uparrow }}}c) \, p~\#\#\mathtt{1} \, @({{\mathrm{\uparrow }}}d) \, q \equiv \ @({{\mathrm{\uparrow }}}c) \, p \cdot d^* \cdot \lnot d \,\cdot \,@({{\mathrm{\uparrow }}}d) \, q$.

We now say that a tre is event-bounded when of the form ${{\mathrm{\uparrow }}}p$, $\psi _1 \cdot \varphi \cdot \psi _2$, $\psi _1 \cup \psi _2$, or $\psi _1 \cap \varphi$ with p a proposition, and $\psi _1, \psi _2$ event-bounded tre. Such expressions, that we call e-tre for short, have an important “well-behaving” property as follows. Given an arbitrary finitely variable signal w, an e-tre can be matched in w only a finite number of times. In the following lemma, we demonstrate that the match set for arbitrary finite signal w and e-tre $\psi$ consists of a finite number of points $(t,t')$ with t an occurrence of a begin event and $t'$ an occurrence of an end event.

Lemma 1

Given an e-tre $\psi$ and a signal w, their associated match set $\mathcal {M}(\psi ,w)$ is finite.

Proof

By induction on the expression structure. Consider an arbitrary signal w and an event ${{\mathrm{\uparrow }}}p$; by finite variability assumption there are finitely many time points in w where ${{\mathrm{\uparrow }}}p$ occurs, so that its match set relatively to w is finite. Now let $\psi$ be an e-tre of the form $\psi = \psi _1 \cdot \varphi \cdot \psi _2$. The signal w matches $\psi$ on the segment $(t,t')$ if and only if there exists some times s and $s'$ such that w matches $\psi _1$ on (t, s) and matches $\psi _2$ on $(s',t')$. By induction hypothesis there are finitely many such times t, $t'$, s and $s'$ so that $\psi$ itself has a finite number of matches. One can easily see that the finiteness of the match set is also preserved by unions and intersections $\psi _1 \cup \psi _2$ and $\psi _1 \cap \varphi$, which concludes our proof.

3 Measuring with Conditional TRE

In this section, we propose a language for describing mixed-signal measures, and a procedure to compute such measures. In our approach, we will use measure patterns based on timed regular expressions to specify signal segments of interest. More precisely, a measure pattern consists of three parts: (1) the main pattern; (2) the precondition; and (3) the postcondition. The main pattern is an e-tre that specifies the portion of the signal over which the measure is taken. Using e-tre to express main patterns ensures the finiteness of signal segments, while pre- and post- conditions expressed as general tre allow to define additional constraints. We formally define measure patterns as follows.

Definition 2

(Measure Pattern). A measure pattern $\varphi$ is a ctre of the form $\alpha {{\mathrm{?}}}\psi {{\mathrm{!}}}\beta$, where $\alpha$ and $\beta$ are tre, while $\psi$ is an e-tre.

Note that preconditions and postconditions can be made optional by using $\epsilon$ as we have $\epsilon {{\mathrm{?}}}\varphi \equiv \varphi$ and $\varphi {{\mathrm{!}}}\epsilon \equiv \varphi$. In what follows we may use simpler formulas to express their semantic equivalent, for instance writing $\varphi$ to refer to the measure pattern $\epsilon {{\mathrm{?}}}\varphi {{\mathrm{!}}}\epsilon$.

According to previous definitions, the match set of a measure pattern $\alpha {{\mathrm{?}}}\psi {{\mathrm{!}}}\beta$ gives us the set of all segments of the signal, represented as couples $(t,t')$, such that $(w,t,t') \models \psi$, and w satisfies both the precondition $\alpha$ before t and the postcondition $\beta$ after $t'$.

Proposition 1

For any signal w and a pattern $\varphi = \alpha {{\mathrm{?}}}\psi {{\mathrm{!}}}\beta$, their associated match set set is given by

$$\begin{aligned} \begin{array}{rll} \mathcal M(\varphi , w) = \{ (t,t') \, : \, \exists s \le t \le t' \le s', &{} (w,s,t) \models \alpha &{}\\ \text {and } &{} (w,t,t') \models \psi &{}\\ \text {and } &{} (w,t',s') \models \beta &{}\} \end{array} \end{aligned}$$

Theorem 2

(Match Set Finiteness). For any signal w and measure pattern $\varphi =\alpha {{\mathrm{?}}}\psi {{\mathrm{!}}}\beta$, their associated match set $\mathcal M(\varphi ,w)$ is finite.

Proof

This is a direct consequence of Lemma 1. The set $\mathcal M(\varphi , w)$ is included in $\mathcal M(\psi ,w)$, which makes it finite.

The match set of a measure pattern may be obtained by selecting the punctual zones of $\mathcal M(\psi ,w)$ that meet a zone of $\mathcal M(\alpha ,w)$ at the beginning, and a zone of $\mathcal M(\beta ,w)$ at the end. Match sets of arbitrary tre are computable following the proof of Theorem 1. The overall procedure to compute the match set of a measure pattern appears as Algorithm 1. It uses the procedure zones $(\varphi ,w)$ as appearing in [19] which returns a set of zones whose union is equal to $\mathcal M(\varphi , w)$ for any timed regular expression $\varphi$ and signal w. For a zone z we denote by $\pi _1(z)$ and $\pi _2(z)$ projections on its first and second coordinates respectively.

The computation of a match set for a measure pattern $\varphi$ and a signal w enables powerful pattern-driven performance evaluation of hybrid or continuous systems. Once the associated match set $\mathcal M(\varphi , w)$ is computed, we propose a two stage analysis of signals.

In the first step, we compute a scalar value for each segment of w that matches $\varphi$, either from absolute times of that match, or from the values of a real signal x in w during that match. A measure is then written with the syntax $\mathrm {op}(\varphi )$ with $\mathrm {op} \in \{ \mathsf time , \mathsf value _x, \mathsf duration , \mathsf inf _x, \mathsf sup _x, \mathsf integral _x, \mathsf average _x \}$ being some sampling or aggregating operator. The semantics $[\![\,]\!]_w$ of these operators is given in Table 1; it associates to a measure $\mathrm {op}(\varphi )$ and trace w a multiset containing the scalar values computed over each matched interval.¹

Table 1. Standard measure operators.

In the second step, we reduce the multiset of scalar values computed over the signal matched intervals in $\mathcal M(\varphi , w)$ to a single scalar. Typically, given the multiset $A=[\![\mathrm {op}(\varphi )]\!]_w$ of scalar values associated with these signal segments, this phase consists in computing standard statistical indicators over A, such as the average, maximum, minimum or standard deviation. This final step is optional, the set of basic measurements sometimes provides sufficient information.

Anti-lock Brake System Example.

We now refer back to our first example from Fig. 1 and propose measure pattern formalization to evaluate performance of the controller. We first formalize the pattern of a brake control signal b under a (heavy) braking situation. The main pattern $\psi$ starts with a rise event on b and a braking period with the duration in I, continues with one or more pulses with duration in J, and ends with a fall event on b:

$$\begin{aligned} \psi :=\ \uparrow b \cdot \langle b\rangle _{I} \cdot \langle \lnot b\cdot b\rangle _{J}^{+} \cdot \downarrow b \end{aligned}$$

We also need to ensure that the speed should be zero at the end of braking situation, with the postcondition $\beta := (v \le 0)$. Finally, we can measure the stopping distance using the expression

$$\begin{aligned} \mathsf integral _v(\psi {{\mathrm{!}}}\beta ) \end{aligned}$$

integrating v over intervals matching the measure pattern.

4 Case Study

4.1 Distributed Systems Interface

Distributed systems interface (DSI3) is a flexible and powerful bus standard [15] developed by the automotive industry. It is designed to interconnect multiple remote sensor and actuator devices to a controller. The controller interacts with the sensor devices via so-called voltage and current lines. In this paper we focus on two phases of the DSI3 protocol:

• the initialization phase called the discovery mode;

• one of the stationary phases called the command and response mode.

In the discovery mode, prior to any interaction the power is turned on, resulting in a voltage ramp from $0\mathrm {V}$ to $V_{high }$. The communication is initiated by the controller that probes the presence/absence of sensors by emitting analog pulses on the voltage line. Connected sensor devices respond in turn with another pulse sent over the current line. At the end of this interaction, a final short pulse is sent to the sensors interfaces, marking the end of the discovery mode.

In the command and response mode, the controller sends a command to the sensor as a series of pulses (or pulse train) on the voltage line, which transmits its response by another pulse train on the current line. For power-demanding applications the command-response pairs are followed by a power pulse, which goes above $V_{h igh}$. This allows the sensor to load a capacitor used for powering its internal operation.

The DSI3 standard provides a number of ordering and timing requirements that determine correct communication between the controller and the sensor devices: (1) minimal time between the power turned on and first discovery pulse; (2) maximal duration of discovery mode; (3) expected time between two consecutive discovery pulses; (4) expected time between command and response. Figure 2 illustrates the discovery mode in the DSI3 protocol and provides a high-level overview of its ordering and timing requirements. In this example, the controller probes five sensor interfaces.

Fig. 2.

DSI3 discovery mode – overview.

The correctness of a DSI3 protocol implementation in an automotive airbag system was studied in [17]. The above requirements were formalized as assertions expressed in signal temporal logic (STL) and the monitoring tool AMT [18] was used to evaluate the simulation traces. In this paper we do more than checking correctness, but evaluate the performance of a controller and sensor implementation. We use measure patterns to specify signal segments of interest and define several measures within the framework introduced in Sect. 3. We study two specific measures: (1) the time between consecutive discovery pulses; and (2) the amount of energy transmitted to the sensor through power pulses.

In order to generate simulation traces, we model our system as follows: the controller is a voltage-source, and the sensor is a current-source in parallel with a resistive-capacitive load. The schematic is shown in Fig. 3. During the discovery phase the load is disabled; the voltage source generates randomized pulses in which the time between two discovery pulses has a Gaussian distribution with a mean of $250\mu \text {s}$ and a standard deviation of $3.65\mu \text {s}$. During the power pulses of the command and response mode, the load is enabled and randomized, with $C=120\text {nF}$ and R uniformly distributed in the range $[25\Omega , 35\Omega ]$. Threshold levels are $4.6\mathrm V$ low, $7.8\mathrm V$ high, $8.3\mathrm V$ power, and $11.5\mathrm V$ idle.

Fig. 3.

Electrical model of the system.

4.2 Measurements

Time Between Consecutive Discovery Pulses.

In order to characterize a discovery pulse, we first define three regions of interest – when the voltage v is (1) below $V_{low }$; (2) between $V_{low }$ and $V_{high }$; and (3) above $V_{high }$. We specify these regions with the following predicates:

$$\begin{aligned} \begin{array}{lcl} v_{l} &{} \equiv &{} v \le V_{low } \\ v_{b} &{} \equiv &{} V_{low } \le v \le V_{high } \\ v_{h} &{} \equiv &{} v \ge V_{high } \\ \end{array} \end{aligned}$$

Next, we describe the shape of a discovery pulse. Such a pulse starts at the moment when the signal v moves from $v_{h}$ to $v_{b}$. The signal then must go into $v_{l}$, $v_{b}$ and finally come back to $v_{h}$. In addition to its shape, the DSI3 specification requires the discovery pulse to have a certain duration between some $d_{min }$ and $d_{max }$. This timing requirement allows distinguishing a discovery pulse from other pulses, such as the end-of-discovery pulse. We illustrate the requirements for a discovery pulse in Fig. 4-a and formalize it with the following e-tre:

$$\begin{aligned} \begin{array}{lcl} \psi _{dp } &{} \equiv &{} \downarrow (v_{h}) \cdot \langle v_{b} \cdot v_{l} \cdot v_{b} \rangle _{[d_{min }, d_{max }]} \cdot \uparrow (v_{h}) \\ \end{array} \end{aligned}$$

In order to measure the time between consecutive discovery pulses, we need to characterize signal segments that we want to measure. The associated pattern shall start at the beginning of a discovery pulse and end at the beginning of the next one, as depicted by the $\psi$ region in Fig. 4-a. It consists of a discovery pulse $\psi _{dp}$, followed by the voltage signal being in the $v_{h}$ region, and terminating when the voltage leaves $v_{h}$. This description is not sufficient – we also need to ensure that this segment is effectively followed by another discovery pulse. Hence we add a postcondition that specifies this additional constraint. The measure pattern $\varphi _1 \equiv \alpha _1 {{\mathrm{?}}}\psi _1 {{\mathrm{!}}}\beta _1$ is formalized as follows.

$$\begin{aligned} \begin{array}{lcl} \alpha _1 &{} \equiv &{} \epsilon \\ \psi _1 &{} \equiv &{} \psi _{dp} \cdot v_{h} \cdot \downarrow (v_{h}) \\ \beta _1 &{} \equiv &{} \psi _{dp} \\ \end{array} \end{aligned}$$

Finally, we evaluate the measure expression $\mu _1 := \mathsf duration (\varphi _1)$ over signal w.

Fig. 4.

(a) Consecutive discovery pulses with timing; (b) Power pulse and flow.

Energy Transfer from Controller to Sensor. In the DSI3 protocol, the discovery mode can be followed by a stationary command and respond mode. A command and respond mode sequence is a pulse train that consists of a command subsequence in the form of potential pulses between $V_{h igh}$ and $V_{l ow}$, a response subsequence by means of current pulses between 0 and $I_{r esp}$, and finishes by a power pulse rising to potential $V_{i dle}$ in which a large current can be drawn by the sensor. We first characterize the power pulse as depicted in Fig. 4-b. It occurs when the voltage goes from below $V_{p wr}$ to above $V_{idle }$, and back under $V_{p wr}$. The three regions of interest are specified with the following predicates.

$$ \begin{array}{lcl} v_{h} &{} \equiv &{} v \ge V_{pwr } \\ v_{t} &{} \equiv &{} V_{pwr } \le v \le V_{idle } \\ v_{p} &{} \equiv &{} v \ge V_{idle } \end{array} $$

Hence the pattern specifying a power pulse is expressed as

$$ \psi _2 \equiv {{\mathrm{\uparrow }}}(v_{h}) \cdot v_{t} \cdot v_{p} \cdot v_{t} \cdot {{\mathrm{\downarrow }}}(v_{h}) $$

The measure pattern does not have pre- or post-conditions as all other communications occur with v below $V_{i dle}$, hence $\alpha _2 = \beta _2 = \epsilon$. The measure pattern $\varphi _2$ is equivalent to its main pattern $\psi _2$. Given v the voltage and i the current on the communication line, the energy transfered to the sensor is given by the area under the signal $v \times i$ between the start and end of power pulse. We assume that such a signal is given in the simulation trace w, and evaluate the measure expression $\mu _2 := \mathsf integral _{v \times i}(\psi _2)$ over signal w.

4.3 Experimental Results

We extended the prototype tool developed in [19] with algorithms for matching zero-duration events and conditional tre as appearing in measure patterns, and with the support of measure operations introduced in Sect. 3. The implementation was done in Python and uses the C library from IF [5] for computing operations on zones. For our experiment we apply a scenario according to which our electrical model is switched on/off 100 times in sequence to stress the discovery mode of DSI3. The set of traces we generate conform to the discovery, and command-and-response modes of the protocol. We then compute match sets for properties presented in Sect. 4.2 over these simulation traces using our prototype implementation. In Fig. 5, we depict measurement results using histograms. The distribution of the times between two discovery pulses follows a normal distribution according to the timing parameters used to generate it. The energy transfered to the sensor through power pulses has a flatter distribution as the result of a uniformly distributed load resistance value.

Fig. 5.

(a) Distribution of $\mu _1$, the time between two consecutive discovery pulses; (b) Distribution of $\mu _2$, the energy transmitted per power pulse.

We then compared the execution times to compute measurements, using a periodic sampling with different sampling rates – note that our method supports variable step sampling without extra cost. The computation times are given in Table 2 with the detailed computation time needed for predicate evaluation ($T_p$), match set computation ($T_m$), measure aggregation ($T_a$) and total computation time (T). Computation of match sets does not depend on the number of samples but on the number of uniform intervals of atomic propositions; evaluation of real predicates by linear interpolation, and computing measures like integration can be done in time linear in the number of samples.

Table 2. Computation times (s)

5 Conclusion and Future Work

We presented a formal measurement specification language that can be used for evaluating cyber-physical systems based on their simulation traces. Starting from a declarative specification of the patterns that should be matched in the segments to be measured, we apply a pattern matching algorithm for timed regular expressions to find out the scope of measurements. The applicability of our framework was demonstrated on a standard mixed-signal communication protocol from the automotive domain.

In the future, we plan to develop an online extension of the presented pattern matching and measurement procedure. It will enable the application of measurements during the simulation process as well as performing measurements on real cyber-physical systems during their execution. We believe that the extension of regular expressions that we introduced is sufficiently expressive to capture common mixed signal properties, and could be used in other application domains, something that we intend to explore further.