Dezyne: Implementing and Testing Components

2024-04-28 5275 words 25 minutes

/posts/dezyne-test-impl/testing-motor-impl.png

Contents

Introduction

In this article, I will present an implementation for a motor in Dezyne. The implementation will build upon the MotorGlue introduced in a previous article, incorporating error handling and an asynchronous protocol. Firstly, I will introduce the necessary interfaces for composing the desired behavior and present the Motor interface. Subsequently, I will provide an implementation for the Motor interface, referred to as MotorImpl. Following that, I will discuss the aspects that require testing and those that are already handled by the Dezyne verification. I will then demonstrate how to create mocks for the Dezyne interfaces. Finally, I will create fixtures and tests to assess the behavior of the MotorImpl component.

All examples in this article are based on Dezyne version 2.18.

Let’s begin by introducing the required interfaces.

The Interfaces

The MotorImpl component will provide the Motor interface and require the following interfaces:

Config
Timer
Async
MotorGlue

Motor Interface

The Motor interface follows a strict design to enable proper verification of its usage in a design by contract fashion. The Initialize and Terminate functions serve as the constructor and destructor, respectively. Prior to usage, the motor must transition to the Operational state. Starting the motor will home the motor axis. Once the motor is in the operational state, the Move command can be used to initiate an asynchronous movement, while the StopMoving command can be used to halt the motor’s movement. In the event of an error, the motor will transition to an error state. While in the error state, the motor cannot be used until it is recovered and restarted. To maintain conciseness, certain details have been omitted from the example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
interface Motor {
    in Result Initialize(in string id);
    in void Terminate();
    in void Start();
    in void Stop();
    in void Recover();
    in void Move(in double position);
    in void StopMoving();

    out void OnOk();
    out void OnError();

    behavior {
        enum State {
            Uninitialized,
            Idle,
            IdleError,
            Starting,
            Operational,
            OperationalError,
            Moving,
        };
        State state = State.Uninitialized;

        [state.Uninitialized]
        {
            on Initialize: { reply(Result.Ok); state = State.Idle; }
            on Initialize: reply(Result.Fail);
            on Terminate: {}
        }

        [state.Idle]
        {
            on Start: state = State.Starting;
            on Stop: {}
            on Recover: {}
            on Terminate: state = State.Uninitialized;
        }

        [state.IdleError]
        {
            on Terminate: state = State.Uninitialized;
            on Stop: {}
            on Recover: state = State.Idle;
        }

        [state.Starting]
        {
            on Stop: state = State.Idle;
            on optional: { OnOk; state = State.Operational; }
            on inevitable: { OnError; state = State.IdleError; }
        }

        [state.Operational]
        {
            on Move: state = State.Moving;
            on Stop: state = State.Idle;
            on StopMoving: {}
        }

        [state.OperationalError]
        {
            on Stop: state = State.IdleError;
            on StopMoving: {}
        }

        [state.Moving]
        {
            on StopMoving: state = State.Operational;
            on optional: { OnOk; state = State.Operational; }
            on inevitable: { OnError; state = State.OperationalError; }
        }
    }
}

Config Interface

The Config interface provides access to the motor’s configuration. It includes two in-events that can be used to query the home and move timeout for the motor with the specified id.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
interface Config {
    in Result GetHomeTimeout(in string id, out double homeTimeout);
    in Result GetMoveTimeout(in string id, out double moveTimeout);

    behavior {
        on GetHomeTimeout, GetMoveTimeout:
        {
            [true] reply(Result.Ok);
            [true] reply(Result.Fail);
        }
    }
}

The interface can be implemented by utilizing a foreign component that has access to the configuration file and exposes its content accordingly. The configuration file could have a structure similar to the following example:

{
    "motor": {
        "xaxis": {
            "home_timeout": 5.0,
            "move_timeout": 3.0,
        },
        "yaxis": {
            "home_timeout": 10.0,
            "move_timeout": 6.0,
        }
    }
}

Timer Interface

The Timer interface allows for managing timeouts. This interface can be implemented by utilizing a foreign component.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
interface Timer {
    in void Create(double timeout);
    in void Cancel();

    out void OnTimeout();

    behavior {
        enum State {
            Idle,
            Armed
        };
        State state = State.Idle;

        [state.Idle]
        {
            on Create: state = State.Armed;
            on Cancel: {}
        }

        [state.Armed]
        {
            on Cancel: state = State.Idle;
            on inevitable: { OnTimeout; state = State.Idle; }
        }
    }
}

Async Interface

The Async interface is as described in the earlier Async post about asynchronous operations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
interface Async {
    in void req();
    in void clr();

    out void ack();

    behavior {
        bool armed = false;

        on clr: armed = false;
        [!armed] on req: armed = true;
        [armed] on inevitable: { ack; armed = false; }
    }
}

MotorGlue Interface

The MotorGlue interface was introduced in the previous post about testing glue.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
interface MotorGlue {
    in Result Initialize(in string id);
    in void Terminate();
    in Result Home();
    in Result IsHoming();
    in Result Move(in double pos);
    in Result IsMoving();
    in Result Stop();

    out void OnHomingChanged();
    out void OnMovingChanged();

    behavior {
        enum State {
            Uninitialized,
            Operational
        };
        State state = State.Uninitialized;

        [state.Uninitialized]
        {
            on Initialize:
            {
                [true] { reply(Result.Ok); state = State.Operational; }
                [true] reply(Result.Fail);
            }
            on Terminate: {}
        }

        [state.Operational]
        {
            on Terminate: state = State.Uninitialized;
            on Home, Move, Stop:
            {
                [true] reply(Result.Ok);
                [true] reply(Result.Fail);
            }
            on IsHoming, IsMoving:
            {
                [true] reply(Result.True);
                [true] reply(Result.False);
                [true] reply(Result.Fail);
            }
        }

        on optional: OnHomingChanged;
        on optional: OnMovingChanged;
    }
}

Implementing the MotorImpl Component

By utilizing these interfaces, we can define the complete behavior of the motor. Let’s start by defining all the provided and required ports. Additionally, the Config interface is injected, meaning that it should be made available through the locator.

 7
 8
 9
10
11
12
component MotorImpl {
    provides Motor api;
    requires MotorGlue glue;
    requires Timer timer;
    requires Async async;
    requires injected Config config;

The behavior begins with defining the State enum. It is important to note that the state of this component may not always match the state of the interface. Therefore, we need to track the component’s state separately and cannot rely on shared state for this purpose. Additionally, the homeTimeout and moveTimeout are two data variables that will be used to store the timeout configuration values.

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    behavior {
        enum State {
            Uninitialized,
            Idle,
            IdleError,
            Starting,
            Starting_SendOk,
            Starting_SendError,
            Operational,
            OperationalError,
            Moving,
            Moving_SendOk,
            Moving_SendError,
        };
        State state = State.Uninitialized;

        double homeTimeout;
        double moveTimeout;

Dezyne does not currently support early returns (early replies) for event blocks. To avoid excessive indentation levels and maintain code readability, I follow a coding style similar to the example shown below in the Initialize function. This approach allows for a more concise and readable code structure, even without deep indentation.

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
        on api.Initialize(id):
        {
            Result res = Result.Ok;
            if (res.Ok) res = config.GetHomeTimeout(id, homeTimeout);
            if (res.Ok) res = config.GetMoveTimeout(id, moveTimeout);
            if (res.Ok) res = glue.Initialize(id);
            if (res.Ok) state = State.Idle;
            reply(res);
        }

        on api.Terminate():
        {
            glue.Terminate();
            state = State.Uninitialized;
        }

        on api.Recover():
        {
            state = State.Idle;
        }

        [state.Uninitialized]
        {
            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

Early return is supported for functions in Dezyne. In this case, a Start function has been defined, which will be utilized later in the Start in-event. When starting the motor, it needs to perform the homing process. This involves stopping the motor and then calling the Home method on the glue. Immediately after that, we check if the homing process has already completed. If it has, it means the homing was very quick, and we can send an OnOk out-event. Otherwise, we start a timeout timer and transition to the Starting state to monitor the OnHomingChanged events and wait for the homing process to finish.

Since Start is an asynchronous operation as specified in the interface, we cannot directly send an OnOk or OnError event here. We need to decouple the event. This can be done by using the defer keyword, but for better testability I choose to utilize the Async interface as explained in the Async article.

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
        void Start()
        {
            Result res = glue.Stop();
            if (res.Fail)
            {
                async.req();
                state = State.Starting_SendError;
                return;
            }

            res = glue.Home();
            if (res.Fail)
            {
                Result dummy = glue.Stop(); // ignore reply value
                async.req();
                state = State.Starting_SendError;
                return;
            }

            res = glue.IsHoming();
            if (res.True)
            {
                timer.Create(homeTimeout);
                state = State.Starting;
            }
            else if (res.False)
            {
                state = State.Starting_SendOk;
                async.req();
            }
            else if (res.Fail)
            {
                Result dummy = glue.Stop(); // ignore reply value
                async.req();
                state = State.Starting_SendError;
            }
            else illegal;
        }

Here we utilize the Start function for the Start in-event.

 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
        [state.Idle]
        {
            on api.Start(): Start();

            on api.Stop(): {}

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

        [state.IdleError]
        {
            on api.Start():
            {
                async.req();
                state = State.Starting_SendError;
            }

            on api.Stop(): {}

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

In the IdleError state, we encounter a situation where the interface does not define the Start in-event. However, in this state, it becomes necessary to implement the Start in-event. This occurs when the component is in the IdleError state, but the interface still assumes it is in the Idle state. This mismatch can arise when the Stop command is sent after an error is detected but not yet reported to the client.

Here we see that the client and component are not synchronized anymore. The api is in the state State:Idle while MotorImpl is in the state State::IdleError.

To address this, the component needs to notify the client that it is in the IdleError state. When the client calls the Start in-event the component will always reply with an OnError out-event. This process effectively synchronizes the client and the component, ensuring both are aware of the current state.

In the Starting state, the component actively monitors the OnHomingChanged signal. Upon receiving this signal, it checks whether the motor has completed the homing process. If the homing process is complete, the component sends an OnOk out-event. However, if the motor fails to finish homing within the specified timeout, the timer triggers an OnTimeout out-event. This event is utilized to send an OnError out-event and transition the component to the IdleError state.

The inclusion of the timer ensures compliance with the interface requirements. Although the OnHomingChanged events are optional and may or may not occur, the timer guarantees the occurrence of an OnTimeout event, enabling the component to fulfill the interface specifications.

123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
        [state.Starting]
        {
            on api.Stop():
            {
                timer.Cancel();
                Result res = glue.Stop();
                if (res.Ok) state = State.Idle;
                else if (res.Fail) state = State.IdleError;
                else illegal;
            }

            on glue.OnHomingChanged():
            {
                Result res = glue.IsHoming();
                if (res.True) {} // ignore, still homing
                else if (res.False)
                {
                    timer.Cancel();
                    api.OnOk();
                    state = State.Operational;
                }
                else if (res.Fail)
                {
                    Result dummy = glue.Stop(); // ignore reply value
                    timer.Cancel();
                    api.OnError();
                    state = State.IdleError;
                }
                else illegal;
            }

            on timer.OnTimeout():
            {
                Result dummy = glue.Stop(); // ignore reply value
                api.OnError();
                state = State.IdleError;
            }

            on glue.OnMovingChanged(): {}
        }

The Starting_SendOk and Starting_SendError states are used to comply to the asynchronous reply to the Start command when needed.

164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
        [state.Starting_SendOk]
        {
            on api.Stop():
            {
                async.clr();
                state = State.Idle;
            }

            on async.ack():
            {
                api.OnOk();
                state = State.Operational;
            }

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

        [state.Starting_SendError]
        {
            on api.Stop():
            {
                async.clr();
                state = State.IdleError;
            }

            on async.ack():
            {
                api.OnError();
                state = State.IdleError;
            }

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

In the Operational state, we define the Move function, which follows a structure similar to the Start function. In this case, since we are in the Operational state, if something goes wrong during the movement, we will transition to the OperationalError state.

200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
        void Move(in double position)
        {
            Result res = glue.Move(position);
            if (res.Fail)
            {
                Result dummy = glue.Stop(); // ignore reply value
                async.req();
                state = State.Moving_SendError;
                return;
            }

            res = glue.IsMoving();
            if (res.True)
            {
                timer.Create(moveTimeout);
                state = State.Moving;
            }
            else if (res.False)
            {
                async.req();
                state = State.Moving_SendOk;
            }
            else if (res.Fail)
            {
                async.req();
                state = State.Moving_SendError;
            }
            else illegal;
        }

The implementation of the Operational and OperationalError states follows the same concepts and structures that we have already discussed. There are no new or surprising elements introduced in these states.

230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
        [state.Operational]
        {
            on api.Move(position): Move(position);

            on api.Stop():
            {
                state = State.Idle;
            }

            on api.StopMoving(): {}

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

        [state.OperationalError]
        {
            on api.Move(position):
            {
                async.req();
                state = State.Moving_SendError;
            }

            on api.Stop():
            {
                state = State.IdleError;
            }

            on api.StopMoving(): {}

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

During the movement process, the OnMovingChanged out-event is monitored to determine when the move is finished. This allows the component to track the progress of the movement. Additionally, a timer is utilized to ensure compliance with the interface requirements.

264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
        [state.Moving]
        {
            on api.StopMoving():
            {
                timer.Cancel();
                Result res = glue.Stop();
                if (res.Ok) state = State.Operational;
                else if (res.Fail) state = State.OperationalError;
                else illegal;
            }

            on glue.OnMovingChanged():
            {
                Result res = glue.IsMoving();
                if (res.True) {} // ignore, still moving
                else if (res.False)
                {
                    timer.Cancel();
                    api.OnOk();
                    state = State.Operational;
                }
                else if (res.Fail)
                {
                    Result dummy = glue.Stop(); // ignore reply value
                    timer.Cancel();
                    api.OnError();
                    state = State.OperationalError;
                }
                else illegal;
            }

            on timer.OnTimeout():
            {
                Result dummy = glue.Stop(); // ignore reply value
                api.OnError();
                state = State.OperationalError;
            }

            on glue.OnHomingChanged(): {}
        }

And lastly the Moving_SendOk and Moving_SendError states are used to asynchronously reply to the Move command when needed.

305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
        [state.Moving_SendOk]
        {
            on api.StopMoving():
            {
                async.clr();
                state = State.Operational;
            }

            on async.ack():
            {
                api.OnOk();
                state = State.Operational;
            }

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }

        [state.Moving_SendError]
        {
            on api.StopMoving():
            {
                async.clr();
                state = State.OperationalError;
            }

            on async.ack():
            {
                api.OnError();
                state = State.OperationalError;
            }

            on glue.OnHomingChanged(): {}
            on glue.OnMovingChanged(): {}
        }
    }
}

About Verification and Unit Testing

For interfaces the following verification checks are done:

No deadlocks - the interface must always be able to make progress.
No unreachable code - the interface does not contain any unreachable code.
No livelocks - the client always has an opportunity to interact with the interface.
Deterministic - all state changes in the interface can be determined by the in- and out-events and reply-values.

For components the verification checks are:

Deterministic - for each state no more than one event handler per event can be defined.
No Illegals - the component never tries to execute an illegal.
No deadlocks - the component can always make progress.
No unreachable code - the component does not contain any unreachable code.
No livelocks - the component never gets into a loop in which it is always busy.
Compliant - the component always follows the contract of the interfaces.

With all these checks in place a lot of things are covered and do not have to be unit tested. The resulting component will be responsive, and all events can be handled. The design of the interfaces are also important. By making an interface restrictive you can ensure the order of calls. For example, by forcing the call to Initialize before allowing any other in-events, you know all components will be initialized before use. Terminate will go back to the state Uninitialized, and therefore you will find out if you forget to terminate a component, because a not terminated component cannot be initialized again. With verification this becomes immensely powerful, because it will check all code paths, which is often not feasible with unit tests.

The usage of the component can be effectively managed by designing strict interfaces and leveraging verification to check correct usage. However, there are also some things that will not be covered by verification and will have to be tested using unit tests.

Data is outside the scope of Dezyne. All data handling and passing will have to be unit tested.
Functional behavior will have to be unit tested. For example, in our motor, before the homing is started, the motor must be stopped.

In our example, additional unit testing is required because the MotorGlue interface is primarily a permissive interface that does not provide guidance on how to use it. The MotorImpl component, therefore, needs to encapsulate the knowledge of how to utilize the MotorGlue interface effectively. The implementation of the usage rules will be coded within the MotorImpl component and will need to be thoroughly unit tested to ensure their correctness and adherence to the desired behavior.

The developers of Dezyne are working on functional testing for Dezyne. When functional testing becomes available, unit testing will primarily be focused on data handling and other handwritten code.

For the MotorImpl we will not test:

If there are any dead locks or life locks.
If we follow the contracts of the interfaces.
That the component needs to be initialized before started.
That the component needs to be started before moving.
That movements can be interrupted.

Because all of that is handled by verification. But we will test that:

Initializing will get the right values from the configuration.
Before the homing, the motor will be stopped.
That the homing and moving timeouts use the right value from the configuration.
That the component correctly detects the end of the movement.
That the component can handle when the movement or homing is directly finished.
That the component will go into error after a timeout.
That stop will be called on the motor when the component goes into error.
That stop and stop moving will call stop on the motor.

Testing

About the `dzn::pump`

The MotorImpl does not use the blocking and defer features of Dezyne, meaning the Dezyne pump is not used. Therefore, we can use the main thread to run the tests on. This way we know there is never an event waiting in the queue (pump) and we can call in- and out-events directly. This makes the tests more direct.

You will notice when the pump is needed. When it cannot be found in the dzn::locator, an exception will be thrown.

Making Mocks for Dezyne Interfaces

By looking at the generated code of the MotorImpl in MotorImpl.hh and MotorImpl.cc we can extract what is needed to manually create a component. This does involve some boilerplate code. Using this, a mock for the Timer interface can be constructed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
struct TimerMock : dzn::component
{
    dzn::meta dzn_meta;
    dzn::runtime& dzn_runtime;
    const dzn::locator& dzn_locator;
    Timer api;

    TimerMock(const dzn::locator& locator) :
        dzn_meta({"timer", "Timer", nullptr, {}, {}, {[this] { api.dzn_check_bindings(); }}}),
        dzn_runtime(locator.get<dzn::runtime>()),
        dzn_locator(locator),
        api({{"api", &api, this, &dzn_meta}, {"api", nullptr, nullptr, nullptr}}, this)
    {
        api.in.Create = [this](double timeout) { Create(timeout); };
        api.in.Cancel = [this] { Cancel(); };
    }

    MOCK_METHOD(void, Create, (double));
    MOCK_METHOD(void, Cancel, ());
};

The idea for the mock is that for each in-event we create a matching mock method. Then we connect the in-event to this mock method. Now we can set expectations on the calls to the in-events. Optionally some helper methods could be added for sending out-events. But because we are making the tests single-threaded without using the pump, out-events can be called directly when needed.

We have to reuse this pattern for the Async, Config and MotorGlue mocks. To reduce the amount of boilerplate code I propose to create the following helper classes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
struct Component : dzn::component
{
    dzn::meta dzn_meta;
    dzn::runtime& dzn_runtime;
    const dzn::locator& dzn_locator;

    Component(const char* name, const char* type, const dzn::locator& locator) :
        dzn_meta({name, type, nullptr, {}, {}, {}}),
        dzn_runtime(locator.get<dzn::runtime>()),
        dzn_locator(locator)
    {
    }
};

template <typename Interface>
struct Provides : Interface
{
    Provides(const char* name, Component* component) :
        Interface(dzn::port::meta{
            {name, this, component, &component->dzn_meta},
            {name, nullptr, nullptr, nullptr}},
            component
        )
    {
        component->dzn_meta.ports_connected.emplace_back([this]
        {
            this->dzn_check_bindings();
        });
    }

    Interface& operator*()
    {
        return *this;
    }
};

Using these helper classes the TimerMock can be written as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
struct TimerMock : Component
{
    Provides<Timer> api;

    TimerMock(const dzn::locator& locator) :
        Component("TimerMock", "TimerMock", locator),
        api("api", this)
    {
        api.in.Create = [this](double timeout) { Create(timeout); };
        api.in.Cancel = [this] { Cancel(); };
    }

    MOCK_METHOD(void, Create, (double timeout));
    MOCK_METHOD(void, Cancel, ());
};

The AsyncMock, MotorGlueMock and ConfigMock follow the same pattern. The last thing we need is a mock to track the out-events of the MotorImpl component.

1
2
3
4
5
struct MotorImplEventsMock
{
    MOCK_METHOD(void, OnOk, ());
    MOCK_METHOD(void, OnError, ());
};

The `MotorImplTest` Fixture

Now that we have all the mocks, we can create the fixture. The fixture will set up the Dezyne environment, instantiate all the objects and connect everything together.

A Dezyne environment will always consist of a dzn::locator and a dzn::runtime. The runtime must be given to the locator. Often also a dzn::pump will be given to the locator, but as discussed above for these tests we do not need the pump. The glue, timer, async, config and events member variables are declared as strict mocks. All Dezyne components are initialized by giving it the locator. Because config is declared as required injected Config config in the MotorImpl component, it will be retrieved from the locator. That is why we added it to the locator here.

In the fixture’s constructor, the components are connected together. The out-events are connected to the MotorImplEventsMock. The last step is to check if everything has been fully connected by calling dzn::check_bindings on every component.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
struct MotorImplTest : testing::Test
{
    MotorImplTest() :
        glue(locator
            .set(runtime)
        ),
        timer(locator),
        async(locator),
        config(locator),
        impl(locator
            .set(*config.api)
        )
    {
        dzn::connect(glue.api, impl.glue);
        dzn::connect(timer.api, impl.timer);
        dzn::connect(async.api, impl.async);

        impl.api.out.OnOk = [this] { events.OnOk(); };
        impl.api.out.OnError = [this] { events.OnError(); };

        dzn::check_bindings(impl);
        dzn::check_bindings(glue);
        dzn::check_bindings(timer);
        dzn::check_bindings(async);
        dzn::check_bindings(config);
    }

    const std::string motorId = "abc";
    dzn::locator locator;
    dzn::runtime runtime;
    testing::StrictMock<MotorGlueMock> glue;
    testing::StrictMock<TimerMock> timer;
    testing::StrictMock<AsyncMock> async;
    testing::StrictMock<ConfigMock> config;
    testing::StrictMock<MotorImplEventsMock> events;
    MotorImpl impl;
};

Testing `Initialize`

For initialize we test if the motorId is used for the calls to config and that the glue is initialized using the same motorId.

1
2
3
4
5
6
7
8
9
TEST_F(MotorImplTest, MotorIdIsPassedAroundAndInitializeReturnsOk)
{
    EXPECT_CALL(config, GetHomeTimeout(motorId, _))
        .WillOnce(Return(Result::Ok));
    EXPECT_CALL(config, GetMoveTimeout(motorId, _))
        .WillOnce(Return(Result::Ok));
    EXPECT_CALL(glue, Initialize(motorId)).WillOnce(Return(Result::Ok));
    ASSERT_EQ(Result::Ok, impl.api.in.Initialize(motorId));
}

The failure cases for Initialize are not tested here. The verification covers this enough. The happy flow of this test together with other tests and the verification will cover all fail cases. The same is true for the call to Terminate.

Testing `Start`

`MotorImplTest_Idle` Fixture

To test Start a fixture representing the Idle state is created. The fixture will set the two timeouts to known values and initialize the glue.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
struct MotorImplTest_Idle : MotorImplTest
{
    void SetUp() override
    {
        EXPECT_CALL(config, GetHomeTimeout(motorId, _)).WillOnce(DoAll(
            SetArgReferee<1>(homeTimeout),
            Return(Result::Ok)
        ));
        EXPECT_CALL(config, GetMoveTimeout(motorId, _)).WillOnce(DoAll(
            SetArgReferee<1>(moveTimeout),
            Return(Result::Ok)
        ));
        EXPECT_CALL(glue, Initialize(motorId)).WillOnce(Return(Result::Ok));
        ASSERT_EQ(Result::Ok, impl.api.in.Initialize(motorId));

        ASSERT_EQ(Motor::State::Idle, impl.api.state);

        ASSERT_TRUE(::testing::Mock::VerifyAndClear(&config));
        ASSERT_TRUE(::testing::Mock::VerifyAndClear(&glue));
    }

    const double homeTimeout = 12.0;
    const double moveTimeout = 34.0;
};

`Start` Tests

This first test is the happy flow for the homing using the following scenario. After sending the home command, it is expected that IsHoming returns true. Because the state of homing changed, an OnHomingChanged event is expected for which the IsHoming still returns true. Sometime later the OnHomingChanged is expected again but then the IsHoming returns false. The result should be that the OnOk event will be received.

It will also test that the correct homeTimeout value is used.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
TEST_F(MotorImplTest_Idle, StartResultsInOnOk)
{
    EXPECT_CALL(timer, Create(homeTimeout)); // Timeout timer with homeTimeout will be created
    EXPECT_CALL(timer, Cancel()); // And cancelled
    EXPECT_CALL(glue, Stop()).WillOnce(Return(Result::Ok));
    EXPECT_CALL(glue, Home()).WillOnce(Return(Result::Ok));
    {
        InSequence s;
        EXPECT_CALL(glue, IsHoming())
            .WillOnce(Return(Result::True)); // Check in Start
        EXPECT_CALL(glue, IsHoming())
            .WillOnce(Return(Result::True)); // Check First OnHomingChanged
        EXPECT_CALL(glue, IsHoming())
            .WillOnce(Return(Result::False)); // Check Second OnHomingChanged
    }
    EXPECT_CALL(events, OnOk());

    impl.api.in.Start();
    glue.api.out.OnHomingChanged();
    glue.api.out.OnHomingChanged();
}

This second happy flow scenario covers that the homing is finished immediately. The IsHoming returns false the first time and the async interface is used to make the OnOk reply asynchronously.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
TEST_F(MotorImplTest_Idle, StartResultInOnOkWhenItFinishedDirectly)
{
    EXPECT_CALL(glue, Stop()).WillOnce(Return(Result::Ok));
    EXPECT_CALL(glue, Home()).WillOnce(Return(Result::Ok));
    EXPECT_CALL(glue, IsHoming()).WillOnce(Return(Result::False));
    EXPECT_CALL(async, req());
    EXPECT_CALL(events, OnOk());

    impl.api.in.Start();
    async.api.out.ack();
}

The next test is about receiving a timeout when starting takes too long. It also makes sure that stop is called twice. Once before homing, and once after the timeout.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
TEST_F(MotorImplTest_Idle, StartResultsInOnErrorAfterTimeout)
{
    EXPECT_CALL(timer, Create(homeTimeout));
    EXPECT_CALL(glue, Stop()).Times(2).WillRepeatedly(Return(Result::Ok));
    EXPECT_CALL(glue, Home()).WillOnce(Return(Result::Ok));
    EXPECT_CALL(glue, IsHoming()).Times(2).WillRepeatedly(Return(Result::True));
    EXPECT_CALL(events, OnError());

    impl.api.in.Start();
    glue.api.out.OnHomingChanged();
    timer.api.out.OnTimeout();
}

The last test makes sure that stop is called on the glue when the starting is aborted.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
TEST_F(MotorImplTest_Idle, WhileStartingAndThenStopThenGlueStopIsCalled)
{
    EXPECT_CALL(timer, Create(homeTimeout));
    EXPECT_CALL(timer, Cancel());
    {
        InSequence s;
        EXPECT_CALL(glue, Stop()).WillOnce(Return(Result::Ok)); // Stop before home
        EXPECT_CALL(glue, Stop()).WillOnce(Return(Result::Ok)); // Stop from stop
    }
    EXPECT_CALL(glue, Home()).WillOnce(Return(Result::Ok));
    EXPECT_CALL(glue, IsHoming()).WillRepeatedly(Return(Result::True));

    impl.api.in.Start();
    glue.api.out.OnHomingChanged();
    impl.api.in.Stop();
}

Testing the Rest

Some more tests could be defined around the error behavior. The Stop call can fail, the Home call can fail and the IsHoming call can fail at different moments. I think it would be wise to make sure that a glue.Stop() is attempted after something goes wrong after starting to home. But I have not added these tests in this example.

The Move tests are like the Start tests and are not included in this article. These tests will also test that the position is correctly passed to the glue.

Test Results

Below is some output from running the test. The test output is intermingled with Dezyne trace output. Normally this should be redirected elsewhere.

Running main() from gmock_main.cc
[==========] Running 9 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 1 test from MotorImplTest
[ RUN      ] MotorImplTest.MotorIdIsPassedAroundAndInitializeReturnsOk
<external>.api.Initialize -> MotorImpl.api.Initialize
MotorImpl.config.GetHomeTimeout -> ConfigMock.api.GetHomeTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.config.GetMoveTimeout -> ConfigMock.api.GetMoveTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.glue.Initialize -> MotorGlueMock.api.Initialize
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.Result:Ok <- MotorImpl.api.Result:Ok
[       OK ] MotorImplTest.MotorIdIsPassedAroundAndInitializeReturnsOk (0 ms)
[----------] 1 test from MotorImplTest (0 ms total)

[----------] 4 tests from MotorImplTest_Idle
[ RUN      ] MotorImplTest_Idle.StartResultsInOnOk
<external>.api.Initialize -> MotorImpl.api.Initialize
MotorImpl.config.GetHomeTimeout -> ConfigMock.api.GetHomeTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.config.GetMoveTimeout -> ConfigMock.api.GetMoveTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.glue.Initialize -> MotorGlueMock.api.Initialize
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.Result:Ok <- MotorImpl.api.Result:Ok
<external>.api.Start -> MotorImpl.api.Start
MotorImpl.glue.Stop -> MotorGlueMock.api.Stop
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.Home -> MotorGlueMock.api.Home
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:True <- MotorGlueMock.api.Result:True
MotorImpl.timer.Create -> TimerMock.api.Create
MotorImpl.timer.return <- TimerMock.api.return
<external>.api.return <- MotorImpl.api.return
MotorImpl.<q> <- MotorGlueMock.api.OnHomingChanged
MotorImpl.glue.OnHomingChanged <- MotorImpl.<q>
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:True <- MotorGlueMock.api.Result:True
MotorImpl.<q> <- MotorGlueMock.api.OnHomingChanged
MotorImpl.glue.OnHomingChanged <- MotorImpl.<q>
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:False <- MotorGlueMock.api.Result:False
MotorImpl.timer.Cancel -> TimerMock.api.Cancel
MotorImpl.timer.return <- TimerMock.api.return
<external>.api.OnOk <- MotorImpl.api.OnOk
[       OK ] MotorImplTest_Idle.StartResultsInOnOk (0 ms)
[ RUN      ] MotorImplTest_Idle.StartResultInOnOkWhenItFinishedDirectly
<external>.api.Initialize -> MotorImpl.api.Initialize
MotorImpl.config.GetHomeTimeout -> ConfigMock.api.GetHomeTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.config.GetMoveTimeout -> ConfigMock.api.GetMoveTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.glue.Initialize -> MotorGlueMock.api.Initialize
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.Result:Ok <- MotorImpl.api.Result:Ok
<external>.api.Start -> MotorImpl.api.Start
MotorImpl.glue.Stop -> MotorGlueMock.api.Stop
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.Home -> MotorGlueMock.api.Home
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:False <- MotorGlueMock.api.Result:False
MotorImpl.async.req -> AsyncMock.api.req
MotorImpl.async.return <- AsyncMock.api.return
<external>.api.return <- MotorImpl.api.return
MotorImpl.<q> <- AsyncMock.api.ack
MotorImpl.async.ack <- MotorImpl.<q>
<external>.api.OnOk <- MotorImpl.api.OnOk
[       OK ] MotorImplTest_Idle.StartResultInOnOkWhenItFinishedDirectly (0 ms)
[ RUN      ] MotorImplTest_Idle.StartResultsInOnErrorAfterTimeout
<external>.api.Initialize -> MotorImpl.api.Initialize
MotorImpl.config.GetHomeTimeout -> ConfigMock.api.GetHomeTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.config.GetMoveTimeout -> ConfigMock.api.GetMoveTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.glue.Initialize -> MotorGlueMock.api.Initialize
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.Result:Ok <- MotorImpl.api.Result:Ok
<external>.api.Start -> MotorImpl.api.Start
MotorImpl.glue.Stop -> MotorGlueMock.api.Stop
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.Home -> MotorGlueMock.api.Home
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:True <- MotorGlueMock.api.Result:True
MotorImpl.timer.Create -> TimerMock.api.Create
MotorImpl.timer.return <- TimerMock.api.return
<external>.api.return <- MotorImpl.api.return
MotorImpl.<q> <- MotorGlueMock.api.OnHomingChanged
MotorImpl.glue.OnHomingChanged <- MotorImpl.<q>
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:True <- MotorGlueMock.api.Result:True
MotorImpl.<q> <- TimerMock.api.OnTimeout
MotorImpl.timer.OnTimeout <- MotorImpl.<q>
MotorImpl.glue.Stop -> MotorGlueMock.api.Stop
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.OnError <- MotorImpl.api.OnError
[       OK ] MotorImplTest_Idle.StartResultsInOnErrorAfterTimeout (0 ms)
[ RUN      ] MotorImplTest_Idle.WhileStartingAndThenStopThenGlueStopIsCalled
<external>.api.Initialize -> MotorImpl.api.Initialize
MotorImpl.config.GetHomeTimeout -> ConfigMock.api.GetHomeTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.config.GetMoveTimeout -> ConfigMock.api.GetMoveTimeout
MotorImpl.config.Result:Ok <- ConfigMock.api.Result:Ok
MotorImpl.glue.Initialize -> MotorGlueMock.api.Initialize
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.Result:Ok <- MotorImpl.api.Result:Ok
<external>.api.Start -> MotorImpl.api.Start
MotorImpl.glue.Stop -> MotorGlueMock.api.Stop
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.Home -> MotorGlueMock.api.Home
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:True <- MotorGlueMock.api.Result:True
MotorImpl.timer.Create -> TimerMock.api.Create
MotorImpl.timer.return <- TimerMock.api.return
<external>.api.return <- MotorImpl.api.return
MotorImpl.<q> <- MotorGlueMock.api.OnHomingChanged
MotorImpl.glue.OnHomingChanged <- MotorImpl.<q>
MotorImpl.glue.IsHoming -> MotorGlueMock.api.IsHoming
MotorImpl.glue.Result:True <- MotorGlueMock.api.Result:True
<external>.api.Stop -> MotorImpl.api.Stop
MotorImpl.timer.Cancel -> TimerMock.api.Cancel
MotorImpl.timer.return <- TimerMock.api.return
MotorImpl.glue.Stop -> MotorGlueMock.api.Stop
MotorImpl.glue.Result:Ok <- MotorGlueMock.api.Result:Ok
<external>.api.return <- MotorImpl.api.return
[       OK ] MotorImplTest_Idle.WhileStartingAndThenStopThenGlueStopIsCalled (0 ms)
[----------] 4 tests from MotorImplTest_Idle (2 ms total)

... snip ...

[----------] Global test environment tear-down
[==========] 9 tests from 3 test suites ran. (5 ms total)
[  PASSED  ] 9 tests.

Download and Run Code

The code can be downloaded here. I have only tested it on Linux. Dezyne 2.8.1 or higher must be installed and in the search path.

Use the following commands to run the unit tests:

unzip code.zip
cd unittesting
mkdir build
cd build
cmake .. -DDEZYNE_RUNTIME_PATH=/opt/dezyne/runtime/c++
make
./DezyneTest

Visual Studio Code settings have been included. Update the settings.json to point to the Dezyne runtime before using it.

Conclusion

Even when using Dezyne with formal verification some testing needs to be done. Especially the functional behavior and data handling needs to be tested. I have shown how and what to test for the MotorImpl component. The behavior of this component especially needs to be tested because it uses the permissive MotorGlue, and the knowledge of how to use the glue is contained in MotorImpl. The unit tests check usage scenarios of the glue.