Creating and using custom objects in Crappy
If you have read over the two first pages of the documentation, you should now have a good understanding of how Crappy works and the possibilities it offers. However, at that point, you are still limited by the functionalities that the Blocks and other objects natively distributed with Crappy. It is now time for you to create your own objects in Crappy, to adapt your scripts to your own needs ! This page of the tutorials covers the basics of the instantiation of custom objects, while the next and last page of the tutorials covers the advanced aspects of custom object instantiation.
1. Custom Modifiers
The first type of custom objects that we’ll cover here are the
Modifiers, because they are by far the simplest objects ! The
Modifiers come in use in a variety of situations, and it is often required
to write your own ones as the catalog of Crappy cannot cover every single
case. Luckily, the only requirement for an object to qualify as a Modifier
is to be a Callable, which means that functions can be
provided !
More precisely, a Modifier should accept a dict as its sole argument and
return a dict as well (None is also accepted). This
dictionary is the representation of a chunk of data flowing through the
Link, and the Modifier has direct access to it ! It can add keys,
delete others, change the value of a key, etc. Each key is a label, and has a
value it carries. In the end, all a Modifier does is to modify the incoming
dictionary and return it after modification. As usual, let’s put these concepts
in application on a real runnable example !
For this first example, let’s create a Modifier that simply doubles the value
of a given label. If you have understood the last paragraph, this Modifier will
basically only perform something like data['label'] = data['label'] * 2.
Now, how to integrate it to your code in practice ? Start by defining the
function somewhere, preferably before the if __name__ == "__main__"
statement. Then, simply pass it to the 'modifier' argument of the target
Link, and that’s it ! Do not forget to return the
modified dictionary in the function !
# coding: utf-8
import crappy
def func_modifier(data):
data['cmdx2'] = data['cmd'] * 2
return data
if __name__ == '__main__':
gen = crappy.blocks.Generator(({'type': 'Sine', 'freq': 0.5,
'amplitude': 2, 'condition': 'delay=15'},),
cmd_label='cmd',
freq=30)
graph = crappy.blocks.Grapher(
('t(s)', 'cmd'),
('t(s)', 'cmdx2'),
)
crappy.link(gen, graph, modifier=func_modifier)
crappy.start()
Note
To run this example, you’ll need to have the matplotlib Python module
installed.
In this first example, you can see that instead of replacing the value of the
'cmd' label with its double, it was chosen to store the double value in
the newly created 'cmdx2' label. A new label was added ! This is just how
powerful of a tool the Modifiers are ! Notice how the Modifier is added to the
Link between the Generator and the Grapher, the syntax couldn’t
be more straightforward. If you need to change the name of the target label, or
the value of the multiplier, they can simply be modified in the definition of
the function. Alternatively, you could add arguments to you function and use
functools.partial when passing it to the Link, but that is quite an
advanced design already.
In the example, a basic function was passed as a Modifier. While functions are
very versatile and well-known even to beginners, there are many situations when
they will show strong limitations. For example, what if you want to store a
value between two chunks of data ? That is simply not possible with functions !
A concrete example of that is the Integrate Modifier,
that integrates a signal over time. It needs to store the integral between
consecutive chunks of data, and therefore cannot rely on a function.
To circumvent this limitation, we made it possible to instantiate Modifiers
as classes instead of functions. It is mentioned above that the Modifiers
need to be Callable objects, but a class defining the
__call__ method is actually callable ! Here is the minimal template of a
Modifier as a class :
import crappy
class MyModifier(crappy.modifier.Modifier):
def __init__(self):
super().__init__()
def __call__(self, dic):
return dic
Some aspects of the code are worth commenting. First, the class should be a
child of the base crappy.modifier.Modifier, and initialize its parent
class during __init__ (via the call to super().__init__()). And
second, it needs to define a __call__ method taking a dict as its
sole argument and returning a dict. The __call__ method plays the
same role as the function in the previous example, but the class structure
makes it possible to store attributes and to achieve much more complex
behaviors ! To demonstrate that, let’s recreate a simpler version of the
Integrate Modifier :
# coding: utf-8
import crappy
from time import time
class ClassModifier(crappy.modifier.Modifier):
def __init__(self, label):
super().__init__()
self.label = label
self.sum = 0
self.last_t = time()
def __call__(self, data):
t = time()
self.sum += data[self.label] * (t - self.last_t)
self.last_t = t
data['cumsum'] = self.sum
return data
if __name__ == '__main__':
gen = crappy.blocks.Generator(({'type': 'Sine', 'freq': 0.5,
'amplitude': 2, 'condition': 'delay=15'},),
cmd_label='cmd',
freq=30)
graph = crappy.blocks.Grapher(
('t(s)', 'cumsum'),
)
crappy.link(gen, graph, modifier=ClassModifier('cmd'))
crappy.start()
Note
To run this example, you’ll need to have the matplotlib Python module
installed.
As you can see, compared to the template, several features have been added.
First, the Modifier takes one argument at instantiation, that indicates the
name of the label to integrate over time. This label is indeed provided in the
line where the Modifier is given as an argument to the Link. And then, several
attributes are defined in the __init__ method to handle the calculation
of the integral during __call__. This ability to store values between
consecutive calls is exactly what was desired when using classes as Modifiers !
The two examples presented in this section can finally be merged into a single
big one :
You can download this custom Modifier example to run it locally on your
machine. An extra example of a custom Modifier can also be found in the
examples folder on GitHub. Except fot what was detailed in
this section, there is actually not much more to know about the definition of
custom Modifiers ! They stand after all among the simplest objects in Crappy.
2. Custom Actuators
After introducing how custom Modifiers work in the first section, this second
section will focus on the use of custom Actuators. Knowing how to add
and use your own Actuator objects in Crappy is
critical for anyone who wants to drive their own tests, as the equipment
that we already integrated in the module will likely not match the one you
have at your disposal. In this situation, you’ll have no choice but to
implement it yourself in Crappy !
Unlike Modifiers, Actuators usually communicate with hardware. The code for driving a custom Actuator is therefore quite different than the one for a Modifier that simply handles data. That being said, what are the steps for writing code for your Actuator ? First, make sure that the hardware you want to use can be driven with Python ! If that is not the case, you unfortunately won’t be able to drive it with Crappy. There are usually many different ways to drive hardware from Python, so just because there is no Python library for your device doesn’t mean you cannot drive it from Python ! Here are a few ways to drive hardware from Python :
Use a Python library provided by the manufacturer or a third party
Get the correct communication syntax and protocol from the datasheet, and code the communication yourself over the right protocol (serial, USB, I2C, SPI, etc.)
Send commands in the terminal from Python, if the manufacturer provides a way to drive hardware from the console
Write Python bindings for a C/C++ library, if the manufacturer provides one
Note that all these solutions require various levels of expertise in Python, some are very simple and others much more difficult ! For now, let’s assume that you have found a way to drive your device from Python. The next step is to write a draft code completely independent from Crappy, and in which you’re able to initialize the connection to the device, set its parameters, set its speed or target position, acquire its current speed or position, and properly de-initialize the device and the connection. We advise you to write this independent draft so that in a first time you don’t have to bother with Crappy’s syntax.
Once you have a working draft, it is time to integrate it in Crappy ! Just like
the Modifier previously, there is also a template for the
Actuator objects :
import crappy
class MyActuator(crappy.actuator.Actuator):
def __init__(self):
super().__init__()
def open(self):
...
def set_position(self, pos, speed):
...
def set_speed(self, speed):
...
def get_position(self):
...
def get_speed(self):
...
def stop(self):
...
def close(self):
...
This template looks much bigger than the one for the Modifier, but actually
part of the methods are optional. Out of the set_position,
set_speed, get_position and get_speed methods, you only need
to implement at least one. You could even get away with implementing none, but
the interest is limited. Let’s review what each method is intended for :
__init__()is where you should initialize the Python objects that your class uses. It is also where the class accepts its arguments, that are given in the dictionary passed to the Machine Block. Avoid interacting with hardware already in this method. Also, don’t forget to initialize the parent class withsuper().__init__()!In
open()you should perform any action required for configuring the device. That includes opening the communication with it, configuring its parameters, or maybe energizing it.set_speed()andset_position()are for sending respectively a target speed or position command to the device. It is possible to implement both if the device supports it, or only one, or even none if the device is only used as a sensor in Crappy. These methods take as an argument the target speed and position respectively, and do not return anything. As you may have guessed,set_speed()is called if the Actuator is driven in speed mode, andset_position()is called if the Actuator is driven in position mode. These methods are only called if the Machine Block receives commands via an incoming Link. Note that theset_position()method always accepts a secondspeedargument, that may be equal toNone. You’ll find more about it in a dedicated section on the next page.In a similar way,
get_speed()andget_position()are for acquiring the current speed or position of the device. These methods do not take any argument, and return the acquired speed or position as afloat. Again, it is possible to define both methods, or only one, or none. They can be called no matter what the driving mode is, provided that theposition_labeland/orspeed_labelkeys are provided as arguments in the dictionary passed to the Machine Block. The data is only sent to downstream Blocks if the Machine Block has outgoing Links.stop()should stop the device in the fastest and more durable possible way. It is called if a problem occurs, and at the very end of the test. If there is no other way to stop the device than setting its speed to 0, this method doesn’t need to be defined.In
close()you should perform any action required for properly de-initializing the device. For example, this is where you put a device to sleep mode or close the connection to it.
Also note how the class inherits from the parent
crappy.actuator.Actuator class. That must not be forgotten, otherwise
the Actuator won’t work ! At that point, you should use your working draft
to fill in the corresponding methods in the template. You should be able to
obtain a working Actuator in no time! To give you a better idea of what the
result could look like, here’s an example inspired from the custom Actuator in
the examples folder on GitHub :
# coding: utf-8
import crappy
from time import time
class CustomActuator(crappy.actuator.Actuator):
def __init__(self, init_speed=1) -> None:
super().__init__()
self._speed = init_speed
self._pos = 0
self._last_t = time()
def open(self):
self._last_t = time()
def set_speed(self, speed):
self._speed = speed
def get_speed(self):
return self._speed
def get_position(self):
t = time()
delta = self._speed * (t - self._last_t)
self._last_t = t
self._pos += delta
return self._pos
As you can see, the __init__ method takes on argument and initializes
various objects used elsewhere in the class. The open method does not
much, as this example emulates hardware and does not interact with any
real-world device. For the same reason, the close and stop methods
are missing. This Actuator can only be driven in speed, so the
set_position method is also missing. The set_speed and
get_speed methods are present for setting the target speed and measuring
the current one, as well as the get_position method since the position
is also measurable. Now that the Actuator is defined, it is time to add some
context to make it run :
Note
To run this example, you’ll need to have the matplotlib Python module
installed.
You can download this custom Actuator example to run it locally on your
machine. The concepts presented in this section will be re-used for all the
other types of custom objects, so make sure to understand them well ! You
can also have a look at the Actuators distributed with Crappy to see how the implementation of real-life Actuators looks like.
3. Custom InOuts
Creating custom In / Out objects is extremely similar to creating custom
Actuators, so make sure to first read and understand the previous
section first ! Just like for Actuators, anyone who wants to drive their
own setups with Crappy will surely need at some point to create their own
InOut objects. This section covers the specificities
of creating new InOuts.
3.a. Regular mode
First, let’s cover the similarities with the creation of Actuator objects, in
the case of a regular usage. The case of the streamer mode is covered in
the next sub-section. Just like for an Actuator,
you’ll need to write your class before the if __name__ == "__main__"
statement, or to import it from another file. You should also start from a
working draft in which you’re able to drive your device in Python. And in both
cases, creating your custom class can be simply achieved by filling in a
template ! That being said, the objects and methods to manipulate will of
course differ, here’s how the template for an InOut looks like :
import crappy
class MyInOut(crappy.inout.InOut):
def __init__(self):
super().__init__()
def open(self):
...
def get_data(self):
...
def set_cmd(self, cmd):
...
def close(self):
...
As you can see, there are two main differences. First, the parent class from
which your InOut must inherit is now crappy.inout.InOut, and second
you now have to define the get_data and/or set_cmd methods. The
__init__(), open() and
close() methods serve the same purpose as for the
Actuators. The new methods are :
get_data(), that takes no argument and should return the data acquired by the device. The first returned value must be the timestamp of the acquisition, as returned bytime.time. Then, you can return as many values as you want, usually corresponding to different channels you device can acquire. The number of returned values should always be the same, and for each value a label should be given in thelabelsargument of the IOBlock. The data will only be acquired if the IOBlock has outgoing Links !set_cmd()takes one or several arguments, and does not return anything. Instead, the arguments it receives should be used to set commands on the device to drive. The number of arguments this method receives only depends on the number of labels given as thecmd_labelsargument to the IOBlock. The order of the arguments is also the same as the one of the labels incmd_labels.
Once again, let’s switch to practice by writing a custom InOut class. We’ll keep it very basic, you can browse the collection of InOuts distributed with Crappy to have an overview of what a real-life InOut looks like.
# coding: utf-8
import crappy
from time import time
class CustomInOut(crappy.inout.InOut):
def __init__(self, init_value=0):
super().__init__()
self.value1 = init_value
self.value2 = init_value
def get_data(self):
return time(), self.value1, self.value2
def set_cmd(self, v1, v2):
self.value1 = v1
self.value2 = v2
In this example, the InOut simply stores two values. When get_data is
called, it simply returns these two values as well as a timestamp. When,
set_cmd is called, it expects two arguments and sets their values as the
new stored values. Let’s now integrate the InOut into a runnable code :
Note
To run this example, you’ll need to have the matplotlib Python module
installed.
In order to obtain two commands from a single Generator, a
Modifier is added to create a new label. In the IOBlock, the two labels
carrying the commands are indicated in the cmd_labels argument. The
values acquired by the get_data method are transmitted to the
Grapher Block over the labels indicated in the labels argument of
the IOBlock. And in the end it all works fine together ! You can
download this custom InOut example to run it locally on your
machine, and have a look at the examples folder on GitHub to find more examples of custom InOut objects.
3.b. Streamer mode
If you want to be able to use your custom InOut object in streamer mode, the methods described above will not be sufficient. Instead, there is a particular framework to follow that is detailed in this sub-section. For more details on how to use the streamer mode, refer to the Dealing with streams section of the tutorials. Getting straight to the point, here’s how the template for an InOut supporting the streamer mode looks like :
import crappy
class MyStreamerInOut(crappy.inout.InOut):
def __init__(self):
super().__init__()
def open(self):
...
def get_data(self):
...
def set_cmd(self, cmd):
...
def start_stream(self):
...
def get_stream(self):
...
def stop_stream(self):
...
def close(self):
...
It is the exact same as the one for the regular InOuts, except there are three
additional methods. You can still define the
get_data() and
set_cmd() methods, so that your InOut can be used
both in regular and streamer mode depending on the value of the
streamer argument of the IOBlock ! Now, what are the new methods
supposed to do ?
start_stream()should perform any action required to start the acquisition of a stream on the device. It can for example configure the device, or send a specific command. It is fine not to define this method if no particular action is required. The actions performed in this method must be specific to the streamer mode, the general initialization commands should still be executed in theopen()method.get_stream()is where the stream data is acquired. This method does not take any parameter, and should return two objects. The first one is anumpyarray of shape (m,), and the second anothernumpyarray of shape (m, n), where m is the number of timestamps and n the number of channels of the InOut. The first array contains only one column with all the timestamps at which data was acquired. It is equivalent to the timestamp value inget_data(), except here there are several timestamps to return. The second array is a table containing for each timestamp and each label the acquired value. Instead of returning one value per channel like in theget_data(), only one object contains all the values.stop_stream()should perform any action required for stopping the acquisition of the stream. It is fine not to define this method if no particular action is required. The actions performed in this method must be specific to the streamer mode, the general de-initialization commands should still be executed in theclose()method.
At that point, the syntax of the objects to return in the
get_stream() method might still not be very clear to
you. A short example should help you understand it, it’s not as difficult as it
first seems ! The following example is the continuation of the one presented in
the previous sub-section :
Note
To run this example, you’ll need to have the matplotlib Python module
installed.
The first difference is that the module numpy must be used, but that is
not a problem since it is a requirement of Crappy. Then, the
get_stream() method is defined. The structure of the
returned arrays should not be too difficult to understand if you’re familiar
with numpy. Note that here the returned arrays are built iteratively,
but for real-life InOuts they are usually derived directly from a big message
sent by the driven device. Just like previously, the
start_stream(),
stop_stream(),
open() and close() methods
don’t need to be defined. At the IOBlock level, the streamer argument is
now set to True, and the labels argument has also been updated.
Finally, a Demux Modifier is now needed on the Link from the
IOBlock to the Grapher in order for the data to be displayed.
You can download this custom streamer InOut example to run it locally on your
machine. The only real difficulty with the instantiation of custom InOuts
supporting the streamer mode is building the arrays to return, but you can
find an additional example of a custom InOut in the examples folder on GitHub and in the InOuts distributed with Crappy.
4. Custom Cameras
Now that you’re getting familiar with the instantiation of custom objects in
Crappy, adding your own Cameras to Crappy should not present any
particular difficulty. The camera management is one of the big strengths of
Crappy, as Crappy handles the parallelization of the acquisition, display,
recording and processing of the images. The result is a very high
performance when dealing with images. Also, Crappy comes with a variety of
Blocks for performing advanced and optimized image processing, that you can
use with your own Camera objects (see the
DIS Correl, DIC VE or Video Extenso Blocks for example).
For these reasons, integrating your own cameras into Crappy might prove very
advantageous.
The first step for integrating a camera in Crappy is to check whether it can be
read by one of the existing Cameras. The Camera OpenCV and
Camera GStreamer objects in particular are designed to be compatible
with a wide range of cameras, using the opencv-python and GStreamer
modules respectively, so it might be worth testing them on your hardware
first ! If your camera is not compatible, you’ll have to write your own
Camera object.
Just like in the previous sections, there is a template for the Camera objects :
import crappy
class MyCamera(crappy.camera.Camera):
def __init__(self):
super().__init__()
def open(self, **kwargs):
...
def get_image(self):
...
def close(self):
...
The base class from which each Camera must inherit is
crappy.camera.Camera. The open() and
close() methods are, as usual, meant for
(de-)initializing the camera and the connection to it. A big difference with
the custom classes that were defined in the previous sections is that here the
__init__() method does not accept any argument.
Instead, all the arguments to pass to the Camera will be given as kwargs to
the open() method. Why did we choose a different
implementation ? As detailed below, it is possible to define settings of the
Camera objects that can be adjusted interactively in a nice
Camera Configurator interface. Because the settings are handled in a
special way and applied during open(), it then
makes sense to catch the arguments here !
The method unique to the Camera objects is
get_image(), that should acquire one image at a
time, normally by communicating with the hardware. This method does not accept
any argument, and should return two values. The first one is the timestamp at
which the image was acquires, as returned by time.time. The second one
is the acquired image, as a numpy array or numpy-compatible object. The
image can be an array of dimension two, if it is a grey level image, or of
dimension three if it is a color image. It can also be encoded over 8 or 16
bits indifferently.
As always, let’s write a basic example to make it clear how the implementation should look like in an actual script, inspired from an example available on GitHub :
In this first example, the Camera object generates a random image with several
settings that can be adjusted in the __init__()
method. If you run the script, you’ll however notice that the settings cannot
be interactively tuned in the configuration window. The possibility to do so
will be introduced in the next paragraphs. Except for the part that customizes
the output image, the syntax is fairly simple. Once the image is acquired, it
just has to be returned by the get_image() method
along with a timestamp. Here, the open() and
close() methods don’t need to be defined as there
is no interactive setting defined nor any hardware to (de-)initialize.
In the previous example, we’ve seen that the settings couldn’t be interactively adjusted in the configuration window. To enable this feature, a set of specific methods has to be used instead of managing the settings ourselves. These methods are :
add_bool_setting(), that allows to add a setting taking a boolean value (TrueorFalse). It is accessible in the configuration window as a checkbox that can be checked and unchecked.add_choice_setting(), for adding a setting that takes onestrvalue out of a given set of possible values. It is accessible in the configuration window as a menu in which you choose one out of several possible values.add_scale_setting(), that adds a setting taking anintorfloatvalue within given limits. It is accessible in the configuration window as a horizontal slider that the user can adjust.
There are actually more methods available, but they are covered in a
dedicated section on the next page. By calling
any of the presented methods, you’ll add a
CameraSetting that manages
automatically the integration of your setting in the configuration window. It
also ensures that any value you would try to set is valid, and manages the
communication with hardware provided that you indicate a getter and a setter
method as arguments. Otherwise, the value of the setting is simply stored
internally like any other attribute. Every setting can be accessed by calling
self.name, with name the name of the setting in plain text, or
getattr(name, self) with the name as a str if the name contains
spaces. Let’s now modify the first example to include a better setting
management :
Note
To run this example, you’ll need to have the opencv-python,
matplotlib and Pillow Python modules installed.
After the changes, notice that the get_image()
method remains unchanged. The values of the settings, that were previously
defined as attributes in the __init__() method, are
still accessed the same way, because the same names were given when adding the
settings. An open() method is now defined, in which
the settings are instantiated and where their initial value can be provided as
arguments. What happens is that set_all() will call
the setter of each setting, effectively setting it on the device with the
indicated value. If set_all() is not called, the
setter is never called and there is no interaction with the hardware until you
modify a setting in the configuration window.
Here, there is no actual hardware to drive so there is no need for getters and
setters. However, an example is still provided for the high setting to
show you how it works. The getter and the setter are usually methods of the
class, in which you communicate with the camera. When changing the value of the
setting, the setter will first be called, followed by the getter to check if
the setting was set to the correct value. It is possible to only provide a
setter, or only a getter.
You can download this custom Camera example to run it locally on your
machine. Cameras are quite complex objects, so there’s much more to
discover by reading the documentation of the Camera
in the API. You can also have a look at the Cameras distributed with Crappy to see how they are implemented.
5. Custom Blocks
For the last section of this tutorial page, we are going to cover the most
difficult but also most interesting and powerful object that you can
customize in Crappy : the Block. Unlike the other objects introduced
on this page, Blocks are much more complex and as a user you are only supposed
to tune a very small part of it for your application. The rest of the code
should remain untouched, as it is the one that allows Crappy to run smoothly.
If you are able to define your own Block, you
should be able to highly customize your scripts in Crappy and to drive
almost any experimental setup ! Remember that the Blocks are usually not
meant to directly interact with hardware, the helper classes like the
Actuators and the Cameras are here for that. Instead, Blocks
usually create data, perform processing on existing data, interact with the
system, display data, etc.
5.a. Methods of the Block
First, all the Blocks must be children of the base crappy.blocks.Block
parent class. Let’s now see what methods you can define when instantiating your
own Blocks :
import crappy
class MyBlock(crappy.blocks.Block):
def __init__(self):
super().__init__()
def prepare(self):
...
def begin(self):
...
def loop(self):
...
def finish(self):
...
There are actually not that many methods for you to fill in, and almost all of them are optional ! For each method, here’s what it does and how it should be used :
__init__()should be used for initializing the Python objects that will be used in your Block. Avoid doing too much in this method, as there is no mechanism for properly de-initializing what you do there in case Crappy crashes very early. This method is also where your Block accepts arguments.prepare()is where you should perform the initialization steps necessary for your Block to run. That can include starting aThread, creating a file, populating an object, connecting to a website, etc. The actions performed here will be properly de-initialized by thefinish()method in case Crappy crashes. It is fine not to define this method if no particular setup action is required.begin()is equivalent to to the first call ofloop(). It is the moment where the Block starts being allowed to send and receive data to/from other Blocks, and performs its main task. For the very first loop, you might want to do something special, like sending a trigger to another application. If so, you should use this method. Otherwise, this method doesn’t need to be defined.loop()is a method that will be called repeatedly during the execution of the script. It is where your Block performs its main task, and can send and receive data to/from other Blocks. This method does not take any argument, and also doesn’t return anything.finish()should perform the de-initialization steps necessary to properly stop your Block before the script ends. This method should always be called, even in case something goes wrong in your script. It is fine not to define this method if no particular action is required in your Block before exiting.
Important
Avoid including any call or structure that would prevent a method of your Block from returning ! For example, avoid using blocking calls without a short timeout (at most a few seconds), and do not use infinite loops that could never end. That is because in the smooth termination scenarios, the Blocks are only told to terminate once their current method call returns. Otherwise, you’ll have to use Control-c to stop your script, which is now considered an invalid way to stop Crappy.
Now that the possible methods have been described, it is time to put them into application in an example. However, as the Block object is quite complex, such an example needs to include aspects described in the next sub-sections. So, instead of building and improving an example iteratively over the sub-sections, we’ll simply comment the relevant parts of one complete example in each sub-section.
For this example, we have created a fully functional Block that can send and/or receive data to/from network sockets. It can be useful for communicating with remote devices over a network, although the Client Server Block already provides this functionality using MQTT. The demo Block is really not advanced enough to be distributed with Crappy, but it will do just fine for this tutorial ! Here is the full code :
Note
To run this example, you’ll need to have the matplotlib Python module
installed.
In this Block, only the begin() method is not
defined. That is not a big deal, most Blocks do not need to define this method,
especially for beginners. Overall, the Block can send the value of a given
input label to a given output network address along with a timestamp. It can
also receive a value and a timestamp from a given input network address and
send it to downstream Blocks over a given output label. It can thus basically
receive and/or send data over the network. Let’s review its methods one by
one :
__init__()only sets attributes, and accepts arguments. It also instantiates two sockets, which is fine since instantiation alone does not actually trigger any connection to anything. In your own Blocks, you can define as many arguments as you want to provide the desired level of granularity, but this comes of course at the cost of complexity. You can see that some attributes have a leading underscore in their name, this is discussed in the next sub-section.In
prepare(), quite a lot of initialization is performed. There are two parts in the implementation : one executed if the Block has input Links, the other if it has output Links. If there are input Links, the Block tries to connect to the provided port at the provided address. If there are output Links, the Block waits for an external connection on the desired address and port, and accepts one connection. If any of these operations fail, an exception is raised and the Block stops.In
loop(), incoming data is first received. Then, if the data contains all the necessary information, the timestamp and the value are cast to bytes and sent over the network. If there are output Links, the Block then checks if data is ready to be read from the network. If so, it unpacks the timestamp and the value and sends them to downstream Blocks.finish()simply closes all the opened network sockets, in order to free the associated resources.
You’ll need to have a closer look at the code if you want to understand every
single line, but you should already have a rough idea of how it works. More
details about the methods and attributes that are used are given in the next
sub-sections. You can download this custom Block example to run it locally on your machine.
5.b. Useful attributes of the Block
While writing your own Blocks, you are free to use whatever names you want for
the attributes you define. Any name, really ? Actually, a few attribute
names are already used by the parent Block
class, and provide some very useful functionalities. This sub-section lists
them, as well as their meaning and effect when applicable. In the general case,
none of the attributes presented here is mandatory to use. Nothing bad can
happen if you choose not to use them, what happens if you override them is a
different story !
Note
When defining your own attributes, you can put a leading underscore in their names to indicate that an attribute is for internal use only and should not be accessed or modified by any external user or program.
Here is the exhaustive list of all the attributes you can access and their meaning :
outputsis alistcontaining the reference to all the incoming Links. It is useful for checking whether the Block has input Links or not. It should not be modified !inputsis alistcontaining the reference to all the outgoing Links. It is useful for checking whether the Block has output Links or not. It should not be modified ! It is sometimes used to put a limit on the number of incoming Links (for example theRecorderBlock raises an error if it has more than one incoming Link).nicenesscan be set during__init__(), and the corresponding niceness value will be set for the Process byrenice_all(). It is only relevant on Linux, and barely used. Most users can ignore it.freqsets the target looping frequency for the Block. It can be set to any positive value, or toNoneto switch to free-run mode. If a value is given, the Block will try to reach it but this is not guaranteed. It can be set anytime, but is usually set during__init__(). Depending on the application, a reasonable value for this attribute is usually somewhere between 20 and 200.display_freqis aboolthat enables the display of the achieved looping frequency of the Block. If set toTrue, the looping frequency is displayed in the terminal every two seconds. It can be set anytime, but is usually set during__init__().debugcan be eitherTrue,False, orNone. If set toFalse(the default), it only displays a limited amount of information in the terminal. If set toTrue, additional debug information is displayed for this Block. When the debug mode is enabled, there is usually way too much information displayed to follow ! The extra information is useful for debugging, for skilled enough users. The last option is to setdebugtoNone, in which case no information is displayed at all for the Block. That is not advised in the general case. This attribute must be set during__init__().labelscontains the names of the labels to send to downstream Blocks. When given, the values to send can be given as atuple(for example), rather than as adictcontaining both the names of the labels and the values. More about it in the next section. This attribute can be set at any moment.t0contains the timestamp of the exact moment when all the Blocks start looping together. It is useful for obtaining the timestamp of the current moment relative to the beginning of the test. This attribute can only be read starting frombegin(), and must not be modified !namecontains the unique name attributed to the Block by Crappy. It can be read at any time, and even modified. This name is only used for logging, and appears in the log messages for identifying where a message comes from.
In the presented example, you may have recognized a few of the presented attributes. They are highlighted here for convenience :
There is not much more to say about the available attributes of the Block that you can use, you’ll see for yourself which ones you need and which ones you don’t when developing !
5.c. Sending data to other Blocks
A very important aspects of Blocks is how they communicate with each other. You
normally already know that for two Blocks to exchange data, they must be linked
by a Link. But when writing your own Blocks, how to tell
Crappy what to send exactly ? That is the topic of this sub-section !
Sending data to downstream Blocks in Crappy is extremely simple, because
there is only one way to achieve it : you have to use the
send() method. This method accepts only one
argument, either a dict or an Iterable (like a
list or a tuple) of values to send (usually the values are
float or str). If a dictionary is given, its keys are the names
of the labels to send. For each label, a single value must be provided, and the
same labels should be sent throughout a given test. If the values are given in
an Iterable without labels, then the labels attribute of the Block must
have been set beforehand. The dictionary to send will be reconstructed from the
labels and the given values. There should, of course, be as many given values
as there are labels. And that’s basically all there is to know about sending
data to downstream Blocks in Crappy !
Note
The dictionary sent through the Links are exactly the same that the Modifiers can access and modify. See the dedicated section for more information.
The line in the example where the data gets sent is outlined below :
def loop(self):
# Receiving the data from upstream Blocks
data = self.recv_last_data()
# Only processing if the time and input labels are present
if data and self._label_in in data and 't(s)' in data:
t = data['t(s)']
val = data[self._label_in]
# Packing to have a standard message format
to_send = pack('<ff', t, val)
# Sending the message
self.log(logging.DEBUG, f"Sending {to_send} on {self._port_out} at "
f"address {self._address_out}")
self._sock_out.send(to_send)
# If this socket is defined, data can be received from a server socket
if self._sock_in is not None:
ready, *_ = select([self._sock_in], list(), list(), 0)
# Only proceeding if there is in-waiting data
if ready:
msg_in = self._sock_in.recv(calcsize('<ff'))
self.log(logging.DEBUG, f"received {msg_in} on {self._port_in} at "
f"address {self._address_in}")
# Unpacking the received data and sending to downstream Blocks
self.send(dict(zip(('t(s)', self._label_out), unpack('<ff', msg_in))))
As you can see, it was chosen to send a dictionary here, but a solution using
the labels attribute would also have worked. The dictionary is built in
a quite elegant way, using the zip method. You can see that the labels
to send are 't(s)' fo the time, and the chosen output label for the
transferred value. The values to send are given by the unpack
function, that returns two float from binary data.
5.d. Receiving data from other Blocks
Now that the method for sending data has been covered, it is time to describe the complementary methods that allow a Block to receive data from upstream Blocks. Things get a bit more complex at that point, because there are no less than four possible methods that you can use ! Each of them serves a different purpose, let’s review them all in this sub-section :
recv_data()is by far the simplest method. It creates an emptydict, that it updates with one message (i.e. one sentdict) from each of the incoming Links, and then returns it. This means that some data might be lost if several Links carry a same label, which is very often the case with the time label ! Also, only the first available message of each Link is read, meaning that if there are several incoming messages in the queue, only one is queued out. For this reason, this method is barely used, but it is still implemented for whoever would find an application to it !recv_last_data()is based on the same principle as the previous method, except it includes a loop that updates the dictionary to return with all the queued messages. In the end, only the latest received value for each label is present in the returned dictionary, hence the name of the method. Afill_missingargument allows to control whether the last known value of each label is included if no newer value is available, thus returning the latest known value of all the known labels (not just the ones whose values were recently received). Just like the previous method, this one doesn’t keep the integrity of the time information if there are several incoming Links, and only returns one value per label even if several messages were received.recv_all_data()allows to keep and return multiple values for each label, if several messages were received from upstream Links. To do so, it returns adictwhose keys are the received labels, but whose values arelistcontaining for each label all the successive values that were received. This way, the history of each label is preserved, which is crucial for certain applications (integration for example). However, just like the previous ones, this method isn’t safe in case several Links carry a same label. Therefore, it also doesn’t preserve the time information. Note that this method possesses two arguments for acquiring data continuously over a given delay, but you’ll need to check the API for more information about them.recv_all_data_raw()is by far the most complex way of receiving data ! This method returns everything that flows into each single incoming Link, with no possible loss ! However, the returned object is of course much more complex. Basically, it is equivalent to a call torecv_all_data()on each Link taken separately. All the results are then put together in one list, so this method returns alistofdict(one per Link) whose keys arestr(labels) and values arelistof all the received data for the given label. Using this method is mandatory when you have to retrieve all the exact timestamps for several labels that can come from different Links. You can check theGrapheror theMultiplexerBlocks for examples of usage.
There is quite much choice, but choosing the right method for your Block is
really not that difficult. Put aside recv_data()
that is almost never used, you can decide for the right method in one or two
steps. Can you work with only the latest value of each label ? If so, go for
recv_last_data(). Otherwise, will you use the
history of the time label and does your Block accept multiple incoming
Links ? If so, no other choice than using
recv_all_data_raw(). Else,
recv_all_data() will do just fine !
Note
An additional data_available() method allows
checking for the availability of new data in the incoming Links. It returns
a bool indicating whether new data is available or not. It can be
useful to avoid useless calls to recv methods.
def loop(self):
# Receiving the data from upstream Blocks
data = self.recv_last_data()
# Only processing if the time and input labels are present
if data and self._label_in in data and 't(s)' in data:
t = data['t(s)']
val = data[self._label_in]
# Packing to have a standard message format
to_send = pack('<ff', t, val)
# Sending the message
self.log(logging.DEBUG, f"Sending {to_send} on {self._port_out} at "
f"address {self._address_out}")
self._sock_out.send(to_send)
# If this socket is defined, data can be received from a server socket
if self._sock_in is not None:
ready, *_ = select([self._sock_in], list(), list(), 0)
# Only proceeding if there is in-waiting data
if ready:
msg_in = self._sock_in.recv(calcsize('<ff'))
self.log(logging.DEBUG, f"received {msg_in} on {self._port_in} at "
f"address {self._address_in}")
# Unpacking the received data and sending to downstream Blocks
self.send(dict(zip(('t(s)', self._label_out), unpack('<ff', msg_in))))
In the custom Block example, you can see that we opted for the
recv_last_data() method. The handling of the
returned data is then fairly simple, as a single dict with single values
is returned. This dictionary must still be checked before processing it, as it
might be empty or not contain all the necessary data at each loop !
Finally, that’s it ! If you have been through this entire page of tutorials and the previous ones, you should now be ready to use Crappy at a reasonably high level to drive all kinds of experimental setups. We know that the module is far from being simple, and that there are many things to keep in mind when using it. This is why we’re trying to keep the documentation as extensive as possible, and we provide a wide variety of ready-to-run examples. At that point of the tutorials, there are still a few uncovered topics only relevant to advanced users. You can check them on the next and last page of the tutorials.