How to use C or C++ code in Crappy ?

If you want to use hardware that’s only available on C or C++ platforms, or if you need to run computationally intensive code in an efficient way, it is quite easy to bind C/C++ code to Crappy.

1. Using an existing library

If you want to integrate a library from an external source and you already have the corresponding file (.so, .dll, etc.), different solutions are available. We’re not going to detail them here, as we could not provide a cross-platform compatible example. You can however look into the pyspcm and comedi_bind tools for an example using the ctypes library and .so files. The pyspcm tool is a binding for using Spectrum high-speed acquisition boards, and comedi is a binding for using the comedi driver. Alternative solutions include Cython, cffi, PyBind11, and many others.

2. Writing your own library

2.1. Prerequisites

Here is detailed a complete example on how to bind C and C++ language with Python and add it to the Crappy package. Linux and Windows are both used for building.

Note

This is not a C++ tutorial, a few programming notions are used here, please refer to the tutorials below if you are not a C or C++ developer.

C++ tutorials :

• openclassrooms C++ (fr)

• cplusplus (en)

C tutorials :

• openclassrooms C (fr)

• cprogramming (en)

In Linux, you must install the python-dev package to ensure that you can use the Python.h library in the C or C++ code :

sudo apt install python-dev

In Windows, there is no python-dev package, but the Python installer for Windows will install a subdirectory in the Python directory which contains the Python.h (on our Windows machine, this folder is located in C:\Users\<username>\AppData\Local\Programs\Python\).

Important

In this tutorial we’ll assume that you’re using Crappy from a setup install (see the Installation section for details). Using C++ modules is otherwise not possible using only a pip install, as we made the choice not to distribute compiled code.

2.2. First example: a simple function

2.2.1. Writing the C++ code

First we need to start with including the Python.h library, and of course any other library we want to use. Then we write our function, that must return a PyObject pointer. If the function takes arguments (that’s the case in the example), they should be passed to args. The arguments then have to be parsed using PyArgs_ParseTuple, and we can finally write the main part of the function. Do not forget to add Py_RETURN_NONE at the end if the function doesn’t return anything.

#include <Python.h>
#include <iostream>

using namespace std;

static PyObject* hello(PyObject* self, PyObject* args){
    const char* name;
    if(!PyArg_ParseTuple(args, "s", &name))
        return NULL;
    cout << "Hello " << name << endl;

    Py_RETURN_NONE;
}

Then, to bind the hello function we need to create a PyMethodDef which contains the function definition. If several functions were to be defined, we would list them all here. The first element will be the name of the function in Python. The second element is the function to bind. The third element is METH_VARARGS if the function gets arguments, or METH_NOARGS otherwise. The last element corresponds to a description of the function, that will appear in the function help.

#include <Python.h>
#include <iostream>

using namespace std;

static PyObject* hello(PyObject* self, PyObject* args){
    const char* name;
    if(!PyArg_ParseTuple(args, "s", &name))
        return NULL;
    cout << "Hello " << name << endl;

    Py_RETURN_NONE;
}

static PyMethodDef HelloMethods[] =
{
    {"hello", hello, METH_VARARGS, "Say hello to somebody."},
    {NULL, NULL, 0, NULL}
};

Once all the Python objects have been defined, we need to define the Python module itself. This is done using PyModuleDef. It has to be initialized with PyModuleDef_HEAD_INIT, then comes the module name, the docstring if any, the size to allocate to the module, the methods of the module, anf finally the slots.

#include <Python.h>
#include <iostream>

using namespace std;

static PyObject* hello(PyObject* self, PyObject* args){
    const char* name;
    if(!PyArg_ParseTuple(args, "s", &name))
        return NULL;
    cout << "Hello " << name << endl;

    Py_RETURN_NONE;
}

static PyMethodDef HelloMethods[] =
{
    {"hello", hello, METH_VARARGS, "Say hello to somebody."},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef helloModule = {
    PyModuleDef_HEAD_INIT,
    "helloModule",
    NULL,
    -1,
    HelloMethods
};

The last step is to initialize the module, which is done using PyMODINIT_FUNC and PyModule_Create. The C++ code is then ready to be compiled !

#include <Python.h>
#include <iostream>

using namespace std;

static PyObject* hello(PyObject* self, PyObject* args){
    const char* name;
    if(!PyArg_ParseTuple(args, "s", &name))
        return NULL;
    cout << "Hello " << name << endl;

    Py_RETURN_NONE;
}

static PyMethodDef HelloMethods[] =
{
    {"hello", hello, METH_VARARGS, "Say hello to somebody."},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef helloModule = {
    PyModuleDef_HEAD_INIT,
    "helloModule",
    NULL,
    -1,
    HelloMethods
};

PyMODINIT_FUNC PyInit_helloModule(void)
{
    return PyModule_Create(&helloModule);
}

2.2.2. Binding the code to Crappy

Once the C++ code is written, most of the work is done. We only need to modify the setup.py and one of the __init__.py files. So let’s start with the setup.py. All we have to do is include our .cpp file as an extension, which is achieved by writing :

helloModule = Extension('tool.helloModule',
                        sources=['sources/hello/hello.cpp'],
                        extra_compile_args=["-l", "python%s" % v],
                        language='c++')

extensions.append(helloModule)

This should be put around line 30. The first argument indicates where to write the binary file that will be generated, the second points to the location(s) of the .cpp and/or .hpp files to use for the extension, and the two last arguments should be left as is.

Then the module should be imported in the __init__.py file of the folder where the compiled file will be written, so in our example in crappy/tool/. The import is similar to all the regular ones, i.e. in our example we should write :

from .helloModule import hello

The last step is to reinstall Crappy, and that’s it ! During install any error or warning related to the compilation of the C files will be displayed. After completing the install, there should be no notable change in the source folder. If you go to the install folder (see here), there should be a binary file in the tool folder as well as a helloModule.py file. This file contains the following code :

def __bootstrap__():
    global __bootstrap__, __loader__, __file__
    import sys, pkg_resources, imp
    __file__ = pkg_resources.resource_filename(__name__, '<binary_file.xxx>')
    __loader__ = None; del __bootstrap__, __loader__
    imp.load_dynamic(__name__,__file__)
__bootstrap__()

It’s this file that actually allows the import in __init__.py to happen. So the hello method is now part of Crappy and lives in crappy.tool.hello. For using it in a script or in a command line, we can simply write :

>>> import crappy
>>> crappy.tool.hello('world')
Hello world

2.3. Second example: a simple class

2.3.1 Writing the C++ code

Now let’s try to build a more advanced Python object in C. We’ll define a class that is similar to this Python class :

class Hello:

    def __init__(self, name="Crappy"):
        self.name = name

    def say_hello(self):
        print 'hello ', self.name

    def get_name(self):
        return self.name

After including the necessary packages, we first need to define the functions to construct our future class :

• a new method

• a constructor

• a destructor and a structure which will contain the class attributes.

Here, the struct contains two elements. The first, PyObject_HEAD must always be defined, it represent the type of object. The second element represents our attribute 'name'.

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

The next method parses the arguments and keywords arguments, to initialize the structure defined before, which will be passed as first argument for each method (similar to the Python self).

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    Hello *self;
    self = (Hello *)type->tp_alloc(type, 0);
    static char *kwlist[] = {(char*)"name", NULL};
    if (self != NULL) {
        if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
            &self->name)){
                return NULL;
        }
    }
    return (PyObject *)self;
}

The constructor parses the arguments and keywords arguments. The "name" argument is optional. Here’s also the destructor.

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    Hello *self;
    self = (Hello *)type->tp_alloc(type, 0);
    static char *kwlist[] = {(char*)"name", NULL};
    if (self != NULL) {
        if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
            &self->name)){
                return NULL;
        }
    }
    return (PyObject *)self;
}

static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = {(char*)"name", NULL};

    self->name = (char*)"Crappy";
    if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
            return 1;
    }
    return 0;
}

static void Hello_dealloc(Hello* self)
{
    Py_TYPE(self)->tp_free((PyObject*)self);
}

We then define our two methods like previously. To return a value, we need to use the Py_BuildValue function, to convert C++ type to python type: this way, we directly get an understandable Python object.

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    Hello *self;
    self = (Hello *)type->tp_alloc(type, 0);
    static char *kwlist[] = {(char*)"name", NULL};
    if (self != NULL) {
        if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
            &self->name)){
                return NULL;
        }
    }
    return (PyObject *)self;
}

static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = {(char*)"name", NULL};

    self->name = (char*)"Crappy";
    if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
            return 1;
    }
    return 0;
}

static void Hello_dealloc(Hello* self)
{
    Py_TYPE(self)->tp_free((PyObject*)self);
}

PyObject*
Hello_get(Hello *self)
{
    return Py_BuildValue("s", self->name);
}

PyObject*
Hello_print(Hello *self)
{
    cout << "Hello " << self->name << endl;
    Py_RETURN_NONE;
}

Now just like in the previous example we need to list the different methods of our module.

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    Hello *self;
    self = (Hello *)type->tp_alloc(type, 0);
    static char *kwlist[] = {(char*)"name", NULL};
    if (self != NULL) {
        if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
            &self->name)){
                return NULL;
        }
    }
    return (PyObject *)self;
}

static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = {(char*)"name", NULL};

    self->name = (char*)"Crappy";
    if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
            return 1;
    }
    return 0;
}

static void Hello_dealloc(Hello* self)
{
    Py_TYPE(self)->tp_free((PyObject*)self);
}

PyObject*
Hello_get(Hello *self)
{
    return Py_BuildValue("s", self->name);
}

PyObject*
Hello_print(Hello *self)
{
    cout << "Hello " << self->name << endl;
    Py_RETURN_NONE;
}

static PyMethodDef Hello_methods[] = {
    {"say_hello", (PyCFunction)Hello_print, METH_VARARGS,
     "Say hello to somebody."},
    {"get_name", (PyCFunction)Hello_get, METH_NOARGS,
    "Return the name attribute."},
    {NULL}
};

To define a class which can be bound with Python, we need to define its structure, with a PyTypeObject. We have to define:

• the constructor

• the destructor

• the new method

• the name of the class

• its size

• its methods

• etc.

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    Hello *self;
    self = (Hello *)type->tp_alloc(type, 0);
    static char *kwlist[] = {(char*)"name", NULL};
    if (self != NULL) {
        if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
            &self->name)){
                return NULL;
        }
    }
    return (PyObject *)self;
}

static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = {(char*)"name", NULL};

    self->name = (char*)"Crappy";
    if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
            return 1;
    }
    return 0;
}

static void Hello_dealloc(Hello* self)
{
    Py_TYPE(self)->tp_free((PyObject*)self);
}

PyObject*
Hello_get(Hello *self)
{
    return Py_BuildValue("s", self->name);
}

PyObject*
Hello_print(Hello *self)
{
    cout << "Hello " << self->name << endl;
    Py_RETURN_NONE;
}

static PyMethodDef Hello_methods[] = {
    {"say_hello", (PyCFunction)Hello_print, METH_VARARGS,
     "Say hello to somebody."},
    {"get_name", (PyCFunction)Hello_get, METH_NOARGS,
    "Return the name attribute."},
    {NULL}
};

static PyTypeObject helloType = {
    PyObject_HEAD_INIT(NULL)
    .tp_name = "crappy.tool.Hello",
    .tp_basicsize = sizeof(Hello),
    .tp_itemsize = 0,
    .tp_dealloc = (destructor) Hello_dealloc,
    .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
    .tp_doc = "Hello objects",
    .tp_methods = Hello_methods,
    .tp_init = (initproc) Hello_init,
    .tp_new = Hello_new,
};

Finally just like in the first example we have to define the module and to initialize it. Here the syntax is a bit more complex but the idea remains the same.

#include <Python.h>
#include <iostream>

using namespace std;

typedef struct {
    PyObject_HEAD
    char *name;
} Hello;

static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    Hello *self;
    self = (Hello *)type->tp_alloc(type, 0);
    static char *kwlist[] = {(char*)"name", NULL};
    if (self != NULL) {
        if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
            &self->name)){
                return NULL;
        }
    }
    return (PyObject *)self;
}

static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = {(char*)"name", NULL};

    self->name = (char*)"Crappy";
    if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
            return 1;
    }
    return 0;
}

static void Hello_dealloc(Hello* self)
{
    Py_TYPE(self)->tp_free((PyObject*)self);
}

PyObject*
Hello_get(Hello *self)
{
    return Py_BuildValue("s", self->name);
}

PyObject*
Hello_print(Hello *self)
{
    cout << "Hello " << self->name << endl;
    Py_RETURN_NONE;
}

static PyMethodDef Hello_methods[] = {
    {"say_hello", (PyCFunction)Hello_print, METH_VARARGS,
     "Say hello to somebody."},
    {"get_name", (PyCFunction)Hello_get, METH_NOARGS,
    "Return the name attribute."},
    {NULL}
};

static PyTypeObject helloType = {
    PyObject_HEAD_INIT(NULL)
    .tp_name = "crappy.tool.Hello",
    .tp_basicsize = sizeof(Hello),
    .tp_itemsize = 0,
    .tp_dealloc = (destructor) Hello_dealloc,
    .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
    .tp_doc = "Hello objects",
    .tp_methods = Hello_methods,
    .tp_init = (initproc) Hello_init,
    .tp_new = Hello_new,
};

static struct PyModuleDef helloClassModule = {
    PyModuleDef_HEAD_INIT,
    "helloClassModule",
    NULL,
    -1,
    Hello_methods
};

PyMODINIT_FUNC
PyInit_helloClassModule(void)
{
    PyObject* m;
    PyType_Ready(&helloType);

    m = PyModule_Create(&helloClassModule);

    Py_INCREF(&helloType);
    PyModule_AddObject(m, "Hello", (PyObject *) &helloType);
    return m;
}

2.3.2 Binding the code to Crappy

Now that the C++ code is ready, let’s add it to the extensions in setup.py :

helloClassModule = Extension('tool.helloClassModule',
                             sources=['sources/hello/hello_class.cpp'],
                             extra_compile_args=["-l", "python%s" % v],
                             language='c++')

extensions.append(helloClassModule)

We also need to import it from __init__.py in crappy/tool/ :

from .helloClassModule import Hello

After reinstalling Crappy, we can now use our class very simply :

>>> import crappy
>>> default = crappy.tool.Hello()
>>> default.get_name()
'Crappy'
>>> default.say_hello()
Hello Crappy
>>> with_arg = crappy.tool.Hello('Bob')
>>> with_arg.get_name()
'bob'
>>> with_arg.say_hello()
Hello bob