How to use C or C++ code in Crappy ?
If you want to use hardware that’s only available on C or C++ platforms, or if you need to run computationally intensive code in an efficient way, it is quite easy to bind C/C++ code to Crappy.
1. Using an existing library
If you want to integrate a library from an external source and you already have
the corresponding file (.so
, .dll
, etc.), different solutions are
available. We’re not going to detail them here, as we could not provide a
cross-platform compatible example. You can however look into the pyspcm and comedi_bind tools for an example using the
ctypes
library and .so
files. The pyspcm tool is a binding for using
Spectrum high-speed acquisition boards, and comedi is a binding for using the
comedi driver. Alternative solutions include Cython, cffi
, PyBind11, and
many others.
2. Writing your own library
2.1. Prerequisites
Here is detailed a complete example on how to bind C and C++ language with Python and add it to the Crappy package. Linux and Windows are both used for building.
Note
This is not a C++ tutorial, a few programming notions are used here, please refer to the tutorials below if you are not a C or C++ developer.
C++ tutorials :
openclassrooms C++ (fr)
cplusplus (en)
C tutorials :
openclassrooms C (fr)
cprogramming (en)
In Linux, you must install the python-dev
package to ensure that you can use
the Python.h
library in the C or C++ code :
sudo apt install python-dev
In Windows, there is no python-dev
package, but the Python installer for
Windows will install a subdirectory in the Python directory which contains the
Python.h
(on our Windows machine, this folder is located in
C:\Users\<username>\AppData\Local\Programs\Python\
).
Important
In this tutorial we’ll assume that you’re using Crappy from a setup
install (see the Installation section for details). Using C++ modules
is otherwise not possible using only a pip
install, as we made the choice
not to distribute compiled code.
2.2. First example: a simple function
2.2.1. Writing the C++ code
First we need to start with including the Python.h
library, and of course
any other library we want to use. Then we write our function, that must return a
PyObject
pointer. If the function takes arguments (that’s the case in the
example), they should be passed to args
. The arguments then have to be
parsed using PyArgs_ParseTuple
, and we can finally write the main part of
the function. Do not forget to add Py_RETURN_NONE
at the end if the function
doesn’t return anything.
#include <Python.h>
#include <iostream>
using namespace std;
static PyObject* hello(PyObject* self, PyObject* args){
const char* name;
if(!PyArg_ParseTuple(args, "s", &name))
return NULL;
cout << "Hello " << name << endl;
Py_RETURN_NONE;
}
Then, to bind the hello
function we need to create a PyMethodDef
which
contains the function definition. If several functions were to be defined, we
would list them all here. The first element will be the name of the function in
Python. The second element is the function to bind. The third element is
METH_VARARGS
if the function gets arguments, or METH_NOARGS
otherwise.
The last element corresponds to a description of the function, that will appear
in the function help.
#include <Python.h>
#include <iostream>
using namespace std;
static PyObject* hello(PyObject* self, PyObject* args){
const char* name;
if(!PyArg_ParseTuple(args, "s", &name))
return NULL;
cout << "Hello " << name << endl;
Py_RETURN_NONE;
}
static PyMethodDef HelloMethods[] =
{
{"hello", hello, METH_VARARGS, "Say hello to somebody."},
{NULL, NULL, 0, NULL}
};
Once all the Python objects have been defined, we need to define the Python
module itself. This is done using PyModuleDef
. It has to be initialized
with PyModuleDef_HEAD_INIT
, then comes the module name, the docstring if
any, the size to allocate to the module, the methods of the module, anf finally
the slots.
#include <Python.h>
#include <iostream>
using namespace std;
static PyObject* hello(PyObject* self, PyObject* args){
const char* name;
if(!PyArg_ParseTuple(args, "s", &name))
return NULL;
cout << "Hello " << name << endl;
Py_RETURN_NONE;
}
static PyMethodDef HelloMethods[] =
{
{"hello", hello, METH_VARARGS, "Say hello to somebody."},
{NULL, NULL, 0, NULL}
};
static struct PyModuleDef helloModule = {
PyModuleDef_HEAD_INIT,
"helloModule",
NULL,
-1,
HelloMethods
};
The last step is to initialize the module, which is done using
PyMODINIT_FUNC
and PyModule_Create
. The C++ code is then ready to be
compiled !
#include <Python.h>
#include <iostream>
using namespace std;
static PyObject* hello(PyObject* self, PyObject* args){
const char* name;
if(!PyArg_ParseTuple(args, "s", &name))
return NULL;
cout << "Hello " << name << endl;
Py_RETURN_NONE;
}
static PyMethodDef HelloMethods[] =
{
{"hello", hello, METH_VARARGS, "Say hello to somebody."},
{NULL, NULL, 0, NULL}
};
static struct PyModuleDef helloModule = {
PyModuleDef_HEAD_INIT,
"helloModule",
NULL,
-1,
HelloMethods
};
PyMODINIT_FUNC PyInit_helloModule(void)
{
return PyModule_Create(&helloModule);
}
2.2.2. Binding the code to Crappy
Once the C++ code is written, most of the work is done. We only need to modify
the setup.py
and one of the __init__.py
files. So let’s start with the
setup.py
. All we have to do is include our .cpp
file as an extension,
which is achieved by writing :
helloModule = Extension('tool.helloModule',
sources=['sources/hello/hello.cpp'],
extra_compile_args=["-l", "python%s" % v],
language='c++')
extensions.append(helloModule)
This should be put around line 30. The first argument indicates where to write
the binary file that will be generated, the second points to the location(s) of
the .cpp
and/or .hpp
files to use for the extension, and the two last
arguments should be left as is.
Then the module should be imported in the __init__.py
file of the folder
where the compiled file will be written, so in our example in crappy/tool/
.
The import is similar to all the regular ones, i.e. in our example we should
write :
from .helloModule import hello
The last step is to reinstall Crappy, and that’s it ! During install any error
or warning related to the compilation of the C files will be displayed. After
completing the install, there should be no notable change in the source folder.
If you go to the install folder (see here), there should be a binary file in the tool
folder as
well as a helloModule.py
file. This file contains the following code :
def __bootstrap__():
global __bootstrap__, __loader__, __file__
import sys, pkg_resources, imp
__file__ = pkg_resources.resource_filename(__name__, '<binary_file.xxx>')
__loader__ = None; del __bootstrap__, __loader__
imp.load_dynamic(__name__,__file__)
__bootstrap__()
It’s this file that actually allows the import in __init__.py
to happen. So
the hello
method is now part of Crappy and lives in crappy.tool.hello
.
For using it in a script or in a command line, we can simply write :
>>> import crappy
>>> crappy.tool.hello('world')
Hello world
2.3. Second example: a simple class
2.3.1 Writing the C++ code
Now let’s try to build a more advanced Python object in C. We’ll define a class that is similar to this Python class :
class Hello:
def __init__(self, name="Crappy"):
self.name = name
def say_hello(self):
print 'hello ', self.name
def get_name(self):
return self.name
After including the necessary packages, we first need to define the functions to construct our future class :
a new method
a constructor
a destructor and a structure which will contain the class attributes.
Here, the struct
contains two elements. The first, PyObject_HEAD
must
always be defined, it represent the type of object. The second element
represents our attribute 'name'
.
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
The next method parses the arguments and keywords arguments, to initialize the
structure defined before, which will be passed as first argument for each method
(similar to the Python self
).
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Hello *self;
self = (Hello *)type->tp_alloc(type, 0);
static char *kwlist[] = {(char*)"name", NULL};
if (self != NULL) {
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
&self->name)){
return NULL;
}
}
return (PyObject *)self;
}
The constructor parses the arguments and keywords arguments. The "name"
argument is optional. Here’s also the destructor.
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Hello *self;
self = (Hello *)type->tp_alloc(type, 0);
static char *kwlist[] = {(char*)"name", NULL};
if (self != NULL) {
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
&self->name)){
return NULL;
}
}
return (PyObject *)self;
}
static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {(char*)"name", NULL};
self->name = (char*)"Crappy";
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
return 1;
}
return 0;
}
static void Hello_dealloc(Hello* self)
{
Py_TYPE(self)->tp_free((PyObject*)self);
}
We then define our two methods like previously. To return a value, we need to
use the Py_BuildValue
function, to convert C++ type to python type: this
way, we directly get an understandable Python object.
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Hello *self;
self = (Hello *)type->tp_alloc(type, 0);
static char *kwlist[] = {(char*)"name", NULL};
if (self != NULL) {
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
&self->name)){
return NULL;
}
}
return (PyObject *)self;
}
static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {(char*)"name", NULL};
self->name = (char*)"Crappy";
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
return 1;
}
return 0;
}
static void Hello_dealloc(Hello* self)
{
Py_TYPE(self)->tp_free((PyObject*)self);
}
PyObject*
Hello_get(Hello *self)
{
return Py_BuildValue("s", self->name);
}
PyObject*
Hello_print(Hello *self)
{
cout << "Hello " << self->name << endl;
Py_RETURN_NONE;
}
Now just like in the previous example we need to list the different methods of our module.
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Hello *self;
self = (Hello *)type->tp_alloc(type, 0);
static char *kwlist[] = {(char*)"name", NULL};
if (self != NULL) {
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
&self->name)){
return NULL;
}
}
return (PyObject *)self;
}
static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {(char*)"name", NULL};
self->name = (char*)"Crappy";
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
return 1;
}
return 0;
}
static void Hello_dealloc(Hello* self)
{
Py_TYPE(self)->tp_free((PyObject*)self);
}
PyObject*
Hello_get(Hello *self)
{
return Py_BuildValue("s", self->name);
}
PyObject*
Hello_print(Hello *self)
{
cout << "Hello " << self->name << endl;
Py_RETURN_NONE;
}
static PyMethodDef Hello_methods[] = {
{"say_hello", (PyCFunction)Hello_print, METH_VARARGS,
"Say hello to somebody."},
{"get_name", (PyCFunction)Hello_get, METH_NOARGS,
"Return the name attribute."},
{NULL}
};
To define a class which can be bound with Python, we need to define its
structure, with a PyTypeObject
. We have to define:
the constructor
the destructor
the new method
the name of the class
its size
its methods
etc.
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Hello *self;
self = (Hello *)type->tp_alloc(type, 0);
static char *kwlist[] = {(char*)"name", NULL};
if (self != NULL) {
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
&self->name)){
return NULL;
}
}
return (PyObject *)self;
}
static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {(char*)"name", NULL};
self->name = (char*)"Crappy";
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
return 1;
}
return 0;
}
static void Hello_dealloc(Hello* self)
{
Py_TYPE(self)->tp_free((PyObject*)self);
}
PyObject*
Hello_get(Hello *self)
{
return Py_BuildValue("s", self->name);
}
PyObject*
Hello_print(Hello *self)
{
cout << "Hello " << self->name << endl;
Py_RETURN_NONE;
}
static PyMethodDef Hello_methods[] = {
{"say_hello", (PyCFunction)Hello_print, METH_VARARGS,
"Say hello to somebody."},
{"get_name", (PyCFunction)Hello_get, METH_NOARGS,
"Return the name attribute."},
{NULL}
};
static PyTypeObject helloType = {
PyObject_HEAD_INIT(NULL)
.tp_name = "crappy.tool.Hello",
.tp_basicsize = sizeof(Hello),
.tp_itemsize = 0,
.tp_dealloc = (destructor) Hello_dealloc,
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
.tp_doc = "Hello objects",
.tp_methods = Hello_methods,
.tp_init = (initproc) Hello_init,
.tp_new = Hello_new,
};
Finally just like in the first example we have to define the module and to initialize it. Here the syntax is a bit more complex but the idea remains the same.
#include <Python.h>
#include <iostream>
using namespace std;
typedef struct {
PyObject_HEAD
char *name;
} Hello;
static PyObject* Hello_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Hello *self;
self = (Hello *)type->tp_alloc(type, 0);
static char *kwlist[] = {(char*)"name", NULL};
if (self != NULL) {
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist,
&self->name)){
return NULL;
}
}
return (PyObject *)self;
}
static int Hello_init(Hello *self, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {(char*)"name", NULL};
self->name = (char*)"Crappy";
if (! PyArg_ParseTupleAndKeywords(args, kwds, "|s", kwlist, &self->name)){
return 1;
}
return 0;
}
static void Hello_dealloc(Hello* self)
{
Py_TYPE(self)->tp_free((PyObject*)self);
}
PyObject*
Hello_get(Hello *self)
{
return Py_BuildValue("s", self->name);
}
PyObject*
Hello_print(Hello *self)
{
cout << "Hello " << self->name << endl;
Py_RETURN_NONE;
}
static PyMethodDef Hello_methods[] = {
{"say_hello", (PyCFunction)Hello_print, METH_VARARGS,
"Say hello to somebody."},
{"get_name", (PyCFunction)Hello_get, METH_NOARGS,
"Return the name attribute."},
{NULL}
};
static PyTypeObject helloType = {
PyObject_HEAD_INIT(NULL)
.tp_name = "crappy.tool.Hello",
.tp_basicsize = sizeof(Hello),
.tp_itemsize = 0,
.tp_dealloc = (destructor) Hello_dealloc,
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
.tp_doc = "Hello objects",
.tp_methods = Hello_methods,
.tp_init = (initproc) Hello_init,
.tp_new = Hello_new,
};
static struct PyModuleDef helloClassModule = {
PyModuleDef_HEAD_INIT,
"helloClassModule",
NULL,
-1,
Hello_methods
};
PyMODINIT_FUNC
PyInit_helloClassModule(void)
{
PyObject* m;
PyType_Ready(&helloType);
m = PyModule_Create(&helloClassModule);
Py_INCREF(&helloType);
PyModule_AddObject(m, "Hello", (PyObject *) &helloType);
return m;
}
2.3.2 Binding the code to Crappy
Now that the C++ code is ready, let’s add it to the extensions in setup.py
:
helloClassModule = Extension('tool.helloClassModule',
sources=['sources/hello/hello_class.cpp'],
extra_compile_args=["-l", "python%s" % v],
language='c++')
extensions.append(helloClassModule)
We also need to import it from __init__.py
in crappy/tool/
:
from .helloClassModule import Hello
After reinstalling Crappy, we can now use our class very simply :
>>> import crappy
>>> default = crappy.tool.Hello()
>>> default.get_name()
'Crappy'
>>> default.say_hello()
Hello Crappy
>>> with_arg = crappy.tool.Hello('Bob')
>>> with_arg.get_name()
'bob'
>>> with_arg.say_hello()
Hello bob