• BoostPythonIntegration

Integrating Boost.Python objects with SIP and PyQt (v3)

After seeing a number of mailing list posts about whether or not it's possible to integrate extension modules made with Boost.Python with PyQt and SIP extensions, but no definite answers, I tried to figure out how to do it. Here is the result of my attempt, reduced to as simple an example as possible. An archive with all the required files is attached to this document: BoostPythonIntegration.tar.gz

The Boost.Python Extension

In my situation, I had already written Boost.Python bindings for an existing C++ library, so I will do the same thing here. This is a very straightfoward library (lifted directly from the Boost.Python documentation) which defines the class World. I put these files in the subdirectory world/.

// world.h
#ifndef WORLD_H
#define WORLD_H

#include <string>

struct World {
  void set(std::string);
  std::string greet() const;
  std::string msg;
};

#endif

// world.cpp
#include "world.h"

void World::set(std::string msg) {
  this->msg = msg;
}
std::string World::greet() const {
  return msg;
}

These will ultimately generate the file libworld.so. Now, as per the Boost.Python tutorial, we wrap it into a Python module hello:

// hello.cpp
#include <boost/python.hpp>
#include "world.h"
using namespace boost::python;

BOOST_PYTHON_MODULE(hello) {
  class_<World>("World")
    .def("greet", &World::greet)
    .def("set",&World::set)
    ;
}

To compile this in Linux, I use a simple makefile (make sure that the bottom half is TABs and not spaces). You may prefer bJam or some other tool.

# Makefile for Boost.Python component
.PHONY:all install
install: libworld.so hello.so
        cp libworld.so hello.so ..
all: libworld.so hello.so

boost_cflags = -ftemplate-depth-100 -DBOOST_PYTHON_DYNAMIC_LIB \
               -isystem /usr/include/python2.4 -I.
boost_libs = -Wl,-rpath-link,. -L/usr/lib/python2.4/config -lboost_python
world_libs = -L. -Wl,-R. -lworld

libworld.so:world.o
        g++ -o libworld.so -shared world.o
world.o:world.cpp world.h
        g++ -c -fpic world.cpp

hello.o:hello.cpp
        g++ -c $(boost_cflags) -fpic hello.cpp
hello.so:hello.o
        g++ -o hello.so -module -shared $(boost_libs) $(world_libs) hello.o

The PyQt/SIP Extension

My basic goal here is to make a PyQt widget that can take the Boost.Python object as a parameter (and hopefully also return it, although I haven't tested that yet). The simplest way to do this is to subclass from QLabel, adding an extra parameter to the constructor to pass in the World object and then to use its greet() text as the label text. I will start with the fully C++ Qt class, which knows nothing of Boost.Python - only the libworld library. The basic structure of this example is based on http://wiki.python.org/moin/EmbedingPyQtTutorial, so please look there for more information.

// mylabel.h
#ifndef MYLABEL_H
#define MYLABEL_H

#include <qlabel.h>
#include "world/world.h"

class MyLabel : public QLabel
{
  Q_OBJECT
public:
  MyLabel(const World &, QWidget *parent=0, const char *name=0 );
};

#endif

// mylabel.cpp
#include "mylabel.h"

MyLabel::MyLabel(const World &W, QWidget *parent, const char *name )
        : QLabel( parent, name )
{
  setText(W.greet());
}

The project file for qmake is straightforward (make sure you're using qmake for Qt 3)

# mylabel.pro
TEMPLATE = lib

CONFIG   += qt warn_on release

HEADERS  = mylabel.h
SOURCES  = mylabel.cpp
TARGET   = mylabel

Then running qmake mylabel.pro && make will generate libmylabel.so. We still need to wrap it into something Python can deal with, so we use SIP (since Boost.Python won't give us a compatible widget). Here is the SIP file (which goes in the same directory with the rest of the files).

// mylabel.sip
%Module mylabel 0

%Import qt/qtmod.sip

%MappedType World
{
%TypeHeaderCode
#include <boost/python.hpp>
#include "../world/world.h"
using namespace boost::python;
%End

%ConvertToTypeCode
  if (!sipIsErr) { // We're being asked whether the conversion is legal
    try {
      handle <> ph(borrowed(sipPy));
      World *w = extract<World *> (object(ph)); // error if no assignment
      return w==w; // prevent silly warning at cost of runtime overhead
    } catch (error_already_set e) {
      return 0;    // the extraction failed, so invalid type
    }
  }
  handle <> ph(borrowed(sipPy)); // Actually do the conversion
  *sipCppPtr = extract<World *> (object(ph));
  return 0;                      // Don't pass ownership
%End

%ConvertFromTypeCode
  return object(sipCpp).ptr();   // Untested - does this work?
%End
};

class MyLabel : QLabel
{
%TypeHeaderCode
#include "../mylabel.h"
%End

public:
    MyLabel(World &,QWidget * /TransferThis/ = 0, const char * = 0);
};

to run SIP, I use a python script,

# configure.py

import os
import sipconfig
import pyqtconfig

# The name of the SIP build file generated by SIP and used by the build
# system.
build_file = "mylabel.sbf"

# Get the PyQt configuration information.
config = pyqtconfig.Configuration()

# Get the extra SIP flags needed by the imported qt module.  Note that
# this normally only includes those flags (-x and -t) that relate to SIP's
# versioning system.
qt_sip_flags = config.pyqt_qt_sip_flags

# Run SIP to generate the code.  Note that we tell SIP where to find the qt
# module's specification files using the -I flag.
os.system(" ".join([ \
    config.sip_bin, \
    "-c", "sip", \
    "-b", "sip/"+build_file, \
    "-I", config.pyqt_sip_dir, \
    qt_sip_flags, \
    "mylabel.sip" \
]))

# Create the Makefile.  The QtModuleMakefile class provided by the
# pyqtconfig module takes care of all the extra preprocessor, compiler and
# linker flags needed by the Qt library.
makefile = pyqtconfig.QtModuleMakefile(
    dir="sip",
    configuration=config,
    build_file=build_file
)

# Add the library we are wrapping.  The name doesn't include any platform
# specific prefixes or extensions (e.g. the "lib" prefix on UNIX, or the
# ".dll" extension on Windows).
makefile.extra_libs = ["mylabel"]
makefile.LFLAGS.append("-L..")
makefile.LFLAGS.append("-Wl,-rpath,.")
makefile.LFLAGS.append("-Wl,-rpath,..")
makefile.LIBS.append(" -Wl,-R.") # Is this necessary?
makefile.LIBS.append("-L..")
makefile.LIBS.append("-lworld")
makefile.LIBS.append("-lboost_python")
# Generate the Makefile itself.
makefile.generate()

Make the directory sip/ and run python configure.py to get a SIP makefile. Change directory to sip/ and run make to get the Python extension module mylabel.so.

Putting It All Together

For now, put all the shared objects (libworld.so, hello.so, libmylabel.so, and mylabel.so) in the same directory to avoid library path problems (hence all the -Wl,-R. above) and we can test it all out with the following Python script

# test.py
import sys
from qt import *
from hello import *
from mylabel import *

app = QApplication(sys.argv)
hello = World()
hello.set("Hello, World")

label = MyLabel(hello)
app.setMainWidget(label)
label.show()
app.exec_loop()

Conclusion

The key step in all this was in the SIP file, declaring World as a %MappedType. This is merely a proof-of-concept, but the principles should extend to more realistic situations.

I am not at all confident that I've done this mapping correctly, but it seems to work, raising a TypeError when passed the wrong thing, and working smoothly with good input. I welcome any comments, suggestions, or improvements. As an interesting addendum, the first working version of this was from adding a World.address() method in the Boost.Python bindings which returned the address of the object typecast to a long, and then the MyLabel object taking a long as input and casting it back to a World *. While this worked, it was terribly unsafe and clearly suboptimal. But it was a key step in getting everything working.

The (original) author

This was worked out and compiled by Steve Hicks (kupopo1 -at- hotpop -dot- com). I welcome any questions or comments (or just edit them into the page...).