So, you know that Qt uses Pimpl (Private Implementation or Opaque pointer) which is very effective mechanism to keep binary compatibility between Qt versions. I use it very often too, but mostly not because of binary compatibility needs, but because of possibility of hiding internals and implementations especialities into cpp files.

And there is something even more important, such practice allows to avoid including 3-rd party headers from your headers – your API – you know, it can often happen that you need some “utility” api, but because of including the header it leads to bring bunch of unexpected includes coming from this “utility”. Even worse, these 3-rd party headers that you probably can not even control, bring unexpected defines, pragmas etc that can makes you puzzled for hours to understand why?, how? and in which order? it conflicts with your coding.

Using d pointer (Private Implementation) allows really avoid including these unneeded headers, but recently in huge project it makes me to think about performance issues. Usually it looks like:

// HEADER *****************
class DynObj {
public:
    DynObj();
    virtual ~DynObj();

private:
    Private * d;
};

// SOURCE *****************
class DynObj::Private {
public:
    //internals
};

DynObj::DynObj() { d = new Private; }
DynObj::~DynObj() { delete d; }

It is very effective for all advantages that I described above except one: if this is expected to be some lightweight object then when you instantiate one you actually get two memory allocation operations: one is for object itself and second one for the Private object. And if you have control for first allocation – stack or heap, you do not have control of second one – always heap.

In my case I used billions of these objects as parameters of methods, return values, local variables etc. Problem is that even if you expect that the object will be allocated in stack, still due to Private sub-object it will request “new” and “delete” so heap is involved in any case, but surely using just stack should be faster. How much? Let’s do some lab to measure.

Lab definition:

1. Define two classes, one is just standard class using d-pointer, second should be fully allocated in stack if needed
2. Still both should use idea of Private Implementation, so need some tricks for second class
3. Measure time spent on allocation of objects of two types

So, we would like to have Private Implementation, but use kind of d-pointer, so when object allocation requests memory it should give enough amount of bytes with reserved space to fit Private object later. Of course we can just define quint8 d_bytes[d_size];, but compiler can not calculate d_size without knowing details of implementation of Private, but it breaks idea of closing internals into cpp only.

We can do just some tricking, we declare some magic number static const int d_size;, but in cpp we use static assert to compare this magic number with real size of Private.

Code to do such static assert is:

namespace static_assert {
template <bool> struct is_fail;
template <> struct is_fail<true> { enum { value = 1 }; };
}

template <int def,int real>
struct check_d_size : ::static_assert::is_fail<(bool)(def == real)> {};

Then in constructor we can place code:

check_d_size<d_size,sizeof(Private)>();

And on compilation we will get error like that:
../cpp-dyn-vs-stack/main.cpp: In instantiation of 'check_d_size<31, 32>':

Now easy to know right magic number.
BTW: Surely totally unportable trick! But fine for our lab and measurements.

Another issue: we have enough memory reserved in object, but we need initialize it in right way – use constructor, but with ready memory. Like that:

class StackObj {
public:
    StackObj() {
        check_d_size<d_size,sizeof(Private)>();
        d = new(d_bytes) Private;
    }
    virtual ~StackObj() { d->~Private(); }
private:
    class Private {
    public:
        inline void * operator new(size_t, quint8 * mem) { return mem; }
        ...
    };
private:
    Private * d;
    static const int d_size = 32;
    quint8 d_bytes[d_size];
};

Now if our StackObj is placed in stack then 1. appropriate memory is reserved, 2. Private sub-object is initialized in right way with all constructors of internals etc. And we still can keep implementation with internals in cpp and leave only API in header without 3-rd party includes.

Time to measures:

• allocate 100,000 objects in stack using heap for Private: 24 msecs
• allocate 100,000 objects in stack using stack for Private: 8 msecs

Wow! 3-times difference.
So, keep track what you can put in stack and what in heap if you count your computing time!

Full listing:

#include <iostream>
#include <string>

#include <QTime>
#include <QDebug>

using namespace std;

class DynObj {
public:
    DynObj() { d = new Private; }
    virtual ~DynObj() { delete d; }

private:
    class Private {
    public:
        int i;
        int j;
        DynObj * p;
        std::string str;
        class Check {
        public: Check() { static bool b=true; if (b) { qDebug() << "ok new dyn"; b = !b; } }
               ~Check() { static bool b=true; if (b) { qDebug() << "ok del dyn"; b = !b; } }
        } chk;
    };
    Private * d;
};

namespace static_assert {
template <bool> struct is_fail;
template <> struct is_fail<true> { enum { value = 1 }; };
}

template <int def,int real>
struct check_d_size : ::static_assert::is_fail<(bool)(def == real)> {};

class StackObj {
public:
    StackObj() {
        check_d_size<d_size,sizeof(Private)>();
        d = new(d_bytes) Private;
    }
    virtual ~StackObj() { d->~Private(); }

private:
    class Private {
    public:
        inline void * operator new(size_t, quint8 * mem) { return mem; }

        int i;
        int j;
        DynObj * p;
        std::string str;

        class Check {
        public: Check() { static bool b=true; if (b) { qDebug() << "ok new stack"; b = !b; } }
               ~Check() { static bool b=true; if (b) { qDebug() << "ok del stack"; b = !b; } }
        } chk;
    };
    Private * d;
    static const int d_size = 32;
    quint8 d_bytes[d_size];
};

int main()
{
    QTime t = QTime::currentTime();

    { DynObj ar_dyn[100000]; }

    qDebug() << "use dyn, call time, msecs" << t.msecsTo(QTime::currentTime());

    t = QTime::currentTime();

    { StackObj ar_stack[100000]; }

    qDebug() << "use stack, call time, msecs" << t.msecsTo(QTime::currentTime());

    return 0;
}

But you may encounter one area which is totally uncovered – there is no way of joining of models into one composition proxy model. So I would like to present some “Gem” for Qt model-view programmers: RowsJoinerProxy class. It does very simple thing: join all top level rows from first source model, second, etc. It does not affect children, so all hierarchies of source models are kept. For root columns it uses qMax(columCount() of roots of source models). Rows inserts, rows removals, data changes of source models are recognized and mapped. Column operations are not supported at the moment, so I may update it later. Inserting/Removing source model from proxy makes model reset, though it can be improved later to do just rows removal.

Files:
RowsJoinerProxy.h
RowsJoinerProxy.cpp

For long time I had one application in my attic. It was an interesting experiment for studying and remembering foreign words. This application do kind of training by repeating words and force you to type correct spelling while application pronounce it to you using TTS (surely you need high-quality voice engine – I used voices from Infovox iVox). That was pretty effective and helped me a lot for studying new words and forming active vocabulary for Norwegian. Because of hearing, reading and typing it makes very effective associations in your brains. The application is cross-platform and requires only QtGui, QtXml and QtCore, also qt plugins are not need.

First I worked on adjustments of widgets and look-and-feel to have app closer to Mac style. It wasn’t complicated because on each release Nokia improves drawing of widgets to have it very close to native views. You can find differences in user interface if you know especialities.

Then application (bundle) was build with inclusion of all required libs as private frameworks. I got too much for such small application — about 60MB. For me it is too heavy and I decided to make it smaller. Also after checking Qt forums I found that some patches are required for Qt to get a ticket into store.

It was chosen Carbon version of Qt because I found some glitches in GUI when used Cocoa – so I decided not to spend time on it. Then disabled everything that is not needed in configure, also used 10.5 sdk and only 32 bit:

./configure -carbon -fast -no-qt3support -no-xmlpatterns -no-multimedia -no-audio-backend -no-phonon -no-phonon-backend -no-svg -no-webkit -no-javascript-jit -no-script -no-scripttools -no-declarative -no-openssl -arch "x86" -sdk "/Developer/SDKs/MacOSX10.5.sdk" -no-exceptions

I got about 30MB for Frameworks. But Framework folders can be downsized — no need for debug versions of libs, no need for header files because frameworks are private. Because I used copying of Frameworks using QMAKE_POST_LINK, it looks like this:

QMAKE_POST_LINK = \
mkdir -p $${BUNDLETARGET}/Contents/Frameworks; \
cp -R $${QTFRAMEWORKSPATH}/QtCore.framework \
$${QTFRAMEWORKSPATH}/QtGui.framework \
$${QTFRAMEWORKSPATH}/QtXml.framework \
$${BUNDLETARGET}/Contents/Frameworks; \
...
\
strip $(TARGET); \
strip -x $${BUNDLETARGET}/Contents/Frameworks/QtCore.framework/Versions/4/QtCore; \
strip -x $${BUNDLETARGET}/Contents/Frameworks/QtGui.framework/Versions/4/QtGui; \
strip -x $${BUNDLETARGET}/Contents/Frameworks/QtXml.framework/Versions/4/QtXml; \
\
rm -rf `find $${BUNDLETARGET} -name "*.prl"`; \
rm -rf `find $${BUNDLETARGET} -name "*.lproj"`; \
rm -rf `find $${BUNDLETARGET} -name "*_debug*"`; \
rm -rf `find $${BUNDLETARGET} -name "Headers"`; \
\
otool -L $(TARGET); \
otool -L $${BUNDLETARGET}/Contents/Frameworks/QtCore.framework/Versions/4/QtCore; \
otool -L $${BUNDLETARGET}/Contents/Frameworks/QtGui.framework/Versions/4/QtGui; \
otool -L $${BUNDLETARGET}/Contents/Frameworks/QtXml.framework/Versions/4/QtXml; \
\
echo "Ok"

Result — 12MB, in Mac App Store – just 5.3MB. This is great just for small application which is based on non-native GUI frameworks.

So we resolved size issue, now it is time to get Qt ready to pass examination and be ready for store.

First — app should keep own data in ~/Library/Application Support/com.organization.appname, so we download patch mac-app-store-cache-location.diff. After applying it, when using QDesktopServices::storageLocation() we get right path to location where to keep app/user data.

Second — Qt apps always read and write ~/Library/Preferences/com.nokia.qt.plist. You can not disable it due to need of common settings of Qt Applications. But Apple has restrictions for applications not to write settings of other applications. So we download second patch for Qt — mac-settings-in-app-area.diff, and in main() in very beginning add a call: qt_force_trolltech_settings_to_app_area(true);

Also there are two patches — first one is for removal Hide/Show All from application menu, but I found this patch later, though my app passed control without it, and second one is for SQLite — but I did not study this one (no need for my app).

Frameworks should be adjusted to have them referenced to each other using @executable_path. You can see it easily by otool utility:

$ otool -L QtGui.framework/QtGui
QtGui.framework/QtGui:
/usr/local/Trolltech/Qt-4.7.3/lib/QtGui.framework/Versions/4/QtGui (compatibility version 4.7.0, current version 4.7.3)
/usr/local/Trolltech/Qt-4.7.3/lib/QtCore.framework/Versions/4/QtCore (compatibility version 4.7.0, current version 4.7.3)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 136.0.0)
...

Add this to QMAKE_POST_LINK:

install_name_tool -id @executable_path/../Frameworks/QtCore.framework/Versions/4/QtCore \
$${BUNDLETARGET}/Contents/Frameworks/QtCore.framework/Versions/4/QtCore; \
install_name_tool -id @executable_path/../Frameworks/QtGui.framework/Versions/4/QtGui \
$${BUNDLETARGET}/Contents/Frameworks/QtGui.framework/Versions/4/QtGui; \
install_name_tool -id @executable_path/../Frameworks/QtXml.framework/Versions/4/QtXml \
$${BUNDLETARGET}/Contents/Frameworks/QtXml.framework/Versions/4/QtXml; \
...
+and same for other references

Now everything is fine:

otool -L QtGui.framework/QtGui
QtGui.framework/QtGui:
@executable_path/../Frameworks/QtGui.framework/Versions/4/QtGui (compatibility version 4.7.0, current version 4.7.3)
@executable_path/../Frameworks/QtCore.framework/Versions/4/QtCore (compatibility version 4.7.0, current version 4.7.3)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 136.0.0)
...

Now just by copying .app bundle onto another Mac without developer tools and Qt your App should start without problems.

Also you need to sign your app by your signature(certificate) as Mac developer. You have to get such certificate from developer.apple.com, pay $99 for year subscription if you haven’t done it yet.

/usr/bin/codesign -f -s 3rd\ Party\ Mac\ Developer\ Application:\ Firstname\ Lastname $${BUNDLETARGET}; \

and add this to “pro” file too.

But for submitting the application into App Store, I had to start Xcode, build it in dsym+dwarf mode, make it signed, added prepared Frameworks by «New Build Phase -> New Copy Files Build Phase». Everything was tested with otool to see all dependencies, checked on another Mac and got ready to send into App Store (If you know how to do all such stuff by scripts please let me know – prefer to do everything in Qt Creator).

Summary —
1. It is not so complicated as for first sight.
2. Apple is okay to accept Qt apps, just need to apply some patches to Qt.
3. Size of Qt App (bundle) is small enough (maybe if you include several resources into app bundle then size of Qt frameworks may become a minor part).

Result — http://itunes.apple.com/us/app/togmeg/id445287955

References:
Main source of information — http://bugreports.qt.nokia.com/browse/QTBUG-16549

Current API is very simple.
First you need to include <QtSpeech>. Then if your application just needed to say something synchronously (execution will wait in the point until speech is finished) you can use such code:

#include <QtSpeech>
...
QtSpeech voice;
voice.say("Hello World!");

If you would like to do this in asynchronous way (so your application is not blocked meanwhile) just use tell() call:

QtSpeech * voice = new QtSpeech(this);
voice->tell("Hello asynchronous world!");

If you need to invoke a slot at the end, use:

voice->tell("Hello!", this, SLOT(onSpeechFinished()));

Also you have possibility to get list of voices and choose voice from available in your system:

QtSpeech::VoiceNames vs = QtSpeech::names()

Current implementation support Windows using SAPI and Mac. I have plans to extend API to include more functionality but will try to choose what is common on all platforms.

Link to repository in Gitorius: http://gitorious.org/qt-speech

UPDATE: Unix/Linux got implementation too! Based on Festival, so only English at the moment, but it is a great start to have TTS working in Qt applications too in Linux systems!

I got a lot of questions about this ideas, but one of frequent questions was about doing same with PySide. Well, PySide is LGPL so this makes the framework applicable for much more projects. So, let’s have a try of doing same but with PySide now.

First, I tried binary packages, but looks like they do not have generatorrunner executable included and other stuff for making bindings for Qt-like libraries – let’s wish for maintainers to include them. So, we need to build it manually. Needed CMake, installed Qt Frameworks, Python.

So I followed instructions listed on PySide web site and got it after hour-two. Let’s better show step-by-step for sure (I have done it on Mac OS 10.6, Qt 4.7 binaries from Nokia, Python 2.6):

# git clone git://gitorious.org/pyside/apiextractor.git
# cd apiextractor
# mkdir build && cd build
# cmake .. && make && make install && cd ../..
#
# git clone git://gitorious.org/pyside/generatorrunner.git
# cd generatorrunner
# mkdir build && cd build
# cmake .. && make && make install && cd ../..
#
# git clone git://gitorious.org/pyside/shiboken.git
# cd shiboken
# mkdir build && cd build
# cmake .. && make && make install && cd ../..
#
# git clone git://gitorious.org/pyside/pyside.git
# cd pyside
# mkdir build && cd build
# cmake .. && make && make install && cd ../..

Now we have additionaly generatorrunner and typesystems needed for making wrappings to c++ mudules.

Let’s return to our hybrid application – create next folder structure:

HybridApp/
 |-data/
 |    |-global.h
 |    |-typesystem.xml
 |-hybrid/
 |    |-MainWindow.h
 |    |-MainWindow.cpp
 |    |-hybrid.pro
 |-hybridpy/
 |    |-hybridpy.pro
 |-build.sh
 |-Main.py
 

Let’s explain:

• Folder hybrid contains c++ part of application which is built into shared lib.
• Folder hybridpy contains wrapping for c++ part – it builds Python module which will be imported into python part of application.
• Folder data contains definition of typesystem used in c++ part. This is Xml file which describes types of used objects and their especialities when converting into python objects. More details about typesystems – on pyside page with following links.

Now we go into hybrid folder and create c++ part of hybrid. First let’s do it as c++ only application with own main() methon.

hybrid/hybrid.pro

TEMPLATE = app
CONFIG += qt
QT += core gui

UI_DIR = build
RCC_DIR = build
MOC_DIR = build
OBJECTS_DIR = build

HEADERS += MainWindow.h
SOURCES += MainWindow.cpp Main.cpp

hybrid/Main.cpp:

#include <QtGui>
#include "MainWindow.h"
int main(int argc, char ** argv) {
    QApplication app(argc, argv);
    MainWindow window;
    window.resize(1000,700);
    window.show();
    return app.exec();
}

hybrid/MainWindow.h:

#ifndef MainWindow_H
#define MainWindow_H

#include <QMainWindow>
class QPushButton;
class QGraphicsView;
class QGraphicsScene;
class QPlainTextEdit;

class MainWindow : public QMainWindow { Q_OBJECT
public:
    MainWindow(QWidget * parent = 0L);
    virtual ~MainWindow();

signals:
    void runPythonCode(QString);

private slots:
    void runPythonCode();

public:
    QGraphicsView * viewer;
    QGraphicsScene * scene;
    QPlainTextEdit * editor;
    QPushButton * pb_commit;
};
#endif // MainWindow_H

hybrid/MainWindow.cpp:

#include <QtGui>
#include "MainWindow.h"

MainWindow::MainWindow(QWidget * parent):QMainWindow(parent) {
    QSplitter * splitter = new QSplitter;
    setCentralWidget(splitter);

    QWidget * editorContent = new QWidget;
    splitter->addWidget(editorContent);

    QVBoxLayout * layout = new QVBoxLayout;
    editorContent->setLayout(layout);

    editor = new QPlainTextEdit;
    layout->addWidget(editor);

    pb_commit = new QPushButton(tr("Commit"));
    connect(pb_commit, SIGNAL(clicked()),
            this, SLOT(runPythonCode()));
    layout->addWidget(pb_commit);

    scene = new QGraphicsScene(this);
    viewer = new QGraphicsView;
    viewer->setScene(scene);
    splitter->addWidget(viewer);

    splitter->setSizes(QList<int>() << 400 << 600);
}

MainWindow::~MainWindow() {;}

void MainWindow::runPythonCode() {
    emit runPythonCode(editor->toPlainText());
}

As result we have a window with editor (left) and canvas (right):

Now it is time to to turn it into PySide-based application. We are going to:

• Change c++ part to dynamic library
• Create wrappings for MainWindow class
• Make Main.py instead of C main() routine

For first point we just change TEMPLATE to lib, exclude Main.cpp, and build the shared lib. Also set TARGET to have shared libraries in root folder of HybridApp

hybrid/hybrid.pro:

TEMPLATE = lib
TARGET = ../Hybrid
CONFIG += qt
QT += core gui

UI_DIR = build
RCC_DIR = build
MOC_DIR = build
OBJECTS_DIR = build

HEADERS += MainWindow.h
SOURCES += MainWindow.cpp

As result you should get libHybrid.dylib in root of our project (HybrydApp)

Now it is time to generate wrappings for c++ part to use in python. The generation is made by build.sh script in root of project. (Actually it builds everything – both c++ and wrapping parts)

build.sh

#!/bin/sh

cd hybrid
qmake
make
cd ..

cd hybridpy

QTGUI_INC=/Library/Frameworks/QtGui.framework/Versions/4/Headers
QTCORE_INC=/Library/Frameworks/QtCore.framework/Versions/4/Headers
QTTYPESYSTEM=/usr/local/share/PySide/typesystems

generatorrunner --generatorSet=shiboken \
    ../data/global.h \
    --include-paths=../hybrid:$QTCORE_INC:$QTGUI_INC:/usr/include \
    --typesystem-paths=../data:$QTTYPESYSTEM \
    --output-directory=. \
    ../data/typesystem.xml

qmake
make
cd ..

rm -rf PyHybrid.so
ln -s libPyHybrid.dylib PyHybrid.so

You see that this is just one call of generatorrunner plus you provide paths for Qt includes, qt typesystem and your typesystem. Then in folder hybridpy we with qmake build the python module.

Listing of our typesystem.xml and global.h:

data/typesystem.xml

<?xml version="1.0"?>
<typesystem package="PyHybrid">
    <load-typesystem name="typesystem_core.xml" generate="no"/>
    <load-typesystem name="typesystem_gui.xml" generate="no"/>
    <object-type name="MainWindow"/>
</typesystem>

data/global.h (fyi: more details about global.h – here)

#undef QT_NO_STL
#undef QT_NO_STL_WCHAR

#ifndef NULL
#define NULL    0
#endif

#include <MainWindow.h>

And pro file for building python module:

hybridpy/hybridpy.pro

TEMPLATE = lib
QT += core gui

INCLUDEPATH += hybrid
INCLUDEPATH += ../hybrid

INCLUDEPATH += /usr/include/python2.6
INCLUDEPATH += /usr/local/include/shiboken
INCLUDEPATH += /usr/local/include/PySide
INCLUDEPATH += /usr/local/include/PySide/QtCore
INCLUDEPATH += /usr/local/include/PySide/QtGui

LIBS += -ldl -lpython2.6
LIBS += -lpyside
LIBS += -lshiboken
LIBS += -L.. -lHybrid

TARGET = ../PyHybrid

SOURCES += \
    pyhybrid/pyhybrid_module_wrapper.cpp \
    pyhybrid/mainwindow_wrapper.cpp \

You can mention that most of used paths of python headers and qt headers can be extracted from utils like pkg-config etc – that’s true, but I showed exact paths intentionally just for demoing what is really included. Also, of course you can do it all with cmake, but I made a choice for qmake pro files to have better (plain) presentation of used files and logic.

Looks complete – if you just run build.sh you will get PyHybrid.so – python module.
Everything is ready, time to connect both parts in Main.py script:

Main.py

import sys
from PySide.QtCore import *
from PySide.QtGui import *
from PyHybrid import *

class RunScript(QObject):
    def __init__(self, mainWindow):
        QObject.__init__(self)
        self.mainWindow = mainWindow

    def runScript(self, script):
        mainWindow = self.mainWindow
        exec(str(script))

a = QApplication(sys.argv)
w = MainWindow()
r = RunScript(w)
w.setWindowTitle('PyHybrid')
w.resize(1000,800)
w.show()
a.connect(w, SIGNAL('runPythonCode(QString)'), r.runScript)
a.connect(a, SIGNAL('lastWindowClosed()'), a, SLOT('quit()') )
a.exec_()

If you run it you will get same window as in c++ application except that python code entered in left panel with editor can be executed just here and the python code has control of our c++ part.

Let’s try small code snippets like:

mainWindow.statusBar().show()

Oh! status bar is appeared in our application. Well, more complex maybe – let’s change background of canvas:

mainWindow.scene.setBackgroundBrush(QColor('#e0e0ff'))

Can we create new objects? Of course yes:

li1 = QGraphicsLineItem(10,10, 500,500)
li1.setPen(QPen(QBrush(QColor("#ff0000")), 3.0, Qt.DashLine))
mainWindow.scene.addItem(li1)

Can we create new classes just in real time and instanciate them???
Sure yes!!!. Let’s create subclass of QGraphicsItem and make custom painting:

mainWindow.viewer.setRenderHint(QPainter.Antialiasing)
class MyItem(QGraphicsItem):
	def boundingRect(self):
		return QRectF(-100,-100,200,200)

	def paint(self, painter, option, widget):
		g = QLinearGradient(-100,-100, 100,100)
		g.setColorAt(0, QColor('#00ff00'))
		g.setColorAt(1, QColor('#ffffff'))
		painter.setBrush(g)
		p = QPen(QBrush(QColor("#ff0000")), 4, Qt.DashLine)
		painter.setPen(p)
		painter.drawRoundedRect(-100,-100,200,200, 30,30)

my1 = MyItem()
mainWindow.scene.addItem(my1)
my1.setPos(200,200)

Well, so we can do everything with our small application – we can program it in real time. We can even generate code by program itself to execute it and more. Anyway, this also can be used for performance balancing when you can quickly prototype your code in python, and then transfer such code into C++ to make it running fast.

Good example – developing canvas-based games, when it can be really helpful and convenient to work with python code to make the logic, drawing etc and then migrate some code into C++.

Also it is very good for mobile devices – you program with python-pyside until you encounter performance issues and in this point you can easily separate critical part into c++ and keep Qt-like interface between these two parts.

Regarding our small application above – you can ask What is the difference to just programming with your editor? – You are coding your application while it is running. This is very new approach to programming with Qt. There is one especiality – you can loose your programming when exit , So, it makes sense to modify the application to save all your code snippets to files.

It’s up to you.

I would like to thanks to PySide team. They are doing really great work and potential of PySide looks very exciting to be used in many projects.

It is often situation when all application data are some kind of hierarchical tree of objects and properties. So, all of them can be kept in just one tree-like model class, but I see that developers do not use the approach and it happens because they do not see how some selective piece of data can be shown or have expectations of implementing own view classes, which is not trivial though and you may spent too much efforts there. Instead a lot of code related to synchronization between models can appear and it leads to complication of logic.

Same time, it can be resolved by implementation proxy models and use just standard view classes from Qt which are very effective. And I would like to show here that it is really simple:

There is simple example that I had to implement recently. The application shows list of objects, and same time it should show list of all properties of all objects. My first impression was to implement two separate models that may have shared data or base data are kept in one of model + some synchronization. But I saw that data manipulations leads to code complication + undo should be implemented smoothly.

We can implement just one model which is tree of objects (two levels). In one view (we can use QListView to show only first level of tree) we show listed objects. But what about another view to list of all objects of second level – we can implement proxy model which will be model of view

Let’s have a look at QAbstractProxyModel. For mapping we need the implementation of mapFromSource() and mapToSource() plus rowCount / columnCount() / parent() and index().
Because destination model is table-like, parent() returns just QModelIndex(), columnCount() returns count of source model, and index() returns just createIndex(row, column).
More complex are rowCount(), mapFromSource() and mapToSource() because they need to calculate. (see a listings below)

Another piece of logic – appropriate behavior on adding /removing rows and data changes. QAbstractItemModel has very convenient signals reporting process of adding rows or columns. With recalculating indexes we can just call same methods of proxy models, like:

...
connect(source, SIGNAL(rowsAboutToBeInserted(QModelIndex, int, int)),
        this, SLOT(s_rowsAboutToBeInserted(QModelIndex, int, int)));
connect(source, SIGNAL(rowsInserted(QModelIndex, int, int)),
        this, SLOT(s_rowsInserted(QModelIndex, int, int)));
...

void Proxy::s_rowsAboutToBeInserted(QModelIndex p, int from, int to)
{
    if (!p.isValid())
        return;

    int f = ... // calculate
    int t = ... // calculate
    beginInsertRows(QModelIndex(), f, t);
}

void Proxy::s_rowsInserted(QModelIndex p, int, int)
{
    // post adding routines if needed
    endInsertRows();
}

So, to summarize: it is not much efforts to implement proxy model which do appropriate data mappings in both directions. As benefit you have data kept in one place, all necessary signals by model to report changes in data (model). By mapping / proxying / filtering you can visualize in view any piece of data, even from different levels of hierarchy, etc. To visualize you can use just standard Qt view which are very effective.

Full listings of my proxy model:

UngroupProxyModel.h
UngroupProxyModel.cpp

Update:
Read also Jongling Qt models 2, Composition Gem

So, what can be done there? Well, good to have it planned from beginning, but usually it does not happen, especially if several people worked on the project, some of them left project, some of them working only on command-line interface, etc.

It is possible that you need to implement undo in project which grown up to 100K lines of code and of course the primary question is how to do it smooth and transparent without affecting much code. I had similar experience and would like to share some tricks about it.

Let’s consider some project which compound of multiple components and one of them is model which used by several views and even used through filter/sorting proxy models. Usually developers quickly goes to cover everything than users can do in views, so they subclass views and start working hard on covering all actions there. Well, even if such idea works, this is not good, considering complexity of your project, and especially if there are multiple components and they are using each other, so they should now about undo actions of every other component in project. Are you using different views of same models? – well, you have to implement same undo few times in different view, oh wait, there is also filter/sorting model in middle – you have to code more, etc

You can get feeling that with implementing undo you started some avalanche in code of new dependencies, new apis, new methods, hundred of undo commands etc… Even more – you have to teach other developers how to apply undo or document it all very well.

Let’s try another approach – we can try to cover one single component without affecting API, and make it totally transparent for other classes in project.

Imaging that we have some model:

class Model : public QAbstractItemModel { Q_OBJECT
public:
    Model(QObject * parent);
    virtual ~Model()

    // model reimpl:
    int rowCount(const QModelIndex &parent = QModelIndex()) const;
    int columnCount(const QModelIndex &parent = QModelIndex()) const;
    QVariant data(const QModelIndex &index,
                  int role = Qt::DisplayRole) const;
    virtual bool setData(const QModelIndex& index,
                         const QVariant & value,
                         int role = Qt::EditRole);

    // you api
    virtual void add(Item *);
    virtual void remove(Item *);
};

There are 3 points where your model can be changed: add(), remove() and setData(). It is ideal to apply undo only in these 3 points and make it automatic. What means by “automatic”? – our code should know when apply undo and when not.

Remember that our project may also be used in command line mode. It makes sense not to mix model logic code with undo code, but we are not going to implement it in view, so let’s subclass our model and implement all Gui stuff and Undo there:

class ModelGui : public Model { Q_OBJECT
public:
    ModelGui(QObject * parent);
    virtual ~ModelGui()

    QUndoStack * undoStack() const;
    void setUndoStack(QUndoStack *);

    // reimplemented for undo:
    virtual bool setData(const QModelIndex& index,
                         const QVariant & value,
                         int role = Qt::EditRole);

    virtual void add(Item *);
    virtual void remove(Item *);

private:
    class AddCmd;
    class RemoveCmd;
    class ChangeCmd;
};

We subclassed Model to reimplement add(), remove(), setData(), also we declared undo commands in “private” section (but we will define them in cpp to make them hidden). Now we need to implement add() where we cover it by undo command. Same time undo AddCmd should access “logic” add() method of superclass – we don’t need a recursion, so some kind of flag is needed, also it makes sense to make the flag kind of “global” to have control of covering actions by undo and in case of macro commands etc. Good trick to use “property” of qobject, also good to use as the qobject – the undo stack itself.

Let’s show implementation to make it clear:

...

class ModelGui::AddCmd : public QUndoCommand {
public:
    AddCmd(ModelGui * m, Item * newItem): model(m), item(newItem) {}

    void redo() {
        bool cover = model->undoStack()->property('cover').toBool();
        model->undoStack()->setProperty('cover', false);
        model->Model::add(item);
        model->undoStack()->setProperty('cover', cover);
    }

    void undo() {
        bool cover = model->undoStack()->property('cover').toBool();
        model->undoStack()->setProperty('cover', false);
        model->Model::remove(item);
        model->undoStack()->setProperty('cover', cover);
    }

private:
    ModelGui * model;
    Item * item;
};

void ModelGui::add(Item * newItem) {
    bool cover = undoStack()->property('cover').toBool();
    if (cover) {
        undoStack()->beginMacro("Add Item");
        undoStack()->push(new AddCmd(this, newItem));
        undoStack()->endMacro();
    }
    else Model::add(newItem);
}

...

What we have accomplished here?

• There is single place where undo applied
• Implementation of undo commands are hidden
• Other classes or components don’t even need to know about undo in the model
• You can use multiple views and sorting/filtering proxy models without worry about undo
• If there is also command-line use of project, you just exclude ModelGui.h/cpp
• If other component/module cast for add(), it is covered by undo automatically
• If other undo command cast for add() from inside, it is not covered by undo – no recursion
• The project can be as complex as possible, you only need to worry about using “cover”

Currently you can find PyQt even for Maemo – link

Anyway, PyQt is great, python programming is fast, but one of frequently asked questions is –
How can I integrate python scripting in my application and operate Qt Api from my scripts?, and why not?

When you can think that it is not possible, I would like to show the way how to do this and let’s even make working example.

Let’s start from scratch, assuming that you have installed and ready Qt Frameworks. (4.6.3 is used). First download such packages from PyQt homepage:

• SIP
• PyQt4 (I used Mac platform for the example)

Build sip:

# tar -zxf sip-4.10.3.tar.gz
# cd sip-4.10.3
# python configure.py
# make
# sudo make install

Build PyQt:

# tar -zxf PyQt-mac-gpl-4.7.4.tar.gz
# cd PyQt-mac-gpl-4.7.4
# python configure.py
# make
# sudo make install

(BTW: you may need to set the environment variable if configure does not work)

# export QMAKESPEC=macx-g++

Now, it is time to design small Qt application which we will use as base for executing PyQt routines.
Let’s choose Canvas-based application – in such application we can prototype canvas items with PyQt
during runtime of the application. Make it simple: two areas: plain text editor for python codes on left and canvas on right.

PyQtHybrid.pro:

TEMPLATE = app
CONFIG += qt
QT += core gui
HEADERS += MainWindow.h
SOURCES += MainWindow.cpp Main.cpp

Main.cpp:

#include <QtGui>
#include "MainWindow.h"
int main(int argc, char ** argv) {
    QApplication app(argc, argv);
    MainWindow window;
    window.resize(1000,700);
    window.show();
    return app.exec();
}

MainWindow.h:

#ifndef MainWindow_H
#define MainWindow_H

#include <QMainWindow>
class QPushButton;
class QGraphicsView;
class QGraphicsScene;
class QPlainTextEdit;

class MainWindow : public QMainWindow { Q_OBJECT
public:
    MainWindow(QWidget * parent = 0L);
    virtual ~MainWindow();

signals:
    void runPythonCode(QString);

private slots:
    void runPythonCode();

public:
    QGraphicsView * viewer;
    QGraphicsScene * scene;
    QPlainTextEdit * editor;
    QPushButton * pb_commit;
};
#endif // MainWindow_H

MainWindow.cpp:

#include <QtGui>
#include "MainWindow.h"

MainWindow::MainWindow(QWidget * parent):QMainWindow(parent) {
    QSplitter * splitter = new QSplitter;
    setCentralWidget(splitter);

    QWidget * editorContent = new QWidget;
    splitter->addWidget(editorContent);

    QVBoxLayout * layout = new QVBoxLayout;
    editorContent->setLayout(layout);

    editor = new QPlainTextEdit;
    layout->addWidget(editor);

    pb_commit = new QPushButton(tr("Commit"));
    connect(pb_commit, SIGNAL(clicked()),
            this, SLOT(runPythonCode()));
    layout->addWidget(pb_commit);

    scene = new QGraphicsScene(this);
    viewer = new QGraphicsView;
    viewer->setScene(scene);
    splitter->addWidget(viewer);

    splitter->setSizes(QList<int>() << 400 << 600);
}

MainWindow::~MainWindow() {;}

void MainWindow::runPythonCode() {
    emit runPythonCode(editor->toPlainText());
}

As result we have a window with editor (left) and canvas (right):

Now it is time to modify application to get it working with PyQt. We are going to:

• Change application to dynamic library
• Create wrappings for MainWindow class
• Make Main.py instead of C main() routine

For first point we just change TEMPLATE to lib, exclude Main.cpp, and build the shared lib.
PyQtHybrid.pro:

TEMPLATE = lib
CONFIG += qt
QT += core gui
HEADERS += MainWindow.h
SOURCES += MainWindow.cpp

Now create folder “sip”. Inside we create a wrappings for our MainWindow class:
MainWindow.sip:

%Module PyQtHybrid 0
%Import QtGui/QtGuimod.sip
%If (Qt_4_2_0 -)

class MainWindow : QMainWindow {
%TypeHeaderCode
#include "../MainWindow.h"
%End
public:
    MainWindow();
    virtual ~MainWindow();

signals:
    void runPythonCode(QString);

private slots:
    void runPythonCode();

public:
    QGraphicsView * viewer;
    QGraphicsScene * scene;
    QPlainTextEdit * editor;
    QPushButton * pb_commit;
};

%End

To generate wrappings from sip we need a configure.py and PyQtHybridConfig.py.in.
configure.py:

import os
import sipconfig
from PyQt4 import pyqtconfig

build_file = "PyQtHybrid.sbf"
config = pyqtconfig.Configuration()
pyqt_sip_flags = config.pyqt_sip_flags

os.system(" ".join([ \
    config.sip_bin, \
    "-c", ".", \
    "-b", build_file, \
    "-I", config.pyqt_sip_dir, \
    pyqt_sip_flags, \
    "MainWindow.sip" \
]))

installs = []
installs.append(["MainWindow.sip", os.path.join(config.default_sip_dir, "PyQtHybrid")])
installs.append(["PyQtHybridConfig.py", config.default_mod_dir])

makefile = pyqtconfig.QtGuiModuleMakefile(
    configuration=config,
    build_file=build_file,
    installs=installs
)

makefile.extra_libs = ["PyQtHybrid"]
makefile.extra_lib_dirs = [".."]

makefile.generate()

content = {
    "PyQtHybrid_sip_dir":    config.default_sip_dir,
    "PyQtHybrid_sip_flags":  pyqt_sip_flags
}
sipconfig.create_config_module("PyQtHybridConfig.py", "PyQtHybridConfig.py.in", content)

PyQtHybridConfig.py.in

from PyQt4 import pyqtconfig
# @SIP_CONFIGURATION@

class Configuration(pyqtconfig.Configuration):
    def __init__(self, sub_cfg=None):
        if sub_cfg: cfg = sub_cfg
        else: cfg = []
        cfg.append(_pkg_config)
        pyqtconfig.Configuration.__init__(self, cfg)

class PyQtHybridModuleMakefile(pyqtconfig.QtGuiModuleMakefile):
    def finalise(self):
        self.extra_libs.append("PyQtHybrid")
        pyqtconfig.QtGuiModuleMakefile.finalise(self)

Time to build Python module in “sip” folder (we assume that C++ lib is already built in top folder)

# python configure.py
# make

Almost everything is ready, time to connect everything in Main.py script:

import sys
sys.path.append('sip')

from PyQt4.QtCore import *
from PyQt4.QtGui import *
from PyQtHybrid import *

class RunScript(QObject):
    def __init__(self, mainWindow):
        QObject.__init__(self)
        self.mainWindow = mainWindow

    def runScript(self, script):
        mainWindow = self.mainWindow
        exec(str(script))

a = QApplication(sys.argv)
w = MainWindow()
r = RunScript(w)
w.setWindowTitle('PyQtHybrid')
w.resize(1000,800)
w.show()
a.connect(w, SIGNAL('runPythonCode(QString)'), r.runScript)
a.connect(a, SIGNAL('lastWindowClosed()'), a, SLOT('quit()') )
a.exec_()

Now, if you run the Main.py you will get same window as we saw before, but, python code entered in left editor panel can be committed for execution by … well application itself…

We have an application which can program itself – that’s cool.

Let’s try small code snippets like:

mainWindow.statusBar().show()

Oh! status bar is appeared in our application. Well, more complex maybe – let’s change background of canvas:

mainWindow.scene.setBackgroundBrush(QColor('#e0e0ff'))

Can we create new objects? Of course yes:

li1 = QGraphicsLineItem(10,10, 500,500)
li1.setPen(QPen(QBrush(QColor("#ff0000")), 3.0, Qt.DashLine))
mainWindow.scene.addItem(li1)

Can we create new classes just in real time and instanciate them???
Sure yes!!!. Let’s create subclass of QGraphicsItem and make custom painting:

mainWindow.scene.clear()
mainWindow.viewer.setRenderHint(QPainter.Antialiasing)
class MyItem(QGraphicsItem):
	def boundingRect(self):
		return QRectF(-100,-100,200,200)

	def paint(self, painter, option, widget):
		g = QLinearGradient(-100,-100, 100,100)
		g.setColorAt(0, QColor('#00ff00'))
		g.setColorAt(1, QColor('#ffffff'))
		painter.setBrush(g)
		p = QPen(QBrush(QColor("#ff0000")), 4, Qt.DashLine)
		painter.setPen(p)
		painter.drawRoundedRect(-100,-100,200,200, 30,30)

my1 = MyItem()
mainWindow.scene.addItem(my1)
my1.setPos(200,200)

Well, so we can do everything with our small application – we can program it in real time. We can even generate code by program itself to execute it and more. Anyway, this also can be used for performance balancing when you can quickly prototype your code in python, and then transfer such code into C++ to make it running fast.

Good example – developing canvas-based games, when it can be really helpful and convenient to work with python code to make the logic, drawing etc and then migrate some code into C++.

You can ask what is the difference to just programming with your editor? – You are coding your application while it is running. This is very new approach to programming with Qt. There is one especiality – you can loose your programming when exit , So, it makes sense to modify the application to save all your code snippets to files.

It’s up to you.

Let’s hope to see LGPL PyQt packages sometimes…

QString buildId()
{
    QString embeddedDate(__DATE__);
    QString m = embeddedDate.left(3);
    QStringList months;
    months << "Jan" << "Feb" << "Mar" << "Apr" << "May" << "Jun"
           << "Jul" << "Aug" << "Sep" << "Oct" << "Nov" << "Dec";

    int month = months.indexOf(m) + 1;
    int day   = embeddedDate.mid(4, 2).trimmed().toInt();
    int year  = embeddedDate.mid(7, 4).trimmed().toInt();

    return QString::number(year*10000+month*100+day);
}

#ifdef Q_WS_MAC
    #include <Carbon/Carbon.h>
    //need a method from qglobal.cpp
    extern QString qt_mac_from_pascal_string(const Str255);
#endif

...

QFont systemMiniFont()
{
#ifdef Q_WS_MAC
    Str255 f_name;
    SInt16 f_size;
    Style f_style;
    static const ScriptCode Script = smRoman;

    GetThemeFont(kThemeSmallSystemFont, Script,
                 f_name, &f_size, &f_style);

    QFont font(qt_mac_from_pascal_string(f_name), f_size,
               (f_style & ::bold) ? QFont::Bold : QFont::Normal,
               (bool)(f_style & ::italic));

    return font;
#endif
    return qApp->font();
}