How to separate serialization, visualization and computation from GUI editable code objects?

Question

This might be an X Y problem, but here's my situation.

I've got a QT5 C++ code base whose task it is to allow for the configuration of a "process chain". A "process chain" here means a list of linked elements who execute in sequential order, taking a single input and returning a single output to the next "process element" in the chain. At the end of the chain, this data is saved.

Each process element is configurable, and while each works on the same data and outputs the same datatype, each does something completely different underneath. They take vastly different configuration parameters, but share the same interface. Each has a different constructor, one may take a single float, another a matrix, etc... in their constructor.

There are dozens of process element types which grows with every iteration of the software. Each process element type needs to be serialized (currently through JSON), and needs its configuration to be configured through a gui, to which each parameter to configure may not have the same gui widget type (ie one may need only a line edit, another may need radio buttons, another a spin box, but all configure the same process element).

My problem is that I currently accomplish this in a very coupled manner. The process element class needs to know and define its own JSON format, how to construct itself from json, and it essentially needs to add itself to a global static unordered map of classname -> constructors, and classname -> json formats.

I do not currently have a way to add in the ability to specify gui type, that is what led me to believe that the current method is unsustainable.

My issues with my current approach are as follows:

• I couple the way the serialization works with the class itself.
• I couple the display implicitly (currently all gui types are the same, but now the requirements have changed and require some gui types to be different).

I want to de couple this functionality, but I'm not happy with other possible solutions. Creating a "serialization class" for each process element seems like code smell, and this would have to be duplicated for the visualization. I would also like to avoid doing work that would have to be replicated for each class. Currently I have a macro set up automating the static adding of classname mappings to the global class maps.

Here is an moc up of what the gui looks like for clarification:

What strategies can I employ to solve the design coupling issue I'm having? Ultimately, is there any way in which I can effectively separate serialization and visualization from the "code objects" (process elements) in my circumstance?

Answer 1

As I understand your problem, the interface of each process element is a set of readable and writable properties. A list of (name, value) pairs is all that is needed for both GUI and serialization. Qt has provides a framework to do this - you can use Q_PROPERTY and QVariant-based setters and getters. Personally, I'd start by creating a base class for a process step which provides information about which properties are needed for display/serialization:

class BaseProcessElement : public QObject
{
    Q_OBJECT

public:
    // Returns type name used for serialization.
    virtual QString typeName() const = 0;

    // Returns a list of properties to be displayed/serialized.
    virtual QStringList elementProperties() const = 0;
};

class SomeProcessElement : public BaseProcessElement
{
    Q_OBJECT

public:
    Q_PROPERTY(int some_prop_1 READ someProp1 WRITE setSomeProp1 NOTIFY someProp1Changed);
    Q_PROPERTY(QString some_prop_2 WRITE someProp2 WRITE setSomeProp2 NOTIFY someProp2Changed);

    int someProp1() const;
    QString someProp2() const;

    QString typeName() const override;

    QStringList elementProperties() const override
    {
        return {"some_prop_1", "some_prop_2"};
    }

public slots:
    void setSomeProp1(int value);
    void setSomeProp2(const QString& value);

signals:
    void someProp1Changed(int newValue);
    void someProp2Changed(const QString& newValue);
};

Now, you can query relevant properties for GUI and serialization without any additional support in element class:

void createUI(BaseProcessElement* element)
{
    for(auto propName : element->elementProperties())
    {
        QVariant propValue = element->property(propName.toStdString().c_str());

        if(propValue.canConvert<int>())
            addSpinBox(propName, propValue.toInt());
        else if(propValue.canConvert<QString>())
            addLineEdit(propName, propValue.toString());
        else....
    }
}

(Obviously, if there are many more possible property types, a factory would be better - this is just to illustrate the idea)

Properties can also be set during deserialization/editing by calling element->setProperty(propName, propValue);.

If a property has more properties than just a value (e. g. bounds for numbers), you could define a type that groups a value with other data:

class IntElementProperty : public ElementProperty
{
public:
    int value() const;
    int maxValue() const;
    int minValue() const;
    ...
 };

And return instances of those classes through Qt's property accessors.

Answer 2

One approach you could take is to have a separate specification language. The specification language would specify each parameter, by detailed type and by presentation type. For instance, you might have a field that ranges between 0.1 and 1.0 by 0.1 increments, and could best be input as a slider. Other parts of the validation could be specified as well.

Taking the specification, you could then generate the serializer/deserializer code for the element, bypassing much of the boiler plate. Similarly, your GUI could read the specification directly and use the GUI description to represent the element parameter visually.

As a for instance:

element Baz:
  field foo: float, min(0.1), max(1.0), increment(0.1), slider;
  field bar: string, maxlen(10), minlen(1), match("[a-z]+");
end element

Creating miniature languages like this is done all the time. You could write the language in Lua, Python or Lisp so that you don't need to write an interpreter.

Answer 3

The way I understand your issue generally is that you ought to embrace the code model,

Program everything as an API

and more specifically and to the point, need to develop a general purpose library for JSON serialization, which consists of two things:

Build a JSON Container/API

which holds all JSON elements in C++ code. Ergo, you must be able to hold something like this:

in a Qt oriented API. You could use the tools here https://doc.qt.io/qt-5/json.html , however if I were to write my own, I would interpret each layer as a QObject basically set up as a QMap<QString,QJSONContainer>, with its json data type set as an enumerator.

I would not use a QVariant, generally because that has many more types than you would need.

In any case, one should be able to do something like this:

QJSONContainer json(QFile("/foo/bar.json")); // Reads from file, creates object api
json["orders"]["custid"].toInt(); // 11045
json["orders"]["fname"].toInt();  // throws error because it is a string
json["orders"]["fname"].setToString("bar"); // changes its type, emits a signal
connect(&json["orders"]["fname"], &QJSONContainer::typeChanged,
[&](QJSONContainer::Type type) {
        switch (type) { 
        case JSONContainer::Null:            break;
        case JSONContainer::Array:  doFoo(); break;
        case JSONContainer::String: doBar(); break;
        default: break;
        }        
});
json.save(); // overwrites old json file 

And with that, you have a JSON api, and it can be used strictly in a console application if you wish.

Build a GUI interpreter:

QWidget QJSONINterpreter::getWidget(QJSONInterpreter::GUIType type)
{
        switch (type) { 
        case JSONInterpreter::Matrix: return getMatrix();
        case JSONInterpreter::Tree:   return getTree();
        default: /* Throw an error */;
        }          
}

thus

QJSONInterpreter jsonGui(json.toUtf8()); // Takes a Json QByteArray
jsonGui.getWidget(QJSONInterpreter::Matrix).show(); // Constructs a matrix from it

Things are now decoupled, and you now have an API you can use for further projects.