6

Recently came across MDDA, I think the term was first coined by Salesforce (pg3) and others have followed it like Metadata Driven Architecture at benefitfocus.com (less dense and well written).

The design sounds useful for enterprise products where the platform is flexible enough to help customers tailor (by end user or a solution engineer) their experience but keeping core system behavior intact.

  1. This might be a common pattern in enterprise SaaS apps, how do they let customers get the flexibility they want without adding a series of if-else logic?
  2. What are some specific engineering design patterns on MDDA?
  3. Are there any open source alternatives where I can look at the code and talk more concretely about it?
  4. It should great in theory but in practice when a system is built with so much flexibility, what are some of the issues with this?
zengr
  • 1,217
  • 11
  • 22
  • 2
    You know, every few years something like this comes up, and it's a bad idea every time. When something tries to solve all problems it will only do so poorly. – Telastyn Sep 11 '17 at 17:32
  • *"If you are in a non-MDDA world and want to get to an MDDA world, you must first take the extreme position. Focus on how nirvana will look, and get that vision deep into your psyche."* Yeah, just another marketing guy trying to peddle the latest "silver bullet." Frankly, it sounds like an [inner platform](https://en.wikipedia.org/wiki/Inner-platform_effect) to me, though there is [some evidence that it might work under the right conditions](http://www.paulgraham.com/avg.html). – Robert Harvey Sep 11 '17 at 17:32
  • 1
    Note that Microsoft had [already been talking about this in 2004](https://msdn.microsoft.com/en-us/library/aa480019.aspx), back when SOA, COM and CORBA were still a big deal. – Robert Harvey Sep 11 '17 at 17:36
  • @Telastyn Yeah totally, I had a bad feeling about building such a flexible system too, updating the question. – zengr Sep 11 '17 at 17:36
  • 4
    Also, note that the example you cite (SalesForce) requires specialized consultants to make it work. We hired one full-time at my last job, and the company wasn't even that large. I've seen this happen several times at different companies: the purchase of extremely complicated, very expensive software that requires equally expensive consultants to maintain and customize it. – Robert Harvey Sep 11 '17 at 17:39
  • @RobertHarvey I don't think that's a bad idea. Keeping the product team focused on building the core system and solution engineering customizing views for customers, in theory, sounds better than one team owning both. – zengr Sep 11 '17 at 17:42
  • My point is, by the time you go to all that expense you might as well hire a second team and write your own software in the conventional way. Far from being a cost-reducing measure, the MDA only serves to further complicate the software and necessitate the hiring of experts to maintain it. – Robert Harvey Sep 11 '17 at 17:43
  • I see your point, makes sense. – zengr Sep 11 '17 at 17:45
  • 1
    The idea is not new and somewhat successful. See [SAP](https://en.wikipedia.org/wiki/SAP_SE), their main product line is pretty much "metadata driven". So if you are trying to build a competing company or product, good luck. – Doc Brown Sep 11 '17 at 17:46
  • @RobertHarvey Although how do you solve the problem of different customers with different needs (and these are F500 companies who mostly have some weird use case because of their internal processes). How can I build a piece of software which is flexible enough and can be sold to such customers with min. effort? – zengr Sep 11 '17 at 17:48
  • That's called "multi-tenant." You can get about 60 to 80 percent there with conventional development techniques. There are a number of ways to approach customization without having to go the SalesForce route. The problem with MDA and similar approaches is that the level of knowledge required almost always outstrips the customer's abilities. It starts out like this: "I know; I'll just write a customer-customizable query/report generator and let them make their own reports," which turns into a multi-tentacled monster that nobody understands, and you're back where you started. – Robert Harvey Sep 11 '17 at 17:50
  • `conventional techniques`: What are some conventional techniques? – zengr Sep 11 '17 at 17:54
  • Anything where XML is not masquerading as code, and databases aren't poor copies of themselves. Ordinary code, in other words. Rules engines. Domain-specific languages. Code as data (see http://www.paulgraham.com/avg.html). Plugin architectures. See [here](https://www.red-gate.com/simple-talk/opinion/opinion-pieces/bad-carma/) for a cautionary tale (a long read, but worth it). – Robert Harvey Sep 11 '17 at 17:56

1 Answers1

5

Oof, big topic. Yes, this can be a typical pattern in enterprise-class software, but I don't necessarily think it's a good one. Consider avoiding it if you can. The core issue here is that it exists at a super-high level of abstraction. When a tool doesn't know what the user wants to do, it inevitably abdicates the decision and gives you a framework for building your own tool. This starts off as a cool sales pitch about customizability, but in some ways MDA is the ultimate scope creep; if you become that flexible, really there's nothing your software can't do (in theory).

This is such a classic mistake that there are all kinds of jokes about it.

Zawinski's Law -- "all truly useful programs experience pressure to evolve into toolkits and application platforms"

Greenspun's Tenth Rule -- and others too.

Bottom line for me, MDA exists at a very high level of abstraction. Systems should avoid over-abstraction. I think it's too much to say categorically that you should never do this, sometimes it might work, but you better have a damn airtight case why it's necessary before you do it, otherwise I'd argue it's over-abstraction.

Specific engineering design patterns - really it's about creating database entities and relationships in a very abstract way, and allowing the user with a UI to specify and relate those things. On the UI side, same concept except instead of using database entities/relationships, you're using panels, inputs, forms, etc.

Open source alternatives - not sure on this one, but I would be wary of them if there are, because usually "super flexible" solutions end up getting customized to the problem domain (CRM, or whatever). Maybe some of the interface and data builders that come with Eclipse?

The issues that come with these approaches are many.

Testing the Beast

Because it's so abstract, testing becomes very hard. It's not possible to test every combination, so designing a representative covering suite is a very hard thing to do. That new feature you developed, does it interact poorly with Customer X's bizarre customization that was really working against how the tool was supposed to function in the first place?

Understanding Your Own System

Second, it allows the user to encode knowledge into the system that the system itself doesn't have. Imagine a back-end engineer looking at a customization of his own system and having a hard time recognizing where to start, because knowing your way around depends on knowledge outside of the system's specs. An earlier commenter pointed out that for salesforce, you have to hire special consultants to help you implement it. Yep, sounds right.

Inappropriate User Expectations

There's a general tradeoff between system power and simplicity; these systems are very powerful, and tend to be the opposite of simple. So basically all of the downsides of complexity come in the bargain for more flexibility. This also creates in the users and product managers a sense of "we can do anything" which makes it very difficult to set and understand proper system scope boundaries.

FrobberOfBits
  • 759
  • 1
  • 6
  • 9
  • Thanks and upvoted! Really helpful. What's your take on building a system where customers (mostly enterprise) want a slightly different version of the report? 1. A chain of if-else is probably hard to manage, 2. MDDA is an overkill. What's the sweet spot for you? (I understand hard to pinpoint one answer) – zengr Sep 11 '17 at 20:02
  • That's better/easier I think. The fundamental is the power / complexity tradeoff. You can have some simple reporting customizations that aren't too bad. And it's already not so bad if all of your metadata is known and locked down, and then you're just supporting dynamic query of that. Customizable metadata is a whole other kettle of fish. – FrobberOfBits Sep 11 '17 at 20:43