Do frameworks put too much abstraction?

Question

I've been programming for a little under a year and have some experience writing systems applications, web apps, and scripts for businesses/organizations. However, one thing I've never really done is working with a framework like Django, Rails or Zend.

Looking over the Django framework, I'm a little frustrated with how much is abstracted away in frameworks. I understand the core goals of DRY and minimal code, but some of this over-reliance on different modules and heavy abstraction of core functions feels like it:

1. Makes programs get dated really fast because of the ever-changing nature of modules/frameworks,

2. Makes code hard to understand because of the plethora of frameworks and modules available and all of their idiosyncrasies,

3. Makes code less logical unless you've read all of the documentation; i.e., I can read through some list comprehensions and conditional logic and figure out what a program is doing, but when you see functions that require passing in arbitrary strings and dictionaries, things get a little hard to understand unless you're already a guru in a given module; and:

4. Makes it difficult and tedious to switch between frameworks. Switching between languages is already a challenge, but it's manageable if you have a strong enough understanding of their core functionality/philosophy. Switching between frameworks seems to be more a matter of rote memorization, which in some ways seems to encourage the very inefficiency these frameworks were designed to eliminate.

Do we really need to put like 50 layers of abstraction on top of something as simple as a MySQL query? Why not use something like PHP's PDO interface, where prepared statements/input testing is handled but the universally understandable SQL query is still a part of the function?

Are those abstractions really useful? Isn't feature bloat making them useless, making applications more difficult compared to similar applications written without using a framework?

Answer 1

It seems to me that you misunderstand abstractions and code reuse.

The whole software development industry is built on abstractions. Just because not using them, i.e. avoiding using frameworks, libraries and in general code which is not written in-house, would increase the cost you need to produce a piece of software by hundred, thousand, probably even more.

Like you don't create a jumbo jet from scratch, without using any engineering knowledge developed for years when building other aircraft, you don't write business software from scratch.

You do get that by using PHP or Ruby in the first place, you already have a huge amount of abstractions, are you? The most obvious ones:

1. The operating system, the programming language and the compiler provide a huge abstraction over Assembler and hardware,

2. The web server provides an abstraction of sockets, failover, HTTP protocol, etc.,

3. SQL provides an abstraction over how data is stored on a permanent storage support and kept in memory for faster access, ensures ACID, mirroring, etc.

You may always try to create a web app without using neither an operating system, nor a web server, a database or a high level programming language. You'll quickly find that you spent years reinventing the wheel, i.e. recreating your programming language, your web server and your database, given that their quality would be far from the quality of products we actually use.

Frameworks are not different from that. You can do what they do. Just that you will waste your time redoing the same functionality, with the difference that those frameworks will do it better and very often their code will be tested and documented better than yours.

Take the example of a cart for an e-commerce website. You may invent your own and spend a few weeks developing it, or you may take the one which was developed for years by a bunch of people inside a framework. Theirs is expected to be tested, not counting the fact that during a few years, those developers discovered and patched a bunch of bugs you won't imagine when developing your own cart.

You may reply that theirs is more complicated because it has more user cases. For example theirs should deal with rebates, while you don't have rebates on your website and don't need this feature in the cart. Well, you're not forced to use this feature, and it's not like it will penalize the performance of your web app.

You may have a very basic scenario when it's really easier to develop your own cart instead of using the existent one. In the same way, if the only thing your app needs is to store a list of numbers, nothing more, you don't need a database: a simple text fill would be enough. In those cases, don't over-engineer your app: simple scenarios require simple solutions.

Another factor is the readability of your code. Imagine you've created an e-commerce website. Later, you left the project, and another developer should maintain it.

• If you had used a cart provided by a framework, your colleague would be relieved: he have to maintain a small piece of code which relies on a reliable, heavily documented framework. Even better: this developer may have been used to work with this framework and is familiar to it. If not, he can ask questions on Stack Overflow: for sure, somebody have already used the framework.

• If you had your own cart, then your colleague would have to maintain some custom code, maybe not even documented, without being able to obtain help anywhere.

By using a framework, you rely on a code which is:

• Written by experienced developers,

• Often pair-reviewed,

• Unit-tested,

• Extensively documented,

• Used for years by thousands of developers,

• Often supported, so you can report a bug and see it actually fixed,

• Known by many developers who know the possible caveats and issues and can help on sites like Stack Overflow.

• Available for you right now, for free.

Answer 2

Frameworks can be tricky indeed. Problems can easily arise when a framework is too "opinionated", i.e. when it really prefers one particular style of application and all parts are geared towards supporting this particular style.

For instance, if the framework completely abstracts the authentication process of a user by allowing you to just add one component, add a login template somewhere and voila, you get user authentication for free. This saved you a lot of repetitive work in worrying about cookies, session storage, password hashing and whatnot.

The problems begin when you realize that the default behavior of the framework's authentication code is not what you need. Maybe it's not following the latest best security practices. Maybe you need a custom hook in the process to trigger some action, but the framework doesn't offer one. Maybe you need to change the details of the cookie that is being set, but the framework offers no way to customize this.

The abstraction afforded by the framework allowed you to add an important feature to your site within minutes instead of days initially, but in the end you may have to fight against the framework to make it do what you need it to do, or you'll have to reinvent the functionality from scratch again anyway to suit your needs.

This is not to say framework abstractions are bad, mind you. It is to say that this is always a possibility you need to keep in mind. Some frameworks are explicitly geared towards this, they offer a way to get something up very quickly as a prototype or even production framework for a very specific, limited type of app. Other frameworks are more like loose collections of components which you can use, but which still allow you a lot of flexibility to change it around later.

When using an abstraction, you should always understand at least roughly what it's abstracting away. If you don't understand cookies and authentication systems, starting from an abstraction is not a good idea. If you do understand what you're trying to do and just need code which already does it instead of having to tediously write your own, abstractions are a great time saver. Poorly written abstractions can get you into trouble later though, so it's a double edged sword.

---

You should also distinguish between technical abstractions and "business rule" abstractions. Programming in a higher level language is an abstraction you probably don't want to miss (Python, PHP, C# vs. C vs. Assembler; Less vs. CSS), while "business rule abstractions" can be difficult if they don't meet your needs exactly (one-click authentication vs. hand-coding cookies).

That's because technical abstractions rarely "leak", i.e. you will hardly have to debug machine code when writing applications in Python. Business rule abstractions work on the same technical level though and are just "code bundles" really. You probably will need to debug the cookie that is set or the password hash that is created at some point, which means you'll be diving through lots of 3rd party code.

Answer 3

I agree - most frameworks become bloated dictators. They promise freedom but you end up in servitude. so look at a framework like a tool - the best tool is the one that stays out of your way. If in PHP I would suggest checking out the Codeigniter framework, because its an elegant balance between conventions and freedom and has a great community. And to the poster who used the example of an e-commerce cart - I so wish I could agree with you. but having looked at the code for many e-commerce solutions - and designed a few - I would respectfully disagree with your example.

Answer 4

Hmmm One aspect of LedgerSMB that we worked on a lot was our approach to framework. There are two fundamental problems that come with this sort of abstraction though. These are:

1. Dependency explosion, and

2. Wrong abstraction

The first is actually better than the alternative which is re-inventing the wheel. The second is a bit of a hard one to label because it can come from a poorly defined problem case or, more often, from people using something outside its intended use.

Let's look at ORMs for example. I avoid using ORMs because it is very often the case that the db ends up designed to the object model of the application because at best they are leaky abstraction layers. This isn't necessarily the case. I have met developers who manage to preserve good DB design and good app while using ORMs but they resort to a lot of things the orm hype says shouldn't be required, like encapsulating the relational API of the db behind updateable views.

A major problem of course is that the more highly automated the code, particularly when it is also opaque, the harder it is to see what is going wrong. This is a point about ORMs that @jhewlett makes above (see https://softwareengineering.stackexchange.com/a/190807/63722).

A good parallel might be advanced avionics as a framework for piloting an aircraft. These have been highly engineered for reliability and are a factor in the increase in flight safety. However, as many articles in the IEEE Spectrum point out this comes at a cost of recoverability from errors outside the bounds of what is considered acceptable from an automation perspective. The same goes for debugging. It's one thing to debug SQL code in your program. It's a very different thing to debug a part of your program that writes SQL code for your program to use.

We wrote the LedgerSMB framework from scratch because at the time we started, there were no really good frameworks that did what we wanted. It's actually one thing I am pretty happy with, and according to a newer developer on the project, it makes customization of the application quite straight forward. (We actually keep the SQL code generation to a minimum and instead focus on hand-written user defined functions, meaning the sql-writing portions are very thin glue). Yes it provides a lot of abstraction in some places, more than some programmers are comfortable with (in particular "map object properties to arguments in that stored procedure" gets us pushback from time to time). We try, however, to keep everything simple and consistent so that it is straight forward to determine what went wrong. This works pretty well.

In the end "too much" is a bit subjective, but on an objective level it also depends on what you are doing. If you are doing exactly what the framework was designed to do, a well-designed framework will provide a perfect amount of abstraction. If you are doing something which is a bit less of a fit, the abstraction will become both too much and too leaky.

Answer 5

Yes, they are useful. They provide you the benefit of lots of other experienced developers doing work to solve a problem you need to solve, with the added benefits of having tested it, fixed lots of bugs, and provided solutions to tricky problems you don't have to worry about yourself by using the framework.

Can they be bloated overkill? Sure. This all depends on what you need, and what the framework brings to the table. If you have a simple web app, maybe Rails is overkill for you, and you should look at something simpler like Sinatra as a solution.

There is definitely a learning curve to all frameworks, and it's steeper with the more work that it saves for you. However, learning the framework means you've saved time and can leverage all of that knowledge on the next project, instead of rewriting your application from scratch, a second time.

But, you say, I'll just copy my code from the old project, and I can use that as a starting point for my new project! I'll just change what is different, and maybe make some of my objects/functions more general, and think of a better way to solve that tricky bit of code! Congratulations, you've just created a framework. Do this a few thousand more times, and you have something like Django, Rails or Zend.

Answer 6

Frameworks usually result in more productivity (perhaps after a slight learning curve), but there is often a trade-off involved. In some cases, for example, you gain programmer productivity at the cost of decreased performance.

Consider Object-Relational Mappers (ORMs). They are great in that they take care of a lot of the tedious mapping code for you. They may even create your objects for you. However, in abstracting away the SQL, it's much harder to reason about performance and optimize your bottlenecks.

If you are building an application that won't have a lot of data or complex queries, performance may not be an issue for you. Indeed, programmer time is the bottleneck for a lot of projects, and frameworks, libraries, and high-level languages help to alleviate that.

Answer 7

I agree that "You use libraries, but frameworks use you". That's a very neat way to put it. I don't agree that as soon as you start re-using code you are beginning to build a framework, as you usually don't need to do much more than a quick copy and paste to make use of your code again - that's a library you are building up; no need to get weird about how you bring the code into your site or app.

For me, the crucial point is that it I have to get more out of a resource than it requires me to invest. Or,from another angle, I would say that as soon as a framework's needs are greater than the reward(s) available, all advantage is absent. So it's 'Yes! Please! And thank you!' to all who make assets that will drop straight and function as needed within my own html/CSS/PHP and SQL.

One thing I find incomprehensible is that frameworks are said to make maintenance easier? If the complex mesh of interworking parts isn't performing as intended, you'd better know everything there is to know about the framework in question. Or be ready for a huge input of new syntax.

Answer 8

Some people say this is the Hollywood Principle of frameworks: "Don't call us, we'll call you". In contrast, whenever you use a library, your code calls the library, not the other way around.

As you can see, the important point is who is in control -- namely, in control of the flow of control. Who decides which statement is run after which one?

There are arguments pro and contra, and whenever I see such endless discussions, I tend to think this must be a matter of taste, rather than an objectively decidable question. Otherwise, it would have been decided already.

In my view, whether you prefer to use frameworks or libraries tells something about what kind of developer you are, and not whether one of both approaches is superior and will eventually prevail.

If you prefer frameworks, you prefer to exchange freedom for security: you are probably more pragmatic and you tend to trust others to do their work properly. If you prefer libraries, you prefer to exchange security for freedom: you are probably more of an idealist and you question other's ideas and claims.

I do not think it would be wise to decide it is better to be like the former or the latter.

Answering your question: Do frameworks put too much abstraction? It depends on who you are, and specifically on what distance from first principles do you feel comfortable with.

...

And even more specifically, if you do not like frameworks like Django, go search for "microframeworks" like Flask :)