7

Software libraries targetting resource constrained environments like embedded systems use conditional compilation to allow consumers to shave space by removing unused features from the final binaries distributed in production.

What are the tradeoffs to building a unique binary? Surely there are the obvious space savings on disk, memory and cpu caches; and the performance benefits brought by space-time tradeoffs. But at least initially it seems that would increase the possibilities to test and design against.

It seems that the implications differ from those of traditional runtime path complexity, not just because of the consideration of 2 different types of branching, but because compiler condition syntax is far unsafer than its runtime counterpart (at least in C).

The specific example that sparked this question is Busybox heavy use of conditionally compiled feature flags. https://git.busybox.net/busybox/tree/networking/httpd.c

TZubiri
  • 435
  • 1
  • 4
  • 9
  • see [What is the problem with "Pros and Cons"?](https://softwareengineering.meta.stackexchange.com/q/6758/31260) – gnat Mar 02 '20 at 06:41
  • I edited the question to focus on the 'cons', we already know the benefits, space. I also narrowed down the suspects to quality, security and code complexity. – TZubiri Mar 02 '20 at 07:00
  • For really large and complicated projects, despite the heavy use of conditional compilations, the goal of testing is not to enumerate every combination of flags. Rather, a limited set of "supported configurations" are tested. Each supported configuration can get their own additional test cases to cover their unique features. Users who use flags outside the supported configurations do so at their own risks. As usual, only the "well traveled paths" would give a sense of "probable correctness"; the least-traveled paths are potentially dangerous and may contain zero-days. – rwong Mar 02 '20 at 19:32
  • I believe conditional compilation can be classified into patterns (like design patterns), anti-patterns (known unsafe usage), and smell (usage of potential concern or warranting review). For example, if conditional compilation is used to stub out a function, a human would understand that function as either running its code or doing nothing, which is easier than a function that uses multiple flags that give rise to O(pow(2, N)) outcomes. – rwong Mar 02 '20 at 19:41
  • After further thought, I agree that the question could still be furthered down, again, the benefits to code complexity and product quality are clear. I am now focusing on the security aspect of the tradeoff, since there are both positive and negative implications, my question deals with quantifying them and comparing them on a unidimensional scale. So I created a new question, any further input is appreciated.thanks! https://softwareengineering.stackexchange.com/questions/406020/does-removing-features-from-a-library-binary-before-consuming-it-have-a-positive – TZubiri Mar 03 '20 at 00:12

2 Answers2

3

Generally speaking, the use of conditional compilation increases complexity which makes it harder to achieve security and other desirable qualities. A part of this complexity is essential if you want to be able to introduce a certain amount of variability. The other part is accidental ans relates to the technical details of conditional compilation and available tool support.

The use of annotation-based variability mechanisms in which the code is annotated with variability information such as conditional compilation affects understandability and testability:

Reduced Understandability How do you know the context of a certain part of the code so you can understand it? How does the compiler know, so you can at least be sure there are no type errors etc.? The problem is that code related to configurable feature is scatteref across the code base. A certain discipline is required to cope with this problem as mentioned in another answer, but does not eliminate the underlying problem.

Reduced Testability How do you know the software works for all combinations of options?

The answer to these questions is that it is not possible for you, the compiler or mainstream testing tools to sufficiently reason about all possible variants before actually building them. The problem is you cannot build all combinations because of the resulting combinatorial explosion. There are concepts for variability-aware analyses but they are not widely adopted, yet.

While as a human you can to a certain extend try to consider a limited amount of variability when understanding code and there exist certain technical concepts to deal with the problem, it is still an #ifdef hell out there in many projects.

As you already mentioned there is no widely available tool support that helps you deal with conditional compilation.

The good news is that people in research are working on solving these problems and investigating alternative variability mechanisms. For instance composition-based variability mechanisms help with local reasoning by modularizing features ( think AOP ). There are also tools that help avoid errors when using annotation-based mechanisms by checking annotations against an explicit variability model for valid combinations.

To deal with testing in the presence of variability im general, there are several sampling approached that help select representative combinations based on different criteria.

There are also variability-aware analysis tools, but those still lack mature implementations and industry acceptance. For instance, many standards require product-bases analysis e.g. for safety-critical domains. Currently, there is a good bunch of concepts that may find their way into mature tools one day.

Hyggenbodden
  • 137
  • 3
  • This is a great answer, but I'll also add that because it's not possible to reason through all of the variants, the use of conditional compilation is discouraged (or even forbidden) when building systems that require a high degree of reliability. I'd have to dig up the appropriate standards or style guides, but I have seen this before in the context of critical systems development. The tools (at least as of a couple of years ago) aren't quite there yet to support the safety critical development processes and give confidence in testing without significant testing of the deployed configuration. – Thomas Owens Mar 02 '20 at 13:18
  • The question of compliance to standards in safety-critical domains becomes particularly interesting when considering novel techniques that are variability-aware, i.e. they intelligently avoid looking at each product separately but are still capable of drawing certain conclusions. In this case, even if you have a nice technical solution the standards often demand a product-based approach. Thus, once the technology is ready, organisational processes will nees to catch up. – Hyggenbodden Mar 02 '20 at 17:05
  • Hello, the question was closed and I made one focused on the security aspects. Your answer was key in solidifying the negative impacts on code complexity and testability, which are now part of the questions assumptions. Now, only the security impacts are left to consider. Check it out! https://softwareengineering.stackexchange.com/questions/406020/does-removing-features-from-a-library-binary-before-consuming-it-have-a-positive – TZubiri Mar 03 '20 at 00:24
2

What are the tradeoffs to building a unique binary? Surely there are the obvious space savings on disk, memory and cpu caches; and the performance benefits brought by space-time tradeoffs. But at least initially it seems that would increase the possibilities to test and design against.

Assuming you're only talking about embedded systems, targeted for resource-constrained environments, like you said:

  • For projects that need to be cross-platform, probably having an unique binary is impossible due to compilation errors yielded by different compilers on unknown structures or symbols.
  • If dynamically linking to other things is not possible during runtime (this will depend on your project and environment targeted), using conditional compilation is a possibility that would solve this.

IMO, if you properly implement your project having separation of concerns and best coding practices in mind, no matter the technique you choose (conditional compilation or not) your project will be easily and safely implemented and maintained.

Example: if you have code for environment A, separate it into a specific file that will be compiled when you target environment A. Wrap/group stuff that belongs together into specific files and folders for that scenario, instead of putting #ifdef SOME_CONDITION spread throughout all of your code.

So, trying to answer your question -- How does conditional compilation impact:

  1. Product quality: if done right (like the example above) it might not impact product quality, since the code does not get polluted or difficult to read (quite the opposite, because you moved code to separated files, and configured those files to be compiled based on some condition); additionally, for testing I don't see how much difference would it take to test several scenarios by: (1) testing different executables generated or (2) testing the same binary, but changing a configuration file; the performance possibly will increase, because a runtime approach will check for all possible options of your program, while a conditionally-compiled program will have only the necessary binary in there;
  2. Security: again, if done right, you can be sure that your program does not contain a specific code that should not be there (for whatever reason of your project), or you can build a version of your application to contain only the capabilities/permissions necessary for a environment (e.g.: to avoid things like privilege escalation); also, you can use conditional compilation if you want to secure the usage of open-source code inside your program (example: you generate a specific version of your program that, for whatever reason, cannot contain GPL code inside of it)
  3. Code complexity: if you spread ifdefs throughout all of your code, it obviously gets more and more complex and eventually unfeasible to maintain; if you can separate them into feature- or platform-dependent files, the complexity for writing code or reviewing will not be impacted that much.
Emerson Cardoso
  • 2,050
  • 7
  • 14
  • What you are basically saying is that if you put in enough effort and discipline it is still possible to create high-quality software despite the disadvantages of using ifdefs. This rather answers the question how to possibly deal with the trade-offs related to ifdefs in practice but does not really explain the drawbacks. – Hyggenbodden Mar 02 '20 at 18:34
  • Thanks for the feedback, I've adjusted my answer to try to be clearer. Regardless, this is my opinion based on some real embedded system projects that I participated; feel free to disagree :) – Emerson Cardoso Mar 02 '20 at 19:23
  • 1
    Oh, I do not think I disagree. I think at least now your answer provides valuable insight. You also adress advantages, related to e.g. security, that compile-time variability provides when compared to run-time variability. Basically you are reducing the attack surface. In think this is very good thought. In my answer I was mainly comparing it to other compile-time or load-time mechanisms. – Hyggenbodden Mar 02 '20 at 19:46
  • Hello, I made a new question focused on the security aspect here https://softwareengineering.stackexchange.com/questions/406020/does-removing-features-from-a-library-binary-before-consuming-it-have-a-positive I agree with Hyggenbodden, 'if done right' at the very least implies that doing it right is harder, there are more ways to do it wrong, so your answer concedes there are negative impacts. A development team has limited resources, if they need to increase efforts to maintain product quality or security, then this design decision had a negative impact, only that it was patched. – TZubiri Mar 03 '20 at 00:29