31

"Premature optimization is the root of all evil"

I think this we can all agree upon. And I try very hard to avoid doing that.

But recently I have been wondering about the practice of passing parameters by const Reference instead of by Value. I have been taught / learned that non-trivial function arguments (i.e. most non-primitive types) should preferably be passed by const reference - quite a few books I've read recommend this as a "best practice".

Still I cannot help but wonder: Modern compilers and new language features can work wonders, so the knowledge I have learned may very well be outdated, and I never actually bothered to profile if there are any performance differences between

void fooByValue(SomeDataStruct data);

and

void fooByReference(const SomeDataStruct& data);

Is the practice that I have learned - passing const references (by default for non-trivial types) - premature optimization?

CharonX
  • 1,633
  • 1
  • 11
  • 23
  • 3
    See also: [F.call in the C++ Core Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#fcall-parameter-passing) for a discussion of various parameter passing strategies. – amon Jun 05 '18 at 11:18
  • 1
    Possible duplicate of [Is it bad practice to write code that relies on compiler optimizations?](https://softwareengineering.stackexchange.com/questions/359027/is-it-bad-practice-to-write-code-that-relies-on-compiler-optimizations) – Doc Brown Jun 05 '18 at 11:20
  • 2
    @DocBrown The accepted answer to that question refers to the *least astonishment principle*, which may apply here too (i.e. using const references is industry standard, etc. etc.). That said, I disagree that the question is a duplicate: The question you refer to asks if it is bad practice to (generally) rely on compiler optimization. This question asks the inverse: Is passing const references a (premature) optimization? – CharonX Jun 05 '18 at 12:18
  • @CharonX: if one can rely on compiler optimization here, the answer to your question is clearly "yes, the manual optimization is not necessary, it is premature". If one cannot rely on it (maybe because you don't know beforehand which compilers will ever be used for the code), the answer is "for larger objects it is probably not premature". So even if those two questions are not literally equal, they IMHO seem to be alike enough to link them together as duplicates. – Doc Brown Jun 05 '18 at 12:52
  • 1
    @DocBrown: So, before you can declare it a dupe, point out where in the question it says that the compiler will be allowed and able to "optimize" that. – Deduplicator Jun 05 '18 at 13:38
  • I have a follow-on question to this, having only recently come back to C++ after a 15 year absence. If the parameter is 'const', is the & reference still necessary or are compilers smart enough to recognize that the variable can't be changed and so will pass a reference if that's more efficient than copying? – David Sep 14 '18 at 03:35
  • 2
    @David Yes, it is still necessary (and will remain so). The compiler is able to perform some (well documented) optimizations like [Copy elision](https://en.wikipedia.org/wiki/Copy_elision), but what you suggest would require changing the type of a variable (from `const T` to `const T&`) which is **not** permitted by the standart for a good number of reasons: From thread-safety (are the functions reentrant?) to application logic (the class may e.g. use a `mutable` cache to optimize calculations) where using a (const) reference instead of a (const) copy would break things. – CharonX Sep 14 '18 at 07:13
  • Fair enough and of course those reasons are obvious once you point them out. Thanks for taking the time to respond. – David Sep 14 '18 at 13:42
  • 1
    @David In addition to the previous comment, the compiler doesn't even know whether or not a parameter is const based on the declaration, since the const only has to be specified in the definition and only applies to the implementation. – nw. Nov 15 '20 at 03:40
  • @CharonX The optimizer would **not** "change the type of `const T` to `const T&`" -- the type would still be `const T` as far as the VM defined by the Standard; in the background the optimizer would decide to copy or add the indirection. The complications you describe don't matter because leaving the `&` out of the function header would move the copy/not copy decision from inside the function to the point of the call. When writing a template function prototype, the parameter may be "expensive to copy" for some specializations but not others. But we're forced to pick one. – Spencer Apr 27 '23 at 15:32
  • @Spencer I can't quite understand what you are trying to say - David asked if it was (still) necessary to explicitely write `const T&` or if the compiler was "smart enough to recognize that the variable can't be changed and so will pass a reference if that's more efficient than copying". To which I replied that it **is** necessary to write `const T&` if the use of a reference is desired, as the compiler **cannot** simply change a `const T` parameter to a `const T&` parameter, giving the example of a `mutable` member inside `T` impacting behaviour. I wrote **nothing** about headers. – CharonX Apr 28 '23 at 10:10
  • @CharonX "if the use of a reference is desired" -- my point is that in a pass-by-value function parameter, a reference is _rarely_ desired as an end in itself. Templates are just the clearest example of why this is so. – Spencer Apr 28 '23 at 12:12
  • @Spencer A tomato is technically a fruit (classified botanically as a berry), not a vegetable. (I still don't understand whatever it is you are trying to say) – CharonX Apr 28 '23 at 13:10
  • @CharonX Bad example, and that's why it's nearly impossible to have productive discussions in comments. If you want to continue, maybe chat... – Spencer Apr 28 '23 at 13:21
  • @Spencer I apologize; I was getting a bit frustrated. My statement - while (technically) correct - had little relevance to the issue at hand... which is basically the impression I got from yours: In my replies to David I did **not** claim the optimizer would "change the type of const T to const T&", nor I say anything about headers, or regarding a reference being desired as an end in itself, which made your comments - while technically correct - bewildering for me. I assume there is simply some mutal misunderstanding, but be it as it may, I don't think further discussions would be fruitful. – CharonX Apr 28 '23 at 14:15

5 Answers5

75

"Premature optimisation" is not about using optimisations early. It is about optimising before the problem is understood, before the runtime is understood, and often making code less readable and less maintainable for dubious results.

Using "const&" instead of passing an object by value is a well-understood optimisation, with well-understood effects on runtime, with practically no effort, and without any bad effects on readability and maintainability. It actually improves both, because it tells me that a call will not modify the object passed in. So adding "const&" right when you write the code is NOT PREMATURE.

gnasher729
  • 42,090
  • 4
  • 59
  • 119
  • 7
    I agree on the "practically no effort" part of your answer. But premature optimization is first and foremost about optimization before their is an noteable, measured performance impact. And I don't think most C++ programmers (which includes myself) make any measurements before using `const&`, so I think the question is quite sensible. – Doc Brown Jun 05 '18 at 20:59
  • 1
    You measure before optimising to know whether any tradeoffs are worth it. With const& the total effort is typing seven characters, and it has other advantages. When you don't intend to modify the variable being passed in, it is of advantage even if there is no speed improvement. – gnasher729 Jun 05 '18 at 21:23
  • Don't get me wrong, I agree with almost anything you wrote. However, I think it is a matter of perspective if one should say "`const&` is no premature optimization" (saying the question a logical fallacy right from the start), or better "`const&` is something like an acceptable form of premature optimization" (saying the question is justfied). But maybe making this distinction is not worth the hassle. – Doc Brown Jun 05 '18 at 21:38
  • 3
    I'm not a C expert, so a question:. `const& foo` says the function won't modify foo, so the **caller** is safe. But a copied value says that no **other thread** can change foo, so the **callee** is safe. Right? So, in a multi-threaded app, the answer depends on correctness, not optimization. – user949300 Jun 05 '18 at 22:35
  • 1
    @user949300 I would posit that (1) it is far better to put one of these into the design of an object: (1.1) immutability, or (1.2) mutability (better with protection), or (1.3) value-semantic; (2) doing so takes the guesswork out of parameter-passing; it reduces mental effort on the user; (3) that this advice applies to both the object on which the method(s) are defined, as well as objects being passed as arguments; (4) when done properly, const-reference can be used without introducing bugs; (5) if we choose to ignore these points, the matter of multithreading doesn't lead to a clear answer. – rwong Jun 05 '18 at 23:16
  • 1
    @DocBrown you may eventually quetion the motivation of the developer which put the const & ? If he did it for performance only without considering the rest it might be considered as premature optimization. Now if he puts it because he knows it will be a const parameters then he's only self documenting his code and giving the opportunity to the compiler to optimize, which is better. – Walfrat Jun 06 '18 at 14:59
  • 1
    @user949300: Few functions allow their arguments to be modified concurrently or by callbacks used, and they explicitly say so. – Deduplicator Jun 06 '18 at 15:24
  • 1
    @user949300: Sadly not. “const T& x” forbids modifying the variable through x. It is legal to take its address: T* p = (T*)&x and assigning to *p. It is possible for other threads to write to the variable if the variable itself is non-const. Both are obviously horrible things to do and should be stopped by a code review but won’t be stopped by the compiler. – gnasher729 May 01 '23 at 12:30
  • @gnasher729 did you intend to hide the `const_cast` as `(T*)` there? – Caleth May 02 '23 at 12:20
21

TL;DR: Pass by const reference is still a good idea in C++, all things considered. Not a premature optimization.

TL;DR2: Most adages don't make sense, until they do.

Aim

This answer just tries to extend the linked item on the C++ Core Guidelines(first mentioned in amon's comment) a little bit.

This answer does not try to address the issue of how to think and apply properly the various adages that were widely circulated within programmers' circles, especially the issue of reconciling between conflicting conclusions or evidence.

Applicability

This answer applies to function calls (non-detachable nested scopes on the same thread) only.

(Side note.) When passable things can escape the scope (i.e. have a lifetime that potentially exceeds the outer scope), it becomes more important to satisfy the application's need for object lifetime management before anything else. Usually, this requires using references that are also capable of lifetime management, such as smart pointers. An alternative might be using a manager. Note that, lambda is a kind of detachable scope; lambda captures behave like having object scope. Therefore, be careful with lambda captures. Also be careful with how the lambda itself is passed - by copy or by reference.

When to pass by value

For values that are scalar (standard primitives that fit within a machine register and have value semantic) for which there is no need for communication-by-mutability (shared reference), pass by value.

For situations where callee require a cloning of an object or aggregate, pass by value, in which the callee's copy fulfills the need for a cloned object.

When to pass by reference, etc.

for all other situations, pass by pointers, references, smart pointers, handles (see: handle-body idiom), etc. Whenever this advice is followed, apply the principle of const-correctness as usual.

Things (aggregates, objects, arrays, data structures) that are sufficiently large in memory footprint should always be designed to facilitate pass-by-reference, for performance reasons. This advice definitely applies when it is hundreds of bytes or more. This advice is borderline when it is tens of bytes.

Unusual paradigms

There are special-purpose programming paradigms which are copy-heavy by intention. For example, string processing, serialization, network communication, isolation, wrapping of third-party libraries, shared-memory inter-process communication, etc. In these application areas or programming paradigms, data is copied from structs to structs, or sometimes repackaged into byte arrays.

How the language specification affects this answer, before optimization is considered.

Sub-TL;DR Propagating a reference should invoke no code; passing by const-reference satisfies this criterion. However, all other languages satisfy this criterion effortlessly.

(Novice C++ programmers are advised to skip this section entirely.)

(The beginning of this section is partly inspired by gnasher729's answer. However, a different conclusion is reached.)

C++ allows user-defined copy constructors and assignment operators.

(This is (was) a bold choice that is (was) both amazing and regrettable. It is definitely a divergence from today's acceptable norm in language design.)

Even if the C++ programmer does not define one, the C++ compiler must generate such methods based on language principles, and then determine whether additional code needs to be executed other than memcpy. For example, a class/struct that contains a std::vector member has to have a copy-constructor and an assignment operator that is non-trivial.

In other languages, copy constructors and object cloning are discouraged (except where absolutely necessary and/or meaningful to the application's semantics), because objects have reference semantics, by language design. These languages will typically have garbage collection mechanism that is based on reachability instead of scope-based ownership or reference-counting.

When a reference or pointer (including const reference) is passed around in C++ (or C), the programmer is assured that no special code (user-defined or compiler-generated functions) will be executed, other than the propagation of the address value (reference or pointer). This is a clarity of behavior that C++ programmers find comfortable with.

However, the backdrop is that the C++ language is unnecessarily complicated, such that this clarity of behavior is like an oasis (a survivable habitat) somewhere around a nuclear fallout zone.

To add more blessings (or insult), C++ introduces universal references (r-values) in order to facilitate user-defined move operators (move-constructors and move-assignment operators) with good performance. This benefits a highly relevant use case (the moving (transfer) of objects from one instance to another), by means of reducing the need for copying and deep-cloning. However, in other languages, it is illogical to speak of such moving of objects.

(Off-topic section) A section dedicated to an article, "Want Speed? Pass by Value!" written in circa 2009.

That article was written in 2009 and explains the design justification for r-value in C++. That article presents a valid counter-argument to my conclusion in the previous section. However, the article's code example and performance claim has long been refuted.

Sub-TL;DR The design of r-value semantics in C++ allows for a surprisingly elegant user-side semantics on a Sort function, for example. This elegant is impossible to model (imitate) in other languages.

A sort function is applied to a whole data structure. As mentioned above, it would be slow if a lot of copying is involved. As a performance optimization (that is practically relevant), a sort function is designed to be destructive in quite a few languages other than C++. Destructive means that the target data structure is modified to achieve the sorting goal.

In C++, the user can choose to call one of the two implementations: a destructive one with better performance, or a normal one which does not modify the input. (Template is omitted for brevity.)

/*caller specifically passes in input argument destructively*/
std::vector<T> my_sort(std::vector<T>&& input)
{
    std::vector<T> result(std::move(input)); /* destructive move */
    std::sort(result.begin(), result.end()); /* in-place sorting */
    return result; /* return-value optimization (RVO) */
}

/*caller specifically passes in read-only argument*/ 
std::vector<T> my_sort(const std::vector<T>& input)
{
    /* reuse destructive implementation by letting it work on a clone. */
    /* Several things involved; e.g. expiring temporaries as r-value */
    /* return-value optimization, etc. */
    return my_sort(std::vector<T>(input));  
}

/*caller can select which to call, by selecting r-value*/
std::vector<T> v1 = {...};
std::vector<T> v2 = my_sort(v1); /*non-destructive*/
std::vector<T> v3 = my_sort(std::move(v1)); /*v1 is gutted*/

Aside from sorting, this elegance is also useful in the implementation of destructive median finding algorithm in an array (initially unsorted), by recursive partitioning.

However, note that, most languages would apply a balanced binary search tree approach to sorting, instead of applying a destructive sorting algorithm to arrays. Therefore, the practical relevance of this technique is not as high as it seems.

How compiler optimization affects this answer

When inlining (and also whole-program optimization / link-time optimization) is applied across several levels of function calls, the compiler is able to see (sometimes exhaustively) the flow of data. When this happens, compiler can apply many optimizations, some of which can eliminate the creation of whole objects in memory. Typically, when this situation applies, it doesn't matter if the parameters are passed by value or by const-reference, because the compiler can analyze exhaustively.

However, if the lower level function calls something that is beyond analysis (e.g. something in a different library outside compilation, or a call graph that is simply too complicated), then the compiler must optimize defensively.

Objects larger than a machine register value might be copied by explicit memory load/store instructions, or by a call to the venerable memcpy function. On some platforms, the compiler generates SIMD instructions in order to move between two memory locations, each instruction moving tens of bytes (16 or 32).

Discussion on the issue of verbosity or visual clutter

C++ programmers are accustomed to this, i.e. as long as a programmer doesn't hate C++, the overhead of writing or reading const-reference in source code isn't horrible.

The cost-benefit analyses might have been done many times before. I don't know if there's any scientific ones which should be cited. I guess most analyses would be non-scientific or non-reproducible.

Here is what I imagine (without proof or credible references)...

  • Yes, it affects the performance of software written in this language.
  • If compilers can understand the purpose of code, it could potentially be smart enough to automate that
  • Unfortunately, in languages that favor mutability (as opposed to functional purity), the compiler would classify most things as being mutated, therefore the automated deduction of constness would reject most things as non-const
  • The mental overhead depends on people; people who find this to be a high mental overhead would have rejected C++ as a viable programming language.
rwong
  • 16,695
  • 3
  • 33
  • 81
  • This is one of the situations where I wish I could accept two answers instead of having to chose only one... *sigh* – CharonX Jun 08 '18 at 10:15
11

In DonaldKnuth's paper "StructuredProgrammingWithGoToStatements", he wrote: "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." - Premature Optimization

This isn't advising programmers to use the slowest techniques available. It's about focusing on clarity when writing programs. Often, clarity and efficiency are a trade-off: if you must pick only one, prefer clarity. But if you can achieve both easily, there's no need to cripple clarity (like signaling that something is a constant) just to avoid efficiency.

Lawrence
  • 637
  • 3
  • 10
  • 4
    "if you must pick only one, pick clarity." The second should be *prefer* instead, as you might be forced to choose the other. – Deduplicator Jun 05 '18 at 23:26
  • @Deduplicator Thank you. In the OP’s context, though, the programmer has the freedom to choose. – Lawrence Jun 05 '18 at 23:44
  • Your answer reads a bit more general than that though... – Deduplicator Jun 05 '18 at 23:57
  • @Deduplicator Ah, but the context of my answer is (also) that the *programmer* chooses. If the choice was forced on the programmer, it wouldn't be "you" that does the picking :) . I considered the change you suggested and wouldn't object to you editing my answer accordingly, but I prefer the existing wording for its clarity. – Lawrence Jun 06 '18 at 00:01
  • Is `const X` more or less clear than `const X &`? – Spencer Apr 28 '23 at 15:06
  • @Spencer I haven't kept up with my C syntax. Is your trailing ampersand referring to the C++11 syntax of assignable r-values (normally, you can't do "f() = x;", but if f() is defined as returning a reference using the trailing ampersand, you can)? But since you are making it const, it doesn't make sense to try to assign to it. On the other hand, if you're referring to passing a parameter by-value vs by-reference, passing by value is clearer while passing by reference may be more efficient without losing too much in clarity. – Lawrence Apr 28 '23 at 17:02
  • PS: you've just pushed me over the fence to @Deduplicator's argument. :) – Lawrence Apr 28 '23 at 17:02
  • There's a bigger problem here. Looking at your answer more closely, you seem to be arguing against const-correctness? Why do you think that reduces clarity? – Spencer Apr 28 '23 at 17:36
  • @Spencer The OP asked about avoiding const markers to counter premature optimisation. I argued that such avoidance isn't what countering premature optimisation is trying to achieve. I'm not sure what you mean by "const-correctness" and why you think I'm arguing against it (or how the term "const-correctness" relates to the topic of clarity). Can you please elaborate? – Lawrence Apr 29 '23 at 17:11
  • @Lawrence OP didn't single out `const`; you did. Logically, it's the _reference_ that's the "optimization". Perhaps OP didn't understand that, but that doesn't matter. Your statement "cripple clarity (like signaling that something is a constant)" is just outright wrong -- lots of C++ code depends on that "signal". It does not cripple clarity, it improves it. – Spencer Apr 30 '23 at 12:19
  • @Spencer Umm, "const" is right there in the question's title. As for crippling clarity, you've misread my answer. I'm saying that clarity and efficiency can sometimes coexist, such as with the use of "const". People shouldn't avoid using "const" just because they think it will be more efficient (i.e. trying to flee from premature optimisation). Put another way, the goal isn't to find the least efficient way to code. The goal should be to find the clearest correct program, and sometimes that would also be the most efficient. And that's good. – Lawrence Apr 30 '23 at 16:38
9

Passing by ([const][rvalue]reference)|(value) should be about the intent and promises made by the interface. It has nothing to do with performance.

Richy's Rule of Thumb:

void foo(X x);          // I intend to own the x you gave me, whether by copy, move or direct initialisation on the call stack.     

void foo(X&& x);        // I intend to steal x from you. Do not use it other than to re-assign to it after calling me.

void foo(X const& x);   // I guarantee not to change your x

void foo(X& x);         // I may modify your x and I will leave it in a defined state
Richard Hodges
  • 227
  • 1
  • 2
3

Theoretically, the answer should be yes. And, in fact, it is yes some of the time--as a matter of fact, passing by const reference instead of just passing a value can be a pessimization, even in cases where the passed value is too large to fit in a single register (or most of the other heuristics people try to use to determine when to pass by value or not). Years ago, David Abrahams wrote an article named "Want Speed? Pass by Value!" covering some of these cases. It's no longer easy to find, but if you can dig up a copy it's worth reading (IMO).

In the specific case of passing by const reference, however, I'd say the idiom is so well established that the situation is more or less reversed: unless you know the type will be char/short/int/long, people expect to see it passed by const reference by default, so it's probably best to go along with that unless you have a fairly specific reason to do otherwise.

Jerry Coffin
  • 44,385
  • 5
  • 89
  • 162