3

I was reading a really great Bjarne Stroustrup's article where he exposes some C++ myths. I look at code that focus my attention, because I wouldn't know that C++ supports this kind of expressions. I'm talking about lambda expressions, or closures. 

void do_my_sort(vector<double>& v)
{
  sort(v,[](double x, double y) { return x>y; });  // sort v in decreasing order
}

I was working with C# and JavaScript for several years, more recently with Java and Objective C and I find this language feature very powerful and useful. I currently doing some works in Smalltalk and, no surprise... the closures are there too...

So I was wondering, why this language feature came so late to C++ (2011)?

Agustin Meriles
  • 141
  • 1
  • 5
  • 2
    committee disagreed on how it should look like and how it owuld work together with other features. – ratchet freak Dec 23 '14 at 20:09
  • @ratchetfreak is that merely an opinion or you can back it up somehow? (not that I don't believe that mind you but blaming design-by-committee seems to be hip even when there are sensible explanations for particular decisions) – gnat Dec 23 '14 at 20:13
  • 1
    If I remember correctly there were a few language features that had to come first (or at the same time) to support lambdas correctly. I have Bjarne's latest C++ book at home, I can check later because I remember he addressed this specific question in there somewhere. –  Dec 23 '14 at 20:17
  • 6
    Closures are hard to get right in a language without fully managed memory. You have to specify all the nasty nuances of ownership, lifetime, copying, etc. Closures in C++ are much, much more complicated than closures in Lisp or JavaScript. And it was not quite clear up until recently that closures are of any use for a typical enterprisey coding, not until LINQ changed the prevailing perception. – SK-logic Dec 23 '14 at 21:13
  • @SK-logic so LINQ may be had influence in C++... That is so curious, C# influencing C++ :) – Agustin Meriles Dec 23 '14 at 21:17
  • 2
    @AgustinMeriles, probably not directly, but most certainly by shifting the public opinion. There is little doubt that it was specifically LINQ which lured the wider programming audience towards some bits of functional programming. – SK-logic Dec 23 '14 at 21:44
  • 1
    Your example is not a closure. But it's exactly closures which are the problem. Maybe that's why you can't see the difficulties? – Jörg W Mittag Dec 24 '14 at 00:20
  • 4
    @SK-logic I have to disagree on the influence of LINQ. Closures are the oldest trick in the book. They existed before the first computer was ever created and they first appear in LISP in the late 50s, ML (70s), Haskell (80s) then other languages adopted them Python, Ruby, JavaScript, then a rebirth of functional programming in 2000 with Scala, Clojure and F# among others. Actually C# and Java are the oldest languages to adopt this. Now you are not going to say that LINQ popularized high order programming. This is just the result of the natural evolution of languages. – edalorzo Dec 24 '14 at 14:30
  • 1
    @edalorzo And SK-Logic probably knows all that, but he's probably right that they weren't popular with the typical enterprisey programmer. James Gosling claims that he didn't consider adding any feature to Java unless multiple people requested it. And yet Java didn't get real lambda expressions until a couple of months ago, and the Collections API has none of the standard high-order functions like `map`, `fold`, or `filter`. So how were closures popular again? – Doval Dec 24 '14 at 14:54
  • @Doval Precisely my point. Closures and lambda expressions were already pretty, pretty popular before LINQ. The fact that Java and C# are mainstream programming languages have simply make them more popular than they were to more programmers. Above all to those who have never programmed in other languages and were probably unfamiliar with the topics this might sound like revolutionary, but for many others there is not a big surprise here. – edalorzo Dec 24 '14 at 15:01
  • Closures have become popular in recent years thanks to languages like JavaScript and maybe C#, and to the FP-hype (I am a big fan of FP, but IMO there is also a FP-hype going on). IMHO they are out of place in C++, and they are difficult to get right, but everybody must have closures nowadays. – Giorgio Dec 24 '14 at 20:31
  • @Giorgio: Then you're nuts. C++ posted the functional STL algorithms long before LINQ, or closures in C# or C++. Lambdas are nothing more than permitting those algorithms to be used in a vaguely sane fashion. – DeadMG Dec 26 '14 at 12:35
  • @eldorado, closures were considered pretty worthless in mainstream ever since they failed miserably in Smalltalk. And now I'm pretty sure that LINQ forced closures into Java and C++, not anything else, because nothing else happened during this time frame. – SK-logic Dec 27 '14 at 20:56
  • Smalltalk itself failed because it was too expensive, and didn't talk to the OS well, but Python & Ruby have pretty much beaten the non-closure languages for mindshare of new projects, and Java/C# ended up adding them as well. – aoeu256 Sep 03 '19 at 17:22

5 Answers5

17

C++ had function objects from 1983 onwards; they took/takes care of many examples where people now use lambdas (and use lambdas in other languages). In fact, a C++ lambda is probably best understood as a simplified notation for defining and immediately using a function object.

Bjarne Stroustrup
  • 171
  • 2
9

Weak compilers, large scope, existing library weaknesses, and potentially negative library interactions.

First, it was very hard for implementations to support all of C++98, and only occurred many years after the Standard shipped. Adding even more features in would only have made this worse. C# doesn't have this problem because C# 1.0/2.0 are very easy to implement in comparison. The fact is that even now C++ tooling is far behind tooling for these other languages because C++ is such a bitch to handle.

Secondly, C++ lambdas have to deal with a lot of additional details. For example, the lifetime of captured objects is something that C#/JS do not have to consider.

Thirdly, I will say that there were few examples of APIs where lambdas would have been genuinely useful. For example, the original STL did a lot with function objects. However, because you couldn't compose or lazy the algorithms, and iterators seriously suck, adding lambdas would not exactly have made them into LINQ. Really addressing these issues and making the C++ algorithms on par would have required a lot more work than just lambdas, a lot of which still hasn't been done. Compared to writing custom iterators and allocators of the time, and chaining together algorithms in-place on containers by hand, using a non-local struct functor was really not that bad.

Finally, it was unclear as to whether library techniques like expression templates could completely obviate the need for a language feature at all. It turns out that they can't, but there are other language features (non-placement new, delete and their array versions, native arrays) that are now not only completely redundant in C++11, but actively seriously harmful. Repeating these mistakes is undesirable. It's better to only resort to language features where you know for sure that a library can't handle it.

DeadMG
  • 36,794
  • 8
  • 70
  • 139
  • I fail to see how `new` and `delete` are *redundant*. Are you saying that because of smart pointers (which need some means of dynamic memory allocation)? Are saying that because of things like `malloc` and `free`? I suppose you can mimic ordinary `new` with malloc and placement `new` (oops... still using new). Otherwise, good answer. – Thomas Eding Dec 24 '14 at 16:39
  • 2
    @ThomasEding: Placement `new` is a different bucket of fish. But regular `new` and `delete` (and their array versions) are nothing more than trivial error-inducing wrappers over `malloc` and `free` you could implement as a library. Preferably a Standard-library internal function that was never exposed to users. – DeadMG Dec 24 '14 at 18:19
  • Perhaps I'm just being daft today, but I don't know how you can write a general purpose library function (using C++ code... as opposed to a compiler primitive) to initialize the memory properly and call the constructor without placement new. And that's even ignoring (single & multiple) inheritance initialization concerns. I'd love to see the code for it if it really is possible to do 100% correctly. – Thomas Eding Dec 24 '14 at 18:24
  • Like I said, placement `new` is a different bucket of fish. – DeadMG Dec 26 '14 at 12:20
6

What a closure does, in essence, is abstract over the lifetime of variables.

Manual memory management OTOH, requires the programmer to be aware of the lifetime of variables. But, how can you be aware of something that is abstracted away from you?

Closures are fundamentally incompatible with manual memory management. Reconciling the two is very hard, and the C++ community has done the best job they can, but actually, you can still get errors (Or maybe even UB? I admit I don't actually know C++.) using closures that capture variables which are then freed.

Or, put another way: closures extend the dynamic scope of variables beyond their lexical scope, but the fact that dynamic scope and lexical scope are identical is the fundamental basis of C++ idioms such as RAII.

C++ was AFAIK the first language with manual memory management to do closures, so they were doing original research and original research takes time. That's the main reason why it took so long. They were doing this research in a committee, which is about the worst place to do research, which probably prolonged the process even more, but the main reason is that they were doing hard original research, not that they were doing so in a committee.

Jörg W Mittag
  • 101,921
  • 24
  • 218
  • 318
  • 3
    RAII is not manual memory management. – DeadMG Dec 24 '14 at 14:03
  • I disagree with `is an abstract over the *lifetime* of variables` (emphasis mine). I would say that a closure is an abstract over the *scope* of variables. Not really a fan of this answer. – Thomas Eding Dec 24 '14 at 16:43
  • 2
    @ThomasEding I don't know what you mean by that. Jorg is right in that a closure forces the variables that were closed over to stay alive for as long as the function does (as opposed to until the end of their scope). The variables remain inaccessible outside of their scope so I don't see how a closure abstracts over their scope. – Doval Dec 24 '14 at 19:12
  • 2
    @Doval: "a closure forces the variables that were closed over to stay alive for as long as the function does": This is true in most languages: you can capture a variable from its lexical scope and keep it alive as long as the closure lives. In C++ it is different: if you capture a stack variable by reference and invoke the closure after you have exited the parent function, you get a crash. So, with C++ you cannot do common FP tricks that involve passing closures around at will: you can easily cause a core dump if you are not careful enough. – Giorgio Dec 24 '14 at 20:45
  • @Jörg W Mittag: I am not sure if they did much research related to RAII and closures. AFAIK, in C++ a closure works somewhat like an object that captures variables passed to its constructor and assigns them to its fields. If such variables are passed by reference, then the fields will point to invalid memory locations as soon as these variables have been disposed of. Then, invoking a method on that object that accesses those fields can crash the program. A closure that captures expired variables will have a similar behaviour. IMO the safest approach it to capture only smart pointers. – Giorgio Dec 24 '14 at 20:54
  • @Doval, the whole point of a closure is to lift the bindings of variables into nested functions. Hence I view them as a scope feature. The rest is an orthogonal feature of the memory model. – Thomas Eding Dec 25 '14 at 06:42
  • @ThomasEding: Maybe the concept of a closure has been adapted lately to accommodate the C++ view. In the commonly accepted notion a closure extends the lifetime of captured variables. See e.g. the discussion under "Implementation and theory" in https://en.wikipedia.org/wiki/Closure_(computer_programming) – Giorgio Dec 25 '14 at 11:29
4

C++ did not have an ISO standard until well after it was first created. Much of its early life was fragmented, with compilers all working differently to the point of being source-incompatible at times.

The first C++ standard was C++98, with relatively minor updates in the C++03 standard. The primary goal of these standards was to take the "wild west" of C++ and bring some semblance of order to it. Once the dust settled and developers got over the changes, the C++ committee would focus on new features.

From Bjarne Stroustrup himself (some parenthesis removed for brevity), The C++ Programming Language (4th Edition), section 1.4.4 "The 2011 Standard":

The current C++, C++11, known for years as C++0x, is the work of the members of WG21. The committee worked under increasingly onerous self-imposed processes and procedures. These processes probably led to a better (and more rigorous) specification, but they also limited innovation. An initial draft standard for public review was produced in 2009. The second ISO C++ standard was ratified by a 21-0 national vote in August 2011.

One reason for the long gap between the two standards is that most members of the committee (including me) were under the mistaken impression that the ISO rules required a "waiting period" after a standard was issued before working on new features. Consequently, serious work on new language features did not start until 2002.

C++ was not even standardized until 1998, and the next few years were spent fixing errors in the existing (new) standard. Then the committee sat on its hands for a couple of years due to procedural issues that were likely unnecessary. It then took seven years from initial inception to a public document, with much arguing and editing along the way. Even then, it took another two years until the document was in a position to be voted on and ratified.

What this means is the addition of lambdas and other language features had to wait until the status quo was stabilized (1998/2003), then politics and procedures of the process dragged out the addition of new features far longer than it probably needed to (2011).

Another reason for the lack of urgency is that functors can do almost anything a lambda can do, they are just more verbose. The language as a whole was not missing much other than the brevity of lambdas and their ability to bind to local variables. However, even that can mostly be worked around by passing references into the functor.

  • The usefulness of lambdas was well-known, and lambdas in widespread use even back when C++ was designed by a single person, and they also weren't added back then. I'm not sure you can blame it on design-by-committee. – Jörg W Mittag Dec 24 '14 at 00:18
  • Design by committee explains why it took so long from the original standard in 1998. Prior to that, the reason was fragmentation in the compilers. Many compilers worked differently in basic things like templates, and it took a long time before the fragmentation was fixed and lambdas and other features could be added. In other words, the C++ community was fixing its existing problems before adding new ones. –  Dec 24 '14 at 00:33
  • Well, but C++ also didn't get them when there was only one compiler. Or even before there was a compiler at all. It's not like lambdas were some newfangled esoteric thing at that time. Lisp had them since 1957, Smalltalk since 1972, the "Lambda: the Ultimate …" series of papers was published 1975-1977, all this happened *years* before *C with Classes* and over a decade before C++. Lambdas in a language without automatic dynamic memory management are just genuinely hard, and AFAIK C++ was the first one to do it. They were doing original research, and that takes time. – Jörg W Mittag Dec 24 '14 at 00:46
  • Granted, a language design committee is about the worst place to do original PLT research (just ask the ECMAScript 4 committee how *that* turned out), so that didn't help at all, but even if they hadn't had the committee it would still have taken a long time … just a slightly shorter long time. – Jörg W Mittag Dec 24 '14 at 00:47
  • 1
    @JörgWMittag: Implementing lambdas in a language with no garbage collection isn't particularly difficult, and Delphi had it in their 2008 release, several years before the C++11 standard even came out, much less got officially supported in compilers. The problem isn't the lambdas; it's that C++ is a horrible language, poorly designed and poorly implemented, and was too messy to implement a feature like this on top of without a lot of very careful effort. – Mason Wheeler Dec 24 '14 at 00:58
  • The vast majority of C++ features do not interact with lambdas in any way. I'm looking at Delphi lambdas right now and they're very limited compared to C++'s lambdas. – DeadMG Dec 24 '14 at 14:02
  • @DeadMG: C++ lambdas are more general than Delphi's but also less safe: they can crash the program if invoked too late, which is against the common intuition closures have in most other programming languages. But without a garbage collector in place you either have unsafe closures (C++) or restricted ones (Delphi). See also http://programmers.stackexchange.com/questions/189856/is-garbage-collection-needed-for-implementing-safe-closures – Giorgio Dec 24 '14 at 20:43
0

What's the difference between a functor and a lambda? The way the code you write looks!

A lambda is just a functor written in-line so to avoid a bit of boring boilerplate (replacing it with less boilerplate). So asking why C++ didn't have this feature is missing the point of what the feature really is. What extra functionality do you gain from a lamdba that you didn't get from a functor?

So apart from a short-hand syntax, there's nothing new to see here. (and remember some shorthand syntax is bad, take a look at some perl sometime and tell me shortening language features so they're easy and quick to type is a good thing :-) )

gbjbaanb
  • 48,354
  • 6
  • 102
  • 172