43

Case: I'm working at a company, writing an application in Python that is handling a lot of data in arrays. I'm the only developer of this program at the moment, but it will probably be used/modified/extended in the future (1-3 years) by some other programmer, at this moment unknown to me. I will probably not be there directly to help then, but maybe give some support via email if I have time for it.

So, as a developer who has learned functional programming (Haskell), I tend to solve, for example, filtering like this:

filtered = filter(lambda item: included(item.time, dur), measures)

The rest of the code is OO, it's just some small cases where I want to solve it like this, because it is much simpler and more beautiful according to me.

Question: Is it OK today to write code like this?

  • How does a developer that hasn't written/learned FP react to code like this?
  • Is it readable?
  • Modifiable?

  • Should I write documentation like explaining to a child what the line does?

     # Filter out the items from measures for which included(item.time, dur) != True

I have asked my boss, and he just says "FP is black magic, but if it works and is the most efficient solution, then it's OK to use it."

What is your opinion on this? As a non-FP programmer, how do you react to the code? Is the code "googable" so you can understand what it does? I would love feedback on this.

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
kd35a
  • 533
  • 1
  • 4
  • 7
  • 14
    In my opinion many modern languages allow *some* degree of functional programming and *using* that becomes more and more common. Personally I'd say go for it (at least for the relatively simple filtering/mapping tasks). – Joachim Sauer Jul 30 '12 at 10:16
  • I can't give you an unaffected answer, because I like FP. It would be a good idea to add a comment, if it is beyond simple though. – Bruno Schäpper Jul 30 '12 at 10:17
  • 3
    Your comment could be clearer; filter includes elements for which the test is True, so your comment could read: `# Select the item's from measures for which included(item.time, dur) == True`, avoiding a double negative always improves understanding. – Martijn Pieters Jul 30 '12 at 10:26
  • 7
    Have you considered using list comprehensions instead? They are often regarded as more "pythonic" than map/filter. – phant0m Jul 30 '12 at 10:31
  • Lucky you. My boss once told me to change my few filter and map functions in PHP back to double/triple nested for loops with countless if-then-else statements... :) – Matjaz Muhic Jul 31 '12 at 11:08
  • 2
    @MatjazMuhic Well, that's beautiful... ;) – kd35a Jul 31 '12 at 14:29
  • "Black magic bars our way but the will of a programer is stronger !" – Arkh Aug 21 '12 at 10:15
  • I like how you act like no one else will be able to understand such trivial code. Does this code count as amazing FP wizardry in your book?? – Josh Oct 27 '18 at 07:44

7 Answers7

50

Is it readable?

For me: Yes, but I have come to understand, that the Python community often seems to consider list comprehensions a cleaner solution than using map()/filter().

In fact, GvR even considered dropping those functions altogether.

Consider this:

filtered = [item for item in measures if included(item.time, dur)]

Further, this has the benefit that a list comprehension will always return a list. map() and filter() on the other hand will return an iterator in Python 3.

Note: If you want to have an iterator instead, it's as simple as replacing [] with ():

filtered = (item for item in measures if included(item.time, dur))

To be honest, I see little to no reason for using map() or filter() in Python.

Is it Modifiable?

Yes, certainly, however, there is one thing to make that easier: Make it a function, not a lambda.

def is_included(item):
    return included(item.time, dur)
filtered = filter(is_included, measures)
filtered = [item for item in measures if is_included(item)]

If your condition becomes more complex, this will scale much easier, also, it allows you to reuse your check. (Note that you can create functions inside other functions, this can keep it closer to the place where it is used.)

How does a developer that hasn't written/learned FP react on code like this?

He googles for the Python documentation and knows how it works five minutes later. Otherwise, he shouldn't be programming in Python.

map() and filter() are extremely simple. It's not like you're asking them to understand monads. That's why I don't think you need to write such comments. Use good variable and function names, then the code is almost self explanatory. You can't anticipate which language features a developer doesn't know. For all you know, the next developer might not know what a dictionary is.

What we don't understand is usually not readable for us. Thus, you could argue it's no less readable than a list comprehension if you've never seen either of them before. But as Joshua mentioned in his comment, I too believe it's important to be consistent with what other developers use - at least if the alternative provides no substantial advantage.

phant0m
  • 2,614
  • 16
  • 25
  • 5
    It might be worth also mentioning [generator expressions](http://www.python.org/dev/peps/pep-0289/), which work the same as list comprehensions but return generators instead of lists, as `map` and `filter` do in python 3. – James Jul 30 '12 at 11:38
  • @James: Great idea, I have added them to my post. Thanks! – phant0m Jul 30 '12 at 11:48
  • 2
    I usually bring up `reduce` as a counter-example to the *you can always use a list comprehension* argument, as it's relatively difficult to replace `reduce` with an inline generator or list comprehension. – kojiro Jul 30 '12 at 12:59
  • @kojiro: I don't see how it is a counter example. A list comprehension is used to create a list, `reduce` consumes a list and gives you a single element, not a list. They are very different. – phant0m Jul 30 '12 at 13:18
  • But sure, it is a good example how expressive FP is. – phant0m Jul 30 '12 at 13:24
  • 4
    +1 for noting the preferred idiom of the language in question. IMHO consistency has tremendous value and using a language in the manner that a significant portion of the "speakers" do can provide more maintainability than even comments at times. – Joshua Drake Jul 30 '12 at 13:38
  • 4
    +1 for "He googles for the Python documentation and knows how it works five minutes later. Otherwise, he shouldn't be programming in Python." – Bjarke Freund-Hansen Aug 21 '12 at 10:51
25

Since developer community is regaining interest in functional programming, it is not unusual to see some functional programming in languages which were originally fully object oriented. A good example is C#, where anonymous types and lambda expressions enable to be much shorter and expressive through functional programming.

This being said, functional programming is weird for beginners. For example, when during a training course, I explained to the beginners how they can enhance their code through functional programming in C#, some of them were not convinced, and some said that the original code was easier to understand for them. The example I gave to them was the following:

Code before refactoring: 

var categorizedProducts = new Dictionary<string, List<Product>>();

// Get only enabled products, filtering the disabled ones, and group them by categories.
foreach (var product in this.Data.Products)
{
    if (product.IsEnabled)
    {
        if (!categorizedProducts.ContainsKey(product.Category))
        {
            // The category is missing. Create one.
            categorizedProducts.Add(product.Category, new List<Product>());
        }

        categorizedProducts[product.Category].Add(product);
    }
}

// Walk through the categories.
foreach (var productsInCategory in categorizedProducts)
{
    var minimumPrice = double.MaxValue;
    var maximumPrice = double.MinValue;

    // Walk through the products in a category to search for the maximum and minimum prices.
    foreach (var product in productsInCategory.Value)
    {
        if (product.Price < minimumPrice)
        {
            minimumPrice = product.Price;
        }

        if (product.Price > maximumPrice)
        {
            maximumPrice = product.Price;
        }
    }

    yield return new PricesPerCategory(category: productsInCategory.Key, minimum: minimumPrice, maximum: maximumPrice);
}

Same code after refactoring by using FP:

return this.Data.Products
    .Where(product => product.IsEnabled)
    .GroupBy(product => product.Category)
    .Select(productsInCategory => new PricesPerCategory(
              category: productsInCategory.Key, 
              minimum:  productsInCategory.Value.Min(product => product.Price), 
              maximum:  productsInCategory.Value.Max(product => product.Price))
    );

This make me think that:

  • you should not worry if you know that the next developer who will maintain your code will have enough overall experience and some knowledge of functional programming, but:

  • otherwise, either avoid functional programming, or put a verbose comment explaining at the same time the syntax, the advantages and the possible caveats of your approach versus a non-functional programming one.

The Guy with The Hat
  • 512
  • 4
  • 12
Arseni Mourzenko
  • 134,780
  • 31
  • 343
  • 513
  • 8
    +1 if FP doesn't make your code more readable to *anyone*, you're doing it wrong. – György Andrasek Aug 21 '12 at 11:06
  • I wanted to post this as comment to your post in the readability topic but your answer was moved. – Esailija Apr 09 '13 at 13:15
  • 7
    I can see how in isolation someone never seeing FP before could consider the former snippet more readable over the latter. But what if we multiple this by 100? Reading the 100 short succinct almost English-like sentences describing the *what* vs thousands of lines of plumbing *how* code. The sheer volume of code, no matter how familiar, should be so overwhelming that the familiarity is not such a big win anymore. At some point it must be easier for anyone to first get comfortable with elements of FP and then read a handful of lines vs reading thousands of lines of familiar code. – Esailija Apr 09 '13 at 13:15
  • Or does familiarity win every time? – Esailija Apr 09 '13 at 13:15
  • @Esailija: I suspect familiarity wins. Probably what scares people the most is that if they are not familiar to some language constructs, they are scared by potential mistakes they can do. FP is not easy, and some basic assumptions won't work. As an example, [see my other answer](http://programmers.stackexchange.com/a/178225/6605) about a wrong assumption that `for` and `foreach` are very close. – Arseni Mourzenko Apr 09 '13 at 13:31
  • 7
    What bothers me most is that we're content to believe it's acceptable for developers not to learn functional style. The advantages have been proven many times over, the paradigm is supported by mainstream languages. If people lack the capability to understand functional techniques, they should be taught or should teach themselves, even if they don't intend to use it. I hate XSLT with a burning passion, but that hasn't stopped me from learning it, as a professional. – Sprague May 15 '13 at 11:50
  • @Sprague: it doesn't bother you that so much people actually call themselves developers and are hired, while they don't know even basics of software development? FP is difficult. Yes, it's useful, but not everyone can start programming in Haskell after a few days of learning. One may hope that over time, FP will be used more and more, but one should also note that FP is more than sixty years old: much older than most developers out there. – Arseni Mourzenko May 15 '13 at 18:15
  • 2
    @MainMa Your point is completely valid, I should have been more specific. What I mean is that a developer in any given language should be expected to use the features of those languages effectively. For example, using 'functional style' in C# (using filter/map/reduce style programming, or passing/returning functions) is nothing new, and so I think that any C# developers which aren't capable of reading such statements should update their skills... probably very quickly. I certainly think every developer should learn a little Haskell, but only for academic reasons. This is a different thing. – Sprague May 16 '13 at 09:25
  • I would also like to point out that I wasn't intending to imply that FP is new, but it has become more mainstream as of late (at least for enterprise type development and for server-side web development.) I think we basically agree in what we're saying! I just hate to see the bar lowered on our profession. – Sprague May 16 '13 at 09:33
  • 2
    @Sprague: I get your point and agree with it. It's just that we can hardly expect for example C# programmers starting using paradigms from FP, when too many of those programmers don't even use generics. Until there are so many unskilled programmers, the bar on our profession will be low, especially since most persons without technical background don't understand at all what development is about. – Arseni Mourzenko May 16 '13 at 11:54
  • I suppose the beginners in your course were similarly scared by SQL? Because the FP version of your example is very similar to SQL. I also suppose that even the people who claimed they were more comfortable with the more verbose imperative version, also made *more mistakes* when writing in this style: plenty of things to get wrong, even when they think they understand it better. – Andres F. Oct 26 '18 at 19:37
20

I am a non-FP programmer and recently I was to modify my colleague's code in JavaScript. There was an Http-request with a callback which looked a lot like the statement included by you. I must say that it took me some time (like half an hour) to figure it all out (the code all in all wasn't very large).

There were no comments, and I think there was no necessity for any. I didn't even ask my colleague to help me understand his code.

Taking into account that I am working for about 1.5 years, I think most programmers will be able to understand and modify such code, since I did.

Besides, as Joachim Sauer said in his comment, there often are pieces of FP in many languages, like C# (indexOf, for instance). So many non-FP programmers deal with this quite often, and the code snippet you included isn't something dreadful or incomprehensible.

superM
  • 7,363
  • 4
  • 29
  • 38
3

I would say definitely yes!

There are many aspects of functional programming, and using higher-order functions, as in your example, is only one of them.

For example, I consider writing pure functions to be extremely important for any software written in any language (where by "pure" I mean no side-effects), because:

  • they're easier to unit test
  • they're much more composable than side-effecting functions
  • they're easier to debug

I also often avoid mutating values and variables -- another concept borrowed from FP.

Both of these techniques work fine in Python and other languages that aren't commonly classified as functional. They are often even supported by the language itself (i.e. final variables in Java). Thus, future maintenance programmers won't face a humongous barrier to understanding the code.

2

We had this same discussion on a company I worked for last year.

The discussion concerned "magical code" and if it was to be encouraged or not. When looking into it a bit more it seemed that people had very different views on what actually was "magical code". The ones who brought up the discussion seemed to mostly mean that expressions (in PHP) which used functional-style was "magical code" whereas developers who originated from other languages who used more FP style in their code seem to think that magical code was rather when you made dynamic inclusion of files through filenames and so on.

We never came to any good conclusion on this, more than that people mostly think code that looks unfamiliar is "magical" or hard to read. So is it a good idea to avoid code that seems unfamiliar to other users? I think that it depends on where it is used. I would refrain from using fp-style expressions (dynamic file inclusion and so on) in a main method (or important central parts of applications) where data should be tunnel in a clear and easy-to-read, intuitive, way. On the other hand, I do not think one should be afraid to push the envelope, the other developers will probably learn FP quickly if they are faced with FP code and maybe have some good in-house resource to consult on the issues.

TL;DR: Avoid in high level central part of the applications (which need to be read for an overview of the functionality of the application). Otherwise use it.

Christopher Käck
  • 121
  • 2
  • Good point in avoid using it in higher level code! – kd35a Jul 30 '12 at 10:51
  • I've always thought there's a lack of formalism, and very much opportunity, in defining techniques of programming in different types of blocks(such as static methods, public methods, constructors, etc.) Good answer... it's making me revisit some of those thoughts. – Sprague May 15 '13 at 11:59
2

The C++ community recently got lambda's as well, and I believe they have roughly the same question. The answer may not be the same, though. The C++ equivalent would be:

std::copy_if(measures.begin(), measures.end(), inserter(filter),
  [dur](Item i) { return included(i, dur) } );

Now std::copy isn't new, and the _if variants aren't new either, but the lambda is. Yet it's defined rather clearly in context: dur is captured and therefore constant, Item i varies in the loop, and the single return statement does all the work.

This looks acceptable to many C++ developers. I haven't sampled opinions on higher-order lambda's, though, and I would expect much less acceptance.

MSalters
  • 8,692
  • 1
  • 20
  • 32
  • Interesting point, that there can be different answers depending on which language it is in. Probably related to [@Christopher Käck post](http://programmers.stackexchange.com/a/158724/60447) about how PHP-coders had more problem with this kind of stuff than Python-coders. – kd35a Jul 30 '12 at 13:34
0

Post a code snippet to a fellow dev that isn't as fluent in python, and ask him if he can spend 5 minutes checking out the code to see if he/she understands it.

If yes, you are probably good to go. If not, you should look at making it clearer.

Could your colleague be daft and not understand something that should be obvious? Yes, but you should always program in accordance with KISS.

Maybe your code is more efficient/good looking/elegant than a more straightforward, idiot-proof approach? Then you need to ask yourself: do I need to do this? Again, if the answer is no, then don't do it!

If after all this, you still think you need and want to do it the FP way, then by all means do it. Trust your instincts, they are better suited for your needs than most people on any forum :)

Arnab Datta
  • 191
  • 8