3

In other words, is there a Python design related reason for it to be so?

Functions like map, filter, reduce etc. are just plain functions.

Is it just a poor design choice (if it is a good one, please explain)? For example in scala, you can chain collection methods like smth.map(func1).reduce(func2). It seems much more convenient to me.

Ivan
  • 141
  • 5
  • 2
    When you discover some new collection function, will you add it to all the collections? What'll happen if someone else tries to do the same and a name clash occurs? As for chaining...it's just syntax. – Doval Apr 23 '15 at 14:59
  • @Telastyn I checked it out; I am asking this because maybe there is a particular Python reason for it. Also it seems pretty natural: those functions are already returning collections. – Ivan Apr 23 '15 at 15:00
  • How this question is duplicate? It is much more concrete than the referenced one (why is it so _in python_?), and in the end I got the answer I consider useful. – Ivan Apr 25 '15 at 15:53

3 Answers3

2

I think this is just a historical artifact. These functions were introduced quite a while ago, when fluid interfaces were not all the rage. Since then everyone got used to them. (So yes, you can write it of as "bad design").

Could these functions could be retrofitted to lists? Possibly, but it was not and should not have been done.

First, There should be one-- and preferably only one --obvious way to do it.

Second, and most importantly, map, reduce and many other functions work not just with lists but with anything that's iterable. They work on tuples, sets, dicts, and, most importantly, any user-defined classes that implement iteration (via __iter__) or generators (via yield).

So, every iterable or generator should somehow receive their own implementations of map, reduce, and probably a bunch of other functions that accept an iterable / generator and return something compatible. This would require that any existing class that happen to be iterable not to expose names like map and reduce. Code-breaking changes are very much frowned-upon by Python maintainers and community alike.

Instead, you can wrap things into your own class that offers the additional interface, and enjoy either the fluid style you mention, or pipe style, or anything else.

9000
  • 24,162
  • 4
  • 51
  • 79
2

Which class would you put these methods on?

In python, I can use map on lists, tuples, dictionaries, files, strings, sets, arrays, etc. There is no collection base class to put a map,reduce,filter etc on.

Now, python could have had a collection class that all these different things inherited from. But that would really go against the "spirit" of python. In python everything is duck typed. Things work by virtue of the fact that you have the right methods. You can use map, reduce, and filter on anything that defines an __iter__. Having to subclass a collection class would go against that.

As it is, map/reduce/filter aren't really considered the pythonic solutions.

Instead of map(x, lambda y: y+1) use [y + 1 for y in x]

Instead of filter(x, lambda y: y % 2 == 0) use [y for y in x if y % 2 == 0]

Instead of reduce(x, lambda x,y: x+y) use 

sum = 0
for y in x:
   sum += y
Winston Ewert
  • 24,732
  • 12
  • 72
  • 103
  • 4
    `Instead of reduce(x, lambda x,y: x+y) use...` In what way is replacing 1 expression with 3 lines of *statements* - potentially introducing a useless intermediate variable - better? Yes, Python lambdas are clumsy, but there's nothing awkward or unwieldy about `reduce(x, add)`. Even if you have to define `add` yourself, it's still not longer than what you suggested, and you only ever have to define `add` once. The other two examples are just a matter of preference for notation. – Doval Apr 23 '15 at 15:43
  • @Doval: read the example again: the list comprehensions take about as much room as function-calling ones. For summation (actually, for everything that uses `+` as the operation) you can just use `sum` (e.g. `sum([(1, 2), (3, 4)], ())` would concatenate tuples). Yo can [`import operator`](https://docs.python.org/2/library/operator.html) if `reduce(int.__add__,...)` looks clumsy to you. Also, `reduce` isn't going anywhere. – 9000 Apr 23 '15 at 15:49
  • 1
    @9000 But I didn't say anything about how much room comprehensions take...? I didn't even say they're different in any way other than syntax. My main objection was against replacing `reduce` with a variable and `for` loop. – Doval Apr 23 '15 at 16:02
  • @Doval, see http://www.artima.com/weblogs/viewpost.jsp?thread=98196 for Guido's thoughts on reduce. He thinks that outside of a few trivial cases reduces are really hard to read and are better done as explicit for loops. – Winston Ewert Apr 23 '15 at 16:19
  • 1
    @WinstonEwert Guido also claims `[x for x in S if P(x)]` is almost always clearer than `filter(P, S)` even though I know people that much prefer the latter. Whether a loop is clearer than `reduce` depends on the problem and how much time you've spent thinking imperatively vs functionally, and who else is going to read the code. But the same holds for comprehensions! They're completely alien to some people. Guido has opinions, and that's fine, and it's also fine to agree with him, but to choose one construct over another *purely because he calls it pythonic* is cargo cult programming. – Doval Apr 23 '15 at 16:36
  • @Doval, who said anything about choosing one construct over another because Guido said so? I'm not actually personally a fan of comprehensions. But, the design of python and the community has gone in the direction of preferring them over the functional language tradition of map/reduce/filter. – Winston Ewert Apr 23 '15 at 16:43
  • 1
    @Doval: The thing is, most languages (both their authors and communities) are opinionated. Ruby and Python are pretty similar in many regards, but they hold very different opinions about the proper style. For instance, Go is highly opinionated, but so are Java, Scala, Haskell, etc. Also note that opinions are in many cases opinions, not hard limits; you can write highly procedural Java, highly functional Python, highly imperative Haskell, etc. It usually goes a bit against the grain of the language, but you can do it if you must, that is, when it has very definite upsides. – 9000 Apr 23 '15 at 16:46
  • "There is no collection base class to put a map,reduce,filter etc on.": Why do you think that such methods should be put in a base class? Each collection classes could implement these methods and they would just happen to be there when you need them (duck typing). – Giorgio Apr 23 '15 at 17:05
  • @Giorgio, you really want every class that I can iterate on to have seperate implementations of all those methods? – Winston Ewert Apr 23 '15 at 17:08
  • @WinstonEwert: What would be so bad about that? After all the existing function implementations have to discriminate between the different collections they are applied to in the same way as the corresponding methods would. – Giorgio Apr 23 '15 at 17:12
  • @Giorgio, no they don't. They all just call `__iter__` to iterate over the element and implement there particular operation on top of that. They don't have to reimplement their logic for every possible collection. – Winston Ewert Apr 23 '15 at 17:16
  • @WinstonEwert: Then each method can be a thin wrapper around this common function implementation using `__iter__`. – Giorgio Apr 23 '15 at 17:35
  • @Giorgio, you're missing the point. The point is that I can use `map`,`reduce`,`filter` on anything that defines `__iter__` if I can only use them on objects that have also decided to specifically add these methods I've lost a great deal of the power of duck typing. – Winston Ewert Apr 23 '15 at 17:41
  • Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/23119/discussion-between-giorgio-and-winston-ewert). – Giorgio Apr 23 '15 at 17:43
  • The for loop is actually not a better solution than reduce, the benefit of reduce is that it can be optimized (maybe sometime in the future) so that the reduction is performed in parallel (Clojure does this) whereas it would be difficult to parallelize a for loop with a shared external variable, whereas parallelizing reduce is merely a matter of changing the function's implementation without making any changes to existing code which uses reduce. – ALXGTV Jun 13 '15 at 13:51
  • @ALXGTV, none of that is likely to be relevant in python. – Winston Ewert Jun 17 '15 at 01:16
0

Some considerations regarding the second part of the question:

Is it just a poor design choice (if it is a good one, please explain)? For example in scala, you can chain collection methods like smth.map(func1).reduce(func2). It seems much more convenient to me.

I do not think that it is poor design: Python supports both procedural and object-oriented styles, and one of the two styles had to be picked for higher-order functions on collections. Ruby and Scala favour the object-oriented style and therefore they chose high-order methods.

Regarding the convenience of method chaining, you have a corresponding compact syntax in languages supporting currying, e.g. in Haskell you can write

(map func1 . filter func2) smthg

Probably you can do something similar in Python (see e.g. PyMonad) even though it would not be the standard functions.

Giorgio
  • 19,486
  • 16
  • 84
  • 135