19

I've recently been delving into functional programming especially Haskell and F#, the prior more so. After some googling around I could not find a benchmark comparison of the more prominent functional languages (Scala,F# etc).

I know it's not necessarily fair to some of the languages (Scala comes to mind) given that they are hybrids, but I just wanna know which outperforms which on what operations and overall.

Farouk
  • 309
  • 1
  • 2
  • 5
  • 21
    Languages are not fast or slow, implementations are. – starblue Dec 16 '12 at 08:10
  • 7
    Language implementations are not fast or slow, programs run using those language implementations are (when compared to some other programs). Be charitable -- when someone talks about one programming language being faster than another, the obvious way that makes sense is to understand that they are talking about particular programs run using particular language implementations. – igouy Dec 16 '12 at 16:49
  • 4
    @starblue: That's a very glib answer, and not a very helpful one. While it's certainly possible to create two implementations of the same language, one of which is slower than the other, the way you put it there implies that there do not exist language design details that necessarily require certain inefficiencies in implementation that other languages, by their design, do not require. And that's simply not true. (Especially in this particular topic; functional languages are notorious for them!) – Mason Wheeler Dec 17 '12 at 02:28
  • 3
    @igouy "Language implementations are not fast or slow" This is not true. `CPython` vs `PyPy` quickly comes to mind. – NlightNFotis Dec 29 '12 at 19:58
  • @NlightNFotis -- How many seconds does CPython take? How many seconds does PyPy take? Language implementations are not fast or slow. How many seconds does that program take with CPython? – igouy Mar 05 '13 at 21:16
  • @igouy Their speed difference is not measured in seconds, for `pypy` is orders of magnitude faster than `CPython`. [You can see for yourself](http://speed.pypy.org/) – NlightNFotis Mar 06 '13 at 07:40
  • @NlightNFotis -- I can see for myself that "benchmark times" of *programs* are compared. (Incidentally, 5.4 is a difference of less than 1 order of magnitude.) – igouy Mar 20 '13 at 13:55
  • @starblue: Language design choices imply performance (or better, the performance of the best possible implementation). – Rok Kralj Aug 04 '14 at 15:21

3 Answers3

28

According the Great Benchmarks Game, ATS is faster than the rest with Haskell, Scala, and one of the variants of Common Lisp in a rough tie for speed close behind that. After that Ocaml and F# are in roughly the same speed category with Racket and Clojure lagging behind...

However, almost none of this means anything at all really. It's all a question of problem, machine, compiler, coding techniques, and in some cases, plain luck. Generally speaking, Directly machine coded languages like Haskell will outperform VM compiled languages like F# and vastly outperform purely interpreted languages. Also generally, Statically typed languages are faster than Dynamically typed due to static analysis allowing all type operations to be calculated at compile rather than run time. Again, these are general rules, there will always be exceptions. "Paradigms" have little to do with it.

LLPJaS4A
  • 5
  • 3
  • I know there are a lot of factors to take into consideration and even if all things were perfect they might perform differently on different data, my question is a pretty vague one. Thanks for the link, really helpful - I never knew ATS was that fast – Farouk Dec 16 '12 at 04:41
  • Nice link with detailed comparison information. I am bit disappointed to see Clojure using up much more memory and taking significantly longer than Java. I remember some claims about Clojure having a similar performance, which doesn't seem to be the case. – DPM Dec 16 '12 at 05:13
  • The ATS website states that ATS supports a variety of programming paradigms -- so before you claim ATS is faster than the rest you need to show that those programs are in fact written in a functional style. It might just be that functional ATS is not faster than the rest. – igouy Dec 16 '12 at 16:45
  • 2
    The Scala website states that Scala integrates object-oriented and functional features. Have you checked that the Scala programs are written as functional programs rather than object oriented programs? – igouy Dec 16 '12 at 16:57
14

It should also be pointed out that you cannot measure / quantify the performance of a programming language. The best you can do is measure the performance of a specific implementation of the language on specific platforms, running specific programs.

So when you ask about "the fastest functional language", what you are really asking about the best of the current implementations of the language(s).

@igouy's comment raises the point that there are other performance measures for language implementation; e.g. compilation time. But that doesn't change the fact that application program's run time is an (indirect) measure of a language's implementation, not a measure of the language itself.

Consider Java for example. Suppose I write a single-threaded benchmark using solely language features of classic (Java 1.0) Java. If I compile and run using JDK 1.0, I will get a poor performance ('cos JDK 1.0 didn't have a native code compiler). If I go from JDK 1.1 to ... JDK 1.7, I will most likely get progressively better results. But this is not due to changes to the Java language ... because my benchmark is using the same language subset. Rather the speedup is due to improvements in the compilers, the runtime system and / or the implementation of class libraries. These are all implementation issues.

The other point is that these implementation differences can be really significant (e.g. orders of magnitude) for the same language. So the fact that the best implementation for language X is faster than the best (or only) implementation of language Y doesn't necessarily tell you a lot about the language itself.

Stephen C
  • 25,180
  • 6
  • 64
  • 87
  • The best you can do is measure the performance of specific programs. When we measure the performance of a language implementation we want to find out how long it takes to compile some program, not how long that program takes to run. – igouy Dec 16 '12 at 17:02
  • The program's run time is a property of that particular program when run using that particular language implementation on that particular hardware etc etc. Given that the program's run time is not a property of the language, it's charitable to allow that in this context the language name is being used as shorthand for the usual well-known language implementations. – igouy Dec 17 '12 at 22:34
  • @igouy - that is true. But a lot of people don't make the distinction, as illustrated by lots of old websites proclaiming that "Java is slow. Unfortunately, this nonsense was read literally by a whole generation of programmers ... and it significantly damaged Java's reputation. THAT is why I am making this point HERE. – Stephen C Dec 18 '12 at 00:05
  • As you want to have the freedom to say - "an (indirect) measure of a language's implementation" - please explain why someone else shouldn't be free to say -- *an (indirect) measure of a language*. – igouy Mar 20 '13 at 14:01
  • @igouy - 1) You are free to say what ever you like. But that doesn't make you right. 2) Consider the case where the only implementation of a language is crap. We benchmark it. Then we update the implementation, benchmark it, and the performance has improved dramatically. Does that mean that the language performance has improved? How does that make sense ... given that the language has NOT changed!!! – Stephen C Mar 12 '14 at 22:59
  • As you agreed, the program's run time is not a property of the language - so "language performance" does not make sense. It's *charitable* to allow that in this context the language name is being used as shorthand for something else. – igouy Mar 28 '14 at 18:46
  • @igouy - Yes ... but being charitable is missing the point. We also need to be realistic. When someone (e.g. a manager) reads somewhere that "the Java language is slow", they won't understand that this is (or might be) a shorthand. They are more likely to interpret as meaning that all implementations of Java are slow. – Stephen C Dec 12 '16 at 23:40
  • You can, however say something about the language itself. For example the design of Python is such that no implementation of it can be faster than say any relevant implementation of C. For example weak typing (virtual dispatching everywhere) is a feature built into language itself and is a real obstacle to many optimizations. Java's "everything is an object" is also such obstacle: heap allocated, requires garbage collection. These are overheads built into language itself. – freakish Jul 04 '23 at 17:51
9

If you are looking at languages only on speed of execution you are missing some major points. Speed is a good thing, but it is not the only thing.

Haskell uses a very robust type system to create programs that will be much more likely to be bug free and robust.

Erlang uses its built in monitoring system to allow you to create fault systems that can give you a huge level of reliability in the face of various types of faults. In addition Erlang can give you a level of concurrency that is hard to match in other languages.

In truth I would consider speed of execution in the modern day to be rather far down on the list of what I would consider in most cases. (OK if I was doing massive numeric computation I would probably want to use fortran for speed but otherwise Its just not important enough to matter)

Zachary K
  • 10,433
  • 2
  • 37
  • 55