61

I'm comparing two technologies in order to reach a recommendation for which one should be used by a company. Technology A's code is interpreted while technology B's code is compiled to machine code. In my comparison I state that tech B in general would have better performance since it doesn't have the additional overhead of the interpretation process. I also state that since a program could be written in many ways it is still possible a program written in tech A could outperform one written in tech B.

When I submitted this report for review, the reviewer stated that I offered no clear reason why in general the overhead of the interpretation process would be large enough that we could conclude that tech B's performance would be better.

So my question is can we ever say anything about the performance of compiled/interpreted technologies? If we can say compiled is generally faster then interpreted, how could I convince the reviewer of my point?

Paul D. Waite
  • 1,164
  • 14
  • 18
EpicSam
  • 870
  • 1
  • 6
  • 10
  • 2
    Possible duplicate of [When an interpreter executes code: Is there a "chain of interpretations" down to the lowest level?](http://softwareengineering.stackexchange.com/questions/231777/when-an-interpreter-executes-code-is-there-a-chain-of-interpretations-down-to) – gnat Jan 10 '17 at 12:43
  • 4
    see also [Understanding the differences: traditional interpreter, JIT compiler, JIT interpreter and AOT compiler](http://softwareengineering.stackexchange.com/questions/246094/understanding-the-differences-traditional-interpreter-jit-compiler-jit-interp) – gnat Jan 10 '17 at 12:44
  • @gnat While these links do not directly answer my question, including information from their answers to create a more detailed argument in my report might be just what I need to convince my reviewer. – EpicSam Jan 10 '17 at 13:02
  • 76
    Since this seems to be a practical problem and not an academic exercise, it might be advisable to not even attempt giving a general answer but actually evaluate technologies A and B. While a general answer probably cannot be given, it is certainly possible to evaluate the performance characteristics of two specific technologies, pointing out their relevance for the kind of applications your organization is interested in. – 5gon12eder Jan 10 '17 at 13:31
  • 26
    The question asks about generalities, but it seems your real problem is with the specifics of Technology A and B. While I think _in general_ interpreted languages _may_ be slower as a whole, that fact likely has absolutely no bearing on your specific technologies. It's quite possible that your specific interpreted B is faster than your compiled A regardless of the general trend. – Bryan Oakley Jan 10 '17 at 13:33
  • It is compiled/interpreted implementations of the same language or two different languages? For the *same* language it is reasonable to expect a compiled implementation is faster. For two different languages, you can't say anything. – JacquesB Jan 10 '17 at 13:34
  • 2
    @EpicSam -- You wrote "If we can say compiled is generally faster then interpreted". I would be nervous advocating this in a professional setting. Basically, the more people have reused other people's work the more likely it is that the "reusable work" will be better and lead to a faster execution time. For example, Java's earlier (1995ish) JVMs ran far slower than today's JVMs (2017). The impact of this maturity cannot be ignored. Well used interpreted languages will eventually get high speed JIT compilation..once that happens all bets about compiled vs. interpreted are off. – Ivan Jan 10 '17 at 13:41
  • 3
    @JacquesB: IronRuby was originally purely compiled. They later added an interpreter for performance. XRuby was a purely compiled implementation of Ruby on the Java platform. Its performance doesn't even come close to JRuby's, which is a mixed-mode implementation. PicoScheme is an AST-interpreted implementation of Scheme that competes very well with bytecode-interpreted, bytecode-JITted, and AOT-compiled implementations. – Jörg W Mittag Jan 10 '17 at 14:07
  • 1
    If they're that worried about it, why not test both languages in a way that is comparable to how they'll be used specifically in this project? – JeffO Jan 10 '17 at 14:54
  • 18
    Randomly pick an interpreted language and a compiled one, and bet a dollar that the compiled one is faster. Repeat enough times and you'll eventually come out far enough ahead to buy lunch at Subway. However, there are faster and more interesting ways for a competent programmer to make that much money. And Subway's not all that great anyhow. – Ed Plunkett Jan 10 '17 at 19:03
  • 23
    You seem to be contradicting yourself. On the one hand, you want to make a general statement about interpreted vs compiled languages. But on the other hand, you want to apply that general statement to a concrete scenario. But once you apply it to a concrete scenario, *it's not generalized anymore*. So even if you can make the case that interpreted languages are slower *in general*, your reviewer doesn't care about that. You're doing an analysis of two very specific technologies. That's literally the opposite of generalizing. – loneboat Jan 10 '17 at 19:42
  • 8
    Honestly, why the heck do you want to give a company-relevant recommendation for one of two technologies based first and foremost on performance? In special cases (like 3D high speed games programming), performance can be a relevant criterion, but for most real-world business programs, it should be the last thing you compare, not the first. – Doc Brown Jan 10 '17 at 22:33
  • 1
    afaik, the most general statement you can get away with is that compiled languages tend to provide more *consistent* memory usage/execution times/etc and offer more *control* over the memory usage/execution time/etc when you need it. Interpreted languages these days often can compete with the "raw performance" of compiled languages, but it's the consistency of and control over that performance that makes certain compiled languages so dominant in embedded systems, scientific computing, 3D game engines and so on. – Ixrec Jan 10 '17 at 22:53
  • @DocBrown for some throughput-oriented things, "performance" is the inverse of "cost". Make something use X% less CPU; handle the same traffic with X% fewer VM instances; get billed X% less :) – hobbs Jan 11 '17 at 07:58
  • 3
    I'd hazard to suggest that the only blanket statement you can reliably make about compiled and interpreted code is that the former is compiled and the latter is interpreted. But, to be honest, I'm not even 100% certain on *that* :-) –  Jan 11 '17 at 08:19
  • 1
    @hobbs: sure, but I don't see any indication in the question that the comparison and the fact the OP wants to use performance as a primary factor is caused by such a reason. However, I have too often seen people trying to use performance for such comparisons because of some superstitious, wrong believe it might become relevant for them in the future. That gives me a big warning sign whenever a person tries to make a technological "A or B" decision that way. – Doc Brown Jan 11 '17 at 08:21
  • 3
    ... I guess this is often just a manifestation of the [Dunning-Kruger effect](https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect). Performance is used as a wrong measure by software engineers because it is a technical aspect of a technology which seems to be measurable, and they had learned a lot about in during their CS course etc. Thus soft factors like the organizational aspects of a technology are overlooked, despite the fact they are often much more important . – Doc Brown Jan 11 '17 at 08:38
  • @paxdiablo You could argue that the CPU interprets the compiled code... but that's probably pushing it :P On the other hand, the newest Intels are so good at branch prediction that the overhead from the extra indirection in interpreted code is almost non-existent (for different scenarios, they measured a speed-up of between 5 and 20 *times*). And code that isn't compiled for a specific computer can be much slower anyway - most C++ 32-bit compilers still target 80s era CPUs for compatibility by default. And then you have dynamic code, which can be much cheaper interpreted... and... – Luaan Jan 11 '17 at 09:14
  • 1
    @Doc Performance was not the primary reason I argued for either technology in my report. The main reason why I re commanded the one I did was based on testability and maintainability. The performance part of the report was however the part that was rejected by the reviewer. And after reading all the comments and answers here I believe the reviewer was correct to do so. – EpicSam Jan 11 '17 at 09:21
  • 4
    @EpicSam: that sounds good. And don't get me wrong, when picking a new technology, it can be important to invest some thoughts into performance. I just would ask such a question in a form like *"do we know that none of those two technologies performs so badly that it can be ruled out immediately"* or *"will both be fast enough for our purposes"*. And if the answer is "yes", discussing a question like "which one is faster" is often obsolete. – Doc Brown Jan 11 '17 at 09:52
  • http://softwareengineering.stackexchange.com/q/340008/ – Robert Harvey Jan 11 '17 at 21:21
  • Hmm... I can think of at least one generalisation that can be made, but it's not actually helpful. Specifically, that it's easier for interpreted code to run slowly than it is for compiled code to run slowly, simply because the interpreter's overhead could divert resources from the interpreted code. This says nothing of actual value, as a good interpreter can easily outperform a bad compiler. It all comes down to the quality of the interpreter vs. the quality of the compiler, the only thing that's different is that the deck is stacked in one's favour more than the other's. – Justin Time - Reinstate Monica Jan 13 '17 at 08:57
  • **Please avoid extended discussions in comments. If you would like to discuss the question further then please visit our chat room. Thank you.** – maple_shaft Jan 13 '17 at 12:52

14 Answers14

109

No.

In general, the performance of a language implementation is primarily dependent on the amount of money, resources, manpower, research, engineering, and development spent on it.

And specifically, the performance of a particular program is primarily dependent on the amount of thought put into its algorithms.

There are some very fast interpreters out there, and some compilers that generate very slow code.

For example, one of the reasons Forth is still popular, is because in a lot of cases, an interpreted Forth program is faster than the equivalent compiled C program, while at the same time, the user program written in Forth plus the Forth interpreter written in C is smaller than the user program written in C.

Mutantoe
  • 103
  • 3
Jörg W Mittag
  • 101,921
  • 24
  • 218
  • 318
  • 12
    as far as I can tell [your own prior answer](http://softwareengineering.stackexchange.com/a/269878/31260) in a linked (possibly a duplicate) question covers these matters much better than this one – gnat Jan 10 '17 at 13:14
  • You answer reminds me of when Java added Just-in-Time Compilation. Early editions of Java (that didn't have as much time/money/engineering invested) were slower. But as more resources were poured in the performance difference btw Java and C++ shrank dramatically. Now the speed difference between java and C++ is pretty trivial (at least I think this is the case). – Ivan Jan 10 '17 at 13:27
  • 11
    If it's impossible to make general conclusions about performance and we can write a certain program in A that outpeforms a similar program in B, but we can also write a program in B that outperforms one written in A. We can not really conclude anything about the speed of language ever then? Anyway Forth looks interesting, I'll have to read more about it later. – EpicSam Jan 10 '17 at 13:35
  • 36
    @EpicSam: Correct. The very idea of "the speed of a language" is fundamentally non-sensical. – Jörg W Mittag Jan 10 '17 at 14:10
  • 1
    I've often thought that for some micros it would be useful to have a C compiler with modes to compile to either native code or a binary representation of a Forth-like language. On the PIC 18xx parts, for example, something like x=y+z; takes 12 bytes of code when using 16-bit values, or 24 bytes if using 32-bit values. A Forth-like language might be able to accomplish the same thing in 7-9 bytes in either case. Not as fast as the native code, but a lot more compact. – supercat Jan 10 '17 at 19:42
  • 23
    In general: be specific. – igouy Jan 10 '17 at 19:57
  • 19
    I assume by "...and some _very_ slow compilers" you meant the code they generate, not their speed of compilation. – martineau Jan 10 '17 at 20:47
  • Do you have any resource that would analyze the performance of Forth interpreter, I understand how a JIT could lead to faster assembly than an AOT compiler, but I'm surprised an interpreter could best assembly and it seems the Forth interpreter you have in mind does not JIT the code (or does it?). – Matthieu M. Jan 11 '17 at 09:05
  • 2
    @MatthieuM. There's a huge benefit you can get from targeting a specific architecture; that usually isn't done with native code (most C++ compilers still target 32-bit 80s CPUs by default for compatibility; it's better with 64-bit), but often is with interpreters. The key saving is that your code doesn't care - you only need one executable, and run it on the correct interpreter. And with modern Intel CPUs, interpreters can avoid most of the overhead of the required indirection. And just like JITters, interpreters can have more opportunities for runtime optimisation. – Luaan Jan 11 '17 at 09:21
  • 2
    @MatthieuM.: Forth is typically compiled to bytecode, then interpreted. You can also compile it, JIT compile it, etc. of course. I am talking about a bytecode compiler. Forth bytecode tends to be both extremely simple (meaning the interpreter is small) and compact (meaning the user program is small). E.g. a Forth interpreter for the PDP-11 can be written in 2(!!!) machine instructions, for the Intel 8080 in about 11 machine instructions. For x86, there are full-featured interpreters in about 100-200 machine instructions for a total code size of about 120-300 bytes. – Jörg W Mittag Jan 11 '17 at 12:31
  • 4
    A full operating system, including Forth compiler, bytecode interpreter, network, disk, keyboard, and graphics drivers, boot loader, text editor, IDE, and debugger clocks in at about 60K, most of which is actually sourcecode that is compiled on the fly (it would be even smaller if it were bytecode instead). That way, the interpreter fits into a single cache line, or maybe two or three, and the entire system, including interpreter, OS, drivers, and user programs fits into the L1 cache. On embedded systems, which have tight space constraints, this can make a huge performance difference. – Jörg W Mittag Jan 11 '17 at 12:37
  • 1
    @MatthieuM. The line between an interpreter and a compiler is becoming increasingly blurred. You see this in tracing interpreters like PyPy, which just-in-time compile the code they're executing by tracing the instructions that are executed during hot paths through their code, and then running compiler optimisations on representations of those instruction traces. – James_pic Jan 11 '17 at 14:29
  • 2
    @James_pic: Actually, PyPy is especially weird, since the code that is traced and JITted is *not* the user program, but the interpreter *while it interprets the user program*. That's how the magic "just write an interpreter for your language, and the JIT is provided by the framework for free" stuff works. Oracle's Truffle AST interpreter framework works in a similar fashion, and the Truffleized experimental version of Ruby has demonstrated cases where a pure Ruby version running on Truffleized JRuby outperforms a pure C extension on YARV. – Jörg W Mittag Jan 11 '17 at 14:31
  • "And specifically, the performance of a particular program is primarily dependent on the amount of thought put into its algorithms." -- or, how well matched the data structures implemented in the language's standard library are to the requirements of the task. I recall one "benchmark" that cropped up a year or so ago showing massive performance benefits of both javascript (using V8) and lua (using luajit) over a C++ implementation of the same task, which involved repeatedly looking up the same strings in an associative array. This was fast for V8 and luajit because ... – Periata Breatta Jan 11 '17 at 21:46
  • ... (1) they both used interned strings and could therefore compare them by address rather than having to do a character-by-character comparison, and (2) both have associative array implementations that are optimized for performance when the number of entries is small (because you need that for languages that support dynamic extension of objects, which is a feature of both javascript and lua) while the C++ implementation used the standard `map` template which is optimized for large numbers of entries (and for providing iteration sorted on the key, which neither V8 nor luajit support). – Periata Breatta Jan 11 '17 at 21:49
  • Obviously it was possible to implement a C++ program that performed as well, but such a program was significantly harder due to having to implement a suitable data structure by hand rather than relying on the standard features of the environment. – Periata Breatta Jan 11 '17 at 21:51
  • 4
    Are there some benchmarks or data available that shows that Forth can be faster than C? The benchmarks I could find showed otherwise. – Alexander Jan 12 '17 at 15:08
  • This link https://www.complang.tuwien.ac.at/forth/performance.html shows that C is faster than compiled forth and a lot faster than interpretted forth by a considerable margin for compute tasks. – rghome Jan 13 '17 at 14:34
  • @Jörg W Mittag: "I remember when…" PLEASE show some reason for us to believe you haven't just made that up. For example, show that "hand-optimized assembly" was actually included in the alioth shootout -- http://web.archive.org/web/20040814114838/http://shootout.alioth.debian.org/langs.php – igouy Jan 13 '17 at 17:15
  • @Jörg W Mittag: "I remember when…" For Heapsort the last element of the sorted array WAS PRINTED OUT. Incidentally the old Doug Bagley Haskell heapsort program was attributed to Julian Assange https://alioth.debian.org/scm/viewvc.php/shootout/bench/heapsort/heapsort.ghc?view=markup&revision=1.1&root=shootout – igouy Jan 13 '17 at 20:39
  • @Luaan: Modern C++ compilers *tune* for modern CPUs, even if they default to emitting code that's compatible with older CPUs. e.g. x86 C++ compilers don't use the `LOOP` instruction, because it's slow. In general the cost model defaults for loop unrolling / addressing modes / latency / multiple LEA + shift insns vs. one IMUL instruction are all tuned for modern CPUs. As far as actual instruction-set extensions, gcc and many other compilers for x86 assume P6 (1995) as a baseline, though. Even with no `-march=` option, gcc uses CMOV instructions. – Peter Cordes Jan 15 '17 at 07:43
  • @Luaan: I think it's pretty rare for `-march=native` to make a huge amount of difference, outside of auto-vectorization using newer SSE/AVX instruction sets. If you'd said CPUs from ~2000 instead of the 1980s, then I'd agree with you on that point, too. (It makes a difference, but often not a big difference, for x86 anyway. Modern out-of-order x86 CPUs aren't too picky about instruction scheduling, which is kind of the point since the CPU designers know that most people don't compile their own binaries for their own systems.) – Peter Cordes Jan 15 '17 at 07:52
80

Generalizations and specific scenarios are literally opposites.

You seem to be contradicting yourself. On the one hand, you want to make a general statement about interpreted vs compiled languages. But on the other hand, you want to apply that general statement to a concrete scenario involving Technology A and Technology B.

Once you apply something to a concrete scenario, it's not generalized anymore. So even if you can make the case that interpreted languages are slower in general, you're still not making your point. Your reviewer doesn't care about generalizations. You're doing an analysis of two very specific technologies. That's literally the opposite of generalizing.

loneboat
  • 961
  • 5
  • 11
37

As a rule of thumb, an interpreted program is about 2x–10x slower than writing the program in the interpreter's host language, with interpreters for more dynamic languages being slower. This is because the interpreted program has to do all the actual work, but additionally has the interpretation overhead.

Depending on the structure of the interpreter, there can be very significant differences. There are two contradictory schools of interpreter design: One says opcodes should be as small as possible so that they can optimized more easily, the other says opcodes should be as large as possible so that we do more work within the interpreter. When the structure of your program matches the philosophy of the interpreter, the overhead becomes negligible.

E.g. Perl is an interpreted language oriented towards text manipulation. An idiomatic Perl program doing text manipulation will not be much slower than a C program, and may even outperform the C program in some cases (possible because Perl uses a different string representation and has various text- and IO-related optimizations built in). However, doing number crunching in Perl is going to be unbearably slow. An increment ++x is a single assembly instruction, but involves multiple pointer traversals and branches for the Perl interpreter. I recently ported a CPU-bound Perl script to C++, and got a 7x–20x speedup, depending on the compiler optimization level.

Speaking about optimizations is important here, since a polished, optimized interpreter may reasonably outperform a non-optimizing naive compiler. Since creating an optimizing compiler is difficult and requires a lot of effort, it is unlikely that your “technology B” has this level of maturity.

(Note: the Computer Language Benchmarks Game site has a confusing structure, but once you reach the table of timings for one problem you'll notice that performance of various languages is all over the place – often, there isn't a clear performance boundary between compiled and interpreted solutions. The most important part of the site isn't the benchmark results, but the discussions on how difficult meaningful benchmarks are.)

When choosing a tech, performance of a language runtime in itself is completely irrelevant. It is more likely to be important that the tech meet some baseline constraints (our budget is $x, we must be able to deliver before yyyy-mm-dd, we must satisfy various non-functional requirements), and that it has a lower total cost of ownership (factoring in developer productivity, hardware costs, business opportunity costs, risk of bugs and unexpected constraints in the tech, maintenance costs, training and hiring costs, …). E.g. in an industry where time-to-market is the most important factor, the tech with the best developer productivity would be the best fit. For a large organization, maintainability and long-term costs may be more interesting.

amon
  • 132,749
  • 27
  • 279
  • 375
  • 10
    If you think "the Computer Language Benchmarks Game site has a confusing structure" then give a URL to the specific page that supports your point, rather than expecting people to search from the top-level looking for something they might never find ! Show; don't just tell. – igouy Jan 10 '17 at 20:01
  • 8
    If you think "The most important part of the site isn't the benchmark results, but the discussions on how difficult meaningful benchmarks are" then give URLs to those pages. Show; don't just tell. – igouy Jan 10 '17 at 20:03
  • @igouy Thanks for your suggestions. However, I don't think expanding on a footnote would significantly improve my answer and/or make it easier to read. For your convenience, you can look at the [k-nucleotide benchmark](https://benchmarksgame.alioth.debian.org/u64q/performance.php?test=knucleotide) which is hash-table heavy. The best scores are the usual suspects C, C++, Java, C#, Go, Rust etc.. But PHP comes in at only 6.9x slower than C, which outperforms some solutions in Go, Rust, OCaml, Java, C#, …. – amon Jan 10 '17 at 20:40
  • 4
    It would be for the convenience of those who read your answer. (I'm just someone who admins the benchmarks game). – igouy Jan 10 '17 at 21:59
  • 2
    @amon linking the k-nucleotide example in the answer would make it more readable and the note more useful, as would linking the discussions (without reading the entire site, it's not obvious to me what discussions you're talking about). – David Moles Jan 10 '17 at 22:12
  • 1
    Where on earth did you get 2-10X from? The ratio depends entirely on how long it takes to fetch and dispatch to the handler for each bytecode as against how long each handler takes to execute, and 10x would only be possible if the fetch cycle took 9x as long as the handler, which is wildly improbably. More usually the execution is dominated by the handler. and the fetch cycle can be vanishingly insignificant. – user207421 Jan 11 '17 at 20:21
  • 1
    @EJP Dispatch can be very cheap, though it possibly has adverse effects regarding caching & branch prediction. But the interpreter isn't just doing the same work as a compiled program: different semantics and different memory models drag down performance. When every value is passed by pointer, every number and boolean is boxed, and each function call involves a hash table lookup, then 10x is a fairly optimistic estimate – but one that is supported by the data (e.g. see the benchmark links above). The 2x estimate assumes no semantic differences to the host language. – amon Jan 11 '17 at 20:55
  • And then you have an optimizing JIT like for Java and Lua that produces better performance than your typical C-compiler... – Martin Schröder Jan 12 '17 at 13:41
  • @ejp where does he get 2–10× : that’s what textbooks have stated for generations now. It might be out of date now that memory is a bottleneck etc. – JDługosz Jan 13 '17 at 17:08
  • 1
    @MartinSchröder You may, but one can argue that JIT is some sort of compiler and thus it is not interpreted anymore. – TomTom Jan 14 '17 at 19:29
18

You absolutely can say something about the performance of compiled/interpreted technologies. But first, you must define "performance". If you're building a computationally simple embedded system, then "performance" would likely lean towards the memory usage side of things. Whereas a computationally complex system operating on large data sets would find itself defining "performance" in the number of computations per unit time since the memory overhead from JVM or .NET would be negligible.

Once you decide on what "performance" is, then you can say something like "we will have 50 billion objects in memory at any given time and interpreted techA adds an additional 8 bytes to each object for internal management which equates to a 400GB memory overhead in comparison to techB which doesnt add this data"

jhbh
  • 289
  • 2
  • 4
12

This is a technical question and you already got many good technical answers, but I'd like to point out a slightly different aspect of your situation: the fact that you can't just base a decision like "technology A or technology B" purely on technical and/or performance reasons.

The technical aspects of something like this is just a small part of the decision between A and B. There are dozens of other factors to keep in mind:

  • does it involve any licensing costs? For example: you have to pay (a substantial amount) to use a cluster of SQL Server machines vs. using a cluster of PostgreSQL machines.
  • do I have employees who are familiar with that technology (stack) and its ecosystem? If yes, are they available? If no, can I hire some? How much will it cost me? Or do I train the existing ones? How much will that cost me?
  • what does the client want? This can be an issue a lot of the time.
  • is the technology I recommend ready for production use? Or is it just a current hype that maybe will die out? (e.g. think about Node.js when it first came out)
  • does the technology I recommend fit well with the existing architecture or the architecture I had in mind? Or do I have to spend a lot of money by making them work together seamlessly?
  • and many many more aspects that depend on your specific situation.

As you can see, there is A TON of stuff to consider when making such a decision.

I know this doesn't specifically answer your question, but I think it brings a more big-picture view over your situation and the specifics of such a decision.

Uyghur Lives Matter
  • 126
  • 1
  • 1
  • 10
Radu Murzea
  • 1,810
  • 2
  • 18
  • 24
10

Partial evaluation is a conceptual framework relevant to relate interpreters and compilers.

Can we make general statements about the performance of interpreted code vs compiled code?

Programming languages are specifications (written in some report, such as R5RS or n1570). They are not software, so it does not even make sense to speak of performance. But some programming language may have several implementations, including interpreters and compilers.

Even in traditionally compiled languages (that is, languages whose implementations are often compilers) like C, some parts are often interpreted. For example, the format control string of printf (defined in the C standard) is often "interpreted" (by the C standard library, which has a printf function using variable arguments techniques) but some compilers (including GCC) are able -in limited specific cases- to optimize it and "compile" it into lower-level calls.

And some implementations, even within "interpreters", are using JIT compilation techniques (so generate machine code at runtime). A good exemple is luajit. Other implementations (e.g. Python, Ocaml, Java, Parrot, Lua) are translating the source code into a bytecode which is then interpreted.

SBCL is a Common Lisp "compiler" which dynamically translates every REPL interaction (and calls to eval etc...) into machine code. So you feel it is an interpreter. Most JavaScript implementations in browsers (e.g. v8) use JIT compilation techniques.

In other words, the difference between interpreters and compilers is very fuzzy (actually there is a continuum between both), and practically speaking, most programming language implementations has often both an interpreter and a compiler (at least to byte code) facet.

An implementation can be fast or slow independently of using most "compiler" or "interpreter" like techniques.

Some language traits favor an interpreting approach (and can only be efficiently compiled thru whole program analysis).

For some types of problems, designing the software with some metaprogramming approaches is worthwhile and gives important speed-ups. You could imagine that given some specific input, your program dynamically generates specialized code to process it. This is even possible with C or C++ (either using some JIT library, or generating some C code, compiling it as a plugin which gets dynamically loaded).

See also this related question about Python, and that

Community
  • 1
Basile Starynkevitch
  • 32,434
  • 6
  • 84
  • 125
7

For code like A = A + B, that can compile down to one or two machine instructions, each taking a certain number of cycles. No interpreter can do the same thing in that number of cycles for a simple reason.

The interpreter also executes an instruction set of its own (call them byte-codes, p-codes, intermediate language, whatever). Each time it sees a byte-code like ADD, it has to look it up somehow and branch to the code that does the addition.

The next time it sees it, it has to repeat that lookup, unless it has a way to remember the prior lookup. If it does have a way to remember the prior lookup, it is no longer what we call an "interpreter", but rather a just-in-time compiler, or JITter.

On The Other Hand...

For code like callSomeFunction( ... some args ...), how many cycles are spent between entering that code and leaving it? It all depends on what happens inside callSomeFunction. It could be a few, and it could be trillions, even if callSomeFunction is itself compiled. If it's a lot, there is no point in debating the interpretation cost of that line of code - the money is elsewhere.

Remember interpreted languages have a value of their own, such as, no need to compile them. (The "compilation" of surface syntax to byte codes takes trivial time. Take R or MATLAB, for example.)

Also, there is flexibility needed for intelligent levels of programming. In Minsky's Society of Mind, Chapter 6.4 B-Brains, there are A programs that deal with the world, and there are B programs that deal with A programs, and there can be further levels. Programs that write and manage other programs can be more easily done in interpretive systems.

In Lisp, you can write (+ A B) to add A and B, but once it is written you only have the choice to run it or not. You can also write (eval (list '+ 'A 'B)) which constructs the program and then executes it. It could construct something different.

The program's subject is another program. This is easier to write in an interpreted language (though, as Jörg points out, newer versions of Lisp, while they have eval, compile-on-the-fly, so they do not have the speed penalty of interpreting).

Peter Mortensen
  • 1,050
  • 2
  • 12
  • 14
Mike Dunlavey
  • 12,815
  • 2
  • 35
  • 58
  • 1
    I find it interesting that you say writing meta-level programs is easier in interpreted languages, yet use Lisp as an example, where most implementations are compiled. – Jörg W Mittag Jan 10 '17 at 14:02
  • 1
    @JörgWMittag: Sure, but they all have `eval` and `apply` functions, which are interpreters. – Mike Dunlavey Jan 10 '17 at 14:04
  • 2
    I'n pretty sure that at least on SBCL, `eval` is not interpreted. And neither is `apply`. There certainly are implementations that contain interpreters, but SBCL does not. – Jörg W Mittag Jan 10 '17 at 14:09
  • 1
    Interpreter can optimize loop condition out in runtime and squash remaining iterations. This is rarely possibly in compiled programs. Specifically, Oracle's Hotspot does exactly that. – Basilevs Jan 10 '17 at 14:42
  • 2
    @JörgWMittag: Sure `eval` is not interpret***ed***. It is an interpret***er***. – Mike Dunlavey Jan 10 '17 at 15:23
  • Not on SBCL. SBCL implements `eval` as a compiler, not an interpreter. SBCL doesn't have an interpreter. Only a compiler. – Jörg W Mittag Jan 10 '17 at 15:36
  • @JörgWMittag: OK, that's news to me. Good for them, and thanks. It's been 30 years since I've done much Lisp. But the rest of my point about flexibility remains. – Mike Dunlavey Jan 10 '17 at 17:41
  • You're confusing cycles a bit. An interpreted addition *can* happen in the same amount of time on modern CPUs (especially Intel's newest, which almost seem like magic :)). A simple "one big switch statement" interpreter has very little overhead nowadays (though obviously, if you have a highly abstracted interpreter, the performance drops all the worse). Of course, there's still places where you can save and lose, but it's not clearly "native is better" - e.g. interpreted code can be smaller than native code (especially obvious on 64-bit), which can mean better caching for example. – Luaan Jan 11 '17 at 09:32
  • 1
    A program that takes a program in source form and runs it is an interpreter. Any efficient interpreter is going to preprocess that source code in some way, but the fact that an interpreter's internal form for the source code in some cases is machine language does not make it not an interpreter. – prosfilaes Jan 12 '17 at 00:21
  • @prosfilaes: machine language programs are not interpreted by a Lisp interpreter, but are run by the CPU. – Rainer Joswig Feb 10 '17 at 08:54
5

Kind of, sort of, it depends but as general rule the compiled - be it through JIT or statically compiled - environment will be faster for many compute-intensive tasks - assuming for simplicity same language.

Part of the reason is that interpreted languages need to have interpreter loop - loop which reads an instruction, selects appropriate action to take and executes it. In best case like interpreting Python or Java bytecode (as old JVM did) it have an overhead of few instructions and played havoc with branch predictor - without the last one you can expect huge penalties due to mispredictions. Even a very dumb JIT should speed this significantly.

That said the interpreted language may cheat. For example Matlab have optimized routines for matrix multiplication and with few changes you can get code running on GPU (disclaimer: I work for nVidia - any opinion expressed here is mine and does not represent view of my employer). That way you can write short and powerful higher level code without worrying about details - someone did take care about it and pour time and resources into optimizing it in low-level language. There is nothing inherited about it and it does not prevent, for example, Matlab to JIT the code but often there is no reason as overhead of calling high-level routine is minimal compared with time spent in low-level ones.

TL;DR - the compiled programs have huge performance benefits over the interpreted ones (for apples-to-apples comparison see PyPy Speed). However the speed of executable is only part of problem and it may not contribute much to overall speed (if time is mostly spent in libraries). Also the implementation matters.

Maciej Piechotka
  • 2,465
  • 2
  • 18
  • 19
5

Your assumption is well founded, although it is an assumption.

I am not going to go over the reasons why compiled code should be faster than interpretted code: if you know how computers work, that will be obvious. The difference can be orders of magnitude for certain types of problem. If your reviewer seriously disputes that general case, they don't know what they are talking about.

Where they may have a point though is whether the difference is significant in the type of application you are developing. If it is mostly I/O or mostly calling compiled libraries and it doesn't have a lot of computation, the overhead of the interpretation process may indeed be insignificant.

But the point of my post is this: as an I.T. expert you will be often called to make snap decisions based on a general knowledge of how things should work. Doing a specific test might get you a more accurate answer, but it will cost a lot more and it won't get you there first.

But from time-to-time you do get caught out. It's happened to me. You make a good assumption and then you find you failed to take into account the stupidity of the world.

But I can't explain as well as my favourite Dilbert cartoon of all time. Nothing shows better than this the perils of being a smart-ass.

alt text

TL;DR: you should be right, but check the real world just in case.

Community
  • 1
rghome
  • 668
  • 5
  • 12
3

Unless you use something somewhat exotic, your problem won't be about performance about an interpreted language A and compiled language B.

Because if you/your team know A and not B and so write way better code in A than B you can end up having much better performance in A than B. If you have people experienced in one language and the language/libraries can does the job you need, stick to it.

Here's a link about regex in various languages; you'll see that regex are implemented better in some language even if compiled or not: http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=regexdna

Peter Mortensen
  • 1,050
  • 2
  • 12
  • 14
Walfrat
  • 3,456
  • 13
  • 26
  • The problem is that the team can use both A and B, and did not indicate a preference for either when I asked them for input. – EpicSam Jan 10 '17 at 13:12
  • @EpicSam what about their past projects then ? – Walfrat Jan 10 '17 at 13:17
  • 1
    Both A and B have been used. However they want to move to using 1 technology for all projects. Both technologies's performance is good enough for current projects. However the potential higher performance of one of them should be considered when looking to potential future projects. – EpicSam Jan 10 '17 at 13:29
  • 3
    @EpicSam better look for the community around the language and the library/Framework available to help you in your projects. – Walfrat Jan 10 '17 at 13:43
  • 6
    I agree with @Walfrat . I would bet the language with "highest potential" is straight-up assembly or some raw CPU instruction set. But we aren't talking about those languages because its "obvious" that having tools like compilers, interpreters, and pre-written libraries is ultra important. I think the availability of tools / community knowledge is more important than the choice between interpreted/compiled – Ivan Jan 10 '17 at 13:57
  • >> However the potential higher performance of one of them should be considered when looking to potential future projects. << Find a way to characterize a specific performance ceiling for tech A. For example, the classic "as-fast-as C when it is C". http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=regexdna – igouy Jan 10 '17 at 20:19
  • @igouy interesting link i put it in my post – Walfrat Jan 10 '17 at 20:22
  • And [pdf] "Although Sawzall is interpreted, that is rarely the limiting factor in its performance" https://static.googleusercontent.com/media/research.google.com/en//archive/sawzall-sciprog.pdf – igouy Jan 10 '17 at 20:32
1

I think it's just not a good idea to talk about performance of two technologies just based on the fact that one is compiled and the other one is interpreted. As stated in other answers, it may depend on the area of application (some languages may be optimised to do some operations very quickly and do other things more slowly) as well as on experience of people that are about to use that technology.

I don't think it's reasonable to expect that you will get a performance boost if you take some excellent interpreted language coders and give them some technology that they are unfamiliar with - maybe in theory the latter MAY result in better performance, but in reality, without necessary skills and experience, you won't use all the optimization opportunities.

From one of the well-known Silicon Valley company employees I've also heard that they prefer the language that is simpler to use as it is more costly and troublesome to pay some skilled developers to maintain a complicated, but highly optimized code than just to buy more rig in order to deal with the less efficient implementation, so that also should be considered while choosing the technology.

KjMag
  • 187
  • 10
0

Once I had to make a similar sweeping statement to justify a big decision.

First, they may not want to believe a humble engineer, so I found some comparable benchmark tests and quoted them. There are a lot of them about, from people like Microsoft, or renowned universities. And they will say stuff like: Method A is between 3 and 10 times faster than method B, depending on variables X and Y.

Second, you might like to run a benchmark of your own, maybe using a representative chunk of the code in question, or something similar you already have. Run it 1000 times overnight so there really is a measurable difference.

At this point the difference (or lack of it) between A and B should be so clear that you only have to present it. So format the results clearly, with diagrams if possible, stating all assumptions and defining all data used.

RedSonja
  • 297
  • 2
  • 4
  • 1
    The problem with doing my own benchmark is that I would effectively just be bench-marking 2 specific programs written in A and B, not A and B as a whole. It should be possible to create programs that solves problem X in both A and B where A is faster and then rewrite them in way B is faster and vice versa. This essentially allows you to work backwards from your conclusion, for example if you prefer A then either choose a scenario in which A is faster or simple optimise the A version till it outperforms the one in B. Therefore we could only conclude performance in a specific case not general. – EpicSam Jan 12 '17 at 15:33
  • Depending on how big and important this decision is, you could implement a representative part of the functionality using both A and B. If this is a really big decision then this is not wasted time. You would have to justify how representative your section is, and that you are not trying to bias in favour of your personal preference. – RedSonja Jan 13 '17 at 09:46
  • 1
    @EpicSam Find someone clearly favoring A and let them optimize the benchmark for A. Do the same for B. You only need to ensure that both programmers are of about the same level and spend about the same time. There's still the problem of selecting the benchmark, but let both programmers be involved in the decision; ideally, let them agree on the benchmark. Another problem is the wasted time, but this can be handled by choosing something simple and useful. – maaartinus Jan 14 '17 at 18:49
0

I would argue that any dynamic language has an advantage over statically compiled ones: "Runtime optimizations"

That is one of the reasons Java can be faster than C++

Yes, loading a dynamically typed language will always have the cost of translation and be at a disadvantage. But once it is running, the interpreter can profile and enhance frequent code paths with runtime information that static languages will never have

NOTE: Well, Java is an interpreted language, not a dynamic one. But it is a great example of what you can speed up with runtime information

SystematicFrank
  • 897
  • 8
  • 18
-3

... I also state that since a program could be written in many ways it is still possible a program written in tech A could outperform one written in tech B.

When I submitted this report for review, the reviewer stated that I offered no clear reason why in general the overhead of the interpretation process would be large enough that we could conclude that tech B's performance would be better. ...

This would be my approach:

In general, interpreters are compiled, so every interpreted technology is nothing else than a compiled technology if looked at it on a low level. Therefore, compiled technologies just are more and with more possibilities you can never get worse if you are smart (which in general you are). It depends on how much information there is available at compile time and how much information available only at runtime and how good the compilers and interpreters are, but it should theoretically always be possible to at least equal the performance of any interpreter with a suitable compiler, just because interpreters are fabricated by compilers. That it's possible, doesn't mean it's the case for your techs A and B though.

In practice, just tell the reviewer about all the benchmarks available where compiled and interpreted systems are compared. Then ask him to suggest an interpreter that beats your optimized Assembly coded specific algorithm.

One should maybe add, that any such general statement doesn't help at all when comparing two specific techs A and B. There the choice of A and B matters much, much more, than if they are interpreted or compiled.

NoDataDumpNoContribution
  • 263
  • 3
  • 16