Lisp obviously is an advantage for the AI stuff, but it doesn't appear to me that Lisp is any faster than Java, C#, or even C. I am not a master of Lisp, but I find it incredibly difficult to understand the advantage one would get in writing business software in Lisp.
Yet it is considered as a hacker's language.
Why does Paul Graham advocate Lisp? Why did ITA Software choose Lisp over other high-level languages? What value does it have over these languages?
There are a few reasons I am working on becoming competent with Common Lisp.
Arguably the only real reasons to choose against Common Lisp is that the standard libraries are dated.
I will go out on a limb and say that in the general case, syntax should not be an issue to a professional software worker.
I like Lisp for its
Programming is fighting complexity. Abstractions are the only effective tool for fighting ever increasing complexity (with our very limited and constant skull size). Managing abstractions with Lisp is like having a genie with n+1 wishes.
I believe the correct Lisp answer is more gnomic. Something like: "If you have to ask, you are not ready."
Then if anyone questions further, the correct answer is either "yes" if it's an either/or question or "You are not ready."
I think that the Lisp advantage in the artificial intelligence (AI) field that everyone mentions is somewhat a historical accident... Lisp started for/in AI, but it's a general-purpose language.
I believe execution speed is not the only important aspect of a language (I did once, though). However, one of the aspects I like about Lisp is that for me, it combines Python and C in one. I can start to code with no declarations and prototype immediately and very quickly (the runtime and the REPL are very important for this). After I have something running, I add type declarations and "optimize" my code, little by little. It is a wonder to press a key in SLIME and watch the machine language generated for the function I'm interested in. In Python, there are no type declarations, so I cannot get more speed, but in C, getting anything done quickly is much more painful. Lisp is very useful in this case.
Having said that, I like Lisp mainly because of macros. When you finally understand what macros can achieve, I think you put up with parentheses easily. Also, editors like Emacs manage parentheses themselves so you don't have to. I admit, however, that I didn't find parentheses all that bad at the beginning, and I know some people just can't stand them. But since the whole purpose of macros is to generate code at compile time, code in Lisp uses a standard data structure, and the parentheses simply are a representation of code as lists, which is necessary to make macros simple to write.
I don't know any other language in which you can write little sublanguages to describe your problem better with the ease of Lisp. That is the advantage Paul Graham talks about in Beating the Averages. It is extreme modularity and conciseness. In Java I have to write a lot of raw text to express a single idea. In Lisp I could write some macros that generate that code automatically, and afterwards just use those. Anyway, you have to understand some examples of this and then judge for yourself. When I "saw" it, I was blown away, and I still think Lisp is the greatest language for this reason alone. I always look for macros in mainstream languages to see if they match the power of Lisp macros, but to date I didn't find any. Forth is a close second.
I'll finish with a couple of criticisms, in relation to business software:
Business software needs libraries, and good ones, and Lisp is not good at this. I usually don't need them, but when I do, I have to choose from a little selection of incomplete software that a few people uses. I should contribute to fix this, I guess...
Business software is usually built by large groups of people, and I think communication can be impeded with macros, since they basically change the language. A lot of programmers are more comfortable detecting certain patterns in code, even if the program text is longer and more repetitive. I suppose at ITA they have some rules regarding macros or they have a huge macro library that makes collaboration easy (or, more simply, all programmers are Lisp experts).
I don't like Lisp.
(I do like many of the concepts it uses, how it makes powerful techniques available natively, and so on.
But I've never been convinced to actually go use it, ((even though several people have tried) because the benefits of the language can be achieved with other programming languages (some directly, some indirectly), so there isn't enough benefit to get me to spend the time learning it and putting up with the horrific syntax.)))
But yeah, for reasons that some people like it, check these Stack Overflow questions:
There's probably a few more in the related questions for those too.
I'll interpret "Lisp" as "Common Lisp"; I've no doubt other answers will say "Scheme". (Hint: Lisp's a family of languages.)
What does "faster" mean? In terms of time taken to run a benchmark, no, it's not faster than C (but it can be).
"Fast" in terms of how long does it take Joe Random Hacker to write up a working program, or fix a bug in large software system? Almost certainly.
As for this hacker, I use it because I want to write code, not boilerplate. I want to write something once, and not continually repeat myself. And I want to interact with the program while I write it.
I like Lisp because it's an excellent medium for expressing my thoughts. The predicate for my favorite language is "If I could pick anything to express ideas in, what would it be?". Currently it's Lisp* (Scheme to be specific), to the point that I find myself writing out programming notes in it. As IRL, paper and pen notes. Even when I'm thinking about programs I need to implement in PHP or Ruby or Python.
This isn't a trick I taught myself, or something I do for nerd credibility (no one gets to see the inside my notebook anyway); it's just that Lisp is so much more natural for me to think in than any of the alternatives, and any language that resonates with you that deeply is one you treasure.
*Just as a footnote though, Haskell is closing the gap pretty quickly as I learn more of it.
Paul Graham sort of answers this question himself in What Made Lisp Different.
Keep in mind he used it for his startup during the mid 1990s, so Python and Ruby weren't really mature at that point (or maybe not even born).
Lisp basically has all the advantages of dynamic languages, and I think for most of today's web applications, Python and Ruby are pretty awesome, and they have the advantage of frameworks and documentation and vibrant communities.
The killer feature is probably that the entire program is made of expressions. This means you can sort of pass around blocks of code to functions (or macros...), because a block of code is nothing more than an expression.
Python doesn't exactly have this feature; you'd have to define functions and pass them around. Ruby seems to have blocks, perhaps it's somewhat limited compared to what Lisp can do (I'm not sure).
The issue is power. Power = Work (program functionality) / Time
"We were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp."
-- Guy Steele, Java spec co-author
Plot some kind of curve between C++ and Java. Keep going, and at some point along the line you'll find Lisp.
The thing that I like about Lisp is that it transcends paradigms. Some people will say that Lisp is functional, others will say it's declarative, and others will say it's multiparadigm. I think all of these miss the point. When you use Lisp, paradigm is no longer a constraint.
Want objects? You can have them. Want functional programming? You can have it. Want Prolog-style logic programming? Write some macros. Want SQL-style declarative programming? Go for it. Want to use some paradigm that hasn't been invented yet? I'm confident it can be done in Lisp.
Aside from Forth-like languages, I have yet to see another language offer this level of flexibility.
I have had a knee-jerk reaction to Scheme in the past, but now I am ready to give Lisp (Clojure, actually) a shot.
You see, over the years I picked up bits of languages like Java, C#, C++, Python, and things are no longer challenging.
Clojure has many promises, appears to be very clean, and can solve many real-world problems. A strong case for a clean language like Clojure is the advent of multi-core computers.
Yay LISP!
EDIT: ITA Software was founded by MIT grads, and Scheme/Lisp was the only language that many of the MIT grads learned. To be fair though, one can hot-swap Lisp algorithms on a running production system, which is a huge plus.
I'm learning Lisp (newLisp) for a couple of reasons.
Reason number one: Lisp makes me think differently, which makes me a better Ruby coder.
It seems very awkward to do things certain ways in Lisp, for example nested iteration to go through multiple lists. So it forces me to use other things, like map. My favorite language, Ruby, has the same map method, but I don't always use it, because it's unfamiliar: I learned to do things using a poor technique, and when the language supports that technique I continue to use it.
Reason number two: Lisp is practical and has good modern libraries.
There's a very nice, lightweight web framework for newLisp called dragonfly. This allows me to use newLisp code instead of PHP for some tasks. I don't really like PHP, and newLisp seems more fun for this specific task than Ruby.
Reason number three: Lisp is syntactically and conceptually consistent.
For me, this is the big difference between Ruby and Python, consistency.
"Faster" isn't a simple thing to measure -- it really depends on which aspect you're benchmarking. Depending on the task and the Lisp implementation, speeds can approach C. Look at the Great Benchmarking Shoot-Out to dive in for details. The SBCL implementation of Lisp is on par with Java 6 Server and is significantly faster than Ruby or Python.
But, pure speed isn't the main reason to choose a programming language -- if it was, we'd all be programming in assembly language still, right? For me, the daily joy of Lisp is that the code is compiled, but I don't have to take down the application, recompile everything, and then start running from scratch. Instead, I can change a single function and that change will take effect everywhere, and I can immediately see the effect in my application. Moreover, that very quick "write, test, write more, test more" approach makes it much easier to test immediately up front while writing the code (and then you can turn those interactive probes into unit tests later on).
Imagine writing email where after every line, you had to hit a button to compile your email output to the screen before continuing your thought. That's what writing in Java or another language like that is for me. Sometimes there's a reason to do that, and I like Java fine, but Lisp is just more responsive, and it's easier to get work done.
When Lisp was created they started from math, not computer science (which didn't really exist yet). And the Lisp team got some things REALLY right. Lisp had garbage collection in 1960 or so! They really did a great job.
I think the The Eternal Flame song covers it.
Can you say "Brand Loyalty"?
I started in Fortran. I loved it.
I switched to Lisp. At first I hated it. Then I learned to love it, and hate Fortran.
Later Pascal, C, C++, various assemblers, C#. (Actually I don't love C#.)
I guess I'm fickle?
A big draw is the community. Lisp has had a draw for the most ambitious and bright developers since the language was invented. Wherever researchers are trying to solve problems that have never been solved you are likely to find Lisp, as in artificial intelligence (AI) research, computer vision, planning, knowlege representation, and complex heuristic optimization. The language lends itself to solving problems both from the bottom up and the top down at the same time, which seems to help in confronting the hairiest challenges.
The exensible syntax via macros means that there is seldom a need to extend the language definition. Much of what would require a language extension in a more restricted language is just a macro away with Lisp. So Lisp programmers are free to make use of newly invented language concepts without a new language standard and without necessarily a real speed penalty. On a basic level, reams of boilerplate code are made unnecessary by small extensions. Whole new ideas in control flow, like Prolog style unification, are implemented efficiently and compactly as extensions.
The OOP system, CLOS, is in a class of its own in terms of flexibility. It is very difficult to go back to rudimentary C++/Java/C# OOP after getting a taste. GoF 5 design patterns become unnecessary as they can be expressed simply and directly.
The language has had no single corporate owner and no single definitive implementation, though it does have an ANSI standard with many conforming implementations. Major new implementations come along every decade and the old ones are still quite active. Experts can plan to use their specialized knowledge for a long time to come. This does cause some anarchistic friction and community fragmentation, but it also means the carpet cannot be pulled out and the language can't become moribund for corporate or project political reasons. There are always multiple commercial and open source implementations being worked on. The more performance focused ones regularly benchmark within a 2x factor of the very fastest, heavily funded imperative language implementations.
The Achilles heel of early Lisp commercialization was memory footprint to accomodate both the type safety features of the language and the advanced software development environments they included, with incredible features like full online documentation including graphics. A 64 MB Symbolics Lisp Machine was not viable cost-wise against an 8 MB Sun workstation. Today, RAM prices have collapsed and there is tremendous interest in the Lisp languages especially considering that the mainstream Java, C#, PHP languages today have advanced only minimally over those of 30 years ago.
There are modern languages now in competition with Lisp for mindshare with intelligent developers: Python, Lua, Erlang, Haskell, and OCaml. But none offers the same mix of maturity, adaptibility, multiple standards-compliant implementations and speed.
I don't actually do Lisp. But the place I work at does finite elements with millions of lines of mainly Fortran. The guy here whom I most respect about computing stuff (codes computational fluid mechanics) thinks the ideal combination is Lisp on the outside (mainly because you avoid messy problems with memory management), and Fortran for the low level algorithms (Fortran is best for exploiting the vector capabilities of SSE/AVX, and we think this lead is unlikely to close).
