What does your Lisp workflow look like?

Question

I'm learning Lisp at the moment, coming from a language progression that is Locomotive BASIC -> Z80 Assembler -> Pascal -> C -> Perl -> C# -> Ruby. My approach is to simultaneously:

• write a simple web-scraper using SBCL, QuickLisp, closure-html, and drakma
• watch the SICP lectures

I think this is working well; I'm developing good 'Lisp goggles', in that I can now read Lisp reasonably easily. I'm also getting a feel for how the Lisp ecosystem works, e.g. Quicklisp for dependencies.

What I'm really missing, though, is a sense of how a seasoned Lisper actually works.

When I'm coding for .NET, I have Visual Studio set up with ReSharper and VisualSVN. I write tests, I implement, I refactor, I commit. Then when I'm done enough of that to complete a story, I write some AUATs. Then I kick off a Release build on TeamCity to push the new functionality out to the customer for testing & hopefully approval. If it's an app that needs an installer, I use either WiX or InnoSetup, obviously building the installer through the CI system.

So, my question is: as an experienced Lisper, what does your workflow look like? Do you work mostly in the REPL, or in the editor? How do you do unit tests? Continuous integration? Packaging & deployment? When you sit down at your desk, steaming mug of coffee to one side and a framed photo of John McCarthy to the other, what is it that you do?

Currently, I feel like I am getting to grips with Lisp coding, but not Lisp development ...

Answer 1

Most of the people on comp.lang.lisp and Lisp Forum recommend a combination of Emacs and SLIME, assuming you don't want to pay for a commercial implementation like Allegro or LispWorks. The SLIME video illustrates the working process. I use Emacs and SLIME for developing personal programs at home, and the combination can be very efficient.

SLIME gives you integration between Emacs and the REPL. What I typically do is load all of my files, then open the one I'm working on in Emacs. After I type in or change each function, I press C-x C-e, which executes the function definition in the REPL. Then I can switch to the REPL buffer and try the function. All of the REPL's history is available and editable using standard Emacs key sequences, so it's easy to redo function calls with different parameters.

Answer 2

To supplement the other answers, I do a combination of the "Think hard about the problem, then write down the solution" kind of development and "Start with a repl, and beat your program into shape iteratively" style. Its usually in the middle. What i do is I start with a general idea of what I want my program to look like, and then I start writing it, trying to acquire new knowledge about the problem domain as i go, in order to revise my approach. In practice that means that I do a lot of experimentation and write a lot of trow away code. I try different approaches and I can switch direction very fast. Lisp is good for this kind of development. That's why I don't like TDD so much, I have to know what I want to do first, in order to to TDD effectively.

Another important thing I try to do is think more about the protocol between the different parts of a program. Unlike other OO languages, In Common Lisp CLOS is focused more on the concept of a Generic function, IOW methods live outside of classes, and they are the focus of development. Sometimes i just start with a set of GFs defining the protocol i want, I write a simple implementation and use it to flesh out the protocol, before I write a real implementation. Say I'm writing a layer that stores a bunch of blog posts, i might have a set of GFs that define how blog posts are created, edited and searched, and I would write a simple implementation that saves blog posts in a list in memory, and after I'm happy with my protocol, i write an implementation that saves blog posts to a database, or writes them to files.

Lisp makes it so easy to change direction and do a different approach when you know that your current solution is suboptimal that I try to maximize on that.

Also important point: Use your environment to the fullest, learn about using different compilation settings to compile your code with more debugging information, take advantage of the fact that lisp can be both compiled and interpreted. Use lisps introspective facilities, such as the MOP, use slimes features to look up documentation, use the inspector to look inside objects, consult the hyperspec regularly, treat your system more like an OS, than a static compiler for your code, it's much more powerful than that.

For a beginner, lisp isn't that much of a win over python and ruby, but as you get more experienced with it, you get enormous wins over lesser environments.

Most important, you have to have fun and love this language, sometimes its just to easy to get discouraged and go use the currently popular blub languages. If you stick with lisp, it will repay you.

Answer 3

I work primarily with Clojure, and here's my setup:

1. Eclipse IDE (I use this as I also do a lot of Java work, sometimes in the same project as Clojure)
2. Counterclockwise plugin for Clojure (this includes a REPL)
3. Maven for dependency and build management
4. Egit for source code control

Workflow while coding is generally like this:

1. Open a REPL, loading all the current source files
2. Code and test in the REPL
3. When I'm happy with some code, copy / paste back into the source file
4. Occasionally reboot the REPL (either when I've messed up a namespace irrevocably, or just to check everything is still working in the source files)
5. Also write tests in the source files, which are run automatically by every Maven build. Sometimes the informal tests I've just done at the REPL get turned straight into unit tests.
6. Commit etc. at this point - pretty much a regular development workflow

Overall it's not too different from a regular Java/C# workflow apart from the more interactive use of the REPL during development.

Answer 4

I tend to work in an editor that has a connection to a REPL (typically, editing in emacs and using SLIME as a bridge to a Common Lisp environment). I tend towards starting both "at the bottom" and "at the top", working towards the middle, revising the design as I go.

As for testing, taht's why I have a REPL. I find that it helps testing code as I go. I will sometimes write more formal tests or benchmarks as I go (usually once I am in the optimisation stage). Have a slower, known-good implementation of a functionality, experiment (in an editor window) with code for an optimised version, send that to the under-lying lisp code and check that it is both faster and returns the same results as the slower code.

As far as packaging goes, I have some utility code to help me pack up ASDF-installable projects, slap them in a download repository and handle versioning.