21

I really enjoy programming language design. Sometimes I think my language projects and their potential users would benefit from a comprehensive standards document. I've looked at many language standards, ranging from the very formal (C++) to the rather informal (ECMAScript), but I can't really get a handle on how I should break things down and organise such a document, even though I think I'm pretty good at technical writing in general.

Should I write it like a long tutorial, or more like a formal math paper? How do I keep it up to date if I'm developing it alongside a reference implementation? Should I just give up and treat the implementation and documentation as the de facto standard? Further, is there really any significant benefit to having a standard? Does requiring a standard mean that the language is needlessly complex?

Jon Purdy
  • 20,437
  • 7
  • 63
  • 95
  • 1
    Have you read Domain Specific Languages by Martin Fowler yet? http://www.amazon.com/Domain-Specific-Languages-Addison-Wesley-Signature-Martin/dp/0321712943 – Gary Dec 03 '10 at 20:09
  • @Gary Rowe: I have not. It looks to be a decent read, though perhaps not exactly what I'm looking for. – Jon Purdy Dec 03 '10 at 20:13
  • The advantage of a standard over a reference implementation is that you can define where other implementations can deviate from what your implementation does. – Bart van Ingen Schenau Apr 29 '15 at 12:59

5 Answers5

8

Read lots and keep it simple

Designing a new language is hard. Really hard. But ultimately very satisfying if it becomes popular and really solves a problem that people are experiencing in an elegant manner.

As I mentioned in the comments, I'd advise you to read Domain Specific Languages by Martin Fowler for the following reasons:

  1. He goes into a lot of practical depth about why you should design a language
  2. There are details on how to do it (parsers, lexical analysers, language workbenches etc)
  3. There are detailed implementation instructions about how your chosen syntax can be made to handle concepts like closures, annotations, literal lists, dynamic reception etc

As for how to go about writing your specification, think about your audience. Obviously, before putting finger to keyboard to design your language you will have thought carefully about what it is intended to do.

If it's a new, interpreted language to replace JavaScript then you'll be wanting a very laissez faire approach in order to reach web developers with a limited attention span and a desire for immediate results - or quicker if possible.

If it's going to be used on the next mission to Titan, then extremely detailed specifications showing exact formal proofs of the behaviour of each component will be the minimal entry level.

So, it's not a straightforward thing. To approach the specification, you would probably be better off gaining a lot of experience in creating your languages and also working with those who actually use them on a day to day basis. If you have willing victims... er... developers, at work who can take some time to learn your language then they can give you feedback on what is needed to get them to use it.

In short, keep it simple and more people will use it.

Gary
  • 24,420
  • 9
  • 63
  • 108
  • Thanks for this. I have a lot of experience developing languages, and even documenting them rather thoroughly, but it's the idea of a standard that consistently gets me. I may just have to pick up the recommended reading and experiment a bit. – Jon Purdy Dec 03 '10 at 20:36
  • @Jon Purdy Do you have any examples of the languages online that you could include in the question? – Gary Dec 03 '10 at 20:41
  • I don't have examples of my current project yet. The only really complete public example of a language I've made (which I do actually use!) is at http://vision-language.sourceforge.net/cgi-bin/Home – Jon Purdy Dec 03 '10 at 20:43
  • @Jon Purdy Vision looks interesting - a sort of souped up Velocity. As an aside, you might want to consider a YouTube screencast showing how to install it and write an example small website (say for a local plumber). This would make the learning curve much easier since people can see it in action and immediately take on the benefits. You can talk around the benefits in comparison to JSP, Velocity, ASP.Net, Freemarker etc – Gary Dec 03 '10 at 21:20
  • That's a good idea; I've been making YouTube videos a lot lately (around three a week), so I think I could definitely fit one in. – Jon Purdy Dec 03 '10 at 23:24
5

I found the Java language spec both formal and readable, and I think it has a sensible structure. Some of the W3C specs could be good examples as well.

Doing the formal work could help you keep language complexity down and see the corner cases.

Headings brain dump: source encoding, lexing, fundamental types, literals, operators, expressions, simple statements, conditionals, loops, functions (definitions and calls), type declarations, modules, compilation units, variable scoping, various kinds of name resolution (eg imports, methods), memory model, side effects, typing, concurrency…

Tobu
  • 186
  • 8
  • Your list of suggestions is very helpful. I think what I'm going to do is brainstorm a similar list, sort it in tutorial-like format, and write a brief informal spec with a few formal addenda such as an EBNF grammar. I'll also definitely have another look at the specs you mentioned for ideas. – Jon Purdy Dec 03 '10 at 23:27
3

Wirth designed and implemented many programming languages: of these, the specifications for the Oberon and Oberon2 languages are notable for there completeness, terseness and readability.

grrussel
  • 469
  • 3
  • 5
2

Common Lisp and Haskell have language standards. Ruby and Python have implementations and documentation. So I would say that a language standard is not necessary, but it might be helpful if you expect there to be more than one implementation of the language you're designing. On the other hand, a standard is premature if you're expecting significant changes in your language definition.

Larry Coleman
  • 6,101
  • 2
  • 25
  • 34
  • Actually, Ruby has *two* things that could be considered "specifications". There's the ISO Ruby Specification, which is currently in its Final Draft state and which is being written by some people who have experience with language specifications (having worked on ANSI Common Lisp and ISO C++). And there is the RubySpec project, which is a set of RSpec-style executable examples forming both a human-readable specification and a machine-executable conformance testing suite for that specification. – Jörg W Mittag Dec 05 '10 at 01:35
1

any specification should be terse and able to stand the test of time

this is why you see an abstraction like BNF used for many language standards...its terse and will still be understood long after many of our current tools have been left behind.

of course there's more to it than just a grammar. look at what others have done...perl6, scheme, C...they address issues that implementor also care about.

Brad Clawsie
  • 740
  • 3
  • 7