10

I'm designing a simple OO programming language.

It's statically typed, compiled, and executed by a VM - similar to Java.

The difference is that I don't want to have such a strong emphasis on OOP. The code itself will mostly resemble C++ (classes, functions, and variables allowed in the file scope).

One of the things that I need to have is a module system. I have the following figured out:

  1. Every file is a module (once compiled) - like Python
  2. Programmers must import a module with the import keyword, which causes the compiler to search for modules in standard directories and the file directory (the VM has to do this at runtime as well)

And now I have no idea how should I introduce the concept of submodules and module hierarchy.

One option, for example, is to depend on the directory hierarchy, so that import engine.graphics.renderer would expect to find a directory called "engine" in the working directory, and inside a directory called "graphics", with a module called "renderer".

What are the drawbacks of such a design? Am I missing anything?

Aber Kled
  • 331
  • 2
  • 10
  • 1
    Well, it wouldn't be the first. Both Rust and OCaml implicitly treat code in a file as being part of a module of the same name as the file. Seems to work well enough there. – KChaloux Oct 01 '13 at 17:24
  • 7
    And Haskell. If you're doing OOP anyways though, I'd encourage you to consider the Scala "Modules are special objects". Also do you intend on supporting first class modules? If so then there's an ambiguity between `foo.bar` being foo/bar.file or some module `foo` with the member (also a module) `bar`. SOmething to consider – daniel gratzer Oct 01 '13 at 17:28
  • How do you plan to handle versioning? Lots of languages don't at all, but you might as well **not** make that mistake up front. – Donal Fellows Oct 02 '13 at 05:35
  • 1
    @DonalFellows Care to elaborate? How's it the language's responsibility to handle versions? But anyway, I meant to introduce `@annotations` for embedding such information. – Aber Kled Oct 02 '13 at 16:05
  • @AberKled: I don't think the issue is so much having the language be responsibility for "handling versions", as ensuring that if Fred has one version of module X when he writes a module Y that works with it, and Joe has a different version of X when he writes module Z that works with it, then in the absence of unresolvable *behavioral* differences between the different versions of X, it should be possible to use Y and Z together without having to recompile either. – supercat Mar 17 '15 at 22:57

2 Answers2

3

Take a look at Python's package/module hierarchy/organization, and especially in historical aspect, major additions over the course of years, most importantly the latest ones. Probably, there is no sense in inventing the wheel.

I do not know how far you want to go with your language, eg

  • Do you want modules bytecode to be read from zip?
  • How do you separate modules/libraries for different interpreter versions?
  • Do you plan to make it seemless with distros?
  • Do not forget easiness of language's own distribution (think distutils / pypi - each platform/language has its own, not always good)

I guess, Java may be another interesting example. Things can be learned from Erlang's way as well (eg https://stackoverflow.com/questions/2968914/code-hot-swapping-in-erlang).

There is a whole lot of design issues (some touched above) if you plan to see your programming language mainstream one day. Fortunately, there are great examples out there.

Some directions programming language module/package/library system design should address:

  • intuitive code for both writing and using the module
  • module/package objects in the code
  • module/package introspection capabilities
  • module/package encapsulation (how to hide unnecessary details?)
  • use of interfaces/header files?
  • fine-graned dispatching system, which both language own distribution utils as well as system level utils can use (avoid "DLL hell")
  • storage places for bytecompiled modules
  • setup system, which automates compiling both "pure" modules and modules/dependencies
  • canonical place of compiled code, tests, documentation, assets in your module
Community
  • 1
Roman Susi
  • 1,763
  • 2
  • 12
  • 20
  • this post is rather hard to read (wall of text). Would you mind [edit]ing it into a better shape? – gnat Oct 01 '13 at 17:42
  • 1
    "_There is a whole lot of design issues if you plan to see your programming language mainstream one day._" I don't. But I'd still appreciate if you told me about the issues... – Aber Kled Oct 01 '13 at 17:42
  • 1
    @gnat Thanks for pointing it out. Hopefully better now. – Roman Susi Oct 01 '13 at 17:44
  • 1
    @AberKled I've tried to add a few. But as I told in the answer, take a look at the history of development of some mainstream language, better if it is driven by programming community nees rather than purists/elitists, to learn lessons from mistakes before making your own. Of course, do not forget main design goals for your language. – Roman Susi Oct 01 '13 at 17:53
3

First of all, let's assume that you map your namespaced model to another namespace, such as the filesystem, as you suggested. Second, I'm assuming that modules can import other modules. In other words, engine.graphics.renderer could very well contain a line like import circle.arc. That raises two issues:

  • Where would the VM look for circle.arc? According to to the filesystem mappimg mentioned, there should be a directory like /etc/mylang/modules/circle/arc (/etc/mylang/modules being the root of your module structure).

  • How would an application reference circle.arc: by importing circle.arc or engine.graphics.render.circle.arc? The first would "spoil" (if not ruin) the hierarchy, because circle.arc is clearly a submodule of engine.graphics.renderer, and the second would imply that there should be a /etc/mylang/modules/engine/graphics/renderer/circle/arc in your filesystem, placing circle.arc in two locations simultaneously.

Having said that, and should you decide to go with the namespace approach, mapping it to the filesystem seems too restrictive to me. I think modules can reside in all different kinds of places (even zip files, as already mentioned, even urls). The VM would start by looking up an entry in some kind of index (maybe a config file) that would map the namespace to actual module locations.

geomagas
  • 151
  • 5
  • 1
    You are probably getting downvoted because you are not really answering the OP's specific question well. If your comment can't convey what you want to say then you probably need to have a broader conversation with the OP offline. Our chat room is a great place to do this. – maple_shaft Oct 13 '13 at 13:23
  • @maple_shaft: What do you mean? Isn't the question _"What are the drawbacks of such a design?"_ Am I not making it clear enough that my points are potential _drawbacks_ to the OP's suggested module architecture? The reason I never asked why I was downvoted is that I come from StackOverflow, where people do it all the time without caring to leave a comment about it -- which is frustrating, and more importantly _non-constructive_, but I'm used to it nevertheless, _and_ tired of asking for an explanation. – geomagas Oct 13 '13 at 13:36
  • 1
    You admit yourself that you aren't intending to answer the question and you are more or less clarifying your thoughts on the example he provided about what to do with the Renderer module. The question really is more abstract than that and needs a broader answer I think. I was providing an explanation for your benefit because I do think you raise good points and you mean well, but indeed it doesn't really address the key parts of the question. – maple_shaft Oct 13 '13 at 14:46
  • Well, if the first sentence was the problem, I removed it, since it didn't agree with the rest of the answer anyway -- meaning that, although I started it as "thoughts", it ended up addressing the real question. As to the question being more abstract, I think not, since it's expressed in a very specific way. If I'm wrong, I expect the OP to specify what is it that he implied but didn't put in words. I know that your comment was meant to help, unlike the _Downvoter Anonymous_ behavior, and I appreciate it. I really do. – geomagas Oct 13 '13 at 15:43