Why do heavily object-oriented languages avoid having functions as a primitive type?

Question

As has been covered to the point of parody, heavily object-oriented languages, such as C# or Java, tend to lack the feature of having functions as a primitive type. You can argue about whether or not their functions are first class, but the pattern is always that you cannot reference a function on its own; It must always have a companion object. C# calls these companions "delegates" and I believe Java calls them "runnables". In a language that doesn't need these companions, higher-order functions can be called with a function directly used as one of their arguments.

My question is this: Why, from a language design point of view, do heavily object-oriented languages tend to lack the ability to reference a function on its own? Why must they be carried around by a companion object? For example, what benefits do these companions provide over the functional-programming like approach of being able to have a function be its own thing? What costs do they avoid?

To pre-empt an objection, I do not believe that this question is opinion-based. The Java and C# designers are very well known for documenting their decisions and I am absolutely certain that they were familiar with heavily functional languages such as Scheme. There'll surely be a good and documented reason why they chose their current approach. In fact, with the rise of more functional languages that intermix with Java, such as Clojure, I wouldn't be surprised if there's already documentation of the costs that functional Java-like languages have paid by choosing the "functions don't need to be carried around by an object" path.

Finally, please note that I'm not specifically asking about C# or Java. Those two languages just happen to be the best examples of languages that insist that functions must be referenced with an object.

Answer 1

IMO...

1. Because Java and C# are not true OO languages.
2. Functional programming was not in vogue when they were designed.

I agree with Jörg W Mittag, neither C# nor Java are true object-oriented languages. They're hybrid procedural/OO languages attempting to improve on C++ (and C# attempting to improve on Java). They have both a traditional primitive type system and a class system. For example, int is a primitive type in Java and requires a wrapper to behave like an object.

Such hybrid design means the language designers get to pick and choose what is and is not an object. The designers of Java and C# didn't feel functions needed to be objects, I'm guessing because functional programming wasn't vogue back then and it was a little faster to not have them be objects. So they weren't objects.

In contrast, true object-oriented languages (Smalltalk, Ruby, Scalar, Eiffel, Emerald, Self, Raku) treat everything as an object which responds to methods. Everything. That includes methods and procedures. They're objects so they can be referenced. Methods being objects are inherent to OO design because everything is an object; the language designers would have to deliberately do otherwise.

For example, Ruby is a pure OO language. Since everything is an object, it has Method objects. Since everything is an object, the code of a Method is a Proc object. Lambdas are just special Procs.

func = lambda { |x | x**2 }

That's syntax sugar for Proc.new. func is an instance of Proc. I can call methods on it.

p func.call(4)  # 16
p func.class    # Proc            

I can ask it for its call method. That is a Method object.

method = func.method(:call)
p method.class    # Method
p method.call(4)  # 16

Anonymous functions and method references are a benefit of Ruby sticking to OO design principles.

In contrast, let's look at how Java and C# implemented lambdas and function references.

---

Java was designed in early 90s as a better C++. C++ is a hybrid procedural/OO language with many, many design problems, but it was extremely popular and influenced what many people thought OO was. Java inherited these problems.

Because Java is a hybrid language, they picked and chose what was and was not an object. For whatever reason, probably micro-optimization and a lack of appreciation for functional programming, they decided that methods were not objects and could not be referenced, and there were no bare functions.

Java attempted to address the need to pass around snippets of functionality with anonymous classes. They finally realized this is awkward and added lambda expressions explaining...

Lambda expressions let you express instances of single-method classes more compactly.

One issue with anonymous classes is that if the implementation of your anonymous class is very simple, such as an interface that contains only one method, then the syntax of anonymous classes may seem unwieldy and unclear. In these cases, you're usually trying to pass functionality as an argument to another method, such as what action should be taken when someone clicks a button. Lambda expressions enable you to do this, to treat functionality as method argument, or code as data.

At the same time they added method references...

You use lambda expressions to create anonymous methods. Sometimes, however, a lambda expression does nothing but call an existing method. In those cases, it's often clearer to refer to the existing method by name. Method references enable you to do this; they are compact, easy-to-read lambda expressions for methods that already have a name.

What you can do is use them to construct instances of any interface which is a FunctionalInterface.

Functional interfaces provide target types for lambda expressions and method references.

Runnable r = () -> System.out.println("Hello World!");

// The equivalent Runnable class.
Runnable r = new Runnable() {
   @Override
   public void run() {
    System.out.println("Hello World!");
   }
};

There's dozens of FunctionalInterfaces which seem to be working around Java's non-OO primitive types.

DoubleToIntFunction: Represents a function that accepts a double-valued argument and produces an int-valued result.

The hybrid nature of Java means rather than flowing naturally from the design, a complex series of adapters is necessary.

Point is, it was bolted on later and it's complicated.

---

C# was designed a better Java. It fixed some mistakes, and repeated others.

When you go back and look, C# version 1.0, released with Visual Studio .NET 2002, looked a lot like Java. As part of its stated design goals for ECMA, it sought to be a "simple, modern, general-purpose object-oriented language." At the time, looking like Java meant it achieved those early design goals.

C# 3.0 added lambda expressions. As with Java, the relationship between lambda expressions and objects is complicated...

Any lambda expression can be converted to a delegate type. The delegate type to which a lambda expression can be converted is defined by the types of its parameters and return value. If a lambda expression doesn't return a value, it can be converted to one of the Action delegate types; otherwise, it can be converted to one of the Func delegate types. For example, a lambda expression that has two parameters and returns no value can be converted to an Action<T1,T2> delegate. A lambda expression that has one parameter and returns a value can be converted to a Func<T,TResult> delegate.

---

This complex system of annotation types and target types and automatic casting and blurring between what is and is not an object is a common problem of hybrid languages, particularly C++ derived languages which mix procedural and object-oriented principles with a C-style type system without choosing one clear paradigm.

And that's my point. True object-oriented languages naturally have function references because functions are objects. Hybrid languages get to pick and choose what is and is not an object; if the language designer didn't think function references were needed, you don't get function references.

Answer 2

This is a little bit of a silly question. You're asking why object-oriented languages are object-oriented. If they passed functions around as first class types we wouldn't describe them as object-oriented languages.

Why, from a language design point of view, do heavily object-oriented languages tend to lack the ability to reference a function on its own?

Because they believe that virtual dispatch (calling the most derived type's implementation of a function) is a better way to model polymorphic behavior than function arguments. "Better" here of course could mean any number of arguments, potentially including non-functional requirements like readability, ease of implementation, etc.

As has been covered to the point of parody, heavily object-oriented languages, such as C# or Java, tend to lack the feature of having functions as a primitive type.

At least for C#, this hasn't been true for nearly 20 years now. Delegates, and later lambda style anonymous functions are first class tokens which are given special treatment in the type system.

I wouldn't be surprised if there's already documentation of the costs that functional Java-like languages have paid by choosing the "functions don't need to be carried around by an object" path.

You mean unlike Javascript or Haskell or every other functional language that supports closures, which carry an object (the state closed over by lambdas) around with their function?

It's just a different model for the same thing.

Answer 3

In these “classic OOP” languages like C++/Java/C#, objects are data + behaviour, where the behaviour is provided by a class. On a technical level, this generally results in a memory layout somewhat like:

using Method = void(*)();  // some function pointer

struct Class {
  Method methods[];
};

struct Object {
  Class* klass;
  Data data;
};

Example ASCII Art:

| object
v
+---+---+
|   |   |
+-+-+---+
  |
  v class 
  +---+---+---+
  |   |   |   |
  +---+-+-+---+
        |
        v
        some_method()

This kind of approach has some nice properties. In particular, we can inspect any object and retrieve its original type. In C# and Java, this also enables reflection for all objects.

A function by itself does not provide the same expected structure, and cannot be used interchangeably. If we want to use the function as an object, the natural approach is to create a wrapper class and wrapper object that then invokes the target function via a call() method or similar. In Java, this generally took the form of “anonymous classes” that could be declared and instantiated inline.

C# initially used a different approach via its “delegate” concept, that did support function pointers to be assigned but not as normal objects.

As these languages have evolved, it has become clearer that a pure OOP approach is often annoying and tedious, and that functional techniques can be quite attractive. So these languages have gotten better, more convenient syntax for this wrapper class approach, often in connection with their syntax for lambdas (anonymous functions).

• C++ has std::function to turn function objects like literal lambdas into objects.
• Java got its “functional interface” concept so that the compiler can auto-implement the necessary wrapper class when given a literal lambda or a method reference. Lambdas themselves do not have a type.
• C#'s lambdas have multiple purposes – depending on the type context the compiler turns them into objects like Action, but in other contexts like LINQ they just serve as abstract syntax – they do not necessarily represent runnable code.

There is another relevant connection between objects and functions if you consider closures – nested functions that retain access to their surrounding variables. A closure is not just a pointer to some machine code, but also needs to keep references to those surrounding variables: a closure is a combination of data + behaviour, a closure is an object. It thus makes sense to recycle the language's facilities for representing objects.

Outside of this C++-style OOP bubble, there has generally been a far more flexible approach to first-class functions. Especially in dynamic languages (e.g. Smalltalk, JavaScript, Python, Lua), Functions and methods are often just ordinary objects. In these languages, that helps keep the data model simple. On a technical level, it makes less sense to create a separate class for each function (this doesn't carry useful type information here), and we might only have a single class for all functions. This class might provide a field for the target function pointer, and for captured variables. Something like:

struct FunctionObject {
  Class* class = &FunctionObjectClass;
  Method target;
  Object* captures[];
};

Class FunctionObjectClass = {
  .methods = { &FunctionObject_call },
};

Object* FunctionObject_call(FunctionObject* self, Object** args) {
  return self->target(self->captures, args);
}

Answer 4

In later C# versions you no longer have to be explicit about delegates. You can pass a function just by stating its name.

I think it is mainly about type safety. In the compiled code there won't be an intermediate object. But on the language level one has to be explicit about the signature of any method, about the arguments and return type, so the compiler can tell if everything checks out when you invoke a method with arguments. If there would be just one function type, this would not be possible.

Your use of the term primitive is confusing to me. Primitive types are things like bytes and floats. Perhaps you mean native (to the language)?

Answer 5

The use of "primitive" is a bit too broad in the question.

In general terms, primitives are things that have simple, measurable, binary storage, without iteration or growth in allocated space (number, byte, boolean, pointer etc). These are basic building blocks of program data, supported by hardware. We can compose complex data types from these.

Considering a function type primitive, really stretches the definition above. A function is a chunk of instructions sitting somewhere in the process memory. A pointer can be used to address it and the pointer itself, can be stored as a primitive.

Apart from this distinction, the everything is an object or everything is a function is just one way of thinking about programming.

Something can be a container of data, the data itself, and can also be executed, all at the same time. JS functions are a close example of this.

My question is this: Why, from a language design point of view, do heavily object-oriented languages tend to lack the ability to reference a function on its own? Why must they be carried around by a companion object?

A function type, at compile time, in a strongly typed language, is not simple or primitive at all. Function types have to deal with signature information. There simply cannot be a single Function type for all kinds of functions one may write, without loosing the signature information.

The heavily object-oriented languages mentioned in the question have this exact shortcoming. They lack the proper ways to define and consider function signatures as first class, standalone types. In-fact, they are behind many other languages in terms of Type expression.

Languages like Typescript allow users to define pretty much any Type Signature as first class Types (aliases) without the need of any interface or class definition:

// dear compiler, this is type of a function that adds two number like things
type add<T extends number> = (x: T, y: T) => T;

// and this, type of a function that compares two compare-ables
type compare<T extends Comparable> = (x: T, Y: T) => number;

Type checking has no need to depend on Object Orient-ism of a language. Types and signatures are just constraints that compiler makes use of.

Unless Java etc go the Typescript way, they have to resort to saving the signature information glued to the enclosing type, a Class. And thus, the inevitable coupling with objects.

Still, such languages have come a long way to minimize the companion type syntax. You can declare variables to be of lambda interface types (Java example):

Supplier<String> fn1 = () -> "Hello World";
Consumer<String> fn2 = System.out::println;

But it is still a long way to go (Java 17 example):

var print = System.out::println;
// fails with: method reference needs an explicit target-type

Answer 6

Without delving into any specific language (yet!), one first reason would probably be ease of implementation (in the compiler/interpreter). It is much much more simple to have a single "internal" data type of objects to pass around, than having to have two different ones.

Functions are a relatively complex data type, compared to integers, strings and whatever "native" data types a language usually has. They have arguments, return values, possibly other attributes. All of that needs to be represented somehow. Then, for a strongly and/or statically typed language like Java, to make a function a real first class citizen, you'd have to implement some kind of inheritance concept for functions; i.e. the same way you can assign any object to a variable of the type Object, but not to a variable of the type Person, you'd probably want to have some feature of this kind for functions as well. Be it some templating scheme or whatever you can think about. Purely functional languages do not have that problem because they start with a concept for exactly this, and then model everything else around it.

Finally, there is nothing wrong with representing a function as an object! In a good object-oriented language, everything is an object, including a function, and everything will then flow together nicely. For example, in Ruby, this is exactly the case: there is nothing that is not an object. Classes are objects. Modules (stateless collections of methods) are objects. All "native" data types like integers etc. are objects. You can define functions dynamically/anonymously and pass them around like variables - because they are also objects (which happen to have a call method).

This means you can write code like this (this is an interactive session, but you could use the same code in actual programs):

# Define a method (lambda/proc in Ruby terms) and assign it to variable fn
irb(main):001:0> fn = -> (arg) { puts arg }         
=> #<Proc:0x007faf00413398@(irb):1 (lambda)>

# Call the function (there are syntactic sugars for that if one prefers to make it look like a "normal" method call)
irb(main):002:0> fn.call("hello")
hello

# Ask about the arity
irb(main):003:0> fn.arity
=> 1

# Ask where the definition (the code) is
irb(main):004:0> fn.source_location
=> ["(irb)", 1]

# Ask for the parameter definition
irb(main):005:0> fn.parameters
=> [[:req, :arg]]

# Use it in an array
irb(main):024:0> arr = [1, 2, 3, fn, 4]
=> [1, 2, 3, #<Proc:0x007faf00413398@(irb):1 (lambda)>, 4]

# Ask which class the object has
irb(main):012:0* fn.class
=> Proc

To stick with this, if you check out the documentation for the Proc class you see that there are a lot of very interesting, cool and powerful ways that open up once you make functions first-class-citizen objects (here, Proc includes a closure (or "binding" in Ruby terms). You can literally do whatever you want with it, including extending the definition of the Proc class to add more features on the fly. Or you could create an instance of class Method which is an object that represents a method call on a given object and otherwise behaves very similarly to a Proc.

If at all, I would ask why these older languages you mention do not have a full-fledged object-representation of functions and methods - and the answer would simply be that that was not en vogue when those languages were designed, or there were pre-existing constraints which would have made it too hard. Languages develop over the decades. Modern versions of Java (and presumably C#) have added more functional features, but it is completely understandable why they shied away from it at the beginning - people were still very used to the silly low-level function-pointer-hell from C or old C++.

Answer 7

Java and C# implement instance methods under the hood as being equivalent to static function with an extra hidden "this" parameter of type "Object", which will be automagically converted to the type of the object containing the method. At least in C#, it's possible to create a delegate for a static object which does not have such a parameter. The first time this is done for any particular function, the system will generate code for a dummy function which accepts the extra parameter but ignores it, and then chains to the static function. Additional requests to create delegates for the same static function will simply reuse the same wrapper. This allows delegates to be invoked without the system having to know or care about whether the target is an instance method or a static method. When invoking a static function through a delegate, calling through the wrapper would be at most slightly more expensive than examining the delegate and determining there was no need to pass "this", but testing whether a delegate invocation requires passing "this" would increase the cost of the common case where it does.

Answer 8

In Objectice-C, (blocks or closures, called lambdas in other languages) are first class objects. They can be stored in variables, arrays, sets, dictionaries, or as instance members, they can be member variables of a class, and they are reference counted.

Differences to most objects are that closures can be called (you can’t call a String, for example), and there is an optimisation in Objective-C that closures which do not actually capture anything have only one instance, with all information in static global data, and reference counting does nothing.

Apart from that, full objects. So you can write a method that tells an object which closure to call before or after any closures that it would call anyway when there is some event, by adding the closure to an array of closures to be called.

Answer 9

I think the premise is wrong, certainly for C#, but the answer is quite simple.

Addressing

If we could declare functions without some kind of containing structure, then how would we address the individual functions? How do you handle the case of importing references from multiple sources if they all define top-level functions that have identical names?

Javascript is a good example that deals with this by only allowing access to the last implementation of each function of the same name, which makes the sequence of the imports critical. But ultimately this is a very ambiguous process for the developer to know in advance what functions might be affected by this and what the correct sequence might be.

• Full disclosure, I'm not a Java developer anymore, I transitioned to C# in 2002, I can relate to Stevey's Blog Rants: Execution in the Kingdom of Nouns (steve-yegge.blogspot.com) and have enjoyed watching C# evolve. For the sake of those we've left behind I do hope that Java has kept up,

The issue is simply that in strongly typed languages we have established rules and conventions around the addressing of types and functions, one of those rules is that function must be contained within a class or type, and that class has a fully qualified namespace that can be used to reference it, by extension we can now reference that function.

In these languages, like C#, Delegates, lambdas, Closures and other forms of functional expressions can be declared but must still be assigned to a variable that is contained within a scope that is ultimately contained within a class. That class could be static and allow access to the function without instantiating an object from that class definition, but it must still be declared within the scope of a class so that we can reference it and importantly differentiate specific functions from each other that might have the same or similar names.

Answer 10

When "a function is a primitive type," IMHO you're basically going back to the bad old days of "C."

Well, a fundamental characteristic of the "object oriented" paradigm is that the surrounding infrastructure is now stronger and more aware. "Executing code" is always associated with 'some object,' variously called this or self, and is "a method or property" of that object.

If we were still to retain the idea of "a pure-function as a primitive type," we would be unleashing a bull in a china shop. We would be introducing, as they say, "out-of-band behavior." Something that isn't aware of the object-oriented runtime framework and which therefore could very easily disrupt it. (IMHO ...)