Information Hiding
What is the advantage of returning a pointer to a structure as opposed
to returning the whole structure in the return statement of the
function?
The most common one is information hiding. C doesn't have, say, the ability to make fields of a struct private, let alone provide methods to access them.
So if you want to forcefully prevent developers from being able to see and tamper with the contents of a pointee, like FILE, then the one and only way is to prevent them from being exposed to its definition by treating the pointer as opaque whose pointee size and definition are unknown to the outside world. The definition of FILE will then only be visible to those implementing the operations that require its definition, like fopen, while only the structure declaration will be visible to the public header.
Binary Compatibility
Hiding the structure definition can also help provide breathing room to preserve binary compatibility in dylib APIs. It allows the library implementers to change the fields in the opaque structure without breaking binary compatibility with those who use the library, since the nature of their code only needs to know what they can do with the structure, not how large it is or what fields it has.
As an example, I can actually run some ancient programs built during the Windows 95 era today (not always perfectly, but surprisingly many still work). Chances are that some of the code for those ancient binaries used opaque pointers to structures whose size and contents have changed from the Windows 95 era. Yet the programs continue to work in new versions of windows since they weren't exposed to the contents of those structures. When working on a library where binary compatibility is important, what the client isn't exposed to is generally allowed to change without breaking backwards compatibility.
Efficiency
Returning a full structure that is NULL would be harder I suppose or
less efficient. Is this a valid reason?
It is typically less efficient assuming the type can practically fit and be allocated on the stack unless there's typically a much less generalized memory allocator being used behind the scenes than malloc, like a fixed-sized rather than variable-sized allocator pooling memory already allocated. It's a safety trade-off in this case, most likely, to allow the library developers to maintain invariants (conceptual guarantees) related to FILE.
It's not such a valid reason at least from a performance standpoint to make fopen return a pointer since the only reason it'd return NULL is on failure to open a file. That would be optimizing an exceptional scenario in exchange for slowing down all common-case execution paths. There might be a valid productivity reason in some cases to make designs more straightforward to make them return pointers to allow NULL to be returned on some post-condition.
For file operations, the overhead is relatively quite trivial compared to the file operations themselves, and the manual need to fclose cannot be avoided anyway. So it's not like we can save the client the hassle of freeing (closing) the resource by exposing the definition of FILE and returning it by value in fopen or expect much of a performance boost given the relative cost of the file operations themselves to avoid a heap allocation.
Hotspots and Fixes
For other cases though, I've profiled a lot of wasteful C code in legacy codebases with hotspots in malloc and needless compulsory cache misses as a result of using this practice too frequently with opaque pointers and allocating too many things needlessly on the heap, sometimes in big loops.
An alternative practice I use instead is to expose structure definitions, even if the client is not meant to tamper them, by using a naming convention standard to communicate that no one else should touch the fields:
struct Foo
{
/* priv_* indicates that you shouldn't tamper with these fields! */
int priv_internal_field;
int priv_other_one;
};
struct Foo foo_create(void);
void foo_destroy(struct Foo* foo);
void foo_something(struct Foo* foo);
If there are binary compatibility concerns in the future, then I've found it good enough to just superfluously reserve some extra space for future purposes, like so:
struct Foo
{
/* priv_* indicates that you shouldn't tamper with these fields! */
int priv_internal_field;
int priv_other_one;
/* reserved for possible future uses (emergency backup plan).
currently just set to null. */
void* priv_reserved;
};
That reserved space is a bit wasteful but can be a life saver if we find in the future that we need to add some more data to Foo without breaking the binaries that use our library.
In my opinion information hiding and binary compatibility is typically the only decent reason to only allow heap allocation of structures besides variable-length structs (which would always require it, or at least be a little bit awkward to use otherwise if the client had to allocate memory on the stack in a VLA fashion to allocate the VLS). Even large structs are often cheaper to return by value if that means the software working much more with the hot memory on the stack. And even if they weren't cheaper to return by value on creation, one could simply do this:
int foo_create(struct Foo* foo);
...
/* In the client code: */
struct Foo foo;
if (foo_create(&foo))
{
foo_something(&foo);
foo_destroy(&foo);
}
... to initialize Foo from the stack without the possibility of a superfluous copy. Or the client even has the freedom to allocate Foo on the heap if they want to for some reason.
