4

Based on the discussion around this question. I'd like to know how .so files/the ELF format/the gcc toolchain avoid problems passing classes defined purely in header files (like the std library). According to Jan in that answer, the dynamic linker/loader only picks one version of such a class to load if its defined in two .so files. So if two .so files have two definitions, perhaps with different compiler options/etc, the dynamic linker can pick one to use.

Is this correct? How does this work with inlining? For example, MSVC inlines templates aggressively. This makes the solution I describe above untenable for dlls. Does Gcc never inline header-only templates like the std library as MSVC does? If so wouldn't that make the functionality of ELF described above ineffective in these cases?

Community
  • 1
Doug T.
  • 11,642
  • 5
  • 43
  • 69

2 Answers2

7

What the ELF linker does is that it ensures that static and dynamic linker produce the same result. So using shared objects does not create any new problems. You can create some if you use of non-default visibility or version or linker scripts, but you obviously wouldn't do that if it isn't safe in that particular case.

Now if you use incompatible sets of compiler flags for building different objects, whether shared or static, in one application, you will have problems. You can have two kinds of problems:

  • The flags are so incompatible that calling convention, structure layout or some other such parameter is different between the modules. Obviously the only solution is to know which compiler options always have to remain set to the platform defaults.

  • The flags modify content of the headers and thus violate one definition rule. Authors of standard libraries, at least the open-source ones, know well how to avoid breaking compatibility between objects. But if you have some special library who's author were not careful, you can get in trouble and will have to make sure the compilation flags are compatible.

In Linux I've not seen these problems happen. Everybody knows not to touch the compiler flags that could break anything and there are no special incompatible versions for debug or such (gcc can emit debug information in optimizing build, so you normally just get unstripped version of the same object or lately even just the debug information for the normal object split out to separate file).

This is different from Windows, where:

  • Due to the way shared libraries need explicit export and import there will not merge symbols (like template instances or class impedimenta) generated into different shared libraries.
  • Have separate debug and release runtimes that differ in both the shared library used and lot of preprocessor magic. It means that you can't link your debug build to libraries built with the release version, so every library needs to ship debug and release variant. And often variant for static and dynamic linking. And despite all this trouble I don't think the debug support on Windows is better; e.g. Linux libc has hooks for replacing allocator (with debug one) which is always gross hack on Windows and Linux has great tools like Valgrind.
  • Have lots of compiler flags that get flipped for various compatibility reasons, causing more complications. It has to be said that Microsoft is in more difficult position here. Since they cater primarily for the closed-source programmers, they have to assume things can't be recompiled and have to provide various compatibility kludges to keep things working. Linux people just assume things can be recompiled and do a breaking change when things get too complicated, like the switch to ELF format was; The ELF format was created between 1997 and 1999, while Windows objects are still backward compatible to the old Windows 3.1 ones. At the cost of all the mess.
Jan Hudec
  • 18,250
  • 1
  • 39
  • 62
  • Considering how easy it is to work around having dlls in old formats lying around in Windows (just spawn a thread and invoke a program that can load the dll and communicate with it over a pipe of similar), I really don't see the fuss windows programmers keep making. They should be doing this anyway. It will even avoid a whole score of problems with x86/x86-64 that windows people are *still* fighting, now, 17 years after x86-64 was released. – Clearer Oct 18 '17 at 20:08
  • @Clearer, it would definitely *not* call that easy; creating interprocess interfaces is a *lot* of work. That said, Microsoft essentially started moving in that direction with the new WinRT APIs based on COM. That does make them much easier to update in a compatible way. – Jan Hudec Oct 19 '17 at 10:16
  • I certainly would consider it easy; just plug whatever functions you need to call behind a simple web service. We all write them all the time and there are a ton of tools that make this kind of thing trivial. – Clearer Oct 22 '17 at 18:00
  • @Clearer, you can do that if the functions are kind of independent and you still have to write the wsdl-or-whatever-idl declarations and generate the glue and you have to serialize all the argument and return types and any shared state is yet another huge barrel of worms. Don't forget we are *not* talking about .NET here—we are talking about plain old C with data types being mess of pointers with complex lifetime requirements. And no, there are **no tools** for **that** – all that ton of tools is for higher level languages. C has nothing. – Jan Hudec Oct 22 '17 at 19:04
1

I'm not sure I understand. If you build parts of your program with different incompatible flags, it's not a .so or .dll problems, the same problems will appear with static libraries as well.

First note that I stopped programming for Windows in 94. So nearly all my understanding about Windows behavior comes from the net and discussions with colleagues working on Windows and may be outdated, misremembered badly understood facts.

  • It is customary on Windows to include the standard library in .DLL. That's not the case on Unix (and I get it isn't mandatory on Windows).

  • Default symbol resolution on Windows prefer the DLL, on Unix it will be the normal search (in both, you may changing it).

  • It's considered bad practice on Unix to make debug libraries which aren't substitutable for the optimized one. Yes this limit the instrumentation which can be done, but it make a lot of thing easier. We may do special instrumented libraries which need a total recompilation when they are worthwhile, but it's not the case for routine debug builds. My understanding is that MSC++ compiler ships with some libraries which aren't substitutable if you compile user code in debug or optimized (I've heard horror stories with #defining new for instance, which seems to me a very bad idea even for special instrumented builds). Nothing of Windows forces you to do that bad practice (but if the standard C++ library is in that case, you probably have to use a saner standard library).

AProgrammer
  • 10,404
  • 1
  • 30
  • 45
  • On Linux (other Unices may differ) standard library is always .so and it can be considered mandatory. It's required by packaging guidelines of most distributions and for legal requirement for closed-source applications, since the standard libraries are LGPL. – Jan Hudec Nov 22 '12 at 06:48