11

I'm a C++ developer and in an attempt to better understand cross-platform development, I'm trying to get a better understanding of some implementation details of compilers and how exactly they create OS specific binaries. In the midst of my study I realized that, at least for a while, most compilers you downloaded for a specific platform only compiled binaries for that platform. So if you downloaded an IDE that came with a compiler exe for Windows, then that compiler would only be able to compile your program for x86-x64 windows applications and not Linux or Mac applications.

Now I understand that different platforms require different binary formats, but what makes it difficult for say the visual C++ compiler on windows to generate a linux binary executable file? As long as you have the assembly instructions for the CPU your running on, as well as the OS specific libraries, shouldn't you be able to compile exectuables for any platform on any machine?

Jason
  • 459
  • 1
  • 4
  • 10
  • 5
    There are many cross-compilers: I am not sure why you think cross-compiling is uncommon. Visual C++ makes sense given Microsoft wants Windows lock-in, but they even provide Android compiler tools in VS2017. –  Mar 23 '17 at 02:09
  • Well a lot of other sites seemed to make it out like a cross compilation compiler is very difficult to implement and only until recently have more cross-platform compilers sprung about. If this is true then I'm just wondering what would make cross compilation difficult if you only need certain CPU assembly instructions and native systems calls for corresponding OS – Jason Mar 23 '17 at 02:15
  • I think you have a terminology understanding problem here that may be confusing. You're using the tend "cross compiler' and " cross-platform compiler " interchangeably, but they're different things (albeit related): a cross compiler is a compiler for a platform that is different to the one you're using, and such things have been available for about as long as compilers have. A cross-platform compiler is a compiler that uses an intermediate step to separate the processing and optimization of the source language from the platform specific code generation so that it can be used ... – Jules Mar 23 '17 at 11:01
  • ... for compiling code for multiple target platforms with minimal changes. Such things are a more recent innovation and much more complex. – Jules Mar 23 '17 at 11:03

3 Answers3

18

what makes it difficult for say the visual C++ compiler on windows to generate a linux binary executable file?

Other than an unwillingness to do that on Microsoft's part, absolutely nothing. The obstacles aren't technical.

Development toolchains are just programs that take input and produce output. Visual C++ produces x86 assembly and then uses an assembler to convert that into a COFF object file. If Microsoft wanted to make it generate ELF instead, it's just code: assembly comes in, ELF goes out. There's nothing magic about object files or libraries; they're just blobs of data in a well-understood format.

Way back in the stone age, cross compilation was a lot more difficult because more often than not, you would have been writing the tool chain for your target platform in assembly for the platform where it would run. This meant that if all there was in the world were the VAX, M68K, and Alpha architectures, a full suite of cross-compilers would require writing nine of them, mostly from scratch. (VAX-to-VAX, VAX-to-M68K, VAX-to-Alpha, M68K-to-VAX, M68K-to-M68K, etc.) That's a bit of an exaggeration since parts of the VAX compiler could be reused and attached to code generators for each target (e.g., VAX, M68K and Alpha, each written for VAX.)

That problem went away when we started writing compilers in a language that wasn't tied to a specific processor, such a C. Going that route means you write the entire toolchain once in C and use a written-for-the-local-platform C compiler to build it. (You'd often use the compiler to recompile itself after it had been bootstrapped on the local platform's compiler, but that's another discussion.) The upshot of this is that building a cross-compiler became essentially the same effort as building a native compiler on the local platform. The only significant difference is that somewhere in the build process, you told it to compile in the code generator for your target platform instead of the one for the local platform, which would have been the logical choice. GCC did (and still does) this by building one binary per local/target platform pair.

As the architecture of compilers evolved, it became convenient to simply include and build all of the code generators with the product and select which one gets used at runtime. Clang/LLVM does this, and I'm sure there are others.

Once you have a working toolchain (compiler, assembler, linker), libraries get built from sources and eventually you end up with everything you need to produce an executable file for some other platform.

Blrfl
  • 20,235
  • 2
  • 49
  • 75
  • This is now the best, most in-depth answer. I think history really helps to understand the context. – Jason Mar 23 '17 at 03:18
  • 3
    @Jason Sometimes it pays to be old. :-) – Blrfl Mar 23 '17 at 03:19
  • 4
    "Other than an unwillingness to do that on Microsoft's part, absolutely nothing." – I wouldn't call it "unwillingness". Microsoft is a publicly-traded profit-oriented business; they have certain responsibilities to their shareholders and stakeholders. They would need to hire, train, **and pay** developers for the Linux backend, they would need to hire, train, **and pay** testers for the Linux backend, they would need to hire, train, **and pay** support staff familiar with the Linux backend, they would need to design, develop, maintain, support, and extend the code, all for a platform that is … – Jörg W Mittag Mar 23 '17 at 09:46
  • … outside their core business. And all of that just to add a n+1th compiler to the already existing n compilers that you could just as well use. What would be the **business benefit** for Microsoft to compete with GCC, Clang, ICC (Intel), xlc (IBM), Digital Mars, tcc, pcc, TenDRA, Metrowerks, PathScale, …? Personally, I don't think there is any. – Jörg W Mittag Mar 23 '17 at 09:49
  • 4
    @JörgWMittag `What would be the business benefit ... I don't think there is any.` -- That sounds like a good reason to be unwilling to do it. – Blrfl Mar 23 '17 at 10:28
  • @Blrfl given that Android is a Linux ( in that it has the Linux kernel, though not all of the GNU tools and libraries many other Linux distributions bundle with them), and Microsoft Visual C++ can target Android (ARM or x86 - https://msdn.microsoft.com/en-us/library/ms235435.aspx ) the premise seems a bit wrong, and Microsoft already do the work to generate ELF. AFAIK, the main difference is the libraries linked to, not the format http://stackoverflow.com/questions/19280391/can-a-shared-library-so-file-built-for-linux-be-included-linked-and-used-in-a#answer-19280602 – Pete Kirkham Mar 24 '17 at 12:22
  • 1
    @PeteKirkham I was pretty much unaware that Redmond had come to its senses because I don't use that platform. Wasn't that long ago that if it wasn't Wintel, they wouldn't support it. Even if the OP's premise that VC++ doesn't do that is off, it's still a good question. – Blrfl Mar 24 '17 at 14:31
  • 2
    @JörgWMittag It's unwillingness. The reasons for that unwillingness are irrelevant. I should add, though, that this weird belief amongst Americans that every company makes every single decision because of some fiduciary duty is incredibly naive. – Miles Rout Mar 28 '17 at 21:42
8

Yes, if you have all of the info about your target platform then it should not matter what platform you're actually running on.

There's two problems that tend to crop up:

  1. People don't focus on it because it's a less common scenario. Often the only thing you cross compile is a compiler so you can then stop cross compiling. Less focus means worse support.
  2. Non-trivial programs need more than just code. Dealing with library inclusion/linkage gets a bit easier when you've got libraries for the platform you're running on. They'll be in a well known location, in a well known encoding.

Those aren't insurmountable of course. Mostly, you get compilers that target the platform they run on because that's what people want.

Telastyn
  • 108,850
  • 29
  • 239
  • 365
2

I disagree with your premise. There are millions of Android and iOS developers. And they all use compilers running on Windows or a Mac, producing code for an entirely different computer.

You won't get a cross compiler if there is no market demand for it. People developing code for a Linux desktop for example mostly have a Linux desktop available and will use a Linux based compiler - much faster if you can run your application directly on the machine where it is compiled without transporting it over the network, much easier and faster to have a debugger running on the same machine and so on.

So how much more money would Microsoft make if their compilers would build for Linux as well? Approximately $0. How much more software for Windows would be created? None. How much more Linux software would be created? No idea, but it's not something that Microsoft cares about. What would be the cost? Quite considerably. Compilers must be bug free. They must be tested.

Another problem: If you write a compiler writing on Windows you need someone who knows how to write Windows software. If you write a compiler for Linux you need someone who knows how to write Linux software. If you write a compiler for Linux running on Windows, suddenly you need the much more rare bread of developer who knows both Windows and Linux.

gnasher729
  • 42,090
  • 4
  • 59
  • 119
  • "There are millions of Android and iOS developers. And they all use compilers running on Windows or a Mac, producing code for an entirely different computer." Um no, not they don't. Many, many Android developers use Linux, and in fact it's the most popular platform for developers according to StackExchange's own survey. – Miles Rout Mar 28 '17 at 21:43