How to have two different programs with two different languages interact?

Question

I don't know if this is too broad or not, but I am a youngish programmer still in college, its my Junior year. I feel like I have a pretty good grasps for different languages and have a pretty good base. But I am stumbling to think how if for example, I am trying to create a program and say I wrote one part in python just because its easy, and does the job, but this program would need to get output from another program that I wrote in C and I am using C because of its speed. I am not sure how to have the two different programs and languages interact with each other to create an overall one total program. I am thinking of like sure you can write to a file, but then what if python and C programs both accessing a file I would need to think of locks.

Most times I have done this was with importing files into a program, but in that case they are the same language so that is easy I just use the import function, but with two languages/programs interacting to create one cohesive output I am having trouble.

I was thinking about this question because I was thinking of diving into creating some basic web applications just to learn but I have no idea how to have say javascript file interactive with something that I wrote in python or vice-versa.

I feel like I am missing something really easy here and just not understanding. Sorry if this question is too broad but I couldn't really find a clear answer online, I was trying to look through an opensource webapp, but couldn't really grasp an answer from it, again pardon me if the question seems dumb I thought this be a good place to ask I love reading on stackexchange.

Thank you for any reply.

Answer 1

Code written in different languages can interact in a number of ways.

At the source level, cross-compilation from one language into the other can be done for some combinations of languages (for example, Google's GWT includes a java-to-javascript compiler; the Glasgow Haskell compiler can compile to C; early versions of C++ compiled to C). Most of the time, however this is not really feasible.

Languages that share a virtual platform, such as the JVM or the .NET runtime, can usually interact through mechanisms exposed by the platform - for example all JVM languages can access Java libraries and use them to communicate among each other, and they can call methods and use classes created in any other JVM language.

Many programming languages, including Python, offer a mechanism to interface with native libraries, typically written in C. Using such a mechanism, it is possible to call native functions from another, more high-level, language. Popular libraries often have bindings readily available. This technique is usually referred to as a "Foreign Function Interface". The Python-into-C interface is the CFFI.

Another option is to build two completely separate programs and have them interact at runtime. There are various mechanisms to achieve this; the easiest is through a pipe (look into the subprocess module for python): basically, one program calls the other, sending input to its stdin and reading the result back from its stdout. This makes one program a subprocess of the other; if you need both to be long-lived and started independently, data can be passed back and forth through named pipes, (local) network sockets, shared files, and (depending on the platform) other means. Which one is best depends.

Answer 2

There are several different well-established ways for two programs to communicate with each other. You've already identified one obvious one, by sharing files, and one obvious difficulty with it. But there are other ways.

Most modern operating systems contain a mechanism called named pipes, which is basically a data stream with its input end in one program and its output end in a different program. If you need to send data from one program to another program running on the same computer, that can be a good method to use.

If you need to set up an API for one program to use to call the other program, you might want to look into RPC, (remote procedure calls,) which again is usually supported at the OS level.

If you want to communicate between two programs on two different computers, things get more complicated. Then you need to work with networking and protocols. You need one of the programs to be able to open a socket and listen for incoming messages, decode them into an internal method call, process them and return a response. There are plenty of libraries available for providing this functionality in various different languages, including C and Python.

Basically, what method you should use depends on the specifics of what you're trying to do. Do some research on various forms of inter-process communication and network messaging, and you should be able to figure out what would best suit your needs.

Answer 3

Message passing would seem to be the general concept here as there are various examples where one can have different languages interacting,e.g. one could use classic ASP with COM components and some JavaScript to get a mix of 3 different languages used in the same application.

In the example you state, wouldn't the locks be something for the Operating System to handle on the file system? Each program doesn't know about the other necessarily remember.

Answer 4

Most software components shouldn't communicate in terms of instructions but send and receive data: interact with a database, send/receive JSON/XML requests, etc.

Answer 5

Depending on your type of problem, the standard shell use of pipes might be sufficient (type in your shell):

program1 | program2

In this way the output of program1 goes directly to the input of program2. Of course this doesn't work for every problem, but a lot of (non-interactive) problems can be solved fine using this approach.

Answer 6

An Operating Systems course will typically cover the various mechanisms for inter-process communications (ipc). Unix/Linux offers a rich variety of ipc mechanisms.

Single (local) computer

• signals
• locks, semaphores (synchronization primitives)
• shared memory (requires synchronization)
• message queues
• named pipes
• local sockets (udp, tcp)
• shared files (requires synchronization)
• remote (local) procedure calls

Most of the above mechanisms handle communication between two processes, although shared memory can be shared between multiple processes. These mechanisms exist on a single system architecture, so messages can be transferred in binary format, or serialized.

Since languages may not share the same binary format for messages, languages will often employ serialization libraries which transform messages to and from a neutral format.

An Operating Systems course will typically cover some mechanisms for distributed communications between networked computers.

Distributed (remote) computers

• network transport (sockets) (udp, tcp)
• distributed shared memory
• remote procedure calls (rpc, orbs, grpc, etc)
• distributed key-value store (redis, memcached, dynamo, et al)
• REST, SOAP (rpc-like, transported over http(s)/sockets)
• message queue (amqp, zeromq, rabbitmq, kafka, pulsar, et al)
• network filesystem
• database

All of these mechanisms rely upon network transport between systems, and are thus built upon network transport (sockets). Since systems may have different architectures, messages may need to be serialized or transformed. Different size and byte-order for integers, different floating point encoding, even different string encoding may necessitate these transformations.