Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [multiprocessing]

30 questions
20
votes
5 answers

Can multiple CPU's / cores access the same RAM simultaneously?

This is what I guess would happen: If two cores tried to access the same address in RAM, one would have to wait for the other to access the RAM. The second time that each core would try to access the same address, they may still have that RAM…
Lost Hobbit
  • 303
  • 1
  • 2
  • 8
10
votes
3 answers

Python Multiprocessing with Queue vs ZeroMQ IPC

I am busy writing a Python application using ZeroMQ and implementing a variation of the Majordomo pattern as described in the ZGuide. I have a broker as an intermediary between a set of workers and clients. I want to do some extensive logging for…
Imraan
  • 201
  • 2
  • 5
6
votes
5 answers

Performance: Is There a Reason to Use Processes over Threads?

This is a general question but I work in the .NET world so I'd like to know if there any specific quirks about the .NET Framework / Core platforms that I should be concerned about here. I think it's safe to say that as a general rule, breaking…
Christian Findlay
  • 279
  • 1
  • 10
6
votes
1 answer

Python: Control subprocess from calling thread

I am working on a database interface to a MongoDB using Python Eve as an API. The database stores several documents containing parameters and values needed for some calculations. I wrote a Python module containing all the calculation routines which…
albert
  • 163
  • 7
5
votes
2 answers

are multithreading, multiprocessing, multitasking implemented at instruction set or OS?

On a computer with a single cpu core, must a multithreading program be implemented based on an OS? Can it be implemented in the instruction set and run without an OS? Can multiple programs run in multitasking way without an OS? Can multitasking…
Tim
  • 5,405
  • 7
  • 48
  • 84
4
votes
2 answers

How can a web browser simultaneously run more videos than the number of CPU cores?

I've been learning about threads and processes and I understand that threads allow you to write concurrent code: where executing threads switch very quickly from one to another, giving the impression that multiple programs are running in parallel…
Avantgarde
  • 151
  • 3
4
votes
1 answer

Multiprocess RPC Architecture Design

I am currently working on a project that has client applications communicating with a server process. The client applications could be local to the same machine as the server process, on the same network, or over the internet. I had written a…
Nonce
  • 43
  • 5
3
votes
2 answers

Difference between multithreaded programs on single and multiple cores

When programming using (pre-emptive) multithreading model on a single processor, we need to deal with synchronization, deadlocks, etc. Do any additional concerns appear (and correspondingly, new techniques used) when switching to multithreading on…
max
  • 1,075
  • 11
  • 19
3
votes
1 answer

How to efficiently implement this background processing chain?

I am working on an audio software that uses The EchoNest web service to identify and retrieve metadata about audio songs and I would like to have some advice on implementing a background processing chain. (to get the full picture, I'll show you what…
aybe
  • 727
  • 6
  • 16
3
votes
3 answers

Opinions on logging in multiprocess applications

We have written an application that spawns at least 9 parallel processes. All processes generate a lot of logging information. Currently we are using Python’s QueueHandler to consolidate all logs into one file. Unfortunately, this sometimes results…
chkorn
  • 33
  • 1
  • 3
2
votes
2 answers

Having tasks that can be ran individually or "in trees", and each task can be blocking or non-blocking

I'm creating a framework that needs to execute certain tasks, I'll use building a car as an example for this. So, an individual task could be something like weld metal or screw bolt. But there are also collections of tasks, I call them jobs. Each…
Markus Meskanen
  • 390
  • 1
  • 10
2
votes
1 answer

How to handle multi-processing of libraries which already spawn sub-processes?

I am having some trouble coming up with a good solution to limit sub-processes in a script which uses a multi-processed library and the script itself is also multi-processed. Both, the library and script are modifiable by us. I believe the question…
exhuma
  • 1,191
  • 1
  • 10
  • 16
1
vote
0 answers

Inter-Process Communication: Stream data from single writer process to multiple reader processes

tl;dr - I want to stream text data from one writer process to multiple reader processes. I'm thinking of using a file to achieve this. Is it a good idea? Using a file would avoid having to maintain client connections in the server and send data on…
Catherine
  • 19
  • 3
1
vote
0 answers

Approach to setup multi-tenant RabbitMQ workers in Python

I'm migrating my current application to multi-tenant setup. Now I've multiple rabbitmq workers to process async job, publish and consume integration events, and other stuffs. I'm planning to use vhosts in RMQ (1 vhost per tenant per service).…
Rohit Jain
  • 167
  • 1
  • 10
1
vote
1 answer

System design : Implementing a common service process with a http server

I have a huge DB,and CalculateTasksFromDB() takes a long time (and lots of memory). Once that method is done,there is a huge list of tasks. There are worker processes in the system (at any point in time the system might have zero to 100 of…
yoni keren
  • 139
  • 3
1
2