2

I have a relatively simple task where I need some 10 consumers to consume work to be produced into a queue, continuously.

This is my first time implementing this design pattern, so I have been searching the web for different approaches.

The ones I found were:

  1. Threads with Monitor.Pulse/Wait
  2. Blocking Collection
  3. Channel
  4. Dataflow

I have picked no. (1), for the sole reason I find it easier to understand and it feels it gives you more control. However, I have the feeling this is not anymore the modern/recommended way to do it (.NET Core days).

Could anyone give some guidance as to the practical advantages in choosing this/that approach over the other ?

This article covers performance comparison between some of these approaches, and states Channel ultimately performs better.

In a short, I have no special requirements such as maximum queue size. For the simplest scenario would the 1st approach be a good choice even today ? Or it is unreasonable to pick a thread-based implementation instead of a TPL or other more modern .NET threading stacks ?

Veverke
  • 383
  • 2
  • 15
  • 2
    Option 1 looks extremely tricky to me, with so much additional fiddly code and complexity which essentially re-invents the wheel in ways which the Channel implementation has already taken care of, around issues such as read/write locking, signalling and managing queues (Indeed, one of the reasons `Channel` exists is to save you from needing to write all that yourself). If you don't have any special requirements then there's no reason to roll-your own when a Channel can do all of that stuff in a single, simple line of code each for the reader (consumer) and for the writer (producer). – Ben Cottrell Jan 19 '21 at 13:00
  • @BenCottrell: thanks for the insights. I felt that (1) gives you more control and that a channel - although wraps part of this control to spare us from that work - gave me the feeling is too tailored/too specific. I guess I do not yet fully understand channel's workings and api. – Veverke Jan 19 '21 at 14:36
  • 2
    I did this several years ago, using channels containing blocking queues I wrote with threads and monitors. Nowadays, I would probably use a Channel, as all of this is already written. – Robert Harvey Jan 19 '21 at 15:35
  • 2
    You can learn more about implementation details by searching in the reference source: [System.Threading.Channels](https://github.com/dotnet/corefx/tree/master/src/System.Threading.Channels/src/System/Threading/Channels), [ConcurrentQueue](https://referencesource.microsoft.com/#mscorlib/system/Collections/Concurrent/ConcurrentQueue.cs), [BlockingCollection](https://referencesource.microsoft.com/#system/sys/system/collections/concurrent/BlockingCollection.cs,b4de4389b8938c7e) – rwong Jan 19 '21 at 16:13
  • You could also use a Semaphore here, given that it clearly conveys, what is being accomplished. From https://docs.microsoft.com/en-us/dotnet/api/system.threading.semaphore?view=net-5.0 "Limits the number of threads that can access a resource or pool of resources concurrently." – hocho Jan 20 '21 at 02:59
  • @hocho: you mean in adopting approach (1) ? Yes, there I have exclusive locks meaning threads can't read simultaneously, whereas with a semaphore this could happen, if I understand it right. – Veverke Jan 20 '21 at 08:14
  • @Veverke, if I understand your question, you want up to 10 consumers to process queue messages simultaneously. I would implement it as a thread reading the queue and creating tasks to process the messages. The semaphore is used to guarantee that at the most there are N (10 in your case) tasks processing the messages. – hocho Jan 20 '21 at 20:07
  • Interestingly, googling `c# producer consumer` returns as first result the [dataflow pattern](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/how-to-implement-a-producer-consumer-dataflow-pattern), specifically. – Veverke Feb 02 '23 at 11:07

1 Answers1

1

Don't go low-level until you need to. Hint: you don't need to.

If you're just 10-wide consuming data, you probably want ActionBlock, or one of its neighbors. This will let you set up a producer (ActionBlock.Post), and consumer (the Action passed into ActionBlock). TPL handles everything else.

Bryan Boettcher
  • 2,754
  • 4
  • 21
  • 31