36

I've been reading about concurrency, multi-threading, and how "the free lunch is over". But I've not yet had the possibility to use MT in my job.

I'm thus looking for suggestions about what I could do to get some practice of CPU heavy MT through exercises or participation in some open-source projects.

Thanks.

Edit: I'm more interested in open-source projects that use MT for CPU-bound tasks, or simply algorithms that are interesting to implement using MT, rather than books or papers that only describe the tools like threads, mutexes and locks, or how MT can be used to have responsive GUIs...

Xavier Nodet
  • 3,684
  • 2
  • 26
  • 33

6 Answers6

16

Joseph Albahari's article on Threading in C# is one of the best resources I've seen.

The Table of Contents is below. Note that some of the topics, like the Task Parallel Library, are specific to .NET, but much of it is applicable to other languages, especially Java. 

GETTING STARTED
Introduction and Concepts
Join and Sleep
How Threading Works
Threads vs Processes
Threading’s Uses and Misuses
Creating and Starting Threads
Passing Data to a Thread
Naming Threads
Foreground vs Background
Thread Priority
Exception Handling
Thread Pooling
Thread Pooling via TPL
Thread Pooling Without TPL
Optimizing the Thread Pool
BASIC SYNCHRONIZATION
+ Synchronization Essentials
+ Locking
+ Thread Safety
+ Event Wait Handles
+ Synchronization Contexts
USING THREADS
+ Event-Based Asynch Pattern
+ BackgroundWorker
+ Interrupt and Abort
+ Safe Cancellation
+ Lazy Initialization
+ Thread-Local Storage
+ Timers
ADVANCED THREADING
+ Nonblocking Synchronization
+ Signaling with Wait and Pulse
+ The Barrier Class
+ Reader/Writer Locks
+ Suspend and Resume
+ Aborting Threads
PARALLEL PROGRAMMING
+ Parallel Programming
+ Why PFX?
+ PLINQ
+ The Parallel Class
+ Task Parallelism
+ Working with AggregateException
+ Concurrent Collections
+ SpinLock and SpinWait

You can also have a look at Jon Skeet's tutorial here: http://www.yoda.arachsys.com/csharp/threads/

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
  • 3
    I add the Jon Skeet's c# mutlithreading page to your answer: http://www.yoda.arachsys.com/csharp/threads/ –  Dec 31 '10 at 00:31
  • 1
    Second that, liked Albahari's 5 chapters extremely helpful. To practice, here's a simple exercise - Create a multiplayer tic-tac-toe game using WCF (if you know), and a simple UI - try updating the UI controls with responses from each player. Have you checked firefox code? – Narayana Jan 03 '11 at 19:06
  • would you mind explaining more on what it does and why do you recommend it as answering the question asked? ["Link-only answers"](http://meta.stackoverflow.com/tags/link-only-answers/info "what's this") are not quite welcome at Stack Exchange – gnat May 12 '13 at 21:50
  • 1
    @gnat: I pasted the table of contents into my answer. – Robert Harvey May 13 '13 at 00:20
  • The link to Jon Skeet's tutorial didn't seem to be working (I was getting an Azure 404 page) - here's an archive of it: https://web.archive.org/web/20181010053742/http://www.yoda.arachsys.com/csharp/threads/ – ArtOfWarfare Jun 04 '19 at 17:36
14

Java Concurrency in Practice is one of the best books about multi-threading and concurrency. Although all the examples in the book are Java based, this book gives a solid explanation of MT world. It helped me a lot when I was developing a M-T system.

Sorantis
  • 2,720
  • 17
  • 24
11

Chapter 11 of the book Intel Threading Building Blocks by James Reinders is devoted to examples of algorithms and projects that make use of Parallel Computing (or Parallel Programming): a substring finder, the Game of Life, a Sieve of Eratosthenes, Matrix Multiply, and then other more advanced topics like network packet filtering and games.

Xavier Nodet
  • 3,684
  • 2
  • 26
  • 33
4

I found Concurrent Programming on Windows by Joe Duffy to be very helpful. There's a lot of depth. It doesn't pull any punches, so you really get a good feel for how many ways there are to shoot yourself in the foot. It helped me to be cautious, which is the best advice I can give anyone starting with MT apps.

Scott Whitlock
  • 21,874
  • 5
  • 60
  • 88
2

There's a difference between concurrency and parallelism. Concurrency is the act of doing more than one thing at a time, like writing to 2 files. Parallelism is the act of speeding up programs by using multiple cores.

Although there's no free lunch when it comes to concurrency, in parallelism the lunch is certainly becoming more free, see developments like http://www.haskell.org/haskellwiki/GHC/Data_Parallel_Haskell

dan_waterworth
  • 7,287
  • 2
  • 34
  • 45
  • 2
    Your distinction is incorrect. Concurrency is the act of completing multiple tasks in overlapping time periods **while not necessarily doing more than one thing at any given time**. Parallelism is the act of doing more than one thing at any given time. – Asad Saeeduddin Nov 12 '14 at 15:39
  • If they overlap, you are in the process of doing more than one thing at a time. – dan_waterworth Nov 17 '14 at 10:31
  • If their start and end times overlap, this does not indicate that both tasks are proceeding simultaneously at any given time. Given two lists of integers to sum, either you can get two people to sum each list independently and simultaneously, which is both concurrent and parallel, or you can sit down and alternate between summing up the entries in one list and the entries in the other list, which is concurrent but not parallel. – Asad Saeeduddin Nov 17 '14 at 14:57
  • I didn't say they were happening simultaneously. I said doing more than one thing at a time. – dan_waterworth Nov 20 '14 at 18:21
  • Doing more than one thing at a time is the same as doing two things simultaneously. If multiple things are occurring at a given time, they are occurring simultaneously. – Asad Saeeduddin Nov 20 '14 at 18:43
  • No it's not. If I told you I was learning French and German, you wouldn't assume I was learning them simultaneously, but I would be learning them both at the same time. – dan_waterworth Nov 21 '14 at 14:39
  • Actually, that's not quite the problem here. The problem is that in English, the progressive tense is ambiguous. If I say "I am learning French". I might mean "I am taking French lessons" or "I am acquiring knowledge of the French language right now". The same ambiguity exists with "the executing threads". Here's the thing; I know the difference between concurrency and parallelism and I did when I wrote this answer. I may not have been clear and I don't mind being corrected on that, but the possibility that you misunderstood me was never considered and you jumped straight to "You're wrong". – dan_waterworth Nov 21 '14 at 15:31
  • If you told me you were learning "French and German at a time" (which is a horrible example because it makes no sense and no one would say that) then it would be quite reasonable of me to assume that you were learning them simultaneously. The possibility I misunderstood you didn't occur to me because what you wrote wasn't opaque or difficult to understand. The obvious interpretation of what you were saying was quite straightforward, and was incorrect, so that was the conclusion I "jumped" to. – Asad Saeeduddin Nov 21 '14 at 17:07
  • Regarding you knowing the difference between concurrency and parallelism, that's great, whatever you say. However, the purpose of an answer here isn't to demonstrate your own knowledge, it is to help people who **don't** know the difference, and can be confused by ambiguity. – Asad Saeeduddin Nov 21 '14 at 17:13
  • The question I'm asking myself, here, is how does drawing this distinction help us answer the original question? Do we not get the gist of the question without drawing a distinction between parallelism and concurrency? – Josiah Jan 16 '16 at 18:17
1

This site has some good project examples in general. www.planet-source-code.com

Just pick a language and search for multi-threading. you should see a number of projects with source code available.

Pemdas
  • 5,385
  • 3
  • 21
  • 41
  • I'm surprised this hasn't been up-voted yet. This is the only answer I saw which focuses on the question "how to PRACTICE parallel & concurrent programming". Every other answer addresses the question "how to learn the concepts", which is a different question. – Josiah Jan 16 '16 at 18:18