Home About Courses Schedule Services Webinars Contact Search

Parallel Computing


Duration: 5.0 days

Method: Instructor led, Hands-on workshops

Price: $3200.00

Course Code: NE2209


This course covers the modern parallel and GPU programming using C# so that the computation time of large problems can be greatly reduced. It starts out with a review of models of parallel computation including shared memory, message passing, SIMD and MIMD architectures. Multithreading concepts in C# including creating and synchronizing threads, coordinating threads, monitors, semaphores, mutexes and thread pooling are presented through examples. The next part of the course covers improved parallel programming through the use of Task Parallel Library (TPL). Major concepts in TPL such as creating, cancelling and coordinating tasks are covered in detail. Different sorting, matrix, image processing, meta-heuristic and artificial intelligence algorithms and their parallelization is described. The last part of the course covers GPU programming and presents examples of algorithms that greatly benefit from the numerous cores available in a GPU.


Some background in programming in a modern language such as C#, or VB.Net, or Java or C++.


  • I. Overview of Parallel Computer Architectures
    • Shared memory and message passing architectures
    • Limitations of shared memory model
    • How CPUs are interconnected using interconnection networks
    • Overview of busses, meshes, hypercubes and product networks in creating massively parallel architectures
    • SIMD and MIMD concepts
  • II. Parallel Programming Using Multithreading
    • Overview of multithreading
    • Thread synchronization issues
    • Creating thread safe classes
    • GUI update issues from a thread
    • Producer consumer implementation
    • Mutexes
    • Semaphores
    • Wait handles
    • Events and signaling between threads
    • Thread pools
  • III. Parallel Programming Using Task Parallel Library
    • Overview of Task Parallel Library
    • Creating and starting simple tasks
    • Cancelling tasks by polling and a wait handle
    • Waiting for tasks
    • Exception handling in tasks
  • IV. Sharing Data between Tasks
    • Synchronizing execution
    • Defining critical regions
    • Defining synchronization primitives
    • Locking and monitoring tasks
    • Interlocked operations
    • Spin locks
    • Wait handles and mutexes
    • Reader-writer locks
    • Working with concurrent collections
  • V. Coordinating Tasks
    • Using Task continuations
    • One-to-many continuations
    • Selective continuations
    • Many-to-one continuations
    • Waiting for continuations
    • ManualResetEventSlim
    • AutoResetEvent,SemaphoreSlim
  • VI. Parallel Loops and Parallel LINQ
    • Setting parallel loops
    • Handling parallel loop exceptions
    • Getting loop results
    • Using PLINQ queries
    • Handling PLINQ exceptions
    • Cancelling PLINQ queries
  • VII. Examples of Parallel Algorithms and their Implementations
    • Parallel sorting algorithms
    • Parallel matrix multiplication
    • Solving equations using parallel LU decomposition
    • Parallel Techniques in Image Processing
    • Parallel Meta-heuristics
    • Parallel Genetic algorithms
    • Different strategies for selection in genetic algorithms
    • Island model and its parallel implementation
    • Swarm Intelligence and its parallel implementation
    • Ant-Colony optimization and its parallel implementation
  • VIII. GPU Programming
    • Overview of GPU architecture
    • Fundamentals of parallel programming for GPU
    • Transferring data between the main memory and the GPU memory
    • Creating threads, atomics and data synchronization in a GPU
  • IX. Examples of Parallel Computing on a GPU
    • Implementing dot products
    • Vector sums
    • Matrix multiplication
    • Fractals on a GPU