Parallel computing in Julia


Tuesday, June 14  &  Thursday, June 16
9:30am–12:30pm Pacific Time

This course will start at 9:30am Pacific Time and will run until 12:30pm Pacific Time. Its format will be a combination of several interactive Zoom sessions and the reading materials in-between the Zoom sessions. Course materials will be added here shortly before the start of the course.

Julia is a high-level programming language well suited for scientific computing and data science. Just-in-time compilation, among other things, makes Julia really fast yet interactive. For heavy computations, Julia supports multi-threaded and multi-process parallelism, both natively and via a number of external packages. It also supports memory arrays distributed across multiple processes either on the same or different nodes. In this hands-on workshop, we will start with a quick review of Julia’s multi-threading features but will focus primarily on Distributed standard library and its large array of tools. We will demo parallelization using two problems: a slowly converging series and a Julia set. We will run examples on a multi-core laptop and an HPC cluster.

Instructors: Alex Razoumov (SFU) & Marie-Hélène Burle (SFU)

Prerequisites: working knowledge of serial Julia (covered in our Julia course) and familiarity with Compute Canada’s HPC cluster environment, in particular, with the Slurm scheduler (covered in our HPC course).

Software: All attendees will need a remote secure shell (SSH) client installed on their computer in order to participate in the course exercises. On Windows we recommend the free Home Edition of MobaXterm. On Mac and Linux computers SSH is usually pre-installed (try typing ssh in a terminal to make sure it is there). No need to install Julia on your computer.


Zoom   Day 1 - 9:30am-12:30pm Pacific
Introduction to Julia language
Intro to parallelism
Multi-threading with Base.Threads (slow series)
Multi-threading with ThreadsX (slow series)
Parallelizing the Julia set with Base.Threads
Parallelizing the Julia set with ThreadsX


Zoom   Day 2 - 9:30am-12:30pm Pacific
Distributed.jl: basics
Distributed.jl: three scalable versions of the slow series
DistributedArrays.jl
Parallelizing the Julia set with DistributedArrays
SharedArrays.jl
Parallelizing the N-body problem (supplemental material)
Parallelizing the additive Schwarz method (supplemental material)