CS294 Modern Parallel Languages

Course Overview

Instructor: Kathy Yelick
Lectures: T-Th 12:30-2
Location: 310 Soda

This course will examine several modern parallel languages and a set of current research topics in parallel languages and their implementations. It will be divided into three parts:

  • Overview of specific languages
  • Cross-cutting topics
  • Final project development and presentations
It will start with a set of lectures on specific languages that cover data parallelism, task parallelism, and SPMD parallelism. We will then discuss cross-cutting topics such as effect systems, memory consistency models, and correctness properties, and have presentations by students on particular topics of interest, such as domain-specific languages, synthesis, compiling for communication-optimality, and new language ideas. Some hands-on programming exercises will be done to familiarize students with a few of the languages, followed by a final project. The course assumes familiarity with parallel computing and programming languages, and strong class participation is expected by all students and auditors.

Lectures
  1. 8/28, Lecture 1: Overview of Language Design and course
  2. 9/3, Lecture 2: NESL (reading #1)
  3. 9/5,Lecture 3: HTA (readings #2)
  4. Note: there was no lecture on Tuesday, 9/10
  5. 9/10, Lecture 4: Chapel (reading #3, and homework #1)
  6. 9/17: Lecture 5: Cilk (reading #4)
  7. 9/19: Lecture 6: X10 (reading #5)
  8. 9/24: Lecture 7: Student presentations about array langauges / libraries
  9. 9/26: Lecture 8: More presentations / discussion about array languages / libraries
  10. 10/1: Lecture 9: More on array languages / libraries
  11. 10/3: Lecture 10: iPython (Guest lecture by Fernando Perez)
  12. 10/8: Lecture 11: TItanium (Guest lecture by Amir Kamil)
  13. 10/10: Lecture 12: UPC++, etc. (Guest lecture by Yili Zheng)
  14. 10/15: Lecture 13: Models of Locality and Tasking (Guest lecture by Harsha Simhadri)
  15. 10/17: Lecture 14: Communication Avoidance
  16. 10/22: Lecture 15: Commmunication Aoviding (lecture by Edgar Solomonik)
  17. 10/24: Lecture 16: Autotuning Overview (Kathy), SEJITS
  18. 10/29: Lecture 17: Autotuning Overview (Kathy), Distributed Arrays (Amir Kamil)
  19. 10/31: Lecture 18: Software Engineering for PDEs (Anshu Dubey), Merging Simulation/Observation (Scott French)
  20. 11/4: Project proposals due
  21. 11/5-11/7: No lectures, project meetings with Kathy (Email Kathy and Tara White, tdwhite at lbl dot gov to schedule). Each project team should meet at least once with Kathy in November.
  22. 11/12: Lecture 19: Memory Consistency
  23. 11/14: Lecture 20: Correctness
  24. 11/19-11/26: No lectures, project meetings with Kathy
  25. 11/28: Thanksgiving holiday
  26. 12/3: Project meetings
  27. 12/3: Project Presentation: Scott and Edgar
  28. 12/10: Final project presentations, 10am-4pm (room TBD).
  29. 12/16: Final project reports due.

Reading assingments
  1. NESL: Nested Data parallelism
  2. HTA: Hierarchically Tiled Arrays
  3. Chapel
  4. Cilk/Cilk Plus
  5. X10
  6. Sequoia/Legion:
  7. Communication Avoiding Algorithms (and hopefully compilers)
    • So-called "HBL" paper on Communication-Avoiding Algorithms
    • 2.5D Matrix Multiply and LU paper
    • 1.5D N-Body paper

Homework and Projects
  • Homework #1: Write a sorting algorithms and a stencil (Read Black Guass-Seidell, aka GSRB) in Chapel using its data parallel operations. Due Thursday 9/12 (email to Kathy).

Staff
Lectures: Tue-Thu 12:30-2:00pm in 310 Soda
Instructor: Professor Kathy Yelick, yelick at cs dot berkeley dot edu
Office: 775 Soda Hall or (at LBNL) 50B-4245
For appointments, contact Tara White (tdwhite at lbl dot gov)