CSC 770: Parallel Computing (syllabus)

Instructor:
Dr. Yumei Huo
Office: / 1N-202
Tel.: / (718) 982-2841
EMail: /
WWW: /
Schedule:

This course meets for 2 hours per week: two hours of lecture and 1 regularly scheduled conference hour.Attendance is expected for all class hours.The conference hour will be used as supplemental independent study.

Class hours / Thursday / 8:20pm - 10:00pm / Room:1N111
Office Hours: / Tuesday / 1:15pm - 2:15pm
Thursday / 1:15pm - 3:15pm
or by special appointment
Contents:

In this course students will learn about the foundations of parallel computing. The emphasis will be on algorithms that can be used on shared- and distributed-memory systems. The course will include both a theoretical component and a programming component. The topics covered will encompass on fundamentals of parallel computing, parallel computer architectures, performance, communication, decomposition techniques for parallel algorithms, parallel programming models such as OpenMP and MPI models, analytical modeling of parallel programs, algorithms and languages. Appropriate examples of existing or proposed parallel architectures will be surveyed as well as recent advances in parallel algorithms for scientific computing. Specific parallel algorithms for solving scientific problems and their implementation on parallel machines related to numerical analysis, scientific applications, runtime environments, performance analysis will be discussed.

Prerequisite: Knowledge of C or C++; CSC 326

This course assumes familiarity with data structures. You will need to be able to program in C/C++ to perform the assignments in the course.

Textbook:

Required textbook: An Introduction to Parallel Computing: Design and Analysis of Algorithms, Second Edition by A. Grama, V. Kumar et al. Pearson Addison Wesley. ISBN: 0201648652.

Reference:

  1. Using MPI - 2nd Edition: Portable Parallel Programming with the Message Passing Interface (Scientic and Engineering Computation) by William Gropp, Ewing Lusk, Anthony Skjellum. MIT Press; 2nd edition (November 26, 1999), ISBN: 0262571323.
  2. Parallel Programming with MPI by Peter S. Pacheco. Morgan Kaufmann; 1st edition (October 1, 1996), ISBN-10: 1558603395 ISBN-13: 978-1558603394

Homeworks:

There are four homework. Homeworks will focus on:

  1. review some important concepts and formulas covered in the class.
  2. extend or modify the algorithms to solve the variants of the problems covered in the class.
  3. design new algorithms for new problems using the techniques covered in the class.

Homework assignments will be assigned during class and also posted on the course Web-page:

Programming Assignments:

There will be two programming assignments. The tasks will be designing and implementing parallel algorithms, compiling and running the programs in the environment of CSI’s cluster, evaluating a solution, and submitting the program and report. Through the programming assignments, the following goals are expected:

  1. Students can solve the problems using the techniques introduced in the class.
  2. Students can implement their parallel programs using MPI programming language on CSI High Performance Center’s Linux cluster.
  3. Students will learn more about doing research in general.

For programming assignment 1, student must work independently.

For programming assignment 2, students can work in groups of size up to two; members of a group will receive the same grade on the project.

You will use CSI’s Linux cluster to do your programming assignments. Basic Information about CSI High Performance Center’s Cluster can be found at

In order to access CSIHigh Performance Center’s cluster:

You will need to use SSH to access CSI’s cluster

You can download SSH from the link: You will also need an account to access CSI’s cluster. You will get your account within first two weeks.

You will also need an account to access CSI’s cluster. You will get your account within first two weeks.

You are strongly encouraged to set up the programming environment and try to be familiar with CSI’s cluster before your first programming assignment.

The solution to each programming assignment may include the source code and a report showing the result.

The source code MUST be turned in by email to before midnight the date they are due.

The report MUST be submitted either per email or be handed in a hard-copy at the beginning of the class on the due date.

Handwritten or typed report will be accepted.

Solutions must be readable(especially handwriting!!!), clear, concise and complete.

Grading and Policies:
  • The exams may test on material covered only in class and on material covered only in the reading assignments.
    Your grade will be based approximately, as follows. These percentages are tentative and subject to change.

20% - Homeworks

25% - Midterm Exam1

25% - Midterm Exam2

30% - Programming Assignments

  • Class participation is essential to succeed in this course.
  • Homework assignments and programming assignments are due at the start of class on their due date. If you think you won't be present in class on that Monday, turn the homework in earlier at instructor's office or by email.
  • Each homework or each programming assignment will receive a maximum of 100 points for full credit.
    Homework or programming assignment submitted up to 1 week late will receive a maximum of 80 points
    Homework or programming assignment submitted up to 2 weeks late will receive a maximum of 60 points.
    NO Homework or programming assignment will be accepted later than two weeks after due date.
  • Check the marks in a homework, programming assignments or an exam and report errors promptly. Make sure you report such problems to the instructor within four weeks from receipt but no later than Dec.6, 2016. If you believe a grade you received is not representative of your effort talk to the instructor.
  • The work you turn in MUST BE your own personal work, composed and written by you. No plagiarism. Otherwise,you will get the grade of zero for the homework or project.
  • DO NOT OBTAIN YOUR SOLUTION THROUGH THE INTERNET. Or you will get the grade of zero for the homework or project.
  • Collaboration of any kind is NOT allowed in the in-class exams (midterms, and final).
  • DO NOT USE pencils to write down your solutions for the homework, programming assignments or exams; if you decide, however, to use a pencil do not complain about grading.

Specific contents (subject to change):

Introduction to Parallel Computing

Parallel Programming Platforms

Analytical Modeling of Parallel Programs

Decomposition Techniques for Parallel Algorithms

MPI programming language

PRAM algorithms

Architecture Independent Algorithm Design and Time analysis

Parallel Algorithm on Matrix Multiplication

Parallel Algorithm on Sorting

Parallel algorithm for scientific computing