Local Clusters

• Canaan User Guide

Online Textbooks

• Pro Git by Scott Chacon and Ben Straub, Apress, 2014.
• Three books by Victor Eijkhout:
  • Introduction to High-Performance Scientific Computing
  • Parallel Programming for Science and Engineering
  • Introduction to Scientific Computing in C++/Fortran 2003
• Programming on Parallel Machines by Normal Matloff (2010)
• High Performance Computing by Charles Severance (1998)
• MPI: The Complete Reference by Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra (1996)
• Designing and Building Parallel Programs by Ian Foster (1995) (Mirror Site)

Tutorials

• Supercomputing in Plain English from the University of Oklahoma. Past presentations are available at https://vcenter.njvid.net/videos/recent/page1/.
• Training materials at Lawrence Livermore National Labs. Particularly useful are
  • Introduction to Parallel Computing
  • Message Passing Interface (MPI)
  • OpenMP
  • POSIX Threads (Pthreads)
  • Linux Clusters Overview provides interesting information about what goes into a modern supercomputer.
• An Overview to OpenMP (slides by Ruud van der Pas from 2009)
• Cyberinfrastructure Tutor: free on-line courses, including topics like MPI, OpenMP, and Debugging serial and parallel codes. Registration is required, but is free.

Message Passing Interface (MPI) and Cluster Computing

• Message Passing Interface Forum
• Argonne National Laboratory MPI page
• A User's Guide to MPI by Peter Pacheco. Older document that uses some deprecated functions, but still a nice overview.
• Open-source implementations: MPICH and OpenMPI.
• Cluster Monkey website
• Warewulf systems management suite
• SLURM cluster resource manager
.

Parallelization through language extensions (directives)

• OpenMP: Directives for shared memory multi-core processing
• OpenACC: Directives to use for multi-core, GPU and other accelerators

Compilers, Debuggers, Profilers, Etc.

• Debugging with GDB and DDD (a GUI frontend for GDB)
• GNU gprof manual and a short but helpful gprof page (read the whole thing!)
• Valgrind Quick Start Guide and User Manual
• Portland Group Community Edition Compiler. Advanced OpenACC compiler

Papers

• A Comparison of Parallel Sorting Algorithms on Different Architectures
• On the Versitility of Parallel Sorting by Regular Sampling
• A Novel Parallel Sorting Algorithm for Contemporary Architectures

News, articles, forums, and blogs

• HPC Admin Magazine
• HPC for Mac OS X
• Parallel numerical linear algebra

LaTeX

LaTeX is a typesetting tool developed for writing mathematics and scientific documents.

• TeXstudio: LaTeX editing system for Windows, Linux, and Mac.
• The Not So Short Introduction to LaTeX 2e: an easy but thorough introduction to LaTeX, the premier (and free) mathematical typesetting package.
• Wikipedia's comparison of TeX editors: a current list of available editors
• TeX Resources