Ctrl+K

Introduction to HPC

Introduction to HPC#

Course Description#

A high performance computing (HPC) cluster is an essential tool for computational tasks that are too intensive for a personal computer, for example analysing large datasets. This course will introduce ISCA, the Exeter’s HPC cluster, and teach participants how to design a HPC workflow, interrogate the Slurm scheduling system, and run simple applications on cluster. At the end of this interactive course, participants will be able to translate what they have learnt to improve the efficiency of their computational research. This course is aimed at those who plan to use ISCA to perform their computational research or who wish to learn what ISCA could enable them to do.

Course Objectives#

  • what an high performance computing (HPC) cluster is and what type of tasks it is suited to.

  • what computing resources are necessary to run a large workload.

  • how to log on to the cluster and navigate around the file system

  • how to submit and manage job tasks in the scheduler’s queue.

  • the array jobs syntax, a tool to efficiently parallel repetitive tasks.

Pre-requisite Knowledge#

  • An account on ISCA

  • Some experience with the command line (e.g. attended an Introduction to Unix Course).

Installation Guide#

Required Software#

To complete the activities in the workshop you will need access to some software on your machine. All of these are available for free via the internet, and it should be as easy as downloading the relevant version and double clicking the download to launch the installation process. It should be fairly quick.

  • A terminal. Mac users will already have one available via Applications – so no need to do anything. Windows users will need to install/download one. We recommend MobaXterm. An alternative is PuTTY.

  • A file transfer client. If you download MobaXterm for (1) then a file transfer client is included in this software. If you are on a Mac or prefer to use an alternative terminal on windows we recommend FileZilla. Alternatives include Cyberduck or Winscp (for windows).

  • A text editor. You likely have a basic text editor already installed (e.g. Notepad or WordPad) and there is a text editor inbuilt into MobaXterm but you may prefer to use one that has some enhanced features for writing code, such as syntax highlighting. We recommend Notepad++ or VSCode.

ISCA Account#

If you have been offered a place on the course we are in the process of setting this up for you.

Acknowledgements#

These materials were adapted from the Carpentries. Specifically we have developed the HPC Carpentry Intro to HPC Shell lessons adding our own custom content to reflect the HPC systems at Exeter. The materials were developed by a team of researchers at the University who are enthusiastic about sharing their skills with the wider research community. Its provision is dependent on the time of these individuals. If you have benefitted in any way from this course and want to support its long term sustainability then please take the time to complete our feedback survey, recommend it to your colleagues, and enthuse about it to your senior leadership team.

This workshop is brought to you by:

The University of Exeter Research Software Engineering Group The Institute of Data Science and Artificial Intelligence The University of Exeter Researcher Development Programme The University of Exeter Doctoral College

Developers#

This course was developed by Ed Hone

License Info#

Instructional Material

The instructional material in this course is copyright © 2024 University of Exeter and is made available under the Creative Commons Attribution 4.0 International licence (https://creativecommons.org/licenses/by/4.0/). Instructional material consists of material that is contained within the “individual_modules/introduction_to_hpc” folders in this repository, with the exception of code snippets and example programs found in files within these folders. Such code snippets and example programs are considered software for the purposes of this licence.

Software

Except where otherwise noted, software provided in this repository is made available under the MIT licence (https://opensource.org/licenses/MIT).

Copyright © 2024 University of Exeter

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

The software in this repository is adapted from software that is covered by the following copyright and permission notice:

Copyright © 2024 Software Carpentry

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Contents