Amazon Web Services provides the most elastic and scalable cloud infrastructure to run your HPC applications. With AWS’s virtually unlimited capacity, engineers, researchers, and HPC system owners can innovate beyond the limitations of on-premises HPC infrastructure.
AWS delivers an integrated suite of services that provides everything you need to quickly and easily build and manage HPC clusters in the cloud. With this elastic infrastructure you can run your most compute-intensive workloads regardless of your industry and application.
With the tools detailed in this workshop, you can run workloads on AWS spanning both traditional HPC applications–genomics, computational chemistry, financial risk modeling, computer aided engineering, weather prediction, and seismic imaging–as well as emerging applications, such as machine learning, deep learning, and autonomous driving.
HPC on AWS removes the long wait times and lost productivity that can come from using fixed, on-premises HPC clusters. Flexible configuration and virtually unlimited scalability allow you to grow and shrink your infrastructure as your workloads dictate, not the other way around. Additionally, with access to a broad portfolio of cloud-based services like Data Analytics, Artificial Intelligence (AI), and Machine Learning (ML), you can redefine traditional HPC workflows to innovate faster.
Today, more HPC is run on AWS than on any other cloud.
In this workshop you will deploy and modify HPC clusters using the AWS ParallelCluster API. To exemplify how the API can be used, you will begin to use PCluster Manager which is an open-source project providing a Web UI on top of the AWS ParallelCluster API. In the second part of the workshop, you will directly interact with the AWS ParallelCluster API using its Python client.
To start with the workshop continue to Workshop Overview.
AWS ParallelCluster is an AWS-supported, open source cluster management tool that makes it easy for you to deploy and manage High Performance Computing (HPC) clusters on AWS. ParallelCluster uses a simple (YAML) text file to model and provision all the resources needed for your HPC applications in an automated and secure manner. It also supports a variety of job schedulers such as AWS Batch and Slurm for easy job submissions.
AWS ParallelCluster is released via the Python Package Index (PyPI). ParallelCluster’s source code is hosted on the Amazon Web Services repository on GitHub. AWS ParallelCluster is available at no additional charge, and you pay only for the AWS resources you use to run your applications.
AWS ParallelCluster API is a serverless application that, once deployed to your AWS account, will enable programmatic access to AWS ParallelCluster features via API.
AWS ParallelCluster API is distributed as a self-contained AWS CloudFormation template mainly consisting of an Amazon API Gateway endpoint, that exposes AWS ParallelCluster features, and an AWS Lambda function, that takes care of executing the invoked features.
The image below shows a high level architecture diagram of the AWS ParallelCluster API infrastructure.