How to Request GPUs on Koa
- davidls
To use GPUs on Koa a user needs to specify an additional option when compared to the basic srun (interactive) or sbatch (batch) job. By default, the scheduler will not allocate GPUs to a given job when the nodes are configured to be shared. This is to prevent the potential of other users on the same node from utilizing resources they did not request.
The additional option needed to request resources is the --gres option. Each possible gres value takes the form of name[[:type]:count]. This means, the only part of the value that is required is the the name, but at least specifying the count may also be desirable so that you are explicit on the number of that resources you require. For example all of the following would be valid gres options
gpu
gpu:1
gpu:NV-A30:1
As gres is explicitly used for selecting GPUs on Koa, this option is only valid when using either the gpu, kill-shared and kill-exclusive partitions, or private partitions for lab that have purchased GPU nodes.
Below we will cover how one would use this option in an interactive job and batch job.
Available Gres Values
This list can also be created by utilizing the sinfo command and viewing the gres column in the output
sinfo -p gpu,kill-shared | grep gpugpu:NV-H100
gpu:NV-V100-SXM2
gpu:NV-L40
gpu:NV-RTX5000
gpu:NV-A30
gpu:NV-RTX-A4000
gpu:NV-RTX2080Ti
gpu:NV-RTX2070
Examples
Interactive job
Interactive job asking for any type of GPU
[user@login001 ~]$ srun -p gpu --gres=gpu:1 --mem=64G -c 10 -t 120 --pty /bin/bashThe above example would request a single gpu, of any type from the gpu partition. This means you could end of up anything from an Nvidia Tesla k40, Quadro RTX 5000 or a Tesla v100.
Interactive job asking for a Nvidia Tesla v100
[user@login001 ~]$ srun -p gpu --gres=gpu:NV-V100-SXM2:1 --mem=64G -c 10 -t 120 --pty /bin/bashThe above example would request a single gpu, that is a Nvidia Tesla v100
Batch script
Interactive job asking for any type of GPU
The above example would request a single gpu, of any type from the gpu partition. This means you could end of up anything from an Nvidia Tesla k40, Quadro RTX 5000 or a Tesla v100.
Interactive job asking for a Nvidia Tesla v100
The above example would request a single gpu, that is a Nvidia Tesla v100