PyTorch

The PyTorch website describes the library as:

Tensors and Dynamic neural networks in Python with strong GPU acceleration.

Setup PyTorch on CSD3

• load a recent python and the pytorch dependencies:

  $ module load python/3.6 cuda/10.0 cudnn/7.5_cuda-10.0
  

• create and activate a virtual environment:

  $ virtualenv --system-site-packages ~/pytorch-env
  $ source ~/pytorch-env/bin/activate
  

• install pytorch into your virtual env:

  $ pip install torch torchvision
  

• download and run an example program:

  $ wget https://pytorch.org/tutorials/_downloads/cifar10_tutorial.py
  $ python cifar10_tutorial.py
  

Running a PyTorch program

PyTorch will be able to run on both CPU and GPU.

CPU submission script:

#/bin/bash
#SBATCH --account MYACCOUNT-CPU
#SBATCH --partition skylake
#SBATCH --nodes 1
#SBATCH --exclusive

unset OMP_NUM_THREADS #let pytorch & MKLDNN use all available CPU cores
python myprogram.py

GPU submission script:

#/bin/bash
#SBATCH --account MYACCOUNT-GPU
#SBATCH --pascal pascal
#SBATCH --nodes 1
#SBATCH --gres=gpu:1
#SBATCH --exclusive

python myprogram.py