PyTorch
=======

The PyTorch `website <https://pytorch.org>`_ describes the library as:

    Tensors and Dynamic neural networks in Python with strong GPU acceleration.

Setup PyTorch on CSD3
---------------------

* load a recent python and the pytorch dependencies::

    $ module load python/3.6 cuda/10.0 cudnn/7.5_cuda-10.0

* create and activate a virtual environment::

    $ virtualenv --system-site-packages ~/pytorch-env
    $ source ~/pytorch-env/bin/activate

* install pytorch into your virtual env::

    $ pip install torch torchvision

* download and run an example program::

    $ wget https://pytorch.org/tutorials/_downloads/cifar10_tutorial.py
    $ python cifar10_tutorial.py

Running a PyTorch program
-------------------------

PyTorch will be able to run on both CPU and GPU.

CPU submission script::

    #/bin/bash
    #SBATCH --account MYACCOUNT-CPU
    #SBATCH --partition skylake
    #SBATCH --nodes 1
    #SBATCH --exclusive 

    unset OMP_NUM_THREADS #let pytorch & MKLDNN use all available CPU cores
    python myprogram.py

GPU submission script::

    #/bin/bash
    #SBATCH --account MYACCOUNT-GPU
    #SBATCH --pascal pascal
    #SBATCH --nodes 1
    #SBATCH --gres=gpu:1
    #SBATCH --exclusive 

    python myprogram.py