RC's community cluster is a great resource to take advantage of! I had a rough time getting the Fast AI course set up, so I wrote down some instructions (hopefully) help others save some time.
- Make an account on recurse.com/settings/cluster - this requires setting up and copying over an SSH key instructions here.
SSH onto one of the GPU-enabled machines - I chose to use mercer (both Mercer and Crosby have powerful graphics cards). You can find more info about the specs for each machine here
Once on the machine, you should be in your own home directory. The command prompt should look something like this:
- Install Conda so that you can create virtual environments to install software that won't affect others using the machine.
Install the fast.ai library (instructions from the course forum)
git clone https://github.com/fastai/fastai.git cd fastai conda env create -f environment.yml
Activate the environment for fast.ai
source activate fastai
Now, your environment should look something like this:
- On your local machine, in a separate tab, set up port forwarding so that the port you're serving up the jupyter notebook to can be opened on a browser on your local machine.
For example, if I want to open up the notebooks on port 8888 on my local machine, and use port 6565 on the server:
ssh -N -f -L localhost:8888:localhost:6565 firstname.lastname@example.org
You can also forward the same ports (8888:8888) if you don't find that less confusing.
Back on the cluster, inside the fast.ai directory:
jupyter notebook --no-browser --port 6565
The --no-browser flag prevents jupyter from trying to open a browser inside the remote server, and the port flag specifies the port that the server will use.
In your browser, navigate to the link that the jupyter notebook gives you, but change the port that you used above. In my case, jupyter served up
so I would navigate to
Sometimes your env won't be automatically populated by conda, so you might have find your env elsewhere to run it:
source activate anaconda/envs/fast-ai
- Make sure you're using python3
To see if your port forwarding is running/working correctly:
ps aux | grep ssh