reminder on GPU

In my deep learning research, a piece of GeForce Titan is mostly sufficient for proof-of-concept experiments. While a GeForce TitanX is enough for larger scale ones. Tesla GPUs are luxury (support double-precision float) but less a good fit in practice.

Some parameters of these gpus are as follows:

model                       memory        cores         price ($)
Geforce Titan            6G               2688          600~800 (amazon)
Geforce TitanX         12G              3072          1100 (@Xi,amazon)
Tesla K20                 5G                2496           1800 (amazon)
Tesla K40                 12G              2880           3000 (amazon)
Tesla K80                 24G              4992           4000+ (amazon)

After plugging the GPU onto the motherborad, always choose the right driver for it:  http://www.nvidia.com/Download/Find.aspx

After installing the driver, install the cuda toolkit,
http://insiderattack.blogspot.com/2014/12/installing-cuda-toolkit-65-in-ubuntu.html

Sometimes, you may want an additional cudnn package for convolutional neural network. Download cudnn library, unpack and copy its files to appropriate directories.

 sudo cp dir_of_cudnn/lib* /usr/local/cuda/lib64  
 sudo cp dir_of_cudnn/cudnn.h /usr/local/cuda/include  

Detailed guidance of other hardware can be found at 
http://timdettmers.com/2015/03/09/deep-learning-hardware-guide/