reminder on GPU

In my deep learning research, a piece of GeForce Titan is mostly sufficient for proof-of-concept experiments. While a GeForce TitanX is enough for larger scale ones. Tesla GPUs are luxury (support double-precision float) but less a good fit in practice.

Some parameters of these gpus are as follows:

model                       memory        cores         price ($)
Geforce Titan            6G               2688          600~800 (amazon)
Geforce TitanX         12G              3072          1100 (@Xi,amazon)
Tesla K20                 5G                2496           1800 (amazon)
Tesla K40                 12G              2880           3000 (amazon)
Tesla K80                 24G              4992           4000+ (amazon)

After plugging the GPU onto the motherborad, always choose the right driver for it:  http://www.nvidia.com/Download/Find.aspx

After installing the driver, install the cuda toolkit,
http://insiderattack.blogspot.com/2014/12/installing-cuda-toolkit-65-in-ubuntu.html

Sometimes, you may want an additional cudnn package for convolutional neural network. Download cudnn library, unpack and copy its files to appropriate directories.

 sudo cp dir_of_cudnn/lib* /usr/local/cuda/lib64  
 sudo cp dir_of_cudnn/cudnn.h /usr/local/cuda/include  

Detailed guidance of other hardware can be found at 
http://timdettmers.com/2015/03/09/deep-learning-hardware-guide/

automatic indenting python when using vim editor

Create a directory under ~/.vim, called ftplugin. Then create a file called python.vim there:

 $mkdir ~/.vim/ftplugin  

 $vim ~/.vim/ftplugin/python.vim  

Insert the following into python.vim
setlocal tabstop=4
setlocal softtabstop=4
setlocal shiftwidth=4
setlocal textwidth=80
setlocal smarttab
setlocal expandtab

create a file named as .vimrc under ~

 $vim ~/.vimrc  

There insert the following text
filetype plugin indent on

Now if you write a .py script in vim, hitting tab once will indent by 4 blankspaces.

Count the Number of Files

We may want to know how many files and subdirectories there are in the current directory. The following is quick solution:

 ls . | wc -l  

Note that the "-l" is an "\ell" not one! This command lists the names of all the files and sub-directories, pipes the list to a tool called wc. Then the wc tool counts the lines of this list, which is the number of the files and sub-directories.

Sometimes, we may want to know the number of files in the sub-directories (of the current directory). This kind of recursive count can be accomplished by

 find . -type f | wc -l  

Here "-type f" restricts the count to be performed on files, not directories.

Similarly, we can do

 find . -type d | wc -l  

to recursively count the directories. But notice that the current directory . is counted in as well. Hence the returned number minus one gives the number of sub-directories.

Two-sided Print from Comand Line in Ubuntu

A network printer can be set up by running

 $ system-config-printer  

If you are assigned a user name in using that printer, you may still have to print a file from command line. The disadvantage of command line is the difficulty of page setups. The following switches between one-sided and double sided, so at least helps a little bit.

 $ lpr path/to/printed/file -U username -o sides=two-sided-long-edge   

Keep Track of Training and Testing Loss When Using Caffe

When using Caffe to train a deep neural network, how to record the training/testing loss or accuracy, throughout iterations?

I know the following works, though without understanding the details.

Run in terminal:

 $ caffe_root/build/tools/caffe train --solver=solver.prototxt 2>&1 | tee mylog.log  

Notice the appended part in red, which will log the information shown in the terminal to "mylog.log"

Then run in terminal,

 $ python caffe_root/tools/extra/parse_log.py mylog.log ./  

Now you will see under ./, there are two files, mylog.log.train, and mylog.log.test. They are two csv files that record training and testing loss.

Then you could run gnuplot to quickly visualize the loss/accuracy during iterations. But first you need to comment the 1st line of the two files by #

The .train (or .test) file is of the following form:

#NumIters,Seconds,LearningRate,accuracy,loss
0,****,****,****,****
100,****,****,****,****
.
.
.

Then in terminal run,

 $ gnuplot  
 gnuplot> set datafile separator ','  
 gnuplot> plot 'mylog.log.train' using 1:4 with line # accuracy throughout the iterations  
 gnuplot> plot 'mylog.log.train' using 1:5 with line # loss throughout the iterations