In this two weeks, I started to use pylearn2 for my deep learning project, however training a convolution neural network with CPU is way too slow that an epoch would take half a hour. Of course GPU is so essential to deep learning, there are indeed many choices of GPU in nvidia which is compatible with cuda toolkit, according to Running things on a GPU, GeForce GTX5xx is better for computation while GTX6xx is optimized for gaming.

To add GPU in my office PC, I brought a graphic card (GTX 560 Ti) that costs 70 pounds with 2GB memory and a external power supply for 35 pounds (Corsair Builder Series CXM 430W).

What to do next is to install the cuda toolkit and driver in my ubuntu 12.04.

sudo apt-get update
sudo apt-get install cuda
export LD_LIBRARY_PATH=/usr/local/cuda-6.0/lib64:$LD_LIBRARY_PATH
export PATH=/usr/local/cuda-6.0/bin:$PATH
finally cuda 6.0 is installed in  /usr/local/cuda-6.0 , and I tried the Theano GPU tests to check whether the GPU is used or not.
there is warming, saying
‘WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: Unable to get the number of gpus available: CUDA driver version is insufficient for CUDA run\ time version).‘
so it seemed that CUDA 6.0 is higher than / not compatible my existing nvidia driver, or CUDA 5.5 is more preferred, so I manually removed all cuda 6.0 and nvidia drivers
apt-get remove nvidia*
apt-get purge cuda
and downloaded and installed the cuda 5.5 run file and the compatible driver.

To install cuda 5.5 run file, make sure you have logged out

  • Hit CTRL+ALT+F1 and login using your credentials.
  • kill your current X server session by typing sudo service stop lightdm 
  • Enter runlevel 3 by typing sudo init 3 and install your *.run file.
  • You might be required to reboot when the installation finishes. If not, run sudo service start lightdm or sudo start lightdm to start your X server again.
  • chmod +x cuda*.run
  • ./cuda_5.5.22_linux_64.run

If the installation encounters a error

Installing the NVIDIA display driver... The driver installation is unable to locate the kernel source. Please make sure that the ker
nel source packages are installed and set up correctly. If you know that the kernel source packages are installed and set up correctly, you may pass the location of the kernel source with the '--kernel-source-path' flag.

then try

  • ./cuda_5.5.22_linux_64.run --kernel-source-path /usr/src/linux-headers-3.8.0-38-generic

(Note: make sure your kernel is above 3.2, 3.11.0 it isn’t supported by any released version of cuda)

at last reboot your computer and export the path

export LD_LIBRARY_PATH=/usr/local/cuda-6.0/lib64:$LD_LIBRARY_PATH
export PATH=/usr/local/cuda-6.0/bin:$PATH
nvidia-smi // check nvidia driver
nvcc // check cuda version

Test the performance of GPU and CPU

THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python test.py  costs 0.49 seconds

device=cpu python test.py costs 3.9