Accelerating Artificial Intelligence, Machine Learning and Deep Learning on a Linux Laptop with an External GPU

Learn how to set up the environment for the data science

With the ability to utilize Intel Thunderbolt3 on laptops, it is now possible to use a dedicated GPU (Graphics Processing Unit) for games and data science that requires a high-end graphics card using an external GPU (eGPU). I am a doctoral researcher of graduate school in the UK, and I am currently doing research related to blockchain technology. However, I am considering the deep learning as a new research topic in the future.

Accordingly, the establishment and utilization of a data science-related development environment was an essential personal task at the moment. For handling the flexibility of graduate research between deep learning and blockchain, it is necessary to build a development environment using a thin and light laptop rather than building a development environment using a high-spec desktop. Ultimately, eGPU (external GPU) became a medium that satisfies needs for Deep Learning.

Previously, I performed Deep Learning study using various cloud computing services such as Microsoft Azure and Amazon AWS. Although these external cloud computing services were able to easily learn and utilize algorithms by using the already established development environment, the excessively expensive fees had a significant impact on my economic status as a graduate student.

Ultimately, the use of laptops and eGPUs is a one-time payment for a development environment that can utilize data science using personal laptops, allowing continuous learning and research of deep learning with a low amount of costs. Ultimately, this post focuses on building a data science environment using Ubuntu 20.04 and eGPU.

This post has the following structure:

  1. Hardware selection
  2. Setting up the development environment for Deep Learning
  3. Validate the deep learning environment with eGPU

1 Hardware selection

The following configuration is required in order to build a data science development environment using Thunderbolt 3 + eGPU:

  • Laptop with Thunderbolt3
  • GPU (Graphics Processing Unit)
  • GPU enclosure

hardware

Basically, the performance of an eGPU solution depends on the performance of the GPU, the performance of the cpu, and the connection bandwith between the GPU and CPU.

1.1 Choosing a Laptop with Thunderbolt 3

The criteria for selecting a laptop are as follows:

  • A thin & light laptop that can install Ubuntu 20.04, a Linux distribution
  • Thunderbolt 3
  • CPU performance
  • Competitive price

My choice is Lenovo Thinkpad T480s with the following specifications:

  • Processor: Intel Core™ i7-8550U CPU / 1.8 GHz (4 GHz) / 8MB Cache
  • Memory: 16GB DDR4(1 X 8GB + 8GB)
  • Storage: 512GB-M.2 SSD TCG Opal Encryption 2, NVMe
  • Graphics: Intel UHD Graphics 620
  • Operating system: Win 10 Pro 64-bit + Ubuntu 20.04 (Dual boot)

1.2 GPU (Graphics Processing Unit)

The criteria for selecting GPU are as follows:

  • List itemNVIDIA GPU
  • Competitive price

My choice is GTX Geforce RTX 2070 Super.

1.3 GPU Enclosure

The criterion for my GPU enclosure selection was only the GPU Enclosure that can be installed with GTX Geforce RTX 2070 super. You can compare and analyze various eGPUs by visiting the eGPU.io website. My final choice is Sonnet Breakaway 550.


2 Setting up the development environment for Deep Learning

The following procedures are required to establish a development environment:

(1) Install NVIDIA GPU drivers (2) Install CUDA Toolkit (3) Install cuDNN (CUDA Deep Learning Neural Network Library) (4) Create a Python Virtual Environment (5) Install Python libraries

2.1 Install NVIDIA GPU drivers

You can visit the NVIDIA website and select the product type, product series, product, operating system (Linux 64-bit), download type (Linux Long-Lived) and Language (English) that are appropriate for the purchased GPU. You can install as following commands on the shell:

nvidia driver

$ chmod +x NVIDIA-Linux-x86_64-450.80.02.run
$ sudo sh NVIDIA-Linux-X86_64-450.80.02.run

This installation requires that the script executable permissions via the chmod command and that the super user’s credential with the root password.

Offloading Xorg to the eGPU

After installing the NVIDIA proprietary driver, make sure that gdm3 (the graphical login manager) for Ubuntu 18.04/20.04 doesn’t use Wayland by commenting out #WaylandEnable=false in /etc/gdm3/custom.conf:

# GDM configuration storage
# 
# See /usr/share/gdm/gdm.schemas for a list of available options.
[daemon]
# Uncomment the line below to force the login screen to use Xorg
WaylandEnable=false

Edit /usr/share/X11/xorg.conf.d/10-nvidia.conf to add an additional option in OutputClass to allow the NVIDIA X driver to accelerate X instead of it being driven by the integrated GPU:

Section "OutputClass"
   Identifier "nvidia"
   MatchDriver "nvidia-drm"
   Driver "nvidia"
   Option "AllowExternalGpus" "True"
   Option "AllowEmptyInitialConfiguration"
   ModulePath "/usr/lib/x86_64-linux-gnu/nvidia/xorg"
EndSection

Check whether the NVIDIA proprietary driver is loaded:

$ nvidia-smi

2.2 Install CUDA Toolkit

The CUDA (Compute Unified Device Architecture) is a parallel computing platform and application programming interface (API) model created by Nvidia.

CUDA toolkit installation is required. For Ubuntu 20.04, the toolkit is available from the standard repository.

$ sudo apt-get install nvidia-cuda-toolkit

Alternatively, you can manually install CUDA Toolkit as follows:

cuda

Alternatively, I suggest the following shortcut:

  • Use the wget command to download the runfile
$ wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda_9.0.176_384.81_linux-run
  • Install the CUDA Toolkit This installation requires that the script executable permissions via the chmod command and that the super user’s credential with the root password.
$ chmod +x cuda_9.0.176_384.81_linux-run
$ sudo ./cuda_9.0.176_384.81_linux-run --override

During the installation, the following selections are required:

  • Accept the terms and conditions
  • Select y for “Install on an unsupported configuration”
  • Select n for “Install NVIDIA Accelerated Graphics Driver for Linux-86_64384.81”
  • Keep all other default values

Bashrc file has to be updated in order to include the CUDA Toolkit: (You can use other text editors such as vim or emacs)

$ nano ~/.bashrc

Scroll to the bottom and add following lines:

export PATH=/usr/local/cuda-9.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64

bash

Once the bash profile is saved and closed, reload the bash profile as follows:

$ source ~/.bashrc

Confirm that the CUDA Toolkit has been installed

$ nvidia-smi

2.3 Install cuDNN (CUDA Deep Learning Neural Network Library)

First of all, you have to create an account on the NVIDIA website and download cuDNN. Please refer to the following procedure:

  • Download cuDNN
  • Log in and check “I agree to the terms of service for the cuDNN Software License Agreement.”
  • Archived cuDNN Releases
  • cuDNN v.7.4.1 (Nov 8, 2018) for CUDA 9.0
  • cuDNN Library for Linux

Install cuDNN on the Ubuntu 20.04 system with the following commands:

$ cd ~
$ tar -zxf cudnn-9.0-linux-x64-v7.4.1.5.tgz
$ cd cuda
$ sudo cp -P lib64/* /usr/local/cuda/lib64/
$ sudo cp -P include/* /usr/local/cuda/include/

I extracted the cuDNN 9.0 v7.4.1.5 file, navigated into the cuda/ directory, copied the lib64/ directory to the path shown and all contents and copied include/ folder to the path shown.

2.4 Create a Python Virtual Environment

I built the python virtual environment using Anaconda distribution. Separate postings are planned for installing Anaconda distribution on Ubuntu, building virtual environments, and installing dependencies required for deep learning. The virtual environment can be built using various methods, but anaconda distribution can be easily built with one line command as follows.

$ conda create --name tf-gpu tensorflow-gpu

The following dependencies were installed in the virtual environment.

  • Tensorflow-gpu
  • Opencv
  • Numpy, scipy matplotlib, pipplow, h5py, request, progressbar2, scikit-learn, scikit-image, etc.

tf

3 Validate the deep learning environment with eGPU

Finally, I verified the use of egpu through performing the face mask detector seojinnseobin

Avatar
Seongha Hwang
PhD Student

My research interests include blockchain technology, digital construction and innovative project governance.