Building HPVM

Dependencies

  • The following components are required for building HPVM:

    • GCC (>=7.0)

    • CMake (>=3.17)

    • GNU Make (>=3.79) or Ninja (>=1.10)

    • Python (==3.6) with pip (>=20)

  • One or more of the following OpenCL libraries; otherwise, only CPU is available:

    • Nvidia OpenCL libraries and runtime: required for compiling applications to run on Nvidia GPUs.

    • Intel FPGA SDK for OpenCL: required for compiling applications to run on Intel FPGAs. (Note: a license may be required depending on the FPGA model being used.)

  • The following components are required to build support for Tensor-domain applications introduced in ApproxHPVM:

    • CUDA (>=9.0, <=10.2) with CUDNN 7

      • GCC (<8.0)

      • CUDNN 7 is unsupported beyond CUDA 10.2 (starting from CUDA 11)

    • OpenMP (>= 4.0)

      • GCC comes with OpenMP support; OpenMP-4.0 is supported by GCC-4.9 onward. see here for the OpenMP version supported by each GCC version.

    • In addition, each version of CUDA-nvcc requires GCC to be not newer than a certain version. See here for the support matrix.

Supported Operating Systems

HPVM has been tested to work on Ubuntu 16.04 and 18.04. Newer Ubuntu distros are expected to work but have not been tested.

Note: The Intel FPGA for SDK does not support Ubuntu versions later than 18.04. Please refer to the Intel documentation and support page for more information.

Supported Hardware

We expect any X86 or ARM CPU to work with HPVM. The following CPUs have been tested and confirmed to work:

  • Intel Xeon E5-2640

  • Intel Xeon W-2135

  • Intel Xeon W-2275

  • ARM Cortex A-57

We expect any Nvidia GPU that is supported by the NVIDIA OpenCL runtime to work with HPVM. The following GPUs have been tested with our OpenCL back end and confirmed to work:

  • Nvidia Quadro P1000

  • Nvidia GeForce GTX 1080

  • Nvidia GeForce RTX 2080 Ti

We expect any Nvidia GPU that is supported by the NVIDIA CUDA/CUDNN libraries to work with HPVM. The following GPUs have been tested with our Tensor back end and confirmed to work:

  • Nvidia Jetson TX2

  • Nvidia GeForce GTX 1080

NOTE: Approximations are tuned for Jetson TX2 and same speedups may not exist for other architectures.

We expect any Intel FPGA with OpenCL support to work with HPVM. The following FPGAs have been tested with our FPGA back end and confirmed to work:

  • Intel Arria 10 GX FPGA Development Kit

Cloning HPVM

First, checkout HPVM and navigate to <root>/hpvm:

git clone --recursive -b main https://gitlab.engr.illinois.edu/llvm/hpvm-release.git
cd hpvm-release/hpvm/

If you have already cloned the repository without using --recursive, the directory hpvm/projects/predtuner should be empty, which can be fixed with git submodule update --recursive --init.

Python Environment

It is strongly recommended to use some Python virtual environment, as HPVM will install a few Python packages during this installation process. These packages are only required for Tensor-domain extensions ApproxHPVM and ApproxTuner.

  • Some HPVM Python packages contain executables. If you don’t use a virtual environment, these executables are installed to your local bin directory, usually $HOME/.local/bin. Please ensure this directory is in your $PATH variable. Below it is assumed that these executables are visible through $PATH.

If you use Anaconda for package management, we provide a conda environment file that covers all Python and package requirements (env.yaml can be found in the repository):

conda env create -n hpvm -f env.yaml

This creates the conda environment hpvm. If you use this method, remember to activate the environment each time you enter a bash shell:

conda activate hpvm

Using HPVM installer Script

The HPVM installer script can be used to automatically download extra components, configure and build HPVM:

./install.sh

  • Without arguments, this script will interactively prompt for parameters. Alternatively, use ./install.sh -h for a list of available arguments and pass arguments as required.

  • ./install.sh supports Ninja, a substitute of Make that is considered to build faster on many IO-bottlenecked devices. Passing --ninja to the installer tells it to use Ninja instead of Make.

  • ./install.sh can relay additional arguments to CMake, but the dash must be dropped regardless of using prompt or CLI arguments. For example,

    ./install.sh -j<n> DCMAKE_BUILD_TYPE=Release

    will compile HPVM in Release mode. In this, `<n>` should be replaced by the number of threads to build with. Note that HPVM requires nearly 16 GB of RAM to build single-threaded and the usage increases (though not linearly) as the number of threads is increased. If you encounter Out of Memory or similar errors during compilation, try decreasing the number of threads.

After configuring HPVM, the installer will also compile HPVM by default, which you can opt out of. (This option is available using the -m CLI flag, or using the interactive prompt). If you do so, follow the instructions in Manually Build HPVM to manually compile HPVM, and in Tests to manually run test cases if desired. Otherwise, the next 2 sections can be skipped.

List of Tools Installed by HPVM lists all the tools that will be built by the installer. These tools can be found under `<root>/hpvm/build/bin.

NOTE: The installer script DOES NOT automatically install Nvidia or Intel FPGA tools. These tools must be installed separately and the user $PATH variables must be configured to correctly point to any required libraries, binaries, etc.

List of Tools Installed by HPVM

A list and description fo the user-facing HPVM tools can be found here.

Full List of Installer Options

The following is a comprehensive list of the installer options:

-m,         Installer will automatically download LLVM and link HPVM components but will not automatically install HPVM. Default: false.
-j,         Sets the number of threads to build with. Default: 2.
-b,         Specifices where the `build` directory will be created. Default: "build".
-t,         Specifies which LLVM targets to build. Default: "X86".
--ninja,    Uses Ninja to build HPVM. Otherwise `make` is used.
-r,         Build and run the test cases.
--no-pypkg, Disables building the HPVM Python packages.
--no-params,Disables downloading the DNN model parameters.
cmake_args, Add arguments that are meant to be relayed to cmake.

How Does the Installer Work

The HPVM installer performs the following tasks:

  • downloads and decompresses LLVM and Clang into hpvm/llvm,

  • links HPVM source code into hpvm/llvm/tools/hpvm,

  • downloads DNN model parameters to test/dnn_benchmarks/model_params (this step is optional – you can opt out of it),

  • installs a few Python packages: the PyTorch frontend, the Keras frontend, the predictive tuner, and the HPVM profiler, (this step is optional),

  • builds the entire HPVM which provides hpvm-clang, HPVM’s main compilation interface,

    • The build system builds HPVM, creates a Python package hpvmpy (which provides the binary hpvm-clang) on the fly, and installs it to your current Python environment.

  • and finally, builds and runs some tests if you explicitly require so.

    • While running tests is recommended, it is not turned on by default as it is very time-consuming.

Skipping Python Package installation

If you are installing HPVM on a “target” device which is just used for profiling, you may not need to install the frontend and the tuner packages. These packages also have Python version requirement and package dependencies that may be hard to meet on some devices, especially edge computing devices with ARM CPUs.

You can instead skip the installation by either passing --no-pypkg flag to the installer, or answering no (“n”) when it prompt the following:

Install HPVM Python Packages (recommended)? [y/n]

In this case, any Python >= 3.6 will work.

TroubleShooting

If CMake did not find your CUDA, some environment variables will help it:

  • CUDA_TOOLKIT_PATH — Path to the CUDA toolkit

  • CUDA_INCLUDE_PATH — Path to the CUDA headers

  • CUDA_LIB_PATH — Path to CUDA libraries

You can use set_paths.sh for this purpose: modify the values of these variables in set_paths.sh according to your system, and source the script:

source set_paths.sh

Manually Build HPVM

Alternatively, HPVM can be manually built with CMake. Please note that in this case, the installer script still must be executed to obtain some required components, but without the build step (i.e. ./install.sh -m). In current directory (hpvm/), do

mkdir build
cd build
cmake ../llvm -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ [options]

Some common additional options that can be used with CMake are:

  • -DCMAKE_INSTALL_PREFIX=directory — Specify for directory the full pathname of where you want the HPVM tools and libraries to be installed.

  • -DCMAKE_BUILD_TYPE=type — Valid options for type are Debug, Release, RelWithDebInfo, and MinSizeRel. Default is Debug.

  • -DLLVM_ENABLE_ASSERTIONS=On — Compile with assertion checks enabled (default is Yes for Debug builds, No for all other build types).

  • -DRTDEBUG=On/Off — Turns Debug mode for the HPVM Runtime ON/OFF. Can be useful for debugging interactions with OpenCL devices (i.e. GPU, FPGA).

Now, compile the HPVM target hpvm using:

make -j<number of threads> hpvm

This will build all the components of hpvm, including the top-level executable tools that are listed under List of Tools Installed by HPVM.

Tests

We provide a number of test cases written in HPVM.

make targets check-hpvm-pass, check-hpvm-dnn, check-hpvm-tensor-rt, check-hetero-pass test the various components of HPVM. You can run tests by simply providing the appropriate target to make: for example,

make -j<number of threads> check-hpvm-pass

runs check-hpvm-pass tests.