The following components are required for building HPVM:
GNU Make (>=3.79) or Ninja (>=1.10)
Python (==3.6) with pip (>=20)
Python must be strictly 3.6 (any subversion from 3.6.0 to 3.6.13). This is needed by some Python packages in HPVM.
If you choose to not install these packages, then any Python >= 3.6 will work. See how to skip installing Python packages in the installer.
One or more of the following OpenCL libraries; otherwise, only CPU is available:
Nvidia OpenCL libraries and runtime: required for compiling applications to run on Nvidia GPUs.
Intel FPGA SDK for OpenCL: required for compiling applications to run on Intel FPGAs. (Note: a license may be required depending on the FPGA model being used.)
The following components are required to build support for Tensor-domain applications introduced in ApproxHPVM:
CUDA (>=9.0, <=10.2) with CUDNN 7
CUDNN 7 is unsupported beyond CUDA 10.2 (starting from CUDA 11)
OpenMP (>= 4.0)
GCC comes with OpenMP support; OpenMP-4.0 is supported by GCC-4.9 onward. see here for the OpenMP version supported by each GCC version.
In addition, each version of CUDA-nvcc requires GCC to be not newer than a certain version. See here for the support matrix.
Supported Operating Systems¶
HPVM has been tested to work on Ubuntu 16.04 and 18.04. Newer Ubuntu distros are expected to work but have not been tested.
Note: The Intel FPGA for SDK does not support Ubuntu versions later than 18.04. Please refer to the Intel documentation and support page for more information.
We expect any X86 or ARM CPU to work with HPVM. The following CPUs have been tested and confirmed to work:
Intel Xeon E5-2640
Intel Xeon W-2135
Intel Xeon W-2275
ARM Cortex A-57
We expect any Nvidia GPU that is supported by the NVIDIA OpenCL runtime to work with HPVM. The following GPUs have been tested with our OpenCL back end and confirmed to work:
Nvidia Quadro P1000
Nvidia GeForce GTX 1080
Nvidia GeForce RTX 2080 Ti
We expect any Nvidia GPU that is supported by the NVIDIA CUDA/CUDNN libraries to work with HPVM. The following GPUs have been tested with our Tensor back end and confirmed to work:
Nvidia Jetson TX2
Nvidia GeForce GTX 1080
NOTE: Approximations are tuned for Jetson TX2 and same speedups may not exist for other architectures.
We expect any Intel FPGA with OpenCL support to work with HPVM. The following FPGAs have been tested with our FPGA back end and confirmed to work:
Intel Arria 10 GX FPGA Development Kit
First, checkout HPVM and navigate to
git clone --recursive -b main https://gitlab.engr.illinois.edu/llvm/hpvm-release.git cd hpvm-release/hpvm/
If you have already cloned the repository without using
hpvm/projects/predtuner should be empty,
which can be fixed with
git submodule update --recursive --init.
It is strongly recommended to use some Python virtual environment,
as HPVM will install a few Python packages during this installation process.
These packages are only required for Tensor-domain extensions
Some HPVM Python packages contain executables. If you don’t use a virtual environment, these executables are installed to your local
$HOME/.local/bin. Please ensure this directory is in your $PATH variable. Below it is assumed that these executables are visible through $PATH.
If you use Anaconda for package management,
we provide a conda environment file that covers all Python and package requirements
env.yaml can be found in the repository):
conda env create -n hpvm -f env.yaml
This creates the conda environment
If you use this method, remember to activate the environment each time you enter a bash shell:
conda activate hpvm
Using HPVM installer Script¶
The HPVM installer script can be used to automatically download extra components, configure and build HPVM:
Without arguments, this script will interactively prompt for parameters. Alternatively, use
./install.sh -hfor a list of available arguments and pass arguments as required.
./install.shsupports Ninja, a substitute of Make that is considered to build faster on many IO-bottlenecked devices. Passing
--ninjato the installer tells it to use Ninja instead of Make.
./install.shcan relay additional arguments to CMake, but the dash must be dropped regardless of using prompt or CLI arguments. For example,
./install.sh -j<n> DCMAKE_BUILD_TYPE=Release
will compile HPVM in Release mode. In this,
`<n>`should be replaced by the number of threads to build with. Note that HPVM requires nearly 16 GB of RAM to build single-threaded and the usage increases (though not linearly) as the number of threads is increased. If you encounter Out of Memory or similar errors during compilation, try decreasing the number of threads.
After configuring HPVM,
the installer will also compile HPVM by default, which you can opt out of.
(This option is available using the
-m CLI flag, or using the interactive prompt).
If you do so, follow the instructions in Manually Build HPVM to manually compile HPVM,
and in Tests to manually run test cases if desired.
Otherwise, the next 2 sections can be skipped.
List of Tools Installed by HPVM lists all the tools that will be built by the installer. These tools can be found
NOTE: The installer script DOES NOT automatically install Nvidia or Intel FPGA tools.
These tools must be installed separately and the user
$PATH variables must be configured
to correctly point to any required libraries, binaries, etc.
List of Tools Installed by HPVM¶
A list and description fo the user-facing HPVM tools can be found here.
Full List of Installer Options¶
The following is a comprehensive list of the installer options:
-m, Installer will automatically download LLVM and link HPVM components but will not automatically install HPVM. Default: false. -j, Sets the number of threads to build with. Default: 2. -b, Specifices where the `build` directory will be created. Default: "build". -t, Specifies which LLVM targets to build. Default: "X86". --ninja, Uses Ninja to build HPVM. Otherwise `make` is used. -r, Build and run the test cases. --no-pypkg, Disables building the HPVM Python packages. --no-params,Disables downloading the DNN model parameters. cmake_args, Add arguments that are meant to be relayed to cmake.
How Does the Installer Work¶
The HPVM installer performs the following tasks:
downloads and decompresses LLVM and Clang into hpvm/llvm,
links HPVM source code into
downloads DNN model parameters to
test/dnn_benchmarks/model_params(this step is optional – you can opt out of it),
installs a few Python packages: the PyTorch frontend, the Keras frontend, the predictive tuner, and the HPVM profiler, (this step is optional),
builds the entire HPVM which provides hpvm-clang, HPVM’s main compilation interface,
The build system builds HPVM, creates a Python package hpvmpy (which provides the binary hpvm-clang) on the fly, and installs it to your current Python environment.
and finally, builds and runs some tests if you explicitly require so.
While running tests is recommended, it is not turned on by default as it is very time-consuming.
Skipping Python Package installation¶
If you are installing HPVM on a “target” device which is just used for profiling, you may not need to install the frontend and the tuner packages. These packages also have Python version requirement and package dependencies that may be hard to meet on some devices, especially edge computing devices with ARM CPUs.
You can instead skip the installation by either passing
--no-pypkg flag to
the installer, or answering no (“n”) when it prompt the following:
Install HPVM Python Packages (recommended)? [y/n]
In this case, any Python >= 3.6 will work.
If CMake did not find your CUDA, some environment variables will help it:
CUDA_TOOLKIT_PATH— Path to the CUDA toolkit
CUDA_INCLUDE_PATH— Path to the CUDA headers
CUDA_LIB_PATH— Path to CUDA libraries
You can use
set_paths.sh for this purpose: modify the values of these variables
set_paths.sh according to your system, and source the script:
Manually Build HPVM¶
Alternatively, HPVM can be manually built with CMake.
Please note that in this case,
the installer script still must be executed to obtain some required components,
but without the build step (i.e.
In current directory (
mkdir build cd build cmake ../llvm -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ [options]
Some common additional options that can be used with CMake are:
-DCMAKE_INSTALL_PREFIX=directory— Specify for directory the full pathname of where you want the HPVM tools and libraries to be installed.
-DCMAKE_BUILD_TYPE=type— Valid options for type are Debug, Release, RelWithDebInfo, and MinSizeRel. Default is Debug.
-DLLVM_ENABLE_ASSERTIONS=On— Compile with assertion checks enabled (default is Yes for Debug builds, No for all other build types).
-DRTDEBUG=On/Off— Turns Debug mode for the HPVM Runtime ON/OFF. Can be useful for debugging interactions with OpenCL devices (i.e. GPU, FPGA).
Now, compile the HPVM target
make -j<number of threads> hpvm
This will build all the components of hpvm, including the top-level executable tools that are listed under List of Tools Installed by HPVM.
We provide a number of test cases written in HPVM.
check-hetero-pass test the various components of HPVM.
You can run tests by simply providing the appropriate target to
make: for example,
make -j<number of threads> check-hpvm-pass