Profiling Tools Interfaces for GPU (PTI for GPU)

[!WARNING] The samples and auxiliary tools (other than tools/unitrace) are no longer under active development and will be removed soon. Candidates for removal:

• chapters
• samples
• loader
• tests
• tools/ze_tracer
• tools/memaccess
• tools/gpuinfo
• tools/oneprof
• tools/onetrace
• tools/instcount
• tools/sysmon

Identical functionality can be found in unitrace for most of the tools. For GTPin-related tools, one can find tools and usage samples at https://www.intel.com/content/www/us/en/developer/articles/tool/gtpin.html.

Overview

This repository describes the ways of collecting performance data for Intel(R) Processor Graphics and provides a set of samples that help to start.

License

Samples for Profiling Tools Interfaces for GPU (PTI for GPU) are distributed under the MIT License.

You may obtain a copy of the License at https://opensource.org/license/MIT

Supported OS

• Linux
• Windows

Supported Platforms

• Intel® Processor Graphics Gen9 (formerly Skylake) and newer
• Intel® Iris® Xe Graphics
• Intel® Data Center GPU Flex Series
• Intel® Data Center GPU Max Series
• Intel® Arc™ Graphics

Some samples may have higher hardware requirements.

Regularly Tested Configurations

• Ubuntu 24.04 with Intel® Arc™ B580 Graphics
• Ubuntu 24.04 with Intel® Data Center GPU Max Series
• Windows 11 with Intel® Arc™ B580 Graphics

Profiling Chapters

1. Runtime API Tracing
   • for OpenCL(TM)
   • for oneAPI Level Zero (Level Zero)
   • for OpenMP*
2. Device Activity Tracing
   • for OpenCL(TM)
   • for oneAPI Level Zero (Level Zero)
   • for SYCL/DPC++
3. Binary/Source Correlation
   • for OpenCL(TM)
   • for oneAPI Level Zero (Level Zero)
4. Metrics Collection
   • based on oneAPI Level Zero (Level Zero) Metric API
   • based on Intel(R) Metrics Discovery Application Programming Interface
   • based on Performance Monitoring (PM) Register
5. Binary Instrumentation
   • based on Graphics Technology Pin (GT Pin)
   • based on OpenCL(TM) built-in intrinsics
6. Code Annotation
   • based on Instrumentation and Tracing Technology API (ITT API)
7. System Management
   • for oneAPI Level Zero (Level Zero)

Profiling & Debug Tools

• unitrace - unified tracing and profiling tool. In addition to Level Zero and/or OpenCL, this tool is capable of profiling software layers in the software stack, for example, SYCL and plugins, oneCCL, MPI etc., for scale-up and scale-out applications. It also supports profiling hardware metrics (including instruction-level EU stalls) and software events at the same time.
• gpuinfo - provides basic information about the GPUs installed in a system, and the list of HW metrics one can collect for it;
• instcount - prints GPU kernel assembly (GEN ISA) annotated by instruction execution count;
• sysmon - Linux "top" like utility to monitor GPUs installed on a system;
• pti - profiling library one can link against to gain the capability of profiling SYCL and Level Zero; used by PyTorch Profiler for the Intel® XPU Activity.

Sample Tools & Utilities

• tools for OpenCL(TM), DPC++ (with OpenCL(TM) backend) and OpenMP* GPU offload (with OpenCL(TM) backend):
  • cl_hot_functions - provides a list of hottest OpenCL(TM) API calls by backend (CPU and GPU);
  • cl_hot_kernels - provides a list of hottest OpenCL(TM) kernels by backend (CPU and GPU);
  • cl_debug_info - prints source and assembly (GEN ISA) for kernels on GPU;
  • cl_gpu_metrics - provides a list of hottest OpenCL(TM) GPU kernels along with percent of cycles it was active, stall and idle (based on continuous metrics collection mode);
  • cl_gpu_query - provides a list of hottest OpenCL(TM) GPU kernels along with percent of cycles it was active, stall and idle (based on query metrics collection mode);
• tools for Level Zero, DPC++ (with Level Zero backend) and OpenMP* GPU offload (with Level Zero backend):
  • ze_hot_functions - provides a list of hottest Level Zero API calls;
  • ze_hot_kernels - provides a list of hottest Level Zero kernels;
  • ze_debug_info - prints source and assembly (GEN ISA) for kernels on GPU;
  • ze_metric_query - provides a list of hottest Level Zero GPU kernels along with percent of cycles it was active, stall and idle (metrics are collected in query mode);
  • ze_metric_streamer - provides a list of hottest Level Zero GPU kernels along with percent of cycles it was active, stall and idle (metrics are collected in streamer mode);
• tools for OpenMP*:
  • omp_hot_regions - provides a list of hottest parallel (for CPU) and target (for GPU) OpenMP* regions;
• utilities:
  • dpc_info - prints information on available platforms and devices in DPC++;
  • ze_info - prints information on available platforms and devices in Level Zero;
  • ze_metric_info - prints the list of HW metrics one can collect with the help of Level Zero;
  • gpu_perfmon_set - allows to choose HW metric for collection in EU PerfMon register;

Prerequisites

• CMake (version 3.22 and above)

• Git (version 1.8 and above)

• Python (version 3.9 and above)

• On Linux one has to be a part of the video (Ubuntu 18 and below) or render (Ubuntu 19 and above) user group to do any computations on Intel(R) Processor Graphics:

  sudo usermod -a -G video <username>
  # OR
  sudo usermod -a -G render <username>
  

• OpenCL(TM) ICD Loader and Headers

  • to use non-standard path to OpenCL ICD library one may add it into LD_LIBRARY_PATH:

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path-to-opencl>
    

• oneAPI Level Zero loader

• Intel(R) Graphics Compute Runtime for oneAPI Level Zero and OpenCL(TM) Driver

• Intel(R) Metrics Discovery Application Programming Interface

  • one may need to install libdrm-dev package to build the library from sources
  • one may need to allow metrics collection for non-root users

    • if using the i915 kernel module (e.g. PVC):

      echo "0" | sudo tee /proc/sys/dev/i915/perf_stream_paranoid
      

    • if using the xe kernel module (e.g. BMG):

      echo "0" | sudo tee /proc/sys/dev/xe/observation_paranoid
      

• Metrics Library for Metrics Discovery API (Metrics Library for MD API)

• Graphics Technology Pin (GT Pin)

• Intel(R) oneAPI Base Toolkit

• libdrm

  • on Ubuntu one may perform:

    sudo apt-get install libdrm-dev
    

More information of what is needed for particular sample can be found on sample description page.

Build and Run

In general, to build samples one need to perform the following steps (specific instructions for particular sample can be found on sample description page):

cd <pti_root>/samples/<sample_root>
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make

To point out to specific headers and libraries one may use -DCMAKE_INCLUDE_PATH and -DCMAKE_LIBRARY_PATH options correspondingly, e.g.:

cmake -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INCLUDE_PATH=/tmp/level_zero/include \
  -DCMAKE_LIBRARY_PATH=/tmp/level_zero/lib \
  ..

Run instructions may vary from sample to sample significantly, so they are provided on particular sample description page.

Testing

There is a way to build and test all the samples in one command, e.g.:

LD_LIBRARY_PATH=/usr/local/lib python <pti_root>/tests/run.py

In case of failed tests, error output will be available in stderr.log file.

It's also possible to test an exact sample or a group of samples, e.g.:

# build and test an exact sample "cl_hot_functions"
python <pti_root>/tests/run.py -s cl_hot_functions
# build and test all L0 samples
python <pti_root>/tests/run.py -s ze

To run testing in debug mode one may use -d option, e.g.:

python <pti_root>/tests/run.py -s ze_gemm -d

The script creates build directory inside each sample folder while testing. To remove all of these folders, use:

python <pti_root>/tests/run.py -c

Unitrace and PTI SDK Experimental Build

The two products in active development are Unitrace and PTI SDK. A new, top-level, CMakeLists.txt file can be used to build them together.
To build them, use CMake in the root of this repository.

source /opt/intel/oneapi/setvars.sh
cmake -S . -B build -GNinja -DCMAKE_BUILD_TYPE=Release
cmake --build build

Tested software versions one may find in SOFTWARE file.

(*) Other names and brands may be claimed as property of others.