What is GPU Boost?

GPU Boost is a new user controllable feature to change theprocessor clock speed on the Tesla K40 GPU. NVIDIA is currentlysupporting 4 selectable Stream Processor clock speeds and twoselectable Memory Clock Speeds on the K40.  Thebase clock for the Stream Processors is 745MHz and the threeselectable options are 666 MHz, 810 MHz and 875MHz (finally, amanufacturer not afraid of superstition!). The base Memory Clockfrequencies are 3004MHz (default) and 324MHz (idle). Only theeffects of tuning the Stream Processor clock are discussed as thereis no application performance increase that results from adjustingthe Memory Clock. This blog shows the impact of GPU Boost on aseismic imaging application (Reverse Time Migration) and anelectromagnetic solver (Finite-difference time-domain).

GPU Boost is useful as not all applications have the same powerprofile. The K40 has a maximum 235W power capacity. For example, anapplication that runs at an average power consumption of 180W atthe base frequency will have a 55W power headroom. By increasingthe clock frequency, the application theoretically can takeadvantage of the full 235W capacity.

Enabling GPU Boost

GPU Boost is controlled using NVIDIA’s System ManagementInterface utility (nvidia-smi) with the followingcommands:

Command

Explanation

nvidia-smi –q –dSUPPORTED_CLOCKS

Show Supported ClockFrequencies

nvidia-smi –ac

Set the Memory and Graphics ClockFrequency

nvidia-smi –q –dCLOCK

Shows current mode

nvidia-smi –rac

Resets all clocks

nvidia-smi –acp0

Allows non-root to changeclocks

On the K40, an nvidia-smi query to find thesupported clock frequencies gives the following output:

[srahim@corsair3 ~]$ nvidia-smi -q -dSUPPORTED_CLOCKS
==============NVSMI LOG==============
Timestamp                          :Mon Mar 10 17:38:22 2014
DriverVersion                     : 331.20

AttachedGPUs                      : 2
GPU 0000:05:00.0
Supported Clocks
Memory                     : 3004 MHz
Graphics               : 875 MHz
Graphics               : 810 MHz
Graphics               : 745 MHz
Graphics               : 666 MHz
Memory                     : 324 MHz
Graphics               : 324 MHz

GPU 0000:42:00.0
Supported Clocks
Memory                     : 3004 MHz
Graphics               : 875 MHz
Graphics               : 810 MHz
Graphics               : 745 MHz
Graphics               : 666 MHz
Memory                     : 324 MHz
Graphics               : 324 MHz

 

To set the clock speed to 666MHz, run

[srahim@corsair3 ~]$ sudo nvidia-smi –ac3004,666
Applications clocks set to "(MEM 3004, SM 666)" for GPU0000:05:00.0
All done.

If you try to set an unsupported clock speed, nvidia-smi shows ahelpful message.

[srahim@corsair3 ~]$ sudo nvidia-smi -ac 3004,888 Specified clock combination "(MEM 3004, SM 888)" is not supportedfor GPU 0000:05:00.0. Run 'nvidia-smi -q -d SUPPORTED_CLOCKS' tosee list of supported clock combinations Terminating early due to previous errors.

The current clock speed is checked with the followingcommand:

[srahim@corsair3 ~]$ nvidia-smi -q -d CLOCK
==============NVSMI LOG==============

Timestamp                          : Tue Mar 11 10:40:16 2014
DriverVersion                     : 331.20

AttachedGPUs                      : 2
GPU 0000:05:00.0
Clocks
Graphics                   : 324 MHz
SM                         : 324 MHz
Memory                     : 324 MHz
Applications Clocks
Graphics                   : 666 MHz
Memory                     :3004 MHz
Default Applications Clocks
Graphics                   : 745 MHz
Memory                     : 3004 MHz
MaxClocks
Graphics                   : 875 MHz
SM                         : 875 MHz
Memory                     : 3004 MHz

The GPU Boost settings are not persistent between reboots ordriver unloads and should be scripted if persistence is desired.Unless the NVIDIA driver persistence mode is set withnvidia-smi –pm 1, the driver may unload when the GPUis idle.

Unlike Intel’s Turbo Boost, GPU Boost is not on by default. Thisputs the impetus on the end user or system administrator to takeadvantage of this feature. An application can programmaticallychange the Boost clock with NVML if run with appropriate permissions. Runnvidia-smi –acp 0 to grant non-root users permissionto change clocks. Two caveats on the uses of GPU Boost from thedocument “NVIDIA GPU Boost for Tesla” are:

1. An important point to remember is that no matter which clocksthe end user selects, if at any time the power monitoring algorithmdetects that the application may exceed the 235 W, the GPU comesdown to a lower clock level as a precaution. Once the power fallsbelow 235 W the GPU will raise its core clock to the selectedclock. This happens automatically and the Tesla K40 does have a fewclock levels below the base clock to handle any powerdigressions.
2. If the workload runs on multiple GPUs and is sensitive to allGPUs running at the same clock, the user may need to try out whichparticular clock works best for all GPUs.

If you are not running on multiple GPUs or multiple nodes, youshould crank up the GPU clock speed to the maximum frequency. Keepthe GPU clock at the default value of 745MHz or lower only if thereare power consumption concerns.

GPU Boost is a relatively simple way to increase performance byup to 30%. It is certainly easier than optimizing a CUDA kernel.Unless you are doing some crazy multi-node MPI application, alwaysboost your clock speed for higher performance!