CUDA Standard Algorithms » Execution Policy

Parameterize Performance

Taskflow parameterizes most CUDA algorithms in terms of the number of threads per block and units of work per thread, which can be specified in the execution policy template type, tf::cudaExecutionPolicy. The design is inspired by Modern GPU Programming authored by Sean Baxter to achieve high-performance GPU computing.

Define an Execution Policy

The following example defines an execution policy object, policy, which configures (1) each block to invoke 512 threads and (2) each of these 512 threads to perform 11 units of work. Block size must be a power of two. It is always a good idea to specify an odd number in the second parameter to avoid bank conflicts.

tf::cudaExecutionPolicy<512, 11> policy;

By default, the execution policy object is associated with the CUDA default stream (i.e., 0). Default stream can incur significant overhead due to the global synchronization. You can associate an execution policy with another stream as shown below.

// assign a stream to a policy at construction time
tf::cudaExecutionPolicy<512, 11> policy(my_stream);
// reassign another stream to a policy
policy.stream(another_stream);

The best-performing configurations for each algorithm, each GPU architecture, and each data type can vary significantly. You should experiment different configurations and find the optimal tuning parameters for your applications. A default policy is given in tf::cudaDefaultExecutionPolicy.

tf::cudaDefaultExecutionPolicy default_policy;