Use the HPC Performance Characterization viewpoint to estimate CPU usage, memory efficiency, and floating-point utilization for compute-intensive or throughput applications. Compute-intensive or throughput applications should use hardware resources efficiently for the duration of their elapsed time. Use the HPC Performance Characterization analysis as a starting point for optimizing application performance and runtime.
To interpret the performance data provided in the HPC Performance Characterization viewpoint, you may follow the steps below:
Tip
Use the Analyzing an OpenMP* and MPI Application tutorial to review basic steps for tuning a hybrid application. The tutorial is available from the Intel Developer Zone at https://software.intel.com/en-us/articles/intel-vtune-amplifier-tutorials.
1. Define a Performance Baseline
Start with exploring the Summary window that provides general information on your application execution. Key areas for optimization include the elapsed time and floating-point operation per second counts (GFLOPS). Red text indicates an area of potential optimization. Hover over a flag to learn more about how to improve your code.
Use the Elapsed Time and GFLOPS or GFLOPS Upper Bound (Intel® Xeon Phi™ only) values as a baseline for comparison of versions before and after optimization.
2. Determine Optimization Opportunities
Review the Summary window to find the key optimization opportunities for your application. Performance metrics that can be improved are marked in red. Issues identified could include CPU Utilization, Memory Bound, FPU Utilization, or a combination of these. The following sections provide suggested next steps for each performance aspect:
CPU Utilization
Compare the serial time to the parallel region time. If the serial portion is significant, consider options to minimize serial execution, either by introducing more parallelism or by doing algorithm or microarchitecture tuning for sections that seem unavoidably serial. For high thread-count machines, serial sections have a severe negative impact on potential scaling (Amdahl's Law) and should be minimized as much as possible.
For applications with Intel OpenMP* or MPI runtime libraries, review the OpenMP Potential Gain to estimate the efficiency of OpenMP parallelization in the parallel part of the code and check for an MPI imbalance. The Potential Gain metric estimates the elapsed time between the actual measurement and an idealized execution of parallel regions, assuming perfectly balanced threads and zero overhead of the OpenMP runtime on work arrangement. Use this data to understand the maximum time that you may save by improving parallel execution. If Potential Gain for a region is significant, you can go deeper and select the link on a region name to navigate to the Bottom-up window employing an OpenMP Region dominant grouping and the region of interest selection. For example, the Tasking (OpenMP) column shows how much time was spent allocating and completing OpenMP tasks. This data is used to help calculate the total potential gain. You can improve your OpenMP overhead by reducing the task granularity and therefore reducing the time spent tasking.
For applications that do not use OpenMP or MPI runtime libraries, review the CPU Usage Histogram, which displays the Elapsed Time of your application, broken down by CPU utilization levels. Use the data in the Bottom-up and Top-down Tree windows to identify the most time-consuming functions in your application by CPU utilization. Focus on the functions with the largest CPU time and low CPU utilization level as your candidates for optimization (for example, parallelization).
Consider running a Locks and Waits or Concurrency analysis to review issues related to threading efficiency.
Memory Bound
A high Memory Bound value might indicate that a significant portion of execution time was lost while fetching data. The section shows a fraction of cycles that were lost in stalls being served in different cache hierarchy levels (L1, L2, L3) or fetching data from DRAM. For last level cache misses that lead to DRAM, it is important to distinguish if the stalls were because of a memory bandwidth limit since they can require specific optimization techniques when compared to latency bound stalls. VTune Amplifier shows a hint about identifying this issue in the DRAM Bound metric issue description. This section also offers the percentage of accesses to a remote socket compared to a local socket to see if memory stalls can be connected with NUMA issues.
For Intel® Xeon Phi™ processors formerly code named Knights Landing, there is no way to measure memory stalls to assess memory access efficiency in general. Therefore Back-end Bound stalls that include memory-related stalls as a high-level characterization metric are shown instead. The second level metrics are focused particularly on memory access efficiency. A high L2 Hit Bound or L2 Miss Bound value indicates that a high ratio of cycles were spent handing L2 hits or misses. A high DRAM Bandwidth Bound or MCDRAM Bandwidth Bound value indicates that a large percentage of the overall elapsed time was spent with high bandwidth utilization. A high DRAM Bandwidth Bound value is an opportunity to run the Memory Access analysis to identify data structures that can be allocated in high bandwidth memory (MCDRAM), if it is available.
The Bandwidth Utilization Histogram shows how much time the system bandwidth was utilized by a certain value (Bandwidth Domain) and provides thresholds to categorize bandwidth utilization as High, Medium and Low. The thresholds are calculated based on benchmarks that calculate the maximum value. You can also set the threshold by moving sliders at the bottom of the histogram. The modified values are applied to all subsequent results in the project.
Switch to the Bottom-up window and review the Memory Bound columns in the grid to determine optimization opportunities.
If your application is memory bound, consider running a Memory Access analysis for deeper metrics and the ability to correlate these metrics with memory objects.
FPU Utilization
Expand the GFLOPS or GFLOPS Upper Bound (Intel Xeon Phi only) section to show the number of Scalar and Packed GFLOPS. This section provides a quick estimate of the amount of FLOPs that were not vectorized.
GFLOP metrics on Intel Xeon Phi processors formerly code named Knights Landing are calculated assuming that vector instructions operate with full 512-bit vectors - FLOP upper bound. A poor upper bound value often reflects a performance issue. Use Intel Advisor for more accurate FLOP numbers.
The Top 5 Hotspot Loops by FPU Usage table shows the top 5 functions that contain floating point operations sorted by CPU time. The FPU Utilization column provides issue descriptions based on whether a loop/function is bandwidth bound, vector instruction set, scalar, or vector and will provide direction on fixing memory bandwidth, changing the vector instruction set, or using Intel Advisor to improve FPU utilization.
Note
FPU and GFLOPS metrics are supported on 3rd Generation Intel Core™ processors, 5th Generation Intel processors, and 6th Generation Intel processors. Limited support is available for Intel® Xeon Phi™ processors formerly code named Knights Landing. The metrics are not currently available on 4th Generation Intel processors. Expand the Details section on the analysis configuration pane to view the processor family available on your system.
3. Analyze Source
Double-click the function you want to optimize to view its related source code file in the Source/Assembly window. You can open the code editor directly from the Intel® VTune™ Amplifier and edit your code (for example, minimizing the number of calls to the hotspot function).
4. Explore Other Analysis Types
Run a memory access analysis to view more detail about cache bound and memory bound issues affecting the performance of your application.
Use the Intel Advisor to analyze the application for vectorization optimization.