OV - - Loop 2

	Loop Id: 2	Module: exec	Source: Step10_orig.c:19-35	Coverage: 0.02%

0x401b70 VMOVSS	(%RDX,%RAX,4),%XMM17    [8]

0x401b77 VMOVSS	(%RSI,%RAX,4),%XMM21    [4]

0x401b7e VXORPS	%XMM31,%XMM31,%XMM31

0x401b84 VMOVSS	(%RCX,%RAX,4),%XMM28    [2]

0x401b8b VSUBSS	%XMM1,%XMM17,%XMM16

0x401b91 VSUBSS	%XMM14,%XMM21,%XMM18

0x401b97 VSUBSS	%XMM2,%XMM28,%XMM30

0x401b9d VMULSS	%XMM16,%XMM16,%XMM0

0x401ba3 VFMADD231SS	%XMM18,%XMM18,%XMM0

0x401ba9 VFMADD231SS	%XMM30,%XMM30,%XMM0

0x401baf VCOMISS	%XMM0,%XMM3

0x401bb3 JBE	401bbc

0x401bb5 VMOVSS	(%R8,%RAX,4),%XMM31    [6]

0x401bbc VCOMISS	%XMM9,%XMM0

0x401bc1 JBE	401c25

0x401bc3 VMOVAPS	%XMM0,%XMM29

0x401bc9 VADDSS	%XMM0,%XMM4,%XMM5

0x401bcd VFMADD132SS	%XMM10,%XMM11,%XMM29

0x401bd3 VCVTSS2SD	%XMM5,%XMM5,%XMM5

0x401bd7 VSQRTSD	%XMM5,%XMM5,%XMM27

0x401bdd VMULSD	%XMM27,%XMM5,%XMM5

0x401be3 VDIVSD	%XMM5,%XMM24,%XMM5

0x401be9 VFMADD132SS	%XMM0,%XMM12,%XMM29

0x401bef VFMADD132SS	%XMM0,%XMM13,%XMM29

0x401bf5 VFMADD132SS	%XMM0,%XMM15,%XMM29

0x401bfb VFMADD132SS	%XMM29,%XMM25,%XMM0

0x401c01 VCVTSS2SD	%XMM0,%XMM0,%XMM0

0x401c05 VADDSD	%XMM0,%XMM5,%XMM5

0x401c09 VCVTSD2SS	%XMM5,%XMM5,%XMM0

0x401c0d VMULSS	%XMM31,%XMM0,%XMM5

0x401c13 VFMADD231SS	%XMM5,%XMM18,%XMM6

0x401c19 VFMADD231SS	%XMM5,%XMM16,%XMM7

0x401c1f VFMADD231SS	%XMM5,%XMM30,%XMM8

0x401c25 INC	%RAX

0x401c28 VMOVSS	(%RDX,%RAX,4),%XMM29    [7]

0x401c2f VMOVSS	(%RSI,%RAX,4),%XMM30    [3]

0x401c36 VXORPS	%XMM20,%XMM20,%XMM20

0x401c3c VMOVSS	(%RCX,%RAX,4),%XMM26    [1]

0x401c43 VSUBSS	%XMM1,%XMM29,%XMM27

0x401c49 VSUBSS	%XMM14,%XMM30,%XMM31

0x401c4f VSUBSS	%XMM2,%XMM26,%XMM23

0x401c55 VMULSS	%XMM27,%XMM27,%XMM0

0x401c5b VFMADD231SS	%XMM31,%XMM31,%XMM0

0x401c61 VFMADD231SS	%XMM23,%XMM23,%XMM0

0x401c67 VCOMISS	%XMM0,%XMM3

0x401c6b JBE	401c74

0x401c6d VMOVSS	(%R8,%RAX,4),%XMM20    [5]

0x401c74 VCOMISS	%XMM9,%XMM0

0x401c79 JBE	401cdd

0x401c7b VMOVAPS	%XMM0,%XMM22

0x401c81 VADDSS	%XMM0,%XMM4,%XMM5

0x401c85 VFMADD132SS	%XMM10,%XMM11,%XMM22

0x401c8b VCVTSS2SD	%XMM5,%XMM5,%XMM5

0x401c8f VSQRTSD	%XMM5,%XMM5,%XMM19

0x401c95 VMULSD	%XMM19,%XMM5,%XMM5

0x401c9b VDIVSD	%XMM5,%XMM24,%XMM5

0x401ca1 VFMADD132SS	%XMM0,%XMM12,%XMM22

0x401ca7 VFMADD132SS	%XMM0,%XMM13,%XMM22

0x401cad VFMADD132SS	%XMM0,%XMM15,%XMM22

0x401cb3 VFMADD132SS	%XMM22,%XMM25,%XMM0

0x401cb9 VCVTSS2SD	%XMM0,%XMM0,%XMM0

0x401cbd VADDSD	%XMM0,%XMM5,%XMM5

0x401cc1 VCVTSD2SS	%XMM5,%XMM5,%XMM0

0x401cc5 VMULSS	%XMM20,%XMM0,%XMM5

0x401ccb VFMADD231SS	%XMM5,%XMM31,%XMM6

0x401cd1 VFMADD231SS	%XMM5,%XMM27,%XMM7

0x401cd7 VFMADD231SS	%XMM5,%XMM23,%XMM8

0x401cdd INC	%RAX

0x401ce0 CMP	%EAX,%EDI

0x401ce2 JG	401b70

/beegfs/hackathon/users/eoseret/qaas_runs/171-094-7986/intel/HACCmk/build/HACCmk/src/Step10_orig.c: 19 - 35

--------------------------------------------------------------------------------

Path /

Metric	Value
CQA speedup if no scalar integer	1.00
CQA speedup if FP arith vectorized	4.00
CQA speedup if fully vectorized	4.00
CQA speedup if no inter-iteration dependency	NA
CQA speedup if next bottleneck killed	1.80
Bottlenecks	P8, P9,
Function	Step10_orig
Source	Step10_orig.c:19-35
Source loop unroll info	not unrolled or unrolled with no peel/tail loop
Source loop unroll confidence level	max
Unroll/vectorization loop type	NA
Unroll factor	NA
CQA cycles	27.00
CQA cycles if no scalar integer	27.00
CQA cycles if FP arith vectorized	6.75
CQA cycles if fully vectorized	6.75
Front-end cycles	12.17
DIV/SQRT cycles	2.50
P0 cycles	0.75
P1 cycles	0.75
P2 cycles	0.50
P3 cycles	2.50
P4 cycles	2.67
P5 cycles	2.67
P6 cycles	2.67
P7 cycles	15.00
P8 cycles	15.00
P9 cycles	10.00
P10 cycles	10.00
P11 cycles	2.00
P12 cycles	2.00
P13 cycles	27.00
Inter-iter dependencies cycles	NA
FE+BE cycles (UFS)	NA
Stall cycles (UFS)	NA
Nb insns	70.00
Nb uops	73.00
Nb loads	8.00
Nb stores	0.00
Nb stack references	0.00
FLOP/cycle	2.22
Nb FLOP add-sub	10.00
Nb FLOP mul	6.00
Nb FLOP fma	20.00
Nb FLOP div	2.00
Nb FLOP rcp	0.00
Nb FLOP sqrt	2.00
Nb FLOP rsqrt	0.00
Bytes/cycle	1.19
Bytes prefetched	0.00
Bytes loaded	32.00
Bytes stored	0.00
Stride 0	NA
Stride 1	NA
Stride n	NA
Stride unknown	NA
Stride indirect	NA
Vectorization ratio all	6.45
Vectorization ratio load	0.00
Vectorization ratio store	NA
Vectorization ratio mul	0.00
Vectorization ratio add_sub	0.00
Vectorization ratio fma	0.00
Vectorization ratio div_sqrt	0.00
Vectorization ratio other	28.57
Vector-efficiency ratio all	8.47
Vector-efficiency ratio load	6.25
Vector-efficiency ratio store	NA
Vector-efficiency ratio mul	8.33
Vector-efficiency ratio add_sub	7.50
Vector-efficiency ratio fma	6.25
Vector-efficiency ratio div_sqrt	12.50
Vector-efficiency ratio other	12.50

Metric	Value
CQA speedup if no scalar integer	1.00
CQA speedup if FP arith vectorized	4.00
CQA speedup if fully vectorized	4.00
CQA speedup if no inter-iteration dependency	NA
CQA speedup if next bottleneck killed	1.80
Bottlenecks	P8, P9,
Function	Step10_orig
Source	Step10_orig.c:19-35
Source loop unroll info	not unrolled or unrolled with no peel/tail loop
Source loop unroll confidence level	max
Unroll/vectorization loop type	NA
Unroll factor	NA
CQA cycles	27.00
CQA cycles if no scalar integer	27.00
CQA cycles if FP arith vectorized	6.75
CQA cycles if fully vectorized	6.75
Front-end cycles	12.17
DIV/SQRT cycles	2.50
P0 cycles	0.75
P1 cycles	0.75
P2 cycles	0.50
P3 cycles	2.50
P4 cycles	2.67
P5 cycles	2.67
P6 cycles	2.67
P7 cycles	15.00
P8 cycles	15.00
P9 cycles	10.00
P10 cycles	10.00
P11 cycles	2.00
P12 cycles	2.00
P13 cycles	27.00
Inter-iter dependencies cycles	NA
FE+BE cycles (UFS)	NA
Stall cycles (UFS)	NA
Nb insns	70.00
Nb uops	73.00
Nb loads	8.00
Nb stores	0.00
Nb stack references	0.00
FLOP/cycle	2.22
Nb FLOP add-sub	10.00
Nb FLOP mul	6.00
Nb FLOP fma	20.00
Nb FLOP div	2.00
Nb FLOP rcp	0.00
Nb FLOP sqrt	2.00
Nb FLOP rsqrt	0.00
Bytes/cycle	1.19
Bytes prefetched	0.00
Bytes loaded	32.00
Bytes stored	0.00
Stride 0	NA
Stride 1	NA
Stride n	NA
Stride unknown	NA
Stride indirect	NA
Vectorization ratio all	6.45
Vectorization ratio load	0.00
Vectorization ratio store	NA
Vectorization ratio mul	0.00
Vectorization ratio add_sub	0.00
Vectorization ratio fma	0.00
Vectorization ratio div_sqrt	0.00
Vectorization ratio other	28.57
Vector-efficiency ratio all	8.47
Vector-efficiency ratio load	6.25
Vector-efficiency ratio store	NA
Vector-efficiency ratio mul	8.33
Vector-efficiency ratio add_sub	7.50
Vector-efficiency ratio fma	6.25
Vector-efficiency ratio div_sqrt	12.50
Vector-efficiency ratio other	12.50

Path /

Average path: Display a virtual path defined by average values of all real paths

Function	Step10_orig
Source file and lines	Step10_orig.c:19-35
Module	exec

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs/171-094-7986/intel/HACCmk/build/HACCmk/src/Step10_orig.c:19-35.

The related source loop is not unrolled or unrolled with no peel/tail loop.
Warnings:

Ignoring paths for analysis
Failed to get the number of paths
RecMII not computed since number of paths is unknown or > max_paths
Streams not analyzed since number of paths is unknown or > max_paths

gain
potential
hint
expert

Vectorization

Your loop is probably not vectorized. Only 8% of vector register length is used (average across all SSE/AVX instructions). By vectorizing your loop, you can lower the cost of an iteration from 27.00 to 6.75 cycles (4.00x speedup).

Details

Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers). Since your execution units are vector units, only a vectorized loop can use their full power.

Workaround

Try another compiler or update/tune your current one
Remove inter-iterations dependences from your loop and make it unit-stride:
- If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly: C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)
- If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA): for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)

Execution units bottlenecks

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck). By removing all these bottlenecks, you can lower the cost of an iteration from 27.00 to 15.00 cycles (1.80x speedup).

Workaround

Reduce the number of division or square root instructions:
- If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
If you accept to loose numerical precision up to 2 ulp, you can speedup your code by passing the following options to your compiler: (ffast-math or Ofast) and mrecip

Expensive FP math instructions/calls

Detected performance impact from expensive FP math instructions/calls. By removing/reexpressing them, you can lower the cost of an iteration from 27.00 to 13.00 cycles (2.08x speedup).

FMA

Detected 20 FMA (fused multiply-add) operations. Presence of both ADD/SUB and MUL operations.

Workaround

Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible. For instance a + b*c is a valid FMA (MUL then ADD). However (a+b)* c cannot be translated into an FMA (ADD then MUL).

Complex instructions

Detected COMPLEX INSTRUCTIONS.

Details

These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.

VCOMISS: 4 occurrences

Conversion instructions

Detected expensive conversion instructions.

Details

VCVTSD2SS (FP64 to FP32, scalar): 2 occurrences
VCVTSS2SD (FP32 to FP64, scalar): 4 occurrences

Workaround

Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements. Use double instead of single precision only when/where needed by numerical stability and avoid mixing precision. In C/C++, FP constants are double precision by default and must be suffixed by 'f' to make them single precision.

Type of elements and instruction set

36 SSE or AVX instructions are processing arithmetic or math operations on single precision FP elements in scalar mode (one at a time). 8 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).

Matching between your loop (in the source code) and the binary loop

The binary loop is composed of 60 FP arithmetical operations:

30: addition or subtraction (20 inside FMA instructions)
26: multiply (20 inside FMA instructions)
2: divide
2: square root

The binary loop is loading 32 bytes (8 single precision FP elements).

Arithmetic intensity

Arithmetic intensity is 1.87 FP operations per loaded or stored byte.

General properties

nb instructions	70
nb uops	73
loop length	376
used x86 registers	6
used mmx registers	0
used xmm registers	32
used ymm registers	0
used zmm registers	0
nb stack references	0
ADD-SUB / MUL ratio	1.67

Front-end

ASSUMED MACRO FUSION FIT IN UOP CACHE

micro-operation queue	12.17 cycles
front end	12.17 cycles

Back-end

	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP0	FP1	FP2	FP3	FP4	FP5
uops	2.50	0.75	0.75	0.50	2.50	2.67	2.67	2.67	15.00	15.00	10.00	10.00	2.00	2.00
cycles	2.50	0.75	0.75	0.50	2.50	2.67	2.67	2.67	15.00	15.00	10.00	10.00	2.00	2.00

Execution ports to units layout:

ALU0/BRU0: ALU
ALU1: ALU
ALU2: ALU
ALU3: ALU
BRU1:
AGU0 (256 bits): store address, load
AGU1 (256 bits): store address, load
AGU2 (256 bits): store address, load
FP0 (256 bits): VPU, DIV/SQRT
FP1 (256 bits): VPU, DIV/SQRT
FP2 (256 bits): VPU
FP3 (256 bits): VPU
FP4 (256 bits): FP store data
FP5 (256 bits): FP store data

Cycles executing div or sqrt instructions

27.00

Cycles summary

Front-end	12.17
Dispatch	15.00
DIV/SQRT	27.00
Overall L1	27.00

Vectorization ratios

all	6%
load	0%
store	NA (no store vectorizable/vectorized instructions)
mul	0%
add-sub	0%
fma	0%
div/sqrt	0%
other	28%

Vector efficiency ratios

all	8%
load	6%
store	NA (no store vectorizable/vectorized instructions)
mul	8%
add-sub	7%
fma	6%
div/sqrt	12%
other	12%

Cycles and memory resources usage

Assuming all data fit into the L1 cache, each iteration of the binary loop takes 27.00 cycles. At this rate:

1% of peak load performance is reached (1.19 out of 96.00 bytes loaded per cycle (GB/s @ 1GHz))

Front-end bottlenecks

Found no such bottlenecks.

ASM code

In the binary file, the address of the loop is: 401c28

Instruction	Nb FU	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP0	FP1	FP2	FP3	FP4	FP5	Latency	Recip. throughput
VMOVSS (%RDX,%RAX,4),%XMM17	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VMOVSS (%RSI,%RAX,4),%XMM21	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VXORPS %XMM31,%XMM31,%XMM31	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VMOVSS (%RCX,%RAX,4),%XMM28	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VSUBSS %XMM1,%XMM17,%XMM16	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM14,%XMM21,%XMM18	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM2,%XMM28,%XMM30	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM16,%XMM16,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM18,%XMM18,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM30,%XMM30,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCOMISS %XMM0,%XMM3	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401bbc <Step10_orig+0x57c>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVSS (%R8,%RAX,4),%XMM31	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VCOMISS %XMM9,%XMM0	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401c25 <Step10_orig+0x5e5>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVAPS %XMM0,%XMM29	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VADDSS %XMM0,%XMM4,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VFMADD132SS %XMM10,%XMM11,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM5,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSQRTSD %XMM5,%XMM5,%XMM27	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	21	8.50
VMULSD %XMM27,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VDIVSD %XMM5,%XMM24,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	13	5
VFMADD132SS %XMM0,%XMM12,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM13,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM15,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM29,%XMM25,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM0,%XMM0,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VADDSD %XMM0,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VCVTSD2SS %XMM5,%XMM5,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM31,%XMM0,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM5,%XMM18,%XMM6	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM16,%XMM7	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM30,%XMM8	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
INC %RAX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	0	0	0	0	1	0.25
VMOVSS (%RDX,%RAX,4),%XMM29	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VMOVSS (%RSI,%RAX,4),%XMM30	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VXORPS %XMM20,%XMM20,%XMM20	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VMOVSS (%RCX,%RAX,4),%XMM26	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VSUBSS %XMM1,%XMM29,%XMM27	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM14,%XMM30,%XMM31	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM2,%XMM26,%XMM23	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM27,%XMM27,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM31,%XMM31,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM23,%XMM23,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCOMISS %XMM0,%XMM3	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401c74 <Step10_orig+0x634>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVSS (%R8,%RAX,4),%XMM20	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VCOMISS %XMM9,%XMM0	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401cdd <Step10_orig+0x69d>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVAPS %XMM0,%XMM22	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VADDSS %XMM0,%XMM4,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VFMADD132SS %XMM10,%XMM11,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM5,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSQRTSD %XMM5,%XMM5,%XMM19	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	21	8.50
VMULSD %XMM19,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VDIVSD %XMM5,%XMM24,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	13	5
VFMADD132SS %XMM0,%XMM12,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM13,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM15,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM22,%XMM25,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM0,%XMM0,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VADDSD %XMM0,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VCVTSD2SS %XMM5,%XMM5,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM20,%XMM0,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM5,%XMM31,%XMM6	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM27,%XMM7	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM23,%XMM8	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
INC %RAX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	0	0	0	0	1	0.25
CMP %EAX,%EDI	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	0	0	0	0	1	0.25
JG 401b70 <Step10_orig+0x530>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1

Function	Step10_orig
Source file and lines	Step10_orig.c:19-35
Module	exec

Ignoring paths for analysis
Failed to get the number of paths
RecMII not computed since number of paths is unknown or > max_paths
Streams not analyzed since number of paths is unknown or > max_paths

gain
potential
hint
expert

Vectorization

Details

Workaround

Try another compiler or update/tune your current one
Remove inter-iterations dependences from your loop and make it unit-stride:
- If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly: C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)
- If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA): for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)

Execution units bottlenecks

Workaround

Reduce the number of division or square root instructions:
- If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
If you accept to loose numerical precision up to 2 ulp, you can speedup your code by passing the following options to your compiler: (ffast-math or Ofast) and mrecip

Expensive FP math instructions/calls

Detected performance impact from expensive FP math instructions/calls. By removing/reexpressing them, you can lower the cost of an iteration from 27.00 to 13.00 cycles (2.08x speedup).

FMA

Detected 20 FMA (fused multiply-add) operations. Presence of both ADD/SUB and MUL operations.

Workaround

Complex instructions

Detected COMPLEX INSTRUCTIONS.

Details

These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.

VCOMISS: 4 occurrences

Conversion instructions

Detected expensive conversion instructions.

Details

VCVTSD2SS (FP64 to FP32, scalar): 2 occurrences
VCVTSS2SD (FP32 to FP64, scalar): 4 occurrences

Workaround

Type of elements and instruction set

Matching between your loop (in the source code) and the binary loop

The binary loop is composed of 60 FP arithmetical operations:

30: addition or subtraction (20 inside FMA instructions)
26: multiply (20 inside FMA instructions)
2: divide
2: square root

The binary loop is loading 32 bytes (8 single precision FP elements).

Arithmetic intensity

Arithmetic intensity is 1.87 FP operations per loaded or stored byte.

General properties

nb instructions	70
nb uops	73
loop length	376
used x86 registers	6
used mmx registers	0
used xmm registers	32
used ymm registers	0
used zmm registers	0
nb stack references	0
ADD-SUB / MUL ratio	1.67

Front-end

ASSUMED MACRO FUSION FIT IN UOP CACHE

micro-operation queue	12.17 cycles
front end	12.17 cycles

Back-end

	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP0	FP1	FP2	FP3	FP4	FP5
uops	2.50	0.75	0.75	0.50	2.50	2.67	2.67	2.67	15.00	15.00	10.00	10.00	2.00	2.00
cycles	2.50	0.75	0.75	0.50	2.50	2.67	2.67	2.67	15.00	15.00	10.00	10.00	2.00	2.00

Execution ports to units layout:

ALU0/BRU0: ALU
ALU1: ALU
ALU2: ALU
ALU3: ALU
BRU1:
AGU0 (256 bits): store address, load
AGU1 (256 bits): store address, load
AGU2 (256 bits): store address, load
FP0 (256 bits): VPU, DIV/SQRT
FP1 (256 bits): VPU, DIV/SQRT
FP2 (256 bits): VPU
FP3 (256 bits): VPU
FP4 (256 bits): FP store data
FP5 (256 bits): FP store data

Cycles executing div or sqrt instructions

27.00

Cycles summary

Front-end	12.17
Dispatch	15.00
DIV/SQRT	27.00
Overall L1	27.00

Vectorization ratios

all	6%
load	0%
store	NA (no store vectorizable/vectorized instructions)
mul	0%
add-sub	0%
fma	0%
div/sqrt	0%
other	28%

Vector efficiency ratios

all	8%
load	6%
store	NA (no store vectorizable/vectorized instructions)
mul	8%
add-sub	7%
fma	6%
div/sqrt	12%
other	12%

Cycles and memory resources usage

Assuming all data fit into the L1 cache, each iteration of the binary loop takes 27.00 cycles. At this rate:

1% of peak load performance is reached (1.19 out of 96.00 bytes loaded per cycle (GB/s @ 1GHz))

Front-end bottlenecks

Found no such bottlenecks.

ASM code

In the binary file, the address of the loop is: 401c28

Instruction	Nb FU	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP0	FP1	FP2	FP3	FP4	FP5	Latency	Recip. throughput
VMOVSS (%RDX,%RAX,4),%XMM17	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VMOVSS (%RSI,%RAX,4),%XMM21	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VXORPS %XMM31,%XMM31,%XMM31	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VMOVSS (%RCX,%RAX,4),%XMM28	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VSUBSS %XMM1,%XMM17,%XMM16	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM14,%XMM21,%XMM18	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM2,%XMM28,%XMM30	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM16,%XMM16,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM18,%XMM18,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM30,%XMM30,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCOMISS %XMM0,%XMM3	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401bbc <Step10_orig+0x57c>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVSS (%R8,%RAX,4),%XMM31	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VCOMISS %XMM9,%XMM0	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401c25 <Step10_orig+0x5e5>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVAPS %XMM0,%XMM29	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VADDSS %XMM0,%XMM4,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VFMADD132SS %XMM10,%XMM11,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM5,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSQRTSD %XMM5,%XMM5,%XMM27	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	21	8.50
VMULSD %XMM27,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VDIVSD %XMM5,%XMM24,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	13	5
VFMADD132SS %XMM0,%XMM12,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM13,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM15,%XMM29	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM29,%XMM25,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM0,%XMM0,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VADDSD %XMM0,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VCVTSD2SS %XMM5,%XMM5,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM31,%XMM0,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM5,%XMM18,%XMM6	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM16,%XMM7	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM30,%XMM8	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
INC %RAX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	0	0	0	0	1	0.25
VMOVSS (%RDX,%RAX,4),%XMM29	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VMOVSS (%RSI,%RAX,4),%XMM30	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VXORPS %XMM20,%XMM20,%XMM20	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VMOVSS (%RCX,%RAX,4),%XMM26	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VSUBSS %XMM1,%XMM29,%XMM27	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM14,%XMM30,%XMM31	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSUBSS %XMM2,%XMM26,%XMM23	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM27,%XMM27,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM31,%XMM31,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM23,%XMM23,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCOMISS %XMM0,%XMM3	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401c74 <Step10_orig+0x634>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVSS (%R8,%RAX,4),%XMM20	1	0	0	0	0	0	0.33	0.33	0.33	0	0	0	0	0	0	1	0.50
VCOMISS %XMM9,%XMM0	2	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0.50	0.50	6	1
JBE 401cdd <Step10_orig+0x69d>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1
VMOVAPS %XMM0,%XMM22	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0.17
VADDSS %XMM0,%XMM4,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VFMADD132SS %XMM10,%XMM11,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM5,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VSQRTSD %XMM5,%XMM5,%XMM19	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	21	8.50
VMULSD %XMM19,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VDIVSD %XMM5,%XMM24,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	13	5
VFMADD132SS %XMM0,%XMM12,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM13,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM0,%XMM15,%XMM22	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD132SS %XMM22,%XMM25,%XMM0	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VCVTSS2SD %XMM0,%XMM0,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VADDSD %XMM0,%XMM5,%XMM5	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VCVTSD2SS %XMM5,%XMM5,%XMM0	1	0	0	0	0	0	0	0	0	0	0	0.50	0.50	0	0	3	0.50
VMULSS %XMM20,%XMM0,%XMM5	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	3	0.50
VFMADD231SS %XMM5,%XMM31,%XMM6	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM27,%XMM7	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
VFMADD231SS %XMM5,%XMM23,%XMM8	1	0	0	0	0	0	0	0	0	0.50	0.50	0	0	0	0	4	0.50
INC %RAX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	0	0	0	0	1	0.25
CMP %EAX,%EDI	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	0	0	0	0	1	0.25
JG 401b70 <Step10_orig+0x530>	1	0.50	0	0	0	0.50	0	0	0	0	0	0	0	0	0	1	0.50-1

▶Scalability Runs Description

Run 1x1	Number processes: 1Number nodes: 1Number processes per node: 1Run Command: <executable> MPI Command: mpirun -np <number_processes> -ppn <number_processes_per_node>Dataset: Run Directory: /home/eoseret/tst_HACCmkI_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spreadOMP_NUM_THREADS: 1
Run 1x2	OMP_NUM_THREADS: 2I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x4	OMP_NUM_THREADS: 4I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x8	OMP_NUM_THREADS: 8I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x16	OMP_NUM_THREADS: 16I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x32	OMP_NUM_THREADS: 32I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x64	OMP_NUM_THREADS: 64I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x128	OMP_NUM_THREADS: 128I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread
Run 1x192	OMP_NUM_THREADS: 192I_MPI_PIN_DOMAIN: auto:scatterOMP_PLACES: threadsOMP_PROC_BIND: spread

(1x1) Efficiency	(1x1) Potential Speed-Up (%)	(1x2) Efficiency	(1x2) Potential Speed-Up (%)	(1x4) Efficiency	(1x4) Potential Speed-Up (%)	(1x8) Efficiency	(1x8) Potential Speed-Up (%)	(1x16) Efficiency	(1x16) Potential Speed-Up (%)	(1x32) Efficiency	(1x32) Potential Speed-Up (%)	(1x64) Efficiency	(1x64) Potential Speed-Up (%)	(1x128) Efficiency	(1x128) Potential Speed-Up (%)	(1x192) Efficiency	(1x192) Potential Speed-Up (%)
1	0	1.38	-0	1.38	-0	1.38	-0	0.69	0	1	0	1	0	1	0	1	0

Speed-Up and Efficiency

▶Detailed Speed-Up and Efficiency

Run	Number of threads	Efficiency (ideal is 1)	Speedup	Ideal Speedup	Time (s)	Coverage (%)
1x1	1	1	1	1	0.11	0.02
1x2	2	1.38	2.75	2	0.05	0.01
1x4	4	1.38	5.5	4	0.04	0.01
1x8	8	1.38	11	8	0.01	0.01
1x16	16	0.69	11	16	0.02	0.01
1x32	32	1	1	32	0.01	0.02
1x64	64	1	1	64	0.01	0.01
1x128	124	1	1	128	0	0.01
1x192	176	1	1	192	0.01	0.01

Report Configuration

Vectorization

Details

Workaround

Execution units bottlenecks

Workaround

Expensive FP math instructions/calls

FMA

Workaround

Complex instructions

Details

Conversion instructions

Details

Workaround

Type of elements and instruction set

Matching between your loop (in the source code) and the binary loop

Arithmetic intensity

General properties

Front-end

Back-end

Cycles summary

Vectorization ratios

Vector efficiency ratios

Cycles and memory resources usage

Front-end bottlenecks

ASM code

Vectorization

Details

Workaround

Execution units bottlenecks

Workaround

Expensive FP math instructions/calls

FMA

Workaround

Complex instructions

Details

Conversion instructions

Details

Workaround

Type of elements and instruction set

Matching between your loop (in the source code) and the binary loop

Arithmetic intensity

General properties

Front-end

Back-end

Cycles summary

Vectorization ratios

Vector efficiency ratios

Cycles and memory resources usage

Front-end bottlenecks

ASM code

▶Scalability Runs Description

▼Colums Filter

Speed-Up and Efficiency

▶Detailed Speed-Up and Efficiency