OV - llama-cli - Loops

MAQAO

options

Loops Index

▶Filters

Loop id	Source Location	Source Function	Level	Max Thread Time / Walltime run_0 (%)	Exclusive Coverage run_0 (%)	Inclusive Coverage run_0 (%)	Max Exclusive Time Over Threads run_0 (s)	Max Inclusive Time Over Threads run_0 (s)	Exclusive Time w.r.t. Wall Time run_0 (s)	Inclusive Time w.r.t. Wall Time run_0 (s)	Nb Threads run_0	Vectorization Ratio (%)	Vector Length Use (%)	Speedup If No Scalar Integer	Speedup If FP Vectorized	Speedup If Fully Vectorized	Speedup If Perfect Load Balancing run_0	Stride 0	Stride 1	Stride n	Stride Unknown	Stride Indirect	Array Access Efficiency	Speedup If Data in L1 run_0
2499	libggml-cpu.so - quants.c:101-1042 [...]	ggml_vec_dot_q8_0_q8_0	Single	87.17	84.78	84.78	94.78	94.78	90.88	90.88	52	72.73	38.07	1	1.41	2.61	1.05	0	2	0	0	1	66.67	2.8
1768	libggml-cpu.so - vec.h:491-497	ggml_compute_forward_flash_attn_ext	Innermost	0.85	0.41	0.41	0.92	0.92	0.44	0.44	37	100	75	1	1	1.33	1.5	0	2	0	0	0	100.00	NA
767	libggml-cpu.so - vec.cpp:311-316	ggml_vec_dot_f16	Single	0.73	0.31	0.31	0.79	0.79	0.33	0.33	34	100	66.67	1	1	1.5	1.55	0	2	0	0	0	100.00	NA
2294	libggml-cpu.so - sgemm.cpp:138-1044 [...]	void (anonymous namespace)::tinyBLAS_Q0_AVX<block_q8_0, block_q8_0, float>::gemm4xN<4>(long, long, long, long)	Innermost	0.16	0.10	0.10	0.17	0.17	0.11	0.11	52	93.43	46.05	1.06	1.19	2.17	1.62	2	4	0	1	1	81.25	NA
61	libggml-cpu.so - ggml-cpu.c:1125-1395 [...]	ggml_compute_forward_mul_mat	Outermost	0.16	0.08	0.19	0.17	0.36	0.09	0.21	52	0	11.49	1	1	2.06	1.96	NA	NA	NA	NA	NA	0.00	NA
2489	libggml-cpu.so - quants.c:298-355 [...]	quantize_row_q8_0	Single	0.15	0.08	0.08	0.16	0.16	0.08	0.08	52	59.83	29.33	1.23	1.67	3.44	1.97	0	2	0	0	0	100.00	NA
1761	libggml-cpu.so - vec.h:375-751 [...]	ggml_compute_forward_flash_attn_ext	InBetween	0.18	0.08	0.49	0.19	1.08	0.08	0.52	33	49.64	45.96	1.59	1.02	1.46	1.48	NA	NA	NA	NA	NA	0.00	NA
2306	libggml-cpu.so - sgemm.cpp:138-1044 [...]	void (anonymous namespace)::tinyBLAS_Q0_AVX<block_q8_0, block_q8_0, float>::gemm4xN<2>(long, long, long, long)	Innermost	0.11	0.07	0.07	0.12	0.12	0.07	0.07	52	90.77	44.47	1.04	1.17	2.15	1.64	0	4	0	1	1	75.00	NA
67	libggml-cpu.so - ggml-cpu.c:1193-1194	ggml_compute_forward_mul_mat	Innermost	0.09	0.05	0.05	0.10	0.10	0.06	0.06	52	0	12.5	1	1	8	1.82	1	0	0	0	0	100.00	NA
64	libggml-cpu.so - ggml-cpu.c:1164-1198 [...]	ggml_compute_forward_mul_mat	InBetween	0.09	0.05	0.11	0.10	0.21	0.05	0.11	52	0	11.37	1	1	2.06	1.94	NA	NA	NA	NA	NA	0.00	NA
82	libggml-cpu.so - ggml-cpu.c:533-2891 [...]	ggml_graph_compute_thread	Single	0.06	0.03	0.03	0.06	0.06	0.03	0.03	52	0	9.82	1	1	14.64	2.27	NA	NA	NA	NA	NA	0.00	NA
2297	libggml-cpu.so - sgemm.cpp:138-1044 [...]	void (anonymous namespace)::tinyBLAS_Q0_AVX<block_q8_0, block_q8_0, float>::gemm4xN<3>(long, long, long, long)	Innermost	0.08	0.02	0.02	0.09	0.09	0.02	0.02	52	92.31	45.3	1.04	1.17	2.15	3.7	0	5	0	1	1	78.57	NA
1437	libggml-cpu.so - ops.cpp:6220-6245 [...]	ggml_compute_forward_rope_f32(ggml_compute_params const, ggml_tensor, bool)	Innermost	0.04	0.01	0.01	0.04	0.04	0.02	0.02	48	15.56	9.17	1.09	1.09	5.83	2.4	1	2	0	0	0	100.00	NA
1760	libggml-cpu.so - ops.cpp:8885-8886 [...]	ggml_compute_forward_flash_attn_ext	Innermost	0.05	0.01	0.01	0.05	0.05	0.01	0.01	31	0	6.25	1	1	16	2.54	0	2	0	0	1	66.67	NA

×