* [MAQAO] Info: Detected 1 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 1 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.1367
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7317
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2097809265
-- (4) Integer space for factors (estimated) = 27856460
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 0
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.335D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Total space in MBytes, IC factorization (INFOG(17)): 24024
Total space in MBytes, OOC factorization (INFOG(27)): 4861
Elapsed time in analysis driver= 27.6506
Analysis time by clock_gettime(): 27.650 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 1 and #OMP = 1
Elapsed time in save structure driver= 0.0003
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 1 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 1
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2097809265
INFOG(4) Integer space for factors (estim.)= 27856460
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m
Statistics on the scaling phase
Elapsed time for scaling = 0.3302
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Effective size of S (based on INFO(39))= 2948442560
Redistrib: total data local/sent = 0 0
Elapsed time to reformat/distribute matrix = 0.3331
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 230000
Size of async. emission buffer (bytes).. = 697700
Small emission buffer (bytes) .......... = 20
** Memory allocated, total in Mbytes (INFOG(19)): 24024
** Memory effectively used, total in Mbytes (INFOG(22)): 18582
Elapsed time for factorization = 293.2915
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.292D+09
------ (3) Operations in node elimination = 1.335D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2097900097
INFOG (10) Integer space for factors = 27856972
INFOG (11) Maximum front size = 18904
INFOG (29) Number of entries in factors = 2046167272
INFOG (12) Number of negative pivots = 171091
INFOG (13) Number of delayed pivots = 256
Number of 2x2 pivots in type 1 nodes = 582
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 0
Elapsed time in factorization driver = 293.9794
Factorization time by clock_gettime(): 293.9693 s
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 1 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 2 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 2 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.1574
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7356
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2138678670
-- (4) Integer space for factors (estimated) = 27888883
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 2
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 15380
Total space in MBytes, IC factorization (INFOG(17)): 27938
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 5237
Total space in MBytes, OOC factorization (INFOG(27)): 10104
Elapsed time in analysis driver= 28.0001
Analysis time by clock_gettime(): 28.000 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 2 and #OMP = 1
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Elapsed time in save structure driver= 0.0004
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 2 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
****** FACTORIZATION STEP ********
[0m GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 2
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2138678670
INFOG(4) Integer space for factors (estim.)= 27888883
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3313
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 1697784856
Elapsed time to reformat/distribute matrix = 0.5472
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 27070540
Size of async. emission buffer (bytes).. = 108552865
Small emission buffer (bytes) .......... = 248
** Memory allocated, max in Mbytes (INFOG(18)): 15390
** Memory allocated, total in Mbytes (INFOG(19)): 27948
** Memory effectively used, max in Mbytes (INFOG(21)): 11983
** Memory effectively used, total in Mbytes (INFOG(22)): 21643
Elapsed time to process root node = 14.8274
Elapsed time for factorization = 203.4249
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.342D+09
------ (3) Operations in node elimination = 1.453D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2198020790
INFOG (10) Integer space for factors = 27897016
INFOG (11) Maximum front size = 18904
INFOG (29) Number of entries in factors = 2055220797
INFOG (12) Number of negative pivots = 166817
INFOG (13) Number of delayed pivots = 6283
Number of 2x2 pivots in type 1 nodes = 576
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 0
Elapsed time in factorization driver = 204.3353
Factorization time by clock_gettime(): 204.3299 s
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 2 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 4 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 4 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.2129
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7347
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2138639144
-- (4) Integer space for factors (estimated) = 28013792
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 5
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 8820
Total space in MBytes, IC factorization (INFOG(17)): 32656
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 4818
Total space in MBytes, OOC factorization (INFOG(27)): 15660
Elapsed time in analysis driver= 28.0821
Analysis time by clock_gettime(): 28.082 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 4 and #OMP = 1
Elapsed time in save structure driver= 0.0004
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 4 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 4
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2138639144
INFOG(4) Integer space for factors (estim.)= 28013792
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m
Statistics on the scaling phase
Elapsed time for scaling = 0.3319
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 989674847
Elapsed time to reformat/distribute matrix = 0.4283
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 18384776
Size of async. emission buffer (bytes).. = 73722943
Small emission buffer (bytes) .......... = 644
** Memory allocated, max in Mbytes (INFOG(18)): 8886
** Memory allocated, total in Mbytes (INFOG(19)): 32846
** Memory effectively used, max in Mbytes (INFOG(21)): 7421
** Memory effectively used, total in Mbytes (INFOG(22)): 24223
Elapsed time to process root node = 18.9217
Elapsed time for factorization = 220.0283
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.499D+09
------ (3) Operations in node elimination = 1.700D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2310163766
INFOG (10) Integer space for factors = 27990991
INFOG (11) Maximum front size = 22093
INFOG (29) Number of entries in factors = 2093677875
INFOG (12) Number of negative pivots = 164689
INFOG (13) Number of delayed pivots = 16221
Number of 2x2 pivots in type 1 nodes = 574
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 1
Elapsed time in factorization driver = 220.8194
Factorization time by clock_gettime(): 220.8242 s
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 4 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 8 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 8 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.2088
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7301
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2138675375
-- (4) Integer space for factors (estimated) = 28116289
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 12
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 5947
Total space in MBytes, IC factorization (INFOG(17)): 40220
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 3789
Total space in MBytes, OOC factorization (INFOG(27)): 23536
Elapsed time in analysis driver= 28.0764
Analysis time by clock_gettime(): 28.076 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 8 and #OMP = 1
Elapsed time in save structure driver= 0.0004
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 8 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
****** FACTORIZATION STEP ********
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 8
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2138675375
INFOG(4) Integer space for factors (estim.)= 28116289
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3320
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 598612486
Elapsed time to reformat/distribute matrix = 0.3589
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 26601996
Size of async. emission buffer (bytes).. = 106673991
Small emission buffer (bytes) .......... = 1936
** Memory allocated, max in Mbytes (INFOG(18)): 5956
** Memory allocated, total in Mbytes (INFOG(19)): 40380
** Memory effectively used, max in Mbytes (INFOG(21)): 4955
** Memory effectively used, total in Mbytes (INFOG(22)): 30104
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 218.9814
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.560D+09
------ (3) Operations in node elimination = 1.150D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2559308040
INFOG (10) Integer space for factors = 28078373
INFOG (11) Maximum front size = 26517
INFOG (29) Number of entries in factors = 1827394071
INFOG (12) Number of negative pivots = 160604
INFOG (13) Number of delayed pivots = 38265
Number of 2x2 pivots in type 1 nodes = 533
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 4
Elapsed time in factorization driver = 219.6771
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 3672023
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 3672023
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 8 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 16 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 16 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.2474
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7394
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2137248125
-- (4) Integer space for factors (estimated) = 28508430
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99692
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 25
Number of split nodes = 1
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 4663
Total space in MBytes, IC factorization (INFOG(17)): 44496
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 2718
Total space in MBytes, OOC factorization (INFOG(27)): 29849
Elapsed time in analysis driver= 28.1362
Analysis time by clock_gettime(): 28.136 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 16 and #OMP = 1
Elapsed time in save structure driver= 0.0005
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 16 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 16
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2137248125
INFOG(4) Integer space for factors (estim.)= 28508430
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99692
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3328
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 323261614
Elapsed time to reformat/distribute matrix = 0.3415
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 22673704
Size of async. emission buffer (bytes).. = 90921550
Small emission buffer (bytes) .......... = 6420
** Memory allocated, max in Mbytes (INFOG(18)): 4668
** Memory allocated, total in Mbytes (INFOG(19)): 45949
** Memory effectively used, max in Mbytes (INFOG(21)): 3680
** Memory effectively used, total in Mbytes (INFOG(22)): 34919
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 61.3408
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.402D+09
------ (3) Operations in node elimination = 8.771D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1745965733
INFOG (10) Integer space for factors = 28169068
INFOG (11) Maximum front size = 32097
INFOG (29) Number of entries in factors = 1659231731
INFOG (12) Number of negative pivots = 156890
INFOG (13) Number of delayed pivots = 47942
Number of 2x2 pivots in type 1 nodes = 511
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 7
Elapsed time in factorization driver = 62.0214
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 175128212
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 175128212
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 16 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 32 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 32 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.9060
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8241
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2137031435
-- (4) Integer space for factors (estimated) = 29486850
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99692
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 52
Number of split nodes = 1
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 1728
Total space in MBytes, IC factorization (INFOG(17)): 42789
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 1105
Total space in MBytes, OOC factorization (INFOG(27)): 28197
Elapsed time in analysis driver= 29.0406
Analysis time by clock_gettime(): 29.040 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 32 and #OMP = 1
Elapsed time in save structure driver= 0.0011
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 32 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 32
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2137031435
INFOG(4) Integer space for factors (estim.)= 29486850
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99692
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3371
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 150239514
Elapsed time to reformat/distribute matrix = 0.3691
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 13108088
Size of async. emission buffer (bytes).. = 52563426
Small emission buffer (bytes) .......... = 23088
** Memory allocated, max in Mbytes (INFOG(18)): 1957
** Memory allocated, total in Mbytes (INFOG(19)): 43794
** Memory effectively used, max in Mbytes (INFOG(21)): 1499
** Memory effectively used, total in Mbytes (INFOG(22)): 31999
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 20.4516
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 3.949D+09
------ (3) Operations in node elimination = 5.160D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1613103981
INFOG (10) Integer space for factors = 28310536
INFOG (11) Maximum front size = 26882
INFOG (29) Number of entries in factors = 1346018556
INFOG (12) Number of negative pivots = 150609
INFOG (13) Number of delayed pivots = 44363
Number of 2x2 pivots in type 1 nodes = 487
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 22
Elapsed time in factorization driver = 21.1683
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 54888469
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 22
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 54888469
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 32 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 64 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 64 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.8756
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8059
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2136397189
-- (4) Integer space for factors (estimated) = 30689915
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99693
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 105
Number of split nodes = 2
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 1037
Total space in MBytes, IC factorization (INFOG(17)): 50115
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 752
Total space in MBytes, OOC factorization (INFOG(27)): 36815
Elapsed time in analysis driver= 28.9967
Analysis time by clock_gettime(): 28.997 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 64 and #OMP = 1
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Elapsed time in save structure driver= 0.0016
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 64 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 64
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2136397189
INFOG(4) Integer space for factors (estim.)= 30689915
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99693
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3389
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 85022769
Elapsed time to reformat/distribute matrix = 0.3949
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 7765960
Size of async. emission buffer (bytes).. = 31141496
Small emission buffer (bytes) .......... = 87124
** Memory allocated, max in Mbytes (INFOG(18)): 1042
** Memory allocated, total in Mbytes (INFOG(19)): 50698
** Memory effectively used, max in Mbytes (INFOG(21)): 916
** Memory effectively used, total in Mbytes (INFOG(22)): 32105
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 5.9698
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 3.336D+09
------ (3) Operations in node elimination = 2.941D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1286833520
INFOG (10) Integer space for factors = 28143870
INFOG (11) Maximum front size = 23281
INFOG (29) Number of entries in factors = 1081197786
INFOG (12) Number of negative pivots = 143150
INFOG (13) Number of delayed pivots = 46677
Number of 2x2 pivots in type 1 nodes = 466
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 15
Elapsed time in factorization driver = 6.7187
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 2452664
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 2452664
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 64 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 128 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 128 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 27.1127
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8232
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2133316699
-- (4) Integer space for factors (estimated) = 33053879
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99698
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 198
Number of split nodes = 7
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 796
Total space in MBytes, IC factorization (INFOG(17)): 56094
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 622
Total space in MBytes, OOC factorization (INFOG(27)): 45536
Elapsed time in analysis driver= 29.3557
Analysis time by clock_gettime(): 29.357 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 128 and #OMP = 1
Elapsed time in save structure driver= 0.0027
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 128 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
[0m
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 128
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2133316699
INFOG(4) Integer space for factors (estim.)= 33053879
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99698
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3378
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 43158526
Elapsed time to reformat/distribute matrix = 0.5090
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 6406008
Size of async. emission buffer (bytes).. = 25688083
Small emission buffer (bytes) .......... = 337816
** Memory allocated, max in Mbytes (INFOG(18)): 798
** Memory allocated, total in Mbytes (INFOG(19)): 56328
** Memory effectively used, max in Mbytes (INFOG(21)): 464
** Memory effectively used, total in Mbytes (INFOG(22)): 34250
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 1.8372
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 2.626D+09
------ (3) Operations in node elimination = 1.560D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1013353222
INFOG (10) Integer space for factors = 27775174
INFOG (11) Maximum front size = 15385
INFOG (29) Number of entries in factors = 851224177
INFOG (12) Number of negative pivots = 135738
INFOG (13) Number of delayed pivots = 39224
Number of 2x2 pivots in type 1 nodes = 426
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 10
Elapsed time in factorization driver = 2.7049
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 104781
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 543448
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 88
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 104781
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 128 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
#################################################################################################################################################################
* [MAQAO] Info: Detected 172 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 172 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 27.7392
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8408
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2132565373
-- (4) Integer space for factors (estimated) = 34509561
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99699
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 0
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 263
Number of split nodes = 8
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 543
Total space in MBytes, IC factorization (INFOG(17)): 58606
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 475
Total space in MBytes, OOC factorization (INFOG(27)): 49726
Elapsed time in analysis driver= 30.0072
Analysis time by clock_gettime(): 30.009 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 172 and #OMP = 1
Elapsed time in save structure driver= 0.0024
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 172 and #OMP = 1
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 172
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2132565373
INFOG(4) Integer space for factors (estim.)= 34509561
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99699
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.3511
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 31698334
Elapsed time to reformat/distribute matrix = 0.5209
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 5375516
Size of async. emission buffer (bytes).. = 21555810
Small emission buffer (bytes) .......... = 605228
** Memory allocated, max in Mbytes (INFOG(18)): 544
** Memory allocated, total in Mbytes (INFOG(19)): 58891
** Memory effectively used, max in Mbytes (INFOG(21)): 420
** Memory effectively used, total in Mbytes (INFOG(22)): 38559
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 2.7806
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 2.801D+09
------ (3) Operations in node elimination = 1.738D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1015939679
INFOG (10) Integer space for factors = 28126721
INFOG (11) Maximum front size = 15690
INFOG (29) Number of entries in factors = 861998779
INFOG (12) Number of negative pivots = 131676
INFOG (13) Number of delayed pivots = 59870
Number of 2x2 pivots in type 1 nodes = 407
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 5.004D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 5.004D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 29
Elapsed time in factorization driver = 3.6901
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 8541847
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 126
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 8541847
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 172 and #OMP = 1
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8
To display your profiling results:
#################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
#################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-172_o1_CurlCurl_allowextra_scala_kptr_probe/tools/lprof_run_8 #
#################################################################################################################################################################
