* [MAQAO] Info: Detected 1 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 1 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.1422
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7351
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2097809265
-- (4) Integer space for factors (estimated) = 27856460
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 0
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.335D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Total space in MBytes, IC factorization (INFOG(17)): 25480
Total space in MBytes, OOC factorization (INFOG(27)): 8978
Elapsed time in analysis driver= 27.6810
Analysis time by clock_gettime(): 27.680 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 1 and #OMP = 2
Elapsed time in save structure driver= 0.0003
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 1 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 1
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2097809265
INFOG(4) Integer space for factors (estim.)= 27856460
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m
Statistics on the scaling phase
Elapsed time for scaling = 0.1785
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Effective size of S (based on INFO(39))= 1517701215
Redistrib: total data local/sent = 0 0
Elapsed time to reformat/distribute matrix = 0.2738
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 230000
Size of async. emission buffer (bytes).. = 697700
Small emission buffer (bytes) .......... = 20
** Memory allocated, total in Mbytes (INFOG(19)): 25480
** Memory effectively used, total in Mbytes (INFOG(22)): 22679
Flops under L0 layer = 2.499D+12
Elapsed time under L0 = 38.3553
Elapsed time for factorization = 215.2940
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.292D+09
------ (3) Operations in node elimination = 1.335D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2097857484
INFOG (10) Integer space for factors = 27856892
INFOG (11) Maximum front size = 18904
INFOG (29) Number of entries in factors = 2046122614
INFOG (12) Number of negative pivots = 172174
INFOG (13) Number of delayed pivots = 216
Number of 2x2 pivots in type 1 nodes = 577
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 0
Elapsed time in factorization driver = 215.7732
Factorization time by clock_gettime(): 215.7659 s
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 1 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_0 #
################################################################################################################################################################
* [MAQAO] Info: Detected 2 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 2 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.2055
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7360
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2138678670
-- (4) Integer space for factors (estimated) = 27888883
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 2
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 16191
Total space in MBytes, IC factorization (INFOG(17)): 29512
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 7308
Total space in MBytes, OOC factorization (INFOG(27)): 14316
Elapsed time in analysis driver= 28.0338
Analysis time by clock_gettime(): 28.033 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 2 and #OMP = 2
Elapsed time in save structure driver= 0.0004
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 2 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
Number of working processes = 2
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
[0m ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2138678670
INFOG(4) Integer space for factors (estim.)= 27888883
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.1784
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 988460161
Elapsed time to reformat/distribute matrix = 0.3598
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 27070540
Size of async. emission buffer (bytes).. = 108552865
Small emission buffer (bytes) .......... = 248
** Memory allocated, max in Mbytes (INFOG(18)): 16191
** Memory allocated, total in Mbytes (INFOG(19)): 29512
** Memory effectively used, max in Mbytes (INFOG(21)): 14103
** Memory effectively used, total in Mbytes (INFOG(22)): 25826
Flops under L0 layer (avg/max across MPI) = 1.307D+12 1.696D+12
Elapsed time under L0 (avg/max across MPI) = 20.8359 25.6995
Elapsed time to process root node = 14.8208
Elapsed time for factorization = 161.9914
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.342D+09
------ (3) Operations in node elimination = 1.453D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2197958850
INFOG (10) Integer space for factors = 27896932
INFOG (11) Maximum front size = 18904
INFOG (29) Number of entries in factors = 2055170354
INFOG (12) Number of negative pivots = 167853
INFOG (13) Number of delayed pivots = 6241
Number of 2x2 pivots in type 1 nodes = 565
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 0
Elapsed time in factorization driver = 162.5583
Factorization time by clock_gettime(): 162.5529 s
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 2 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_1 #
################################################################################################################################################################
* [MAQAO] Info: Detected 4 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 4 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.0032
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7223
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2138639144
-- (4) Integer space for factors (estimated) = 28013792
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 5
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 9378
Total space in MBytes, IC factorization (INFOG(17)): 34386
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 5369
Total space in MBytes, OOC factorization (INFOG(27)): 19554
Elapsed time in analysis driver= 27.8011
Analysis time by clock_gettime(): 27.800 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 4 and #OMP = 2
Elapsed time in save structure driver= 0.0004
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 4 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 4
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2138639144
INFOG(4) Integer space for factors (estim.)= 28013792
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Statistics on the scaling phase
Elapsed time for scaling = 0.1787
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 641174522
Elapsed time to reformat/distribute matrix = 0.3285
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 18384776
Size of async. emission buffer (bytes).. = 73722943
Small emission buffer (bytes) .......... = 644
** Memory allocated, max in Mbytes (INFOG(18)): 9378
** Memory allocated, total in Mbytes (INFOG(19)): 34065
** Memory effectively used, max in Mbytes (INFOG(21)): 8574
** Memory effectively used, total in Mbytes (INFOG(22)): 28422
Flops under L0 layer (avg/max across MPI) = 6.384D+11 1.049D+12
Elapsed time under L0 (avg/max across MPI) = 9.6806 13.2344
Elapsed time to process root node = 18.7964
Elapsed time for factorization = 164.8893
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.499D+09
------ (3) Operations in node elimination = 1.700D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2310099129
INFOG (10) Integer space for factors = 27990911
INFOG (11) Maximum front size = 22093
INFOG (29) Number of entries in factors = 2093629542
INFOG (12) Number of negative pivots = 165685
INFOG (13) Number of delayed pivots = 16181
Number of 2x2 pivots in type 1 nodes = 564
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 3
Elapsed time in factorization driver = 165.4190
Factorization time by clock_gettime(): 165.4271 s
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 4 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_2 #
################################################################################################################################################################
* [MAQAO] Info: Detected 8 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 8 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.1633
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7272
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2138675375
-- (4) Integer space for factors (estimated) = 28116289
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99691
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 12
Number of split nodes = 0
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 6319
Total space in MBytes, IC factorization (INFOG(17)): 42066
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 4168
Total space in MBytes, OOC factorization (INFOG(27)): 27500
Elapsed time in analysis driver= 28.0137
Analysis time by clock_gettime(): 28.013 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 8 and #OMP = 2
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Elapsed time in save structure driver= 0.0005
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 8 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 8
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2138675375
INFOG(4) Integer space for factors (estim.)= 28116289
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99691
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Statistics on the scaling phase
Elapsed time for scaling = 0.1816
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 447062291
Elapsed time to reformat/distribute matrix = 0.3126
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 26601996
Size of async. emission buffer (bytes).. = 106673991
Small emission buffer (bytes) .......... = 1936
** Memory allocated, max in Mbytes (INFOG(18)): 6319
** Memory allocated, total in Mbytes (INFOG(19)): 42065
** Memory effectively used, max in Mbytes (INFOG(21)): 5653
** Memory effectively used, total in Mbytes (INFOG(22)): 33914
Flops under L0 layer (avg/max across MPI) = 2.357D+11 3.329D+11
Elapsed time under L0 (avg/max across MPI) = 3.8917 4.8795
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 167.2655
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.561D+09
------ (3) Operations in node elimination = 1.150D+13
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 2559238294
INFOG (10) Integer space for factors = 1696311
INFOG (11) Maximum front size = 26517
INFOG (29) Number of entries in factors = 1827365761
INFOG (12) Number of negative pivots = 161585
INFOG (13) Number of delayed pivots = 38225
Number of 2x2 pivots in type 1 nodes = 530
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 5
Elapsed time in factorization driver = 167.7647
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 20659496
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 6
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 20659496
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 8 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_3 #
################################################################################################################################################################
* [MAQAO] Info: Detected 16 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 16 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.0736
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.7254
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2137248125
-- (4) Integer space for factors (estimated) = 28508430
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99692
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 25
Number of split nodes = 1
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 4897
Total space in MBytes, IC factorization (INFOG(17)): 46520
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 3006
Total space in MBytes, OOC factorization (INFOG(27)): 33613
Elapsed time in analysis driver= 27.8992
Analysis time by clock_gettime(): 27.899 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 16 and #OMP = 2
Elapsed time in save structure driver= 0.0005
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 16 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 16
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2137248125
INFOG(4) Integer space for factors (estim.)= 28508430
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99692
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.1837
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 264155796
Elapsed time to reformat/distribute matrix = 0.3209
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 22673704
Size of async. emission buffer (bytes).. = 90921550
Small emission buffer (bytes) .......... = 6420
** Memory allocated, max in Mbytes (INFOG(18)): 4897
** Memory allocated, total in Mbytes (INFOG(19)): 47703
** Memory effectively used, max in Mbytes (INFOG(21)): 4078
** Memory effectively used, total in Mbytes (INFOG(22)): 38519
Flops under L0 layer (avg/max across MPI) = 6.063D+10 1.216D+11
Elapsed time under L0 (avg/max across MPI) = 1.2940 2.0809
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 49.7607
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 4.402D+09
------ (3) Operations in node elimination = 8.771D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1745930976
INFOG (10) Integer space for factors = 2671592
INFOG (11) Maximum front size = 32096
INFOG (29) Number of entries in factors = 1659208200
INFOG (12) Number of negative pivots = 157814
INFOG (13) Number of delayed pivots = 47899
Number of 2x2 pivots in type 1 nodes = 508
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 8
Elapsed time in factorization driver = 50.2714
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 182449234
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 182449234
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 16 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_4 #
################################################################################################################################################################
* [MAQAO] Info: Detected 32 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 32 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.9119
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8348
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2137031435
-- (4) Integer space for factors (estimated) = 29486850
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99692
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 52
Number of split nodes = 1
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 1826
Total space in MBytes, IC factorization (INFOG(17)): 44914
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 1233
Total space in MBytes, OOC factorization (INFOG(27)): 31692
Elapsed time in analysis driver= 29.0442
Analysis time by clock_gettime(): 29.044 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 32 and #OMP = 2
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
Elapsed time in save structure driver= 0.0009
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 32 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 32
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2137031435
INFOG(4) Integer space for factors (estim.)= 29486850
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99692
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.1869
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 123052557
Elapsed time to reformat/distribute matrix = 0.3672
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 13108088
Size of async. emission buffer (bytes).. = 52563426
Small emission buffer (bytes) .......... = 23088
** Memory allocated, max in Mbytes (INFOG(18)): 1825
** Memory allocated, total in Mbytes (INFOG(19)): 44908
** Memory effectively used, max in Mbytes (INFOG(21)): 1502
** Memory effectively used, total in Mbytes (INFOG(22)): 32712
Flops under L0 layer (avg/max across MPI) = 2.297D+10 4.610D+10
Elapsed time under L0 (avg/max across MPI) = 0.6156 1.1700
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 10.8227
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 3.611D+09
------ (3) Operations in node elimination = 4.013D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1445641663
INFOG (10) Integer space for factors = 2559956
INFOG (11) Maximum front size = 21358
INFOG (29) Number of entries in factors = 1236609118
INFOG (12) Number of negative pivots = 151492
INFOG (13) Number of delayed pivots = 27400
Number of 2x2 pivots in type 1 nodes = 479
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 13
Elapsed time in factorization driver = 11.3901
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 642225
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 11
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 642225
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 32 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_5 #
################################################################################################################################################################
* [MAQAO] Info: Detected 64 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 64 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.9027
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8185
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2136397189
-- (4) Integer space for factors (estimated) = 30689915
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99693
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 105
Number of split nodes = 2
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 1095
Total space in MBytes, IC factorization (INFOG(17)): 52002
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 776
Total space in MBytes, OOC factorization (INFOG(27)): 40059
Elapsed time in analysis driver= 29.0372
Analysis time by clock_gettime(): 29.037 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 64 and #OMP = 2
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
PRE FACTO START LPROF----------------------
Elapsed time in save structure driver= 0.0013
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 64 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m ** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 64
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2136397189
INFOG(4) Integer space for factors (estim.)= 30689915
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99693
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.1859
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 73698574
Elapsed time to reformat/distribute matrix = 0.4315
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 7765960
Size of async. emission buffer (bytes).. = 31141496
Small emission buffer (bytes) .......... = 87124
** Memory allocated, max in Mbytes (INFOG(18)): 1095
** Memory allocated, total in Mbytes (INFOG(19)): 52386
** Memory effectively used, max in Mbytes (INFOG(21)): 941
** Memory effectively used, total in Mbytes (INFOG(22)): 35806
Flops under L0 layer (avg/max across MPI) = 6.821D+09 1.694D+10
Elapsed time under L0 (avg/max across MPI) = 0.3506 0.6998
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 5.3741
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 3.364D+09
------ (3) Operations in node elimination = 2.971D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1305201510
INFOG (10) Integer space for factors = 3519325
INFOG (11) Maximum front size = 23281
INFOG (29) Number of entries in factors = 1085072892
INFOG (12) Number of negative pivots = 143987
INFOG (13) Number of delayed pivots = 46043
Number of 2x2 pivots in type 1 nodes = 453
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 21
Elapsed time in factorization driver = 6.0066
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 3571664
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 43
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 3571664
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 64 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_6 #
################################################################################################################################################################
* [MAQAO] Info: Detected 86 Lprof instances in igk-0805.
If this is incorrect, rerun with number-processes-per-node=X
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 1 2380515 14448191
executing #MPI = 86 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
=================================================
MUMPS compiled with option -Dmetis
MUMPS compiled with option -Dpord
MUMPS compiled with option -Dptscotch
MUMPS compiled with option -Dscotch
=================================================
L D L^T Solver for general symmetric matrices
Type of parallelism: Working host
****** ANALYSIS STEP ********
Processing a graph of size: 2380515
Average density of rows/columns = 10
Ordering based on METIS
ELAPSED TIME SPENT IN METIS reordering = 26.8487
SYMBOLIC based on column counts
ELAPSED TIME IN symbolic factorization = 0.8144
A root of estimated size 9530 has been selected for Scalapack.
Leaving analysis phase with ...
INFOG(1) = 0
INFOG(2) = 0
-- (20) Number of entries in factors (estim.) = 2046072726
-- (3) Real space for factors (estimated) = 2134713545
-- (4) Integer space for factors (estimated) = 31684934
-- (5) Maximum frontal size (estimated) = 18904
-- (6) Number of nodes in the tree = 99695
-- (32) Type of analysis effectively used = 1
-- (7) Ordering option effectively used = 5
ICNTL (6) Maximum transversal option = 0
ICNTL (7) Pivot order option = 7
ICNTL(12) Ordering symmetric indef. matrices = 1
ICNTL(13) Parallelism/splitting of root node = 0
ICNTL(14) Percentage of memory relaxation = 30
ICNTL(15) Analysis by block effectively used = 0
ICNTL(18) Distributed input matrix (on if >0) = 0
ICNTL(32) Forward elimination during facto. = 0
ICNTL(35) BLR activation = 0
ICNTL(48) Tree based multithreading (effective)= 1
ICNTL(58) Symbolic factorization option = 2
Number of level 2 nodes = 137
Number of split nodes = 4
RINFOG(1) Operations during elimination (estim)= 1.364D+13
MEMORY ESTIMATIONS ...
Estimations with standard Full-Rank (FR) factorization:
Maximum estim. space in Mbytes, IC facto. (INFOG(16)): 1557
Total space in MBytes, IC factorization (INFOG(17)): 55707
Maximum estim. space in Mbytes, OOC facto. (INFOG(26)): 1282
Total space in MBytes, OOC factorization (INFOG(27)): 44630
Elapsed time in analysis driver= 28.9966
Analysis time by clock_gettime(): 28.997 s
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 7 2380515 14448191
executing #MPI = 86 and #OMP = 2
Elapsed time in save structure driver= 0.0018
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
On return from DMUMPS, INFOG(1)= -71
On return from DMUMPS, INFOG(2)= 0
PRE FACTO START LPROF----------------------
Entering DMUMPS 5.8.2 from C interface with JOB, N, NNZ = 2 2380515 14448191
executing #MPI = 86 and #OMP = 2
Advanced settings:
KEEP(370) Static mapping = 1
KEEP(371) Advanced optimizations = 0
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
** ERROR RETURN ** FROM DMUMPS INFO(1)= -71
** INFO(2)= 0
PRE FACTO START LPROF----------------------
* [MAQAO] Info: STARTING COUNTERS (igk-0805)
[0m
****** FACTORIZATION STEP ********
GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
Number of working processes = 86
ICNTL(22) Out-of-core option = 0
ICNTL(35) BLR activation (eff. choice) = 0
ICNTL(37) BLR CB compression (eff. choice) = 0
ICNTL(49) Compact workarray S (end facto.) = 0
ICNTL(56) Effective value during facto. = 0
ICNTL(14) Memory relaxation = 30
INFOG(3) Real space for factors (estimated)= 2134713545
INFOG(4) Integer space for factors (estim.)= 31684934
Maximum frontal size (estimated) = 18904
Number of nodes in the tree = 99695
ICNTL(23) Memory allowed (value on host) = 0
Sum over all procs = 0
Memory provided by user, sum of LWK_USER = 0
Effective threshold for pivoting, CNTL(1) = 0.1000D-01
Statistics on the scaling phase
Elapsed time for scaling = 0.1862
Max difference from 1 after scaling the entries for ONE-NORM (option 7/8) = 0.32D-01
Average Effective size of S (based on INFO(39))= 56351569
Elapsed time to reformat/distribute matrix = 0.4705
Allocated buffers
------------------
Size of reception buffer in bytes ...... = 10406616
Size of async. emission buffer (bytes).. = 41730533
Small emission buffer (bytes) .......... = 154820
** Memory allocated, max in Mbytes (INFOG(18)): 1557
** Memory allocated, total in Mbytes (INFOG(19)): 55856
** Memory effectively used, max in Mbytes (INFOG(21)): 750
** Memory effectively used, total in Mbytes (INFOG(22)): 35912
Flops under L0 layer (avg/max across MPI) = 3.982D+09 1.049D+10
Elapsed time under L0 (avg/max across MPI) = 0.3089 0.6062
Elapsed time to process root node = 0.0000
Elapsed time for (failed) factorization = 3.3802
Leaving factorization with ...
RINFOG (2) Operations in node assembly = 2.968D+09
------ (3) Operations in node elimination = 2.171D+12
ICNTL (8) Scaling effectively used = 7
INFOG (9) Real space for factors = 1139835736
INFOG (10) Integer space for factors = 3580913
INFOG (11) Maximum front size = 18619
INFOG (29) Number of entries in factors = 961946201
INFOG (12) Number of negative pivots = 141104
INFOG (13) Number of delayed pivots = 34502
Number of 2x2 pivots in type 1 nodes = 427
Number of 2x2 pivots in type 2 nodes = 0
RINFOG(19) Smallest pivot WITH perturbed pivots = 6.209D-19
RINFOG(20) Smallest pivot WITHOUT perturbed pivots = 6.209D-19
RINFOG(21) Largest pivot in absolute value = 5.326D-01
INFOG (24) Effective value of ICNTL(12) = 1
INFOG (14) Number of memory compress = 13
Elapsed time in factorization driver = 4.0689
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -9
** INFO(2)= 632319
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
** ERROR RETURN ** FROM DMUMPS INFO(1)= -1
** INFO(2)= 46
On return from DMUMPS, INFOG(1)= -9
On return from DMUMPS, INFOG(2)= 632319
Entering DMUMPS 5.8.2 from C interface with JOB = -2
executing #MPI = 86 and #OMP = 2
Your experiment path is /home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7
To display your profiling results:
################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Functions | Per-node | maqao lprof -df -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Functions | Per-process | maqao lprof -df -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Functions | Per-thread | maqao lprof -df -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Cluster-wide | maqao lprof -dl xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Per-node | maqao lprof -dl -dn xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Per-process | maqao lprof -dl -dp xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/home/mlkaps_org/kevin/matrices/test_m1-86_o2_curlcurl_allowextra_scala_kptr_probe/tools/lprof_run_7 #
################################################################################################################################################################
