* Info: Selecting the 'perf-high-ppn' engine for node inti6230
* Info: Selecting the 'perf-high-ppn' engine for node inti6231
* Info: Selecting the 'perf-high-ppn' engine for node inti6232
* Info: Selecting the 'perf-high-ppn' engine for node inti6229
* Info: "ref-cycles" not supported on inti6230: fallback to "cpu-clock"
* Info: "ref-cycles" not supported on inti6231: fallback to "cpu-clock"
* Info: "ref-cycles" not supported on inti6232: fallback to "cpu-clock"
* Info: "ref-cycles" not supported on inti6229: fallback to "cpu-clock"
* Warning: Process/thread 319067 (host inti6231) exited with 1
* Warning: Process/thread 319068 (host inti6231) exited with 1
* Warning: Process/thread 354427 (host inti6229) exited with 1
* Warning: Process/thread 318972 (host inti6232) exited with 1
* Warning: Process/thread 354421 (host inti6229) exited with 1
* Warning: Process/thread 354430 (host inti6229) exited with 1
* Warning: Process/thread 319070 (host inti6231) exited with 1
* Warning: Process/thread 319072 (host inti6231) exited with 1
* Warning: Process/thread 354425 (host inti6229) exited with 1
* Warning: Process/thread 354435 (host inti6229) exited with 1
* Warning: Process/thread 354429 (host inti6229) exited with 1
* Warning: Process/thread 354431 (host inti6229) exited with 1
* Warning: Process/thread 354428 (host inti6229) exited with 1
* Warning: Process/thread 318976 (host inti6232) exited with 1
* Warning: Process/thread 354432 (host inti6229) exited with 1
* Warning: Process/thread 354426 (host inti6229) exited with 1
* Warning: Process/thread 318969 (host inti6232) exited with 1
* Warning: Process/thread 923155 (host inti6230) exited with 1
* Warning: Process/thread 354422 (host inti6229) exited with 1
* Warning: Process/thread 354424 (host inti6229) exited with 1
* Warning: Process/thread 923145 (host inti6230) exited with 1
* Warning: Process/thread 319077 (host inti6231) exited with 1
* Warning: Process/thread 923148 (host inti6230) exited with 1
* Warning: Process/thread 319066 (host inti6231) exited with 1
* Warning: Process/thread 318970 (host inti6232) exited with 1
* Warning: Process/thread 923144 (host inti6230) exited with 1
* Warning: Process/thread 923146 (host inti6230) exited with 1
* Warning: Process/thread 319071 (host inti6231) exited with 1
* Warning: Process/thread 923149 (host inti6230) exited with 1
* Warning: Process/thread 319078 (host inti6231) exited with 1
* Warning: Process/thread 319085 (host inti6231) exited with 1
* Warning: Process/thread 923154 (host inti6230) exited with 1
* Warning: Process/thread 319076 (host inti6231) exited with 1
* Warning: Process/thread 319082 (host inti6231) exited with 1
* Warning: Process/thread 923152 (host inti6230) exited with 1
* Warning: Process/thread 319079 (host inti6231) exited with 1
* Warning: Process/thread 319083 (host inti6231) exited with 1
* Warning: Process/thread 319069 (host inti6231) exited with 1
* Warning: Process/thread 319084 (host inti6231) exited with 1
* Warning: Process/thread 319080 (host inti6231) exited with 1
* Warning: Process/thread 923153 (host inti6230) exited with 1
* Warning: Process/thread 318977 (host inti6232) exited with 1
* Warning: Process/thread 923150 (host inti6230) exited with 1
* Warning: Process/thread 923147 (host inti6230) exited with 1
* Warning: Process/thread 923158 (host inti6230) exited with 1
* Warning: Process/thread 923161 (host inti6230) exited with 1
* Warning: Process/thread 923160 (host inti6230) exited with 1
* Warning: Process/thread 318973 (host inti6232) exited with 1
* Warning: Process/thread 923159 (host inti6230) exited with 1
* Warning: Process/thread 923151 (host inti6230) exited with 1
* Warning: Process/thread 318980 (host inti6232) exited with 1
* Warning: Process/thread 318971 (host inti6232) exited with 1
* Warning: Process/thread 318979 (host inti6232) exited with 1
* Warning: Process/thread 318983 (host inti6232) exited with 1
* Warning: Process/thread 318975 (host inti6232) exited with 1
* Warning: Process/thread 318985 (host inti6232) exited with 1
* Warning: Process/thread 318984 (host inti6232) exited with 1
* Warning: Process/thread 318986 (host inti6232) exited with 1
* Warning: Process/thread 318978 (host inti6232) exited with 1
* Warning: Process/thread 318987 (host inti6232) exited with 1
* Warning: Process/thread 354434 (host inti6229) exited with 1
* Warning: Process/thread 354438 (host inti6229) exited with 1
* Warning: Process/thread 354436 (host inti6229) exited with 1
* Warning: Process/thread 354437 (host inti6229) exited with 1
* Warning: Process/thread 319114 (host inti6231) exited with 1
* Warning: Process/thread 319116 (host inti6231) exited with 1
* Warning: Process/thread 319017 (host inti6232) exited with 1
* Warning: Process/thread 319019 (host inti6232) exited with 1
* Warning: Process/thread 354469 (host inti6229) exited with 1
* Warning: Process/thread 923192 (host inti6230) exited with 1
* Warning: Process/thread 319119 (host inti6231) exited with 1
* Warning: Process/thread 319118 (host inti6231) exited with 1
* Warning: Process/thread 354471 (host inti6229) exited with 1
* Warning: Process/thread 354470 (host inti6229) exited with 1
* Warning: Process/thread 319021 (host inti6232) exited with 1
* Warning: Process/thread 319022 (host inti6232) exited with 1
* Warning: Process/thread 319024 (host inti6232) exited with 1
* Warning: Process/thread 354475 (host inti6229) exited with 1
* Warning: Process/thread 319122 (host inti6231) exited with 1
* Warning: Process/thread 319123 (host inti6231) exited with 1
* Warning: Process/thread 354476 (host inti6229) exited with 1
* Warning: Process/thread 923194 (host inti6230) exited with 1
* Warning: Process/thread 319126 (host inti6231) exited with 1
* Warning: Process/thread 354477 (host inti6229) exited with 1
* Warning: Process/thread 319027 (host inti6232) exited with 1
* Warning: Process/thread 319030 (host inti6232) exited with 1
* Warning: Process/thread 319028 (host inti6232) exited with 1
* Warning: Process/thread 354478 (host inti6229) exited with 1
* Warning: Process/thread 319127 (host inti6231) exited with 1
* Warning: Process/thread 354480 (host inti6229) exited with 1
* Warning: Process/thread 923195 (host inti6230) exited with 1
* Warning: Process/thread 923196 (host inti6230) exited with 1
* Warning: Process/thread 319129 (host inti6231) exited with 1
* Warning: Process/thread 354481 (host inti6229) exited with 1
* Warning: Process/thread 319130 (host inti6231) exited with 1
* Warning: Process/thread 354483 (host inti6229) exited with 1
* Warning: Process/thread 923197 (host inti6230) exited with 1
* Warning: Process/thread 354484 (host inti6229) exited with 1
* Warning: Process/thread 319132 (host inti6231) exited with 1
* Warning: Process/thread 354488 (host inti6229) exited with 1
* Warning: Process/thread 354493 (host inti6229) exited with 1
* Warning: Process/thread 319131 (host inti6231) exited with 1
* Warning: Process/thread 319134 (host inti6231) exited with 1
* Warning: Process/thread 319136 (host inti6231) exited with 1
* Warning: Process/thread 319140 (host inti6231) exited with 1
* Warning: Process/thread 319139 (host inti6231) exited with 1
* Warning: Process/thread 923202 (host inti6230) exited with 1
* Warning: Process/thread 923200 (host inti6230) exited with 1
* Warning: Process/thread 319034 (host inti6232) exited with 1
* Warning: Process/thread 319033 (host inti6232) exited with 1
* Warning: Process/thread 319035 (host inti6232) exited with 1
* Warning: Process/thread 354496 (host inti6229) exited with 1
* Warning: Process/thread 923201 (host inti6230) exited with 1
* Warning: Process/thread 923206 (host inti6230) exited with 1
* Warning: Process/thread 354497 (host inti6229) exited with 1
* Warning: Process/thread 319036 (host inti6232) exited with 1
* Warning: Process/thread 923205 (host inti6230) exited with 1
* Warning: Process/thread 319038 (host inti6232) exited with 1
* Warning: Process/thread 319037 (host inti6232) exited with 1
* Warning: Process/thread 319041 (host inti6232) exited with 1
* Warning: Process/thread 923208 (host inti6230) exited with 1
* Warning: Process/thread 319043 (host inti6232) exited with 1
* Warning: Process/thread 923207 (host inti6230) exited with 1
* Warning: Process/thread 923213 (host inti6230) exited with 1
* Warning: Process/thread 923212 (host inti6230) exited with 1
* Warning: Process/thread 923214 (host inti6230) exited with 1
* Warning: Process/thread 923215 (host inti6230) exited with 1
* Warning: Process/thread 354495 (host inti6229) exited with 1
* Warning: Process/thread 319052 (host inti6232) exited with 1
* Warning: Process/thread 354517 (host inti6229) exited with 1
* Warning: Process/thread 354519 (host inti6229) exited with 1
* Warning: Process/thread 319067 (host inti6232) exited with 1
* Warning: Process/thread 923240 (host inti6230) exited with 1
* Warning: Process/thread 319165 (host inti6231) exited with 1
* Warning: Process/thread 319164 (host inti6231) exited with 1
* Warning: Process/thread 319151 (host inti6231) exited with 1
* Warning: Process/thread 319150 (host inti6231) exited with 1
* Warning: Process/thread 319069 (host inti6232) exited with 1
* Warning: Process/thread 354522 (host inti6229) exited with 1
* Warning: Process/thread 923242 (host inti6230) exited with 1
* Warning: Process/thread 319074 (host inti6232) exited with 1
* Warning: Process/thread 319172 (host inti6231) exited with 1
* Warning: Process/thread 354531 (host inti6229) exited with 1
* Warning: Process/thread 354521 (host inti6229) exited with 1
* Warning: Process/thread 354528 (host inti6229) exited with 1
* Warning: Process/thread 354532 (host inti6229) exited with 1
* Warning: Process/thread 319174 (host inti6231) exited with 1
* Warning: Process/thread 923244 (host inti6230) exited with 1
* Warning: Process/thread 319177 (host inti6231) exited with 1
* Warning: Process/thread 923245 (host inti6230) exited with 1
* Warning: Process/thread 354527 (host inti6229) exited with 1
* Warning: Process/thread 319176 (host inti6231) exited with 1
* Warning: Process/thread 319181 (host inti6231) exited with 1
* Warning: Process/thread 319184 (host inti6231) exited with 1
* Warning: Process/thread 319170 (host inti6231) exited with 1
* Warning: Process/thread 923248 (host inti6230) exited with 1
* Warning: Process/thread 923250 (host inti6230) exited with 1
* Warning: Process/thread 923251 (host inti6230) exited with 1
* Warning: Process/thread 319179 (host inti6231) exited with 1
* Warning: Process/thread 319171 (host inti6231) exited with 1
* Warning: Process/thread 354526 (host inti6229) exited with 1
* Warning: Process/thread 319182 (host inti6231) exited with 1
* Warning: Process/thread 923255 (host inti6230) exited with 1
* Warning: Process/thread 923256 (host inti6230) exited with 1
* Warning: Process/thread 923258 (host inti6230) exited with 1
* Warning: Process/thread 923259 (host inti6230) exited with 1
* Warning: Process/thread 923263 (host inti6230) exited with 1
* Warning: Process/thread 923266 (host inti6230) exited with 1
* Warning: Process/thread 354529 (host inti6229) exited with 1
* Warning: Process/thread 354530 (host inti6229) exited with 1
* Warning: Process/thread 923262 (host inti6230) exited with 1
* Warning: Process/thread 923254 (host inti6230) exited with 1
* Warning: Process/thread 923265 (host inti6230) exited with 1
* Warning: Process/thread 354548 (host inti6229) exited with 1
* Warning: Process/thread 354537 (host inti6229) exited with 1
* Warning: Process/thread 354550 (host inti6229) exited with 1
* Warning: Process/thread 319183 (host inti6231) exited with 1
* Warning: Process/thread 319180 (host inti6231) exited with 1
* Warning: Process/thread 354525 (host inti6229) exited with 1
* Warning: Process/thread 354538 (host inti6229) exited with 1
* Warning: Process/thread 354559 (host inti6229) exited with 1
* Warning: Process/thread 354539 (host inti6229) exited with 1
* Warning: Process/thread 354524 (host inti6229) exited with 1
* Warning: Process/thread 923277 (host inti6230) exited with 1
* Warning: Process/thread 319206 (host inti6231) exited with 1
* Warning: Process/thread 319210 (host inti6231) exited with 1
* Warning: Process/thread 319208 (host inti6231) exited with 1
* Warning: Process/thread 319209 (host inti6231) exited with 1
* Warning: Process/thread 319078 (host inti6232) exited with 1
* Warning: Process/thread 923287 (host inti6230) exited with 1
* Warning: Process/thread 319076 (host inti6232) exited with 1
* Warning: Process/thread 319216 (host inti6231) exited with 1
* Warning: Process/thread 319079 (host inti6232) exited with 1
* Warning: Process/thread 319071 (host inti6232) exited with 1
* Warning: Process/thread 319088 (host inti6232) exited with 1
* Warning: Process/thread 319090 (host inti6232) exited with 1
* Warning: Process/thread 319072 (host inti6232) exited with 1
* Warning: Process/thread 319091 (host inti6232) exited with 1
* Warning: Process/thread 319075 (host inti6232) exited with 1
* Warning: Process/thread 319081 (host inti6232) exited with 1
* Warning: Process/thread 319080 (host inti6232) exited with 1
* Warning: Process/thread 319092 (host inti6232) exited with 1
* Warning: Process/thread 319085 (host inti6232) exited with 1
* Warning: Process/thread 923292 (host inti6230) exited with 1
* Warning: Process/thread 319084 (host inti6232) exited with 1
* Warning: Process/thread 354566 (host inti6229) exited with 1
* Warning: Process/thread 923294 (host inti6230) exited with 1
* Warning: Process/thread 354569 (host inti6229) exited with 1
* Warning: Process/thread 319218 (host inti6231) exited with 1
* Warning: Process/thread 354573 (host inti6229) exited with 1
* Warning: Process/thread 354575 (host inti6229) exited with 1
* Warning: Process/thread 319083 (host inti6232) exited with 1
* Warning: Process/thread 319222 (host inti6231) exited with 1
* Warning: Process/thread 319082 (host inti6232) exited with 1
* Warning: Process/thread 923296 (host inti6230) exited with 1
* Warning: Process/thread 923298 (host inti6230) exited with 1
* Warning: Process/thread 354577 (host inti6229) exited with 1
* Warning: Process/thread 923301 (host inti6230) exited with 1
* Warning: Process/thread 319224 (host inti6231) exited with 1
* Warning: Process/thread 319226 (host inti6231) exited with 1
* Warning: Process/thread 319225 (host inti6231) exited with 1
* Warning: Process/thread 319227 (host inti6231) exited with 1
* Warning: Process/thread 354584 (host inti6229) exited with 1
* Warning: Process/thread 923303 (host inti6230) exited with 1
* Warning: Process/thread 319232 (host inti6231) exited with 1
* Warning: Process/thread 319234 (host inti6231) exited with 1
* Warning: Process/thread 923305 (host inti6230) exited with 1
* Warning: Process/thread 319236 (host inti6231) exited with 1
* Warning: Process/thread 923307 (host inti6230) exited with 1
* Warning: Process/thread 354586 (host inti6229) exited with 1
* Warning: Process/thread 354587 (host inti6229) exited with 1
* Warning: Process/thread 354590 (host inti6229) exited with 1
* Warning: Process/thread 354593 (host inti6229) exited with 1
* Warning: Process/thread 354594 (host inti6229) exited with 1
* Warning: Process/thread 319125 (host inti6232) exited with 1
* Warning: Process/thread 319127 (host inti6232) exited with 1
* Warning: Process/thread 319243 (host inti6231) exited with 1
* Warning: Process/thread 319245 (host inti6231) exited with 1
* Warning: Process/thread 923309 (host inti6230) exited with 1
* Warning: Process/thread 923313 (host inti6230) exited with 1
* Warning: Process/thread 923315 (host inti6230) exited with 1
* Warning: Process/thread 923314 (host inti6230) exited with 1
* Warning: Process/thread 923310 (host inti6230) exited with 1
* Warning: Process/thread 923311 (host inti6230) exited with 1
* Warning: Process/thread 923325 (host inti6230) exited with 1
* Warning: Process/thread 354603 (host inti6229) exited with 1
* Warning: Process/thread 354601 (host inti6229) exited with 1
* Warning: Process/thread 354602 (host inti6229) exited with 1
* Warning: Process/thread 354607 (host inti6229) exited with 1
* Warning: Process/thread 354608 (host inti6229) exited with 1
* Warning: Process/thread 923329 (host inti6230) exited with 1
* Warning: Process/thread 319254 (host inti6231) exited with 1
* Warning: Process/thread 319258 (host inti6231) exited with 1
* Warning: Process/thread 319260 (host inti6231) exited with 1
* Warning: Process/thread 319259 (host inti6231) exited with 1
* Warning: Process/thread 319264 (host inti6231) exited with 1
* Warning: Process/thread 923334 (host inti6230) exited with 1
* Warning: Process/thread 319130 (host inti6232) exited with 1
* Warning: Process/thread 319129 (host inti6232) exited with 1
* Warning: Process/thread 354614 (host inti6229) exited with 1
* Warning: Process/thread 319133 (host inti6232) exited with 1
* Warning: Process/thread 319135 (host inti6232) exited with 1
* Warning: Process/thread 923336 (host inti6230) exited with 1
* Warning: Process/thread 319266 (host inti6231) exited with 1
* Warning: Process/thread 354617 (host inti6229) exited with 1
* Warning: Process/thread 319136 (host inti6232) exited with 1
* Warning: Process/thread 354620 (host inti6229) exited with 1
* Warning: Process/thread 319270 (host inti6231) exited with 1
* Warning: Process/thread 319137 (host inti6232) exited with 1
* Warning: Process/thread 319139 (host inti6232) exited with 1
* Warning: Process/thread 354623 (host inti6229) exited with 1
* Warning: Process/thread 319141 (host inti6232) exited with 1
* Warning: Process/thread 319140 (host inti6232) exited with 1
* Warning: Process/thread 319143 (host inti6232) exited with 1
* Warning: Process/thread 319147 (host inti6232) exited with 1
* Warning: Process/thread 319154 (host inti6232) exited with 1
* Warning: Process/thread 319145 (host inti6232) exited with 1
* Warning: Process/thread 923344 (host inti6230) exited with 1
* Warning: Process/thread 319153 (host inti6232) exited with 1
* Warning: Process/thread 354625 (host inti6229) exited with 1
* Warning: Process/thread 923346 (host inti6230) exited with 1
* Warning: Process/thread 923349 (host inti6230) exited with 1
* Warning: Process/thread 319272 (host inti6231) exited with 1
* Warning: Process/thread 319274 (host inti6231) exited with 1
* Warning: Process/thread 319276 (host inti6231) exited with 1
* Warning: Process/thread 319277 (host inti6231) exited with 1
* Warning: Process/thread 354632 (host inti6229) exited with 1
* Warning: Process/thread 923352 (host inti6230) exited with 1
* Warning: Process/thread 923354 (host inti6230) exited with 1
* Warning: Process/thread 319281 (host inti6231) exited with 1
* Warning: Process/thread 319280 (host inti6231) exited with 1
* Warning: Process/thread 923356 (host inti6230) exited with 1
* Warning: Process/thread 319284 (host inti6231) exited with 1
* Warning: Process/thread 354634 (host inti6229) exited with 1
* Warning: Process/thread 354636 (host inti6229) exited with 1
* Warning: Process/thread 354638 (host inti6229) exited with 1
* Warning: Process/thread 319160 (host inti6232) exited with 1
* Warning: Process/thread 354641 (host inti6229) exited with 1
* Warning: Process/thread 354643 (host inti6229) exited with 1
* Warning: Process/thread 319165 (host inti6232) exited with 1
* Warning: Process/thread 319292 (host inti6231) exited with 1
* Warning: Process/thread 319293 (host inti6231) exited with 1
* Warning: Process/thread 923360 (host inti6230) exited with 1
* Warning: Process/thread 923361 (host inti6230) exited with 1
* Warning: Process/thread 923364 (host inti6230) exited with 1
* Warning: Process/thread 923366 (host inti6230) exited with 1
* Warning: Process/thread 923368 (host inti6230) exited with 1
* Warning: Process/thread 923367 (host inti6230) exited with 1
* Warning: Process/thread 923374 (host inti6230) exited with 1
* Warning: Process/thread 354649 (host inti6229) exited with 1
* Warning: Process/thread 354650 (host inti6229) exited with 1
* Warning: Process/thread 354652 (host inti6229) exited with 1
* Warning: Process/thread 354653 (host inti6229) exited with 1
* Warning: Process/thread 354655 (host inti6229) exited with 1
* Warning: Process/thread 319301 (host inti6231) exited with 1
* Warning: Process/thread 319306 (host inti6231) exited with 1
* Warning: Process/thread 319308 (host inti6231) exited with 1
* Warning: Process/thread 319309 (host inti6231) exited with 1
* Warning: Process/thread 319177 (host inti6232) exited with 1
* Warning: Process/thread 319178 (host inti6232) exited with 1
* Warning: Process/thread 923377 (host inti6230) exited with 1
* Warning: Process/thread 319312 (host inti6231) exited with 1
* Warning: Process/thread 923382 (host inti6230) exited with 1
* Warning: Process/thread 923384 (host inti6230) exited with 1
* Warning: Process/thread 354662 (host inti6229) exited with 1
* Warning: Process/thread 319314 (host inti6231) exited with 1
* Warning: Process/thread 319183 (host inti6232) exited with 1
* Warning: Process/thread 319182 (host inti6232) exited with 1
* Warning: Process/thread 319316 (host inti6231) exited with 1
* Warning: Process/thread 354665 (host inti6229) exited with 1
* Warning: Process/thread 354668 (host inti6229) exited with 1
* Warning: Process/thread 319187 (host inti6232) exited with 1
* Warning: Process/thread 923386 (host inti6230) exited with 1
* Warning: Process/thread 354671 (host inti6229) exited with 1
* Warning: Process/thread 319189 (host inti6232) exited with 1
* Warning: Process/thread 354673 (host inti6229) exited with 1
* Warning: Process/thread 319190 (host inti6232) exited with 1
* Warning: Process/thread 319193 (host inti6232) exited with 1
* Warning: Process/thread 923390 (host inti6230) exited with 1
* Warning: Process/thread 319194 (host inti6232) exited with 1
* Warning: Process/thread 923393 (host inti6230) exited with 1
* Warning: Process/thread 319196 (host inti6232) exited with 1
* Warning: Process/thread 319198 (host inti6232) exited with 1
* Warning: Process/thread 319199 (host inti6232) exited with 1
* Warning: Process/thread 319200 (host inti6232) exited with 1
* Warning: Process/thread 319203 (host inti6232) exited with 1
* Warning: Process/thread 319320 (host inti6231) exited with 1
* Warning: Process/thread 319322 (host inti6231) exited with 1
* Warning: Process/thread 319324 (host inti6231) exited with 1
* Warning: Process/thread 354680 (host inti6229) exited with 1
* Warning: Process/thread 923399 (host inti6230) exited with 1
* Warning: Process/thread 319326 (host inti6231) exited with 1
* Warning: Process/thread 923401 (host inti6230) exited with 1
* Warning: Process/thread 923402 (host inti6230) exited with 1
* Warning: Process/thread 319328 (host inti6231) exited with 1
* Warning: Process/thread 319330 (host inti6231) exited with 1
* Warning: Process/thread 319332 (host inti6231) exited with 1
* Warning: Process/thread 354682 (host inti6229) exited with 1
* Warning: Process/thread 354683 (host inti6229) exited with 1
* Warning: Process/thread 319207 (host inti6232) exited with 1
* Warning: Process/thread 319211 (host inti6232) exited with 1
* Warning: Process/thread 354686 (host inti6229) exited with 1
* Warning: Process/thread 354689 (host inti6229) exited with 1
* Warning: Process/thread 354691 (host inti6229) exited with 1
* Warning: Process/thread 319341 (host inti6231) exited with 1
* Warning: Process/thread 319342 (host inti6231) exited with 1
* Warning: Process/thread 923409 (host inti6230) exited with 1
* Warning: Process/thread 923410 (host inti6230) exited with 1
* Warning: Process/thread 923414 (host inti6230) exited with 1
* Warning: Process/thread 923417 (host inti6230) exited with 1
* Warning: Process/thread 923419 (host inti6230) exited with 1
* Warning: Process/thread 923420 (host inti6230) exited with 1
* Warning: Process/thread 354697 (host inti6229) exited with 1
* Warning: Process/thread 923418 (host inti6230) exited with 1
* Warning: Process/thread 354699 (host inti6229) exited with 1
* Warning: Process/thread 354700 (host inti6229) exited with 1
* Warning: Process/thread 354701 (host inti6229) exited with 1
* Warning: Process/thread 354703 (host inti6229) exited with 1
* Warning: Process/thread 319349 (host inti6231) exited with 1
* Warning: Process/thread 319351 (host inti6231) exited with 1
* Warning: Process/thread 319225 (host inti6232) exited with 1
* Warning: Process/thread 319228 (host inti6232) exited with 1
* Warning: Process/thread 319356 (host inti6231) exited with 1
* Warning: Process/thread 319358 (host inti6231) exited with 1
* Warning: Process/thread 923428 (host inti6230) exited with 1
* Warning: Process/thread 319360 (host inti6231) exited with 1
* Warning: Process/thread 923430 (host inti6230) exited with 1
* Warning: Process/thread 354710 (host inti6229) exited with 1
* Warning: Process/thread 923432 (host inti6230) exited with 1
* Warning: Process/thread 319231 (host inti6232) exited with 1
* Warning: Process/thread 319232 (host inti6232) exited with 1
* Warning: Process/thread 319363 (host inti6231) exited with 1
* Warning: Process/thread 354716 (host inti6229) exited with 1
* Warning: Process/thread 923434 (host inti6230) exited with 1
* Warning: Process/thread 354714 (host inti6229) exited with 1
* Warning: Process/thread 319235 (host inti6232) exited with 1
* Warning: Process/thread 354719 (host inti6229) exited with 1
* Warning: Process/thread 354721 (host inti6229) exited with 1
* Warning: Process/thread 319237 (host inti6232) exited with 1
* Warning: Process/thread 923436 (host inti6230) exited with 1
* Warning: Process/thread 319241 (host inti6232) exited with 1
* Warning: Process/thread 923438 (host inti6230) exited with 1
* Warning: Process/thread 319362 (host inti6231) exited with 1
* Warning: Process/thread 319239 (host inti6232) exited with 1
* Warning: Process/thread 319243 (host inti6232) exited with 1
* Warning: Process/thread 319368 (host inti6231) exited with 1
* Warning: Process/thread 319245 (host inti6232) exited with 1
* Warning: Process/thread 319370 (host inti6231) exited with 1
* Warning: Process/thread 319372 (host inti6231) exited with 1
* Warning: Process/thread 319248 (host inti6232) exited with 1
* Warning: Process/thread 319247 (host inti6232) exited with 1
* Warning: Process/thread 319249 (host inti6232) exited with 1
* Warning: Process/thread 319251 (host inti6232) exited with 1
* Warning: Process/thread 354724 (host inti6229) exited with 1
* Warning: Process/thread 923447 (host inti6230) exited with 1
* Warning: Process/thread 319374 (host inti6231) exited with 1
* Warning: Process/thread 319377 (host inti6231) exited with 1
* Warning: Process/thread 319379 (host inti6231) exited with 1
* Warning: Process/thread 923450 (host inti6230) exited with 1
* Warning: Process/thread 319380 (host inti6231) exited with 1
* Warning: Process/thread 923449 (host inti6230) exited with 1
* Warning: Process/thread 354730 (host inti6229) exited with 1
* Warning: Process/thread 354731 (host inti6229) exited with 1
* Warning: Process/thread 319255 (host inti6232) exited with 1
* Warning: Process/thread 319259 (host inti6232) exited with 1
* Warning: Process/thread 354734 (host inti6229) exited with 1
* Warning: Process/thread 354737 (host inti6229) exited with 1
* Warning: Process/thread 354739 (host inti6229) exited with 1
* Warning: Process/thread 319388 (host inti6231) exited with 1
* Warning: Process/thread 319389 (host inti6231) exited with 1
* Warning: Process/thread 354745 (host inti6229) exited with 1
* Warning: Process/thread 923458 (host inti6230) exited with 1
* Warning: Process/thread 923461 (host inti6230) exited with 1
* Warning: Process/thread 354747 (host inti6229) exited with 1
* Warning: Process/thread 354749 (host inti6229) exited with 1
* Warning: Process/thread 354750 (host inti6229) exited with 1
* Warning: Process/thread 354751 (host inti6229) exited with 1
* Warning: Process/thread 923463 (host inti6230) exited with 1
* Warning: Process/thread 923465 (host inti6230) exited with 1
* Warning: Process/thread 923466 (host inti6230) exited with 1
* Warning: Process/thread 319397 (host inti6231) exited with 1
* Warning: Process/thread 923469 (host inti6230) exited with 1
* Warning: Process/thread 923470 (host inti6230) exited with 1
* Warning: Process/thread 319399 (host inti6231) exited with 1
* Warning: Process/thread 319273 (host inti6232) exited with 1
* Warning: Process/thread 319275 (host inti6232) exited with 1
* Warning: Process/thread 319404 (host inti6231) exited with 1
* Warning: Process/thread 319406 (host inti6231) exited with 1
* Warning: Process/thread 319408 (host inti6231) exited with 1
* Warning: Process/thread 923474 (host inti6230) exited with 1
* Warning: Process/thread 923478 (host inti6230) exited with 1
* Warning: Process/thread 923480 (host inti6230) exited with 1
* Warning: Process/thread 319279 (host inti6232) exited with 1
* Warning: Process/thread 319280 (host inti6232) exited with 1
* Warning: Process/thread 354762 (host inti6229) exited with 1
* Warning: Process/thread 923482 (host inti6230) exited with 1
* Warning: Process/thread 354763 (host inti6229) exited with 1
* Warning: Process/thread 354759 (host inti6229) exited with 1
* Warning: Process/thread 354768 (host inti6229) exited with 1
* Warning: Process/thread 319283 (host inti6232) exited with 1
* Warning: Process/thread 354767 (host inti6229) exited with 1
* Warning: Process/thread 923484 (host inti6230) exited with 1
* Warning: Process/thread 319410 (host inti6231) exited with 1
* Warning: Process/thread 319413 (host inti6231) exited with 1
* Warning: Process/thread 319414 (host inti6231) exited with 1
* Warning: Process/thread 319418 (host inti6231) exited with 1
* Warning: Process/thread 923486 (host inti6230) exited with 1
* Warning: Process/thread 354772 (host inti6229) exited with 1
* Warning: Process/thread 319420 (host inti6231) exited with 1
* Warning: Process/thread 354778 (host inti6229) exited with 1
* Warning: Process/thread 354779 (host inti6229) exited with 1
* Warning: Process/thread 319422 (host inti6231) exited with 1
* Warning: Process/thread 319285 (host inti6232) exited with 1
* Warning: Process/thread 354782 (host inti6229) exited with 1
* Warning: Process/thread 319425 (host inti6231) exited with 1
* Warning: Process/thread 319428 (host inti6231) exited with 1
* Warning: Process/thread 923495 (host inti6230) exited with 1
* Warning: Process/thread 319430 (host inti6231) exited with 1
* Warning: Process/thread 319287 (host inti6232) exited with 1
* Warning: Process/thread 923497 (host inti6230) exited with 1
* Warning: Process/thread 923498 (host inti6230) exited with 1
* Warning: Process/thread 354789 (host inti6229) exited with 1
* Warning: Process/thread 319289 (host inti6232) exited with 1
* Warning: Process/thread 319435 (host inti6231) exited with 1
* Warning: Process/thread 319436 (host inti6231) exited with 1
* Warning: Process/thread 319291 (host inti6232) exited with 1
* Warning: Process/thread 319293 (host inti6232) exited with 1
* Warning: Process/thread 354792 (host inti6229) exited with 1
* Warning: Process/thread 923507 (host inti6230) exited with 1
* Warning: Process/thread 319295 (host inti6232) exited with 1
* Warning: Process/thread 319296 (host inti6232) exited with 1
* Warning: Process/thread 354794 (host inti6229) exited with 1
* Warning: Process/thread 354795 (host inti6229) exited with 1
* Info: Process launched (host inti6229, process 354085)
* Warning: Process/thread 319298 (host inti6232) exited with 1
Version : master-2023-07-03
MPI : 64 processes
CPU : 256 cores (max 256)
OpenMP : 8 threads (v5.0)
SIMD : AVX2
SOATL : HFA P1 / A32 / C8
Cuda : v12.0.76 (no GPU)
Grids : MultiMatFieldSet FullFieldMechSet MultimatMechFieldSet
* Warning: Process/thread 319299 (host inti6232) exited with 1
+ exanbCorePlugin
* Warning: Process/thread 319304 (host inti6232) exited with 1
* Warning: Process/thread 319307 (host inti6232) exited with 1
* Warning: Process/thread 923509 (host inti6230) exited with 1
* Warning: Process/thread 923514 (host inti6230) exited with 1
* Warning: Process/thread 923518 (host inti6230) exited with 1
* Warning: Process/thread 923512 (host inti6230) exited with 1
* Warning: Process/thread 923519 (host inti6230) exited with 1
* Warning: Process/thread 319321 (host inti6232) exited with 1
* Warning: Process/thread 319323 (host inti6232) exited with 1
* Warning: Process/thread 319327 (host inti6232) exited with 1
* Warning: Process/thread 319329 (host inti6232) exited with 1
* Warning: Process/thread 319331 (host inti6232) exited with 1
* Warning: Process/thread 319332 (host inti6232) exited with 1
* Warning: Process/thread 319336 (host inti6232) exited with 1
* Warning: Process/thread 319338 (host inti6232) exited with 1
* Warning: Process/thread 319340 (host inti6232) exited with 1
* Warning: Process/thread 319342 (host inti6232) exited with 1
+ exanbIOPlugin
+ exanbMPIPlugin
+ exaStampIOPlugin
+ exaStampParticleSpeciesPlugin
+ exanbGridCellParticlesPlugin
+ raveloPlugin
+ exanbParticleNeighborsPlugin
+ exanbLogicPlugin
+ exaStampComputePlugin
+ exanbAMRPlugin
+ exanbDefBoxPlugin
+ exaStampMPIPlugin
+ exaStampAnalyticsPlugin
*SUPPRESS* sim.first_iteration.init_particles.parallel_update_particles.update_particle_neighbors.resize_particle_locks
*SUPPRESS* sim.loop.scheme.check_and_update_particles.update_particles_full.update_particles_full_body.parallel_update_particles.update_particle_neighbors.resize_particle_locks
*SUPPRESS* sim.simulation_epilog.simulation_thermodynamic_state
███████╗██╗ ██╗ █████╗ ███████╗████████╗ █████╗ ███╗ ███╗██████╗ ██╗ ██╗██████╗
██╔════╝╚██╗██╔╝██╔══██╗██╔════╝╚══██╔══╝██╔══██╗████╗ ████║██╔══██╗ ██║ ██║╚════██╗
█████╗ ╚███╔╝ ███████║███████╗ ██║ ███████║██╔████╔██║██████╔╝ ██║ ██║ █████╔╝
██╔══╝ ██╔██╗ ██╔══██║╚════██║ ██║ ██╔══██║██║╚██╔╝██║██╔═══╝ ╚██╗ ██╔╝ ╚═══██╗
███████╗██╔╝ ██╗██║ ██║███████║ ██║ ██║ ██║██║ ╚═╝ ██║██║ ╚████╔╝ ██████╔╝
╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝ ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═══╝ ╚═════╝
=========== Cuda ================
No GPU found
=================================
======== Atom species ========
- Ta: { mass: 1.80948000e+02 , z: 73 , charge: 0.00000000e+00 }
==============================
============ atoms_000100000.MpiIO ============
fields = rx ry rz vx vy vz id type
domain bounds = (0.00000000e+00,0.00000000e+00,0.00000000e+00)-(2.98000000e+03,6.60000000e+02,6.60000000e+02) , size=2.98000000e+03,6.60000000e+02,6.60000000e+02
domain xform = {(1.00000000e+00,0.00000000e+00,0.00000000e+00),(0.00000000e+00,1.00121212e+00,0.00000000e+00),(0.00000000e+00,0.00000000e+00,1.00121212e+00)} , inv = {(1.00000000e+00,0.00000000e+00,0.00000000e+00),(0.00000000e+00,9.98789346e-01,0.00000000e+00),(0.00000000e+00,0.00000000e+00,9.98789346e-01)}
domain size = 2.98000000e+03,6.60800000e+02,6.60800000e+02
cell size = 2.00000000e+01
grid size = 149,33,33
periodicity = false , true , true
expandable = true
particles = 50000000
time step = 100000
time = 1.00000000e+02
opt. size = 856 (compressed size 172)
data chunks = 780
================================
========= Neighborhood ==========
transform = {(1.00000000e+00,0.00000000e+00,0.00000000e+00),(0.00000000e+00,1.00121212e+00,0.00000000e+00),(0.00000000e+00,0.00000000e+00,1.00121212e+00)}
scale = 1.00000000e+00 / 1.00121212e+00
rcut_max = 5.30000000e+00
ghost_dist_max = 1.06000000e+01
rcut_inc = 1.00000000e+00
nbh_dist_lab = 6.30000000e+00
nbh_dist = 6.30000000e+00
max_displ = 4.99394673e-01
bond_max_dist = 0.00000000e+00
bond_max_stretch = 0.00000000e+00
ghost_dist = 1.16000000e+01
=================================
======= Simulation Domain =======
bounds = (0.00000000e+00,0.00000000e+00,0.00000000e+00)-(2.98000000e+03,6.60000000e+02,6.60000000e+02)
dom size = 2.98000000e+03,6.60000000e+02,6.60000000e+02
grid = 149,33,33
cell size = 2.00000000e+01
grid size = 2.98000000e+03,6.60000000e+02,6.60000000e+02
periodic = false , true , true
xform = {(1.00000000e+00,0.00000000e+00,0.00000000e+00),(0.00000000e+00,1.00121212e+00,0.00000000e+00),(0.00000000e+00,0.00000000e+00,1.00121212e+00)} (diagonal)
inv_xform = {(1.00000000e+00,0.00000000e+00,0.00000000e+00),(0.00000000e+00,9.98789346e-01,0.00000000e+00),(0.00000000e+00,0.00000000e+00,9.98789346e-01)}
scale = 1.00000000e+00 / 1.00121212e+00
phys size = 2.98000000e+03,6.60800000e+02,6.60800000e+02 , cell size = 2.00000000e+01,2.00242424e+01,2.00242424e+01
=================================
========= Performance ===========
Particles = 50,000,000
Cells = 162,261
MPI Processes = 64
OMP threads = 8
MPI x OMP = 512
Cells / MPI Proc = 2,535
Cells / Thread = 316
Part. / MPI Proc = 781,250
Part. / Thread = 97,656
=================================
======== Atom species ========
- Ta: { mass: 1.80948000e+02 , z: 73 , charge: 0.00000000e+00 }
==============================
Step Time (ps) Particles Mv/Ext/Imb. Tot. E. (eV/part) Kin. E. (eV/part) Pot. E. (eV/part) Temp. (K) Press. (Pa) sMises (Pa) Vol. (ang^3) Mass
100000 1.000000e+02 50000000 -6.1082554854e+00 1.6715470686e+00 -7.7798025540e+00 5353.060 -2.432e+09 9.261e+08 1.301237e+09 9.047e+09
100001 1.000010e+02 50000000 -6.1082554855e+00 1.6715473375e+00 -7.7798028230e+00 5353.062 -2.432e+09 9.261e+08 1.301237e+09 9.047e+09
100050 1.000500e+02 50000000 3 1.4e-01 -6.1082554950e+00 1.6716409021e+00 -7.7798963971e+00 5353.786 -2.426e+09 9.251e+08 1.301237e+09 9.047e+09
100100 1.001000e+02 50000000 3 -6.1082555001e+00 1.6716933529e+00 -7.7799488530e+00 5354.192 -2.419e+09 9.230e+08 1.301237e+09 9.047e+09
100150 1.001500e+02 50000000 4 1.2e-01 -6.1082554978e+00 1.6716569404e+00 -7.7799124381e+00 5353.910 -2.412e+09 9.206e+08 1.301237e+09 9.047e+09
100200 1.002000e+02 50000000 3 -6.1082555065e+00 1.6717093839e+00 -7.7799648904e+00 5354.316 -2.405e+09 9.193e+08 1.301237e+09 9.047e+09
100250 1.002500e+02 50000000 3 d 1.2e-01 -6.1082555070e+00 1.6717357717e+00 -7.7799912788e+00 5354.520 -2.382e+09 9.116e+08 1.309970e+09 9.047e+09
100300 1.003000e+02 50000000 4 -6.1082555125e+00 1.6717794773e+00 -7.7800349898e+00 5354.858 -2.376e+09 9.100e+08 1.309970e+09 9.047e+09
100350 1.003500e+02 50000000 3 1.1e-01 -6.1082555130e+00 1.6717896852e+00 -7.7800451982e+00 5354.937 -2.369e+09 9.086e+08 1.309970e+09 9.047e+09
100400 1.004000e+02 50000000 4 1.1e-01 -6.1082555171e+00 1.6718332496e+00 -7.7800887666e+00 5355.274 -2.361e+09 9.069e+08 1.309970e+09 9.047e+09
100450 1.004500e+02 50000000 3 -6.1082555227e+00 1.6718805745e+00 -7.7801360972e+00 5355.640 -2.354e+09 9.058e+08 1.309970e+09 9.047e+09
100500 1.005000e+02 50000000 4 1.1e-01 -6.1082555231e+00 1.6718825446e+00 -7.7801380676e+00 5355.655 -2.346e+09 9.042e+08 1.309970e+09 9.047e+09
Profiling ......................................... tot. time ( GPU ) avginb maxinb count percent
sim ............................................... 6.158e+05 0.000 0.000 1 100.00%
message
hw_device_init
mpi_comm_world
init_cuda
grid_flavor_full_mechanics
grid_flavor_full_mechanics
global
species
particle_regions
preinit_rcut_max
domain
ravelo_force
nbh_dist
domain
read_dump_atoms ................................. 1.283e+03 0.009 0.009 1 0.21%
species
grid_memory_compact
init_rcut_max
ravelo_force
nbh_dist
print_domain
performance_adviser
do_init_temperature
greater_than
greater_than
block3
gaussian_noise_v
init_temperature
block4
init_temperature
species
first_iteration ................................. 3.038e+03 0.000 0.000 1 0.49%
init_particles ................................ 1.536e+03 0.242 0.242 1 0.25%
move_particles
extend_domain
load_balance
simple_cost_model
load_balance_rcb
parallel_update_particles ................... 1.509e+03 0.246 0.246 1 0.25%
migrate_cell_particles .................... 6.405e+02 0.122 0.122 1 0.10%
rebuild_amr
backup_r
ghost_full_update ......................... 1.789e+02 0.893 0.893 1 0.03%
ghost_comm_scheme ....................... 9.476e+01 1.498 1.498 1 0.02%
ghost_update_all ........................ 8.417e+01 0.534 0.534 1 0.01%
grid_memory_compact
update_particle_neighbors ................. 6.473e+02 0.506 0.506 1 0.11%
amr_grid_pairs
chunk_neighbors ......................... 6.459e+02 0.508 0.508 1 0.10%
zero_force_energy
ravelo_force .................................. 8.715e+02 0.384 0.384 1 0.14%
force_to_accel
simulation_thermodynamic_state ................ 6.261e+02 2.357 2.357 1 0.10%
print_thermodynamic_state
dump_thermodynamic_state
next_time_step
loop ............................................ 6.111e+05 0.000 0.000 1 99.24%
begin_iteration
nth_timestep
nth_timestep
nth_timestep
trigger_thermo_state
boolean_or
boolean_or
nth_timestep
scheme ........................................ 5.698e+05 0.089 0.520 500 92.54%
push_f_v_r .................................. 7.684e+02 0.932 1.958 500 0.12%
push_f_v .................................... 4.754e+02 0.825 1.698 500 0.08%
check_and_update_particles .................. 1.995e+05 1.986 4.092 500 32.40%
particle_displ_over ....................... 1.674e+05 2.226 4.811 500 27.19%
update_particles_full ..................... 2.818e+04 0.413 0.477 34 4.58%
update_particles_full_body .............. 2.818e+04 0.413 0.477 34 4.58%
move_particles ........................ 2.400e+02 0.985 1.078 34 0.04%
trigger_load_balance .................. 2.389e+02 0.972 1.073 34 0.04%
trigger_lb_tmp
nth_timestep
extend_domain ....................... 2.385e+02 0.975 1.076 34 0.04%
boolean_or
load_balancing_if_triggered
load_balance
simple_cost_model
load_balance_rcb
parallel_update_particles ............. 2.767e+04 0.420 0.487 34 4.49%
migrate_cell_particles .............. 1.976e+03 0.265 1.113 34 0.32%
rebuild_amr ......................... 4.895e+02 0.467 0.561 34 0.08%
backup_r ............................ 1.101e+02 0.424 0.592 34 0.02%
ghost_full_update ................... 3.534e+03 0.362 0.502 34 0.57%
ghost_comm_scheme ................. 1.197e+03 1.456 1.917 34 0.19%
ghost_update_all .................. 2.337e+03 0.539 0.607 34 0.38%
grid_memory_compact ................. 2.061e+02 1.706 2.859 34 0.03%
update_particle_neighbors ........... 2.135e+04 0.494 0.518 34 3.47%
amr_grid_pairs
chunk_neighbors ................... 2.135e+04 0.494 0.518 34 3.47%
update_particles_fast ..................... 3.906e+03 0.546 1.138 466 0.63%
update_particles_fast_body .............. 3.905e+03 0.546 1.138 466 0.63%
with_analysis ......................... 1.678e+03 0.595 0.768 44 0.27%
ghost_update_r_v_vir ................ 1.678e+03 0.595 0.768 44 0.27%
with_analysis ......................... 2.227e+03 0.541 1.139 422 0.36%
ghost_update_r ...................... 2.225e+03 0.541 1.139 422 0.36%
lb_event_counter
zero_force_energy ........................... 3.727e+03 0.513 2.005 500 0.61%
ravelo_force ................................ 3.645e+05 0.468 0.486 500 59.19%
force_to_accel .............................. 3.487e+02 0.759 1.437 500 0.06%
push_f_v .................................... 5.403e+02 0.755 4.068 500 0.09%
end_iteration ................................. 4.125e+04 0.770 2.176 500 6.70%
thermo_state_if_triggered ................... 2.049e+04 2.121 2.269 51 3.33%
simulation_thermodynamic_state ............ 2.049e+04 2.121 2.269 51 3.33%
dump_data_if_triggered
print_log_if_triggered
print_thermodynamic_state
dump_thermo_if_triggered
dump_thermodynamic_state
dump_analysis_if_triggered .................. 2.076e+04 0.108 0.131 50 3.37%
dump_analysis ............................. 2.076e+04 0.108 0.131 50 3.37%
resize_grid_cell_values
grid_cell_particle_splatting ............ 1.174e+04 0.375 0.428 50 1.91%
grid_connected_components ............... 8.313e+03 1.510 1.635 50 1.35%
timestep_file
connected_grid_analysis ................. 4.548e+02 11.659 15.213 50 0.07%
timestep_file
grid_slice_analysis ..................... 2.428e+02 3.831 4.088 50 0.04%
next_time_step
sim_continue
simulation_epilog
final_dump
finalize_cuda
==================================
* Info: Process launched (host inti6229, process 354091)
* Info: Process finished (host inti6229, process 354091)
* Warning: (host inti6229, process 354091) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354097)
* Info: Process finished (host inti6229, process 354097)
* Warning: (host inti6229, process 354097) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354122)
* Info: Process finished (host inti6229, process 354122)
* Warning: (host inti6229, process 354122) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354086)
* Info: Process finished (host inti6229, process 354086)
* Warning: (host inti6229, process 354086) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354096)
* Info: Process finished (host inti6229, process 354096)
* Warning: (host inti6229, process 354096) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354098)
* Info: Process finished (host inti6229, process 354098)
* Warning: (host inti6229, process 354098) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354095)
* Info: Process finished (host inti6229, process 354095)
* Warning: (host inti6229, process 354095) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process finished (host inti6229, process 354085)
* Warning: (host inti6229, process 354085) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318735)
* Info: Process finished (host inti6231, process 318735)
* Warning: (host inti6231, process 318735) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318732)
* Info: Process finished (host inti6231, process 318732)
* Warning: (host inti6231, process 318732) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318739)
* Info: Process finished (host inti6231, process 318739)
* Warning: (host inti6231, process 318739) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318733)
* Info: Process finished (host inti6231, process 318733)
* Warning: (host inti6231, process 318733) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318734)
* Info: Process finished (host inti6231, process 318734)
* Warning: (host inti6231, process 318734) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318743)
* Info: Process finished (host inti6231, process 318743)
* Warning: (host inti6231, process 318743) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922818)
* Info: Process finished (host inti6230, process 922818)
* Info: Process launched (host inti6231, process 318742)
* Info: Process finished (host inti6231, process 318742)
* Warning: (host inti6231, process 318742) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Warning: (host inti6230, process 922818) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318769)
* Info: Process finished (host inti6231, process 318769)
* Warning: (host inti6231, process 318769) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922817)
* Info: Process finished (host inti6230, process 922817)
* Warning: (host inti6230, process 922817) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354088)
* Info: Process finished (host inti6229, process 354088)
* Warning: (host inti6229, process 354088) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318736)
* Info: Process finished (host inti6231, process 318736)
* Warning: (host inti6231, process 318736) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318738)
* Info: Process finished (host inti6231, process 318738)
* Warning: (host inti6231, process 318738) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922819)
* Info: Process finished (host inti6230, process 922819)
* Warning: (host inti6230, process 922819) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318744)
* Info: Process finished (host inti6231, process 318744)
* Info: Process launched (host inti6229, process 354094)
* Info: Process finished (host inti6229, process 354094)
* Warning: (host inti6231, process 318744) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Warning: (host inti6229, process 354094) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922815)
* Info: Process finished (host inti6230, process 922815)
* Warning: (host inti6230, process 922815) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318746)
* Info: Process finished (host inti6231, process 318746)
* Warning: (host inti6231, process 318746) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318737)
* Info: Process finished (host inti6231, process 318737)
* Warning: (host inti6231, process 318737) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922811)
* Info: Process finished (host inti6230, process 922811)
* Warning: (host inti6230, process 922811) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354087)
* Info: Process finished (host inti6229, process 354087)
* Info: Process launched (host inti6231, process 318745)
* Info: Process finished (host inti6231, process 318745)
* Warning: (host inti6229, process 354087) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Warning: (host inti6231, process 318745) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922816)
* Info: Process finished (host inti6230, process 922816)
* Warning: (host inti6230, process 922816) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922813)
* Info: Process finished (host inti6230, process 922813)
* Warning: (host inti6230, process 922813) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922823)
* Info: Process finished (host inti6230, process 922823)
* Warning: (host inti6230, process 922823) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318740)
* Info: Process finished (host inti6231, process 318740)
* Warning: (host inti6231, process 318740) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354090)
* Info: Process finished (host inti6229, process 354090)
* Warning: (host inti6229, process 354090) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922822)
* Info: Process finished (host inti6230, process 922822)
* Warning: (host inti6230, process 922822) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318636)
* Info: Process finished (host inti6232, process 318636)
* Warning: (host inti6232, process 318636) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354093)
* Info: Process finished (host inti6229, process 354093)
* Warning: (host inti6229, process 354093) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354092)
* Info: Process finished (host inti6229, process 354092)
* Warning: (host inti6229, process 354092) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922810)
* Info: Process finished (host inti6230, process 922810)
* Info: Process launched (host inti6232, process 318637)
* Info: Process finished (host inti6232, process 318637)
* Warning: (host inti6230, process 922810) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Warning: (host inti6232, process 318637) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354089)
* Info: Process finished (host inti6229, process 354089)
* Warning: (host inti6229, process 354089) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922821)
* Info: Process finished (host inti6230, process 922821)
* Warning: (host inti6230, process 922821) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922814)
* Info: Process finished (host inti6230, process 922814)
* Warning: (host inti6230, process 922814) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922820)
* Info: Process finished (host inti6230, process 922820)
* Warning: (host inti6230, process 922820) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318634)
* Info: Process finished (host inti6232, process 318634)
* Warning: (host inti6232, process 318634) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922847)
* Info: Process finished (host inti6230, process 922847)
* Warning: (host inti6230, process 922847) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318638)
* Info: Process finished (host inti6232, process 318638)
* Warning: (host inti6232, process 318638) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922812)
* Info: Process finished (host inti6230, process 922812)
* Warning: (host inti6230, process 922812) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318644)
* Info: Process finished (host inti6232, process 318644)
* Warning: (host inti6232, process 318644) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318635)
* Info: Process finished (host inti6232, process 318635)
* Warning: (host inti6232, process 318635) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318633)
* Info: Process finished (host inti6232, process 318633)
* Info: Process launched (host inti6232, process 318632)
* Info: Process finished (host inti6232, process 318632)
* Warning: (host inti6232, process 318632) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Warning: (host inti6232, process 318633) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318642)
* Info: Process finished (host inti6232, process 318642)
* Warning: (host inti6232, process 318642) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318640)
* Info: Process finished (host inti6232, process 318640)
* Warning: (host inti6232, process 318640) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318639)
* Info: Process finished (host inti6232, process 318639)
* Warning: (host inti6232, process 318639) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6231, process 318741)
* Info: Process finished (host inti6231, process 318741)
* Warning: (host inti6231, process 318741) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318646)
* Info: Process finished (host inti6232, process 318646)
* Warning: (host inti6232, process 318646) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6229, process 354099)
* Info: Process finished (host inti6229, process 354099)
* Warning: (host inti6229, process 354099) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318641)
* Info: Process finished (host inti6232, process 318641)
* Warning: (host inti6232, process 318641) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318668)
* Info: Process finished (host inti6232, process 318668)
* Warning: (host inti6232, process 318668) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6230, process 922809)
* Info: Process finished (host inti6230, process 922809)
* Warning: (host inti6230, process 922809) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318643)
* Info: Process finished (host inti6232, process 318643)
* Warning: (host inti6232, process 318643) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Process launched (host inti6232, process 318645)
* Info: Process finished (host inti6232, process 318645)
* Warning: (host inti6232, process 318645) Observed more threads (18) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=18.
* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.
Info: 15/64 lprof instances finished
Info: 16/64 lprof instances finished
Info: 27/64 lprof instances finished
Info: 57/64 lprof instances finished
* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.* Info: Detected stop signals that could make lprof alter execution. In case of issues, rerun with -verbose for more information.
Your experiment path is /ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0
To display your profiling results:
###################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
###################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Functions | Per-node | maqao lprof -df -dn xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Functions | Per-process | maqao lprof -df -dp xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Functions | Per-thread | maqao lprof -df -dt xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Loops | Cluster-wide | maqao lprof -dl xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Loops | Per-node | maqao lprof -dl -dn xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Loops | Per-process | maqao lprof -dl -dp xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/ccc/scratch/cont001/ocre/oserete/exastamp/ov_dirs/scal_16x8_killian_20230703_v2/tools/lprof_npsu_run_0 #
###################################################################################################################################################################
