Segmentation fault when iterative Hirshfeld analysis is performed

Dear developers,
I encountered a segmentation fault when performing iterative Hirshfeld population analysis using Q-Chem 6.0.2. The input is shown below:

$molecule
0 1
 O        -4.56977         5.57593         6.84542
 O        -0.62516         0.57814         4.03546
 O        -5.46843         4.20063         5.31248
 H        -6.19207         4.50606         5.61012
 O         1.61369         4.45840         5.92759
 O         0.35957         5.87346         7.12335
 O        -2.53572         3.80468         1.46019
 N        -1.65841         2.16391         2.77537
 C        -4.45980         4.76895         5.94175
 C        -2.20829         1.69879         0.42309
 C        -3.12753         4.32017         5.42759
 C        -3.01814         3.44572         4.34725
 H        -3.79712         3.12349         3.90931
 C        -1.97995         4.79968         6.05871
 H        -2.05795         5.39950         6.79131
 C        -0.37150        -1.18999         1.84046
 H         0.07297        -1.43935         2.64230
 C         0.51772         4.95915         6.28094
 C        -1.61028         0.43151         0.55360
 C        -2.15532         2.65220         1.55761
 C        -0.61452         3.51568         4.54362
 H         0.24070         3.23253         4.24208
 C        -0.72118         4.40208         5.61888
 C        -1.75997         3.05203         3.91958
 C        -2.84032         2.05031        -0.74927
 H        -3.27783         2.89114        -0.81359
 C        -1.00782         0.01799         1.75767
 C        -1.05861         0.90413         2.95481
 O         1.35371        -5.58640        -6.85427
 O        -2.59090        -0.58861        -4.04431
 O         2.25237        -4.21110        -5.32133
 H         2.97602        -4.51653        -5.61897
 O        -4.82975        -4.46887        -5.93644
 O        -3.57562        -5.88393        -7.13220
 O        -0.68034        -3.81515        -1.46904
 N        -1.55764        -2.17438        -2.78422
 C         1.24375        -4.77942        -5.95060
 C        -1.00776        -1.70926        -0.43194
 C        -0.08853        -4.33064        -5.43644
 C        -0.19792        -3.45619        -4.35610
 H         0.58106        -3.13396        -3.91816
 C        -1.23610        -4.81015        -6.06756
 H        -1.15811        -5.40997        -6.80016
 C        -2.84456         1.17952        -1.84931
 H        -3.28903         1.42888        -2.65115
 C        -3.73377        -4.96962        -6.28978
 C        -1.60578        -0.44198        -0.56245
 C        -1.06074        -2.66267        -1.56645
 C        -2.60153        -3.52615        -4.55247
 H        -3.45676        -3.24300        -4.25093
 C        -2.49487        -4.41255        -5.62773
 C        -1.45608        -3.06250        -3.92843
 C        -0.37574        -2.06078         0.74043
 H         0.06177        -2.90161         0.80474
 C        -2.20824        -0.02847        -1.76652
 C        -2.15745        -0.91460        -2.96366
 C         3.27745         0.60639         1.28244
 C         1.63765         2.31173         0.58915
 C         2.38290         1.16140         0.29967
 C         5.61283        -2.27913         1.63255
 H         6.18234        -2.66644         2.28662
 C         4.90123        -1.09767         1.93911
 C         4.30476         0.67055         3.48436
 H         4.40698         1.08396         4.33368
 C         4.03847        -0.55120         0.96696
 C         3.40574         1.23845         2.53545
 C         5.01806        -0.44686         3.20089
 H         5.60688        -0.80274         3.85619
 C         1.78420         2.90694         1.87786
 H         1.27712         3.68155         2.09079
 C         2.62869         2.39440         2.80477
 H         2.70144         2.81731         3.65222
 C         5.49093        -2.87296         0.40632
 H         5.97564        -3.66970         0.22457
 C         3.01709        -0.61686        -1.29129
 C         4.65689        -2.32220        -0.59800
 C         3.91164        -1.17187        -0.30852
 C         0.68172         2.26866        -1.64140
 H         0.11220         2.65597        -2.29547
 C         1.39331         1.08719        -1.94796
 C         1.98979        -0.68102        -3.49321
 H         1.88757        -1.09443        -4.34253
 C         2.25608         0.54073        -0.97581
 C         2.88881        -1.24892        -2.54430
 C         1.27648         0.43639        -3.20974
 H         0.68766         0.79227        -3.86504
 C         4.51034        -2.91741        -1.88671
 H         5.01743        -3.69202        -2.09964
 C         3.66585        -2.40487        -2.81361
 H         3.59311        -2.82778        -3.66107
 C         0.80362         2.86249        -0.41517
 H         0.31891         3.65923        -0.23342
 H        -0.50206         6.18327         7.32232
 H        -4.84767        -3.78527        -5.29586
$end

$rem
METHOD wb97xd
hirshiter true
BASIS 6-311++g**
job_type sp
symmetry false
sym_ignore true
scf_convergence 10
solvent_method pcm
mem_total 30000
mem_static 10000
$end

$pcm
theory iefpcm
$end

$solvent
dielectric 6.3
$end                                  

And the last three lines in the error information:

/opt/soft/q-chem-v6.0/bin/qchem: line 126: 264343 Segmentation fault      (core dumped) ${QCPROG_S} ${inp} ${scr}
Error in Q-Chem run part 1
Error in the Q-Chem run

Please let me know how to fix this problem. Thank you very much.

I vaguely recall that Hirshfeld charges may not be very memory efficient. What happens if you try
(1) Setting MEM_TOTAL equal to the physical memory on the machine? (I don’t think that MEM_STATIC needs to be so large, by the way.)
(2) Setting XC_GRID = 1?

I wonder if one of your single-atom fragment jobs crashed, or was that part successfully done?

I have tried setting MEM_TOTAL=180000 and XC_GRID=1. The same segmentation fault shows up again. The last line of the output file:

Performing Hirshfeld-I population analysis.
================ Generating on-the-fly SAD Guess ================
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg1/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg2/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg3/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg4/
 Creating input file /tmp/qchem-182471/qchem253255/Frg1.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 0
 Done Job For Fragment 0
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg2.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 1
 Done Job For Fragment 1
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg3.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 2
 Done Job For Fragment 2
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg4.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 3
 Done Job For Fragment 3
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Time for running child jobs:  CPU 0.03 s  wall 8.20 s
 ============= Method Specific SAD Guess Constructed =============
 Guess from on-the-fly superposition of atomic densities
 Warning: Energy on first SCF cycle will be non-variational

 ---------- Iterative Hirshfeld Requested, Building Reference State ----------
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg1/
 Failed to make child job directory:     /tmp/qchem-182471/qchem253255//Frg1/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg2/
 Failed to make child job directory:     /tmp/qchem-182471/qchem253255//Frg2/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg3/
 Failed to make child job directory:     /tmp/qchem-182471/qchem253255//Frg3/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg4/
 Failed to make child job directory:     /tmp/qchem-182471/qchem253255//Frg4/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg5/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg6/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg7/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg8/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg9/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg10/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg11/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg12/
 Attempting to make child job directory: /tmp/qchem-182471/qchem253255//Frg13/
 Creating input file /tmp/qchem-182471/qchem253255/Frg1.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 0
 Done Job For Fragment 0
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg2.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 1
 Done Job For Fragment 1
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg3.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 2
 Done Job For Fragment 2
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg4.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 3
 Done Job For Fragment 3
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg5.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 4
 Done Job For Fragment 4
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg6.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 5
 Done Job For Fragment 5
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg7.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 6
 Done Job For Fragment 6
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg8.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 7
 Done Job For Fragment 7
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg9.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 8
 Done Job For Fragment 8
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg10.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 9
 Done Job For Fragment 9
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg11.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 10
 Done Job For Fragment 10
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg12.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 11
 Done Job For Fragment 11
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Creating input file /tmp/qchem-182471/qchem253255/Frg13.input
 using default PutDataFrgm: nothing
 calling putenv with: QCSCRATCH=/tmp/qchem-182471/qchem253255/
 Spawning Job For Fragment 12
 Done Job For Fragment 12
 calling putenv with: QCSCRATCH=/tmp/qchem-182471
 calling putenv with: QCTHREADS=40
 Time for running child jobs:  CPU 0.13 s  wall 36.88 s
 ------------------ Isolated Atomic Densities Converged ------------------

It is saying ‘Isolated Atomic Densities Converged’, so I guess everything goes well in this part.

Does your job begin the Hirshfeld analysis? In poking around with this, I am reminded that Hirshfeld is very slow (I don’t think it’s well parallelized, if at all), so after 4 hours on 40 cores your job did not finish for me but the Hirshfeld analysis was running when it ran out of time. With the SG-1 grid this is clear because it prints warnings about the grid quality:

Performing Hirshfeld-I population analysis.
 warning:  vdwdata pointer should have been initialzed before dftman
 Inaccurate integrated density:
   Number of electrons =   460
   Numerical integral  =   461.650401223656
   Relative error      =     0.3587828747 %

This is with the latest version of Q-Chem, v. 6.2. I will run the full job requesting more time.

One other note, why are you using SCF_CONVERGENCE=10? That’s excessive, usually 8 is sufficient and for complicated electronic structure (which this system isn’t), my experience is that it’s not always possible to converge reliably to 10^(-10).

  1. The running time of my job on 40 cores is about 2 hours, so I guess the job stops as soon as Hirshfeld analysis begins.
  2. Ok, I will change the convergence condition to 8. Thank you for your advice.

Actually the Q-Chem defaults for SCF_CONVERGENCE are usually fine (=5 for single-point energies only, =8 for anything that needs the density matrix, like a geometry optimization or excited-state calculation). Will save you a lot of time. I edited the output above to show that the grid warnings (using SG-1) appear just below the message that Hirshfeld is beginning. I will run mine through to make sure it doesn’t segfault but I’m not sure what your issue is. What version of Q-Chem are you running?

I am using Q-Chem 6.0.2.

With v. 6.2 this job seems to run but is extremely slow. I don’t think the Hirshfeld part is parallelized. I have opened a ticket to report this.