Hello Q-Chem Community,
I am working on some high-energy collision simulations with Prof. Krylov, and she suggested I reach out here for tips on how to most efficiently simulate these systems. Apologies in advance for the length, but this touches on a number of different issues.
The initial setup is an incident ion traveling at high relative velocity (order of km/s) toward a target cluster under the influence of a high external electric field (order of V/nm). Both have been independently equilibrated and I’m confident in that setup.
Interestingly, most of the simulations seem to perform fine through the initial impact and associated bond-breaking events, but some of them fail SCF convergence later in the trajectory. For reference, here is the last geometry of one such trajectory:
I was going to include a plot of V(elec) vs. time, but as a new user I’m limited to one embedded image. The V(elec) peaks shortly after the collision onset and then displays oscillatory ringing behavior until the failed step.
It’s important to continue the trajectory long enough to determine the final fragmentation products and estimate the fragments’ individual center-of-mass velocity vectors for follow-up work. Also, note that the initial setup has nonzero charge and closed-shell species.
Here is a typical $rem section for the AIMD jobs. I’m also including the applied field section for reference:
$rem
JOBTYPE aimd
EXCHANGE wB97X-D
BASIS 6-31+G*
MEM_TOTAL 28000
MEM_STATIC 2000
SYMMETRY off
SYM_IGNORE true
AIMD_STEPS 1000
TIME_STEP 10
AIMD_PRINT 1
AIMD_INTEGRATION vverlet
SCF_MAX_CYCLES 200
THRESH 14
SCF_CONVERGENCE 8
SCF_ALGORITHM DIIS_GDM
THRESH_DIIS_SWITCH 4
FOCK_EXTRAP_ORDER 6
FOCK_EXTRAP_POINTS 12
$end
$multipole_field
Z 1.944690e-3
$end
As mentioned, I found that my setup with DIIS_GDM works well for these systems, and even for the difficult cases, the better converged the DIIS portion is, the better the GDM portion performs. Leveraging the Fock matrix extrapolation to construct the initial guess typically leads to DIIS reaching the 1e-4 threshold within 2 cycles prior to collision or 4-5 cycles after collision onset. I’ve also noticed that as the incident molecule approaches the cluster, the SCF time increases even for steps which use the same number of cycles, though I imagine this is due to the integral threshold?
When the SCF fails to converge with this setup, it does so in a relatively consistent manner. Here is the $molecule input for the failed time step pictured above:
$molecule
-1 1
C 1.7233212920680792 -5.8563622222002030 2.6048786747123778
C 2.4375397879196297 -1.0874228051560904 4.0718598933933032
N 2.1628857898648932 -5.4199695014953182 1.5360147878851433
N 3.2273781371073684 -0.0164398517616878 4.5508603287990637
C -2.4440665533422590 0.0938076307047873 4.3807062014564240
C 0.8222764330423260 -4.6654134289634470 1.2655345836184473
C -2.5260832990681146 5.5018194840497339 1.3008540234951256
C -1.9674860953278790 4.4610388975035642 0.2037391193483711
H 0.3600197760813975 -3.9023104945299907 1.9531865921891594
H 0.1479132583075785 -5.4586840620234316 0.7648475549804422
H 1.0557076420117912 -4.0689456350053330 0.2628422689404784
H -2.9344508232436293 -0.6827222814351923 3.4174123036637583
H -0.3134875250742652 -3.5833952051163229 5.5242226293273369
H 2.7560602083933312 -2.0122525889190683 4.5314335047718650
H -1.7032065327449961 6.1739214422997701 1.3739078204717476
H -2.4245445809894517 5.2223902474620436 2.2811378272165772
H -1.2051280630425107 4.6078192548812202 -0.5554919431957626
H -1.5828310344967087 3.7843055433666857 0.9835582256715187
H -2.9211615304447687 3.7693160889112511 0.0220553094153600
B -1.0749647377131921 -0.3671176989324121 4.6588148350359830
F 4.9290700848318192 2.1087473889413122 7.4972414237638052
F -0.5359710362822977 -0.8478983513127627 6.0147436240049270
F -0.1956505658557586 -4.2169237517430647 4.5794648394871178
F -3.0604364719123045 1.3217485369889355 4.5842459293610789
B 0.9411896695308501 -0.6903418526930895 3.9644195024876741
F -3.7219260244529622 6.4330459249696021 1.6331972145068072
F 7.8460070137447993 0.1339441223649654 1.4999100072509646
F -8.6811709239861301 -1.7885950419161010 2.5996066454476798
F 0.3061884474334090 0.1711794773968840 2.5867000898032018
$end
The SCF output for the failed time step in that AIMD job followed:
A restricted SCF calculation will be
performed using DIIS, GDM
SCF converges when RMS gradient is below 1.0e-08
---------------------------------------
Cycle Energy DIIS error
---------------------------------------
1 -1191.9799575958 5.04e-04
2 -1191.9803084320 1.62e-04
3 -1191.9766375361 4.93e-04
4 -1191.9797060029 2.61e-04
5 -1191.9797198145 2.58e-04
6 -1191.9808492254 3.75e-05 Done DIIS. Switching to GDM
---------------------------------------
Cycle Energy RMS Gradient
---------------------------------------
1 -1191.9808492254 6.98e-03 Normal BFGS step
2 -1191.9801442181 5.92e-02 Line search: overstep
3 -1191.9808681903 2.63e-03 Normal BFGS step
4 -1191.9808437580 1.01e-02 Line search: overstep
5 -1191.9808760083 2.64e-03 Normal BFGS step
6 -1191.9808693666 7.88e-03 Line search: overstep
7 -1191.9808828670 2.61e-03 Normal BFGS step
8 -1191.9808606451 1.31e-02 Line search: overstep
9 -1191.9808905269 2.03e-03 Normal BFGS step
10 -1191.9808760909 6.23e-03 Line search: overstep
11 -1191.9808960377 1.83e-03 Normal BFGS step
12 -1191.9809008387 1.14e-03 Normal BFGS step
13 -1191.9809057086 4.98e-04 Normal BFGS step
14 -1191.9809074041 2.36e-04 Normal BFGS step
15 -1191.9809078349 9.32e-05 Normal BFGS step
16 -1191.9809079403 5.88e-05 Normal BFGS step
17 -1191.9809079976 3.70e-05 Normal BFGS step
18 -1191.9809080182 3.65e-05 Normal BFGS step
19 -1191.9809080350 3.58e-05 Normal BFGS step
20 -1191.9809080523 3.49e-05 Normal BFGS step
21 -1191.9809080815 3.35e-05 Normal BFGS step
22 -1191.9809081790 8.24e-05 Normal BFGS step
23 -1191.9809082751 8.39e-05 Normal BFGS step
24 -1191.9809084024 9.23e-05 Normal BFGS step
25 -1191.9809085061 7.61e-05 Normal BFGS step
26 -1191.9809085028 4.10e-04 Line search: overstep
27 -1191.9809085221 1.45e-04 Normal BFGS step
28 -1191.9809084645 5.19e-04 Line search: overstep
29 -1191.9809085293 5.64e-05 Normal BFGS step
30 -1191.9809084993 4.90e-04 Line search: overstep
31 -1191.9809085333 4.25e-05 Normal BFGS step
32 -1191.9809085148 1.75e-04 Line search: overstep
33 -1191.9809085375 1.75e-05 Normal BFGS step
34 -1191.9809085374 7.52e-05 Normal BFGS step
35 -1191.9809085398 4.09e-05 Normal BFGS step
36 -1191.9809085406 3.26e-05 Normal BFGS step
37 -1191.9809085418 1.24e-05 Normal BFGS step
38 -1191.9809085425 8.54e-06 Normal BFGS step
39 -1191.9809085435 8.52e-06 Normal BFGS step
40 -1191.9809085459 1.16e-05 Normal BFGS step
41 -1191.9809085514 1.97e-05 Normal BFGS step
42 -1191.9809085637 2.96e-05 Normal BFGS step
43 -1191.9809085882 3.70e-05 Normal BFGS step
44 -1191.9809086148 4.04e-05 Normal BFGS step
45 -1191.9809086304 2.68e-05 Normal BFGS step
46 -1191.9809086359 2.89e-05 Normal BFGS step
47 -1191.9809086379 1.72e-05 Normal BFGS step
48 -1191.9809086383 2.86e-05 Normal BFGS step
49 -1191.9809086379 6.87e-05 Normal BFGS step
50 -1191.9809086379 4.85e-05 Normal BFGS step
51 -1191.9809086378 5.81e-05 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
52 -1191.9809086364 5.16e-05 Normal BFGS step
53 -1191.9809086386 3.05e-05 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
54 -1191.9809086368 4.67e-05 Normal BFGS step
55 -1191.9809086394 1.36e-05 Normal BFGS step
56 -1191.9809086391 2.43e-05 Normal BFGS step
57 -1191.9809086399 6.26e-06 Normal BFGS step
58 -1191.9809086401 6.02e-06 Normal BFGS step
59 -1191.9809086405 4.04e-06 Normal BFGS step
60 -1191.9809086419 5.98e-06 Normal BFGS step
61 -1191.9809086442 1.04e-05 Normal BFGS step
62 -1191.9809086481 1.45e-05 Normal BFGS step
63 -1191.9809086528 2.00e-05 Normal BFGS step
64 -1191.9809086566 1.65e-05 Normal BFGS step
65 -1191.9809086586 1.12e-05 Normal BFGS step
66 -1191.9809086600 1.94e-05 Normal BFGS step
67 -1191.9809086595 9.35e-05 Normal BFGS step
68 -1191.9809086605 4.89e-05 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
69 -1191.9809086550 1.63e-04 Normal BFGS step
70 -1191.9809086611 1.33e-05 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
71 -1191.9809086592 5.17e-05 Normal BFGS step
72 -1191.9809086614 7.31e-06 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
73 -1191.9809086609 2.15e-05 Normal BFGS step
74 -1191.9809086616 6.11e-06 Normal BFGS step
75 -1191.9809086614 8.90e-06 Normal BFGS step
76 -1191.9809086618 3.82e-06 Normal BFGS step
77 -1191.9809086619 6.85e-06 Normal BFGS step
78 -1191.9809086623 3.72e-06 Normal BFGS step
79 -1191.9809086640 1.72e-05 Normal BFGS step
80 -1191.9809086662 3.01e-05 Normal BFGS step
81 -1191.9809086686 2.47e-05 Normal BFGS step
82 -1191.9809086698 8.45e-06 Normal BFGS step
83 -1191.9809086702 4.66e-06 Normal BFGS step
84 -1191.9809086703 4.52e-06 Normal BFGS step
85 -1191.9809086704 5.33e-06 Normal BFGS step
86 -1191.9809086704 2.49e-05 Normal BFGS step
87 -1191.9809086705 1.02e-05 Normal BFGS step
88 -1191.9809086708 1.50e-05 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
89 -1191.9809086701 4.80e-05 Normal BFGS step
90 -1191.9809086710 3.83e-06 Normal BFGS step
91 -1191.9809086710 1.58e-05 Normal BFGS step
92 -1191.9809086714 4.62e-06 Normal BFGS step
93 -1191.9809086716 1.16e-05 Normal BFGS step
94 -1191.9809086717 1.98e-05 Normal BFGS step
95 -1191.9809086719 4.93e-06 Normal BFGS step
96 -1191.9809086720 2.41e-06 Normal BFGS step
97 -1191.9809086720 1.80e-06 Normal BFGS step
98 -1191.9809086720 8.00e-07 Normal BFGS step
99 -1191.9809086720 4.13e-07 Normal BFGS step
100 -1191.9809086720 4.68e-07 Normal BFGS step
101 -1191.9809086720 4.63e-07 Normal BFGS step
102 -1191.9809086720 3.80e-07 Normal BFGS step
103 -1191.9809086720 9.67e-07 Normal BFGS step
104 -1191.9809086720 5.18e-06 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 9.75e-09
105 -1191.9809086720 1.34e-05 Normal BFGS step
106 -1191.9809086721 2.42e-06 Normal BFGS step
107 -1191.9809086721 1.77e-06 Normal BFGS step
108 -1191.9809086721 1.50e-05 Normal BFGS step
109 -1191.9809086721 5.64e-06 Normal BFGS step
110 -1191.9809086721 2.17e-06 Normal BFGS step
111 -1191.9809086721 1.60e-06 Normal BFGS step
112 -1191.9809086721 1.34e-06 Normal BFGS step
113 -1191.9809086721 8.18e-07 Normal BFGS step
114 -1191.9809086721 1.50e-06 Normal BFGS step
115 -1191.9809086721 1.21e-06 Normal BFGS step
116 -1191.9809086721 2.61e-07 Normal BFGS step
117 -1191.9809086721 2.59e-07 Normal BFGS step
118 -1191.9809086721 3.18e-07 Normal BFGS step
119 -1191.9809086721 4.28e-07 Normal BFGS step
120 -1191.9809086721 3.06e-07 Normal BFGS step
121 -1191.9809086721 8.32e-07 Normal BFGS step
122 -1191.9809086721 1.80e-06 Normal BFGS step
123 -1191.9809086721 4.17e-06 Normal BFGS step
124 -1191.9809086721 1.14e-06 Normal BFGS step
125 -1191.9809086721 1.45e-06 Normal BFGS step
126 -1191.9809086722 2.30e-06 Normal BFGS step
127 -1191.9809086722 7.85e-06 Normal BFGS step
128 -1191.9809086722 4.14e-06 Normal BFGS step
129 -1191.9809086722 2.17e-06 Normal BFGS step
130 -1191.9809086722 8.81e-07 Normal BFGS step
131 -1191.9809086722 3.07e-06 Normal BFGS step
132 -1191.9809086722 2.92e-06 Normal BFGS step
133 -1191.9809086722 2.48e-06 Normal BFGS step
134 -1191.9809086722 2.23e-06 Normal BFGS step
135 -1191.9809086722 1.07e-06 Normal BFGS step
136 -1191.9809086722 1.54e-06 Normal BFGS step
137 -1191.9809086722 7.58e-07 Normal BFGS step
138 -1191.9809086722 8.43e-07 Normal BFGS step
139 -1191.9809086722 9.86e-07 Normal BFGS step
140 -1191.9809086722 1.16e-06 Normal BFGS step
141 -1191.9809086722 1.90e-06 Normal BFGS step
142 -1191.9809086722 4.78e-06 Normal BFGS step
143 -1191.9809086722 3.89e-07 Normal BFGS step
144 -1191.9809086722 1.16e-06 Normal BFGS step
145 -1191.9809086722 5.16e-07 Normal BFGS step
146 -1191.9809086722 5.41e-07 Normal BFGS step
147 -1191.9809086722 3.98e-07 Normal BFGS step
148 -1191.9809086722 1.26e-06 Normal BFGS step
149 -1191.9809086722 1.44e-07 Normal BFGS step
150 -1191.9809086722 1.55e-07 Normal BFGS step
151 -1191.9809086722 2.05e-07 Normal BFGS step
152 -1191.9809086722 5.14e-07 Normal BFGS step
153 -1191.9809086722 6.03e-07 Normal BFGS step
154 -1191.9809086722 4.93e-07 Normal BFGS step
155 -1191.9809086722 4.39e-07 Normal BFGS step
156 -1191.9809086722 1.94e-07 Normal BFGS step
157 -1191.9809086722 1.92e-07 Normal BFGS step
158 -1191.9809086722 2.07e-07 Normal BFGS step
159 -1191.9809086722 2.28e-07 Normal BFGS step
160 -1191.9809086723 7.94e-07 Normal BFGS step
161 -1191.9809086722 1.68e-06 Normal BFGS step
162 -1191.9809086722 4.99e-07 Normal BFGS step
163 -1191.9809086723 7.09e-07 Normal BFGS step
164 -1191.9809086722 4.91e-07 Normal BFGS step
165 -1191.9809086722 5.16e-07 Normal BFGS step
166 -1191.9809086722 3.21e-07 Normal BFGS step
167 -1191.9809086723 1.25e-06 Normal BFGS step
168 -1191.9809086722 1.65e-06 Normal BFGS step
169 -1191.9809086723 3.07e-07 Normal BFGS step
170 -1191.9809086723 1.17e-07 Normal BFGS step
171 -1191.9809086723 1.85e-07 Normal BFGS step
172 -1191.9809086723 3.24e-07 Normal BFGS step
173 -1191.9809086723 4.46e-07 Normal BFGS step
174 -1191.9809086723 1.01e-07 Normal BFGS step
175 -1191.9809086723 6.55e-08 Normal BFGS step
176 -1191.9809086723 1.23e-07 Normal BFGS step
177 -1191.9809086723 7.08e-08 Normal BFGS step
178 -1191.9809086723 1.49e-07 Normal BFGS step
179 -1191.9809086723 9.53e-08 Normal BFGS step
180 -1191.9809086723 1.63e-07 Normal BFGS step
181 -1191.9809086723 3.03e-07 Normal BFGS step
182 -1191.9809086723 4.58e-08 Normal BFGS step
183 -1191.9809086723 1.13e-07 Normal BFGS step
184 -1191.9809086723 7.36e-08 Normal BFGS step
185 -1191.9809086723 7.47e-08 Normal BFGS step
186 -1191.9809086723 1.43e-07 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 1.27e-08
187 -1191.9809086723 3.45e-07 Normal BFGS step
188 -1191.9809086723 4.32e-07 Normal BFGS step
189 -1191.9809086723 4.92e-08 Normal BFGS step
190 -1191.9809086723 4.01e-08 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 1.27e-08
191 -1191.9809086723 2.50e-08 Normal BFGS step
192 -1191.9809086723 1.94e-08 Normal BFGS step
193 -1191.9809086723 1.96e-08 Normal BFGS step
194 -1191.9809086723 8.12e-08 Normal BFGS step
195 -1191.9809086723 4.73e-08 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 1.27e-08
196 -1191.9809086723 3.04e-08 Normal BFGS step
197 -1191.9809086723 3.23e-08 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 1.27e-08
198 -1191.9809086723 6.26e-08 Normal BFGS step
199 -1191.9809086723 3.33e-08 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 1.27e-08
200 -1191.9809086723 2.85e-08 Normal BFGS step
gen_scfman_exception: SCF failed to converge
The GDM warning from the output is typical for failed simulations, always first appearing at an RMS gradient on the order of 1e-05 (even for expensive time steps which do wind up converging the SCF before reaching the max cycles). The effective_threshold it provides is always on the order of 1e-09 for any cycle containing that warning. This job showed that the energy was practically unchanging near the end – and it likely would have reached convergence within another 100 cycles – but this wasn’t always the case for other jobs. For example, another failed job (different species) had the following SCF output:
...
60 -1455.8860708558 1.47e-05 Normal BFGS step
61 -1455.8860708560 1.79e-05 Normal BFGS step
62 -1455.8860708564 1.21e-05 Normal BFGS step
GDM::WARNING energy changes are now smaller than effective accuracy.
GDM:: calculation will continue, but THRESH should be increased
GDM:: or SCF_CONVERGENCE decreased.
GDM:: effective_thresh = 7.30e-09
63 -1455.8860708550 1.06e-04 Normal BFGS step
64 -1455.8860708566 1.29e-05 Normal BFGS step
65 -1455.8860708566 1.44e-05 Normal BFGS step
...
195 -1455.8860747263 4.96e-04 Normal BFGS step
196 -1455.8860748652 1.15e-03 Normal BFGS step
197 -1455.8860749210 5.48e-04 Normal BFGS step
198 -1455.8860754361 8.07e-04 Normal BFGS step
199 -1455.8860757413 1.11e-04 Normal BFGS step
200 -1455.8860759923 1.02e-03 Normal BFGS step
gen_scfman_exception: SCF failed to converge
Prof. Krylov suggested the issue might be due to a very small energy gap between occupied and virtual orbitals. Are there any general techniques to improve performance on these kinds of systems? I should also note that although some of the fragmented species are likely open-shell, it doesn’t make sense to us to do unrestricted SCF since there’s no guarantee that any step will identify the same global minimum state. For these high-energy systems, we’re comfortable with whatever small energy error may be introduced by requiring restricted SCF. We’re also looking into whether dropping the diffuse orbitals will ease convergence with a smaller basis set without significantly affecting the results.
My natural instinct for the failed jobs is to try different initial guesses. Indeed, many of them are able to trudge along slowly but successfully upon a naive restart of the scratch MOs from the previous step. I believe these restarts do not read in any history for the Fock matrix extrapolation. This leads me to question whether maybe extrapolation after the “peak” collision period is hurting rather than helping? Is it possible that the system retains memory from a vastly different chemical environment as the atoms were slamming into one another, and “resetting” the extrapolation would resolve it?
I have tried other initial guesses, typically without success if reading the MOs didn’t work. Because some fragments are charged species (e.g. H+ or F-) and SAD uses the density of neutral atoms, it may not be the most appropriate. Trying SAP has also shown little success. Do you have any other recommendations or general strategies for these kinds of jobs to balance performance/efficiency/robustness?
Thanks!