SCF convergence issue with diffuse functions on basis set

Questions
#1

Hello, I am trying to run a sp calculation for a large system with def2-tzvpd but I am struggling to convert the calculation. When I run with Def2-TZVP it converges easily with ~4,5 scf cycles but when I add the diffusion it just give me really noisy and weird results and do not converge. I already tried the BASIS2 (basis set projection) but It gives me the same problem (gonna put an input example below). Do you know what I can try to change? (I already tried without the LEVEL SHIFT, without the SCF_GUESS flag, without S2THRESH as well but nothing worked)
Input:`$molecule
0 1
H 8.929130 15.324930 3.187710
H 8.930730 15.331990 10.130110
O 17.959280 14.394630 15.997290
H 21.131970 9.313330 1.398210
C 20.000300 11.332980 15.970020
H 21.459350 7.601420 12.446460
Mg 16.619130 14.509750 3.653030
Mg 16.618420 14.511030 10.595030
C 20.088090 10.283970 -2.621040
H 21.461050 7.598710 -1.437680
Mg 18.325420 14.668470 5.967170
Mg 18.324720 14.669750 12.909170
H 10.614310 14.362670 -0.895210
H 18.960930 11.288590 -1.072700
C 20.482050 9.215110 19.150700
H 10.582310 14.756880 12.965960
H 21.459430 7.599840 5.504170
O 17.757260 15.887300 -2.515010
C 20.002410 11.329130 -4.855980
H 10.580620 14.753470 6.025000
Mg 17.335540 16.066380 1.338820
Mg 17.334840 16.067660 8.280820
H 20.931130 21.840000 5.336660
C 19.388210 19.185700 -2.543490
C 20.492560 10.285190 -4.027420
C 19.245310 11.312980 18.715550
O 17.756550 15.888580 4.426990
O 17.755850 15.889860 11.368990
O 9.578690 15.153940 2.487820
O 9.577990 15.155220 9.429810
C 18.920740 17.945250 -2.032450
H 20.450280 8.404490 20.840910
H 20.279180 11.363710 -5.897830
Mg 18.326130 14.667190 -0.974830
H 21.853870 22.311420 7.843540
H 21.871160 22.308660 14.776590
O 16.562620 14.964390 6.741040
O 16.561910 14.965660 13.683040
O 21.287740 8.248760 4.804280
O 21.287030 8.250050 11.746280
O 21.414010 21.841250 0.173780
O 21.413310 21.842530 7.115780
C 18.755160 12.356920 17.886980
O 10.999570 15.740930 18.106100
H 18.971680 11.269820 19.757300
C 18.765680 13.427000 -5.291130
O 17.960690 14.392070 2.113290
O 17.959980 14.393350 9.055290
H 19.228620 19.427410 -3.579580
C 20.490450 10.289030 16.798580
O 17.962110 17.412180 2.692480
O 17.961400 17.413460 9.634480
O 11.000970 15.738360 4.222100
O 11.000270 15.739640 11.164100
O 21.288440 8.247480 -2.137720
C 10.813940 15.371230 16.834580
O 17.926630 13.308600 -2.376490
O 19.178630 13.450630 -6.562670
O 15.641330 15.537330 16.134870
O 15.140340 14.379960 5.006750
O 15.139640 14.381240 11.948750
O 20.070500 9.188920 6.538230
O 20.069800 9.190200 13.480230
O 21.208440 20.317640 1.908290
O 21.207730 20.318920 8.850290
C 19.247420 11.309130 -2.110450
O 17.962810 17.410900 -4.249520
H 18.797350 14.228450 -6.996000
O 20.069090 9.191480 20.422230
O 19.177920 13.451910 0.379330
O 19.177220 13.453190 7.321330
O 20.458690 21.330640 4.663750
O 20.457980 21.331930 11.605760
H 17.815730 13.113650 19.386820
H 20.258390 11.364410 14.924610
O 15.642740 15.534770 2.250870
O 15.642030 15.536050 9.192870
O 21.321100 9.333520 16.236050
O 21.323210 9.329670 -4.589950
C 18.189770 17.065550 -2.977860
O 9.577280 15.156500 16.371820
O 10.499280 14.582270 0.035990
O 10.498580 14.583560 6.977990
O 21.207020 20.320200 15.792300
C 13.249260 15.393270 16.400190
O 17.925920 13.309880 4.565510
O 17.925220 13.311160 11.507510
H 15.539480 15.703000 17.078460
O 18.711160 16.400460 6.879020
O 18.710460 16.401740 13.821020
C 11.941250 15.177710 15.889230
O 20.459400 21.329360 -2.278250
O 21.322500 9.330950 2.352050
O 21.321800 9.332240 9.294050
H 13.329290 15.654290 17.442990
O 17.924510 13.312440 18.449510
C 18.189070 17.066830 3.964140
C 18.188360 17.068110 10.906140
C 10.815350 15.368670 2.950580
C 10.814650 15.369950 9.892580
C 19.159630 12.358140 16.480610
H 20.935560 21.837750 12.276650
C 19.161740 12.354300 -4.345390
O 17.961390 14.390780 -4.828710
C 15.325960 14.749660 6.278270
C 15.325260 14.750940 13.220270
C 20.483450 9.212550 5.266700
C 20.482750 9.213830 12.208700
C 20.981490 20.662990 0.636640
C 20.980780 20.664280 7.578640
H 21.132120 9.317410 8.340280
C 20.085980 10.287810 18.204960
O 21.286330 8.251320 18.688280
C 18.764970 13.428280 1.650870
C 18.764270 13.429560 8.592870
C 18.920030 17.946530 4.909550
C 18.919320 17.947810 11.851550
C 11.942660 15.175150 2.005230
C 11.941960 15.176430 8.947230
C 14.199360 14.941910 0.281630
C 14.198650 14.943190 7.223620
C 20.087390 10.285250 4.320960
C 20.086690 10.286530 11.262960
C 20.249820 19.784580 6.633230
C 20.249110 19.785870 13.575230
C 19.161030 12.355580 2.596610
C 19.160330 12.356860 9.538610
C 20.046520 20.133730 5.226720
C 20.045820 20.135010 12.168730
C 14.400080 15.293060 1.688000
C 14.399370 15.294340 8.630000
C 11.741940 14.823990 0.598860
C 11.741240 14.825270 7.540860
C 18.756560 12.354360 4.002980
C 18.755860 12.355640 10.944980
C 19.123330 17.597380 6.316050
C 19.122620 17.598660 13.258060
C 20.491860 10.286470 2.914580
C 20.491160 10.287750 9.856580
C 19.387510 19.186980 4.398510
C 19.386800 19.188260 11.340510
C 13.250660 15.390710 2.516190
C 13.249960 15.391990 9.458180
C 12.890650 14.727620 6.712670
C 12.889940 14.728900 13.654670
C 19.246720 11.310420 4.831550
C 19.246020 11.311700 11.773550
C 19.783050 18.542850 0.202270
C 19.782340 18.544130 7.144270
C 20.001700 11.330420 2.086020
C 20.001000 11.331700 9.028020
H 19.242440 19.424830 3.358860
H 19.241170 19.427710 10.301180
H 13.377740 15.649240 3.556130
H 13.383180 15.651530 10.497170
H 12.783270 14.484460 5.668150
H 12.790830 14.490070 12.608540
H 18.957640 11.291780 5.871010
H 18.955340 11.290940 12.811430
H 19.942290 18.309580 1.244210
H 19.937180 18.308850 8.186510
H 20.261130 11.353650 1.039810
H 20.262380 11.358650 7.982490
O 21.412590 21.843820 14.057800
H 21.442370 9.538990 -5.522740
O 19.176510 13.454480 14.263330
H 21.134280 9.313610 15.280170
C 18.757270 12.353080 -2.939020
H 8.939870 15.325670 17.082610
O 10.497870 14.584840 13.919990
O 20.071200 9.187630 -0.403770
O 16.563320 14.963100 -0.200960
O 18.711870 16.399180 -0.062980
C 14.197950 14.944470 14.165620
C 20.250530 19.783300 -0.308780
C 12.891360 14.726340 -0.229330
C 15.326670 14.748370 -0.663730
C 19.781640 18.545410 14.086270
C 20.980070 20.665560 14.520650
C 11.740530 14.826560 14.482860
C 19.124030 17.596090 -0.625950
C 18.763560 13.430840 15.534870
C 20.484160 9.211270 -1.675300
C 14.398670 15.295620 15.572000
C 20.047230 20.132440 -1.715280
H 12.789980 14.468370 -1.271140
O 15.141050 14.378680 -1.935250
H 19.953600 18.316360 15.124070
H 21.858680 22.312100 0.897900
H 20.894760 21.862090 -1.597760
H 17.715230 15.015010 15.270820
H 16.034640 14.244070 -2.345400
H 17.617140 16.598930 -4.678420
O 17.030923 20.180604 7.674996
C 16.390923 19.250604 7.458996
O 15.750923 18.321604 7.241996
$end

$rem
JOBTYPE sp
METHOD HF
BASIS Def2-TZVPD
CHELPG TRUE
POP_MULLIKEN TRUE
MEM_TOTAL 100000
MBDVDW MBD_SP
MBDVDW_BETA 10000

MAX_SCF_CYCLES 200

SCF_CONVERGENCE 8

THRESH 14

GUI 2

BASIS2 def2-TZVP

SCF_GUESS CORE

S2THRESH 14

LEVEL_SHIFT TRUE

LSHIFT 500

MEM_STATIC 4096

$end

And the SCF cycles is looking like this:

Cycle Energy DIIS Error

1   23543.6375163963      6.40E-02
2   27046.9758839808      5.88E-02
3   10036.6916014008      5.20E-02
4   18612.4468479886      6.59E-02
5   11519.1274259245      6.45E-02
6    5100.6012273234      4.56E-02
7    2531.7699012155      3.46E-02
8   13060.9571608931      6.18E-02
9   -2386.2616358222      3.03E-02

10 3602.7734002637 4.18E-02
11 184.3086006999 3.08E-02
12 1845.7506287005 3.91E-02
13 -5540.6657684927 1.71E-02
14 8275.1396201772 4.50E-02
15 6736.8052764968 4.13E-02
16 6617.4984400721 4.07E-02
17 3612.3299893461 3.91E-02
18 7746.9681010764 4.67E-02
19 -1596.9264381589 2.68E-02
20 -2695.1503788261 3.27E-02
21 -6150.9475788403 2.09E-02
22 -2744.0773566149 2.75E-02
23 -5007.9259096748 2.06E-02
24 -8073.1359170381 9.63E-03
25 6802.3277443085 5.34E-02
26 5390.2176533551 4.96E-02
27 3816.1072316557 4.72E-02
28 4757.6331588731 4.85E-02
29 13176.5142077793 6.59E-02
30 12436.5863208156 6.30E-02
31 12423.1385126460 6.29E-02
32 12440.6486662224 6.31E-02
33 12469.1834072102 6.34E-02
34 12852.1243656530 6.39E-02
35 11941.3820602145 6.12E-02
36 8702.0762265380 5.69E-02
37 11943.1318666469 6.32E-02
38 7582.0776518703 5.22E-02
39 19023.0413736442 7.36E-02
40 35210.3669653650 8.22E-02
Thanks a lot for the attention

#2

This is almost certainly the result of setting SCF_GUESS = CORE. That’s a terrible guess, why would you use that one? SAD (which is the default) or AUTOSAD should work much better.

#4

Hello, Thanks for the reply.

I tried to use the default (SAD) but the job just crashed when tried to split into fragments. Than I tried the CORE because I was using ghost atoms. But I did not tried the AUTOSAD yet, I will do it.

Thanks for the guess.

#5

If you want to use fragments, try SCF_GUESS = FRAGMO. Other options include specifying a BASIS2 (i.e., using basis-set projection). All of these are better options then CORE, which has the advantage of always being available but the disadvantage of always being a terrible guess.

#6

I will try to use this options with the BASIS2 that I am using.

Thank you very much for the reply.

#7

I looked at this more carefully and I’m a bit surprised about the convergence problems. AUTOSAD guess is being used by default, which is typically fine. The issue may simply be the size and the number of linear dependencies, and you’re doing the right thing in terms of thresholds. I suggest setting BASIS2 = def2-TZVP (i.e., without the diffuse functions). You could also consider a minimally augmented version (def2-ma-TZVP) for your target BASIS, instead of def2-TZVPD. We designed that in this paper
https://doi.org/10.1021/acs.jctc.1c01302
and those diffuse functions are usually good enough, certainly for SCF applications.

#9

Thanks a lot for your reply.

Just to see if I understood properly. You suggest than to change the target basis set? (to def2-ma-TZVP for example). Because for the other suggestion that you made, I am already using BASIS2=def2-TZVP. Or should I consider doing something else?.

Thanks again for all the attention and help

#10

Edit: I don’t think that MBD hampers convergence because it’s not computed until near the end (i.e., MBD is not switched on until the DIIS error falls below a certain threshold). Might need to turn it off in $rem_frgm, however. See example below.

#11

Yeah I will test all the possibilities to see if I can get it right. But I also did with HF-R2SCAN (DC-DFT) and also had the same error so I don’t think is the MBD. But I will keep it update.

Thanks again

#12

The following input file converges in 7 SCF interations:

$rem
METHOD HF
BASIS Def2-ma-TZVP
MEM_TOTAL 100000
MBDVDW MBD_SP
MBDVDW_BETA 10000
thresh 14
s2thresh 16
MEM_STATIC 4096
$end

$rem_frgm
mbdvdw 0
$end
#13

Hi Thanks a lot for the answer. So I guess the problem is really the diffuse functions on the Def2-TZVPD right? I will still try to cover with Def2-TZVPD because I did some other calculations with this basis set to be consistent. But if I cannot I will use this minimal augmentation.

Thanks a lot for the effort and for the attention

#15

Hi, so I used your suggested input format:

$rem
JOBTYPE sp
METHOD HF
BASIS Def2-TZVPD
CHELPG TRUE
POP_MULLIKEN TRUE
MEM_TOTAL 100000
MBDVDW MBD_SP
MBDVDW_BETA 10000
thresh 14
s2thresh 16
MEM_STATIC 4096
$end

$rem_frgm
mbdvdw 0
$end

And now I am having reasonable scf cycles:

 ---------------------------------------
  Cycle       Energy         DIIS Error
 ---------------------------------------
    1   -9180.1568493429      7.06E-03
    2   -9133.0790524675      9.68E-04
    3   -9135.0756292203      7.70E-04
    4   -9137.0478627480      1.18E-04
    5   -9137.1141171755      5.29E-05

But it’s too slow the convergence (almost 20h of calculation and it only did 5 scf cycles with 256 cores). Do you know if it’s normal or if I can do something to be faster?

Thanks again.

#16

First, I doubt there’s any reason to use 256 cores. (Unless you have scaling data to demonstrate otherwise, I think Q-Chem’s parallel efficiency will saturate well before that.) My suggestion is to use def2-ma-TZVP.