[nicoloangileri]

Follow-up to #2908. After it was merged, HestonModelTests/testAnalyticVsBlack took 21.8 s in the "Check test times" job, over the 20 s limit.

The time goes into FdmHestonVarianceMesher, not into the distribution as such. With a small vol of vol (sigma = 1e-4 in testAnalyticVsBlack, 0.001 or 0.005 in several other Heston tests), the degrees of freedom 4·theta·kappa/sigma² are between about 1e5 and 1e8. Before #2908 the CDF underflowed there, the inverse failed and the mesher went to the default mesh at once. Now the CDF converges, so the mesher computes all the quantiles of the grid: each inverse takes 15 to 46 CDF evaluations, since its bracket starts at df+ncp and doubles, which is very wide compared with the width of the distribution, and each evaluation costs up to about 1 ms at these sizes. In testAnalyticVsBlack the distribution is so narrow that the grid is degenerate in the end, and the mesher uses the default mesh anyway after computing all of it.

Two commits:

• "Skip the variance quantiles beyond 1-epsilon in the Heston mesher": once p reaches 1-epsilon, the next target probability can only be lower and its quantile is below the current point, so the point is the previous one plus the minimum step whatever the quantile is. The inverse is no longer computed in that case; the grids are the same.
• "Use the default Heston variance mesh for very narrow distributions": a normal approximation of the quantiles (mean df+ncp, standard deviation sqrt(2(df+2ncp))) checks, before any inverse is computed, whether the (size-1) minimum steps are wider than the range between the quantiles at every averaging step. In that case most points would end up beyond the 1-epsilon quantile, and the mesher now uses the default mesh, as it did before Avoid underflow in the non-central chi-square distribution #2908.

I checked both on a sweep of 3024 parameter sets (sigma from 1e-4 to 2, kappa from 0.1 to 5, v0 from 0.01 to 0.2, maturities from 1e-5 to 5 years, size 3, 10 and 100, tAvgSteps 5 and 20):

• with the first commit, all meshes and volatility estimates are bitwise identical to master, and the sweep takes 3.6 minutes instead of 19 (1.3 minutes with both commits);
• the second commit changes 164 cases, all of which used the default mesh before Avoid underflow in the non-central chi-square distribution #2908 and now get exactly that mesh again. The grids master builds there are of the kind [0, v0, v0+δ], with volatility estimates up to 67% away from the ones before Avoid underflow in the non-central chi-square distribution #2908. In testFdmHestonBlackScholes, for instance, master builds [0, 0.0625, 0.0629] with a volatility estimate of 0.2085, where the default mesh gives 0.25, which is the volatility of the test. No grid that was built before Avoid underflow in the non-central chi-square distribution #2908 changes, and the other 1094 grids that Avoid underflow in the non-central chi-square distribution #2908 made possible stay as they are.

Test times on my machine (Release, Apple clang, seconds):

test	before #2908	master	first commit	both
HestonModelTests/testAnalyticVsBlack	0.34	5.70	0.56	0.34
HestonModelTests/testFdBarrierVsCached	2.03	3.73	2.73	2.16
FdHestonTests/testFdmHestonBlackScholes	0.35	1.43	1.40	0.35
FdHestonTests/testFdmHestonBarrierVsBlackScholes	0.66	1.09	1.04	1.06
HestonModelTests/testFdVanillaWithDividendsVsCached	0.69	0.91	0.84	0.88
HestonModelTests/testFdAmerican	0.70	0.94	0.86	0.90

The last three are still slower than before #2908 because there the mesher builds proper quantile grids that it couldn't build when the CDF underflowed (one grid of 100 points in testFdAmerican and in testFdVanillaWithDividendsVsCached, 144 grids of 3 and 10 points in testFdmHestonBarrierVsBlackScholes). Getting those back to the old times would need a faster inverse or CDF for large arguments.

I also looked at boost::math::non_central_chi_squared, but it doesn't change the picture: at df = 1e8 its CDF is 1.3 to 1.8 times faster than ours and its quantile about 3.6 times faster, while at small arguments (df = 10, ncp = 50) ours is about 3 times faster; and at df = 1e8 the two CDFs differ by about 2.6e-7. So I left the distribution as it is.

The full test suite passes locally, also with QL_HIGH_RESOLUTION_DATE=ON.

[coveralls]

coverage: 79.073% (+0.001%) from 79.072% — nicoloangileri:speed-up-heston-variance-mesher into lballabio:master