Avoiding Boundary-Seeking Parameters in Heston Calibration

# Useful methods to avoid degenerate calibration? (Heston model in my case)


# Useful methods to avoid degenerate calibration? (Heston model in my case)












I have implemented a Levenberg-Marquardt(LM) based method to calibrate the Heston model against market data by minimizing a weighted $L^2$-norm of differences of market vs model prices. Pretty standard stuff. The calibration works pretty well and fits the market data as good as could be expected.

As a reference, the Heston model is given by the processes: $$ dS_t = \mu \cdot S_t dt + \sqrt{\nu_t} \cdot S_t dW^S_t$$ $$ d\nu_t = \kappa \cdot (\theta - \nu_t) dt + \xi \sqrt{\nu_t} dW^{\nu}_t $$ where $$ dW^S_t \cdot dW^{\nu}_t = \rho dt$$ and $v_t$ has initial value $= \nu_0.$ The Heston parameters are given by the tuple $(\kappa, \rho, \xi, \theta, \nu_0).$ The parameters of the model have some natural boundaries. The correlation, $\rho$, for example, should be in the interval $[-1, 1]$ and the rest of the parameters should be non-negative. My problem is that the LM minimizer sometimes end up hitting the boundary of the allowed parameter space and "wants" to go further in that direction. So the calibration then ends up with say $\rho = -1$ or some other parameter equal or close to zero. This might not be a huge problem when it comes to fitting the market data, but it will affect the quality of the stochastic behavior of the calibrated Heston model if used in Monte Carlo simulations for example. To make a simple example, let us say that the vol-vol, $\xi$, is calibrated to some value very close to or equal to zero. Then we practically have a deterministic volatility model, which probably will not be very useful. In this case we would probably have been more satisfied with a slightly worse fit to market data, but with no degenerate values of the model parameters. My question is to get some advice on methods to prevent degenerate model values in the calibration. A read a bit about penalty functions to prevent the minimizer to search too close to the boundary of the parameter space. Could this be a way to go? What forms of penalty functions are appropriate then? My worry here is that the penalty function might affect the calibration too much even when we are not close to the boundary?

This question is, of course, not specific to the Heston model. This seems to be a reoccurring problem in most model calibrations.