Blog

Overfitting is a major concern that needs to be dealt with while development of trading models. This turbulence is largely observed in quantitative finance and algorithmic trading. This is a situation where a model formulates a conclusion based on randomness rather than most likely tracing the general and most logical pattern. Consequently, strong results are observed from backtesting but subsequent live trading renders changes into extreme futility.

What is Overfitting?

A model becomes over fitted when the number of its parameters unproportionately surpasses the number of data points available consequently making the model overly complicated. Backtest results become overwhelming yet the model proves to be incapable of adapting and dealing with novel data which would ultimately lead to heavy losses from real-time trading.

Example of Overfitting in Trading

For example, consider a trading strategy that attempts to forecast stock price movement by relying on previously charged stock prices. Should the model utilize patterns that are far too general such as a certain day’s price fluctuation, it may record huge gains in backward tests. However such trivial patterns are rarely replicated, thus the model becomes redundant during live trading.

Causes of Overfitting in Trading Models

Excessive Complexity
Using many indicators or features in a model can force it to memorize all the noise rather than focusing on structures worth replicating.

Insufficient Data
Overfitting can arise when there is scarce historical data, thus making the accuracy of the model heavily dependent on minute details.

Data Snooping Bias

Overfitting occurs when model retraining and testing is done on the same dataset repeatedly.

High Parameterization

More the parameters added, more the flexibility of the model increases but generalization decreases.

Inadequate Cross Validation

Widespread use of backtesting approaches often leads to a positive test bias even if the model does not work as intended.

Signs of Overfitting in Trading Models

Strikingly High Backtested Returns And Disappointing Results When Trading

In backtesting the model works superbly but in real time, it fails to achieve even the simplest of objectives.

Unrealistic Complexity

It is common for overfitting models to be complex, consisting of many rules or parameters.

Divergence in Metrics

The difference between the in-sample performance and out-of-sample performance is huge signifying a classical love tale’s stereotype, someone who is all about overfitting.

Strategies to Prevent Overfitting

Simplify the Model

Focus on fewer features and parameters rather than a million, so that you do not overfit. Furthermore, less complex models are often more generalizable and intuitive.

Increase Dataset Size

Include a wider scope of historical data in terms of years, quarters , months to make the model more robust.

Use Regularization

Lasso L1 and L2 (Ridge) regularization prevents over parameterization or fitting by including a penalty.

Cross Validation

Divide the data set into a training data set, a validation data set and a test data set, such that the K fold cross validation will further help maximize performance when testing.

Out-of-Sample Testing

Assess the model’s performance on data that it has never seen before during training in order to determine its real-life usefulness.

Avoid Data Snooping

Decrease the frequency of model experiments on the same dataset. If possible, always use a new dataset for validation.

Apply Robust Performance Metrics

Rather than appealing only to the raw returns, appeal to other measures of performance which include the Sharpe ratio, the Sortino ratio, the maximum drawdown, and so forth.

Tools and Techniques for Overfitting Prevention

Walk-Forward Testing

This is achieved by training and testing the model on successively forwards moving parts of the data set to mimic the real world.

Bootstrapping

Create multiple training and testing datasets by resampling the dataset to ensure that the model tests on many different sets.

Dropout in Neural Networks

During the training phase, certain neurons are excluded at random from the model in order to avoid dependence on particular patterns of the data.

Noise Injection

Use of noise on the training data restricts the model from being very sensitive to small changes.

Practical Case: Overfitting Reduction

Let’s take an example of a trader who has developed a ml model for stock market predictors and takes the backtesting establishing accuracy of 95% with machine learning model having 50 indicators. But the moment the model goes into live trading it starts making losses.

Measures Adopted to avoid Overfitting:

Feature Reduction: Cut down indicators from the existing number of 50 to 10 that have shown accuracy in terms of prediction.

Cross-Validation: Implement k-fold method so as to evaluate the model’s performance when trained in different subsets of the data.

Regularization: Invoke L1 regularization so as to get rid of redundant parameters.

Walk-Forward Testing: Implement the successful model across different time frames to test its consistency.

Result: returns of more test accuracy of 85 % were availed with the modified model but live trading proved to earn steady returns.

Striking the balance between Underfitting and Overfitting

Overshooting of over fitting usually leads to the heavy emphasis on the models fitted on historical data in turn giving rise to a new problem which is underfitting characterized by simple models that do not capture the right features. Therefore the aim is to find such a modell with the most adequate complexity .

Overfitting Consequences in Algorithmic Trading

Capital Loss

Overfitting has negative results on the live performance making it much worse than backtesting which increases likely hood of losses.

Trust Crisis

Maximum trust is placed in the model or the strategy but in most cases even after one defeat the trust is lost and further down the line many more such defeats cause extreme trust deficit.

Higher Development Costs

Expending resources on such overfitted models is not only a waste of time but leads to increased inefficiencies.

Conclusion

The fundamental problem with overfitting exists when building up a trading model, regardless of the specific focus being on algorithms or quantitative strategies. Certain procedures such as model parsimony, cross-validation or walk forward analysis have to be implemented in practice to minimize the risk of overfitting and thus improve the expected future performance of models. The important aspect is to reach a degree of complexity that will not hamper the model from the ability of generalization while still executing in different markets.

To avail our algo tools or for custom algo requirements, visit our parent site Bluechipalgos.com