Before I give you my strongest argument(s) against backtesting, let me preface my answer by saying this is not one of those answers against backtesting. I believe backtesting serves as a valuable tool for evaluating and refining trading strategies. However, I also think it's crucial to understand its limitations and avoid relying solely on its results. It should be used in conjunction with other risk management strategies, fundamental analysis, and a deep understanding of market dynamics before deploying capital in real-world trading.
The Limitations of Backtesting Trading Strategies
Having said that, my first strongest argument against backtesting trading strategies is that past performance is not indicative of future results. This is a fundamental principle in finance and investing. Just because a strategy performed well in the past using historical data doesn't guarantee it will translate to success in the future. Market conditions, investor sentiment, and regulations can change drastically, rendering past performance irrelevant.
You can overcome this challenge by complementing backtesting with forward-testing in simulated or live trading environments, as well as ongoing monitoring and adjustment of strategies based on real-world market feedback.
Another limitation of backtesting is the issue of survivorship bias. Backtesting often overlooks survivorship bias, leading to an overestimation of performance. By focusing only on assets that have survived until the present day, backtests may paint a rosier picture of strategy performance than would be the case in a real-world scenario where failed assets are included.
You can overcome this limitation by using historical data that is survivorship bias-free. You can get this survivorship bias-free historical stock data from specialized providers like Algoseek. This data includes symbols that have been delisted due to bankruptcy, M&A, or any other reason, making it especially suitable for backtesting.
Another downside of backtesting s is that it typically does not account for transaction costs, slippage, bid-ask spreads, and other real-world trading expenses, which can significantly eat into profits. Ignoring these factors can lead to unrealistic expectations of profitability and may result in strategies that are not economically viable when implemented in live trading.
Finally, it’s just difficult to replicate real-world conditions: Backtesting often occurs in a simulated environment that doesn't account for the psychological aspects of real-world trading. Factors like emotions, fear of missing out (FOMO), greed, and the pressure of managing real capital can significantly impact decision-making and potentially lead to deviations from the backtested strategy.
Backtesting is a Tool, Not a Guarantee
While recognizing its limitations, backtesting can still be a valuable tool for traders. It can help identify potential strategies, highlight potential risks and weaknesses, and serve as a starting point for further research and analysis. However, it's crucial to remember that past performance is not indicative of future results, and backtesting alone should not be the sole basis for making investment decisions.
For successful trading, it's essential to combine backtesting with other crucial elements like thorough fundamental analysis, understanding market psychology, managing risk effectively, and maintaining a disciplined approach in the face of market volatility and emotional impulses.
N.B: I first publish an excerpt of this article as an answer on Quora. You can read my other answers here
FAQs
Backtesting often overlooks survivorship bias, leading to an overestimation of performance. By focusing only on assets that have survived until the present day, backtests may paint a rosier picture of strategy performance than would be the case in a real-world scenario where failed assets are included.
What are the disadvantages of backtesting? ›
Disadvantages of backtesting
Because the outcome of backtesting relies on a simulation, it's subject to biases. Investors can manipulate the data to achieve a desirable result, without realizing they're doing it. It's important to create the strategy before having access to the data to avoid this bias.
What is the problem with backtesting? ›
Limited data quality: Backtesting relies on historical data, and the quality and accuracy of the data used can have a significant impact on the results. Data may contain errors, gaps, or other inconsistencies, which can distort the backtest results and lead to inaccurate conclusions about the strategy's performance.
Is backtesting good for trading? ›
' By leveraging the power of historical data, backtesting enables traders to analyze the potential profitability and risks associated with their strategies, helping them make informed decisions and improve their overall trading performance.
What are the biases of backtesting? ›
A: Common backtesting biases include survivorship bias, look-ahead bias, data snooping bias, and curve-fitting and optimization bias. These biases can distort the backtesting results and lead to inaccurate performance estimations.
Can you trade without backtesting? ›
It's important to note that backtesting isn't a guarantee that a strategy will be successful in the current market. Past results are never a fool-proof indicator of future performance. Rather, it's part of doing your due diligence before opening a position.
What is backtesting in risk? ›
Backtesting measures the accuracy of the value at risk calculations. Backtesting is the process of determining how well a strategy would perform using historical data. The loss forecast calculated by the value at risk is compared with actual losses at the end of the specified time horizon.
What is the opposite of backtesting? ›
Backtesting is the process of recreating the work of your strategies on historical data, essentially all of your past strategic work. Forward testing allows for the recreation of your strategy work in real-time, all while your charts refresh their data.
How do you backtest accurately? ›
Choose Your Data
Select the historical data you want to use for backtesting. This data should closely resemble the market conditions you expect to encounter when trading live. Ensure that the data is accurate, reliable, and includes all relevant information (price, volume, spreads, etc.).
Is 100 trades enough for backtesting? ›
If you're backtesting a day trading strategy, 100 trades is not nearly enough to see if a strategy is reliable. Let's say that you're backtesting a day trading strategy that averages 1 trade per day. There are about 20 trading days per month. So if you have 20 trades per month, 100 trades will only represent 5 months.
Unlike retail traders who dabble with different strategies they never know work or not, professional traders only employ strategies they have confirmed through backtesting to have an edge in the market and then execute them in the right way and at the right time.
How many times should I backtest my trading strategy? ›
When you are backtesting a strategy on a higher timeframe, you will have to go back 6 to 12 months. Ideally, you want to end up with 30 to 50 trades in your backtest to get a meaningful sample size. Anything below 30 trades does not have enough explanatory power.
Is backtesting on TradingView accurate? ›
In summary, TradingView provides powerful tools for both manual and automated backtesting. However, remember that backtesting is just one part of strategy development. Past performance doesn't guarantee future results, so always trade with caution and proper risk management.
What are the pitfalls of backtesting? ›
One of the most prevalent pitfalls in back-testing is over-optimization, also known as “curve-fitting.” This occurs when a trading strategy is excessively tailored to historical data, performing exceptionally well in the past but failing to generalize to future market conditions.
What is the assumption of backtesting? ›
A successful backtest will show traders a strategy that's proven to show positive results historically. While the market never moves the same, backtesting relies on the assumption that stocks move in similar patterns as they did historically.
What is Overfitting in backtesting? ›
Backtesting is a powerful tool for evaluating the performance of a trading strategy based on historical data. However, it can also lead to overfitting, which means that the strategy fits the data too well and does not generalize to new or unseen market conditions.
What are the disadvantages of test retest reliability? ›
The disadvantage of the test-retest method is that it takes a long time for results to be obtained. The reliability can be influenced by the time interval between tests and any events that might affect participants' responses during this interval.
What are the disadvantages of expected return? ›
Disadvantages of calculating expected return
The most notable is the volatility of the market. Market conditions aren't consistent, so the expected return relies on speculating on different market conditions and the probability of each scenario.
What are the disadvantages of scripted testing? ›
Disadvantages
- Results can be non-standardized across different testers.
- This technique doesn't find many bugs.
- It focuses on limited areas constrained by the script from which you cannot deviate.
It drives students and teachers away from learning, and at times from school. It narrows, distorts, weakens and impoverishes the curriculum while fostering forms of instruction that fail to engage students or support high-quality learning.
