Category: Backtest
Aug 31, 2021
in Backtest, Moving Average, Channel, Ratio, Parameter Summary
We have added three new Parameter Performance Summaries to the website:
As with the Relative Strength and TRD summaries that we introduced in July, each of the above can be accessed from their respective backtests.
Set the min, max and step / increment for each parameter, then click 'Run Backtests' and the tabulated results will be displayed:
Parameter Performance Summaries are available to all (regular and pro) annual subscribers.
** studying the guidelines that we published within the original Parameter Summaries announcement is highly recommended **
Jul 21, 2021
in Backtest, Parameter Summary
We have just added new functionality to the site that makes it possible to backtest numerous different parameter combinations in one go. We have started with two Parameter Performance Summaries, one focused on relative strength and the other on mean reversion:
Parameter Performance Summaries are available to annual subscribers, both regular and pro, and can be accessed from their respective backtests.
Set the weight, min, max and step / increment for each required parameter, then click 'Run Backtests' and the tabulated results will be displayed:
Backtesting is the only way to know if a strategy works, but it obviously does not guarantee good future performance. Following a solid testing procedure, however, puts the odds in your favor. To that end, we recommend keeping the following in mind when using the Parameter Performance Summaries:
- Backtest results shouldn't be used to justify a model; it needs to have a sound underlying rationale to begin with.
- Choose a range of parameter values that make sense for the strategy. i.e., very short lookbacks aren't suited to a strategy targeting longer-term trends and vice versa. Sticking to appropriate parameter values lessens the possibility of being misled by an isolated / lucky result.
- Larger step / increment values can be used initially to identify the approximate range of parameter values that have produced the best returns. More detailed testing, with smaller step values, can then be done on that range of parameter values..
i.e., begin with a wider spread between min and max and a larger step value. After identifying a narrower min / max spread, the step value can be reduced.
Alternatively, when employing 2 or more factors, choose 'Serial' rather than 'All Backtests'. Rather than backtesting every parameter permutation, Serial employs a multi-stage process that reduces the total number of backtests performed, thereby allowing longer periods of time and/or a wider range of parameter values to be tested.
- When examining the results:
- A robust model will be moderately sensitive to small differences in parameter value. i.e. performance will vary, but slightly different parameter values should not produce wildly different returns. Larger differences in parameter value, by contrast, should be expected to have a bigger impact on performance. (note: adjacent monthly parameters will exhibit larger performance differences than adjacent daily lookbacks.)
- If the top performing parameter's returns are far above the rest, then it indicates that its results likely benefited from good luck.
- The best parameter / lookback values are generally those that show consistency over time. i.e. parameter values that were hugely successful in favorable environments but performed poorly in other conditions, are less desirable than parameter values that produced solid returns across different market environments.
- The overall backtest should be of sufficient length to include a mix of environments; up, down and sideways markets. Examining shorter periods within that backtest is also worthwhile, as it’s unlikely that the best overall performers were strongest in each and every sub-period. Recognizing that even the best strategies have endured periods of under-performance can help set realistic expectations.
- The Parameter Summary provides an overview, but it’s important to go beyond the headline statistics once a set of parameter values has been identified. Run a backtest and examine the return and drawdown for each year vs your benchmark. Was it a wild ride? Could you have actually stuck with it (be realistic)? Does huge out-performance in only one or two years mask under-performance the rest of the time? If so, was that because the model does well only in a particular environment, (if so, can you live with that?) or, was it just lucky at certain times?
- The future is all that matters and it won't be exactly like the past, so there's little point in trying to precisely optimize historic performance.
You may need to refresh the site to force your browser to download all the necessary files from the server afresh in order for the pages to render correctly. To do this, hold down the Shift key on your keyboard and click on the circular arrow icon to the left of address field at the top of the browser window.
Feb 15, 2021
in Backtest, Mean Reversion, Video, Sequential RS
Note: during the launch of our new application called Sequential Relative Strength, we are allowing all accounts to create portfolios using individual stocks. This app module is able to expand on the core Portfolio Relative Strength and add a 2nd stage to help improve entry points.
Reviewing some recent actual trading from the lens of some of our backtesting applications. This has been a very good market environment for ETF rotation. #STUDY
to expand video on screen, click the '4 expanding arrows' icon in the bottom right corner of the video screen. Use the settings icon to change to 1080 quality if it seems at all blurry
Jun 20, 2020
in Backtest, Video, Sequential RS
Note: during the launch of our new application called Sequential Relative Strength, we are allowing all accounts to create portfolios using individual stocks. This app module is able to expand on the core Portfolio Relative Strength and add a 2nd stage to help improve entry points.
The idea behind this module is to more easily combine intermediate-term relative strength with short-term mean reversion. Short term weakness within a strong long-term trend is normal. Try running your backtest with this mean-reversion added and then compare it to a model which does the opposite. Analyze the results. Think about the volatility of the resulting equity curve. Look at sub-periods. Are the results highly lumpy? Were the results achieved in earlier years but fade in recent years? These are some of the things ETFreplay was created to do -- not just look at the return of a strategy -- but to decompose it and analyze it to try to find something that is more robust. #STUDY
to expand video on screen, click the '4 expanding arrows' icon in the bottom right corner of the video screen. Use the settings icon to change to 1080 quality if it seems at all blurry
Mar 03, 2020
in Backtest, Mean Reversion, TRD Total Return Diff
We have introduced a new backtest for subscribers that is focused on short term mean reversion. It is intended to be used with highly related securities, which are unlikely to drift apart for very long before coming back together. Consequently, when one of the securities underperforms by a non-insignificant margin, an opportunity occurs.
The backtest works by comparing the difference in the x-day Total Return between the securities / ETFs. When that x-day return difference is more than y standard deviations below the mean, the backtest will buy Security 1. It will then hold that security until it reaches z standard deviations above the mean, when it will be sold in favor of Security 2.
Suitable primarily for non-taxable accounts only, the idea is that these short-term trades among correlated assets can be used to materially enhance the returns through compounding without a material difference in the underlying beta exposure. More return not for less risk -- but for very similar risk. Said another way, taking a modicum of extra risk can be quite profitable.
Go to Total Return Difference (TRD) - Mean Reversion Backtest