Category: Backtest
Jul 05, 2022
in Backtest, Dashboards, Tracking Error
We have added some new functionality to the Core-Satellite backtest, Tracking Error tool and Dashboard.
Core-Satellite
The Core-Satellite backtest has an updated look and the Hold Filter, which we recently added to the Portfolio RS and RS - Combine Portfolios, can now be applied to either (or both) of the satellites.
click image to view full size version
The appearance of the other parallel relative strength backtests, the sequential RS and various regime backtests has been similarly updated.
Tracking Error tool
The tracking error values now link through to the ETF charts page, which will automatically load the two securities over the specified duration and display their performance.
click image to view full size version
Dashboard
We recently added a new Summary window option that displays the 1-Day, Year-To-Date, 2021 and 2020 Total Return of each security in your chosen list.
click image to view full size version
Apr 05, 2022
in Backtest, Relative Strength
We have added a Hold Filter option to the Portfolio Relative Strength and RS - Combine Portfolios backtests.
When the Hold Filter is turned on, the ETFs will be held as long as they remain ranked in the top (or bottom) Y.
i.e. If you choose a Buy Top of 2 and a Hold rank of 4, then the backtest will invest in 2 ETFs, but will only rotate out of an ETF when it drops below the top 4.
Using the Hold Filter makes a model less reactive and as a consequence it reduces trade activity. By not switching each and every time a security moves in and out of the top X, it can also result in fewer whipsaws. However, the higher the hold rank number is, the less responsive to changes the model will be, so a balance must be struck.
click image to view full size version
Note: the hold rank must be greater than the buy top / bottom number and be less than the number of securities in the chosen portfolio(s).
Oct 17, 2021
in Backtest, Relative Strength, TRD Total Return Diff, Video, Ratio, Parameter Summary
Instructional video on how to use the Parameter Performance Summary functionality. #STUDY
to expand video on screen, click the '4 expanding arrows' icon in the bottom right corner of the video screen. Use the settings icon to change to 1080 quality if it seems at all blurry
Aug 31, 2021
in Backtest, Moving Average, Channel, Ratio, Parameter Summary
We have added three new Parameter Performance Summaries to the website:
As with the Relative Strength and TRD summaries that we introduced in July, each of the above can be accessed from their respective backtests.
Set the min, max and step / increment for each parameter, then click 'Run Backtests' and the tabulated results will be displayed:
Parameter Performance Summaries are available to all (regular and pro) annual subscribers.
** studying the guidelines that we published within the original Parameter Summaries announcement is highly recommended **
Jul 21, 2021
in Backtest, Parameter Summary
We have just added new functionality to the site that makes it possible to backtest numerous different parameter combinations in one go. We have started with two Parameter Performance Summaries, one focused on relative strength and the other on mean reversion:
Parameter Performance Summaries are available to annual subscribers, both regular and pro, and can be accessed from their respective backtests.
Set the weight, min, max and step / increment for each required parameter, then click 'Run Backtests' and the tabulated results will be displayed:
Backtesting is the only way to know if a strategy works, but it obviously does not guarantee good future performance. Following a solid testing procedure, however, puts the odds in your favor. To that end, we recommend keeping the following in mind when using the Parameter Performance Summaries:
- Backtest results shouldn't be used to justify a model; it needs to have a sound underlying rationale to begin with.
- Choose a range of parameter values that make sense for the strategy. i.e., very short lookbacks aren't suited to a strategy targeting longer-term trends and vice versa. Sticking to appropriate parameter values lessens the possibility of being misled by an isolated / lucky result.
- Larger step / increment values can be used initially to identify the approximate range of parameter values that have produced the best returns. More detailed testing, with smaller step values, can then be done on that range of parameter values..
i.e., begin with a wider spread between min and max and a larger step value. After identifying a narrower min / max spread, the step value can be reduced.
Alternatively, when employing 2 or more factors, choose 'Serial' rather than 'All Backtests'. Rather than backtesting every parameter permutation, Serial employs a multi-stage process that reduces the total number of backtests performed, thereby allowing longer periods of time and/or a wider range of parameter values to be tested.
- When examining the results:
- A robust model will be moderately sensitive to small differences in parameter value. i.e. performance will vary, but slightly different parameter values should not produce wildly different returns. Larger differences in parameter value, by contrast, should be expected to have a bigger impact on performance. (note: adjacent monthly parameters will exhibit larger performance differences than adjacent daily lookbacks.)
- If the top performing parameter's returns are far above the rest, then it indicates that its results likely benefited from good luck.
- The best parameter / lookback values are generally those that show consistency over time. i.e. parameter values that were hugely successful in favorable environments but performed poorly in other conditions, are less desirable than parameter values that produced solid returns across different market environments.
- The overall backtest should be of sufficient length to include a mix of environments; up, down and sideways markets. Examining shorter periods within that backtest is also worthwhile, as it’s unlikely that the best overall performers were strongest in each and every sub-period. Recognizing that even the best strategies have endured periods of under-performance can help set realistic expectations.
- The Parameter Summary provides an overview, but it’s important to go beyond the headline statistics once a set of parameter values has been identified. Run a backtest and examine the return and drawdown for each year vs your benchmark. Was it a wild ride? Could you have actually stuck with it (be realistic)? Does huge out-performance in only one or two years mask under-performance the rest of the time? If so, was that because the model does well only in a particular environment, (if so, can you live with that?) or, was it just lucky at certain times?
- The future is all that matters and it won't be exactly like the past, so there's little point in trying to precisely optimize historic performance.
You may need to refresh the site to force your browser to download all the necessary files from the server afresh in order for the pages to render correctly. To do this, hold down the Shift key on your keyboard and click on the circular arrow icon to the left of address field at the top of the browser window.