Dec 22, 2015
in Backtest
In one of the best poker books ever written The Theory Of Poker, the author makes a point quite relevant to investing. He writes,
“Beginning poker players sometimes ask, ‘What do you do in this particular situation?’”
The problem is that it is simply the wrong question and indicates overly-simplistic mode of thought. The right question is:
“What do you consider in this particular situation before determining what to do?”
Likewise, the term ‘backtest’ is sometimes viewed critically by those who are beginners. Some make simplistic assumption that just by applying a longer timeframe to a given backtest then validates a strategy and that therefore they can just follow a simple model and don’t have to burden themselves anymore with any critical thinking.
But ETF backtesting isn’t anything more (or less) than information, and in the investment business, information is research. Research is what leads you to make informed decisions. Often times you run backtests that seem to conflict with each other in terms of what to do at a given point in time. A quarterly updating backtest with a 12-month lookback might have good results and indicates to hold on until Dec 31 while a monthly backtest with a 3-month lookback might be indicating a switch is called for immediately. Some argue that whichever one has a higher return over the longest timeframe available must be correct. But frankly, your strategy backtested to the year 1900 might not be nearly as relevant as you think (see Intro to Regime Change ETF Backtesting). It may even be an awful strategy for the present-day environment.
At the end of the day it is sometimes going to be a judgment call based on the weight of the evidence and the conclusions you reach based on your overall research/information.
No matter whether you stand pat and do nothing or make some trades due to perceived increased risk, you might end up wrong in the short-run. This is why you focus on the decision-making process and not short-term results because any single decision can lead to a poor result. But performance over time is the culmination of many decisions.
Some might say to this…. ‘well I don’t want to play’ and I do have the choice to ‘just buy index funds.’ But ask anyone maintaining a buy and hold global portfolio how those ‘just buy index funds’ portfolios have done? They are loaded in international stocks and emerging markets and have portions allocated to gold and other commodities. Have you seen how poorly these segments have done? Moreover, do you think the S&P 500 is immune to similar bouts of very poor relative performance? Think again.
The point is that any judgment you make can be right or wrong, so don’t think anyone is removed from the process just because they buy index funds. They aren’t. You didn’t see buy & hold investors do much marketing in 2001-02 during a bear market. Nor in 2008-09. Buy and hold global advisors of today can’t market their returns because they have losses, not returns.
Summary: Focus on what to ‘consider’ and then make an informed decision. Good investors know how to control their drawdowns while still exposing themselves on the long side as a general framework. Backtesting is very much a part of this broader decision-making process because be it buy hold portfolio allocation backtesting or tactical backtesting, backtesting is simply about research.
Mar 18, 2011
in Backtest
It is well accepted in professional money management that having a quantitative aspect to your investment process is additive. That is, quantitative methods can greatly help in screening and monitoring lists of securities into a manageable ranking for further analysis. The vast majority of institutional-oriented firms do this kind of thing.
The classic, basic steps of an investment process involve:
- Install a (Quantitative) Method To Rank A Relevant Universe of Securities
- Take The Top-Ranked Securities And Do Further Research
- Construct a Portfolio Using Securities That Pass The First 2 Steps
- Monitor And Update The Portfolio
- Repeat
We view backtesting as a very practical and useful part of the research process. The way you rank securities should be based on something consistent with your beliefs on what actually works.
Once you have done some research and found a method to rank securities, run some backtests. You will learn about your method greatly and can understand more aspects and characteristics of such strategies. You will speed up your understanding and you will be forced to think through all the details of how you are going to execute your process.
Once you have created a portfolio (per Step 3 above), that portfolio effectively now becomes a ‘forward-test.’ Monitoring these portfolios and seeing these various rotation strategies behave is a crucial part of the process. Track your various rotation strategies. You can learn a lot by running many simultaneous ‘forward tests’ at once. Importantly, you will get a feeling for the ‘short-term noise’ that occurs around your strategy. Even for professionals, the psychological aspects of short-term volatility will cause them to doubt themselves. Back/Forward tests will make you much more aware of the kind of thing you will face in the future. You must get used to this as psychological aspects to investing are absolutely critical.
Back/forward-testing accelerates the learning process and you can then feed the incremental improvements you discover back into your actual portfolio. The point is that just doing a backtest and then ‘stopping your research’ is very limiting. Don’t stop learning, don’t stop improving. Small incremental improvements add up over time.
We are entering the golden age of active indexing. The specialized, targeted index fund is really a somewhat new phenomenon. Index funds in the 1980s were all very broad vehicles. Many specialized index products (Vanguard REIT index mutual fund, country fund ETFs) actually only have histories back to 1996. You can simulate prior performance -- but they weren't so inexpensively accessible. TIPS securities weren’t even issued by the Treasury until 1997. The World Equity Index products (mutual fund AND etf) didn’t launch until 2008.
Important new areas of future investment may come from newly investable products. For example, the emerging markets small cap index might become as mainstream a product as some other well-accepted benchmarks are today. Understanding the important indices of tomorrow might be as good an idea as understanding REIT and emerging markets indices when they were new PRODUCTS (ie, investable and tradeable at reasonable cost).
Understanding and processing relative performance, relative volatilities and observing relative drawdowns in present ‘forward-test’ environment strikes us as a pretty good idea.
iShares is bringing to market an Internationally focused preferred stock index. Wisdomtree just launched an Asia-Pacific regional intermediate bond ETF. Remember that at one point, the emerging markets index mutual fund was brand new (1994). Today it is a primary index everyone follows. Growth vs value wasn’t mainstream until the 1990s -- and indexing these products came later. The world evolves. Embrace the change and learn from it – let many simultaneous forward tests accelerate the learning.