2000 to 2003 focus QQQ's large drawdown backtest

Jan 07, 2018 in Backtest | moving average

Study many different sub-periods for many different markets.  It helps you understand scenarios, it helps you understand strengths and weaknesses of various techniques in backtesting.

Learn how to put the odds in your favor.   If you study many different time periods across many types of markets, you will gain understanding of a strategy that is fragile vs a strategy that is more durable.   You will have ideas that cannot be supported and you realize their weaknesses.   Running bad backtests and learning from that is part of the process.

Below is one look at the 2000-2003 bear market.  We suggest you look at that time period and many other time periods using many different types of funds.   QQQ's downturn was especially bad due to the extended run-up in the prior years (QQQ index existed in prior years but the ETF product QQQ of course did not exist prior to the middle of 1999). 

 

 

 **A pro subscription allows you to backtest to 12/31/99 using daily total return data. 

 

Follow us on Follow etfreplay on Twitter

Tags:

Short-Sell Backtesting with ETFs

Nov 02, 2017 in Backtest | Short Selling | VIX | Volatility

 If you would like to test Short-Selling on ETFreplay, use the versatile app called Rel Str - Combine Portfolios.

You can test going short the top or bottom ranking securities in a list.

 

Follow us on Follow etfreplay on Twitter

Tags:

Adding Rotation Options For Backtesting ETFs

Jun 29, 2017 in Backtest

We have added a new backtest rotation option, which we think delivers some useful flexibility.

Previously Relative Strength strategies could be rotated quarterly, monthly or semi-monthly.   Now you can choose a different schedule in the 'Relative Strength - Combine Portfolios' backtest module utilizing a useful feature we call Skip Rotation:

 

 

This way you can set for example a 3-month (quarterly) or a semi-annual rotation but not necessarily on CALENDAR quarter ends.  So for example, you could offset a quarterly rotation by 1 month and choose Jan, Apr, Jul, Oct.  Or you could choose 'every other month' such as in the example below:

 

Then below we reverse it so now it skips the opposite months as the above example and instead test an earlier 7-year period:

 

Separately, we can actually use this same structure to do some basic seasonality testing.   In the test below, we test going long Small & Midcap stocks for the period November to April and then invest simply in the benchmark S&P 500 from May to the end of October.  We do this by using checkmarks to move to the cash security (set to SPY) for May, June, July, Aug, Sep & Oct:

 

The green question mark icon next to 'Skip Rotation' will, when clicked, produce a pop-up help note with more information about the function. However,  if you have any further questions, please contact us

Follow us on Follow etfreplay on Twitter

Tags:

ETF Regime Change Backtesting – update of a 2011 example

Dec 02, 2016 in Backtest | Regime Change

Back in 2011 we produced a little video that compared the performance of two very different 60-40 allocations: an aggressive portfolio invested in Emerging Markets, Financials and High Yield; and a defensive strategy based around Treasury Bonds, Utilities and Healthcare.

 

The purpose of that video, which can be seen here, was not to show which allocation was best but rather to illustrate that ‘that different sectors perform differently during the course of the business cycle’. It therefore makes sense that when there is a change in the overall regime, allocations should be materially adjusted. 

Below is an update to that original example. The same two aggressive and defensive allocations are used, but this time we have employed the Regime Portfolios Backtest to dynamically switch between them depending on the prevailing regime. For this example with have used a simple credit spread style ratio to define the regime. When high yield bonds are outperforming treasuries, the backtest invests in the aggressive allocation. When the opposite is true, it switches to the defensive portfolio.

This is not meant to be a comprehensive strategy by any means, it's just a simple example to illustrate the concept of adapting to change. Hopefully though it provides a solid starting point for subscribers to conduct their own regime based research.

Follow us on Follow etfreplay on Twitter

Tags:

Trailing Stops vs Time Interval Stops. Yes ETFreplay has stops built into its architecture.

Sep 28, 2016 in Backtest

From subscriber on email:  "I would like to put a trailing stop on the backtest and/or investigate what would happen if I stopped and moved to cash."

This is one of the more common questions we get on email from members.

First off, built into the ETFreplay architecture IS a natural form of stop.  A stop is of course a point at which you exit -- either by moving to cash or moving to another security.   A good relative strength backtest will naturally move towards the performing ETF(s) and away from the non-performers.   We have different trade interval choices on ETFreplay and if you choose say monthly, then you have a stop built-in -- it is just a 'time-interval (monthly) stop.'   

Yes you are essentially locking in a full months performance and not exiting immediately and people often view this as risky -- but that is NOT what the evidence shows.  Having the perception of being 'less risky' instead just means you 'feel emotionally better because you are out of the market.'  Holding on until month-end and accepting that extra time-risk is often dramatically better than locking in a loss at a percentage.

Why?   Because first, stops usually get hit as market gets oversold and then it bounces back and you end up rotating at a dis-advantageous on the stop-price.   

But importantly there is a second reason, the market often not only bounces -- it recovers back strongly and sometimes back to a new swing high and you end up NOT rotating and are sitting on a paper non-taxable gain rather than having taking a real loss and now in cash and out of the market potentially missing more upside.

This second case has happened many times during the bull market that began a few years ago.  If you stopped out, did you get back in higher??  Many times that might be emotionally tough to do.   If you accept the calendar month return, you might occassionally be worse off -- but this is usually offset by the 'death by a thousand cuts' underperformance risk so many twitchy traders suffer from....

But what about stops as many people read about in all those trading books?  

Trading books are almost always based on things that have no ETF research or backtest support, many no backtest support at all.  Some recommend a 'percentage based stop'  (ie,  stop if XYZ drops by -8% from your purchase price).   Those that say they have research behind their method don't present the actual results and instead have done limited testing and made conclusions based on one set of detailed assumptions.   These are also almost always done on individual stocks -- not ETFs.   We are experts in ETF backtests and people should be aware that backtesting an ETF is NOT the same thing as backtesting an individual stock.   We have done a fair bit of work on individual stocks too -- but find that individual stocks are extremely noisy and it takes a tremendous amount of trading activity (many small positions to offset the added noise) to actually implement anything where the statistics can back it up.  Many invstment advisors simply can't do such things as trade hundreds and thousands of individual stocks every month or quarter.   (And as you can see in hedge fund results, neither can hedge funds that try to do it).

Also with individual stocks, you are always worried about a total cratering -- your stock potentially becoming a penny stock or bankrupt.   But even if you forget that ultimate risk, many stocks can lose tremendously more than a typical index ETF and simply not recover wheras an index of many stocks will recover.  Many past stocks that had large weightings in an index collapsed and the index not that soon after went back to new highs -- think Bank of America or Cisco Systems or worse Worldcom and countless others that were at one point considered core holdings.   With individual stocks, you cannot take hits like that that and never rotate away.  The losses can be huge and importantly, the opportunity cost of sitting in a dead stock long past its prime can cause you to dramatically underperform.  

Some of the hidden beauty of an index is that they by rules-based methodology let winning stocks and groups of winning stocks grow in weighting while broken stocks of past cycles lose their significance (weighting).  

While time-interval stops can work for individual stocks too, we are more concerned with how this all applies to ETFs since ETFs work much better for trading practicality reasons (after all, trading an ETF is exactly the same thing as trading a large basket of stocks all at once).   You get hundreds or even thousands of trades (and the associated altered exposure) for the cost of zero  (assuming you are in a free trading program like Schwab, Ameritrade or others have).

 

 

Follow us on Follow etfreplay on Twitter

Tags:

Ratio Moving Average Backtest Example

Sep 02, 2016 in Backtest | Ratio

There are lots of ways to skin a cat of course.   Here is a look at results of using a simple total return ratio between Emerging Markets (VWO) and an iShares Treasury ETF (IEF):

 

 

 

 

 

Follow us on Follow etfreplay on Twitter

Tags:

In ETF Investing, 'Backtest' should be synonymous with 'Research'

Dec 22, 2015 in Backtest

In one of the best poker books ever written The Theory Of Poker, the author makes a point quite relevant to investing.   He writes,

“Beginning poker players sometimes ask, ‘What do you do in this particular situation?’”   

The problem is that it is simply the wrong question and indicates overly-simplistic mode of thought.   The right question is:

“What do you consider in this particular situation before determining what to do?”

Likewise, the term ‘backtest’ is sometimes viewed critically by those who are beginners.   Some make simplistic assumption that just by applying a longer timeframe to a given backtest then validates a strategy and that therefore they can just follow a simple model and don’t have to burden themselves anymore with any critical thinking.

But ETF backtesting isn’t anything more (or less) than information -- and in the investment business,  information is research.   Information is what leads you to make informed decisions.    Often times you run backtests that seem to conflict with each other in terms of what to do at a given point in time.  A quarterly updating backtest with a 12-month lookback might have good results and indicates to hold on until Dec 31 while a monthly backtest with a 3-month lookback might be indicating a switch is called for immediately.  Some argue that whichever one has a higher return over the longest timeframe available must be correct.   But frankly, your strategy backtested to the year 1900 might not be nearly as relevant as you think  (See Intro to Regime Change ETF Backtesting).   It may even be an awful strategy for the present-day environment.

At the end of the day it is sometimes going to be a judgment call based on the weight of the evidence and the conclusions you reach based on your overall research/information.       

No matter whether you stand pat and do nothing or make some trades due to perceived increased risk, you might end up wrong in the short-run.   This is why you focus on the decision-making process and not short-term results because any single decision can lead to a poor result.    But performance over time is the culmination of many decisions.

Some might say to this…. ‘well I don’t want to play’ and I do have the choice to ‘just buy index funds.’  But ask anyone holding a buy and hold global portfolio how those ‘just buy index funds’ portfolios have done?   They are loaded in international stocks and emerging markets and have portions allocated to gold and other commodities.   Have you seen how poorly these segments have done?   Moreover, do you think the S&P 500 is immune to similar bouts of very poor relative performance?  Think again.

The point is that any judgment you make can be right or wrong so don’t think anyone is removed from the process just because they buy index funds.   They aren’t.   You didn’t see buy & hold investors do much marketing in 2001-02 during a bear market.  Nor in 2008-09.   Buy and hold global advisors of today can’t market their returns because they have losses, not returns.  

Summary:  Focus on what to ‘consider’ and then make an informed decision.  Good investors know how to control their drawdowns while still exposing themselves on the long side as a general framework.   Backtesting is very much a part of this broader decision-making process because be it buy hold portfolio allocation backtesting or tactical backtesting, backtesting is simply about research.

 

 

 

Follow us on Follow etfreplay on Twitter

Tags:

Introduction to Regime Change ETF Backtesting

Mar 10, 2015 in Backtest | Regime Change

Q. What is regime change?

Regime change in financial markets terms means that there is a shift in a basic financial relationship that has governed a backtest periods results. Some examples might be an inflationary regime changing to a disinflationary era, reflected in how long-term treasury bonds behave. Or it might be a period of sustained high equity market volatility changing to a more narrow range market. Or a rising US Dollar vs a US dollar bear market. Rising price of oil vs declining etc....

Q. Why is understanding what regime you are in important?

Models and backtests are good at identifying relationships and how to profit from those relationships. But standard backtesting is only good if that financial relationship you are backtesting actually stays in-tact. Relationships develop, exist for a period of time --- often lengthy -- and then they can change.

Recall that the rules of financial markets are not like the rules of other areas of observation like in science. The temperature at which water freezes is the same across time. However, the financial markets are different and need to be thought about differently. You cannot be sure that a given financial law will continue into the future. Water doesn't learn from its freezing in the past and adapt -- but investors do.

Thus, regime change is one of the main problems with many articles you see written that study only very long periods of time. You get caught thinking something is true over the long-run and so -- since you are a long-term investor after all -- you don't care about a 3-7 year period of poor performance.

But there is a key problem with that, what if the underlying regime assumption has been mined and through that long-term analysis papers were written about this financial 'law' that exists. But we know that the thing you are counting on is NOT a law of nature, it is just a financial relationship. If you are wrong in your belief system, how much drawdown (or lost performance) will it take you before you figure out your previous thinking was simply flawed in terms of a reasonable time frame of performance analysis (3-7 years) ?

Q. How do you identify regime change?

This takes ongoing research. This takes ongoing thinking. This takes work. You can and should think about very long-term backtests too -- we are not advising taking only a short-term approach --- but then you should also put time and effort into intermediate-term backtesting to help determine whether the model you are relying upon is acting correctly. Balance the long-term backtests with shorter/intermediate term backtests. There is no way to know exactly how relationships will hold/change in the future --- but you can monitor them in ongoing fashion. You don't have to ride a relationship down into oblivion.

New ETFreplay Backtesting Module: "Regime Relative Strength"

This new module uses some existing methods that have been on ETFreplay.com for years --- but it now combines them into a single process. Before, you had to build this using component modules and then assemble and enter the information as an imported backtest. Now we have automated portions to take more of the monotonous sludgework out and give you more time to do what you should be doing with your time, doing research and thinking about the relationships to test -- not the mechanics of how to make a backtest work and the associated de-bugging.

Think of this as a multi-step process rolled into a single backtest. First, you come up with a rule of how to define Regime 1 vs Regime 2. Once the regime is determined, you line up a backtest strategy for that regime. When the regime switches, you associate a 2nd strategy for that regime. You are always in one regime or the other, there is no grey area -- (backtests need to be specific in the details).

We have defined the 'rule' in this case based on the concept of the Ratio Moving Average. Recall that if you are judging many ETFs (more than 2) against each other, you should use a multi-factor model. But if you are just looking at 2 ETFs, one vs the other, then you may want to just use the total return ratio of the 2 and define rules that way.

Once you have defined what constitutes Regime 1 or Regime 2, you can then set up a strategy for each regime.

Example:

Regime 1 Rule: When NASDAQ is beating SHY Over ~100 day lookback (Absolute Momentum)

Regime 1 Portfolio: Own the top 2 performing basic NASDAQ Sectors

Regime 2 Rule: All Other Times

Regime 2 Portfolio: Reduce drawdown risk but maintain equity exposure by owning lower vol ETFs.

This is a very simple example. You can of course add filters such as cash or moving average filters just like the regular backtesting applications elsewhere on ETFreplay.

Follow us on Follow etfreplay on Twitter

Tags:

Using Different Weightings Based On Rank In an ETF Relative Strength Backtest

Dec 11, 2014 in Advanced Relative Strength | Backtest

User Question:

I run a portfolio relative strength backtest with 5 ETF but all are assigned an equal weight of 20%. How can I assign different weights to the ranked ETFs? Example: Top 5 Weighted as 30%, 30%, 20%, 10%, 10% respectively."

Answer:

For this you would use the Advanced Relative Strength Backtest Module (subscriber link).

By layering the strategies using different numbers of selections while at the same time using the same ETF list, you can create weightings based on Rank.

Note that the top ranked security in this portfolio list would receive 10% from portfolio 1, 10% from portfolio 2 and 10% from portfolio 3. The backtest report combines weights and it becomes simply 30% for the top ranked ETF. The backtest will appropriately rotate the 30% weight to the top ranked security and likewise for the other ETFs. Dive into the backtest report to see all the breakdown of sub-periods and the weightings of each ETF and its contribution to return for that period.

 

 

 

 

Follow us on Follow etfreplay on Twitter

Tags:

ETF Ratio Backtest And Ratio MA Backtest For Portfolios

Apr 12, 2013 in Backtest

Ratio Moving Average backtest was a module we added in 2012.    We have since added Ratio MA Backtest for portfolios.    

Ratio Moving Strategies are good ways to do added research and help you gain comfort in how your portfolio is positioned.  We find that this type of analysis can add some perspective and can help think about some other options to fine-tune your actual entries and exits.   We outlined one way to use it here: Allocation ETF Overweight Example. 

We also recently added a slideshow to highlight our thinking on the layout of ETFreplay.com.    That is located near bottom right of the Backtest Page.  

 

Follow us on Follow etfreplay on Twitter

Tags:

Follow ETFreplay

Archives