Backtest Strategy Enhancement. The Use Of 'First Date' Of New Period.

May 27, 2011

In response to feedback we have improved the suite of Relative Strength backtesting applications.  The "Use Next to Last Day Pick(s)" option, which some users found confusing, has been replaced by an option to "Invest on first day of next period". 

This change has two benefits:

A) Consistency throughout the site - because the "first day" option still uses the Screener’s end of period picks, the ETFs chosen will always match those shown on the Screener Ranks and RS Reader pages

B) Because the "first day" option invests in exactly the same ETFs, the performance is generally much closer to that of the standard backtest

To be clear, does not use opening prices in any of its backtesting apps.  This is because opening prices can often be skewed quite significantly.   Closing prices cannot be manipulated nearly as much because the final 30 mins is a very volume-intensive time and anyone trying to set a price could easily get 'run-over' trying to do it.  By contrast, opening prices are somewhat guesswork.  The underlying securities of an ETF haven't opened either - so it is impossible in some cases to precisely know how all the various gaps will play out in terms of the overall index.  Moreover, market makers can sometimes manipulate the opening price to fill existing market orders and then let the price go back to its natural point.   This is especially true on options expiration.  Sometimes the opening price might print a small number of shares and then move quickly in the other direction to a more natural price.   Consequently we think it's best, as a rule of thumb, to avoid trading the open.

In addition, it should be clear that if you trade on the first close of a month, then the comparison index return should also be set from the first close of the month.   You should not compare an entry on the first close to an index return that uses a prior day -- it is an unfair comparison.   You want to go for consistency and match the time periods.

Let’s go through one example using the 'Sample' Portfolio.   This portfolio was added to member’s lists to show an example of a few different ideas -- everyone got the same portfolio since we launched this idea in the Spring of 2010.

First note not only the raw return -- but the difference between the return and the index.   +12.0% vs +6.2% for a spread of +5.8%:


Then view using the first close to track performance for both the strategy return as well as the benchmark:


And then finally,  take note of the fact that the dates are offset by 1 day.   We go deep into the issues of dates throughout our site as there are many problems with dates in most financial databases.  If you 'miss' dates, your output will be garbage.  We make sure there are no missing dates for any ETF through the use of SQL database queries.


If we leave the "Invest on first day of next period" option unchecked, then the back test will invest in the chosen ETFs on the close of the last day of the period.  If we use the checkbox, two things happen: 

 1) the 'update' period will now be the first closing price of the new period 

2)  the comparison index will be tracked from the matching date.

 Since the picks are the same regardless of method, this tightens up the performance greatly.  The only difference is the performance on the first day of the period and the performance of the extra closing day on the update date.   These will affect the overall result -- but they treat both the chosen ETF and the comparison index the same.

Note the slight difference in year-to-date returns between the standard (unchecked) backtest and the first day option backtest. It will be better sometimes and worse sometimes and over time, some of the differences will offset each other.  To the extent the first day of the month goes up, you will be long the pick from the previous month and will benefit from this.   To the extent it goes down, so too will the comparison index.  We think this consistency will clarify some of the issues we've been having in email.


Follow Us On Twitter:


Comments (19) -

May 27, 2011 07:40 #

Thanks very much for the new option and espceially the explanation.  I had assumed "Invest on first day of next period" meant at the Open.  I really appreciate this new option; however, as I've commented previously, I would like to have a choice of "Use Next to Last Day Pick(s)"  or "Invest on first day of next period".  Since "Use Next to Last Day Pick(s)" was introduced,  I have all of my spreadsheets setup with the proper Time Period and Weight paramenters along with backtest results and actual results.


Ernest United States

May 27, 2011 08:01 #

Hi Ernest, there is no reason you can't continue to buy 'next to last' on the last day now --- or for that matter any other method -- just use the Screener.   The only difference is how we are automating the performance reports here.    

So far, we have not tried to build applications focused on 1 or 2 or 3 day tendencies --- we are just trying to capture the big intermediate moves that ultimately drive portfolio performance.  

The problem with the next-to-last is only that it caused some confusion amongst some users.    

Regarding assuming the open,   for many months now it has clearly said "All returns are total return, which includes dividends and distributions, and assume the entry and exit is executed at the closing price."

So think of the website as 2 parts --- you can do almost anything you want with the flexibility in the date controls.   But when it comes to AUTOMATING the backtesting into advanced reports, then we have to have limits on the amount of options or else the application set-up will begin to look like the cockpit of the Space Shuttle with so many controls.  

If you want to send an example spreadsheet so I can see what you mean, send it through the contact us tab and we will take a look.

Chris United States

May 27, 2011 12:00 #

Hello all!

Chris, this is a very good addition to backtesting module, but I have to agree with Ernest. I would also like to have three options:

E= end of period
C=closing price

E-1 =  picks from E-1, buy/sell on E @ C  //Use Next to Last Day Pick(s)
E = picks from E, buy/sell on E @ C //Last Day Pick(s)
E+1 = picks from E, buy/sell on E+1 @ C //Invest on first day of next period

I know I can still use screener to see which ETFs are chosen, but I can't backtest E-1 anymore! Three backtest dates = more robust strategy!


May 27, 2011 23:00 #

I too am a little disappointed to see the "Use Next to Last Day Pick(s)" removed.  Having said that, I understand Chris's reasoning.  I still think the best setup would be to allow the user to specify the exact rotation dates, AND provide a "Next to Last Day" pick checkbox.  

I think Chris has hinted about this in the past, but I'm finding that a robust system is not too dependent on rotation schedule or whether the next to last day picks are utilized.  Backtests that do change dramatically based on these selections are likely suffering from a degree of curve fitting.

Alecs Hong Kong S.A.R.

May 28, 2011 03:36 #


I think some of the confusion regarding "Use Next to Last Day Pick(s)" may be the terminology used.  I think it might be a little less confusing if something like "Use Previous to Last Day Pick(s)" is used.  

Ernest United States

May 28, 2011 12:21 #

This is a great improvement. It is a much more functional option.

TrickPony United States

May 29, 2011 01:10 #

I believe there would be no need for any option if the picks from E-1  are the ones the Screener Ranks and RS Reader pages show till the period's next E-1 (E=End of period). In that case, there is entire consistency between buy/sell at E and picks shown throughout the new period. And also the period's Returns will be consistent with the returns from charting programs (which are calculated from close at E to close at next E).

Rudy Belgium

May 29, 2011 05:20 #


That is a nice presentation!

Ernest United States

May 29, 2011 08:56 #

This is all good feedback.   Realize that we have spent many weeks testing this and were quite impressed with the results and their consistency with the original app  (same as the 'unchecked' option).    So take it for a spin and I think you will find the same -- the results are good -- and consistent.  The reason is that it puts this '1 day difference' issue into perspective well ---  the decision to buy on next-to-last vs last vs last+1 is really of much less importance in the long-run.  

When returns for the markets are low for a few months, it will 'seem' like these small move are  important --- but then these will all be very small blips on the chart when looking back X months from now.

We base our decisions first on the research we do -- trust that we have built all sorts of things that were mildly interesting but in the end are just not very compelling.   In this case, the research we've done greatly supports the move we've made here.    

Be sure that we are internally discussing the points made here and in emails and will continue to enhance the standard applications so keep the feedback coming.

Chris United States

May 29, 2011 09:24 #

I really appreciate the research and development the team does.  I think this new enhancement is a great improvment!  I am in the process of updating all my workbooks/spreadsheets with results from backtesting with the new option.

Thanks guys!

Ernest United States

Jun 01, 2011 00:44 #

Guys,  Backtesting is just that, Backtesting !    Performances based on the securities in the portfolio.   Anyone get choose ETFS  portfolio and create a great backtesting performance.


Bruce United States

Jun 01, 2011 07:54 #

The key is 'synthesis' ---  combining backtesting with your own beliefs and research about where you want to be invested.   Keeping your foot on the accelerator at all times will take you to the poorhouse.    Knowing when reward warrants the risk and when to ease up is very crucial.  

Backtesting and then stopping is not that helpful.   But ongoing testing of ideas helps you see market relationships develop as they occur.

We saw a good rotation in May out of risk assets.  This was made easier to see through the relative strength backtesting process.

Chris United States

Jun 01, 2011 08:26 #

Thanks Chris for these general comments. It's not about setting up (curve fitting) a system with the best backtesting results, but rather finding out what rules help to make good investment choices.

Looking at the Portfolio Screener Ranks (last 12 weeks) one can see best the rotation of ETFs. Which ETFs, over the last weeks,  are gaining/loosing most in the ranks?  

hugos Germany

Jun 02, 2011 00:06 #

What "rules" ?  This is about simple backtesting based on   Return A, Return B and Volatility.  Any one this parameters can be modified to produce great backtesting results.    Add  ETF or remove a ETF from a portfolio, you can produce even better results.  I don't see how you can use this manipulated system and call it "RULES".   Please explain. Chris or whoever.  

Bruce United States

Jun 02, 2011 00:46 #


I suggest you explore this site a little further.  Rotating ETFs or any other assets based on pre-defined parameters can be described as a "system".  A system has rules.  It is that simple.

Any system / backtest can be over-optimized (curve fitted) however as Chris has pointed out, the key to this and any other mechanical system is to find a combination that provides an edge.  Sometimes that edge is simply the removal of emotion from investment decisions.

Alecs Hong Kong S.A.R.

Jun 02, 2011 03:08 #

Alecs, removal of emotions was indeed an edge for me. Plus an impressive simplification of diversification - I adjusted my investing now to the equivalent of the Permanent Portfolio and a couple of enhanced versions of that, delivering stable and nice returns, and good sleeping again.

Rudy Belgium

Jun 02, 2011 06:46 #


I would suggest you watch some of the video examples and slideshow examples within the member applications.    We did not do wild examples involving ultra-narrow exchange traded products .    Instead we did solid methods based on balanced portfolios  (equities and bonds).   These models were based on some thought and logic put in up-front ---  and were definitely NOT done by 'solving for the best possible backtest'.    Using backtesting as researcj will lead you to a greater understanding of the markets.  

Go watch the examples from 6-12 months ago and run the updated performances.   I think it speaks for itself.

Chris United States

Jun 03, 2011 00:30 #

Thanks, Chris.  I will do that.

Bruce United States

Aug 09, 2011 04:22 #

Invest first day next period - Y/N?   Maybe I've missed something here.  I've noticed that sometimes the differences between the two options are significant.  

With the NO option, it is obviously impossible to simultaneously close an existing portfolio and open a new one at the EOD closing prices.    

However it is possible to close the existing portfolio "at market" 15 minutes before EOD, and then open the new one at the EOD close.   Provided you use liquid ETFs then the differences will be minuscule compared with the YES option.  Maybe I've missed something?

GrahamCC United States

Comments are closed

Follow ETFreplay