Posted By: pcz
Are all backtest results potentially flawed? - 11/21/16 12:22
I haven't received an answer for reported problems with slippage in another thread yet so I'll try to reformulate the issue and also provide a code to reproduce the whole thing.
It seems that when you set slippage greater than zero, the results might be optimistically biased. The effects can be quite profound even with values around 5, which is the default in new Zorro. It can effectively ruin your account. You can try for yourself with this simple mean reversion strategy:
In Zorro 1.51 it gives me AR 50% and SR 1.15 - not that bad, right? Now lets add line Slippage = 0;
With zero slippage AR is 0% and SR 0.01. Isn't that weird? Lets try larger value, for example one minute (i.e. 60). Now the AR is 1293% and SR is 11.12. The number of trades and entry / exit times are the same as with slippage set to 0, yet the results are very different. This is for example the first trade for both extreme cases (i.e. Slippage = 0 and Slippage = 60 in the second line):
Exported price data for the asset (around open / exit time):
As you can see with slippage the entry and exit prices are way out of line.
Sphin noted that in the new beta version there should be a fix for some slippage issues but I tried it and the results are the same so I assume it was a different problem than this one I described.
Also - I asked for this months ago: it would be really useful to increase the precision of price representation in log files. With only four decimal places it's really hard to debug the fine details.
It seems that when you set slippage greater than zero, the results might be optimistically biased. The effects can be quite profound even with values around 5, which is the default in new Zorro. It can effectively ruin your account. You can try for yourself with this simple mean reversion strategy:
Code:
//Asset settings in AssetFix.csv:
// EUR/USD,1.12000,0.00005,-0.418,0.0560,0.0001,10.00,2236,0,100000,0,
function run()
{
Asset = "EUR/USD";
StartDate = 20140101;
EndDate = 20141231;
if(IBS() < 0.2 && NumOpenLong == 0)
enterLong();
if(IBS() > 0.8 && NumOpenShort == 0)
enterShort();
}
In Zorro 1.51 it gives me AR 50% and SR 1.15 - not that bad, right? Now lets add line Slippage = 0;
With zero slippage AR is 0% and SR 0.01. Isn't that weird? Lets try larger value, for example one minute (i.e. 60). Now the AR is 1293% and SR is 11.12. The number of trades and entry / exit times are the same as with slippage set to 0, yet the results are very different. This is for example the first trade for both extreme cases (i.e. Slippage = 0 and Slippage = 60 in the second line):
Code:
Osc_spread,Long,EUR/USD,8201,1,2014-01-07 19:00,2014-01-08 02:00,1.3617,1.3628,108.00,0.00,Reverse Osc_spread,Long,EUR/USD,8201,1,2014-01-07 19:00,2014-01-08 02:00,1.3613,1.3630,163.99,0.00,Reverse
Exported price data for the asset (around open / exit time):
Code:
07/01/14 19:00, 1.36175, 1.36184, 1.36163, 1.36163 07/01/14 19:01, 1.36163, 1.36163, 1.36154, 1.36157 ... 08/01/14 02:00, 1.36283, 1.36290, 1.36283, 1.36289 08/01/14 02:01, 1.36289, 1.36290, 1.36274, 1.36275
As you can see with slippage the entry and exit prices are way out of line.
Sphin noted that in the new beta version there should be a fix for some slippage issues but I tried it and the results are the same so I assume it was a different problem than this one I described.
Also - I asked for this months ago: it would be really useful to increase the precision of price representation in log files. With only four decimal places it's really hard to debug the fine details.
