Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
5 registered members (Nymphodora, AndrewAMD, TipmyPip, Quad, Imhotep), 847 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 2 1 2
Are all backtest results potentially flawed? #463193
11/21/16 12:22
11/21/16 12:22
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
I haven't received an answer for reported problems with slippage in another thread yet so I'll try to reformulate the issue and also provide a code to reproduce the whole thing.

It seems that when you set slippage greater than zero, the results might be optimistically biased. The effects can be quite profound even with values around 5, which is the default in new Zorro. It can effectively ruin your account. You can try for yourself with this simple mean reversion strategy:

Code:
//Asset settings in AssetFix.csv: 
// EUR/USD,1.12000,0.00005,-0.418,0.0560,0.0001,10.00,2236,0,100000,0,

function run()
{
	Asset = "EUR/USD";
	StartDate = 20140101;
	EndDate = 20141231;
	if(IBS() < 0.2 && NumOpenLong == 0)
		enterLong();
	if(IBS() > 0.8 && NumOpenShort == 0)
		enterShort();
}



In Zorro 1.51 it gives me AR 50% and SR 1.15 - not that bad, right? Now lets add line Slippage = 0;

With zero slippage AR is 0% and SR 0.01. Isn't that weird? Lets try larger value, for example one minute (i.e. 60). Now the AR is 1293% and SR is 11.12. The number of trades and entry / exit times are the same as with slippage set to 0, yet the results are very different. This is for example the first trade for both extreme cases (i.e. Slippage = 0 and Slippage = 60 in the second line):

Code:
Osc_spread,Long,EUR/USD,8201,1,2014-01-07 19:00,2014-01-08 02:00,1.3617,1.3628,108.00,0.00,Reverse
Osc_spread,Long,EUR/USD,8201,1,2014-01-07 19:00,2014-01-08 02:00,1.3613,1.3630,163.99,0.00,Reverse



Exported price data for the asset (around open / exit time):
Code:
07/01/14 19:00, 1.36175, 1.36184, 1.36163, 1.36163
07/01/14 19:01, 1.36163, 1.36163, 1.36154, 1.36157
...
08/01/14 02:00, 1.36283, 1.36290, 1.36283, 1.36289
08/01/14 02:01, 1.36289, 1.36290, 1.36274, 1.36275



As you can see with slippage the entry and exit prices are way out of line.

Sphin noted that in the new beta version there should be a fix for some slippage issues but I tried it and the results are the same so I assume it was a different problem than this one I described.

Also - I asked for this months ago: it would be really useful to increase the precision of price representation in log files. With only four decimal places it's really hard to debug the fine details.

Re: Are all backtest results potentially flawed? [Re: pcz] #463199
11/21/16 13:45
11/21/16 13:45
Joined: Feb 2014
Posts: 73
Montreal, Qc Canada
F
Finstratech Offline
Junior Member
Finstratech  Offline
Junior Member
F

Joined: Feb 2014
Posts: 73
Montreal, Qc Canada
I noticed that some systems behave very differently with and without the TICKS flag. Try setting it and the EXTRADATA flag. I haven't played with the slippage yet though. Will try.

Re: Are all backtest results potentially flawed? [Re: Finstratech] #463203
11/21/16 15:07
11/21/16 15:07
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
Originally Posted By: Finstratech
I noticed that some systems behave very differently with and without the TICKS flag. Try setting it and the EXTRADATA flag. I haven't played with the slippage yet though. Will try.


TICKS + EXTRADATA (especially the second flag) change the results somewhat but there's still a huge difference even between 0 and 5 sec slippage backtest. You can try it for yourself with the code provided above.

Re: Are all backtest results potentially flawed? [Re: pcz] #463205
11/21/16 15:51
11/21/16 15:51
Joined: Dec 2013
Posts: 568
Fuerth, DE
Sphin Offline
User
Sphin  Offline
User

Joined: Dec 2013
Posts: 568
Fuerth, DE
I noted that What's New page promotes a slippage bug to be solved in 1.52.0 but this version seems to be not available yet because the actual beta on the download page is 1.51.9. Did you already get 1.52.0 there?

Re: Are all backtest results potentially flawed? [Re: Sphin] #463206
11/21/16 16:18
11/21/16 16:18
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
Originally Posted By: Sphin
I noted that What's New page promotes a slippage bug to be solved in 1.52.0 but this version seems to be not available yet because the actual beta on the download page is 1.51.9. Did you already get 1.52.0 there?


Sorry, I misunderstood your last post. I thought that it's fixed in 1.51.9. So we'll see if it is fixed in the next version.

Nevertheless it's worth noting that all previous backtests should be done again as Slippage > 0 has been default.

Last edited by pcz; 11/21/16 16:21.
Re: Are all backtest results potentially flawed? [Re: pcz] #463208
11/21/16 19:03
11/21/16 19:03
Joined: Jun 2016
Posts: 9
Germany, BW
MattY Offline
Newbie
MattY  Offline
Newbie

Joined: Jun 2016
Posts: 9
Germany, BW
Hey, test with this, so you are always on the pessimistic and more real side:

Quote:
Setting Slippage to a negative amount simulates asymmetric slippage that is always in adverse direction of the trade. Asymmetric slippage is illegal, but some trading platforms allow the broker to automatically apply asymmetric slippage for reducing the trader's profit.


I trade with FXCM and the slippage seems to be much greater on the negative side (hope they don't do this on purpose....).

Re: Are all backtest results potentially flawed? [Re: pcz] #463209
11/21/16 21:15
11/21/16 21:15
Joined: Nov 2016
Posts: 4
Taipei
DavidC911 Offline
Guest
DavidC911  Offline
Guest

Joined: Nov 2016
Posts: 4
Taipei
Does it mean that all Z system's results are rather optimistic?

If so, how can we deal with that issue since we can't modify the Slippage or Fill parameter of the Z system?(or can we?)

Re: Are all backtest results potentially flawed? [Re: DavidC911] #463216
11/22/16 08:40
11/22/16 08:40
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
Originally Posted By: DavidC911
Does it mean that all Z system's results are rather optimistic?

If so, how can we deal with that issue since we can't modify the Slippage or Fill parameter of the Z system?(or can we?)


Not necessarily. I've tried the same for my other strategy and greater values of slippage had actually decreased its performance. But I have no clue which strategies might be affected.

Re: Are all backtest results potentially flawed? [Re: pcz] #463218
11/22/16 13:01
11/22/16 13:01
Joined: Nov 2016
Posts: 4
Taipei
DavidC911 Offline
Guest
DavidC911  Offline
Guest

Joined: Nov 2016
Posts: 4
Taipei
Originally Posted By: pcz
Not necessarily. I've tried the same for my other strategy and greater values of slippage had actually decreased its performance. But I have no clue which strategies might be affected.

Thanks for your info, pcz. I'll keep this potential flaw in mind.

Re: Are all backtest results potentially flawed? [Re: DavidC911] #463219
11/22/16 14:07
11/22/16 14:07
Joined: Feb 2014
Posts: 73
Montreal, Qc Canada
F
Finstratech Offline
Junior Member
Finstratech  Offline
Junior Member
F

Joined: Feb 2014
Posts: 73
Montreal, Qc Canada
I tried my own systems and slippage works as expected - decreasing results when slippage increased.

Page 1 of 2 1 2

Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1