Gamestudio Links
Zorro Links
Newest Posts
folder management functions
by VoroneTZ. 04/17/24 06:52
lookback setting performance issue
by 7th_zorro. 04/16/24 03:08
zorro 64bit command line support
by 7th_zorro. 04/15/24 09:36
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:48
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:46
LPDIRECT3DCUBETEXTUR
E9

by Ayumi. 04/12/24 11:00
Sam Foster Sound | Experienced Game Composer for Hire
by titanicpiano14. 04/11/24 14:56
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (firecrest, AndrewAMD), 387 guests, and 3 spiders.
Key: Admin, Global Mod, Mod
Newest Members
11honza11, ccorrea, sakolin, rajesh7827, juergen_wue
19045 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 2 1 2
Bug? Backtest performance getting worse with smaller timeframes #465643
05/07/17 18:50
05/07/17 18:50
Joined: Dec 2014
Posts: 206
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 206
Germany
This strategy uses only simple pending orders and shows a whopping 360% annual profit in H4, but an annual loss in M1.

This is how I thought the backtest works with pending orders:

bar crosses limit order -> simulated trade opened (add spread and slippage)
bar hits stop -> book a loss
bar hits TP -> book a profit
bar hits both TP and SL within the same candle -> book a loss.

Using a smaller Timeframe, some trades that the backtester booked as a loss before, because it wasn't sure if TP or SL was hit first should turn into winners and performance should get even better.

Code:
/*
Mean Reversion Strategy

ATR Band - looks similar to BollingerBands, without widening of the bands during spikes in Volatility

Entry: Pending order at the perimeter of the ATR Band
Exit: TP at the middle of the ATR Band
*/

function run()
{
//settings

set(PLOTNOW);
StartDate = 20100101;
EndDate = 20161231;
LookBack = 500;
BarPeriod = 240; //set this to 60, 15 or even 1 to see what I mean
NumWFOCycles = 8;

//PriceSeries

vars Price = series(price());

//Optimization Parameters

var ATR_multi = 1; //optimize(2,1,4,0.2); //optimization disabled

//static parameters

TimeFrame = 240 / BarPeriod; //keep Timeframe for the ATR Bands at H4 at all times

//ATR Bands

vars middle = series(LowPass(Price,10));
vars upperATR = series(LowPass(Price,10)+ATR_multi*ATR(100));
vars lowerATR = series(LowPass(Price,10)-ATR_multi*ATR(100));

//Trendfilter

// vars Trend = series(LowPass(Price,200)); //Trendfilter disabled

//TradeManagement

Stop = 2*ATR(25); //optimize(2, 1, 6, 1)*ATR(25);
TakeProfit = middle[0];
EntryTime = 1;

//TradeLogic

if (true) //(rising(Trend)) Trendfilter disabled
{
Entry = -lowerATR[0];
reverseLong(1);

}

if (true) //(falling(Trend))
{
Entry = -upperATR[0];
reverseShort(1);
}

PlotBars=300;
plot("upperATR",upperATR,MAIN,GREY);
plot("lowerATR",lowerATR,MAIN,GREY);

}


Re: Bug? Backtest performance getting worse with smaller timeframes [Re: Smon] #465644
05/07/17 19:16
05/07/17 19:16
Joined: Feb 2017
Posts: 369
D
Dalla Offline
Senior Member
Dalla  Offline
Senior Member
D

Joined: Feb 2017
Posts: 369
You cannot change BarPeriod and expect you system to behave the same way.

I think what you are looking for is TICKS mode.
Check here
http://zorro-project.com/manual/en/mode.htm

Basically just use set(TICKS); to enable

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: Dalla] #465653
05/08/17 13:25
05/08/17 13:25
Joined: Dec 2014
Posts: 206
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 206
Germany
@Dalla, I took care of that with this line:

TimeFrame = 240 / BarPeriod;

Pending orders are always calculated in the H4 timeframe.

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: Smon] #465675
05/09/17 11:38
05/09/17 11:38
Joined: Jul 2000
Posts: 27,978
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,978
Frankfurt
There is a general and a detailed answer. The general answer is that bar period and time frame is not exactly the same. There can be huge differences in the backtest and in live trading. The main differences are explained in the manual under "TimeFrame".

For the detailed answer, print logs of both strategy versions and compare the trades. Then you can see exactly where the differences come from.

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: jcl] #465691
05/09/17 20:32
05/09/17 20:32
Joined: Dec 2014
Posts: 206
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 206
Germany
Sorry, I don't get it.

I cite Kris from Robotwealth (I'm a member there): "If you look closely at the two 4-hour plots, you can see a number of trades that lasted for a single bar in the first plot, but which are absent altogether from the last one. Most of these seem to be winning trades, so their absence probably accounts for the difference. I have no explanation for why they are missing, but it does appear to be due to the granularity of the simulation. Definitely a question for the Zorro developers – this is pretty important!"

I looked into the log files and found some interesting differences. I set

StartDate = 20170101;
EndDate = 20170404;

The first strange difference is the date of the first respective trade, which Kris already mentioned:

H4: 2017.01.16 08:00
H1: 2017.01.13 14:00
M1: 2017.01.17 05:36

Here is a trade that was taken in H4 and H1, but not in M1:

H4: 18.01.2017 12:00 exit at the same bar
H1: 18.01.2017 13:00 exit at 15:00
M1: no trade

The trade looks legit on the chart, even if it's scary accurate. I dind't calculate my ATR Bands at this time yet, but I'm willing to do so if I have to to get this riddle solved.


The first trade that comes from the same signal is this short trade and fortunately, it's an interesting one with significant differences. It was very close to SL in between. I pencilled down the stops from the STEPWISE function, as I don't know how to let these write into the logfile yet:

H4: Entry 2017.01.17 04:00 @ 1.06499 Exit 2017.01.18 04:00 @ 1.06978 STOP 1.0730
H1: Entry 2017.01.17 05:00 @ 1.06489 Exit 2017.01.18 02:00 @ 1.06901 STOP 1.0729
M1: Entry 2017.01.17 05:36 @ 1.06466 Exit 2017.01.17 11:29 @ 1.07061 STOP 1.0724

The M1 trade was slightly different Entry with STEPWISE: Entry at 6:25 @ 1.0655

The highest HIGH between Entry and Exit of the trade was the 11:00 (UTC) BAR with 1.07196

M1 got stopped out even if it missed the SL with 4,4 PIPS (maybe due to different price data? I got the HIGH from tradingview.com)
H1 missed the STOP by 5,0 PIPS and didn't get stopped out (due to the 0,6 PIP difference or due to the Timeframe???)

However, I re-read the manual and found the section about BarPeriod and TimeFrame pretty easy to understand and straightfoward. If I set BarPeriod to H1 (60) and Timeframe to 240 / BarPeriod = 4, then my ATR should be calculated just as if I set BarPeriod = 240, as 4*60 = 240. Wrong???

I understand that the backtester will produce very different results with market orders, as it doesn't know how the signal and price will change during an H4 bar. But with pending orders, Entry, SL and TP should have the exact same values from my understanding, because these values only change at 00:00, 04:00, 08:00 and so on...
And even if the backtester can't tell the entry time down to the minute in H4 (of course not), it shouldn't have a hard time to come up with the exact same prices for entries and exits in such a liquid market (EURUSD).


I searched for the email of the helpdesk, but I couldn't find it.

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: Smon] #465704
05/10/17 09:32
05/10/17 09:32
Joined: May 2016
Posts: 180
Prague
pcz Offline
Member
pcz  Offline
Member

Joined: May 2016
Posts: 180
Prague
Btw. the backtest gives a nice profit (at least with my params) but when you look at the performance report it's all due to slippage ($8203 profit and $12474 slp in my case). Setting Fill and Slippage to 0 gives me -$7524 loss. The reasons are described in the manual (in Fill and Slippage sections). I think that the numbers in the second case are more realistic. What do you think, jcl? Which values should give more realistic results?

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: pcz] #465706
05/10/17 10:07
05/10/17 10:07
Joined: Jul 2000
Posts: 27,978
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,978
Frankfurt
Probably neither. When I understand the code right, TakeProfit is set very close to the current price. Thus many bars will trigger Entry and TakeProfit at the same time. You can not test such a system in low resolution. You need TICKS mode and probably T1 data. From what I understand, the code is also strongly dependent on BarPeriod since the price series is derived from it.

Test the system first without any stop or takeprofit for finding the best suited bar period. For a TimeFrame mechanism I see no reason in the code.

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: jcl] #465720
05/10/17 15:14
05/10/17 15:14
Joined: Dec 2014
Posts: 206
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 206
Germany
@pcz, thanks! I wouldn't have noticed the huge slippage.

@jcl, yes, as far as I understand my own code grin and as I intended this system, the price is 1 ATR(100) in H4 away from both TP and SL when the trade gets stopped in.

Quote:
"Thus many bars will trigger Entry and TakeProfit at the same time."


Of course, I'm all with you but again, as I learned from the robotwealth course, Zorro makes sure to not give overly optimistic results. It does so by counting a loss, if SL and TP are both hit within the same candle. Isn't that correct? If it's not, it should be implemented. And if SL isn't hit, well - why doesn't the backtester just calculates the Pips between the former pending order and the TP?

Quote:
"You can not test such a system in low resolution."


I would aggree if we weren't talking about pending limit orders with static SL and TP. I think the backtester isn't well designed if it can be confused with such simple trades. The only thing that probably needs less ifs and elses to simulate are Binary Options.

Quote:
"From what I understand, the code is also strongly dependent on BarPeriod since the price series is derived from it."


I don't understand the difference between Pending Limit orders calculated once per H4 bar or every 240 bars on M1....? From how TimeFrame is explained in the manual I conclude, that TimeFrame acts like I had set a different BarPeriod for all subsequent functions. For how the script is working it shouldn't make a difference if I set BarPeriod = 1 and Timeframe = 240 or BarPeriod = 240 and TimeFrame = 1 (or comment out TimeFrame alltogether).

I'm still learning. This script was just an exercise, so TimeFrame was used for practicing reasons and I used these ATR Bands because I'm using them in my manual trading. I fixed them to H4 in my MT4, to always have the same price levels.

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: Smon] #465721
05/10/17 16:11
05/10/17 16:11
Joined: Jul 2000
Posts: 27,978
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,978
Frankfurt
No. TimeFrame affects only series and trades. But in your script the entry levels and price series are still based on 60 minutes. Please read the manual how to use TimeFrame. Aside from that - entry and exit at the same price tick will not give you an optimistic or pessimistic, but just a meaningless result, as the trade outcome is unknown. Blaming the evil backtester won't help.

It's clear that when you're learning to write scripts and are just experimenting with entries and exits, you probably won't see such problems already in the script. But you can easily see them in the log. That's why you should check the log after running a test. If you do not understand the value of a certain variable, print them also in the log. Then you're not helpless when you get unexpected results, but can see where they come from.

Re: Bug? Backtest performance getting worse with smaller timeframes [Re: jcl] #465746
05/11/17 04:46
05/11/17 04:46
Joined: Dec 2014
Posts: 206
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 206
Germany
But I'm using only series (middle, upperATR, lowerATR).

I just tried

vars atr = series(2*ATR(25));
Stop = atr[0];

=> same result.

I just re-studied Workshop 6 and the specifications of TimeFrame in the manual for the 5th time at least.

The only potential problem with my script I could find was that my price series is declared before TimeFrame is changed.

I switched the order to

TimeFrame = 240 / BarPeriod;
vars Price = series(price());
....

=> same result.

(I assume that this is an H4 Price series now?)

Even before I changed the order, I could see the ATR Bands looking like stairs in lower timeframes, which confirms that they are calculated in H4, and as my entries are equal to these lines, they should stay the same as well, don't they?



Sorry if I keep nagging, but I'm feeling if I don't understand this now, I'm building my knowledge without a foundation which will lead to total confusion and failure.

Page 1 of 2 1 2

Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1