Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
3 registered members (VoroneTZ, monk12, Quad), 829 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 3 of 5 1 2 3 4 5
Re: bar aggregation problem + Download from FXCM problem [Re: Grat] #470809
02/07/18 12:17
02/07/18 12:17
Joined: Jul 2017
Posts: 783
Z
Zheka Offline
User
Zheka  Offline
User
Z

Joined: Jul 2017
Posts: 783
Quote:
If using the TimeFrame you got different. Try without Timeframe and change only BarPeriod.
I tried and with BarPeriod=5 it works as expected.

It seems TimeFrame formation simply begins from the first available bar in a data file. Timeframes have to be manually coded around anyway to work properly.

Re: bar aggregation problem + Download from FXCM problem [Re: Zheka] #470811
02/07/18 12:45
02/07/18 12:45
Joined: Dec 2017
Posts: 129
Halifax, NS
K
kujo Offline
Member
kujo  Offline
Member
K

Joined: Dec 2017
Posts: 129
Halifax, NS
So what are the results? There is no bug? Only a question about the Timeframe?

Zorro is a great product! Many kudos for the team! There always be bugs. There is no bugs only if you don't develop anything. Saying that, it could be an area for improvement when it comes to quality (for example, Z3 system in the release version of Zorro, I've downloaded 3 release versions just to find out that Z3 isn't working again and again. Sometimes I think that QA doesn't check it at all. I hope I am wrong). As it was mentioned, better attitude will be beneficial for all and make Zorro even more powerful and of an excellent quality. I believe that it's a common goal for the Zorro's team and users.

Re: bar aggregation problem + Download from FXCM problem [Re: kujo] #470812
02/07/18 13:43
02/07/18 13:43
Joined: Jul 2017
Posts: 783
Z
Zheka Offline
User
Zheka  Offline
User
Z

Joined: Jul 2017
Posts: 783
JCL,

this is my communication with support that shows everything:
Quote:

thank you for your email. The Zorro MT4 plugin uses the iTime function for M1 bars, which returns the bar close time according to MQ support.

If you have MT4 data that has different timestamps than the same data from IB, can you send us both?
-----------------------------------------------------------
Me:
iTime returns the bar open time!

This is indeed vague in the English version of MQL manual, but is stated clearly in the Russian.

MT4 in general operates with bar open times. https://docs.mql4.com/predefined/time
-----------------------------------------------------------
Hi,
if you have any evidence that the MQ support was wrong with the iTime function, you can anytime forward it us.


1. Why does a user need to get involved in the implementation of a software? What matters is if it works correctly.

2. To establish that, why can't Support do the quick QA check between MT4 and IB's data?

3. What do I do now after the last message?
- I indicated there is a problem
- I specified the process to replicate (compare to IB data)
- I went to MQL4 manual to confirm that MT4 in general works with bar open times AND that iTime works with bar open times...

IF this is not enough to get someone to simply download both IB's data and MT4's and do the checking firsthand - what else??
(who would blindly trust some files from a user anyway)

My understanding now is that there is a bar formation bug with MT4 plug-in.

Last edited by Zheka; 02/07/18 13:44.
Re: bar aggregation problem + Download from FXCM problem [Re: Zheka] #470813
02/07/18 14:28
02/07/18 14:28
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
I do not understand this problem. You claim to have data that proves a difference of IB and MT4. But you don't want to send it to us. We do not have such data and are not able to produce it, because all our MT4 and IB data contain totally different prices anyway. MT4 generates prices on its server and IB gets them from the market.

So I do not know why you cannot submit the details that we need for looking into the matter. Other users have no problem with that. You find a bug, send us the script or data, we reproduce it, we fix it, or find out that it's no bug. As simple as that. Happens all the time. What is your problem with this procedure? Lengthy discussions or manual interpretation speculations won't get any bug fixed, they will only get you to not being taken seriously anymore at some point.

Re: bar aggregation problem + Download from FXCM problem [Re: jcl] #470824
02/07/18 17:22
02/07/18 17:22
Joined: Jul 2017
Posts: 783
Z
Zheka Offline
User
Zheka  Offline
User
Z

Joined: Jul 2017
Posts: 783
The problem - again - is with the attitude.

I did not want to send the data this time as a matter of principle - because last time I did it (with IB plug-in), it was not taken seriously, and because this time it is really elementary.

IB's and MT4's data(from any STP/ECN broker) differ by only 0.1-0.3pip per side, but price direction is 100% the same.
So, comparing direction and range of bars with the same timestamp between sources will show if there is a problem...
Nothing mysterious. Just needs good, constructive attitude.

Secondly, Support believe(!) iTime function returns bar close time.
But MT4 works with bar open times. It is written in the MQL manual in numerous places, but also specifically for this function.

So, this looks like a possible reason for the error and should be checked and truth found, right?

Instead, I am asked to provide proof of communication with MQ support.
Attitude?


Messages from support like above, absence of QA, support refusing to read the forum - are not normal for people switching from other software.
In the same way, as Zorro's EndMarket=1900 excludes 19:00, while the whole world around (from exchanges to cafes) operates otherwise.

Re: bar aggregation problem + Download from FXCM problem [Re: Zheka] #470829
02/07/18 22:53
02/07/18 22:53
Joined: Dec 2017
Posts: 129
Halifax, NS
K
kujo Offline
Member
kujo  Offline
Member
K

Joined: Dec 2017
Posts: 129
Halifax, NS
No speculation, only facts.

Broker: !Global Prime Pty Ltd

EURUSD M1 history from MT4:


Let's look at this tick:
2018.02.07,23:55,1.22700,1.22705,1.22683,1.22683,170

Now Zorro's history downloaded from this MT4 (time zone shift is 2h):


The same tick:
2018.02.07 21:55 1.227 1.22705 1.22683 1.22683 0 263.8

The timestamp is identical BUT Zorro uses bar close time while MT4 uses bar open time.
So, obviously it's a bug.

If there is a doubt that MT4 uses bar open time, well... here comes the proof:
Proof1:
Quote:
The bar closing time is not considered in the online trading system MetaTrader 4 (formally, the time of the last tick coming within a timeframe or the beginning time of the next timeframe can be considered as the bar closing time, as shown in Fig. 143).

Source: https://book.mql4.com/functions/datetime. Please read the full article if there are still doubts.

Proof2
Let's check iTime function definition but in Russian since MetaQuotes is a Russian company and their RnD is in Russia. I guess tutorial was translated from Russian to English. But in English version it's not directly said that it returns open bar time only indirectly. However, the Russian version explicitly says that it's an open bar time.
Russian version: https://docs.mql4.com/ru/series/itime
Translated with Google: link

Translated section with Google:
Quote:

Return value
The value of the opening time of the bar (indicated by the shift parameter) of the corresponding chart or 0 in case of an error. For more information about the error, you need to call the GetLastError () function.


I forward the link to this post to Support.
Thank you!

Re: bar aggregation problem + Download from FXCM problem [Re: kujo] #470830
02/08/18 01:06
02/08/18 01:06
Joined: Dec 2017
Posts: 129
Halifax, NS
K
kujo Offline
Member
kujo  Offline
Member
K

Joined: Dec 2017
Posts: 129
Halifax, NS
Quick check of FXCM data shows the similar issue. FXCM has an open bar timestamp but Zorro treats it as a close bar timestamp. I assume you will be able to conduct a detailed examination if you need.

Maybe data from the other sources needs to be checked as well.

Last edited by kujo; 02/08/18 01:09.
Re: bar aggregation problem + Download from FXCM problem [Re: kujo] #470835
02/08/18 09:20
02/08/18 09:20
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
We received today several data files and logs from various users with bar issues. So, no need for Zheka's mystery data anymore.

No bar aggregation problem could be found with any of the received data files. So we must still conclude that Zorro bar aggregation is fine.

As to FXCM, it is clearly stated in the API manual that the time stamp is the bar start. However, we received today 2 different FXCM .t6 files, both also different to our own .t6 file, so it appears that the FXCM servers have slightly different history files. Maybe they are generating them local. We will look into this. Until then we will stick to treating FXCM time stamp as bar start.

As to MT4, when we developed the bridge we received a description by MetaQuotes how historical bars are generated. From this description the time stamp is the bar end. We'll contact them again for clarification. Inbetween, if you want to treat it as start time, just edit the Zorro.mq4 and add 60 to the iTime return value. Maybe we'll make this optional.

That's all info that I have about bar aggregation issues and worries.

Re: bar aggregation problem + Download from FXCM problem [Re: jcl] #470841
02/08/18 15:12
02/08/18 15:12
Joined: Dec 2017
Posts: 129
Halifax, NS
K
kujo Offline
Member
kujo  Offline
Member
K

Joined: Dec 2017
Posts: 129
Halifax, NS
I've continued my investigation of FXCM, MT4 and Zorro data.

FXCM provides a tool to download their data: FXCM Historical Data Downloder. It's possible to download both Bid and Ask M1 data (BTW maybe this explains different price history from FXCM: simply Bid and Ask data, more on this below).

As it's known Zorro uses ask prices and bar close time. FXCM uses bar open time.
Data downloaded from FXCM API with Zorro (Ask):
2018.02.07 21:57:00 1.22685 1.22686 1.22662 1.2267 119
2018.02.07 21:56:00 1.22704 1.22707 1.22685 1.22685 170
2018.02.07 21:55:00 1.22678 1.22705 1.22678 1.22704 151
2018.02.07 21:54:00 1.22666 1.22679 1.22663 1.22678 98
2018.02.07 21:53:00 1.22685 1.22696 1.22646 1.22666 301

Data downloaded from FXCM with their tool (Ask):
2/7/2018 21:57:00 1.2267 1.22673 1.22654 1.22655 119
2/7/2018 21:56:00 1.22685 1.22686 1.22662 1.2267 119
2/7/2018 21:55:00 1.22704 1.22707 1.22685 1.22685 170
2/7/2018 21:54:00 1.22678 1.22705 1.22678 1.22704 151
2/7/2018 21:53:00 1.22666 1.22679 1.22663 1.22678 98

Data looks good taking into account that FXCM uses open time and Zorro close time. Good!

Let's continue and compare FXCM bid price, FXCM MT4 price (bid as well) and Zorro's data downloaded from FXCM MT4.
Data downloaded from FXCM with their tool (Bid):
2/7/2018 21:53:00 1.22663 1.22676 1.2266 1.22675
2/7/2018 21:54:00 1.22675 1.22703 1.22675 1.22701
2/7/2018 21:55:00 1.22701 1.22705 1.22682 1.22682
2/7/2018 21:56:00 1.22682 1.22682 1.22662 1.2267
2/7/2018 21:57:00 1.2267 1.2267 1.2265 1.22652

Data exported from FXCM MT4 (bid as well cause MT4 uses bid):
2018.02.07 23:53 1.22662 1.22676 1.2266 1.22675
2018.02.07 23:54 1.22676 1.22702 1.22676 1.22701
2018.02.07 23:55 1.227 1.22705 1.22682 1.22682
2018.02.07 23:56 1.22682 1.22682 1.22662 1.22669
2018.02.07 23:57 1.2267 1.2267 1.22651 1.22652

Data almost identical (small differences less then PIP and also timezone shift: 2h). As we remember, FXCM uses bar open time. Data in MT4 is the same. So, it seems that MT4 uses bar open time as well.

Now data downloaded from FXCM MT4 with Zorro:
2018.02.07 21:53:00 1.22662 1.22676 1.2266 1.22675
2018.02.07 21:54:00 1.22676 1.22702 1.22676 1.22701
2018.02.07 21:55:00 1.227 1.22705 1.22682 1.22682
2018.02.07 21:56:00 1.22682 1.22682 1.22662 1.22669
2018.02.07 21:57:00 1.2267 1.2267 1.22651 1.22652

Data identical to FXCM's and MT4 data. But it should be identical because both FXCM and MT4 uses bar open time and Zorro bar close time.

I hope these results explain the issue.
To sum up: MT4 uses bar open time (as well as FXCM). Zorro downloads data from FXCM API correctly. Zorro's data downloaded from MT4 is shifted because of bar time confusion.

If you have any questions please feel free to ask!

Re: bar aggregation problem + Download from FXCM problem [Re: kujo] #470846
02/08/18 21:20
02/08/18 21:20
Joined: Jan 2018
Posts: 29
L
luisd Offline OP
Newbie
luisd  Offline OP
Newbie
L

Joined: Jan 2018
Posts: 29
gents, wouldn't it be simpler to print a M5 chart from Zorro and compare it visually to any other reliable source (like FXCM)?

If it does not look the same...

anyway, this is my last shot at it. Very last! attached the final evidence.

Attached Files FINAL EVIDENCE FXCM VS ZORRO.png
Page 3 of 5 1 2 3 4 5

Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1