Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
3 registered members (Edgar_Herrera, VoroneTZ, Akow), 973 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
EOD data import using dataFromCSV() #471405
03/02/18 20:10
03/02/18 20:10
Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
M
mhdus Offline OP
Junior Member
mhdus  Offline OP
Junior Member
M

Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
I tested a very simple system using 2 external EOD data sets as "indicators":

#include <contract.c>

var ExtInd1() { return dataFromCSV(101,"+%m/%d/%Y,f","Data1.csv",1); }
var ExtInd2() { return dataFromCSV(102,"+%m/%d/%Y,f","Data2.csv",1); }

void run()
{
BarPeriod = 1440;

vars Diff = series(ExtInd1()-ExtInd2());

if(crossUnder(Diff,0))
enterLong();

if(crossOver(Diff,0))
enterShort();
}


Test results were looking too good to be true and I finally realized that Zorro is entering positions at the OPEN of the SAME daily bar the crossovers are signalled based on the external data. However, the external data is from END of day, so effectively Zorro was using indicator data that would not have been known yet in reality, hence the almost perfect results.

Now I wonder if this is just a conceptual misunderstanding of mine or eventually a Zorro bug? Or in other words, how to avoid this from happening? Actually Zorro should open the trade either at the close of the current bar or at the open of the next daily bar... Ok, I could shift the series Diff by 1 bar for the crossOver() condition but this looks like a dirty workaround to me.

Any recommendations how to do this the "correct" way?

Re: EOD data import using dataFromCSV() [Re: mhdus] #471410
03/02/18 21:18
03/02/18 21:18
Joined: Sep 2017
Posts: 235
H
Hredot Offline
Member
Hredot  Offline
Member
H

Joined: Sep 2017
Posts: 235
If your external data is from end of day, then it has no business being used during that same day anyways. End of day data can possibly only be used for next day, which makes shifting your external data back by one day the only sensible thing to do, and not a "dirty workaround".

If you still like the results after testing it that way, all you'll have to do to reproduce the same behavior in live trading is to trade as the market opens. Pretty much a non-issue.

Last edited by Hredot; 03/02/18 21:31.
Re: EOD data import using dataFromCSV() [Re: Hredot] #471413
03/02/18 22:44
03/02/18 22:44
Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
M
mhdus Offline OP
Junior Member
mhdus  Offline OP
Junior Member
M

Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
Thanks. Well, I was at least surprised of this behaviour since EOD data series should be rather common and I did not expect it required such explicit code in my script to avoid peeking into the future.
Maybe it relates to the more general question "which defaults apply when testing and trading with daily bars?". Trades always entering and exiting at the open? And how to change this behaviour? In fact I would still like to use my EOD signal to trade the very same day at the close (in live trading I would then use the indicator data e. g. 3 minutes before the close, of course - yes, it is available by that time). I believe I can achieve this in live trading using BarOffset but any chance to enter/exit trades at the daily bar close in test mode?
There is probably a section in the manual discussing this - I just haven't found it yet. Any hints appreciated.

Re: EOD data import using dataFromCSV() [Re: mhdus] #471423
03/03/18 13:02
03/03/18 13:02
Joined: Sep 2003
Posts: 929
Spirit Offline

Moderator
Spirit  Offline

Moderator

Joined: Sep 2003
Posts: 929
It is here: http://manual.zorro-project.com/fill.htm

Normally it enters at close plus slippage, fill mode 3 enters at open of next day.

Re: EOD data import using dataFromCSV() [Re: Spirit] #471424
03/03/18 15:25
03/03/18 15:25
Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
M
mhdus Offline OP
Junior Member
mhdus  Offline OP
Junior Member
M

Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
That's what I was looking for, thanks!

However, while playing with this I realized that I may have previously misinterpreted my test data. In fact it rather looks like Zorro (I'm using the latest release build 1.74.8) applies a 1-day lag between the actual data in the .t6 file (loaded from AV, verified with the Z History Editor and confirmed with Yahoo Finance).

Test script:

void run()
{
BarPeriod = 1440;
set(PRELOAD|LOGFILE);
if(is(FIRSTINITRUN)) {
assetList("AssetsIB");
assetHistory("SPY",FROM_AV);
}
printf("nTime: %s, Open: %f, Close: %f",strdate("%Y-%m-%d %H:%M:%S"),priceOpen(),priceClose());
}

logfile:

...
[1299: Fri 18-03-02 00:00] (267.70)
Time: 2018-03-02 00:00:00, Open: 271.410004, Close: 267.700012
[1300: Sat 18-03-03 00:00] (269.08)
Time: 2018-03-03 00:00:00, Open: 265.799988, Close: 269.079987

SPY.t6 in Z History Editor: (png image attached)

Is this a serious bug or am I doing something really stupid??

Attached Files ZEditor.png
Last edited by mhdus; 03/03/18 19:54.
Re: EOD data import using dataFromCSV() [Re: mhdus] #471437
03/04/18 11:45
03/04/18 11:45
Joined: Sep 2003
Posts: 929
Spirit Offline

Moderator
Spirit  Offline

Moderator

Joined: Sep 2003
Posts: 929
Well whats the price at midnight 00:00? Its not the future price of next day, its the last price of last day. So all is fine here, otherwise your strategy would be future peeking.

Re: EOD data import using dataFromCSV() [Re: Spirit] #471447
03/04/18 18:00
03/04/18 18:00
Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
M
mhdus Offline OP
Junior Member
mhdus  Offline OP
Junior Member
M

Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
Ok, I could follow that logic if it applied the same way to dataFromCSV(). But there all data is apparently "known" the same day specified by the date in the same row. Referring back to my initial sample script: This script is effectively peeking 1 day into the future due to these two different ways the EOD data is processed. I find this quite counter-intuitive and I bet this is an unnoticed error in many scripts, at least by beginners. It just happened to be very relevant to my trading idea, otherwise I would have never noticed it (before trading it live). I suggest a related note in the dataFromCSV() documentation, at least.

Re: EOD data import using dataFromCSV() [Re: mhdus] #471455
03/05/18 10:34
03/05/18 10:34
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
You're right, one must be careful here. When EOD data contains a date, but no time, the timestamp is the begin of the day. But the data is from the end of the day. That's future peeking. Zorro corrects this automatically for EOD price history, but not for data imported from arbitrary files. This must be done by the user. I'll put a warning in the manual and maybe we'll provide a time offset feature on import in a future version.

Re: EOD data import using dataFromCSV() [Re: jcl] #471456
03/05/18 11:22
03/05/18 11:22
Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
M
mhdus Offline OP
Junior Member
mhdus  Offline OP
Junior Member
M

Joined: Jul 2016
Posts: 93
Düsseldorf, Germany
Sounds good, many thanks.


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1