Gamestudio Links
Zorro Links
Newest Posts
Zorro FIX plugin - Experimental
by flink. 04/21/24 07:12
Data from CSV not parsed correctly
by EternallyCurious. 04/20/24 21:39
M1 Oversampling
by 11honza11. 04/20/24 20:57
Scripts not found
by juergen_wue. 04/20/24 18:51
zorro 64bit command line support
by 7th_zorro. 04/20/24 10:06
StartWeek not working as it should
by jcl. 04/20/24 08:38
folder management functions
by VoroneTZ. 04/17/24 06:52
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
1 registered members (AndrewAMD), 533 guests, and 2 spiders.
Key: Admin, Global Mod, Mod
Newest Members
EternallyCurious, howardR, 11honza11, ccorrea, sakolin
19047 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
How to parse CSV file with tick data to .t1 correctly? #467400
07/31/17 18:37
07/31/17 18:37
Joined: Jul 2016
Posts: 8
H
Hlopetz Offline OP
Newbie
Hlopetz  Offline OP
Newbie
H

Joined: Jul 2016
Posts: 8
I'm parsing CSV file with tick data with the script:

string Format = "f,f,i";
void main()
{
int N,Row,Record,Records;
Records = dataParse(1,Format,FileName); // read BBO data

printf("n%d rows read",Records);

dataNew(2,Records,2); // create T1 dataset

for(Record = 0,Row = 0; Record < Records; Record++)
{
T1* t1 = dataStr(2,Row,0); // store record in T1 format
t1->fVal = dataVar(1,Record,2);
t1->time = dataVar(1,Record,1)/86400+25569;
Row++;
}
printf(", %d stored",Row);
dataAppend(3,2,0,Row); // append dataset
if(!wait(0)) return;

dataSave(3,OutName); // store complete dataset
}

The result: 8876715 rows read, 887615 stored. I checked with ZHistoryEditor that it is fine.
However Zorro prints Error 016: Incosistent start/end dates.


In manual about dataparse:

The CSV file can be in ascending or descending chronological order, but the resulting dataset should be always in descending order, i.e. the newest records are at the begin.

So the right order should be done automaticly.
However I see the problem.

Shoud I sort the records before saving to .t1 file?

Last edited by Hlopetz; 07/31/17 18:38.
Re: How to parse CSV file with tick data to .t1 correctly? [Re: Hlopetz] #467407
08/01/17 09:45
08/01/17 09:45
Joined: Jul 2000
Posts: 27,982
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,982
Frankfurt
All historical data needs timestamps. But your file seems not to have timestamps, at least there is no time code in your format string.

Re: How to parse CSV file with tick data to .t1 correctly? [Re: jcl] #467411
08/01/17 11:04
08/01/17 11:04
Joined: Jul 2016
Posts: 8
H
Hlopetz Offline OP
Newbie
Hlopetz  Offline OP
Newbie
H

Joined: Jul 2016
Posts: 8
the data contain unix time.

The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT)

I simply convert it to Windows DATE format by:

t1->time = dataVar(1,Record,1)/86400+25569

Now I sorted all with ZHistoryEditor and Zorro works on it!

How to do it in the sript?

I tried:
for(Record = Records-1,Row = 0; Record = 0; Record--)

but it does not work.

Last edited by Hlopetz; 08/01/17 11:05.
Re: How to parse CSV file with tick data to .t1 correctly? [Re: Hlopetz] #467412
08/01/17 11:44
08/01/17 11:44
Joined: Jul 2016
Posts: 8
H
Hlopetz Offline OP
Newbie
Hlopetz  Offline OP
Newbie
H

Joined: Jul 2016
Posts: 8
Dear JCL, one more question.
I would like to backtest the strategy on price based bars that is calculted with the function bar() on tick data. I realy need time and volume for each bar. It would be great if it can be calculted in bar() function. Now it is impossible to do as T1 does not contain the volume, but T2 does.
Would it be possible to implement using T2 in Zorro or all the above can be done with .t8 files?

Last edited by Hlopetz; 08/01/17 13:15.
Re: How to parse CSV file with tick data to .t1 correctly? [Re: Hlopetz] #467414
08/01/17 14:13
08/01/17 14:13
Joined: Jul 2000
Posts: 27,982
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,982
Frankfurt
For the opposite order in the script, either use a '+' in the format string, or call dataSort() afterwards.

For volume, use T6. OHLC can then be set to the same values.

Re: How to parse CSV file with tick data to .t1 correctly? [Re: jcl] #472641
05/09/18 13:46
05/09/18 13:46
Joined: Oct 2017
Posts: 17
J
JRA Offline
Newbie
JRA  Offline
Newbie
J

Joined: Oct 2017
Posts: 17
Originally Posted By: jcl
For the opposite order in the script, either use a '+' in the format string, or call dataSort() afterwards.

For volume, use T6. OHLC can then be set to the same values.


It´s possible to use .t6 data with seconds timestamp? Can a .t6 be handled in a seconds resolution?

Thanks.

Re: How to parse CSV file with tick data to .t1 correctly? [Re: JRA] #472644
05/09/18 19:40
05/09/18 19:40
Joined: Sep 2003
Posts: 929
Spirit Offline

Moderator
Spirit  Offline

Moderator

Joined: Sep 2003
Posts: 929
Why not, t6 is a file format, it does not restrict time stamps.

Re: How to parse CSV file with tick data to .t1 correctly? [Re: Spirit] #472649
05/10/18 07:20
05/10/18 07:20
Joined: Oct 2017
Posts: 17
J
JRA Offline
Newbie
JRA  Offline
Newbie
J

Joined: Oct 2017
Posts: 17
Originally Posted By: Spirit
Why not, t6 is a file format, it does not restrict time stamps.


Yes, I think so, but I never seen a .t6 with seconds timestamp on the ZHistory editor.
And when I try it I got the 057 Error:
" T1 data is required for testing with bar periods less than one minute. The test will run with interpolated price data, but won't be very accurate".




Attached Files format_to_post.PNG
Last edited by JRA; 05/10/18 07:30.

Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1