How to parse CSV file with tick data to .t1 correctly?

Posted By: Hlopetz

How to parse CSV file with tick data to .t1 correctly? - 07/31/17 18:37

I'm parsing CSV file with tick data with the script:

string Format = "f,f,i";
void main()
{
int N,Row,Record,Records;
Records = dataParse(1,Format,FileName); // read BBO data

printf("n%d rows read",Records);

dataNew(2,Records,2); // create T1 dataset

for(Record = 0,Row = 0; Record < Records; Record++)
{
T1* t1 = dataStr(2,Row,0); // store record in T1 format
t1->fVal = dataVar(1,Record,2);
t1->time = dataVar(1,Record,1)/86400+25569;
Row++;
}
printf(", %d stored",Row);
dataAppend(3,2,0,Row); // append dataset
if(!wait(0)) return;

dataSave(3,OutName); // store complete dataset
}

The result: 8876715 rows read, 887615 stored. I checked with ZHistoryEditor that it is fine.
However Zorro prints Error 016: Incosistent start/end dates.


In manual about dataparse:

The CSV file can be in ascending or descending chronological order, but the resulting dataset should be always in descending order, i.e. the newest records are at the begin.

So the right order should be done automaticly.
However I see the problem.

Shoud I sort the records before saving to .t1 file?
Posted By: jcl

Re: How to parse CSV file with tick data to .t1 correctly? - 08/01/17 09:45

All historical data needs timestamps. But your file seems not to have timestamps, at least there is no time code in your format string.
Posted By: Hlopetz

Re: How to parse CSV file with tick data to .t1 correctly? - 08/01/17 11:04

the data contain unix time.

The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT)

I simply convert it to Windows DATE format by:

t1->time = dataVar(1,Record,1)/86400+25569

Now I sorted all with ZHistoryEditor and Zorro works on it!

How to do it in the sript?

I tried:
for(Record = Records-1,Row = 0; Record = 0; Record--)

but it does not work.
Posted By: Hlopetz

Re: How to parse CSV file with tick data to .t1 correctly? - 08/01/17 11:44

Dear JCL, one more question.
I would like to backtest the strategy on price based bars that is calculted with the function bar() on tick data. I realy need time and volume for each bar. It would be great if it can be calculted in bar() function. Now it is impossible to do as T1 does not contain the volume, but T2 does.
Would it be possible to implement using T2 in Zorro or all the above can be done with .t8 files?
Posted By: jcl

Re: How to parse CSV file with tick data to .t1 correctly? - 08/01/17 14:13

For the opposite order in the script, either use a '+' in the format string, or call dataSort() afterwards.

For volume, use T6. OHLC can then be set to the same values.
Posted By: JRA

Re: How to parse CSV file with tick data to .t1 correctly? - 05/09/18 13:46

Originally Posted By: jcl
For the opposite order in the script, either use a '+' in the format string, or call dataSort() afterwards.

For volume, use T6. OHLC can then be set to the same values.


Itīs possible to use .t6 data with seconds timestamp? Can a .t6 be handled in a seconds resolution?

Thanks.
Posted By: Spirit

Re: How to parse CSV file with tick data to .t1 correctly? - 05/09/18 19:40

Why not, t6 is a file format, it does not restrict time stamps.
Posted By: JRA

Re: How to parse CSV file with tick data to .t1 correctly? - 05/10/18 07:20

Originally Posted By: Spirit
Why not, t6 is a file format, it does not restrict time stamps.


Yes, I think so, but I never seen a .t6 with seconds timestamp on the ZHistory editor.
And when I try it I got the 057 Error:
" T1 data is required for testing with bar periods less than one minute. The test will run with interpolated price data, but won't be very accurate".





Attached picture format_to_post.PNG
© 2024 lite-C Forums