Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (Quad, aliswee), 835 guests, and 5 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 3 1 2 3
Get a csv file into an array of structs, confused about lite-c #470687
01/30/18 14:58
01/30/18 14:58
Joined: Nov 2017
Posts: 39
Germany
E
easyX Offline OP
Newbie
easyX  Offline OP
Newbie
E

Joined: Nov 2017
Posts: 39
Germany
I am struggeling to read a file line by line. Playing arround with all kind of funtions, some seem not to work?

the data looks like this:

Code:
2007/01/14,23:50,JPY,M,CoreMachineryOrdersm/m,3.8%,4.2%,2.8%,,03398
2007/01/15,00:30,AUD,M,HomeLoansm/m,-0.6%,0.0%,-0.2%,-0.1%,10266
2007/01/15,02:00,CNY,L,M2MoneySupplyy/y,16.9%,16.7%,16.8%,,24236
2007/01/15,02:00,CNY,M,NewLoans,214B,,194B,,24331
2007/01/15,05:01,USD,N,BankHoliday[AllDay],,,,,03343




First step would be to split the file into it's lines.
After that i want to split the values of each line line to copy them into a struct.

Now there are c funtions i would like to use e.g. getline() or strsep() but both seems to be missing in litec?
Can i add them somehow manually? Or there other function which will work for sure?
Often examples in c also use the term != EOF for loops regarded to reading files, but this also does not work in litec? (sometimes -1 or NULL seems to work instead.

Iam just very new to all of this and i am so confused, maybe i just do it wrong, but maybe it's all because of lite-c limitations? I would appreciate any help/hint!

Edit: just wondering if DataSet handlign would work for the data i got? With dataParse, ... what do you think? And if so how?

Kind Regards

Isaak

Last edited by easyX; 01/30/18 15:38.
Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX] #470689
01/30/18 16:26
01/30/18 16:26
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
dataParse would be ok, unless you need the full strings in the struct. Then you must parse it by script. Clib functions like getline() are normally in the stdio header - if not, you can add them like the other functions in that header.

Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl] #470694
01/30/18 18:01
01/30/18 18:01
Joined: Nov 2017
Posts: 39
Germany
E
easyX Offline OP
Newbie
easyX  Offline OP
Newbie
E

Joined: Nov 2017
Posts: 39
Germany
Thanks for the reply, does dataParse has a problem with ",,,," if there are no values sometimes?

What format string will i need for the example?

"+%Y/%m/%d,%H:%M,s,%c, ??? ,%i "

is it %s or s for the 3 character string (currency, aka market)?

iam unsure about the strings with different lenght and also containing characters like "[" or "/" or "%", only thing for sure is that ',' is the delimiter. so maybe %[^,] ? What about the lenght?

for the values after the name could i use f? would this automatic skip any characters like %,B,K (non numeric, no '.') like it behaves with i??



And after parsing...it stores the data somewhere, but how to access this data, e.g. if i want to find values from a certain day, oder by name, or by id (last value). No idea yet

Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX] #470701
01/31/18 14:47
01/31/18 14:47
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
I don't know how dataParse will react on irregular or no values - just try it. For generating the format string, follow the suggestions in the manual. You can always check the right format with Verbose = 7 or by exporting the resulting dataset.

Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl] #470840
02/08/18 13:40
02/08/18 13:40
Joined: Jul 2017
Posts: 783
Z
Zheka Offline
User
Zheka  Offline
User
Z

Joined: Jul 2017
Posts: 783
To access data you use a Handle. It is just an int variable, any number convenient for you.
Using it in dataParse associates the whole dataset with this variable/number.
So, to access data later, just use this variable in functions like dataFind(),dataVar(),etc..

If format for each type of events is constant, then it might be easier to parse one big file into several datasets, per event, with
Quote:
dataParse (int Handle, string Format, string Filename, string Filter): int
parses only lines that contain the string Filter. The string is case senstitive and can cover several adjacent fields, including the delimiters. This way only lines with a certain asset name, year number, or other field content are parsed.

I personally did not try it, so I do not know the requirements for 'string Filter' and how it should look like. Experiment.

To search by date - use dataFind, but do dataSort before.
Searching by other fields will be like working with an array, calling dataVar(row,col), dataInt and dataStr. and comparing to the filter value you need.

Re: Get a csv file into an array of structs, confused about lite-c [Re: Zheka] #470933
02/13/18 13:07
02/13/18 13:07
Joined: Nov 2017
Posts: 39
Germany
E
easyX Offline OP
Newbie
easyX  Offline OP
Newbie
E

Joined: Nov 2017
Posts: 39
Germany
Ok thanks for your input, i am going to try a bit more later.

Another thing i am confused right now:

i try to import price data with the CSVtoHistory script, (with an older version i think it worked)

The point is the Data is formated like this (and it's a huge file (500mb+):

1986.12.01,01:00,0.90110,0.90110,0.90110,0.90110,4
1986.12.01,01:01,0.90110,0.90110,0.90110,0.90110,4
1986.12.01,01:02,0.90110,0.90110,0.90110,0.90110,4
1986.12.01,01:03,0.90110,0.90110,0.90110,0.90110,4

i am using this format code: string Format = "+%Y.%m.%d,%H:%M,f3,f1,f2,f4,f6";

To notice is that there are no seconds, since it's M1 data

if i start the parsing script i get an error:

Error 058: Bad date 01:01,1.43
(1.43 is the start of the first price for the file i tried in this example)

That look like dataparse wants to have the seconds for sure? How to solve this? to Edit the files is not very easy for me as well, if it would be smaller i could just change format of time in excel, but it's to big file, would need to split it into 10+ files smirk The Point is, it must have worked like this in version 1.72.x if i remember right, but not anymore.

Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX] #470952
02/14/18 08:02
02/14/18 08:02
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
The format string looks syntactically ok. Check if the file has really a header line. Set Verbose = 7 and check the log - there you see the first two parsed lines and can see what's wrong.

Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl] #470955
02/14/18 09:37
02/14/18 09:37
Joined: Nov 2017
Posts: 39
Germany
E
easyX Offline OP
Newbie
easyX  Offline OP
Newbie
E

Joined: Nov 2017
Posts: 39
Germany
THe file has no header, with Verbose = 7 it shows only the same as always:

CSVtoHistory compiling...........

Parse HistoryGBPUSD_1_MT4.csv.......
Error 058: Bad date 01:02,1.43 - can't parse GBPUSD_1_MT4.csv
0 lines read

Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX] #470957
02/14/18 09:44
02/14/18 09:44
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
If it has no header, format begins with +0.

Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl] #470958
02/14/18 10:02
02/14/18 10:02
Joined: Nov 2017
Posts: 39
Germany
E
easyX Offline OP
Newbie
easyX  Offline OP
Newbie
E

Joined: Nov 2017
Posts: 39
Germany
Yeah, noticed that and tested it (also before).


CSVtoHistory compiling...........

Parse HistoryAUDCAD_1_MT4.csv.......
Error 058: Bad date 01:00,0.90 - can't parse AUDCAD_1_MT4.csv
0 lines read


This time i used the file where the above Data sample comes from and added the 0 into the format string.

So it seems data parse really WANT to have seconds? Since it's trying to read it, even without putting it into the format string. Maybe if not in format string it should be 0 by default.

Last edited by easyX; 02/14/18 11:12.
Page 1 of 3 1 2 3

Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1