Gamestudio Links
Zorro Links
Newest Posts
What are you working on?
by Emre
Today at 01:03
New Zorro version 1.83
by claudio
Yesterday at 17:22
Ahead bias when retrain
by jcl
Yesterday at 12:27
Huge array for Deep Learning
by jcl
Yesterday at 12:06
[LBGUI] Memory not released!
by EVO
05/23/18 22:39
http_post undeclared identifier
by Superku
05/23/18 22:10
Pan/Tilt in relation to entity?
by Superku
05/23/18 20:59
AUM Magazine
Latest Screens
Lake scene
Forgotten Tales - Demo version
RPG PARTY
Triton Wing now available on Steam
Who's Online
8 registered (Slin, Quad, Superku, AndrewAMD, 3run, jyd, 2 invisible), 460 Guests and 3 Spiders online.
Key: Admin, Global Mod, Mod
Newest Members
Rayharry, AgentShark, rpauletto, tuuPaul, LaserJock2000
17960 Registered Users
Page 1 of 3 1 2 3 >
Topic Options
Rate This Topic
#470687 - 01/30/18 15:58 Get a csv file into an array of structs, confused about lite-c
easyX Offline
Newbie

Registered: 11/29/17
Posts: 39
Loc: Germany
I am struggeling to read a file line by line. Playing arround with all kind of funtions, some seem not to work?

the data looks like this:

Code:
2007/01/14,23:50,JPY,M,CoreMachineryOrdersm/m,3.8%,4.2%,2.8%,,03398
2007/01/15,00:30,AUD,M,HomeLoansm/m,-0.6%,0.0%,-0.2%,-0.1%,10266
2007/01/15,02:00,CNY,L,M2MoneySupplyy/y,16.9%,16.7%,16.8%,,24236
2007/01/15,02:00,CNY,M,NewLoans,214B,,194B,,24331
2007/01/15,05:01,USD,N,BankHoliday[AllDay],,,,,03343




First step would be to split the file into it's lines.
After that i want to split the values of each line line to copy them into a struct.

Now there are c funtions i would like to use e.g. getline() or strsep() but both seems to be missing in litec?
Can i add them somehow manually? Or there other function which will work for sure?
Often examples in c also use the term != EOF for loops regarded to reading files, but this also does not work in litec? (sometimes -1 or NULL seems to work instead.

Iam just very new to all of this and i am so confused, maybe i just do it wrong, but maybe it's all because of lite-c limitations? I would appreciate any help/hint!

Edit: just wondering if DataSet handlign would work for the data i got? With dataParse, ... what do you think? And if so how?

Kind Regards

Isaak


Edited by easyX (01/30/18 16:38)

Top
#470689 - 01/30/18 17:26 Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26338
Loc: Frankfurt
dataParse would be ok, unless you need the full strings in the struct. Then you must parse it by script. Clib functions like getline() are normally in the stdio header - if not, you can add them like the other functions in that header.

Top
#470694 - 01/30/18 19:01 Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl]
easyX Offline
Newbie

Registered: 11/29/17
Posts: 39
Loc: Germany
Thanks for the reply, does dataParse has a problem with ",,,," if there are no values sometimes?

What format string will i need for the example?

"+%Y/%m/%d,%H:%M,s,%c, ??? ,%i "

is it %s or s for the 3 character string (currency, aka market)?

iam unsure about the strings with different lenght and also containing characters like "[" or "/" or "%", only thing for sure is that ',' is the delimiter. so maybe %[^,] ? What about the lenght?

for the values after the name could i use f? would this automatic skip any characters like %,B,K (non numeric, no '.') like it behaves with i??



And after parsing...it stores the data somewhere, but how to access this data, e.g. if i want to find values from a certain day, oder by name, or by id (last value). No idea yet

Top
#470701 - 01/31/18 15:47 Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26338
Loc: Frankfurt
I don't know how dataParse will react on irregular or no values - just try it. For generating the format string, follow the suggestions in the manual. You can always check the right format with Verbose = 7 or by exporting the resulting dataset.

Top
#470840 - 02/08/18 14:40 Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl]
Zheka Offline
Member

Registered: 07/03/17
Posts: 294
To access data you use a Handle. It is just an int variable, any number convenient for you.
Using it in dataParse associates the whole dataset with this variable/number.
So, to access data later, just use this variable in functions like dataFind(),dataVar(),etc..

If format for each type of events is constant, then it might be easier to parse one big file into several datasets, per event, with
Quote:
dataParse (int Handle, string Format, string Filename, string Filter): int
parses only lines that contain the string Filter. The string is case senstitive and can cover several adjacent fields, including the delimiters. This way only lines with a certain asset name, year number, or other field content are parsed.

I personally did not try it, so I do not know the requirements for 'string Filter' and how it should look like. Experiment.

To search by date - use dataFind, but do dataSort before.
Searching by other fields will be like working with an array, calling dataVar(row,col), dataInt and dataStr. and comparing to the filter value you need.

Top
#470933 - 02/13/18 14:07 Re: Get a csv file into an array of structs, confused about lite-c [Re: Zheka]
easyX Offline
Newbie

Registered: 11/29/17
Posts: 39
Loc: Germany
Ok thanks for your input, i am going to try a bit more later.

Another thing i am confused right now:

i try to import price data with the CSVtoHistory script, (with an older version i think it worked)

The point is the Data is formated like this (and it's a huge file (500mb+):

1986.12.01,01:00,0.90110,0.90110,0.90110,0.90110,4
1986.12.01,01:01,0.90110,0.90110,0.90110,0.90110,4
1986.12.01,01:02,0.90110,0.90110,0.90110,0.90110,4
1986.12.01,01:03,0.90110,0.90110,0.90110,0.90110,4

i am using this format code: string Format = "+%Y.%m.%d,%H:%M,f3,f1,f2,f4,f6";

To notice is that there are no seconds, since it's M1 data

if i start the parsing script i get an error:

Error 058: Bad date 01:01,1.43
(1.43 is the start of the first price for the file i tried in this example)

That look like dataparse wants to have the seconds for sure? How to solve this? to Edit the files is not very easy for me as well, if it would be smaller i could just change format of time in excel, but it's to big file, would need to split it into 10+ files smirk The Point is, it must have worked like this in version 1.72.x if i remember right, but not anymore.

Top
#470952 - 02/14/18 09:02 Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26338
Loc: Frankfurt
The format string looks syntactically ok. Check if the file has really a header line. Set Verbose = 7 and check the log - there you see the first two parsed lines and can see what's wrong.

Top
#470955 - 02/14/18 10:37 Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl]
easyX Offline
Newbie

Registered: 11/29/17
Posts: 39
Loc: Germany
THe file has no header, with Verbose = 7 it shows only the same as always:

CSVtoHistory compiling...........

Parse HistoryGBPUSD_1_MT4.csv.......
Error 058: Bad date 01:02,1.43 - can't parse GBPUSD_1_MT4.csv
0 lines read

Top
#470957 - 02/14/18 10:44 Re: Get a csv file into an array of structs, confused about lite-c [Re: easyX]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26338
Loc: Frankfurt
If it has no header, format begins with +0.

Top
#470958 - 02/14/18 11:02 Re: Get a csv file into an array of structs, confused about lite-c [Re: jcl]
easyX Offline
Newbie

Registered: 11/29/17
Posts: 39
Loc: Germany
Yeah, noticed that and tested it (also before).


CSVtoHistory compiling...........

Parse HistoryAUDCAD_1_MT4.csv.......
Error 058: Bad date 01:00,0.90 - can't parse AUDCAD_1_MT4.csv
0 lines read


This time i used the file where the above Data sample comes from and added the 0 into the format string.

So it seems data parse really WANT to have seconds? Since it's trying to read it, even without putting it into the format string. Maybe if not in format string it should be 0 by default.


Edited by easyX (02/14/18 12:12)

Top
Page 1 of 3 1 2 3 >



Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de