The new scripts are intended for batch conversion so you can run it on multiple files exported using Tick Downloader with no additional user input. It's more convenient because sometimes the conversion of multiple symbols can take hours or even days. Again I'm using Cygwin to run it. You must have python installed. You can check by running 'which python' command. If it doesn't return a python path, something is wrong.


How to use it - the short version

1. Save conversion bash script and python script into a folder. Edit history_path and zorro_path variables in convert.sh file and set them accordingly. The first variable is the path to the folder containing .csv files with tick data (exported using Tick Downloader).
2. Save T1 data conversion and T6 data conversion scripts in Zorro/Strategy folder and modify the basePath variable. It should be set to the same path as history_path in convert.sh but in Windows path notation style.
3. Run ./convert -p x, where x is the desired period of the converted data in minutes. -p 0 will produce T1 data, -p 1 will produce M1 T6 data with spread stored in fVal variable; higher periods don't work yet.


How to use it - the long version

The first file is a conversion bash script. It's important to edit it and set these two variables correctly: history_path and zorro_path. The first is a path of the folder containing tick data .csv files exported using Tick Downloader (by default they have names like EURUSD_tick.csv etc.). It shouldn't contain any other files otherwise unexpected things might happen Folders are fine. The second variable is a path of the folder containing Zorro executable.

In the same folder as the previously mentioned script you should put this python script used for data preprocessing. It will be called automatically.

Then there are two Zorro strategies, one used for T1 data conversion and one for T6 data conversion. They belong to Zorro/Strategy folder. You have to edit them and set basePath variable correctly (should be the same folder as the previously mentioned history_path but in Windows notation style).

You can run the bash script for example like this:

The first parameter is the desired period of the converted data. If you set it to 0 the conversion script will produce tick data in T1 format. If #define SPREAD directive is defined in the strategy file, two files will be created - one for ask and one for bid prices.

If you set -p to 1 it will produce M1 data in T6 format. Spread for opening price is stored in fVal variable. Higher periods don't work yet

You can use T1 export script to check the converted T1 prices. For T6 you can use the script included in Zorro.

The whole thing has been tested but not very thoroughly, it's an ongoing work. So if you find a bug please let me know.


How it works

The bash script loads list of files in history_path folder. Then it loops through them and calls python script with given parameters on each of them.

The python script reverses the line order of each file, splits them by year and for periods greater than 0 it converts the tick data to OHLC with opening price spread stored in the last column.

After that the bash script takes over again and loads the list of newly split files. It stores the first and the last available year together with each symbol. After that it calls Zorro with these arguments to convert the .csv files to .t1 or .t6 files depending on the chosen period.


Some final warnings

- Don't trust the scripts unless you check the converted data yourself
- Don't use special distributions of Python (e.g. Anaconda) in Cygwin for this task. It might complicate things.
- Don't use periods higher than 1
- Timestamps can be shifted using -s parameter but it works only for T6 data

EDIT: T6 Zorro script fixed (wrong variable names)