Gamestudio Links
Zorro Links
Newest Posts
Data from CSV not parsed correctly
by dr_panther. 05/06/24 18:50
Help with plotting multiple ZigZag
by degenerate_762. 04/30/24 23:23
M1 Oversampling
by 11honza11. 04/30/24 08:16
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
1 registered members (AndrewAMD), 1,135 guests, and 2 spiders.
Key: Admin, Global Mod, Mod
Newest Members
firatv, wandaluciaia, Mega_Rod, EternallyCurious, howardR
19050 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 3 of 5 1 2 3 4 5
Re: price() clarification [Re: Sundance] #435195
01/03/14 19:48
01/03/14 19:48

L
liftoff
Unregistered
liftoff
Unregistered
L



So I am guessing you cant actually test a strategy with tick data in zorro. closest you can come to this is using 1 minute bars.

Re: price() clarification [Re: DdlV] #435197
01/03/14 19:57
01/03/14 19:57

A
acidburn
Unregistered
acidburn
Unregistered
A



Originally Posted By: DdlV
I agree that discussions about what something "ought" to be called are sometimes superfluous. However, agreeing on the definitions of terms is rarely superfluous - in fact, it's usually critical to a useful discussion and good decisions. So, I start there:

Tick: Timestamp, Bid, and Ask provided whenever one of the latter 2 changes in live trading; or provided as historical data from live trading. (Or I suppose could be made up to simulate future trading...)

Bar: Timestamp, Open, Close, High, and Low built from the Ticks of whatever the Bar's historical time period is, possibly ending with the most recent live data, Open coming from the last Tick prior to the start of the time period. Also called Candle.

Is this correct? Is there any other useful term to define?


FWIW, I completely agree with your definitions, and that is how I have seen those terms, always.

I also agree that the proper terminology is essential. Especially in complex topics, and automated trading is certainly complex.

Re: price() clarification [Re: ] #435198
01/03/14 19:58
01/03/14 19:58

A
acidburn
Unregistered
acidburn
Unregistered
A



Originally Posted By: liftoff
So I am guessing you cant actually test a strategy with tick data in zorro. closest you can come to this is using 1 minute bars.


That is correct.

Re: price() clarification [Re: ] #435226
01/04/14 03:05
01/04/14 03:05
Joined: Aug 2013
Posts: 22
P
pipclown Offline
Newbie
pipclown  Offline
Newbie
P

Joined: Aug 2013
Posts: 22
I put together a Python script that parses and counts wicks in the .bar history files. A wick is defined as high price > open price or low price < close price.

Here's the output from that script:
Code:
_Asset | 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013
--------------------------------------------------------------------------------
AUDUSD |   66    78    82    89    87    76     0     0     0    93    93    91 
EURUSD |   71    81    84     0     0     0     0     0     0    96    94    92 
EURCHF |    -     -     -     -     -     -     -     -     -    94    78    87 
GBPUSD |   73    80    85    90    91     0     0     0     0    93    92    92 
 GER30 |    -     -     -     -     -     -     -     0     0     0    88    86 
NAS100 |    -     -     -     -     -     -     -     0     0    76    75    74 
NZDUSD |   67    78    83    86    88    76    81    85    87     -    91     - 
SPX500 |    -     -     -     -     -     -     -     0     0    80    77    76 
 UK100 |    -     -     -     -     -     -     -     0     0    88    83    82 
  US30 |    -     -     -     -     -     -     -     0     0    87    85    84 
USDCAD |   67    79    83    88    88    74     0     0     0    91    89    84 
USDCHF |   78    81    82    87    90     0     0     0     0    93    92    91 
USDJPY |   74    80    82    89    89     0     0     0     0    89    88    94 
 USOil |    -     -     -     -     -     -     -     0     0    82    83    80 
XAGUSD |    -     -     -     -     -     -     -     -     0    89    86    87 
XAUUSD |    -     -     -     -     -     -     -     -     0    93    93    93



Entries marked with a dash ("-") are simply missing data (no .bar file found). Tried to download all .bar files found on the download page.
The numbers represent the wick percentage (num wicks / total num of HLOCs).

The wick percentage can of course vary from each asset and year depending on if the asset is trending or not. But what looks peculiar are the zeros. Seems strange these should be zero?

jcl, do you know why this could be?

Then there's the question what effect this has on backtesting, if any. Rarely do you test on the 1-min bar period IMHO.

I included the script I used. Requires a working Python installation. Run from the History folder.
Code:
import sys
from struct import calcsize, unpack_from
from collections import namedtuple

Tick = namedtuple('Tick', ['open', 'close', 'high', 'low', 'time'])
YEARS = range(2002, 2013 + 1)
ASSETS = [
    'AUDUSD', 'EURUSD', 'EURCHF', 'GBPUSD', 'GER30', 'NAS100',
    'NZDUSD', 'SPX500', 'UK100', 'US30', 'USDCAD', 'USDCHF',
    'USDJPY', 'USOil', 'XAGUSD', 'XAUUSD'
]

def parse_bar_file(filename):
    ticks = open(filename, 'rb').read()
    format = '<ffffd'
    format_size = calcsize(format)
    num_ticks = len(ticks) / format_size
    return [Tick(*unpack_from(format, ticks, format_size * i)) for i in xrange(num_ticks)]

def count_wicks(ticks):
    return len([t for t in ticks if t.high > t.open or t.low < t.close])

def print_summary_matrix():
    print "%6s | %s" % ("Asset", "  ".join(map(str, YEARS)))
    print "-" * 80
    for asset in ASSETS:
        wick_percentages = []
        for year in YEARS:
            try:
                ticks = parse_bar_file("%s_%d.bar" % (asset, year))
            except IOError:
                # Some years are missing
                wick_percentages.append("   - ")
                continue

            num_wicks = count_wicks(ticks)
            if len(ticks) == 0:
                # Some files are empty
                prc = wick_percentages.append("   - ")
                continue

            prc = num_wicks / float(len(ticks))
            wick_percentages.append("%4.0f " % (prc * 100))
        
        print "%6s | %s" % (asset, " ".join(wick_percentages))


print_summary_matrix()


Last edited by pipclown; 01/04/14 03:15.
Re: price() clarification [Re: pipclown] #435231
01/04/14 09:45
01/04/14 09:45
Joined: Jul 2000
Posts: 27,986
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,986
Frankfurt
Yes, that explains your finding. The FXCM API apparently composed price ticks in 2008-2010 in a different way than the ticks from more recent years. Possibly FXCM have filtered price quotes in some way before generating ticks from them.

I had not noticed this previously - I've only checked the 2013 data after your first post. I'll ask FXCM about that. We had downloaded the wickless price files in 2011, using their old API back then, so maybe unfiltered ticks are now available with the new API. We'll then upload new historic files soon.

- For overcoming the apparent confusion of quotes, ticks, bars, and candles, I'll put up a glossary of those terms in the manual. I hope this helps then to understand my clumsy technical explanations.

Re: price() clarification [Re: jcl] #435240
01/04/14 14:55
01/04/14 14:55
Joined: Aug 2013
Posts: 22
P
pipclown Offline
Newbie
pipclown  Offline
Newbie
P

Joined: Aug 2013
Posts: 22
Cool, thanks jcl.

Re: price() clarification [Re: pipclown] #435242
01/04/14 15:41
01/04/14 15:41
Joined: May 2013
Posts: 627
Bonn
Sundance Offline
User
Sundance  Offline
User

Joined: May 2013
Posts: 627
Bonn
Most appreciated JCL!

Re: price() clarification [Re: DdlV] #435286
01/05/14 17:27
01/05/14 17:27

A
acidburn
Unregistered
acidburn
Unregistered
A



Originally Posted By: DdlV

Bar: Timestamp, Open, Close, High, and Low built from the Ticks of whatever the Bar's historical time period is, possibly ending with the most recent live data, Open coming from the last Tick prior to the start of the time period. Also called Candle.


Actually, I see one mistake in the above. If Open came from the last tick prior to the start of the time period, then we would never have gaps. But we do. Open comes from the first tick in the new time period. At least that's how I see it.

Re: price() clarification [Re: ] #435289
01/05/14 18:19
01/05/14 18:19
Joined: Jun 2013
Posts: 1,609
D
DdlV Offline
Serious User
DdlV  Offline
Serious User
D

Joined: Jun 2013
Posts: 1,609
Thanks acidburn! True - Open is a bit more complicated - something like:

if((Market has been closed) or (Tick time == Bar start time)) Open = Bar's first Tick price;
else Open = last Bar Close;

???

Re: price() clarification [Re: DdlV] #435293
01/05/14 18:39
01/05/14 18:39
Joined: Sep 2013
Posts: 504
California
G
GPEngine Offline
User
GPEngine  Offline
User
G

Joined: Sep 2013
Posts: 504
California
Great script, pipclown. I used this to identify some problems in my History data.

Can we think of some other criteria to add to this Python script, to make it more of a sanity check?
How about
- starts near jan 1
- ends near dec 31, unless current year.
- no gaps > 3 days

Page 3 of 5 1 2 3 4 5

Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1