0 registered members (),
1,397
guests, and 7
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
Really really annoying problem with histories
#464377
02/08/17 15:45
02/08/17 15:45
|
Joined: Apr 2013
Posts: 35
Joaquin
OP
Newbie
|
OP
Newbie
Joined: Apr 2013
Posts: 35
|
I keep strugling with my Pairs Trading Strategy. This time I a suffering a problem with what I think it is a problem with histories. I download my histories from an Oanda Demo account, which runs much better and faster than MT4. This is a snippet of the code I am running:
function run()
{
BarPeriod =1;
StartDate = 20160615;
EndDate = 20170201;
LookBack = 1000;
UnstablePeriod = 50;
vars pairRatio, priceClose1, priceClose2;
//---------1 ----------//
asset(asset1);
Spread = Slippage = RollShort = RollLong = 0;
priceClose1 = series(log(priceClose()) / log(10));
//---------2 ----------//
asset(asset2);
Spread = Slippage = RollShort = RollLong = 0;
priceClose2 = series(1/log(priceClose()) / log(10));
pairRatio = series((priceClose1[0]) * (priceClose2[0]));
if(is(FIRSTINITRUN)) {
while(loop(asset1,asset2)) assetHistory(Loop1,1);
}
int halfLife = LookBack;
vars zScore = series((pairRatio[0] - SMA(pairRatio,halfLife)) / StdDev(pairRatio, halfLife));
if (Bar > LookBack) printf("nzScore: %f",zScore[0]);
With this code snippet, it runs fine in "Trade" mode for LookBack smaller than 1000 (or so, I don't know exactly where is the frontier). If I try with LookBack = 2000 and bigger, then I get
zScore: -1.#IND00
zScore: -1.#IND00
zScore: -1.#IND00
zScore: -1.#IND00
zScore: -1.#IND00
zScore: -1.#IND00
zScore: -1.#IND00
zScore: -1.#IND00
[2093: Wed 08.02.17 14:55] 0.99387
zScore: -1.#IND00
Logout.. ok
I guess there must be a problem with histories: either I don't have enough histories, or there's a gap and at some point one of the asset's price is zero, or something like that. I would like to know what could I do to 1) First, to be sure that this issue is due to a histories problem 2) Second, if it is actually a history problem, to solve it somehow... like using latest known price if there's a gap, or another intermediate solution. If I run this same script in Trading mode with Oanda, it runs wonderfully. I think that what is happening is that MT4 history sucks horribly, and when I'm in trading mode with MT4 accounts, it loads MT4 history, it does not get correct histories, and then it gets this error... How do I solve this nightmare?
|
|
|
Re: Really really annoying problem with histories
[Re: Joaquin]
#464385
02/09/17 09:24
02/09/17 09:24
|
Joined: Apr 2013
Posts: 35
Joaquin
OP
Newbie
|
OP
Newbie
Joined: Apr 2013
Posts: 35
|
I don't like to leave my researching projects abandoned, so I just keep looking for an answer of what is happening with the histories. It's very difficult to explain in just a few lines the strange things that I have seen. But I will try. As I keep growing LookBack, probabilities of getting an error are higher and higher. I've noticed that MT4 passes histories to Zorro in batches of 300 candles each time. This is the log I get in MT4 with LookBack = 7000 and BarPeriod = 1, loading 2 assets:
2017.02.09 10:01:01.481 Zorro AUDUSD,M1: Zorro Disconnected!
2017.02.09 10:01:00.172 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.02 15:53 (6886 - 7186)
2017.02.09 10:00:59.828 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.02 20:53 (6585 - 6885)
2017.02.09 10:00:59.500 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.03 01:54 (6285 - 6585)
2017.02.09 10:00:59.188 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.03 06:53 (5986 - 6286)
2017.02.09 10:00:58.860 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.03 11:53 (5686 - 5986)
2017.02.09 10:00:58.563 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.03 16:55 (5387 - 5687)
2017.02.09 10:00:58.282 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.03 21:54 (5088 - 5388)
2017.02.09 10:00:58.000 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.06 03:04 (4788 - 5088)
2017.02.09 10:00:57.688 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.06 08:03 (4489 - 4789)
2017.02.09 10:00:57.406 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.06 13:02 (4190 - 4490)
2017.02.09 10:00:57.125 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.06 18:01 (3891 - 4191)
2017.02.09 10:00:56.844 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.06 23:01 (3591 - 3891)
2017.02.09 10:00:56.578 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.07 04:02 (3292 - 3592)
2017.02.09 10:00:56.281 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.07 09:01 (2993 - 3293)
2017.02.09 10:00:55.985 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.07 14:01 (2693 - 2993)
2017.02.09 10:00:55.672 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.07 19:00 (2394 - 2694)
2017.02.09 10:00:55.375 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.08 00:01 (2094 - 2394)
2017.02.09 10:00:55.110 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.08 05:04 (1795 - 2095)
2017.02.09 10:00:54.766 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.08 10:03 (1496 - 1796)
2017.02.09 10:00:54.485 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.08 15:02 (1197 - 1497)
2017.02.09 10:00:54.141 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.08 20:02 (897 - 1197)
2017.02.09 10:00:53.797 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.09 01:02 (598 - 898)
2017.02.09 10:00:53.438 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.09 06:01 (299 - 599)
2017.02.09 10:00:53.063 Zorro AUDUSD,M1: Send 300 USDCHF bars at 2017.02.09 11:00 (0 - 300)
2017.02.09 10:00:52.563 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.02 16:05 (6881 - 7181)
2017.02.09 10:00:52.203 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.02 21:04 (6582 - 6882)
2017.02.09 10:00:51.875 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.03 02:03 (6283 - 6583)
2017.02.09 10:00:51.532 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.03 07:02 (5984 - 6284)
2017.02.09 10:00:51.203 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.03 12:01 (5685 - 5985)
2017.02.09 10:00:50.891 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.03 17:04 (5385 - 5685)
2017.02.09 10:00:50.594 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.03 22:04 (5085 - 5385)
2017.02.09 10:00:50.281 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.06 03:11 (4786 - 5086)
2017.02.09 10:00:50.000 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.06 08:10 (4487 - 4787)
2017.02.09 10:00:49.719 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.06 13:09 (4188 - 4488)
2017.02.09 10:00:49.422 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.06 18:08 (3889 - 4189)
2017.02.09 10:00:49.125 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.06 23:08 (3589 - 3889)
2017.02.09 10:00:48.844 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.07 04:08 (3290 - 3590)
2017.02.09 10:00:48.547 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.07 09:07 (2991 - 3291)
2017.02.09 10:00:48.250 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.07 14:06 (2692 - 2992)
2017.02.09 10:00:47.969 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.07 19:05 (2393 - 2693)
2017.02.09 10:00:47.685 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.08 00:05 (2094 - 2394)
2017.02.09 10:00:47.364 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.08 05:04 (1795 - 2095)
2017.02.09 10:00:46.921 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.08 10:03 (1496 - 1796)
2017.02.09 10:00:46.530 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.08 15:02 (1197 - 1497)
2017.02.09 10:00:46.189 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.08 20:02 (897 - 1197)
2017.02.09 10:00:45.824 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.09 01:02 (598 - 898)
2017.02.09 10:00:45.449 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.09 06:01 (299 - 599)
2017.02.09 10:00:45.188 Zorro AUDUSD,M1: Send 300 EURUSD bars at 2017.02.09 11:00 (0 - 300)
2017.02.09 10:00:45.078 Zorro AUDUSD,M1: Zorro Connected!
The histories are there. I would not tell that the problem is in bad MT4 histories. The problem, I think, is in the DLL, in the "bridge" between MT4 and Zorro. I think that this bridge tries in some way to synchronize both timeseries, and because there are not exactly the same number of candles for the same period of time, the script disconnects in the MT4 end, and in the Zorro end it just says that it lacks X candles. This is a problem. Another one is that between different runs of the same script, sometimes it works, sometimes it doesn't, sometimes it says that it lacks X candles and other times it says it lacks Y candles. I think that when LookBack is long enough and you are loading several assets, there should be a better check of histories correctly synchronized. This test could be done in the MT4 end (the DLL) or in the Zorro part. Personally, I would do it in the DLL... The sooner the problem is solved, the better.
|
|
|
Re: Really really annoying problem with histories
[Re: Joaquin]
#464391
02/09/17 17:08
02/09/17 17:08
|
Joined: Apr 2013
Posts: 35
Joaquin
OP
Newbie
|
OP
Newbie
Joined: Apr 2013
Posts: 35
|
Well, I have solved something... I just noticed that the error I was getting with my zScore was because some candles before this error I was getting something like #INF00 error, and I assumed that there was a division by zero somewhere. And yes, in fact there's a division of the two price series. It turns out that priceClose() sometimes returns zero. Don't ask me why, because I think it shouldn't. So I've just coded an indicator to eliminate these zeroes:
var myFilter(var* Data, double number)
{
int __i = 0;
while (Data[__i] == number) __i++;
if (Data[__i] != number) return(Data[__i]);
}
Passing the price series as first parameter and 0.0 as second parameter I have solved this error of zero division. Also, I had a lot of outliers in the zScore, maybe due to incredibly huge gaps in history or just due to flash crashes. To eliminate these outliers, I've written another function:
var removeOutliers(var* Data, double number)
{
int __i = 0;
while (abs(Data[__i]) >= abs(number)) __i++;
if (abs(Data[__i]) <= abs(number)) return(Data[__i]);
}
being "number" the outlier level you want to chop off. Previously I had seen in a chart what I had in Data, in order to see intuitively where I should set the threshold for the outliers. I'm happier now, after having found a solution to some of my problems. I will keep on my research with Zorro Trader, I think it is one the best options I have for doing multiasset backtests...
|
|
|
|