|
1 registered members (AndrewAMD),
1,014
guests, and 1
spider. |
|
Key:
Admin,
Global Mod,
Mod
|
|
|
Surprising behavior NumWFOCycles = {0,1,2}
#452081
06/02/15 05:49
06/02/15 05:49
|
Joined: Sep 2013
Posts: 504
California
GPEngine
OP
User
|
OP
User
Joined: Sep 2013
Posts: 504
California
|
I have a strategy. Call it "W0". I set up the Training like this
StartDate = 20110101;
EndDate = 20121231;
BarPeriod = 60;
Weekend = 1;
LookBack = 600;
Hedge = 2;
DataSplit = 85;
set(TESTNOW);
set(PARAMETERS+FACTORS);
NumOptCycles = 2;
The backtest results are like Profit 149$ MI 80$ DD 23$ Capital 572$ Trades 13 Win 46% Avg +10.6p Bars 30 AR 169% PF 3.94 SR 3.07 UI 19% R2 0.70 This is my starting point. I add Call this "W1". The behavior of the Training is the same. The performance report is identical. I'm not sure what I was expecting. But, I wanted to point out that these two settings are equivalent
NumWFOCycles = 0; // default[
NumWFOCycles = 1; // equivalent
Alright, on to
< NumWFOCycles = 1;
---
> NumWFOCycles = 2;
call this "W2". The difference in the performance report is interesting. The numbers and date ranges are the same (good!). We gain glyphs indicating per asset x algo performance in our only cycle.
< Out-of-Sample Test w1 portfolio - performance report
---
> Walk-Forward Test w2 portfolio - performance report
3a4
> Simulation period 15.11.2010-31.12.2012
6,7c7,8
< Out of sample test 1868 bars (16 weeks)
< Training period 10585 bars (91 weeks)
---
> WFO test cycles 1 x 1868 bars (16 weeks)
> Training cycles 2 x 10585 bars (91 weeks)
55c56
< Portfolio analysis OptF ProF Win/Loss Wgt%
---
> Portfolio analysis OptF ProF Win/Loss Wgt% Cycles
57,69c58,70
< EUR/USD avg .999 2.37 12/15 95.9
< GBP/USD avg .000 0.00 0/2 -0.6
< USD/JPY avg .499 1.54 6/5 4.7
<
< EUR/USD .999 2.37 12/15 95.9
< EUR/USD:L .999 3.28 7/6 69.6
< EUR/USD:S .999 1.66 5/9 26.3
< GBP/USD .000 0.00 0/2 -0.6
< GBP/USD:L .000 0.00 0/1 -0.5
< GBP/USD:S .000 0.00 0/1 -0.1
< USD/JPY .999 1.54 6/5 4.7
< USD/JPY:L .999 5.23 5/1 9.1
< USD/JPY:S .000 0.32 1/4 -4.4
---
> EUR/USD avg .999 2.37 12/15 95.9 /
> GBP/USD avg .000 0.00 0/2 -0.6 \
> USD/JPY avg .499 1.54 6/5 4.7 X
>
> EUR/USD .999 2.37 12/15 95.9 /
> EUR/USD:L .999 3.28 7/6 69.6 /
> EUR/USD:S .999 1.66 5/9 26.3 /
> GBP/USD .000 0.00 0/2 -0.6 \
> GBP/USD:L .000 0.00 0/1 -0.5 \
> GBP/USD:S .000 0.00 0/1 -0.1 \
> USD/JPY .999 1.54 6/5 4.7 X
> USD/JPY:L .999 5.23 5/1 9.1 /
> USD/JPY:S .000 0.32 1/4 -4.4 \
But, what is surprising about this is the Training time. W0 and W1 finish in 43 seconds. W2 takes almost twice as long, 84 seconds. Looking at the train.log, it appears that W2 (NumWFOCycles=2) spends time to create a second w2.par file. But this calculation has no effect on the performance report compared to NumWFOCycles={0,1}. Can you elaborate?
Last edited by GPEngine; 06/03/15 05:28. Reason: I meant TRAINing time
|
|
|
Re: Surprising behavior NumWFOCycles = {0,1,2}
[Re: jcl]
#452101
06/02/15 16:13
06/02/15 16:13
|
Joined: Sep 2013
Posts: 504
California
GPEngine
OP
User
|
OP
User
Joined: Sep 2013
Posts: 504
California
|
Here's the script.
function run() {
if (is(INITRUN)) {
StartDate = 20110101;
EndDate = 20121231;
BarPeriod = 60;
Weekend = 1;
LookBack = 600;
Hedge = 2;
DataSplit = 85;
set(TESTNOW);
set(PARAMETERS+FACTORS);
NumOptCycles = 2;
set(LOGFILE);
}
while(asset(loop("EUR/USD", "USD/JPY", "GBP/USD"))) {
vars Price = series(price());
var v0 = optimize(200, 20, 500);
var v1 = optimize(v0, v0, 600);
Margin = max(OptimalF, 0.001) * 250.;
vars S0 = series(LowPass(Price, v0));
vars S1 = series(LowPass(Price, v1));
Stop = 2.0 * ATR(100);
Trail = 2.0 * ATR(100);
if (crossOver(S0, S1)) {
exitShort();
enterLong();
}
if (crossOver(S1, S0)) {
exitLong();
enterShort();
}
}
}
Here is a train+test timing breakdown.
w0 & w1
18s Train O1 cycle
24s Train O2 cycle
1s Compute FACTORS & Test
---
43s Total
w2
18s W[1]:O[1]
25s W[1]:O[2]
18s W[2]:O[1]
25s W[2]:O[2]
1s Compute FACTORS & Test
---
87s Total
What's Zorro doing? At least with this script, the training time depends on the number of WFO Cycles. Keeping everything else the same,
NumWFOCycles = 1 takes 43s
NumWFOCycles = 2 takes 87s
NumWFOCycles = 3 takes 111s
NumWFOCycles = 4 takes 128s
NumWFOCycles = 5 takes 142s
Linear fit: Base Training time is 31 seconds and each WFO Cycle adds 24 seconds.
Last edited by GPEngine; 06/03/15 05:27. Reason: I meant TRAINing time
|
|
|
Re: Surprising behavior NumWFOCycles = {0,1,2}
[Re: jcl]
#452135
06/03/15 05:33
06/03/15 05:33
|
Joined: Sep 2013
Posts: 504
California
GPEngine
OP
User
|
OP
User
Joined: Sep 2013
Posts: 504
California
|
Indeed, where I said "backtest" I often meant "training". I have edited both posts. If there is nothing surprising in those posts to you, then I guess there is no problem.
With NumWFOCylces = 2, the first cycle is just like the case where NumWFOCylces = 0 or 1. But Zorro spends an additional Train cycle to produce final parameters, which have no effect on the Performance Report. Is there a clean way to skip this step? The use case is, I am interested in the performance symbols ('.', '/' and '\') for each component, but I am just getting a preliminary evaluation of performance on a large number of potential strategies, not preparing a Strategy for deployment (Trade). So that last cycle is just wasted compute time.
|
|
|
Re: Surprising behavior NumWFOCycles = {0,1,2}
[Re: GPEngine]
#452341
06/10/15 05:08
06/10/15 05:08
|
Joined: Sep 2013
Posts: 504
California
GPEngine
OP
User
|
OP
User
Joined: Sep 2013
Posts: 504
California
|
If I set
function run() {
if (is(INITRUN)) {
StartDate = 20131108;
EndDate = 20150508;
set(LOGFILE+TESTNOW+PARAMETERS+FACTORS);
NumWFOCycles = 5;
if (WFOCycle == NumWFOCycles - 1) {
// Want to skip last WFO because this is a PRELIMINARY_TEST.
WFOCycle += 1;
}
}
...
}
Then I see this error in both the train.log and the test.log
Error 062: Can't open ff6ed7a00e084258a2b6233e6a5acb9b_4.par (rt:2)
|
|
|
Re: Surprising behavior NumWFOCycles = {0,1,2}
[Re: GPEngine]
#452346
06/10/15 08:49
06/10/15 08:49
|
Joined: Jul 2000
Posts: 27,986
Frankfurt
jcl
Chief Engineer
|
Chief Engineer
Joined: Jul 2000
Posts: 27,986
Frankfurt
|
Hmm, it seems you're increasing the wrong cycle - you want to skip cycle 5, not cycle 4 - and then run through the whole cycle, now with a wrong WFOCycle number. This is most likely the reason of the error message.
You have to experiment a little because this is not an official method. You could check if WFOCycle is 5, and then just quit(), or jump to the end of the WFO cycle by setting Bar to the final bar or a very high number. Or increase the WFOCycle in the EXITRUN, not in the INITRUN. All this is experimental, so no guarantee. If you can't find a solution, please post again and I'll try it myself.
|
|
|
|
