Hi

I am not sure if this can workout but I write it here as an idea to avoid the peeking bias introduced by optimalF parameteres:
The point is to calculate the OptimalF parameters for each test period after each training period of each WFO cycle. Then we would get 1.fac 2.fac and so on as there are 1.par 2.par files for the parameteres of each training cycle. The Test period after each WFO cylce could be splitted in the test period to calculate the OptimalF and then a pure out-of-sample period where the optimalF calculated are applied. So each training cylce has in reality two test periods, one to calculate the optimalF and another to actually test the strategy.
At the end, the whole test would be cover by different out-of-sample periods which use diferent optimalF parameteres.


As well it maybe possible that the robustness of the strategy could be tested by slightly changing the OptimalF parameteres. Something like NumSampleCycles applied to the OptimalF values where at each cycle the OptimalF parameters are slightly different from the previous cycle. Since the strategy will behave slightly different during live trading, the optimalF parameteres would change if that live period would be considered as well into the calculation but that difference should not produce an unprofitable strategy from one that looks profitable in first place.

Well spotted, and you raise a very good point. OptimalF is calculated over the entire simulation period, but is then applied to the simulation from the start. This introduces a weak form of look-ahead bias to strategies that trade using OptimalF for position sizing. I would also like to see separate .fac files for each WFO cycle.

Yes, we can do that. The reason why we calculated OptimalF for the whole period was that this factor needs a lot of data, including all drawdowns, for being useful. The factors calculated from single WFO periods turned out more or less random.

Could the test period be splitted between some to calculate the optimalF and other to applied it?
Or could the NumSampleCycles increase the data so that enough data is used to calculated each fac file?

what about changing slightly the optimalF parameters through some cycles to evaluate how the final performance is affected by that change? I robuts strategy should remain profitable after chaging slightly the optimalF parameters... I guess.

The strategy result has a linear dependence on OptimalF. If you have a portfolio of 4 components, as in the tutorial, and change one of the factors by 10%, the overal result will change by about 2.5%.

A more severe bias problem is excluding unprofitable components that have a zero OptimalF factor. This can affect the result much more.

NumSampleCycles won't help here, but I believe the best idea is still to use different factors in the backtest than in live trading. The factors for the backtest are then calculated from the last cycle, for live trading from the whole period. This will reduce the profit in the backtest, but eliminate the bias.

I see it is kind of tricky.
I was thinking about splitting the test period in one to calculate the optimalF and one to apply it, but it maybe not enough data for that.

The bias when optimalF=0 will be allways there somehow because those assets/algo combinations have a low PF anyways and anyone would take them away during the strategy development since there are allways some assets who do not fit into the behaviour that the strategy wants to exploit.

Another thing is when one get too optimistic optimalF. It can be reduceded so that a trade those not take all the margin as explained in the tutorial, but if the avg optimalF of all nonzero values is x and the standard desvitation of that mean is y, maybe artifitially the optimalF whichs is above that range could be reduced to that value or 2 or 3 times that value. If OptimalF is cero one avoid enter in an asset/algo which could behave good or bad. But if the optimalF is too good, then too optimistic results introduce bias.

This is something that anyone can do. But I do not know if it make sense to reduce artifitially the optimalF is this way for the too optimistic values.

The thread is three months old, nevertheless...
Is there meanwhile a way to obtain more realistic (not selection biased due to OptimalF from the future) test results for Z systems?
I was thinking about the following approach to solve this task at least partially:
- first, delete (move to a different directory) history files for 2016;
- then let Zorro calculate OptimalF on data up to 2014. Copy OptF column from Z1.txt test result file and paste it to .par file;
- then put 2016 history files back where they were; run Test again and check the equity curve during 2016 (the earlier part is biased).

Is it a feasible way?