WFO and curve fitting

The manual states that a good rule of thumb for avoiding curve fitting is to generate at least 20 trades per optimization parameter per asset.

I have a system with 10 assets and 5 parameters. Therefore I need at least 1000 trades to have confidence that my system isn't curve fitted.

If I run a WFO test, I assume I need 1000 trades PER CYCLE to get the same level of confidence. Is that correct?

Yes. But it's not 1000 trades, it's 100 per asset. It won't do if you have one asset with 5 trades and another one with 195.

OK, thanks for the reply. That's an important distinction.

What conclusions could I draw if I used a relatively large number of WFO cycles, with say only 30-40 trades per asset, and got good performance in the WFO test? Obviously the concern is that the strategy would be curve fitted to each training cycle, but in my mind this shouldn't matter because each test cycle that makes up the WFO test is out of sample.

Therefore it seems to make sense to test various numbers of WFO cycles and find the number that gives good performance and is stable when the numbers adjacent to it are tested (as we do when optimizing the strategy's parameters). This number would then be used to set the re-train interval in live trading.

Is this a statistically robust approach to WFO testing?

Thought I'd bump this. I've been revisiting this idea and would really like to hear jcl's and other's thoughts.

My thinking is that WFO is not 100% reliable, though by now it might be one of the best tools we have. If you test it in a different period of time than the one you are testing, you probably will find quiet different results. So, it would be interesting to test the strategy in periods when the market has movement up, down, lateral or flat, and see how the system behaves.

Alternatively, if you test it only in the immediate past, say last few years or so, then if your system suffers draw down higher than your WFO results it might be the time to considere in abandoning it. Also you can aply one formula jcl gave. I am pasting his communication:

"When you begin trading an automated system, you normally start in a drawdown. But how can you determine if the drawdown is normal, or if something is wrong and you should pull the plug?

The simplest method is comparing the drawdown with the recommended capital of the simulation. A more precise estimate is this:

E >= C + P*t/y - D*sqrt((t+l)/y)

E = your equity
C = initial capital (= Required Capital)
P = test profit
t = trade time
y = test period
D = test max drawdown
l = length of the drawdown

Example: the backtest shows $8,000 profit after 5 years with $2000 drawdown of 1 year length, and $1500 required capital. You invest it and are down to $1000 after 6 months.

E = 1500 + 8000*0.5/5 - 2000*sqrt(1.5/5) = 1204.

With $1000 you're below the minimum profit, meaning that you should pull out.

Keep in mind that this is also just an estimate and does not take into account that the max drawdown can happen anytime, even right at the beginning. So there may be reasons to stop the system earlier, or later."

All this doesn't answer directly to your question, but it may help to keep thinking about it.

Thanks for posting, Mangal. I don't suppose you know where that formula comes from? It would be interesting to see the derivation and the assumptions behind it.

No, I don't. Jcl posted it sometime ago.