In my design process, I've enabled NumSampleCycles at one stage but I noticed it can have a dramatic effect depending if I enable it during Training or Testing or both. What does this mean?
For example:
--> WFO rolling Train, no oversampling
--> Test, no oversampling
--> good result
--> WFO rolling Train with oversampling on
--> Test with oversampling on
--> bad result
--> WFO rolling Train with no oversampling
--> Test with oversampling on
--> good result
It seems that oversampling is "bad" to use during Training, but what does that mean?
Since a WFO rolling train includes several OOS cycles for optimizing, why would oversampling cause it to produce such poor optimal parameters?
I just want to understand better.
Thanks