Step 1f: Goal: identify modifiers that work wellYes WFO; Yes oversample; Yes maxtrades; Yes all param optimizing; No emodeThe introduction of what I call "modifiers" has been one of the most exciting parts of strategy development for me. These seemingly slight adjustments to trading rules have the capability to make or break a system. If I think about my discretionary trading, it was always these types of rules that were hardest to follow; they add a level of complexity that is hard for a human trader, but easy for Zorro. The other reason I like this section of strategy development is that the sky is the limit -- there are so many ideas to explore. I am still writing modifiers as I think them up and I think this is one area where human creative expression can be very useful in tradebot design.
I have tried to write modifiers in a way that they can be modular, so that I can easily repurpose them for each new tradebot. Potentially each tradebot prefers its own unique combination of modifiers, so they are meant to be easily switched on/off, and leveraging Zorro's optimizer feature (perhaps in some ways not intended).
The process I use in Step 1f is to first identify and record the results of each modifier
individually, and then I try combinations of the most successful. The goal is to try to find the best combination of modifiers that works well together. As with all other parts of this tradebot design, it's important to utilize Zorro's optimizer where applicable (ie, where a parameter value other than "on" or "off" exists) because the intent here is to build a dynamic and global-minded tradebot, one that can swiftly follow rules for multiple assets. Any time I catch myself setting hardcoded parameter values, I have to ask myself "is this something I can let optimizer control in a more dynamic way?"
Also, it's
very important to provide an "estimated best start value" for the optimizer in cases where you will be Train/Test-ing multiple optimized parameters. I will explain below the process I use to learn this value when starting from scratch:
using an incorrect start value can make or break your strategy!!I will walk through each modifier I currently have in the toolbox individually, and give comments. As a goal, I try to keep these extracted as much as possible from the core logic code. Also, I try to avoid making adjustments
within the modifier code for an individual tradebot. Instead, it's preferable to write more dynamic functions that can be controlled from the run() function, or separate helper functions. In this way, my hope is that one day the toolbox of modifiers could be completely extracted into a separate header file that would just be included at the top of new strategies (my coding abilities are not quite at that level yet).
fridayClose() is a switch that provides a simple way to exit all positions prior to the weekend market close. In practice, I have not yet used this tool, because it has not yet shown me that it is statistically worthy in any of the bots I've designed so far. With that said, I am keeping it in the toolbox because it seems like it "should be" a good idea.
function fridayClose(int fridayclose)
{
//allows Friday trading up until NYSE 3pm; close trades and don't allow after this
if(fridayclose && dow() == FRIDAY && lhour(ET) >= 15)
{
exitLong("*");
exitShort("*");
return 1; //condition met; indicate no further trades
}
return 0; //condition not met; safe to take new trades
}
is another modifier tool I have never used, but that I keep in the toolbox nonetheless. It's getting rusty and probably does need some logic adjustment. Perhaps some day I will have a need for something like this and decide to polish it up.
function hourOpen(int hourblockstart, int hourblockend)
{
//blocks new open trades between selected hours
//uses NYSE time, including DST
if ( (lhour(ET) >= hourblockstart) && (lhour(ET) < hourblockend) )
return 0; //between blocked hours, do not allow trade opens
else
return 1; //no conditions met, allow trades by default
}
is a modifier that allows Zorro's optimizer to find the best combination of days for opening new trades. The days that (forex) trades can be opened are: Mon, Tues, Wed, Thurs, Fri, and Sun. I have found, usually to my surprise, that constraining a trade logic to open trades only on specific days can be dramatic. The combo modifiers in my toolbox all support multiple combinations of parameters, and are all compatible with Zorro's optimizer. This
binary table is also helpful when trying to envision what a combo equates to (unless your brain naturally thinks in binary LOL).
function todayOpenCombo(var dayopencombo)
{
//allows optimizer to specify the best combo of days for opens
//bit position 0 = Monday
//bit position 1 = Tuesday
//bit position 2 = Wednesday
//bit position 3 = Thursday
//bit position 4 = Friday
//bit position 5 = Sunday
//given a combination #, the function will return whether
//current dow() is in the combination
int dayopencombobits = dayopencombo+.5; //truncate to rounded int
int today = dow() - 1; //Mon is 0
if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)
if (dayopencombobits & (1 << today)) return 1; //current dow() is in the combo
else return 0; //current dow() not in combo, do not allow trade opens
}
is todayOpenCombo's cousin, and allows Zorro to find the best day combination for closing trades (currently by NYSE 4pm).
function todayCloseCombo(var dayclosecombo)
{
//allows optimizer to specify the best combo of days to close by NYSE 4pm
//bit position 0 = Monday
//bit position 1 = Tuesday
//bit position 2 = Wednesday
//bit position 3 = Thursday
//bit position 4 = Friday
//bit position 5 = Sunday
//given a combination #, the function will determine if we are beyond
//a combo close time, close all trades if necessary, and return 1
//if no further trades allowed today
int dayclosecombobits = dayclosecombo+.5; //truncate to rounded int
int today = dow() - 1; //Mon is 0
if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)
if ((dayclosecombobits & (1 << today)) && lhour(ET) >= 16)
{
exitLong("*");
exitShort("*");
return 1; //current dow() is in the combo; indicate no further trades
}
else return 0; //current dow() not in combo, safe to take new trades
}
is one of my favorite modifiers, because I find myself saying "WOW!" a lot when I utilize this. This tool allows Zorro's optimizer to find the best markets that trades should be opened in: New York, Sydney, Tokyo and/or London.
function marketOpenCombo(var marketopencombo)
{
//allows optimizer to specify best markets to initiate trades
//bit position 0 = New York 8am-5pm Eastern
//bit position 1 = Sydney 5pm-2am Eastern
//bit position 2 = Tokyo 7pm-4am Eastern
//bit position 3 = London 3am-12pm Eastern
//given a combination #, the function will determine if current time is within
//a market part of the combination (returns 1 to allow trading if true)
int marketcombobits = marketopencombo+.5; //truncate to rounded int
if ( (lhour(ET) >=8) && (lhour(ET) <17) && (marketcombobits & (1 << 0)) ) return 1; //inside New York
if ( (lhour(ET) >=17) || (lhour(ET) <2) && (marketcombobits & (1 << 1)) ) return 1; //inside Sydney
if ( (lhour(ET) >=19) || (lhour(ET) <4) && (marketcombobits & (1 << 2)) ) return 1; //inside Tokyo
if ( (lhour(ET) >=3) && (lhour(ET) <12) && (marketcombobits & (1 << 3)) ) return 1; //inside London
return 0; //default - current market not in combination, don't allow trade opens
}
I use a helper function which is like a control interface to the toolbox. In this way, I have again extracted most of this from the core trading logic, which supports my goal of rapid development. On some bots, I have found that I need multiple helper functions: for example, on a bot that trades with 2 core logics, such as the example in
Workshop 6 with both TRND and CNTR logic.
function checkModifiers()
{
int reversedir = 0; //default normal trade direction (0) unless specified otherwise
int fridayclose = 0; //enforce auto-close and no trades after NYSE 3pm Friday
int hourblockstart = 0; //block trade opens beginning at NY hour
int hourblockend = 0; //block trade opens ending at NY hour
int dayopencombo = 63; //combo of days to open; 63=every day
int dayclosecombo = 0; //combo of days to close after NYSE 4pm; 0=none; 63=every day
int marketopencombo = 15; //combo of markets to allow trade opens; 15=every market
if ( (!fridayClose(fridayclose) //close NYSE 3pm on Friday
|| !todayCloseCombo(dayclosecombo) ) //close NYSE 4pm on selected days
&& todayOpenCombo(dayopencombo) //open on selected days only
&& marketOpenCombo(marketopencombo) //open during selected markets only
&& hourOpen(hourblockstart,hourblockend) ) //open during selected hours only
return 1; //ok to place new trades
else
return 0; //no trade, restricted by a modifier
}
Before I can fully implement each modifier though, I need to provide Zorro with an "estimated best start value" so that it can be Train'd appropriately with my other optimizable parameters (ie, core logic and Stop/Trail). To do this, the process I use is as follows:
1. DISABLE WFO temporarily (comment out the NumWFOCycles and DataSplit lines)
2. Hardcode all currently optimizable parameters. I do it this way, for example:
var TimeCycle = 64; //optimize(64,55,75,1,0);
3. Set an optimize on the first modifier to check, such as this:
int marketopencombo = optimize(1,1,15,1);
I don't have any idea what the "estimated best start value" is yet, so I've used just 1.
IMPORTANT: no other parameters should be set to optimize except the modifier at this stage.
4. Now tell Zorro to Train. It will recurse through each value of the modifier, as compared with the known-reasonable values that we hardcoded in all the other parameters. This produces a figure in the Zorro window after Train that can then become your "estimated best start value". I will now store this value in my script like this:
int marketopencombo = 15; //optimize(12,1,15,1);
Shown here: 12 was the best value in that Train, so I'm recording that as part of my optimize statement. It is commented-out because my next step is to check another modifier. The value 15 in this case is the default "all markets" value for this particular modifier.
5. Go back and repeat for each optimizable modifier, starting from step #3. This will allow you to realize the "estimated best start value" for each optimizable modifier.
6. Don't forget to re-ENABLE the WFO that we disabled in step #1. Also re-ENABLE all core parameter and Stop/Trail optimization statements before proceeding.
So putting all this together... the tradebot now looks like the below, after identifying all the "estimated best start values" for each modifier. If you recall, in Step 1f, I will go through each modifier individually and record how it affects the AR%. Then when I have a grasp of those modifiers that seem to work good, I try combining them to find the best mix. For example, to tell Zorro to recurse through and find the best mix of days to open trades, use an optimizer command like:
int dayopencombo = optimize(54,1,63,1); and then refer to the chart on
this page if you're interested to see which days were chosen.
Note that from this point forward, I keep the optimizer running in all the core logic and Stop areas.
function fridayClose(int fridayclose)
{
//allows Friday trading up until NYSE 3pm; close trades and don't allow after this
if(fridayclose && dow() == FRIDAY && lhour(ET) >= 15)
{
exitLong("*");
exitShort("*");
return 1; //condition met; indicate no further trades
}
return 0; //condition not met; safe to take new trades
}
function hourOpen(int hourblockstart, int hourblockend)
{
//blocks new open trades between selected hours
//uses NYSE time, including DST
if ( (lhour(ET) >= hourblockstart) && (lhour(ET) < hourblockend) )
return 0; //between blocked hours, do not allow trade opens
else
return 1; //no conditions met, allow trades by default
}
function todayOpenCombo(var dayopencombo)
{
//allows optimizer to specify the best combo of days for opens
//bit position 0 = Monday
//bit position 1 = Tuesday
//bit position 2 = Wednesday
//bit position 3 = Thursday
//bit position 4 = Friday
//bit position 5 = Sunday
//given a combination #, the function will return whether
//current dow() is in the combination
int dayopencombobits = dayopencombo+.5; //truncate to rounded int
int today = dow() - 1; //Mon is 0
if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)
if (dayopencombobits & (1 << today)) return 1; //current dow() is in the combo
else return 0; //current dow() not in combo, do not allow trade opens
}
function todayCloseCombo(var dayclosecombo)
{
//allows optimizer to specify the best combo of days to close by NYSE 4pm
//bit position 0 = Monday
//bit position 1 = Tuesday
//bit position 2 = Wednesday
//bit position 3 = Thursday
//bit position 4 = Friday
//bit position 5 = Sunday
//given a combination #, the function will determine if we are beyond
//a combo close time, close all trades if necessary, and return 1
//if no further trades allowed today
int dayclosecombobits = dayclosecombo+.5; //truncate to rounded int
int today = dow() - 1; //Mon is 0
if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)
if ((dayclosecombobits & (1 << today)) && lhour(ET) >= 16)
{
exitLong("*");
exitShort("*");
return 1; //current dow() is in the combo; indicate no further trades
}
else return 0; //current dow() not in combo, safe to take new trades
}
function marketOpenCombo(var marketopencombo)
{
//allows optimizer to specify best markets to initiate trades
//bit position 0 = New York 8am-5pm Eastern
//bit position 1 = Sydney 5pm-2am Eastern
//bit position 2 = Tokyo 7pm-4am Eastern
//bit position 3 = London 3am-12pm Eastern
//given a combination #, the function will determine if current time is within
//a market part of the combination (returns 1 to allow trading if true)
int marketcombobits = marketopencombo+.5; //truncate to rounded int
if ( (lhour(ET) >=8) && (lhour(ET) <17) && (marketcombobits & (1 << 0)) ) return 1; //inside New York
if ( (lhour(ET) >=17) || (lhour(ET) <2) && (marketcombobits & (1 << 1)) ) return 1; //inside Sydney
if ( (lhour(ET) >=19) || (lhour(ET) <4) && (marketcombobits & (1 << 2)) ) return 1; //inside Tokyo
if ( (lhour(ET) >=3) && (lhour(ET) <12) && (marketcombobits & (1 << 3)) ) return 1; //inside London
return 0; //default - current market not in combination, don't allow trade opens
}
function checkModifiers()
{
int reversedir = 0; //default normal trade direction (0) unless specified otherwise
int fridayclose = 0; //enforce auto-close and no trades after NYSE 3pm Friday
int hourblockstart = 0; //block trade opens beginning at NY hour
int hourblockend = 0; //block trade opens ending at NY hour
int dayopencombo = 63; //optimize(54,1,63,1); //combo of days to open; 63=every day
int dayclosecombo = 0; //optimize(33,1,63,1); //combo of days to close after NYSE 4pm; 0=none; 63=every day
int marketopencombo = 15; //optimize(12,1,15,1); //combo of markets to allow trade opens; 15=every market
if ( (!fridayClose(fridayclose) //close NYSE 3pm on Friday
|| !todayCloseCombo(dayclosecombo) ) //close NYSE 4pm on selected days
&& todayOpenCombo(dayopencombo) //open on selected days only
&& marketOpenCombo(marketopencombo) //open during selected markets only
&& hourOpen(hourblockstart,hourblockend) ) //open during selected hours only
return 1; //ok to place new trades
else
return 0; //no trade, restricted by a modifier
}
function run()
{
set(PARAMETERS);
StartDate = 20080101;
EndDate = 20130531;
BarPeriod = 15;
LookBack = 600;
if(is(TESTMODE)) NumSampleCycles = 15; //oversampling on Test only, not Train
if (Train) { RollLong = 0; RollShort = 0; } //help prevent asymmetry in parameters & profit factors
DataSplit = 70; //70% training, 30% OOS test
NumWFOCycles = 5;
int maxtrades = 1;
//require minimum 30 trades per WFO cycle or stop training
static int LastWFOCycle = 0, LastNumTrades = 0;
if(Train && (WFOCycle != LastWFOCycle) )
{
if(LastNumTrades > 0 and LastNumTrades < 30)
{
char tradecount[100];
sprintf(tradecount,"Not enough trades per cycle: %d",LastNumTrades);
quit(tradecount);
}
LastWFOCycle = WFOCycle;
}
LastNumTrades = NumWinTotal+NumLossTotal;
//edge trading logic
var TimeCycle = optimize(64,55,75,1,0);
var TimeFactor = optimize(2.6,0.2,5,0.2,0);
//Stop = BarPeriod*PIP; //simple stop level
Stop = ATR(200) * optimize(1.99,1,15,0.5,-3); // allow 3% tolerance for preferring low stop distances
Trail = ATR(200) * optimize(8.5,3,13,0.5);
vars Price = series(price(0));
vars MA1 = series(SMA(Price,TimeCycle));
vars MA2 = series(SMA(Price,TimeCycle*TimeFactor));
if (checkModifiers())
{
//OK to trade, let's evaluate signals then
if (crossOver(MA1,MA2) && rising(MA1))
{
//enterLong(); //standard entry
reverseLong(maxtrades);
}
else if(crossUnder(MA1,MA2) && falling(MA2))
{
//enterShort(); //standard entry
reverseShort(maxtrades);
}
}
}
After checking each modifier individually, combined with all the core optimizable parameters, I'm able to determine which modifiers work best, and can also test combinations of high performers. I record all this info in my spreadsheet.
For the particular bot, I have recorded these AR% figures:
best markets combo: AR 45%
best day open combo: AR 3%
best day close combo: (loss)
friday close option: AR 31%
If you recall from Step 1e, the best AR% I had before applying any modifiers was 33%. Therefore, I'm happy with the improvements that the "best markets combo" modifier has added.
Coming next... Step 1g: equity-curve trading (emode)
