Gamestudio Links
Zorro Links
Newest Posts
lookback setting performance issue
by 7th_zorro. 04/16/24 03:08
folder management functions
by 7th_zorro. 04/15/24 10:10
zorro 64bit command line support
by 7th_zorro. 04/15/24 09:36
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:48
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:46
LPDIRECT3DCUBETEXTUR
E9

by Ayumi. 04/12/24 11:00
Sam Foster Sound | Experienced Game Composer for Hire
by titanicpiano14. 04/11/24 14:56
SGT_FW
by Aku_Aku. 04/10/24 16:09
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (7th_zorro, Quad), 373 guests, and 3 spiders.
Key: Admin, Global Mod, Mod
Newest Members
11honza11, ccorrea, sakolin, rajesh7827, juergen_wue
19045 Registered Users
Previous Thread
Next Thread
Print Thread
Rating: 5
Page 2 of 2 1 2
Re: DT design process: step 1: identify the edge [Re: royal] #432212
11/02/13 12:51
11/02/13 12:51
Joined: May 2013
Posts: 627
Bonn
Sundance Offline
User
Sundance  Offline
User

Joined: May 2013
Posts: 627
Bonn
Just in short. It seems you have spend many hours to get this result. I think that everybody can profit from the way you took this aim.

When i have enough time i also will read your posts.

As a result for everyone is that a profitable strategy can't be seen from only programming some entry rule. Make some WFOs and hope to see some profits.

I remember that somebody says that it is not the most important thing when to enter a trade but to know when to exit. Even when you enter trades randomly you still can be profitable....


happy WE to you all

Re: DT design process: step 1: identify the edge [Re: Sundance] #432220
11/02/13 16:04
11/02/13 16:04
Joined: Jun 2013
Posts: 1,609
D
DdlV Offline
Serious User
DdlV  Offline
Serious User
D

Joined: Jun 2013
Posts: 1,609
Wow! Thanks dusktrader for taking the time to share in such detail what's obviously the result of much time and trial & error. And here I am still stuck "on the edge"... laugh Like others, I will absorb as time permits; and hopefully the time will come when I can return you something equally useful...

Thanks again.

Re: DT design process: step 1: identify the edge [Re: Sundance] #432222
11/02/13 16:09
11/02/13 16:09
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
Thanks very much for your feedback guys! I've been working on polishing the logic a bit further and I've been able to extract a few more routines into helper functions (that keeps the run() function cleaner and easier to follow). I'll post the code below.

Next I'm going to work on dynamic switching of parameter ranges depending on the asset. My thinking is that if I incorporate a reasonable parameter range that is also personalized for each asset... that it could have a major impact.

Code:
function fridayClose(int fridayclose)
{
	//allows Friday trading up until NYSE 3pm; close trades and don't allow after this
	if(fridayclose && dow() == FRIDAY && lhour(ET) >= 15) 
		{
			exitLong("*");
			exitShort("*");
			return 1; //condition met; indicate no further trades
		}
	return 0; //condition not met; safe to take new trades
}

function hourOpen(int hourblockstart, int hourblockend)
{
	//blocks new open trades between selected hours
	//uses NYSE time, including DST
	if ( (lhour(ET) >= hourblockstart) && (lhour(ET) < hourblockend) )
		return 0; //between blocked hours, do not allow trade opens
	else
		return 1; //no conditions met, allow trades by default
}

function todayOpenCombo(var dayopencombo)
{
	//allows optimizer to specify the best combo of days for opens
	//bit position 0 = Monday
	//bit position 1 = Tuesday
	//bit position 2 = Wednesday
	//bit position 3 = Thursday
	//bit position 4 = Friday
	//bit position 5 = Sunday
	//given a combination #, the function will return whether
	//current dow() is in the combination

	int dayopencombobits = dayopencombo+.5; //truncate to rounded int
	int today = dow() - 1; //Mon is 0
	if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)

	if (dayopencombobits & (1 << today)) return 1; //current dow() is in the combo
		else return 0; //current dow() not in combo, do not allow trade opens
}

function todayCloseCombo(var dayclosecombo)
{
	//allows optimizer to specify the best combo of days to close by NYSE 4pm
	//bit position 0 = Monday
	//bit position 1 = Tuesday
	//bit position 2 = Wednesday
	//bit position 3 = Thursday
	//bit position 4 = Friday
	//bit position 5 = Sunday
	//given a combination #, the function will determine if we are beyond
	//a combo close time, close all trades if necessary, and return 1
	//if no further trades allowed today

	int dayclosecombobits = dayclosecombo+.5; //truncate to rounded int
	int today = dow() - 1; //Mon is 0
	if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)

	if ((dayclosecombobits & (1 << today)) && lhour(ET) >= 16) 
	{
		exitLong("*");
		exitShort("*");
		return 1; //current dow() is in the combo; indicate no further trades
	}
	else return 0; //current dow() not in combo, safe to take new trades
}

function marketOpenCombo(var marketopencombo)
{
	//allows optimizer to specify best markets to initiate trades
	//bit position 0 = New York 8am-5pm Eastern
	//bit position 1 = Sydney 5pm-2am Eastern
	//bit position 2 = Tokyo 7pm-4am Eastern
	//bit position 3 = London 3am-12pm Eastern
	//given a combination #, the function will determine if current time is within
	//a market part of the combination (returns 1 to allow trading if true)
	
	int marketcombobits = marketopencombo+.5; //truncate to rounded int
	if ( (lhour(ET) >=8) && (lhour(ET) <17) && (marketcombobits & (1 << 0)) ) return 1; //inside New York
	if ( (lhour(ET) >=17) || (lhour(ET) <2) && (marketcombobits & (1 << 1)) ) return 1; //inside Sydney
	if ( (lhour(ET) >=19) || (lhour(ET) <4) && (marketcombobits & (1 << 2)) ) return 1; //inside Tokyo
	if ( (lhour(ET) >=3) && (lhour(ET) <12) && (marketcombobits & (1 << 3)) ) return 1; //inside London
	return 0; //default - current market not in combination, don't allow trade opens
}

function checkEquity(var emode)
{
	//emode 1 = standard: sets phantom/normal mode only (via Lots)
	//emode 2 = switch hitter: always in market (Lots=1), fades direction (via dir)
	//emode 3 = reward success with weighting: increase trades based on degree of improvement
	//emode 4 = mean reversion: trade when equity curve falls (Lots=1), sit out when it rises (Lots=-1)
	vars EquityCurve = series(EquityLong+EquityShort); //includes all phantom equity
	var dir; //indicates normal trade direction (dir=1) or reverse (dir=-1)

	//narrower curves
	//var slow = 50;
	//var fast = 10;

	//wider curves
	//var slow = 100;
	//var fast = 10;

	//mega-wide curves
	var slow = 200;
	var fast = 10;

	//uber-wide curves
	//var slow = 300;
	//var fast = 10;

	//optimized curves
	//var slow = optimize(50,50,300,12);
	//var fast = 10;

	vars EquityLP = series(LowPass(EquityCurve,fast));
	var EquityLPfalling = LowPass(EquityLP,slow);
	var EquityLPrisingBigger = LowPass(EquityLP,slow*3.2);
	var EquityLPrisingBig = LowPass(EquityLP,slow*1.5);
	//plot("EquityLPslow",LowPass(EquityLP,slow),1,BLUE);
	//plot("EquityLPfast",LowPass(EquityLP,fast),0,GREEN);
	
	if(EquityLP[0] < EquityLPfalling && falling(EquityLP)) { //drawdown
		if (emode==1) Lots = -1; //set phantom trade mode
		if (emode==2) return 1; //fade: take signals when losing
		if (emode==3) { //reward success with weighting
			Lots = -1; //set phantom trade mode
			return 1; //allow max 1 phantom trade in drawdown
		}
		if (emode==4) Lots = 1; //mean-reversion: start trading when equity curve falls
		
	}
	else { //positive equity curve
		if (emode==1) Lots = 1; //set normal trade mode
		if (emode==2) return -1; //fade: take reverse signals when winning
		if (emode==3) { //reward success with weighting
			Lots = 1; //set normal trade mode
			if (EquityLP[0] > EquityLPrisingBigger && rising(EquityLP)) return 3; //very big rising
			else if (EquityLP[0] > EquityLPrisingBig && rising(EquityLP)) return 2; //big rising
			else return 1; //rising but not yet significantly
		}
		if (emode==4) Lots = -1; //mean-reversion: stop trading when equity curve rises
	}
}

function checkTradesPerCycle()
{
	//require minimum 30 trades per WFO cycle or stop training
	static int LastWFOCycle = 0, LastNumTrades = 0;
	if(Train && (WFOCycle != LastWFOCycle) )
	{
		if(LastNumTrades > 0 and LastNumTrades < 30)
		{
			char tradecount[100];
			sprintf(tradecount,"Not enough trades per cycle: %d",LastNumTrades);
			quit(tradecount);
		}
		LastWFOCycle = WFOCycle;
	}
	LastNumTrades = NumWinTotal+NumLossTotal;
}

function calculateMargin(int direction)
{
	//calculate risk Margin based on OptimalF and trade direction
	Capital = 1000; //simulated account balance
	var riskCapital = 600; //basis to trade with
	
	if (direction && OptimalF>.001) //long trade, historically profitable
		return OptimalFLong * riskCapital;
	else if (!direction && OptimalF>.001) //short trade, historically profitable
		return OptimalFShort * riskCapital;
	else if (is(TRADEMODE)) //non-historically profitable = phantom live trades only
		Lots = -1;

	return 0; //no Margin allocated for non-historically profitable
}

function checkModifiers()
{
	int reversedir = 0; //default normal trade direction (0) unless specified otherwise
	int fridayclose = 0; //enforce auto-close and no trades after NYSE 3pm Friday
	int hourblockstart = 0; //block trade opens beginning at NY hour
	int hourblockend = 0; //block trade opens ending at NY hour
	int dayopencombo = 63; //optimize(54,1,63,1); //combo of days to open; 63=every day
	int dayclosecombo = 0; //optimize(33,1,63,1); //combo of days to close after NYSE 4pm; 0=none; 63=every day
	int marketopencombo = optimize(12,1,15,1); //combo of markets to allow trade opens; 15=every market

	if ( (!fridayClose(fridayclose) //close NYSE 3pm on Friday
		|| !todayCloseCombo(dayclosecombo) ) //close NYSE 4pm on selected days
		&& todayOpenCombo(dayopencombo) //open on selected days only
		&& marketOpenCombo(marketopencombo) //open during selected markets only
		&& hourOpen(hourblockstart,hourblockend) ) //open during selected hours only
			return 1; //ok to place new trades
	else
		return 0; //no trade, restricted by a modifier	
}


function run()
{
	set(PARAMETERS+FACTORS);
	StartDate = 20080101;
	EndDate = 20130531;
	BarPeriod = 15;
	LookBack = 600;
	DataSplit = 70; //70% training, 30% OOS test
	NumWFOCycles = 5;
	if(is(TESTMODE)) NumSampleCycles = 15; //oversampling on Test only, not Train
	if (Train) { RollLong = 0; RollShort = 0; } //help prevent asymmetry in parameters & profit factors
	checkTradesPerCycle(); //stop Train early if not enough trades
	int maxtrades = 1;
	int reinvestprofits = 1; //invoke margin setting during trade logic

	//equity-curve trading
	checkEquity(1); //emode 1: normal/phantom trading
	//reversedir = checkEquity(2); //emode 2: switch hitter
	//maxtrades = checkEquity(3); //emode 3: reward success
	//checkEquity(4); //emode 4: mean-reversion mode
   
   while(asset(loop("NZDJPY","EURUSD","AUDUSD","AUDCHF","AUDJPY","USDCHF","USDJPY")))
	{
		//edge trading logic
		var TimeCycle = optimize(64,55,75,1,0);
		var TimeFactor = optimize(2.6,0.2,5,0.2,0);
		//Stop = BarPeriod*PIP; //simple stop level
		Stop = ATR(200) * optimize(1.99,1,15,0.5,-3); // allow 3% tolerance for preferring low stop distances
		Trail = ATR(200) * optimize(8.5,3,13,0.5);

		vars Price = series(price(0));
		vars MA1 = series(SMA(Price,TimeCycle));
		vars MA2 = series(SMA(Price,TimeCycle*TimeFactor));

		if (checkModifiers())
		{
			//OK to trade, let's evaluate signals then
			if (crossOver(MA1,MA2) && rising(MA1))
			{
				if (reinvestprofits) Margin = calculateMargin(1); //long
				//enterLong(); //standard entry
				reverseLong(maxtrades);
			}
			else if(crossUnder(MA1,MA2) && falling(MA2))
			{
				if (reinvestprofits) Margin = calculateMargin(0); //short
				//enterShort(); //standard entry
				reverseShort(maxtrades);
			}
		}
	}

	PlotWidth = 1100;
	PlotHeight1 = 800;
}


Re: DT design process: step 1: identify the edge [Re: dusktrader] #432270
11/03/13 20:44
11/03/13 20:44
Joined: Apr 2013
Posts: 107
UK
G
Geek Offline
Member
Geek  Offline
Member
G

Joined: Apr 2013
Posts: 107
UK
Very impressed dusktrader, I also echo the thoughts of others here.

I hope I can one day contribute in similar fashion, the problem for me at the moment is I am still a novice and need to put the time in to learn, understand and digest everything regarding Zorro.

One day everything will click, hopefully.

All the best.

Re: DT design process: step 1: identify the edge [Re: Geek] #434609
12/22/13 12:21
12/22/13 12:21

L
liftoff
Unregistered
liftoff
Unregistered
L



Totally blown away by your work dusktrader ... a real contribution to this community. I just spent the last 3 hours digesting your post. I have to admit I have learnt so much and my whole conception of the design process has been greatly improved. This will definitely be the template on which I build my own design process. Thank you again.

Re: DT design process: step 1: identify the edge [Re: ] #437901
02/28/14 16:05
02/28/14 16:05
Joined: Oct 2013
Posts: 18
5
512mb Offline
Newbie
512mb  Offline
Newbie
5

Joined: Oct 2013
Posts: 18
dusk, I was using your minimum 30 trades per WFO cycle code. When it goes through
all the cycles without stopping the script one would assume its all good.
But then often tradecount is printed in Zorro window in the end.
This somehow prevents generating of fac file.
So what you might end up using is fac file of whatever you did before.

Re: DT design process: step 1: identify the edge [Re: 512mb] #437908
03/01/14 01:03
03/01/14 01:03
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
Hi 512mb,
the routine can only check the # of trades at the end of the WFO cycle. If it happens to be the last cycle, then I think that situation could happen. So you definitely have to be on the lookout. You could try making the alert more obvious, or even a popup window.

Re: DT design process: step 1: identify the edge [Re: dusktrader] #438489
03/15/14 17:35
03/15/14 17:35
Joined: Jan 2013
Posts: 68
I
ibra Offline
Junior Member
ibra  Offline
Junior Member
I

Joined: Jan 2013
Posts: 68
Hi dusk,

I've always thought the oversampling tool is used when you feel that you don't get enough trades from the original script... but since you use it on this low timeframe, can you explain to me why? I guess your strategy produces a whole lot of entries as it is?

Re: DT design process: step 1: identify the edge [Re: ibra] #438498
03/16/14 00:00
03/16/14 00:00
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
Well I suppose its all a matter of perspective. There are some combinations that barely produce the minimum required trades during WFO cycles. But that's not the reason I use oversampling. My feeling is that oversampling in general is a way to improve test results by reducing the effect of random entries with your logic (ie, success due to luck only).

My understanding is that oversampling will never harm the accuracy of your simulation result. On the contrary, it adds more realism to the simulation, in my opinion.

In another thread somewhere on this forum, jcl confirmed exactly how oversampling works, but I will try to explain it again here from memory:

As an example, if your strategy looks for entries on BarPeriod 15, that means during simulation, it would only look at one price sample every 15 minutes. That should theoretically be fine, as long as you have "enough" trades overall to be statistically significant when looking at the Performance Report analysis.

However, by using the NumSampleCycles oversampling feature, you have the opportunity to introduce more simulated trades "while maintaining the trend, periodic behavior, and other characteristics", per the manual.

In laymens terms, the simulation looks at the "what if" scenario of where exactly the 15 minute bar begins. If the strategy logic is truly robust, then it shouldn't really matter too much if the 15-minute bar begins at 9:15, 9:16 or 9:17. That is what oversampling does -- it shifts where that demarcation point is, at least for simulation purposes.

So my understanding is that it runs and re-runs the strategy assuming the demarcation point of the bar is at 9:15, 9:16, 9:17, etc... and then takes the average result of those tests.

Technically speaking, I believe it works like this:
NumSampleCycles=5

cycle 5 --> 9:19 bar boundary
cycle 4 --> 9:18 bar boundary
cycle 3 --> 9:17 bar boundary
cycle 2 --> 9:16 bar boundary
cycle 1 --> 9:15 bar boundary

If the strategy logic is robust enough, it shouldn't matter too much where the bar boundary is. So oversampling helps validate robust logic.

Re: DT design process: step 1: identify the edge [Re: dusktrader] #438522
03/16/14 13:47
03/16/14 13:47
Joined: Jan 2013
Posts: 68
I
ibra Offline
Junior Member
ibra  Offline
Junior Member
I

Joined: Jan 2013
Posts: 68
Thanks!

Page 2 of 2 1 2

Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1