Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
5 registered members (AndrewAMD, monk12, TipmyPip, Quad, aliswee), 1,031 guests, and 6 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 2 1 2
Save trade stats between Zorro restarts #437015
02/07/14 21:03
02/07/14 21:03
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
jcl says they will incorporate a way to save state information between Zorro restarts. I'm really looking forward to that feature because I can definitely see how you could be hurt if your Zorro stops at any point during live trading. The stop could be intentional or unintentional.

I've gone ahead and written this proof-of-concept which I will test a little further, then plan to implement (I'm using Zorro 1.20.1; don't know when the official features would be available).

I'm definitely a n00b when it comes to C programming so it took me quite a while to get this just right. I would greatly appreciate any feedback or improvements.

I'm working on incorporating this into my infrastructure that I use for building slopbots. I have tried to keep it as modular as possible. (I'll share the full latest infrastructure once I can verify it's all working correctly.)

Here is the theory why I built this:
When Zorro starts for the VERY FIRST time, it does not know anything about a given asset's current behavior with regard to equity-curve trading (ie, it doesn't know if the asset should currently be on phantom-only restriction due to a string of losses, for example). The way I deal with that situation can be seen in the logic of the CalculateMargin() function I use. This logic has been tested on my live account and I do believe it works.

When Zorro starts a SUBSEQUENT TIME after live trading has already begun, it only knows about phantom or real trades that are currently open (my understanding as of v1.20.1). Therefore, it is somewhat of a tragedy to stop/restart Zorro because that would almost definitely hurt the performance if equity-curve trading is used.

While this state info I'm saving to disk is not a perfect solution, I do believe it is "better" than nothing. One other thing I plan to look into is serializing a series array, so that potentially the equity curve itself could be saved to disk. That might be ideal. But my coding skills are weak, and the official Zorro feature is due out soon...

This is the margin calculation routine I use:
Code:
function calculateMargin(int direction)
{
	//calculate risk Margin based on OptimalF and trade direction
	Capital = 1000; //simulated account balance
	var riskCapital = 300; //basis to trade with

	if (direction && OptimalFLong>.001) //long trade, historically profitable
		{
		if (getOpt("emode") //uses equity-curve trading
			&& (NumWinLong<1 && NumWinShort<1 && NumLossLong<1 && NumLossShort<1) //no completed trades yet
			&& (EquityLong<=0 && EquityShort<=0)) //initial active trades are losing or non-existent
			{
				Lots = -1; //phantom only on first trade each asset
				if(is(TRADEMODE)) printf("\n%s initial trade restricted to Phantom only",Asset);
			}

		if(is(TRADEMODE)) printf("\nAsset=%s; direction=%i; OptimalF=%f;\nMargin=%f; Lots=%i",Asset,direction,OptimalF,(OptimalFLong*riskCapital),Lots);
		return OptimalFLong * riskCapital;
		}
	else if (!direction && OptimalFShort>.001) //short trade, historically profitable
		{
		if (getOpt("emode") //uses equity-curve trading
			&& (NumWinLong<1 && NumWinShort<1 && NumLossLong<1 && NumLossShort<1) //no completed trades yet
			&& (EquityLong<=0 && EquityShort<=0)) //initial active trades are losing or non-existent
			{
				Lots = -1; //phantom only on first trade each asset
				if(is(TRADEMODE)) printf("\n%s initial trade restricted to Phantom only",Asset);
			}

		if(is(TRADEMODE)) printf("\nAsset=%s; direction=%i; OptimalF=%f;\nMargin=%f; Lots=%i",Asset,direction,OptimalF,(OptimalFShort*riskCapital),Lots);
		return OptimalFShort * riskCapital;
		}

	return 0; //no Margin allocated for non-historically profitable
}



And here is the proof-of-concept for saving trade stats to disk:
Code:
#include <default.c>
#include <stdio.h>

function loadStats(string filename)
{
	if(!is(TRADEMODE)) return; //only read state info when live trading
	
	char path[100]; //full path to parameter .ini file
	char param[30]; //asset-labeled parameter in file
	sprintf(path,"Strategy\\%s",filename);
	string fcontent = file_content(path);
	
	//load from disk current state info for each asset
	while(asset(loop("EURUSD","USDJPY")))
	{
		sprintf(param,"%s_NumWinLong",Asset);
		NumWinLong = strvar(fcontent,param);
		
		sprintf(param,"%s_NumWinShort",Asset);
		NumWinShort = strvar(fcontent,param);	
	
		sprintf(param,"%s_NumLossLong",Asset);
		NumLossLong = strvar(fcontent,param);
	
		sprintf(param,"%s_NumLossShort",Asset);
		NumLossShort = strvar(fcontent,param);
		
		sprintf(param,"%s_WinLong",Asset);
		WinLong = strvar(fcontent,param);

		sprintf(param,"%s_WinShort",Asset);
		WinShort = strvar(fcontent,param);
		
		sprintf(param,"%s_LossLong",Asset);
		LossLong = strvar(fcontent,param);
		
		sprintf(param,"%s_LossShort",Asset);
		LossShort = strvar(fcontent,param);
	}
}

function saveStats(string filename)
{
	if(!is(TRADEMODE)) return; //only save state info when live trading

	FILE *out;
	char path[100]; //full path to parameter .ini file
	char line[2500], tmp[2500]; //line of data for .ini file
	sprintf(path,"Strategy\\%s",filename);

	if (!(out = fopen(path, "w")))
	{
		printf("\nERROR trying to open file for write:\n%s\n",path);
		return;
	}

	//store to disk current state info for each asset
	while(asset(loop("EURUSD","USDJPY")))
	{
		sprintf(line,"%s_NumWinLong = %i\n",Asset,NumWinLong);
		sprintf(tmp,"%s_NumWinShort = %i\n",Asset,NumWinShort);
		strcat(line,tmp);
		sprintf(tmp,"%s_NumLossLong = %i\n",Asset,NumLossLong);
		strcat(line,tmp);
		sprintf(tmp,"%s_NumLossShort = %i\n",Asset,NumLossShort);
		strcat(line,tmp);
		sprintf(tmp,"%s_WinLong = %f\n",Asset,WinLong);
		strcat(line,tmp);
		sprintf(tmp,"%s_WinShort = %f\n",Asset,WinShort);
		strcat(line,tmp);
		sprintf(tmp,"%s_LossLong = %f\n",Asset,LossLong);
		strcat(line,tmp);
		sprintf(tmp,"%s_LossShort = %f\n",Asset,LossShort);
		strcat(line,tmp);
		
		if (!fwrite(line, sizeof(char), strlen(line), out))
		{
			printf("\nWRITE ERROR:\n%s\n",path);
			fclose(out);
			return;
		}
		else printf("\nSaved trade stats to disk");
	}

	fclose(out);
}

function run()
{
	StartDate = 20130520;
	EndDate = 20130920;
	BarPeriod = 15;
	LookBack = 200;
	static int numOpenLastCheck; //track changes in trade open/closes

	//load most recent stats that were saved to disk
	if(is(INITRUN)) loadStats("dt-file-readwrite.ini");

	//save stats to disk periodically, or upon trade count changes
	if(NumOpenTotal != numOpenLastCheck || minute()==30 ) saveStats("dt-file-readwrite.ini");
	numOpenLastCheck = NumOpenTotal;

	while(asset(loop("EURUSD","USDJPY")))
	{
		if(is(INITRUN)) printf("\nINIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);
		if(is(EXITRUN)) printf("\nEXIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);

		vars Price = series(price());
		vars Trend = series(LowPass(Price,1000));
	
		Stop = 4*ATR(100);
	
		if(valley(Trend))
			enterLong();
		else if(peak(Trend))
			enterShort();
	}
}



Thanks for any feedback.

Re: Save trade stats between Zorro restarts [Re: dusktrader] #437064
02/08/14 14:54
02/08/14 14:54
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
@jcl is it possible that there could be some issue with using file_content() and set(FACTORS) ??

I'm trying to integrate the above code but it seems whenever I use file_content() it breaks loading the .fac file with the following error:
Error 062: Can't open dt-e9-htc-30min.fac (rt)

I will try rewriting the function to use file_read() instead of file_content()

EDIT: it might be something else I'm doing wrong; it does seem to work on the proof-of-concept code above, even with set(FACTORS) so I'll have to keep poking at the code

EDIT2: it seems the issue has something to do with setting the trade stat variables, combined with reading .par and .fac files. I realize I may be doing something illegal by setting these values, but it seems to work as expected when there is no .par or .fac to be read. Therefore, I'll try another hack, which would be to let Zorro read the .par and .fac, and then reset those global stat variables after-the-fact. I just need a way to identify an "early iteration" of Zorro, perhaps not the INITRUN though.

EDIT3: interestingly, I think my theory is correct. If I simply allow Zorro to have the INITRUN to itself, it will allow me to read and reset the trade stats on an early non-first iteration. In WFO testing, it is later then unable to read the 2nd .par file, giving the error:
Quote:
dt-e9-htc-30min run..
Walk-Forward Test: dt-e9-htc-30min EURUSD 2008..2014
Read dt-e9-htc-30min_EURUSD.fac dt-e9-htc-30min_EURUSD_1.par
Error 062: Can't open dt-e9-htc-30min_2.par (rt)
...

That makes sense to me... it seems Zorro needs the stats to be reset when it begins testing a new parameter set, for some reason. Since .fac is only read once per entire WFO, it was able to do that on the INITRUN. Therefore, my tentative conclusion is that during TRADEMODE (live trading), it would work correctly, as there would only be one .fac and .par file read. After that iteration was complete, I would then read my .ini and reset the trad stats.

Here is the code I used to call my code from iteration 2:
Code:
static int zorroIteration;
if(is(INITRUN)) zorroIteration=0; else zorroIteration++;

//load most recent stats that were saved to disk
if (zorroIteration==1) loadStats("dt-e9-htc-30min.ini");



(I'm still testing and hope to run this on a demo account next week.)

EDIT4: seems like no matter what conditions I add, Zorro does not like to read the second .par file after I've read my text file (and manipulated the trade stat variables). I'm not sure if this is a "bug" or just an illegal operation in trying to set those trade stats. What I think it means is that... during WFO testing, the functions would interfere with normal WFO calculations. You wouldn't normally use them anyway, except for TRADEMODE. Here is the latest code that seems to work. I'll know better next week when I can get it tested on a demo account:
Code:
static int zorroIteration, LastWFOCycle;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if(WFOCycle != LastWFOCycle) printf("\nWFOCycle = %i",WFOCycle);
if(is(INITRUN)) LastWFOCycle=0; else LastWFOCycle = WFOCycle;

//load most recent stats that were saved to disk
if (zorroIteration==2) loadStats("dt-e9-htc-30min.ini");

//save stats to disk periodically, or upon trade count changes
if(minute()==60 || NumOpenTotal != numOpenLastCheck) saveStats("dt-e9-htc-30min.ini");
numOpenLastCheck = NumOpenTotal;



Last edited by dusktrader; 02/08/14 16:45.
Re: Save trade stats between Zorro restarts [Re: dusktrader] #437151
02/10/14 14:36
02/10/14 14:36
Joined: Jan 2013
Posts: 68
I
ibra Offline
Junior Member
ibra  Offline
Junior Member
I

Joined: Jan 2013
Posts: 68
*Wall of code*

Hey Dusk!

Stop calling yourself "n00b"!! You make me feel like I'm an ÜBERNOOB grin

Re: Save trade stats between Zorro restarts [Re: ibra] #437154
02/10/14 14:49
02/10/14 14:49
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
Well I guess we are all n00bs basically!! I think you can definitely get burned if you start thinking you're an expert. Trading's a head-game, remember??

Besides, I think it's neat how the Zorro project attracts such a wide range of thinkers... they range from n00b to mad scientist, and everything inbetween.

Btw in the above code, it occurred to me that I do not need the "periodic save" logic, because the only items being saved currently are metrics that are linked to trade opens/closes. Therefore that activity in-and-of-itself is the best trigger I think. Here is what I'm using now on my demo account:
Code:
static int numOpenLastCheck, zorroIteration; //track changes in trade open/closes
if(is(INITRUN)) numOpenLastCheck=0; else numOpenLastCheck=NumOpenTotal;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if (zorroIteration==2) loadStats("dt-e9-htc-30min.ini"); //load most recent stats that were saved to disk
if(NumOpenTotal != numOpenLastCheck) saveStats("dt-e9-htc-30min.ini"); //save stats to disk upon trade count changes


Re: Save trade stats between Zorro restarts [Re: ibra] #437155
02/10/14 15:25
02/10/14 15:25
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
I can not immediately see why reading an external file should interfere with loading factors - at least, our systems do that without problem. But there might be some hidden reason. This is hard to tell without seeing the full code. From what I see at a first glance in your example, you're doing it right.

One of the next Zorro versions, most likely 1.24, will store trade statistics and the equity curve trading status.

Re: Save trade stats between Zorro restarts [Re: jcl] #437157
02/10/14 16:23
02/10/14 16:23
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
Sorry I didn't provide a complete code... here I've modified the test script to demonstrate the issue. Going through this also helped me fix a bug. First Train the script, then Test and you'll see the issue. It seems like a file handle is possibly not being closed by Zorro's read operation (either .par or .fac):

Code:
#include <default.c>
#include <stdio.h>

function loadStats(string filename)
{
	if(!is(TESTMODE)) return; //only read state info when live trading
	
	char path[100]; //full path to parameter .ini file
	char param[30]; //asset-labeled parameter .ini file
	char fcontent[2500]; //file content from parameter .ini file
	sprintf(path,"Strategy\\%s",filename);
	if(!file_date(path)) msg("#>>WARNING<< no stats file found!\nThis is normal if this is the very first run of a new strategy");
	string fcontent = file_content(path);
		
	//load from disk current state info for each asset
	while(asset(loop("EURUSD")))
	{
		sprintf(param,"%s_NumWinLong",Asset);
		NumWinLong = strvar(fcontent,param);
		
		sprintf(param,"%s_NumWinShort",Asset);
		NumWinShort = strvar(fcontent,param);	
	
		sprintf(param,"%s_NumLossLong",Asset);
		NumLossLong = strvar(fcontent,param);
	
		sprintf(param,"%s_NumLossShort",Asset);
		NumLossShort = strvar(fcontent,param);
		
		sprintf(param,"%s_WinLong",Asset);
		WinLong = strvar(fcontent,param);

		sprintf(param,"%s_WinShort",Asset);
		WinShort = strvar(fcontent,param);
		
		sprintf(param,"%s_LossLong",Asset);
		LossLong = strvar(fcontent,param);
		
		sprintf(param,"%s_LossShort",Asset);
		LossShort = strvar(fcontent,param);
	}
}

function saveStats(string filename)
{
	if(!is(TESTMODE)) return; //only save state info when live trading

	FILE *out;
	char path[100]; //full path to parameter .ini file
	char line[2500], tmp[2500]; //line of data for .ini file
	sprintf(path,"Strategy\\%s",filename);

	if (!(out = fopen(path, "w")))
	{
		printf("\nERROR trying to open file for write:\n%s\n",path);
		return;
	}

	//store to disk current state info for each asset
	while(asset(loop("EURUSD")))
	{
		sprintf(line,"%s_NumWinLong = %i\n",Asset,NumWinLong);
		sprintf(tmp,"%s_NumWinShort = %i\n",Asset,NumWinShort);
		strcat(line,tmp);
		sprintf(tmp,"%s_NumLossLong = %i\n",Asset,NumLossLong);
		strcat(line,tmp);
		sprintf(tmp,"%s_NumLossShort = %i\n",Asset,NumLossShort);
		strcat(line,tmp);
		sprintf(tmp,"%s_WinLong = %f\n",Asset,WinLong);
		strcat(line,tmp);
		sprintf(tmp,"%s_WinShort = %f\n",Asset,WinShort);
		strcat(line,tmp);
		sprintf(tmp,"%s_LossLong = %f\n",Asset,LossLong);
		strcat(line,tmp);
		sprintf(tmp,"%s_LossShort = %f\n",Asset,LossShort);
		strcat(line,tmp);
		
		if (!fwrite(line, sizeof(char), strlen(line), out))
		{
			printf("\nWRITE ERROR:\n%s\n",path);
			fclose(out);
			return;
		}
		//else printf("\nSaved trade stats to disk");
	}

	fclose(out);
}

function run()
{
	set(PARAMETERS+FACTORS);
	StartDate = 2010;
	EndDate = 2011;
	BarPeriod = 15;
	LookBack = 200;
	NumWFOCycles = 5;

	static int LastWFOCycle;
	if(WFOCycle != LastWFOCycle) printf("\nWFOCycle = %i",WFOCycle);
	if(is(INITRUN)) LastWFOCycle=0; else LastWFOCycle = WFOCycle;

	static int numOpenLastCheck, zorroIteration; //track changes in trade open/closes
	if(is(INITRUN)) numOpenLastCheck=0;
	if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
	if (zorroIteration==2) loadStats("dt-file-readwrite.ini"); //load most recent stats that were saved to disk
	if(NumOpenTotal != numOpenLastCheck)
	{
		saveStats("dt-file-readwrite.ini"); //save stats to disk upon trade count changes
		numOpenLastCheck = NumOpenTotal;
	}

	asset("EURUSD");
	{
		if(is(INITRUN)) printf("\nINIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);
		if(is(EXITRUN)) printf("\nEXIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);

		vars Price = series(price());
		vars Trend = series(LowPass(Price,1000));
	
		Stop = ATR(100) * optimize(4,1,5,1);
	
		if(valley(Trend))
			enterLong();
		else if(peak(Trend))
			enterShort();
	}

}


Re: Save trade stats between Zorro restarts [Re: dusktrader] #437208
02/12/14 09:12
02/12/14 09:12
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
Thanks for the code, but I can't reproduce the problem. The .par files open here.

Maybe it has something to do with the access rights or the UAC on your PC? Do you have the .par files in the normal Data folder, or in a shadow folder?

Re: Save trade stats between Zorro restarts [Re: jcl] #437216
02/12/14 11:24
02/12/14 11:24
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
No it should not be UAC or access rights (my system does not have shadow folders).

I just noticed something a little strange in my live (demo) account related to this. Upon stopping/starting the strategy, it complains about the .fac file but then a debug line I have below that shows that it does have the correct OptimalF value in memory. I wonder if the error message could be bogus for some reason?

EDIT: I'm replacing output below with a cleaner example:
Code:
Broker: IBFX, Inc.  connected at UTC 12.02. 11:36
Loading EURUSD prices.. 4500 min

Trade: dt-e9-htc-30min EURUSD 12.02.2014
[EURUSD::L9488] continuing
Read dt-e9-htc-30min_EURUSD.fac dt-e9-htc-30min_EURUSD.par
Error 062: Can't open dt-e9-htc-30min.fac (rt)
Saved trade stats to disk
Asset=EURUSD; direction=1; OptimalF=0.451000;
Margin=211.950000; Lots=0
[Wed 12.02. 11:36]  977 +0 -23 \.
[EURUSD::L9488] Stop 77@1.3603: -30.26 at 11:36



Btw I am now using the following logic (I keep tweaking it). I will keep poking at this issue and see if I can create any more easily-reproducible sample code. Note: to test this theory about the bogus error message, I also set OptimalF to 0 during the INITRUN and it seems to support the theory:
Code:
if(is(INITRUN)) OptimalF = 0;
static int numOpenLastCheck, zorroIteration; //track changes in trade open/closes
if(is(INITRUN)) numOpenLastCheck=0;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if (zorroIteration==2) loadStats("dt-e9-htc-30min.ini"); //load most recent stats that were saved to disk
if (zorroIteration>2 && NumOpenTotal != numOpenLastCheck)
{
	saveStats("dt-e9-htc-30min.ini"); //save stats to disk upon trade count changes
	numOpenLastCheck = NumOpenTotal;
}



Last edited by dusktrader; 02/12/14 11:41.
Re: Save trade stats between Zorro restarts [Re: dusktrader] #437233
02/12/14 15:36
02/12/14 15:36
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
Hmm, I see in your post that there seems to be something wrong with the file names. The files should have the name xxxx_EURUSD.fac, but it complains about not being able to open xxxx.fac. That explains of course why the files can't be opened. However, when I test your script the file names are correct.

Please rename the script to f.i. "Test.c" - that's the name I'm using when testing user scripts - and try again. Does this change something?



Re: Save trade stats between Zorro restarts [Re: jcl] #437234
02/12/14 15:48
02/12/14 15:48
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
Good catch... I'll try this as soon as I can. I believe the _EURUSD is tacked onto the name depending on whether or not the asset is being called from a loop or not. That is probably the issue (I'm speculating here) ... because I bet the loadStats() and saveStats() routines are doing it "the other way" as compared to the run() function.

Page 1 of 2 1 2

Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1