|
Save trade stats between Zorro restarts
#437015
02/07/14 21:03
02/07/14 21:03
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
jcl says they will incorporate a way to save state information between Zorro restarts. I'm really looking forward to that feature because I can definitely see how you could be hurt if your Zorro stops at any point during live trading. The stop could be intentional or unintentional. I've gone ahead and written this proof-of-concept which I will test a little further, then plan to implement (I'm using Zorro 1.20.1; don't know when the official features would be available). I'm definitely a n00b when it comes to C programming so it took me quite a while to get this just right. I would greatly appreciate any feedback or improvements. I'm working on incorporating this into my infrastructure that I use for building slopbots. I have tried to keep it as modular as possible. (I'll share the full latest infrastructure once I can verify it's all working correctly.) Here is the theory why I built this: When Zorro starts for the VERY FIRST time, it does not know anything about a given asset's current behavior with regard to equity-curve trading (ie, it doesn't know if the asset should currently be on phantom-only restriction due to a string of losses, for example). The way I deal with that situation can be seen in the logic of the CalculateMargin() function I use. This logic has been tested on my live account and I do believe it works. When Zorro starts a SUBSEQUENT TIME after live trading has already begun, it only knows about phantom or real trades that are currently open (my understanding as of v1.20.1). Therefore, it is somewhat of a tragedy to stop/restart Zorro because that would almost definitely hurt the performance if equity-curve trading is used. While this state info I'm saving to disk is not a perfect solution, I do believe it is "better" than nothing. One other thing I plan to look into is serializing a series array, so that potentially the equity curve itself could be saved to disk. That might be ideal. But my coding skills are weak, and the official Zorro feature is due out soon... This is the margin calculation routine I use:
function calculateMargin(int direction)
{
//calculate risk Margin based on OptimalF and trade direction
Capital = 1000; //simulated account balance
var riskCapital = 300; //basis to trade with
if (direction && OptimalFLong>.001) //long trade, historically profitable
{
if (getOpt("emode") //uses equity-curve trading
&& (NumWinLong<1 && NumWinShort<1 && NumLossLong<1 && NumLossShort<1) //no completed trades yet
&& (EquityLong<=0 && EquityShort<=0)) //initial active trades are losing or non-existent
{
Lots = -1; //phantom only on first trade each asset
if(is(TRADEMODE)) printf("\n%s initial trade restricted to Phantom only",Asset);
}
if(is(TRADEMODE)) printf("\nAsset=%s; direction=%i; OptimalF=%f;\nMargin=%f; Lots=%i",Asset,direction,OptimalF,(OptimalFLong*riskCapital),Lots);
return OptimalFLong * riskCapital;
}
else if (!direction && OptimalFShort>.001) //short trade, historically profitable
{
if (getOpt("emode") //uses equity-curve trading
&& (NumWinLong<1 && NumWinShort<1 && NumLossLong<1 && NumLossShort<1) //no completed trades yet
&& (EquityLong<=0 && EquityShort<=0)) //initial active trades are losing or non-existent
{
Lots = -1; //phantom only on first trade each asset
if(is(TRADEMODE)) printf("\n%s initial trade restricted to Phantom only",Asset);
}
if(is(TRADEMODE)) printf("\nAsset=%s; direction=%i; OptimalF=%f;\nMargin=%f; Lots=%i",Asset,direction,OptimalF,(OptimalFShort*riskCapital),Lots);
return OptimalFShort * riskCapital;
}
return 0; //no Margin allocated for non-historically profitable
}
And here is the proof-of-concept for saving trade stats to disk:
#include <default.c>
#include <stdio.h>
function loadStats(string filename)
{
if(!is(TRADEMODE)) return; //only read state info when live trading
char path[100]; //full path to parameter .ini file
char param[30]; //asset-labeled parameter in file
sprintf(path,"Strategy\\%s",filename);
string fcontent = file_content(path);
//load from disk current state info for each asset
while(asset(loop("EURUSD","USDJPY")))
{
sprintf(param,"%s_NumWinLong",Asset);
NumWinLong = strvar(fcontent,param);
sprintf(param,"%s_NumWinShort",Asset);
NumWinShort = strvar(fcontent,param);
sprintf(param,"%s_NumLossLong",Asset);
NumLossLong = strvar(fcontent,param);
sprintf(param,"%s_NumLossShort",Asset);
NumLossShort = strvar(fcontent,param);
sprintf(param,"%s_WinLong",Asset);
WinLong = strvar(fcontent,param);
sprintf(param,"%s_WinShort",Asset);
WinShort = strvar(fcontent,param);
sprintf(param,"%s_LossLong",Asset);
LossLong = strvar(fcontent,param);
sprintf(param,"%s_LossShort",Asset);
LossShort = strvar(fcontent,param);
}
}
function saveStats(string filename)
{
if(!is(TRADEMODE)) return; //only save state info when live trading
FILE *out;
char path[100]; //full path to parameter .ini file
char line[2500], tmp[2500]; //line of data for .ini file
sprintf(path,"Strategy\\%s",filename);
if (!(out = fopen(path, "w")))
{
printf("\nERROR trying to open file for write:\n%s\n",path);
return;
}
//store to disk current state info for each asset
while(asset(loop("EURUSD","USDJPY")))
{
sprintf(line,"%s_NumWinLong = %i\n",Asset,NumWinLong);
sprintf(tmp,"%s_NumWinShort = %i\n",Asset,NumWinShort);
strcat(line,tmp);
sprintf(tmp,"%s_NumLossLong = %i\n",Asset,NumLossLong);
strcat(line,tmp);
sprintf(tmp,"%s_NumLossShort = %i\n",Asset,NumLossShort);
strcat(line,tmp);
sprintf(tmp,"%s_WinLong = %f\n",Asset,WinLong);
strcat(line,tmp);
sprintf(tmp,"%s_WinShort = %f\n",Asset,WinShort);
strcat(line,tmp);
sprintf(tmp,"%s_LossLong = %f\n",Asset,LossLong);
strcat(line,tmp);
sprintf(tmp,"%s_LossShort = %f\n",Asset,LossShort);
strcat(line,tmp);
if (!fwrite(line, sizeof(char), strlen(line), out))
{
printf("\nWRITE ERROR:\n%s\n",path);
fclose(out);
return;
}
else printf("\nSaved trade stats to disk");
}
fclose(out);
}
function run()
{
StartDate = 20130520;
EndDate = 20130920;
BarPeriod = 15;
LookBack = 200;
static int numOpenLastCheck; //track changes in trade open/closes
//load most recent stats that were saved to disk
if(is(INITRUN)) loadStats("dt-file-readwrite.ini");
//save stats to disk periodically, or upon trade count changes
if(NumOpenTotal != numOpenLastCheck || minute()==30 ) saveStats("dt-file-readwrite.ini");
numOpenLastCheck = NumOpenTotal;
while(asset(loop("EURUSD","USDJPY")))
{
if(is(INITRUN)) printf("\nINIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);
if(is(EXITRUN)) printf("\nEXIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);
vars Price = series(price());
vars Trend = series(LowPass(Price,1000));
Stop = 4*ATR(100);
if(valley(Trend))
enterLong();
else if(peak(Trend))
enterShort();
}
}
Thanks for any feedback.
|
|
|
Re: Save trade stats between Zorro restarts
[Re: dusktrader]
#437064
02/08/14 14:54
02/08/14 14:54
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
@jcl is it possible that there could be some issue with using file_content() and set(FACTORS) ?? I'm trying to integrate the above code but it seems whenever I use file_content() it breaks loading the .fac file with the following error: Error 062: Can't open dt-e9-htc-30min.fac (rt)I will try rewriting the function to use file_read() instead of file_content() EDIT: it might be something else I'm doing wrong; it does seem to work on the proof-of-concept code above, even with set(FACTORS) so I'll have to keep poking at the code EDIT2: it seems the issue has something to do with setting the trade stat variables, combined with reading .par and .fac files. I realize I may be doing something illegal by setting these values, but it seems to work as expected when there is no .par or .fac to be read. Therefore, I'll try another hack, which would be to let Zorro read the .par and .fac, and then reset those global stat variables after-the-fact. I just need a way to identify an "early iteration" of Zorro, perhaps not the INITRUN though. EDIT3: interestingly, I think my theory is correct. If I simply allow Zorro to have the INITRUN to itself, it will allow me to read and reset the trade stats on an early non-first iteration. In WFO testing, it is later then unable to read the 2nd .par file, giving the error: dt-e9-htc-30min run.. Walk-Forward Test: dt-e9-htc-30min EURUSD 2008..2014 Read dt-e9-htc-30min_EURUSD.fac dt-e9-htc-30min_EURUSD_1.par Error 062: Can't open dt-e9-htc-30min_2.par (rt) ... That makes sense to me... it seems Zorro needs the stats to be reset when it begins testing a new parameter set, for some reason. Since .fac is only read once per entire WFO, it was able to do that on the INITRUN. Therefore, my tentative conclusion is that during TRADEMODE (live trading), it would work correctly, as there would only be one .fac and .par file read. After that iteration was complete, I would then read my .ini and reset the trad stats. Here is the code I used to call my code from iteration 2:
static int zorroIteration;
if(is(INITRUN)) zorroIteration=0; else zorroIteration++;
//load most recent stats that were saved to disk
if (zorroIteration==1) loadStats("dt-e9-htc-30min.ini");
(I'm still testing and hope to run this on a demo account next week.) EDIT4: seems like no matter what conditions I add, Zorro does not like to read the second .par file after I've read my text file (and manipulated the trade stat variables). I'm not sure if this is a "bug" or just an illegal operation in trying to set those trade stats. What I think it means is that... during WFO testing, the functions would interfere with normal WFO calculations. You wouldn't normally use them anyway, except for TRADEMODE. Here is the latest code that seems to work. I'll know better next week when I can get it tested on a demo account:
static int zorroIteration, LastWFOCycle;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if(WFOCycle != LastWFOCycle) printf("\nWFOCycle = %i",WFOCycle);
if(is(INITRUN)) LastWFOCycle=0; else LastWFOCycle = WFOCycle;
//load most recent stats that were saved to disk
if (zorroIteration==2) loadStats("dt-e9-htc-30min.ini");
//save stats to disk periodically, or upon trade count changes
if(minute()==60 || NumOpenTotal != numOpenLastCheck) saveStats("dt-e9-htc-30min.ini");
numOpenLastCheck = NumOpenTotal;
Last edited by dusktrader; 02/08/14 16:45.
|
|
|
Re: Save trade stats between Zorro restarts
[Re: dusktrader]
#437151
02/10/14 14:36
02/10/14 14:36
|
Joined: Jan 2013
Posts: 68
ibra
Junior Member
|
Junior Member
Joined: Jan 2013
Posts: 68
|
*Wall of code* Hey Dusk! Stop calling yourself "n00b"!! You make me feel like I'm an ÜBERNOOB
|
|
|
Re: Save trade stats between Zorro restarts
[Re: ibra]
#437154
02/10/14 14:49
02/10/14 14:49
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
Well I guess we are all n00bs basically!! I think you can definitely get burned if you start thinking you're an expert. Trading's a head-game, remember?? Besides, I think it's neat how the Zorro project attracts such a wide range of thinkers... they range from n00b to mad scientist, and everything inbetween. Btw in the above code, it occurred to me that I do not need the "periodic save" logic, because the only items being saved currently are metrics that are linked to trade opens/closes. Therefore that activity in-and-of-itself is the best trigger I think. Here is what I'm using now on my demo account:
static int numOpenLastCheck, zorroIteration; //track changes in trade open/closes
if(is(INITRUN)) numOpenLastCheck=0; else numOpenLastCheck=NumOpenTotal;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if (zorroIteration==2) loadStats("dt-e9-htc-30min.ini"); //load most recent stats that were saved to disk
if(NumOpenTotal != numOpenLastCheck) saveStats("dt-e9-htc-30min.ini"); //save stats to disk upon trade count changes
|
|
|
Re: Save trade stats between Zorro restarts
[Re: jcl]
#437157
02/10/14 16:23
02/10/14 16:23
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
Sorry I didn't provide a complete code... here I've modified the test script to demonstrate the issue. Going through this also helped me fix a bug. First Train the script, then Test and you'll see the issue. It seems like a file handle is possibly not being closed by Zorro's read operation (either .par or .fac):
#include <default.c>
#include <stdio.h>
function loadStats(string filename)
{
if(!is(TESTMODE)) return; //only read state info when live trading
char path[100]; //full path to parameter .ini file
char param[30]; //asset-labeled parameter .ini file
char fcontent[2500]; //file content from parameter .ini file
sprintf(path,"Strategy\\%s",filename);
if(!file_date(path)) msg("#>>WARNING<< no stats file found!\nThis is normal if this is the very first run of a new strategy");
string fcontent = file_content(path);
//load from disk current state info for each asset
while(asset(loop("EURUSD")))
{
sprintf(param,"%s_NumWinLong",Asset);
NumWinLong = strvar(fcontent,param);
sprintf(param,"%s_NumWinShort",Asset);
NumWinShort = strvar(fcontent,param);
sprintf(param,"%s_NumLossLong",Asset);
NumLossLong = strvar(fcontent,param);
sprintf(param,"%s_NumLossShort",Asset);
NumLossShort = strvar(fcontent,param);
sprintf(param,"%s_WinLong",Asset);
WinLong = strvar(fcontent,param);
sprintf(param,"%s_WinShort",Asset);
WinShort = strvar(fcontent,param);
sprintf(param,"%s_LossLong",Asset);
LossLong = strvar(fcontent,param);
sprintf(param,"%s_LossShort",Asset);
LossShort = strvar(fcontent,param);
}
}
function saveStats(string filename)
{
if(!is(TESTMODE)) return; //only save state info when live trading
FILE *out;
char path[100]; //full path to parameter .ini file
char line[2500], tmp[2500]; //line of data for .ini file
sprintf(path,"Strategy\\%s",filename);
if (!(out = fopen(path, "w")))
{
printf("\nERROR trying to open file for write:\n%s\n",path);
return;
}
//store to disk current state info for each asset
while(asset(loop("EURUSD")))
{
sprintf(line,"%s_NumWinLong = %i\n",Asset,NumWinLong);
sprintf(tmp,"%s_NumWinShort = %i\n",Asset,NumWinShort);
strcat(line,tmp);
sprintf(tmp,"%s_NumLossLong = %i\n",Asset,NumLossLong);
strcat(line,tmp);
sprintf(tmp,"%s_NumLossShort = %i\n",Asset,NumLossShort);
strcat(line,tmp);
sprintf(tmp,"%s_WinLong = %f\n",Asset,WinLong);
strcat(line,tmp);
sprintf(tmp,"%s_WinShort = %f\n",Asset,WinShort);
strcat(line,tmp);
sprintf(tmp,"%s_LossLong = %f\n",Asset,LossLong);
strcat(line,tmp);
sprintf(tmp,"%s_LossShort = %f\n",Asset,LossShort);
strcat(line,tmp);
if (!fwrite(line, sizeof(char), strlen(line), out))
{
printf("\nWRITE ERROR:\n%s\n",path);
fclose(out);
return;
}
//else printf("\nSaved trade stats to disk");
}
fclose(out);
}
function run()
{
set(PARAMETERS+FACTORS);
StartDate = 2010;
EndDate = 2011;
BarPeriod = 15;
LookBack = 200;
NumWFOCycles = 5;
static int LastWFOCycle;
if(WFOCycle != LastWFOCycle) printf("\nWFOCycle = %i",WFOCycle);
if(is(INITRUN)) LastWFOCycle=0; else LastWFOCycle = WFOCycle;
static int numOpenLastCheck, zorroIteration; //track changes in trade open/closes
if(is(INITRUN)) numOpenLastCheck=0;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if (zorroIteration==2) loadStats("dt-file-readwrite.ini"); //load most recent stats that were saved to disk
if(NumOpenTotal != numOpenLastCheck)
{
saveStats("dt-file-readwrite.ini"); //save stats to disk upon trade count changes
numOpenLastCheck = NumOpenTotal;
}
asset("EURUSD");
{
if(is(INITRUN)) printf("\nINIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);
if(is(EXITRUN)) printf("\nEXIT %s\nEquityLong=%f; EquityShort=%f\n",Asset,EquityLong,EquityShort);
vars Price = series(price());
vars Trend = series(LowPass(Price,1000));
Stop = ATR(100) * optimize(4,1,5,1);
if(valley(Trend))
enterLong();
else if(peak(Trend))
enterShort();
}
}
|
|
|
Re: Save trade stats between Zorro restarts
[Re: jcl]
#437216
02/12/14 11:24
02/12/14 11:24
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
No it should not be UAC or access rights (my system does not have shadow folders). I just noticed something a little strange in my live (demo) account related to this. Upon stopping/starting the strategy, it complains about the .fac file but then a debug line I have below that shows that it does have the correct OptimalF value in memory. I wonder if the error message could be bogus for some reason? EDIT: I'm replacing output below with a cleaner example:
Broker: IBFX, Inc. connected at UTC 12.02. 11:36
Loading EURUSD prices.. 4500 min
Trade: dt-e9-htc-30min EURUSD 12.02.2014
[EURUSD::L9488] continuing
Read dt-e9-htc-30min_EURUSD.fac dt-e9-htc-30min_EURUSD.par
Error 062: Can't open dt-e9-htc-30min.fac (rt)
Saved trade stats to disk
Asset=EURUSD; direction=1; OptimalF=0.451000;
Margin=211.950000; Lots=0
[Wed 12.02. 11:36] 977 +0 -23 \.
[EURUSD::L9488] Stop 77@1.3603: -30.26 at 11:36
Btw I am now using the following logic (I keep tweaking it). I will keep poking at this issue and see if I can create any more easily-reproducible sample code. Note: to test this theory about the bogus error message, I also set OptimalF to 0 during the INITRUN and it seems to support the theory:
if(is(INITRUN)) OptimalF = 0;
static int numOpenLastCheck, zorroIteration; //track changes in trade open/closes
if(is(INITRUN)) numOpenLastCheck=0;
if(is(INITRUN)) zorroIteration=1; else zorroIteration++;
if (zorroIteration==2) loadStats("dt-e9-htc-30min.ini"); //load most recent stats that were saved to disk
if (zorroIteration>2 && NumOpenTotal != numOpenLastCheck)
{
saveStats("dt-e9-htc-30min.ini"); //save stats to disk upon trade count changes
numOpenLastCheck = NumOpenTotal;
}
Last edited by dusktrader; 02/12/14 11:41.
|
|
|
|