Gamestudio Links
Zorro Links
Newest Posts
Data from CSV not parsed correctly
by EternallyCurious. 04/25/24 10:20
Trading Journey
by howardR. 04/24/24 20:04
M1 Oversampling
by Petra. 04/24/24 10:34
Zorro FIX plugin - Experimental
by flink. 04/21/24 07:12
Scripts not found
by juergen_wue. 04/20/24 18:51
zorro 64bit command line support
by 7th_zorro. 04/20/24 10:06
StartWeek not working as it should
by jcl. 04/20/24 08:38
folder management functions
by VoroneTZ. 04/17/24 06:52
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
5 registered members (SBGuy, Petra, flink, AndrewAMD, 1 invisible), 687 guests, and 7 spiders.
Key: Admin, Global Mod, Mod
Newest Members
Mega_Rod, EternallyCurious, howardR, 11honza11, ccorrea
19048 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 3 1 2 3
Memory limit too low for tick data? #459632
05/29/16 11:31
05/29/16 11:31
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
Hello, I'm getting Error 060: TIC memory limit exceeded. Citation from manual:

"Zorro can not allocate a too large memory resource - such as the ticks buffer for a simulation in TICKS mode, or the price buffer for a large simulation period and small bar period. The script can not be executed and Zorro must be restarted. This happens when the PC is either running low on memory (a Win32 process has 3 GB max.), when a too-large single block of memory is allocated, or when the memory is fragmented through many previous simulation or training runs. In the latter case the fragmented memory can be released simply by re-starting Zorro. Otherwise, memory can be saved by splitting the portfolio into separate strategies, reducing the simulation period (f.i. by setting a EndDate or MaxBars limit), not using the TICKS flag, and using M1 rather than T1 price data."

I'm trying to backtest only one year of tick data for one symbol with no optimization. I have Win7 64bit with 12GB in VirtualBox so the PC memory size is not a problem. The .t1 file has 328 MB. The original .csv file has 0.91 prices per second which is not that much for tick data.

The memory is not full or fragmented. I can't split the portfolio as it is only one symbol and one strategy. Setting MaxBars, EndDate or turning off TICKS flag doesn't help either - I guess it's because the whole .t1 file is loaded at once.

Using M1 instead of T1 is not an option for me. I've noticed that when doing so, the results can be quite different (I'm testing a strategy with thousands of trades per year - even small details matter).

The ability to test tick data was one of the main reasons I bought Zorro in the first place. So my question is: Is there any way to get around this problem? Is it maybe possible to increase the memory limit somehow? Thank you in advance for the reply. Cheers!

Re: Memory limit too low for tick data? [Re: pcz] #459643
05/29/16 17:21
05/29/16 17:21
Joined: Dec 2014
Posts: 206
Germany
Smon Offline
Member
Smon  Offline
Member

Joined: Dec 2014
Posts: 206
Germany
Exact same question here. I can't get lower than 2s Ticks until running into the memory issue. Limiting the backtest to one month doesn't help either.

By the way, what is

Error 057: not enough price ticks

?

Backtest is running after this Error without problems.

Last edited by sdh309795gaas; 05/29/16 17:21.
Re: Memory limit too low for tick data? [Re: Smon] #459660
05/30/16 10:41
05/30/16 10:41
Joined: Jul 2000
Posts: 27,982
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,982
Frankfurt
1 year of 1 price per second tick data is 60*60*24*252*16 = about 350 MB. So that should be normally not a problem. I suppose that the running out of memory has a different reason, like using more memory for something else. Can you post your script, or contact Support? They'll check.

Re: Memory limit too low for tick data? [Re: jcl] #459662
05/30/16 11:04
05/30/16 11:04
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
The minimal example would be quite simple:

Code:
function run()
{
	History = ".t1";
	StartDate = 2015;
	EndDate = 2015;
	while(asset(loop("EUR/USD"))){}
}



This is for the FXCM data (EUR/USD for 2015 is the largest file).

The exact error message is: "Error 060: TIC memory limit exceeded (788 MB)"

788 is twice the file size.

Re: Memory limit too low for tick data? [Re: pcz] #459666
05/30/16 13:00
05/30/16 13:00
Joined: Jul 2000
Posts: 27,982
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,982
Frankfurt
Indeed! That error message is in fact wrong, the memory did not run out. I suppose that this is a bug in the memory size calculation. I've notified the developer, and we'll upload later today a new beta version 1.45.2 where this is fixed.

Re: Memory limit too low for tick data? [Re: jcl] #459674
05/30/16 15:50
05/30/16 15:50
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
I've downloaded the new beta version and it works fine, thank you!

Re: Memory limit too low for tick data? [Re: pcz] #459808
06/07/16 12:16
06/07/16 12:16
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
Another similar problem - I get the same error when trying to backtest two years of data for a symbol if the sum of file sizes is cca 500 MB+.

Minimal example for FXCM data:

Code:
function run()
{
	History = ".t1";
	StartDate = 2014;
	EndDate = 2015;
	while(asset(loop("EUR/USD"))){}
}



Again - trying in VirtualBox with Win7 64bit and 12+ GB RAM.

Re: Memory limit too low for tick data? [Re: pcz] #459812
06/07/16 15:03
06/07/16 15:03
Joined: Jul 2000
Posts: 27,982
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,982
Frankfurt
Yes, this error is indeed intentional for limiting .t1 data to one year. For two years you must run two separate tests. Reason is that the current version loads all ticks into memory. It's planned to cache the data in a future version and then there's no limit to .t1 data - but with the current version, one year is max.

Re: Memory limit too low for tick data? [Re: jcl] #464129
01/24/17 11:41
01/24/17 11:41
Joined: May 2016
Posts: 180
Prague
pcz Offline OP
Member
pcz  Offline OP
Member

Joined: May 2016
Posts: 180
Prague
I'm running again into the memory issue with my own collected data. If I want to store every price tick the .t1 file can have over 600 MB. In that case it's not even possible to backtest one year for one asset without getting Error 60 despite the fact that the system has 9 GB free memory. It's quite inconvenient to split the date ranges further. Is it possible to increase the limit somehow?

Re: Memory limit too low for tick data? [Re: pcz] #464138
01/24/17 19:11
01/24/17 19:11
Joined: Dec 2013
Posts: 568
Fuerth, DE
Sphin Offline
User
Sphin  Offline
User

Joined: Dec 2013
Posts: 568
Fuerth, DE
Quote:
system has 9 GB free memory
I guess you won't use it with a 32 bit app. laugh

Page 1 of 3 1 2 3

Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1