Gamestudio Links
Zorro Links
Newest Posts
Get BMAP's filename...?
by Firoball
20 minutes 59 seconds ago
bone ang/pos
by 3run
Today at 15:41
Takeprofit and Stoploss
by sdelatorre
Today at 10:54
Save as .wmb
by Emre
Today at 05:57
Connors RSI implementation
by MatPed
Today at 00:37
multiple results graph in multi-asset scripts?
by SBGuy
Yesterday at 21:19
Verbose messages
by Sphin
Yesterday at 20:24
AUM Magazine
Latest Screens
Evonet Beta v1.0 : Online Project Development Template
Lake scene
Forgotten Tales - Demo version
RPG PARTY
Who's Online
13 registered (3run, JRA, alibaba, ChrstphFr, mig, AndrewAMD, MatPed, jyd, krial057, 1 invisible), 477 Guests and 6 Spiders online.
Key: Admin, Global Mod, Mod
Newest Members
bbn1982, selma, mbullied, Raisinbran, Grodt
17984 Registered Users
Topic Options
Rate This Topic
#470150 - 12/29/17 09:54 R bridge running very slow
Dalla Offline
Member

Registered: 02/24/17
Posts: 263
I have a weird issue where my R bridge seems to be running very slow. I'm training a DNN using Keras for R with tensorflow backend. Training takes some time, which is expected (three layers with 20 neurons in each).

My issue is that during testing, it takes 5-10 seconds of time to call the neural.load-function.
With R debugging enabled, the console will output something like
"Load KerasDeepLearn_USDJPY_1" and then after 10 seconds or so
"neural.load('....KerasDeepLearn_USDJPY_1')

I'm not that familiar with the bridge, but the long wait between the Load message from Zorro and the subsequent neural.load message from R seems strange to me. I would expect the log from R milliseconds after the call from Zorro.

I've attached my files (.R extensions are apparently not allowed by the file manager, so I renamed it to .txt)

I'm running Zorro 1.74.2 with R 3.4.1


Attachments
KerasDeepLearn.c (83 downloads)
KerasDeepLearn.txt (89 downloads)



Edited by Dalla (12/29/17 09:55)

Top
#470183 - 01/01/18 18:11 Re: R bridge running very slow [Re: Dalla]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26372
Loc: Frankfurt
We have not observed bad slowness with Keras so far, but you could look into it by adding timer calls to the neural() function.

Top
#470189 - 01/01/18 22:47 Re: R bridge running very slow [Re: jcl]
Dalla Offline
Member

Registered: 02/24/17
Posts: 263
I added a timer to the neural load function like this
Code:
if(mode == NEURAL_LOAD) {
		printf("nLoad %s",strrchr(Data,'')+1);
		timer();
		int neuralLoadReturnCode = Rx(strf("neural.load('%s')",slash(Data)),3); 		
		printf("nNEURAL_LOAD call time = %.3f ms",timer()); 
		return neuralLoadReturnCode;
 	}



This is the output (only included first five model calls)
Code:
Load KerasDeepLearn_USDJPY_1.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_1.ml')
NEURAL_LOAD call time = 14981.734 ms
Load KerasDeepLearn_USDJPY_2.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_2.ml')
NEURAL_LOAD call time = 11214.417 ms
Load KerasDeepLearn_USDJPY_3.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_3.ml')
NEURAL_LOAD call time = 13523.271 ms
Load KerasDeepLearn_USDJPY_4.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_4.ml')
NEURAL_LOAD call time = 17167.255 ms
Load KerasDeepLearn_USDJPY_5.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_5.ml')
NEURAL_LOAD call time = 19805.827 ms



As you can see the calls take 10 to 20 seconds

I should add the the models are small, only ~70kb in size
I also tried setting up a completly new environment on AWS and ran the same script, but the issue remains


Edited by Dalla (01/01/18 22:51)

Top
#470192 - 01/02/18 09:56 Re: R bridge running very slow [Re: Dalla]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26372
Loc: Frankfurt
Hmm, when I run the Deeplearn script with a 2-layer Keras net, I'm also getting about 500 ms loading time. That's indeed a bit slow. I don't know what takes so long.

Maybe you can get some hints about how to speed up loading on Keras or Tensorflow forums - otherwise you have to live with it.

Top
#470196 - 01/02/18 11:06 Re: R bridge running very slow [Re: jcl]
Dalla Offline
Member

Registered: 02/24/17
Posts: 263
Thanks, good to know it's not an issue on my side at least. Ive found others having similar issues, but no solution yet.

Top
#473115 - 06/14/18 12:33 Re: R bridge running very slow [Re: Dalla]
JRA Online
Newbie

Registered: 10/30/17
Posts: 17
Hi Dalla:

Have you tested the GPU version? Im running it but at the moment its not an improvement over CPU.

Top



Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de