Gamestudio Links
Zorro Links
Newest Posts
Error 072 & Orphaned Rogue Trade - Bug?
by DdlV
Yesterday at 04:09
Why history files are different since version 1.83?
by kvm
08/16/18 17:39
Stop, Trail etc
by AndrewAMD
08/16/18 15:00
Performance reporting showing Annual and monthly % returns
by jcl
08/16/18 11:09
Trade only within regular trading hours
by Jeff1228
08/16/18 10:18
Issue with inconsistent test results
by kvm
08/15/18 17:24
Access string content through another string
by ChrstphFr
08/15/18 11:30
AUM Magazine
Latest Screens
AckCon'18 - Lotter vs the World 2 - Preview Release
Evonet Beta v1.0 : Online Project Development Template
Lake scene
Forgotten Tales - Demo version
Who's Online
15 registered (mk_1, 3DCat, Michael_Schwarz, Angerbeer, alibaba, Ivan3333, LDM, 5 invisible), 443 Guests and 6 Spiders online.
Key: Admin, Global Mod, Mod
Newest Members
Ulenspegel, toto, FloMi, behr065, OptimusPrime
18027 Registered Users
Topic Options
Rate This Topic
#470150 - 12/29/17 09:54 R bridge running very slow
Dalla Offline
Member

Registered: 02/24/17
Posts: 271
I have a weird issue where my R bridge seems to be running very slow. I'm training a DNN using Keras for R with tensorflow backend. Training takes some time, which is expected (three layers with 20 neurons in each).

My issue is that during testing, it takes 5-10 seconds of time to call the neural.load-function.
With R debugging enabled, the console will output something like
"Load KerasDeepLearn_USDJPY_1" and then after 10 seconds or so
"neural.load('....KerasDeepLearn_USDJPY_1')

I'm not that familiar with the bridge, but the long wait between the Load message from Zorro and the subsequent neural.load message from R seems strange to me. I would expect the log from R milliseconds after the call from Zorro.

I've attached my files (.R extensions are apparently not allowed by the file manager, so I renamed it to .txt)

I'm running Zorro 1.74.2 with R 3.4.1


Attachments
KerasDeepLearn.c (91 downloads)
KerasDeepLearn.txt (99 downloads)



Edited by Dalla (12/29/17 09:55)

Top
#470183 - 01/01/18 18:11 Re: R bridge running very slow [Re: Dalla]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26431
Loc: Frankfurt
We have not observed bad slowness with Keras so far, but you could look into it by adding timer calls to the neural() function.

Top
#470189 - 01/01/18 22:47 Re: R bridge running very slow [Re: jcl]
Dalla Offline
Member

Registered: 02/24/17
Posts: 271
I added a timer to the neural load function like this
Code:
if(mode == NEURAL_LOAD) {
		printf("nLoad %s",strrchr(Data,'')+1);
		timer();
		int neuralLoadReturnCode = Rx(strf("neural.load('%s')",slash(Data)),3); 		
		printf("nNEURAL_LOAD call time = %.3f ms",timer()); 
		return neuralLoadReturnCode;
 	}



This is the output (only included first five model calls)
Code:
Load KerasDeepLearn_USDJPY_1.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_1.ml')
NEURAL_LOAD call time = 14981.734 ms
Load KerasDeepLearn_USDJPY_2.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_2.ml')
NEURAL_LOAD call time = 11214.417 ms
Load KerasDeepLearn_USDJPY_3.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_3.ml')
NEURAL_LOAD call time = 13523.271 ms
Load KerasDeepLearn_USDJPY_4.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_4.ml')
NEURAL_LOAD call time = 17167.255 ms
Load KerasDeepLearn_USDJPY_5.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_5.ml')
NEURAL_LOAD call time = 19805.827 ms



As you can see the calls take 10 to 20 seconds

I should add the the models are small, only ~70kb in size
I also tried setting up a completly new environment on AWS and ran the same script, but the issue remains


Edited by Dalla (01/01/18 22:51)

Top
#470192 - 01/02/18 09:56 Re: R bridge running very slow [Re: Dalla]
jcl Offline

Chief Engineer

Registered: 07/22/00
Posts: 26431
Loc: Frankfurt
Hmm, when I run the Deeplearn script with a 2-layer Keras net, I'm also getting about 500 ms loading time. That's indeed a bit slow. I don't know what takes so long.

Maybe you can get some hints about how to speed up loading on Keras or Tensorflow forums - otherwise you have to live with it.

Top
#470196 - 01/02/18 11:06 Re: R bridge running very slow [Re: jcl]
Dalla Offline
Member

Registered: 02/24/17
Posts: 271
Thanks, good to know it's not an issue on my side at least. Ive found others having similar issues, but no solution yet.

Top
#473115 - 06/14/18 12:33 Re: R bridge running very slow [Re: Dalla]
JRA Offline
Newbie

Registered: 10/30/17
Posts: 17
Hi Dalla:

Have you tested the GPU version? Im running it but at the moment its not an improvement over CPU.

Top



Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de