Gamestudio Links
Zorro Links
Newest Posts
folder management functions
by 7th_zorro. 04/16/24 13:19
lookback setting performance issue
by 7th_zorro. 04/16/24 03:08
zorro 64bit command line support
by 7th_zorro. 04/15/24 09:36
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:48
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:46
LPDIRECT3DCUBETEXTUR
E9

by Ayumi. 04/12/24 11:00
Sam Foster Sound | Experienced Game Composer for Hire
by titanicpiano14. 04/11/24 14:56
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (rki, Ayumi), 475 guests, and 2 spiders.
Key: Admin, Global Mod, Mod
Newest Members
11honza11, ccorrea, sakolin, rajesh7827, juergen_wue
19045 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
R bridge running very slow #470150
12/29/17 08:54
12/29/17 08:54
Joined: Feb 2017
Posts: 369
D
Dalla Offline OP
Senior Member
Dalla  Offline OP
Senior Member
D

Joined: Feb 2017
Posts: 369
I have a weird issue where my R bridge seems to be running very slow. I'm training a DNN using Keras for R with tensorflow backend. Training takes some time, which is expected (three layers with 20 neurons in each).

My issue is that during testing, it takes 5-10 seconds of time to call the neural.load-function.
With R debugging enabled, the console will output something like
"Load KerasDeepLearn_USDJPY_1" and then after 10 seconds or so
"neural.load('....KerasDeepLearn_USDJPY_1')

I'm not that familiar with the bridge, but the long wait between the Load message from Zorro and the subsequent neural.load message from R seems strange to me. I would expect the log from R milliseconds after the call from Zorro.

I've attached my files (.R extensions are apparently not allowed by the file manager, so I renamed it to .txt)

I'm running Zorro 1.74.2 with R 3.4.1

Attached Files
KerasDeepLearn.c (557 downloads)
KerasDeepLearn.txt (484 downloads)
Last edited by Dalla; 12/29/17 08:55.
Re: R bridge running very slow [Re: Dalla] #470183
01/01/18 17:11
01/01/18 17:11
Joined: Jul 2000
Posts: 27,978
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,978
Frankfurt
We have not observed bad slowness with Keras so far, but you could look into it by adding timer calls to the neural() function.

Re: R bridge running very slow [Re: jcl] #470189
01/01/18 21:47
01/01/18 21:47
Joined: Feb 2017
Posts: 369
D
Dalla Offline OP
Senior Member
Dalla  Offline OP
Senior Member
D

Joined: Feb 2017
Posts: 369
I added a timer to the neural load function like this
Code:
if(mode == NEURAL_LOAD) {
		printf("nLoad %s",strrchr(Data,'')+1);
		timer();
		int neuralLoadReturnCode = Rx(strf("neural.load('%s')",slash(Data)),3); 		
		printf("nNEURAL_LOAD call time = %.3f ms",timer()); 
		return neuralLoadReturnCode;
 	}



This is the output (only included first five model calls)
Code:
Load KerasDeepLearn_USDJPY_1.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_1.ml')
NEURAL_LOAD call time = 14981.734 ms
Load KerasDeepLearn_USDJPY_2.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_2.ml')
NEURAL_LOAD call time = 11214.417 ms
Load KerasDeepLearn_USDJPY_3.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_3.ml')
NEURAL_LOAD call time = 13523.271 ms
Load KerasDeepLearn_USDJPY_4.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_4.ml')
NEURAL_LOAD call time = 17167.255 ms
Load KerasDeepLearn_USDJPY_5.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_5.ml')
NEURAL_LOAD call time = 19805.827 ms



As you can see the calls take 10 to 20 seconds

I should add the the models are small, only ~70kb in size
I also tried setting up a completly new environment on AWS and ran the same script, but the issue remains

Last edited by Dalla; 01/01/18 21:51.
Re: R bridge running very slow [Re: Dalla] #470192
01/02/18 08:56
01/02/18 08:56
Joined: Jul 2000
Posts: 27,978
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,978
Frankfurt
Hmm, when I run the Deeplearn script with a 2-layer Keras net, I'm also getting about 500 ms loading time. That's indeed a bit slow. I don't know what takes so long.

Maybe you can get some hints about how to speed up loading on Keras or Tensorflow forums - otherwise you have to live with it.

Re: R bridge running very slow [Re: jcl] #470196
01/02/18 10:06
01/02/18 10:06
Joined: Feb 2017
Posts: 369
D
Dalla Offline OP
Senior Member
Dalla  Offline OP
Senior Member
D

Joined: Feb 2017
Posts: 369
Thanks, good to know it's not an issue on my side at least. I´ve found others having similar issues, but no solution yet.

Re: R bridge running very slow [Re: Dalla] #473115
06/14/18 10:33
06/14/18 10:33
Joined: Oct 2017
Posts: 17
J
JRA Offline
Newbie
JRA  Offline
Newbie
J

Joined: Oct 2017
Posts: 17
Hi Dalla:

Have you tested the GPU version? I´m running it but at the moment it´s not an improvement over CPU.


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1