Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (AndrewAMD, Imhotep), 567 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 2 1 2
_bmap shader feature wasting resources...? #468565
10/10/17 12:26
10/10/17 12:26
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Superku Offline OP
Senior Expert
Superku  Offline OP
Senior Expert

Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Hello! - You can skip this segment if you're short on time. --->
I was wondering why my game's performance dropped notably and steadily over the last year or 2 despite optimizing it regularly (in an actual level it sometimes only ran as well as DOOM 2016 on max settings on my machine, which is a little ridiculous). After writing a basic profiler and other stuff I found out something strange:

- Everything deactivated, no pp, no (other) functions running. Simple test scene (A), a couple of objects and blocks, most of the screen empty blue sky -> 500 fps (~170 fps with actual game stuff running on the same scene, no pp - I know you have to take the ms into account for best comparison).

- Is A8 that "slow"? No: In another project (B), a first person prototype with full post processing (including SSAO and godrays), movement and other code and over 100k triangles visible - meaning something way more demanding - I get over 400fps on 1920x1080 as well, 500fps on the next lower resolution. -> Problem had to be somewhere else.

- Turns out, when I level_load(NULL) from scene (A) on the press of a button the FPS only jumps up to about 750 with no functions running, not a lot more than in the other actual game (B) on lower resolutions. If I level_load(NULL) immediately or a frame after I get 2500fps. Hm!

-------------------->

This led me to the following self contained example: http://superku.de/materialTest.zip
Click to reveal..
Code:
///////////////////////////////
#include <acknex.h>
#include <default.c>
///////////////////////////////

BMAP* bmpTest = "tex1024x1024.tga";
BMAP* bmpTestb = "tex1024x1024b.tga";

void levelNull()
{
	level_load(NULL);
}

void main()
{
	fps_max = 9999;
	video_mode = 10;
	level_load(NULL);
	int i;
	for(i = 0; i < 500; i++)
	{
		MATERIAL* matNew = mtl_create();
		if(1)
		{
			you = ent_create(CUBE_MDL,vector(100+random(100),random(100)-50,random(100)-50),NULL); // "testCube.mdl"
			effect_load(matNew,"test.fx");
		}
		else
		{
			you = ent_create("testCube.mdl",vector(100+random(100),random(100)-50,random(100)-50),NULL);
			effect_load(matNew,"testEntSkin.fx");
		}
		your.material = matNew;
	}
	on_space = levelNull;
	def_debug();
	while(1)
	{
		DEBUG_VAR(16.0/time_frame,200);
		wait(1);
	}
}


It takes a couple seconds to create 500 materials but it makes the problem more apparent. Please start it once with if(1), note the performance before and after you press space. Then do it again but change to the second case if(0) first.

- if(1): Two 3MB textures are requested from the materials via _bmap feature.
~290fps -> space -> ~580fps
- if(0): Model with two external 3MB textures and entSkin1/2.
~600fps -> space -> ~3800fps




This leads me to a few questions:
1) Does it make sense that the shader with *_bmap textures is rendering (texture swapping?) slower than the entSkin alternative? If so, why?
2) Why does the application only run at < 600fps in the first case after level_load(NULL)? It seems as if the textures are still being updated for every material. Oh, when you def_move/ [0] and turn the camera around so nothing gets rendered you (more or less obviously) pretty much get the same FPS before and after level_load(NULL).
3) Can this be "fixed"?


I have 816 fx files in my project folder, I guess I use 750+ of them. Most of them share a bunch of textures using the *_bmap feature, and oftentimes 1-4 entSkins as additional textures. The 1700+ models have a corresponding setup. Changing it all to entSkins (where possible because of the amount of textures) would be a pain and I'd highly prefer a code solution.

Thanks in advance for your time.


"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual

Check out my new game: Pogostuck: Rage With Your Friends
Re: _bmap shader feature wasting resources...? [Re: Superku] #468599
10/11/17 15:07
10/11/17 15:07
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
When a level is loaded, several loops are active, one is a loop over all shaders that are assigned to something. This loops checks all shader variables for possible updates. I do not know why a particular shader is faster or slower than another one - I guess that simply depends on the environnment.

The difference between 500 and 1000 fps is 1 millisecond. Can this millisecond can be "fixed", maybe reduced to half a millisecond? Probably not, but even if it could, I don't think anyone would do that.

Re: _bmap shader feature wasting resources...? [Re: jcl] #468604
10/11/17 15:56
10/11/17 15:56
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Superku Offline OP
Senior Expert
Superku  Offline OP
Senior Expert

Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Originally Posted By: jcl
This loops checks all shader variables for possible updates.

It never stops updating those shaders though I think when I use the _bmap feature, as outlined by the bold text and the two cases in my previous post. This has to be a bug.


"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual

Check out my new game: Pogostuck: Rage With Your Friends
Re: _bmap shader feature wasting resources...? [Re: Superku] #468608
10/11/17 16:10
10/11/17 16:10
Joined: Jun 2009
Posts: 2,210
Bavaria, Germany
Kartoffel Offline
Expert
Kartoffel  Offline
Expert

Joined: Jun 2009
Posts: 2,210
Bavaria, Germany
Maybe there's a problem that the texture is assigned to the shader more often than it should, or the texture it's being copied into a shader resource for each material? I'm not sure.

as a possible "workaround", you could try to set the texture manually (in case you're exceeding the entSkin1-4 limit):

Code:
void material_set_texture(MATERIAL * mat, BMAP * bmap, char * name)
{
	LPDIRECT3DTEXTURE9 tex = (LPDIRECT3DTEXTURE9)bmap->d3dtex;
	
	if(tex == NULL)
	{
		printf("error in 'material_set_texture()': empty d3dtex pointer");
		return;
	}
	
	if(mat == NULL)
	{
		printf("error in 'material_set_texture()': empty material pointer");
		return;
	}
	
	if(name == NULL)
	{
		printf("error in 'material_set_texture()': empty name pointer");
		return;
	}
	
	LPD3DXEFFECT eff = (LPD3DXEFFECT)mat->d3deffect;
	
	if(eff != NULL)
	{
		eff->SetTexture(name, tex);
	}
	else
	{
		printf("error in 'material_set_texture()': empty effect");
		return;
	}
}


I've used it a few times, although I'm not entirely sure if the above function works since I haven't used it in a while. Note that the material's shader has to be compiled before executing this function (otherwise the d3deffect-pointer will be invalid/empty). If I remember correctly you can force the material to compile by using effect_load. Otherwise you have to wait until the material has been used.

Last edited by Kartoffel; 10/11/17 16:15. Reason: grammar

POTATO-MAN saves the day! - Random
Re: _bmap shader feature wasting resources...? [Re: Kartoffel] #468615
10/11/17 16:23
10/11/17 16:23
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Superku Offline OP
Senior Expert
Superku  Offline OP
Senior Expert

Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
I was just looking into SetTexture(), so thanks for the function and endorsement! - I'm dreading changing all shaders and models of course though.

The _bmap updates waste around 2ms of performance per frame in my game (if I'm not mistaken), probably more when I go through all the levels.


"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual

Check out my new game: Pogostuck: Rage With Your Friends
Re: _bmap shader feature wasting resources...? [Re: Superku] #468619
10/11/17 16:55
10/11/17 16:55
Joined: Jun 2009
Posts: 2,210
Bavaria, Germany
Kartoffel Offline
Expert
Kartoffel  Offline
Expert

Joined: Jun 2009
Posts: 2,210
Bavaria, Germany
Glad I could help! Let me know if you run into any problems, I remember having some issues when I tried using SetTexture()


POTATO-MAN saves the day! - Random
Re: _bmap shader feature wasting resources...? [Re: Kartoffel] #468621
10/11/17 17:07
10/11/17 17:07
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Superku Offline OP
Senior Expert
Superku  Offline OP
Senior Expert

Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
I will!

New test scenario, _bmap vs manual setTexture: http://superku.de/materialTest2.zip

Click to reveal..
Code:
///////////////////////////////
#include <acknex.h>
#include <default.c>
#include <d3d9.h>
///////////////////////////////

BMAP* bmpTest = "tex1024x1024.tga";
BMAP* bmpTestb = "tex1024x1024b.tga";

void levelNull()
{
	level_load(NULL);
}

void material_set_texture(MATERIAL * mat, BMAP * bmap, char * name)
{
	LPDIRECT3DTEXTURE9 tex = (LPDIRECT3DTEXTURE9)bmap->d3dtex;
	
	if(tex == NULL)
	{
		printf("error in 'material_set_texture()': empty d3dtex pointer");
		return;
	}
	
	if(mat == NULL)
	{
		printf("error in 'material_set_texture()': empty material pointer");
		return;
	}
	
	if(name == NULL)
	{
		printf("error in 'material_set_texture()': empty name pointer");
		return;
	}
	
	LPD3DXEFFECT eff = (LPD3DXEFFECT)mat->d3deffect;
	
	if(eff != NULL)
	{
		eff->SetTexture(name, tex);
	}
	else
	{
		printf("error in 'material_set_texture()': empty effect");
		return;
	}
}

MATERIAL* mats[500];

void main()
{
	fps_max = 9999;
	video_mode = 10;
	level_load(NULL);
	
	int testMode = 0; // (0 == manual), (1 == _bmap)
	int i;
	for(i = 0; i < 500; i++)
	{
		MATERIAL* matNew = mtl_create();
		if(testMode)
		{
			you = ent_create(CUBE_MDL,vector(100+random(100),random(100)-50,random(100)-50),NULL); // 
			effect_load(matNew,"testBmap.fx");
		}
		else
		{
			you = ent_create(CUBE_MDL,vector(100+random(100),random(100)-50,random(100)-50),NULL);
			effect_load(matNew,"test.fx");
		}
		your.material = matNew;
		mats[i] = matNew;
	}
	on_space = levelNull;
	while(1)
	{
		DEBUG_VAR(time_frame*1000.0/16.0,200);
		if(key_e)
		{
			for(i = 0; i < 500; i++)
			{
				material_set_texture(mats[i],bmpTest,"bmpTest_bma");
				material_set_texture(mats[i],bmpTestb,"bmpTestb_bma");
			}
		}
		if(total_frames < 3)
		{
			draw_quad(bmpTest,vector(0,0,0),NULL,NULL,NULL,NULL,100,0);
			draw_quad(bmpTestb,vector(0,0,0),NULL,NULL,NULL,NULL,100,0);
		}
		wait(1);
	}
}


Results:

- testMode = _bmap: 2.9-3ms per frame, over 1.5ms after level_load(NULL) (or no entities on screen)

- testMode = manual: 1.45ms per frame, 0.18ms after level_load(NULL) (or no entities on screen)

Holding down the [e] key to call setTexture() for 2 textures for 500 materials makes a difference of ~0.04-0.06ms.


Obversation: 1.5ms additional computation time for the _bmap approach. If it was the fault of setTexture() calls, why is it 1.5ms and not < 0.06ms?


"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual

Check out my new game: Pogostuck: Rage With Your Friends
Re: _bmap shader feature wasting resources...? [Re: Superku] #468648
10/12/17 08:51
10/12/17 08:51
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
When a shader gets a _bmap variable, its texture is updated once per frame from the bitmap. This is not a bug, but just the purpose of the _bmap variable. Just as _var or _flt variables that are also updated once per frame.

Code:
if(once_per_frame) {
            if (paramDesc.Type == D3DXPT_TEXTURE) {
                char* found = MAT_FindParam(paramDesc,PID_BMAP);
                if (found) {
                    BMAP_T* bmap = (BMAP_T*)CS->GetObject(found,BMAP_IDX);
                    if (bmap)
                        m->d3deffect->SetTexture(hParam,D3D_Tex(bmap));
                    else
                        m->d3deffect->SetTexture(hParam,NULL);
                }
            }
}


Re: _bmap shader feature wasting resources...? [Re: jcl] #468655
10/12/17 11:12
10/12/17 11:12
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Superku Offline OP
Senior Expert
Superku  Offline OP
Senior Expert

Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
Hmmm.
In case it was working as intended I started to write my own system/ approach yesterday with the hopes of it performing better for some reason. First I wrote a converter to replace all _bmap occurrences in my effect files with _AUTO. That converter scans through all *.c and *.h files as well for global materials, then outputs a single effect setup file with a structure as follows:
Quote:
fxFileName[material1,material2,...](textureAUTO1,textureAUTO1,...)
(In case some materials use the same effect file - probably unnecessary but I had material events and mtl switching in mind.)

Now an AUTOTextureInit() function reads that file, gets MATERIAL* and BMAP* pointers and saves the corresponding *_AUTO names.
An AUTOTextureUpdate() loop sets all _AUTO textures of each material. It's working, and luckily more performant - for whatever reason - than _bmap/ your code snippet.

I now get 1024fps in my "simple test scene (A)" (up from 500), and 1260fps if I don't call the update function (key press).
During actual gameplay in a real level I have found a performance increase of ~1.3ms (140->170, 170->220fps).


If I don't run the update function (for all materials) I obviously get a small speed boost on top of that. I'd like to do that, preferably by setting MATERIAL (and/ or BMAP?) flags (request update/ is postprocessing material/ ...).
This leads me to my last question for now:

Quote:
// note: all other flag bits are used internally and must not be modified!!
[...]
// MATERIAL flags
#define PASS_SOLID (1<<11) // enforce solid pass
#define TANGENT (1<<16)
#define ENABLE_RENDER (1<<17) // call event before object rendering
#define ENABLE_VIEW (1<<18) // call event before view rendering
#define ENABLE_TREE (1<<19) // call event before entity sorting
#define OVERRIDE (1<<22) // override view material
#define AUTORELOAD (1<<24) // reload effects when changed

// BMAP flags
#define BMPF_ARGB (1<<8) // bitmap contains alpha channel
#define BMPF_SAVE (1<<9) // bitmap was modified
#define BMPF_MIPMAPS (1<<10) // bmap->pixels contains 3 mipmaps


Could you please tell me 2-3 flags/ bits per MATERIAL and BMAP struct that are *not* used internally, despite the note?

Thanks for your time and the insight.


"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual

Check out my new game: Pogostuck: Rage With Your Friends
Re: _bmap shader feature wasting resources...? [Re: Superku] #468657
10/12/17 13:22
10/12/17 13:22
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
In both structs the flags 1<<5,1<<6,1<<7 are unused.

Page 1 of 2 1 2

Moderated by  jcl, Nems, Spirit, Tobias 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1