1 registered members (SBGuy),
652
guests, and 3
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
video memory and d3d_texfree
#464863
03/15/17 19:49
03/15/17 19:49
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Hello, I am developing my game on a laptop with a 840m gpu which has 2 gb ddr3 video memory ( http://www.notebookcheck.net/NVIDIA-GeForce-840M.105681.0.html). When I check d3d_texfree in my current game project, I seem to have alot of video memory free (on startup its something like 4095 and than goes to circa 3100). Except when I am nearing circa 3100 it eventually gives error messages when I am placing more models in the map with fairly big textures (2048x2048 with normal maps too). (It does not seem to be related to a specific model or texture). The error message -> "Error E2005 - Can't create D3D texture The texture format is invalid or unsupported by your video card. Very old video cards often don't support DirectX 9 and can not create textures of the given size or format." So how come I get these messages when d3d_texfree seem to show that I have still enough free video memory? Thanks for taking the time.
Last edited by Reconnoiter; 03/15/17 19:49.
|
|
|
Re: video memory and d3d_texfree
[Re: Reconnoiter]
#465072
03/28/17 23:24
03/28/17 23:24
|
Joined: Jul 2004
Posts: 785 Serbia
Iglarion
User
|
User
Joined: Jul 2004
Posts: 785
Serbia
|
Unfortunately i also have this problem with randomly geting e2005 error and i really stuck with this. First i think that problem was in textures but after converting all textures today i got again this error. Why this error/crash appear randomly, and how i can be sure that texture is problem? For example i just now get error where i can see some untextured model on screen and now i know exactly what is suspicious texture (it's an tga texture 2048x2048x32). This is first time that this texture is problem. A big problem is i need to play my game almost one hour to reach this crash. Manual say that old video cards can't support some texture size, but i have a modern gpu (GTX 1060)and this should not be a problem. Any idea what else can make this crash, could it be possible that this is a engine bug beacuse i don't know what is wrong with this texture? Only what i can try now is resize all textures to 1024 dds and wait did crash will appear again.
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#465091
03/30/17 16:44
03/30/17 16:44
|
Joined: May 2005
Posts: 868 Chicago, IL
Dooley
User
|
User
Joined: May 2005
Posts: 868
Chicago, IL
|
I was able to solve it (I hope) by reducing a bunch of texture sizes. Whether the problem will reappear after extended play sessions is still a concern for me, but it's much more stable than it was before.
One thing I realized is that I could reduce the size of my normal maps without impacting the appearance as much.
I'm not sure what metric to use as far as how much texture size I can use, what's the limit, etc... it would be nice to have a specific number that we can make sure we stay under. Of course it would vary depending on video card, computer memory, nexus, etc...
I think this all falls under optimizing the game. I'm guessing that big name games have all sorts of tricks and methods of getting really big textures on screen seamlessly. Reduce, reuse, recycle ... that sort of thing.
|
|
|
Re: video memory and d3d_texfree
[Re: Dooley]
#465095
03/31/17 11:02
03/31/17 11:02
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Manual say that old video cards can't support some texture size, but i have a modern gpu (GTX 1060)and this should not be a problem. Any idea what else can make this crash, could it be possible that this is a engine bug beacuse i don't know what is wrong with this texture? , I dont think your videocard is the bottleneck, but its the 32 bit Acknex engine I guess. I sometimes get this error too when I switch resolutions (video_set) and it's worrying me. , I am not sure but perhaps higher video resolutions use more ram? Like they always say that for 4k or VR you need alot of ram. I had it all (deactivating render stages, switching resolutions, freeing memory and allocating textures for the new screen size) logged into acklog.txt and a console but can't seem to find any of those logs right now. Will report and come back here when I have more information to share. , that would be helpful ty. I also worked alot on this problem and I simply couldnt find a real fix. It usually worked using dds textures which are compressed and damn small
, tnx for the info, they seem to be the way to go when lossless/perfect image is not important. Though with .dds you cannot read the texture pixels right? (I am using pixel_for_bmap and pixel_to_bmap for a few things) One thing I realized is that I could reduce the size of my normal maps without impacting the appearance as much. , reducing normal maps indeed helps, I did that for e.g. terrain where the difference in loss is not that noticeable. The 'free mb' in the f11/diag window gives the same values as d3d_texfree. I noticed that when that gives e.g. 3667 (for a small intro map) in windows task manager it shows that Acknex uses 635 mb. For another map the d3d_texfree/free mb was something like ~3340 and in task manager Acknex used ~1620 mb (which is about the highest I can let Acknex get before I get that error again). There seem to be some difference between these values but I dont fully understand why. I also tried setting nexus to e.g. 200 in Sed preferences but that doesn't seem to help either. I did notice that other 32 bit programs like e.g. jasc paint shop pro also that that limit of ~1650 mb. While 64 bit corel paint shop pro can go much futher to e.g. somethig like >5600 mb in the task manager. So checking task manager is probably the most reliable way to check if the game uses much ram. I am personally using mostly .bmp. While the textures are not that high (with 2048x2048 as a max), I guess stuff like normal maps and shadow maps for lights double/triple their memory usage. In my game editor that doesn't use these shaders I could place alot more stuff (still not that much though). So just keep to small maps than? Or maybe ent_purge some stuff (like terrain or buildings) when not visible? (though that does give a bit of lag when backtracking etc within maps...)
|
|
|
Re: video memory and d3d_texfree
[Re: Iglarion]
#465177
04/09/17 23:34
04/09/17 23:34
|
Joined: May 2005
Posts: 868 Chicago, IL
Dooley
User
|
User
Joined: May 2005
Posts: 868
Chicago, IL
|
I would try making a backup copy of your game, and reducing all textures by half, i.e. 2048 to 1024, 1024 to 512, etc...
This way, you can see if the problem is fixed, and also see if you can live with the smaller texture sizes. I was not thrilled to reduce all my texture sizes, but it's better than having the game crash.
Also, I believe you can go up to 500 with your nexus. I know the manual recommends keeping it lower, but I think that was written a long time ago... this could give you some extra needed memory.
|
|
|
Re: video memory and d3d_texfree
[Re: Dooley]
#465180
04/10/17 10:45
04/10/17 10:45
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
I don't think the issue really lies with big textures. I barely use any (less than 128MB maybe) in my game, have near unlimited RAM and VRAM and yet it still crashes with DirectX errors, right now more than ever (which is not a bad thing for debugging). There is something wrong with this engine I fear, I wish it was a user problem. EDIT: The error: Error E2005: Can't create DirectX texture unnamed D3D_TexCreate: 8007000e - E_OUTOFMEMORY How is "unnamed" set (or not set), could you, Mr. Lotter, use that field to add debugging information?
Last edited by Superku; 04/10/17 13:10.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: Dooley]
#465192
04/10/17 16:15
04/10/17 16:15
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
@Iglarion, your game is a sidescroller right? Have you tried ent_purge big models/textures that the player has passed? @Superku, how much memory does task manager say your game/project is using? Maybe you can debug your problem with dx debug runtime; https://www.microsoft.com/en-us/download/details.aspx?id=6812https://msdn.microsoft.com/en-us/library/windows/desktop/bb173355(v=vs.85).aspx https://legalizeadulthood.wordpress.com/2009/06/28/direct3d-programming-tip-5-use-the-debug-runtime/ Also, I believe you can go up to 500 with your nexus. I know the manual recommends keeping it lower, but I think that was written a long time ago... this could give you some extra needed memory.
, this does not seem to work for me (maybe because I dont use blocks but only models in my maps/.wmb). Cutting all normal maps in half is so far the best one for me since it doesn't decrease visual quality much. Also I noticed that Shade-C uses a noticeable amount of extra memory (probably for shadowmaps etc). I found this quote also informative: In case you're not familar with address space starvation, on a 32-bit operating system, each executable has 4GB of address space to use, between 0x00000000 and 0xffffffff. The top 2GB is reserved by the operating system, some address ranges are reserved (E.g. the low 64KB (I think? Maybe more?)), and the EXE file and all the various system DLLs loaded into the process usually means that a single application has around 1GB to 1.5GB of free space to use for the heap (At most, the address space could be fragmented meaning there's not one contiguous chunk of memory that size).
( https://www.gamedev.net/topic/616046-d3dxcreatecubetexture-fails-with-e-outofmemory/) Lastly found another A8 developer that had this problem to and solved it (maybe it helps): https://steamcommunity.com/app/336200/discussions/0/618453594750526027/
|
|
|
Re: video memory and d3d_texfree
[Re: Reconnoiter]
#465195
04/10/17 19:00
04/10/17 19:00
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
It's at 1066MB in development mode and 232MB in a published build (which temporarily goes up to 296MB for 5+ seconds after changing resolutions on 1920x1080, which is a tad strange IMO). That DirectX SDK/ debugger stuff sounds like a good idea, too bad this does not work on Win7 SP1, Win 8 and Win 10 though. -.- It's just not supported anymore. EDIT: Uhh, got it running on an old laptop but there the crash has not happened so far - obviously. EDIT2: Activating the debug version of DirectX 9 results in a different error, usually related to outdated DirectX versions (apparently): D3D_DrawPrimitive: D3DERR_INVALIDCALL D3D_DrawPrimitive: D3DERR_INVALIDCALL 0 lmaps 0 textures D3D_DrawPrimitive: D3DERR_INVALIDCALL Malfunction W1246: DirectX driver failure: D3DERR_INVALIDCALL...ok LevelReady at 214.750 (listed as E1246 in the manual)
Last edited by Superku; 04/11/17 07:44.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: Reconnoiter]
#465200
04/11/17 19:03
04/11/17 19:03
|
Joined: Jul 2004
Posts: 785 Serbia
Iglarion
User
|
User
Joined: Jul 2004
Posts: 785
Serbia
|
@Iglarion, your game is a sidescroller right? Have you tried ent_purge big models/textures that the player has passed? My game has multiple views, and i already use ent_purage on the few places where this is possible. I haven't yet had a time to play my game a little longer time to check whether the error still occurs, currently i prepare complete game for this beacuse i need almost one+ hour to reach crash and before test i want to go once again through all suspicious parts. First i must see when exactly the e2005 occurs and where. I just found my topic from the year 2013 when i have the same problem but without any error message. It seems that the issue is only reduced after converting textures to DDS, but is still here. The RAM usage for my game is 1280MB in development mode and i try to keep all things as simple as possible - lowpoly models, extern dds textures 512 and 1024 and just five 2048 for terrain colormap and shadowmap. Error "Can't create DirectX texture unnamed" in my case also happens only after switch resolution. Error "Can't create DirectX texture skin(number)" appear after i pass trought many of my levels and at begining of new loaded level. The part that scares me is that Superku not used at all too much video memory, and especially Starlaxis, and the question is how much reducing memory consumption can help in my game. Maybe we also should contact the author of Starlaxis and see if his update actually removed e2005 error from his game and what was the problem. I also afraid there is something wrong with engine and if this is the truth it will be very difficult for us to isolate this error for support.
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#465240
04/13/17 20:34
04/13/17 20:34
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
@Iglarion, yeah contacting Starlaxis dev is a good idea. @Superku, could try setting this d3d_texdepth = 16; before level loading? (so e.g. in your main function) I tried that and it saved me ~25% video memory without noticeable losses so far (not sure if its important but my maps only have models and not blocks). The RAM usage shows pretty much the same value (~1055MB), both for a nexus of 150MB and 250MB. Does that make sense? , from what I read every 32 app that uses above 1 gb ram could technically give the error if the engine or used dll's etc use alot or there is to little ram and/or video ram in the computer itself. Maybe you use many plugins or such? Also the nexus does not seem make any difference for me too, I think it only matters for loading big maps with lots of blocks and wad textures or perhaps with big terrain shadow/light maps etc.
Last edited by Reconnoiter; 04/13/17 20:41.
|
|
|
Re: video memory and d3d_texfree
[Re: Ch40zzC0d3r]
#465242
04/13/17 20:43
04/13/17 20:43
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
x86 processes can only take a maximum of 2gb ram if the app wasnt compiled with a special flag (/LARGEADDRESSAWARE) which allows to address up to 4gb ram. , maybe its possible to enforce that flag?
Last edited by Reconnoiter; 04/13/17 20:43.
|
|
|
Re: video memory and d3d_texfree
[Re: Reconnoiter]
#465245
04/13/17 22:26
04/13/17 22:26
|
Joined: Jul 2004
Posts: 785 Serbia
Iglarion
User
|
User
Joined: Jul 2004
Posts: 785
Serbia
|
x86 processes can only take a maximum of 2gb ram if the app wasnt compiled with a special flag (/LARGEADDRESSAWARE) which allows to address up to 4gb ram. , maybe its possible to enforce that flag? But we here talking about out of memory problem if game use just 1GB, so something is definitely wrong here. I agre with Superku idea about one unofficial relase for us who have this problem. I just hope is possible to put some extra debuging in engine to finally isolate the problem. Jcl what do you think about this did you can help somehow? I really understand it's very difficult without enough information and test project say something more, but do you have any advice on what we should pay attention to find reason of this error. Too many people with a completely different games have this problem, but none of us has any idea why this is happening.
|
|
|
Re: video memory and d3d_texfree
[Re: Reconnoiter]
#465248
04/14/17 09:46
04/14/17 09:46
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
@Superku, could try setting this d3d_texdepth = 16; before level loading? (so e.g. in your main function) Never tried that but I'm not trying to avoid this error message, I'm trying to isolate (and indirectly fix) that bug or error. As I've said the game even gave me the DirectX error on my laptop (8GB RAM) while using less than 300 MB according to the task manager. Trying to avoid that error by various measures is not helping, it's rather counterproductive IMO.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465380
04/22/17 01:52
04/22/17 01:52
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
///////////////////////////////
#include <acknex.h>
#include <default.c>
///////////////////////////////
void main()
{
fps_max = 60;
video_mode = 6;
camera.stage = view_create(0);
set(camera.stage,PROCESS_TARGET);
level_load(NULL);
me = ent_create(CUBE_MDL,vector(128,0,0),NULL);
var videoSwitchCounter = 32;
var videoSwitchMode = 8;
while(1)
{
my.pan += 6*time_step;
draw_text(str_printf(NULL,"d3d_texbmaps: %.1f",(double)d3d_texbmaps),20,20,COLOR_WHITE);
videoSwitchCounter = maxv(videoSwitchCounter-time_step,0);
if(!videoSwitchCounter)
{
videoSwitchCounter = 16;
videoSwitchMode++;
if(videoSwitchMode > 12) videoSwitchMode = 8;
video_switch(videoSwitchMode,0,2);
}
wait(1);
}
}
d3d_texbmaps rises continuously (except a small bump going from highest to lowest resolution). Let a similar more resource heavy test run until > 65GB. Does that make sense? Is this just a bug with the variable d3d_texbmaps (like that one nexus variable bug) or something else with the memory allocation for the post processing stage (target)?
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465397
04/24/17 15:01
04/24/17 15:01
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
But can't you just check and print the d3d_tex... variables in several steps during initialization of your game? This should tell you where the problem is. Maybe some particularly large textures. [...] But you should still be able to find the problem this way when you don't change the resolution.
I'm sorry but I don't really understand how. Here's the IMO relevant part of the acklog just created:
Video memory found: 4094 MB
main_event() at frame 4... (nexus: 31.7/ 150.0) (sys_memory: 219.3) (d3d_texfree: 4072.0) (d3d_texbmaps: 1.0) (d3d_texskins: 0.0) (d3d_texsmaps: 0.0) (d3d_texsurfs: 0.0).
create_rendertargets() START at frame 17... (nexus: 35.8/ 150.0) (sys_memory: 219.3) (d3d_texfree: 4071.0) (d3d_texlimit: 8192.0) (d3d_texbmaps: 1.0) (d3d_texskins: 0.0) (d3d_texsmaps: 0.0) (d3d_texsurfs: 0.0)
bmap_createblackDEBUG(1920,1080,24)...DONE. (CAMTARG1, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(1920,1080,24)...DONE. (KUREF, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(1920,1080,24)...DONE. (REFRAC, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(1920,1080,24)...DONE. (CAMBMAP, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(960,540,24)...DONE. (COMBINE, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(480,270,24)...DONE. (BLOOM2, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(1920,1080,24)...DONE. (COMBINE2, d3d_texbmaps: 1.0)
bmap_createblackDEBUG(1920,1080,24)...DONE. (LOADDISPLAY, d3d_texbmaps: 1.0)DONE
MEMORY NOW (after all bmap creations): (nexus: 35.8/ 150.0) (sys_memory: 219.3) (d3d_texfree: 4071.0) (d3d_texlimit: 8192.0) (d3d_texbmaps: 1.0) (d3d_texskins: 0.0) (d3d_texsmaps: 0.0) (d3d_texsurfs: 0.0) (sum: 1.0MB)
level_load(NULL)
main_event() at frame 20... (nexus: 42.8/ 150.0) (sys_memory: 219.4) (d3d_texfree: 4071.0) (d3d_texbmaps: 26.0) (d3d_texskins: 0.0) (d3d_texsmaps: 0.0) (d3d_texsurfs: 0.0)
main_event() at frame 21... (nexus: 42.8/ 150.0) (sys_memory: 219.4) (d3d_texfree: 4045.0) (d3d_texbmaps: 111.3) (d3d_texskins: 0.0) (d3d_texsmaps: 0.0) (d3d_texsurfs: 0.0)
main_event() at frame 22... (nexus: 42.8/ 150.0) (sys_memory: 219.4) (d3d_texfree: 3993.0) (d3d_texbmaps: 111.3) (d3d_texskins: 0.0) (d3d_texsmaps: 0.0) (d3d_texsurfs: 0.0)
level_load(...)
main_event() at frame 57... (nexus: 61.2/ 150.0) (sys_memory: 219.4) (d3d_texfree: 3993.0) (d3d_texbmaps: 111.3) (d3d_texskins: 0.0) (d3d_texsmaps: 0.4) (d3d_texsurfs: 0.1)
// now playing the game normally, task manager RAM usage at 1060MB
main_event() at frame 394... (nexus: 64.0/ 150.0) (sys_memory: 219.4) (d3d_texfree: 3962.0) (d3d_texbmaps: 127.5) (d3d_texskins: 4.5) (d3d_texsmaps: 0.4) (d3d_texsurfs: 0.1)
All values are the same as they used to be, right now it doesn't crash though. I had the d3d_... stuff in place before when it was crashing every time, did not see anything unusual there. I don't think I've even added a single new texture or changed an old one (in size), just stuff in the code. This cannot be "another" compiler related issue like in some past bug hunts where changing stuff in the code lead to crashes (mostly on MDL loading I think), right?
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465399
04/24/17 16:32
04/24/17 16:32
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Yes, sorry, I meant the old "Error E2005: Can't create DirectX texture unnamed D3D_TexCreate: 8007000e - E_OUTOFMEMORY" message. Got this message again 15min ago after switching resolutions a couple of times. However, I noticed I can continue playing after closing all of these messages. In fact, depending on the size of the resolution I get 0, 1 (just once) or 8 E_OUTOFMEMORY errors which is the number of newly (re-)allocated render targets. The error messages appear at the end of the frame (esp. after all bmap_createblack(DEBUG) calls in the log above and after MEMORY NOW). I had added a GetAvailableTextureMem() call after my previous post. It returned (GetAvailableTextureMem: 4278190080) before the D3D messages. What can I do or try next? EDIT: The ninja edit! -> However, I've meanwhile looked through some developer forums and it seems that E_OUTOFMEMORY can also happen when in fact memory is still available. [...] After closing all of the E_OUTOFMEMORY errors it seems like I can continue playing normally with my render chain still in tact. The bitmap pointers all have received new and non-zero addresses, too.
Last edited by Superku; 04/24/17 16:35.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465403
04/24/17 19:33
04/24/17 19:33
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Uhh... I'm not sure what I'm doing and what those values mean exactly but I've tried a program called VMMap (which I found in some non-Gamestudio thread looking for E_OUTOFMEMORY) and looked at my game's (process's) address space ~ I guess. Committed: This graph shows the committed (memory that represents data or code) memory usage of the process: 1.402GB Working Set: This graph shows the working set usage of the process by memory type. Working set represents the amount of commited virtual memory that's in physical memory and owned by the process: 1.120GB Summary of the virtual and physical usage of the process by type (like Image, Mapped File, Heap, Private Data, ...), "Total": 1.845GB RAM usage according to the task manager: 1.050GB When I now change the resolution (from 1920x1080 to 1600x1050), memory for the render targets is reallocated. I read somewhere that DirectX does not flush (?) memory immediately on "release", instead you need to flush the data manually if you really need to (let's say when you're low on memory). I get the standard E2005. The values are now as follows: 1.489, 1.153, 1.920, 1.079GB. The second to last value looks a little alarming to me (esp. if contiguous or fragmented memory is important in this case, which I assume it is). The acklog part (after all bmap_create instructions are done): MEMORY NOW (after all bmap creations): (nexus: 63.5/ 150.0) (sys_memory: 210.7) (d3d_texfree: 3963.0) (d3d_texlimit: 8192.0) (d3d_texbmaps: 73.7) (d3d_texskins: 4.3) (d3d_texsmaps: 1.7) (d3d_texsurfs: 0.1) (sum: 79.7MB) (GetAvailableTextureMem: 4280287232) Error E2005: Can't create DirectX texture unnamed D3D_TexCreate: 8007000e - E_OUTOFMEMORY Is that one value, the "summary/ total" value the one that matters? And the RAM usage in the task manager "totally irrelevant"? To make sure this is not project related (or because of a damaged file):
///////////////////////////////
#include <acknex.h>
#include <default.c>
///////////////////////////////
#define MAX_BMAPS 196 // start with the default nexus/ "run current script"
BMAP* test_bmps[MAX_BMAPS];
#define EXTRA_BMAPS 1
BMAP* extra_bmps[EXTRA_BMAPS];
#define MAX_VIEWS 64
MATERIAL* mat_chain[MAX_VIEWS];
VIEW* view_chain[MAX_VIEWS];
void createExtraBmaps()
{
int i;
for(i = 0; i < EXTRA_BMAPS; i++) extra_bmps[i] = bmap_create("test8MB.tga");
}
void switchResolution()
{
video_mode = 10+(video_mode == 10);
video_switch(video_mode,0,2);
}
void main()
{
int i;
fps_max = 60;
video_mode = 6;
level_load(NULL);
me = ent_create(CUBE_MDL,vector(128,0,0),NULL);
for(i = 0; i < MAX_BMAPS; i++) test_bmps[i] = bmap_create("test8MB.tga");
wait(1);
VIEW* view_prev = camera;
for(i = 0; i < MAX_VIEWS; i++)
{
//mat_chain[i] = mtl_create();
//effect_load(mat_chain[i],"blur_h_test.fx");
view_chain[i] = view_create(0);
set(view_chain[i],PROCESS_TARGET);
//view_chain[i].material = mat_chain[i];
//if(i < MAX_VIEWS-1) view_chain[i].bmap = bmap_createblack(screen_size.x,screen_size.x,24);
view_prev.stage = view_chain[i];
view_prev = view_chain[i];
}
on_1 = createExtraBmaps;
on_space = switchResolution;
while(1)
{
my.pan += 6*time_step;
draw_text(str_printf(NULL,"sys_memory: %.1f",(double)sys_memory),20,20,COLOR_RED);
draw_text(str_printf(NULL,"d3d_texbmaps: %.1f",(double)d3d_texbmaps),20,40,COLOR_RED);
wait(1);
}
}
There I create a ton of 2048x1024 (8+MB) textures to get the RAM usage above 1.6GB and the "summary/ total" value > 1.9GB. When I now press [Space] twice, E2005 happens. (Which seems somewhat reasonable to me that it errors here, except the difference of 600MB -task manager- compared to my real project.) Btw. if you just add additional bitmaps (with the createExtraBmaps()) function you get a (probably legit) Error E1005: Out of memory.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465405
04/25/17 10:00
04/25/17 10:00
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
I get no error with your script regardless of how often I press space Hmmm just tried it again and no error at first. However, I increased MAX_BMAPS gradually until I got an "Out of memory" error at start, then reduced it again (to 202) so it only gave me the DirectX error when pressing space. (Oh, it's a 32bit texture btw, 8.193MB.) Maybe a certain render target size or format is not supported by your hardware. Well those usually all work, no matter what I set the target to, except when there's a notable amount of stuff loaded into memory. For finding out, print GetAvailableTextureMem(). d3d_texfree works also, but is only updated once per frame. If it's really a video memory issue, GetAvailableTextureMem() should return a low value immediately before the error. If not, it's something different. Tried doing that as you can see in my previous post. The lowest value I ever got was still greater than 4.1GB. EDIT: Tried printing before and after manual bmap creation (not applicable in this test code), at the end of a frame and on_d3d_lost, on_d3d_reset).
Last edited by Superku; 04/25/17 10:01.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465412
04/25/17 23:55
04/25/17 23:55
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Well I have 8GB of VRAM, not sure how that would work then. Therefore I've now grabbed my laptop with an Intel HD Graphics 5500, 3839MB shared system memory ("Gemeinsam genutzter Systemspeicher") - hopefully that doesn't make a difference in this case. Same issue but with MAX_BMAPS = 212. Surprisingly, GetAvailableTextureMem() shows similar values here as well, and always > 4.0GB. "Before the crash" I put the call in the main function again, the switchResolution function, on_d3d_lost, on_d3d_reset, always > 4.0GB with only minor differences. I must be doing something wrong. Tbh I'm majorly confused right now what I'm looking at or checking. Just for clarification: UINT GetAvailableTextureMem()The function returns an estimate of the available texture memory. Remarks: The returned value is rounded to the nearest MB. This is not true, right? Or is it? I assumed this was in bytes, as I get values like 4278190080 (which I just interpreted as 4,278,190,080 ~ 4.278GB). This value is awfully close to 4294967294, the range of uint (took the int range -2147483648 to 2147483647 from the manual, times 2). However, there's no unsigned int in lite-C - it's 4 bytes either way, so I guess %u interprets the bytes correctly. So... back to the example/ test code: I moved the bitmap creation into the main function like this:
if(num_bmaps < MAX_BMAPS)
{
test_bmps[num_bmaps] = bmap_create("test8MB.tga");
num_bmaps++;
}
sys_memory slowy builds up with each bitmap creation up until ~1641MB. However, d3d_texfree remains the same, that is the initial Video memory found: 4094 MB (3838MB on my laptop) minus 2MB (the nexus/ d3d_texbmaps). It just stays at 4092MB. GetAvailableTextureMem() always returns the same value, too (laptop and desktop). It only ever changes slightly when I change resolutions. Just removed the render chain and this is what I got: D3D_DrawPrimitive: D3DERR_INVALIDCALL at 0.393 switchResolution() at frame 149: (GetAvailableTextureMem: 4290772992) D3D_Resize Window: 1440x900 on_d3d_lost_event() at frame 149: (GetAvailableTextureMem: 4291821568) on_d3d_reset_event() at frame 149: (GetAvailableTextureMem: 4284481536) -> Window: 1x1440x900x32 D3D_DrawPrimitive: D3DERR_INVALIDCALL switchResolution() at frame 175: (GetAvailableTextureMem: 4278190080) D3D_Resize Window: 1680x1050 on_d3d_lost_event() at frame 175: (GetAvailableTextureMem: 4279238656) on_d3d_reset_event() at frame 175: (GetAvailableTextureMem: 4280287232) -> Window: 1x1680x1050x32 D3D_DrawPrimitive: D3DERR_INVALIDCALL Malfunction W1246: DirectX driver failure: D3DERR_INVALIDCALL Error E1005: Out of memory This may be something else though, I get the first warning immediately (after "1st frame with 4094 MB").
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#465421
04/26/17 13:13
04/26/17 13:13
|
Joined: Jul 2000
Posts: 27,978 Frankfurt
jcl
Chief Engineer
|
Chief Engineer
Joined: Jul 2000
Posts: 27,978
Frankfurt
|
Hmm. If GetAvailableTextureMem really returned a wrong value with your 3D card, then the error message could be correct and you're in fact exceeding the memory. What 3D card are you using? Just to be sure, can you run a simple test like this?
void main()
{
fps_max = 60;
video_mode = 6;
int i;
for(i = 0; i < 10000; i++) {
wait(1);
BMAP *Bmap = bmap_createblack(1048,1024,32);
draw_quad(Bmap,vector(10,10,10),NULL,NULL,NULL,NULL,50,0);
draw_text(str_printf(NULL,"d3d_texfree: %d MB",d3d_texfree),20,60,COLOR_RED);
}
}
At some point this will exceed either the available video memory or the available system memory, and the texture creation will fail. Does d3d_texfree then really display still your full 4 GB?
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465424
04/26/17 15:34
04/26/17 15:34
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Ran the test, draw_quad makes the difference here - in your or my example code. With draw_quad, d3d_texbmaps increases and d3d_texfree gradually decreases! 1) Desktop (GTX 1080): E2005 appears after 201 bitmaps ~ d3d_texbmaps 1608MB, RAM usage of 1655MB and a remaining free memory of 2492MB = d3d_texfree. sys_memory does not increase.
2) Laptop (HD 5500): E2005 after 213 bitmaps, d3d_texfree 2141MB.
3) Old Laptop, < 4GB RAM: A) Integrated graphics, HD 2200 or something like that, only ~1640MB shared memory: The test virtually kills this laptop, memory allocations take forever when d3d_texfree gets low. E2005 appears at d3d_texfree = 5MB (or less because of double buffering?). Memory usage in the task manager crawls up to > 1.5GB, then, when d3d_texfree gets lower, it jumps to 353-416MB in the task manager.(?) B) Dedicated graphics card, Geforce 310M (not quite sure how much VRAM there is): E2005 after 210 bitmaps, d3d_texfree 990MB. Here the RAM usage in the task manager looks normal/ as expected, probably because of dedicated memory/ VRAM.
My code example (tested on computer 2): - Without draw_quad, sys_memory increases with each bitmap creation, d3d_texbmaps does not and d3d_texfree does not decrease. This is probably working as intended, right? Memory is copied over/ sent to the graphics card/ VRAM when it's needed..? I can create 212 bitmaps. - With draw_quad(test_bmps[num_bmaps],...) (see previous post) sys_memory increases as well as d3d_texbmaps. E2005 after 105 bitmaps at 889MB sys_memory, 824MB d3d_texbmaps. RAM sits at 1722MB though. -> If I replace bmap_create("test8MB.tga") with bmap_createblack(2048,1024,32), I can create 212 bitmaps again. sys_memory does not increase, d3d_texbmaps does and d3d_texfree decreases.
Btw. texture creation should already fail at > 2GB, right? Because of reasons mentioned earlier in this thread. Oh, and yesterday I read that GetAvailableTextureMem() is unreliable and includes some system/ shared memory as well.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465435
04/27/17 12:43
04/27/17 12:43
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
So you must have enough of both. But I do have enough of both, no? Can you please comment on the following: - With draw_quad(test_bmps[num_bmaps],...) (see previous post) sys_memory increases as well as d3d_texbmaps. E2005 after 105 bitmaps at 889MB sys_memory, 824MB d3d_texbmaps. RAM sits at 1722MB though. -> If I replace bmap_create("test8MB.tga") with bmap_createblack(2048,1024,32), I can create 212 bitmaps again. sys_memory does not increase, d3d_texbmaps does and d3d_texfree decreases. Does that make sense? If so, why?
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465444
04/27/17 17:06
04/27/17 17:06
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Yes, that's fine since bmap_createblack, as long as nothing else happens, just creates an empty bmap and has no impact on sys_memory or d3d_texbmaps. But d3d_texbmaps does increase as I'm using draw_quad...
///////////////////////////////
#include <acknex.h>
#include <default.c>
///////////////////////////////
#define MAX_BMAPS 196 // start with the default nexus/ "run current script"
BMAP* test_bmps[MAX_BMAPS];
int num_bmaps = 0;
void main()
{
int i;
fps_max = 60;
video_mode = 6;
while(1)
{
draw_text(str_printf(NULL,"sys_memory: %.1f",(double)sys_memory),20,20,COLOR_RED);
draw_text(str_printf(NULL,"d3d_texbmaps: %.1f",(double)d3d_texbmaps),20,40,COLOR_RED);
draw_text(str_printf(NULL,"nexus: %.1f/ %d",(double)nexus,(int)max_nexus),20,60,COLOR_RED);
draw_text(str_printf(NULL,"d3d_texfree: %.1f",(double)d3d_texfree),20,80,COLOR_RED);
if(num_bmaps < MAX_BMAPS)
{
test_bmps[num_bmaps] = bmap_createblack(2048,1024,32); //bmap_create("test8MB.tga"); //
draw_quad(test_bmps[num_bmaps],vector(10,10,10),NULL,NULL,NULL,NULL,50,0);
num_bmaps++;
}
draw_text(str_printf(NULL,"num_bmaps: %d",(int)num_bmaps),20,120,COLOR_RED);
wait(1);
}
}
That code creates 196 bitmaps, 8MB per image, and consumes about 1.6GB of memory. EDIT: d3d_texbmaps at 1552MB, no error. If I replace bmap_createblack(2048,1024,32) with bmap_create("test8MB.tga"); // 2048x1024x32 the application errors E2005 after ~100 bitmaps, ~1.6GB of memory (task manager), with sys_memory at 849MB and d3d_texbmaps at 776MB. After closing that message it prints "Out of Memory" (E1005). This cannot be correct, right?
Last edited by Superku; 04/27/17 17:07.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465448
04/27/17 18:27
04/27/17 18:27
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
So it is intended that version A)
test_bmps[num_bmaps] = bmap_create("test8MB.tga"); draw_quad(test_bmps[num_bmaps],...);
uses twice the memory (read: 16MB per 8MB (source) image) compared to version B)
BMAP* source_bmp = "test8MB.tga"; // global bitmap ... test_bmps[num_bmaps] = bmap_createblack(2048,1024,32); bmap_blit(test_bmps[num_bmaps],source_bmp, NULL, NULL); draw_quad(test_bmps[num_bmaps],...);
(which both display the same content to the screen)?
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465471
04/29/17 14:43
04/29/17 14:43
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Please download this code plus bitmap then and give it a try: http://superku.de/crash_test5.zipThe problem: the application errors E2005 after ~100 bitmaps, ~1.6GB of memory (task manager), with sys_memory at 849MB and d3d_texbmaps at 776MB. After closing that message it prints "Out of Memory" (E1005). Changing
if(0) // < ---------- here
to true results in sys_memory not rising and me being able to create twice as many bitmaps. EDIT: If I'm not mistaken I only then have ~800MB of memory to work with to make PC games with A8.
Last edited by Superku; 04/29/17 14:44.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#465475
04/29/17 23:09
04/29/17 23:09
|
Joined: Aug 2003
Posts: 118 Deutschland
Ezzett
Member
|
Member
Joined: Aug 2003
Posts: 118
Deutschland
|
I tested it also with the engine sdk in Visual Studio Community 2015. Maybe this is helpfull, because it's possible to look inside the structs.
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <stdlib.h>
#include <malloc.h>
#include <memory.h>
#include <tchar.h>
#include "var.h"
#include "adll.h"
ENGINE_VARS *ev;
COLOR COLOR_RED = { _VAR(255), _VAR(0), _VAR(0) };
COLOR COLOR_WHITE = { _VAR(255), _VAR(255), _VAR(255) };
#define MAX_BMAPS 196 // start with the default nexus/ "run current script"
BMAP* test_bmps[MAX_BMAPS];
int num_bmaps = 0;
int APIENTRY WinMain(HINSTANCE hInstance,
HINSTANCE hPrevInstance,
LPSTR lpCmdLine,
int nCmdShow)
{
ev = engine_open(NULL);
int i;
v(fps_max) = 60;
v(video_mode) = 6;
engine_frame();
static BMAP* source_bmp = bmap_create("test8MB3.tga");
while (engine_frame()) {
if (num_bmaps < MAX_BMAPS)
{
if (1) // < ---------- here
{
test_bmps[num_bmaps] = bmap_createblack(_VAR(2048), _VAR(1024), _VAR(32));
bmap_blit(test_bmps[num_bmaps], source_bmp, NULL, NULL);
}
else test_bmps[num_bmaps] = bmap_create((char *)"test8MB3.tga");
draw_quad(test_bmps[num_bmaps], vector(_VAR(10), _VAR(10), _VAR(10)), NULL, NULL, NULL, NULL, _VAR(100), _VAR(0));
//cprintf4("n%d) w %d, h %d, b %d",(int)(num_bmaps+1),(int)test_bmps[num_bmaps]->finalwidth,(int)test_bmps[num_bmaps]->finalheight,(int)test_bmps[num_bmaps]->finalbytespp);
num_bmaps++;
}
draw_text(_CHR(str_printf(NULL, "sys_memory: %.1f", (double)v(sys_memory))), _VAR(20), _VAR(20), &COLOR_RED);
draw_text(_CHR(str_printf(NULL, "d3d_texbmaps: %.1f", (double)v(d3d_texbmaps))), _VAR(20), _VAR(40), &COLOR_RED);
draw_text(_CHR(str_printf(NULL, "nexus: %.1f/ %d", (double)v(nexus), (int)v(max_nexus))), _VAR(20), _VAR(60), &COLOR_WHITE);
draw_text(_CHR(str_printf(NULL, "d3d_texfree: %.1f", (double)v(d3d_texfree))), _VAR(20), _VAR(80), &COLOR_WHITE);
draw_text(_CHR(str_printf(NULL, "num_bmaps: %d", (int)num_bmaps)), _VAR(20), _VAR(120), &COLOR_WHITE);
}
engine_close();
return 0;
}
For me it has the exact same result like running the script with SED. I don't know if this is relevant (I really doubt it has something to do with the memory problems), but when I use if(1), the u1,v1- and u2,v2-coordinates of the bitmaps are not zero but slightly off:
u1 0.000244140625 float
v1 0.000488281250 float
u2 0.999755859 float
v2 0.999511719 float
This is even true for the source_bmp which is declared before the if branch. I have no idea what is going on here. Here is something more interesting. When I'm using if(0) and I look inside the bmap members I can see that the pixel member is filled:
pixels 0x0bd3d020 "T^nÿWaqÿYctÿZduÿZeuÿ\gwÿ]hxÿ]hxÿ\fvÿYduÿXbsÿYctÿYctÿXbsÿWbrÿWbrÿXbrÿWaqÿV`pÿU_nÿT^mÿT^mÿS]kÿR\jÿS^kÿT^lÿS]kÿQ[iÿPYgÿOXfÿOXfÿPYgÿOYfÿNWeÿOXeÿOXeÿOXfÿQZhÿQ[iÿQ[iÿQZhÿOXeÿNWdÿNWdÿNWdÿNWdÿOXfÿOXfÿPYgÿPYgÿ... 84 'T' unsigned char *
But when I'm changing the branch to if(1) the pixel member of the bitmap created by bmap_createblack can't be read:
pixels 0x00000000 <NULL> <Unable to read memory> unsigned char *
This is also true, after bmap_blit is used. Could it be, that the pixel member of a bitmap takes up 8 MB of memory? If bmap_create is used, the pixel member is filled. But when bmap_createblack or bmap_blit is used, the pixel member is NULL. Maybe this is the cause the first function can only create half as many bitmaps until the memory is full. (1.6 GB equals 200 * 8 MB or 100 * 16 MB)
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#465492
04/30/17 17:30
04/30/17 17:30
|
Joined: Aug 2003
Posts: 118 Deutschland
Ezzett
Member
|
Member
Joined: Aug 2003
Posts: 118
Deutschland
|
I've searched a little bit on maximum 32 bit heap sizes per application and it seems the 1.6GB heap size limitation is caused by Windows. For example, here is a small explanation by Oracle: Why can't I get a larger heap with the 32-bit JVM?
The maximum theoretical heap limit for the 32-bit JVM is 4G. Due to various additional constraints such as available swap, kernel address space usage, memory fragmentation, and VM overhead, in practice the limit can be much lower. On most modern 32-bit Windows systems the maximum heap size will range from 1.4G to 1.6G. On 32-bit Solaris kernels the address space is limited to 2G. On 64-bit operating systems running the 32-bit VM, the max heap size can be higher, approaching 4G on many Solaris systems.
As of Java SE 6, the Windows /3GB boot.ini feature is not supported.
If your application requires a very large heap you should use a 64-bit VM on a version of the operating system that supports 64-bit applications. See Java SE Supported System Configurations for details. http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#gc_heap_32bit2GB should be possible if Gamestudio is ported to Solaris. I would suggest not to come even close to 1.6GB. On some Windows systems this might work while on others 1.4GB could be to much.
Last edited by Ezzett; 04/30/17 17:40.
|
|
|
Re: video memory and d3d_texfree
[Re: Ezzett]
#465506
05/01/17 10:51
05/01/17 10:51
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Ty Ezzett that was informative . So far for me reducing the size of normal maps and setting d3d_texdepth helped the most (while keeping quality). Though lowering d3d_texdepth probably doesn't make much of a difference if one already uses .dds textures right? (/since the .dds is often already compressed?) @jcl, thanks for the info.
Last edited by Reconnoiter; 05/01/17 10:52.
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#465510
05/01/17 13:32
05/01/17 13:32
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Both methods use video memory, but only bmap_create allocates also system memory. Yes, but I'm wondering what bmap_create is using that (pixels) memory for? Does it really need to be there or can that memory allocation be omitted (in the next update, optionally with a new addition to the compatibility variable)? @Ezzett: I see, thanks!
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
compiler and memory usage
[Re: Superku]
#466330
06/09/17 10:21
06/09/17 10:21
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Well, the trouble continues: Since yesterday I have not been able to play test my game. The first launch already took a while if I'm not mistaken, then I got an "Error E1241: Can't open DirectX device" (three D3D_Init Window: 640x352 failed messages before that in the acklog). Since then my game has taken twice as long to compile as it did before. Old values: 15.Xs for the first compile, then 12.Xs for subsequent compiles. Now it takes 29.Xs, until the game starts with an OUT OF MEMORY DirectX error. Just once it worked correctly with the old compiling time after a reboot, otherwise always the same result. I've tried several old builds from my external hard drive, including one from over a year ago, surprisingly the same result. Reinstalled everything, no Windows update since May, thought it could be the DirectX loader that's making trouble but no change. (1) Do you have any idea what could be causing this behavior? When I put
#define PRAGMA_PRINT "n>superku.c END<";
at the end of my main file (finally found this helpful define in the manual) I can see that the "..." progress almost comes to a halt after it's done compiling my code. Most dots now take about 3-5times as long, the first dot after my last PRAGMA_PRINT/ last line in my code takes about 10 times as long as the previous average. I'm not sure what the compiler is doing right after it's done with my source code as it takes a while until it reaches the first global dummy TEXT or PANEL object (where I put in invalid code). (2) Can you please share some insight on this process? Publishing and compiling the exe there takes as long as it does now too, however, the published version seems to work fine. This is not quite feasible for development though. VMMap returns a Heap size of 380MB for my game running in the published version and a level loaded and running.When I put invalid code after my last PRAGMA_PRINT and stop the compile process that way, VMMap returns a Heap size of about 1245MB (EDIT: 961MB on my laptop?). After that it crawls up to about 1550MB, starting the game and then resulting in that OUT OF MEMORY error on level_load (which seems reasonable because of the 2GB/ ~1.6GB heap limit - the previous 380MB seem to coincide as well). (3) I am aware that the development build is different to the published build but does it seem reasonable that the compiling of less than 3MB in code result in that huge heap size in the development version? Thanks for your time. EDIT2: Although it probably does not matter in this case, the exact error is "DirectX driver failure: D3DERR_OUTOFVIDEOMEMORY".
Last edited by Superku; 06/09/17 15:24.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: compiler and memory usage
[Re: jcl]
#466437
06/14/17 08:35
06/14/17 08:35
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Strangely, the problem did not happen for the last 3 days. I cannot explain that or even guess why.
However, my concern is the following: Right now compiling my source code alone uses 980MB of (heap) memory (and 1245MB+ as something was wrong a couple of days ago), using "Run current script" from SED with the default nexus (40MB?).
Assuming my code base does not grow (which it does continuously though), this leaves me with a very finite pool of memory of ~500MB for my whole game, including sound/ music, render targets and textures (which even gets subjectively less because the BMAP struct may occupy twice the memory for one image as outlined earlier in this thread) - at least in the development version.
This is only getting worse the bigger the project becomes, from a code perspective. If I'm not mistaken that's a considerable restriction on all acknex products, not just mine. The development version needs extra memory for memory allocation safety and debugging features and similar stuff, right? Can this memory footprint be reduced though, even if that would mean cutting a few safety measures?
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: compiler and memory usage
[Re: jcl]
#466453
06/14/17 20:40
06/14/17 20:40
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
I'm just using the default nexus and "run current script". I've created a small test example thingy that demonstrates the problem: http://superku.de/codeCompileTest.zipVMMap if you do not have it already: https://technet.microsoft.com/en-us/sysinternals/vmmap.aspxThe code_creator.c creates a few files with copy pasted dummy functions and different names. That has been done already though, just run main.c. The compiler will stop after it has processed all the code and reached "asasda" after the main function. When you select that acknex process with VMMap, it will show a heap value of ~1041MB (unless my system is bugging out right now again, which I don't think it is). No images or global objects, except stuff from default.c (and acknex.h). Remove "asasda" and run current script again. The A8 window appears. VMMap on that process returns a heap size of ~~1223MB. If you now press [Space], bmap_createblack(2048,2048,32)+draw_quad is executed each frame. The application throws the known DirectX error after about (on my machine) 22 bitmaps (which makes sense, 22*2*8MB = 352MB, default critical heap value). ------------------------------ Btw. I've copy pasted some more (of the same) dummy code into dummyFunction.txt, ran code_creator.c again and main.c after. Got no code size warning but a Microsoft Vusual C++ Runtime Library Runtime Error! Program: ... acknex.exe This application has requested the Runtime to terminate it in an unusual way. VMMap: Total 2028MB, Heap 1864MB. This is not working as intended, right? Thanks for your time.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: compiler and memory usage
[Re: HenWoll]
#466503
06/18/17 21:33
06/18/17 21:33
|
Joined: Dec 2011
Posts: 1,823 Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Now, however i can only Maximum change to 2 level,s after this i become the Error E1513, In the most diverse varieties!
The levels individually can be run and error-free, only that my video memory fills quickly after Level Load .
Someone has a suggestion where I should start? , perhaps it some texture(s) that is not freed by level_load (bmap_create and/or extra panels?). If yes, try bmap_purge / ptr_remove them. I think I can keep loading quite big (in terms of memory) levels but I mostly use entities so they get automatically purged. My panels are currently mostly placeholder tiles and I dont think I really use bmap_create except for some initialization of some icons. I quess its something bmap / bmap_create related. Perhaps also try debugging with sys_marker incase you already haven't. Also d3d_texdepth = 16; helps (but not for memory leaks ofcourse).
Last edited by Reconnoiter; 06/18/17 21:35.
|
|
|
Re: compiler and memory usage
[Re: Reconnoiter]
#466519
06/19/17 09:33
06/19/17 09:33
|
Joined: Jul 2000
Posts: 27,978 Frankfurt
jcl
Chief Engineer
|
Chief Engineer
Joined: Jul 2000
Posts: 27,978
Frankfurt
|
Ok, I've checked your example. The compiled code size is not 980 MB, but about 10 MB. This is indeed a problem, since the compiler first compiles the script to an interim code and then to the final machine code. The interim code can be much larger than the machine code, up to factor 30. The compiler also allocates all sorts of additional data structures since it's made for speed, not for memory preservation. All this inflates the memory allocation at compile time to a large multiple of the original code size.
When the script is then run afterwards, some of the memory is freed, but a large part stays allocated for the purpose of debugging and accessing variables. Only when the script is compiled to a published exe and then executed, that memory is not allocated.
There's certainly room for optimization since I think 90% of the allocated memory by the compiler is not needed at runtime. But that's how it works at the moment.
|
|
|
Re: compiler and memory usage
[Re: jcl]
#466566
06/21/17 13:43
06/21/17 13:43
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Thanks for looking into it. a large part stays allocated for the purpose of debugging and accessing variables Is that for the "Debug run" in SED (only)? If so, could that be optionally skipped/ excluded from the acknex build for "Run current script"/ "Test run"? I actually never use the "Debug run" as it for some reason never worked for me. When I see its features I should probably have dug more into it. If that (quote) includes stuff like sys_marker, sys_malloc and so on it has definitely got to stay though. There's certainly room for optimization since I think 90% of the allocated memory by the compiler is not needed at runtime. But that's how it works at the moment. Ok, glad that this memory mystery is finally solved! Would be nice if this could be improved/ done some time in the future as it's such a big issue for non-published test runs.
Last edited by Superku; 06/21/17 13:45.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#469261
11/09/17 21:38
11/09/17 21:38
|
Joined: Sep 2005
Posts: 352 Germany
preacherX
Senior Member
|
Senior Member
Joined: Sep 2005
Posts: 352
Germany
|
d3d_texbmaps is indeed not reduced by releasing the old textures when switching the resolution. I have put this on the bug list. But you should still be able to find the problem this way when you don't change the resolution. I'm getting many reports on Steam from people who have problems when switching between programs and then returning to the game that they get "Error E2005: Can't create DirectX texture unnamed" errors and the game stops. I'm not sure if I understand the discussion here correctly, but is this problem the d3d_texbmaps bug? When will this be fixed? I also don't find this on the bug list? Is there a workaround?
Last edited by preacherX; 11/09/17 21:59.
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#470735
02/04/18 12:38
02/04/18 12:38
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Ok, I've checked your example. The compiled code size is not 980 MB, but about 10 MB. This is indeed a problem, since the compiler first compiles the script to an interim code and then to the final machine code. The interim code can be much larger than the machine code, up to factor 30. [...] There's certainly room for optimization since I think 90% of the allocated memory by the compiler is not needed at runtime. But that's how it works at the moment. [...] It had to be explicitely implemented. Although I've optimized it already over the months and years (not loading any big files like music, creating any big textures only on demand (of which there are < 10) and deleting 99% of all unused code), I've finally reached the point where I cannot play-test my game anymore. The code is the issue, not other files. Most of the code is already in place but a little more is yet to come. Mr. Lotter, can you free some of the memory (not 90% but let's say 30%, optimistically, some big chunks) and release a new version? That would be a huge thing/ relief, probably not just for me. How much time would that take you to do/ implement? I'd throw some money in the company's coffee tip jar if it helped thinking.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: jcl]
#470803
02/07/18 10:54
02/07/18 10:54
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Thanks for the response. I went through the code again after my last post, optimizing it and commenting/ excluding temporarily unused code (for levels I'm not working on currently). This saved me about 45MB (compile time 14.9->14.1) which means I could test the game again for now but that's obviously going to change/ break in the near future. Putting code into a DLL is a neat idea! Too bad I wrote this project's code very acknex-y but I will do that if I have no other choice. The license allows us to give the lite-C source not to the public, but to select developers. If someone with enough C++ knowledge volunteers, I'm happy to give him a contract and the source code. Sounds great! I have sent an e-mail to the support.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: Garrettwademan]
#474877
11/10/18 12:43
11/10/18 12:43
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
Not that I'm aware of, sorry. I've signed said contract and dug through the compiler source code (and looked at test interim code) for a bunch of days. I think I roughly know how it works but actually changing anything in there is something completely else, the memory management code might as well be Chinese. You'd probably need some wise guy and compiler enthusiast like JustSid to have a look at this. *wink wink*
Btw. as a workaround I wrote a small tool in lite-C which checks the alteration dates of code files, compiles a new exe version like this: exec_wait("C:\GStudio8\WED.EXE","-r superku.c -nx 128 -ns"); and then starts the exe. If I only do changes on levels, models or something like that it heavily cuts down on the compile times, otherwise (changing code) it's only slightly slower. I don't have any memory issues then and the game is running faster as well, but... you don't have any debugging features (like sys_marker) or informative error messages then. Not an optimal solution.
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: sitrep]
#474951
11/14/18 17:00
11/14/18 17:00
|
Joined: Sep 2003
Posts: 6,861 Kiel (Germany)
Superku
Senior Expert
|
Senior Expert
Joined: Sep 2003
Posts: 6,861
Kiel (Germany)
|
@Garrettwademan: My "tool" is the following code:
///////////////////////////////
#include <acknex.h>
#include <default.c>
#include <windows.h>
///////////////////////////////
STRING* str_tmp = "";
TEXT* tDir = { pos_y = 40; strings = 200; flags = SHOW;}
void main()
{
fps_max = 60;
video_screen = 0;
video_window(NULL,NULL,0,"Superku Launcher");
var fhandle = file_open_write("launchLog.txt");
file_str_write(fhandle,str_printf(NULL,"Superku Launcher Logn%02d:%02d:%04d - %02d:%02d:%02d...",(int)sys_day,(int)sys_month,(int)sys_year,(int)sys_hours,(int)sys_minutes,(int)sys_seconds));
file_str_write(fhandle,str_printf(NULL,"nCurrent work_dir: %s",_chr(work_dir)));
str_cpy(work_dir,"C:\GStudio8\Superku\");
str_cpy(save_dir,work_dir);
SetCurrentDirectory(_chr(work_dir));
char cbuffer[256];
GetCurrentDirectory(256,cbuffer);
file_str_write(fhandle,str_printf(NULL,"
nGetCurrentDir: %s",cbuffer));
int doCompile = 0;
int numFiles = txt_for_dir(tDir,"*.c");
int i;
for(i = 0; i < numFiles; i++)
{
file_str_write(fhandle,str_printf(NULL,"n
Is file (%s) newer than superku.exe? ",_chr((tDir.pstring)[i])));
if(file_date((tDir.pstring)[i]) > file_date("superku.exe"))
{
file_str_write(fhandle,"Yes!");
doCompile = 1;
break;
}
else file_str_write(fhandle,"No...");
}
if(!doCompile)
{
int numFiles = txt_for_dir(tDir,"*.h");
int i;
for(i = 0; i < numFiles; i++)
{
file_str_write(fhandle,str_printf(NULL,"n
Is file (%s) newer than superku.exe? ",_chr((tDir.pstring)[i])));
if(file_date((tDir.pstring)[i]) > file_date("superku.exe"))
{
file_str_write(fhandle,"Yes!");
doCompile = 1;
break;
}
else file_str_write(fhandle,"No...");
}
}
printf("nndoCompile : %d",doCompile);
file_str_write(fhandle,str_printf(NULL,"n
ndoCompile: %d",doCompile));
if(doCompile)
{
printf("nSet Build options: %s...",_chr("options_skbuild.scr"));
file_cpy("C:\GStudio8\data\options.scr","C:\GStudio8\data\options_skbuild.scr");
printf(" Done.");
printf("nCompile exe and dll...");
exec_wait("C:\GStudio8\WED.EXE","-r superku.c -nx 128 -ns");
printf(" Done.");
wait(1);
if(key_esc) sys_exit(""); // in case of compile errors
file_cpy("C:\GStudio8\Superku\acknex.dll","C:\GStudio8\Superku\superku.cd\acknex.dll");
}
file_close(fhandle);
if(key_esc) sys_exit("");
wait(1);
if(key_esc)
{
sys_exit("");
return;
}
exec("C:\GStudio8\Superku\superku.exe","-nx 128 -ns -diag");
sys_exit("");
}
I have no clue about C-Script, forgot everything about it, but it should work still. Go to the "data" subfolder in your Gamestudio installation folder, duplicate (and backup) the options.scr file and create one for your script compilation. I've pretty much excluded all files there (like images, MDL, WMB) so it only compiles the code. I think there's a command line option for the publish process to do that normally but it did not work for me for some reason, which I forgot. Btw. have you checked the amount of memory your game uses when it crashes using VMMap?
"Falls das Resultat nicht einfach nur dermassen gut aussieht, sollten Sie nochmal von vorn anfangen..." - Manual Check out my new game: Pogostuck: Rage With Your Friends
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#474986
11/15/18 23:58
11/15/18 23:58
|
Joined: Jan 2006
Posts: 245 PA
Garrettwademan
Member
|
Member
Joined: Jan 2006
Posts: 245
PA
|
Thanks for sharing, will review and try it! I just found the command d3d_texlimit and was about to do a cartwheel after reading the description, but when I actually implemented it, the game crashes every time for certain models on several different machines.
I have used VMmap, so I clocked 1.984Gb RAM on my Desktop machine, and about 1.8Gb something on my laptop. My Desktop DOES NOT CRASH or throw any texture memory errors (it runs perfectly), but my laptop always has the issue. I cleaned up my code so I am seeing about 1.7Gb but my laptop still says NOPE and craps itself! Now I use built in checks for vid_memory and nexus, and I am only using 78/600Mb nexus, and about 250Mb of video memory. Its like these are not logging properly or related to the bug. At game launch it says it found 750Mb of video memory.
Now I did some testing, the same texture file that kept crashing on my laptop didn't crash when I didn't load the world models (walls, floors, tables, etc) with the culprit model, and it only crashes when I look at the item in the game. There may be things I can potentially do to work around this (but not coming to me right now).
All in all Windows OS only really supports up to 2GB of RAM per software instance unless you do some crazy stuff.
BTW Superku, your game on Steam looks awesome. The specialFX and detail is amazing! 7 years too. I am a little worried about posting my game on steam even though I paid the fee and working on the page layout. I wanted to finish this last detail before going live, but worried this issue may severely delay my project now. I have been dragging playing other games now, thinking, researching, and performing little tests here and there. I don't want to be bummed because I have been able to get around and even abuse the bugs from A8 to my advantage lol, but this one is scary. This is my first 1 million+ lines of code project, so I plan on grinding until its done! Thank you for your input and experience too! It has been very helpful to me over the years.
Last edited by Garrettwademan; 11/16/18 00:02.
|
|
|
Re: video memory and d3d_texfree
[Re: Garrettwademan]
#474991
11/16/18 10:01
11/16/18 10:01
|
Joined: Aug 2003
Posts: 118 Deutschland
Ezzett
Member
|
Member
Joined: Aug 2003
Posts: 118
Deutschland
|
I have used VMmap, so I clocked 1.984Gb RAM on my Desktop machine, and about 1.8Gb something on my laptop. . My Desktop DOES NOT CRASH or throw any texture memory errors (it runs perfectly), but my laptop always has the issue. Depending on the system even 1.4Gb could be too much for a 32 Bit application. You should stay way below that. I wouldn't risk to make a Gamestudio game that needs more than 1 Gb at a time. Today's HD textures and high poly models fill up RAM very quickly. Because Gamestudio is a 32 Bit software you would need to work around that limit by a clever resource management in the game. Now I use built in checks for vid_memory and nexus, and I am only using 78/600Mb nexus, and about 250Mb of video memory. If I remember correctly some things like textures are saved twice in memory to recreate them if you ALT+TAB out and back into the game. This is not shown correctly when you use these helper variables. You basically need to multiply some of the allocated memory by two.
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#475141
11/25/18 00:28
11/25/18 00:28
|
Joined: Apr 2002
Posts: 1,246 ny
jumpman
Serious User
|
Serious User
Joined: Apr 2002
Posts: 1,246
ny
|
im getting out of memory and d3d unamed errors as well its been happening when recreating render chains, but also when I add in objects with huge textures. However I did not optimize my levels nor the texures, but its a little concerning nonetheless. what Im worried about is optimizign all these textures and models and levels and still hit that out of error when creating new objects in realtime.
Last edited by jumpman; 11/25/18 00:29.
|
|
|
Re: video memory and d3d_texfree
[Re: Superku]
#475147
11/25/18 21:02
11/25/18 21:02
|
Joined: Apr 2002
Posts: 1,246 ny
jumpman
Serious User
|
Serious User
Joined: Apr 2002
Posts: 1,246
ny
|
Thanks for taking a look superku. Yes, this is my test scene. My nexus is 500 I put "fart" at the bottom of my script lol, saved, ran the engine, it caught the error, then i ran VMMap, and here is what I see: 901,468 k total 742,624 k heap What is the difference between total and heap? Does my screenshot/stats look ok?
Last edited by jumpman; 11/25/18 21:02.
|
|
|
|