Engine crash after level_load
Posted By: pegamode
Engine crash after level_load - 02/01/16 08:20
Hi JCL,
could you please take a look at this thread:
http://www.opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=457609#Post457609 Maybe you could give me a hint where to take a deeper look at.
Also: Is there any chance to get a SED debugger that works with large projects? Currently the debugger allocates too much memory for a 32bit application so that the SED just crashes. Maybe you could get rid of the internally used xml structure?
Thanks.
Regards,
Sascha.
Posted By: jcl
Re: Engine crash after level_load - 02/01/16 09:17
Crashes when loading a particular level B from a particular level A are normally either caused by a function that continues to run after level change, but accesses a now-removed object of the old level. Or it's an entity action of the new level.
Put a diag() call at the begin and after the wait() of all entity functions. This way you can see which function does not survive the first wait(). Similarly you can also find out which functions continue to run after level change.
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 09:28
Hi JCL,
thanks for your reply.
I've already removed all entity actions ... no change at all.
I've also logged out all currently running functions right before the level_load and removed them. The crash still occured.
Then I logged out all running functions right after the level_load and before the wait(1) ... the list doesn't differ from the one taken before the level_load.
After that I resaved all models used in both levels and removed the sky texture. Still no change.
Everything hints to a memory issue somewhere, but I wasn't able to find anything yet.
Is there any possibility to get more information about what the engine is currently processing?
Posted By: jcl
Re: Engine crash after level_load - 02/01/16 12:13
Only what is printed in the log file. If it's not a function, next guess would be a damaged entity file - not necessarily a model. Maybe even the WMB file. Resaving is not guaranteed to fix it, but you could replace the entities with dummy models as a next step. When the crash is reproducible, you can find it by excluding.
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 12:24
The problem is that the crash disappears once I change anything in WED:
- I delete a model ... crash is gone
- I add a model ... crash is gone
- I replace a model ... crash is gone
- even if I delete a model and add it again ... crash is gone
Posted By: Reconnoiter
Re: Engine crash after level_load - 02/01/16 14:58
The problem is that the crash disappears once I change anything in WED:
- I delete a model ... crash is gone
- I add a model ... crash is gone
- I replace a model ... crash is gone
- even if I delete a model and add it again ... crash is gone
, what do you mean with your 4th bullet/point; do you mean delete it in WED and create it in your script?
Have you tried to recreate the map through script (as in first load a new empty map than use ent_create etc. to put every used model/sprite/texture in it)?
As a last resort, have you tried just adding a dummy model (bullet/point 2) and does it give signs of memory/random errors?
Posted By: jenGs
Re: Engine crash after level_load - 02/01/16 16:26
Wait, if you delete an object and add it again the crash is gone? I would guess the wmb is't compiled right. Perhaps an unfortunate combination of models. Or an error in the saved wmp file wich got resolved, when you changed the level.
Why do you bite your teeth into this, if you can resolve the problem. I know you want to know what causes the error. But at this point I would delete and readd a model resave the Level and be happy with it.
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 18:30
Bullet point 4 seems to be related to changing the order of the models inside the level.
If I delete a model (not the last one listed in WED), then click on "undo" (the model appears now at the end of the list in WED) and then build the crash is gone. So the crash only appears if the models are in exact this order.
The reason why I'm not happy with this solution is that it just shifts the crash to another point of the game.
About 2 years ago I had exactly the same crash. The only difference was that the first level was a different one (but the second, the one that crashes, was the same). Then I just changed the order of the models inside the WED and was happy that it solved the problem. At least I thought so.
Now I noticed that the crash is still there, only the first level is different.
So editing the level just seems to move the crash, but doesn't fix it.
Posted By: FBL
Re: Engine crash after level_load - 02/01/16 18:56
Did you try a full publish on the project? Does it crash in a different way or somewhere else? Maybe this gives a new idea where to further investigate.
Even if it does not crash - does the log of a published run show a proper engine shutdown or does it just stop logging (meaning it is not shutting down in a proper way)?
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 19:01
One of our beta testers noticed the same crash in the published version some months ago. It just took me some time to reproduce it.
The logfile of the beta tester looks the same as mine. No error message at all.
But I can give it another try.
Posted By: FBL
Re: Engine crash after level_load - 02/01/16 19:08
That's the important part:
Normal exit at 1859.663
Close level,DLL,objects
Free input,funcs,panels,defs,syns,views,strings,vars..ok
Free sounds,bmaps,fonts,hash,defs1,script..ok
Close dx,multimedia,D3D,engine,nexus..ok
A8 Engine - Pro Edition V8.46.3 - Sep 10 2015
(c) Conitec - www.3dgamestudio.com
Close window at 1859.874
It can happen that the game does not crash, but still the log is incomplete. And then there's something wrong, even if you don't notice.
So if you patch your level with the delete and undo trick, what happens to the log?
Posted By: FBL
Re: Engine crash after level_load - 02/01/16 19:22
I remember I had a nasty bug with pssm shadows and wait (1).
The pssm_run function internally uses a while() - wait(1) loop for the effect. By calling the same function with (0) as parameter it will deactivate itself - but ONE FRAME LATER, due to the while() - wait(1) loop.
So deactivating pssm, loading level, setting it again in a row caused to not stop the effect and instead have it running twice. Game got slower and slower after soem iterations and eventually crashed at some point.
A wait(1) after level loading fixed this. wait(1). As usual.
While this shadow thingy is sort of specific, the function construct with the while loop is not. Maybe you have such a trap running somewhere.
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 19:59
[20:53:54:177] INFO [void print_all_running_tasks()]: starting function
[20:53:54:177] INFO [void print_all_running_tasks()]: calculateFreeGPUmemory
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: mouse_key_observer
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: animate_mouse_map
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: garbageObserver
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: fmod_eventUpdater
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: fmod_trackMemoryUsage
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: inv_arrows
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: internalClock
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: scheduleHandler
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: load_new_room
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: mouse_key_observer
[20:53:54:177] INFO [void print_all_running_tasks()]: entity is NULL
[20:53:54:177] INFO [void print_all_running_tasks()]: exiting function
[20:53:54:177] INFO [void load_new_room (STRING* lName, STRING* entPosName)]: loading level: telescope_room.wmb
LevelInit at 765.687
14 entities 0 cameras 0 lights 0 sounds 2 paths
1 lmaps 1 textures 7 meshes 7 submeshes 104 faces...ok
LevelReady at 767.752
[20:53:56:255] INFO [void load_new_room (STRING* lName, STRING* entPosName)]: level was loaded.
This is my logfile (published version of the game ... dev looks the same).
The "level was loaded." message is placed right before the "wait(1);". With reaching the wait the engine just crashes without any error message.
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 20:00
I'll now do the "delete-undo" trick and check the logfile again.
And here it is:
[21:08:07:534] INFO [void print_all_running_tasks()]: starting function
[21:08:07:534] INFO [void print_all_running_tasks()]: calculateFreeGPUmemory
[21:08:07:534] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:534] INFO [void print_all_running_tasks()]: mouse_key_observer
[21:08:07:534] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: animate_mouse_map
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: garbageObserver
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: fmod_eventUpdater
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: fmod_trackMemoryUsage
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: inv_arrows
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: internalClock
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: scheduleHandler
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: load_new_room
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: mouse_key_observer
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: exiting function
[21:08:07:566] INFO [void load_new_room (STRING* lName, STRING* entPosName)]: loading level: telescope_room.wmb
LevelInit at 44.897
15 entities 0 cameras 0 lights 0 sounds 2 paths
1 lmaps 1 textures 7 meshes 7 submeshes 104 faces...ok
LevelReady at 45.528
[21:08:08:206] INFO [void load_new_room (STRING* lName, STRING* entPosName)]: level was loaded.
[21:08:08:456] INFO [action useable_object()]: starting function
[21:08:08:456] INFO [action useable_object()]: start action for: telescope_controls
[21:08:08:456] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:456] INFO [action useable_object()]: exiting function
[21:08:08:456] INFO [action useable_object()]: starting function
[21:08:08:456] INFO [action useable_object()]: start action for: telescope_right_button
[21:08:08:472] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:472] INFO [action useable_object()]: exiting function
[21:08:08:472] INFO [action useable_object()]: starting function
[21:08:08:472] INFO [action useable_object()]: start action for: telescope_left_button
[21:08:08:472] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:472] INFO [action useable_object()]: exiting function
[21:08:08:472] INFO [action useable_object()]: starting function
[21:08:08:472] INFO [action useable_object()]: start action for: telescope_coinslot
[21:08:08:472] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:472] INFO [action useable_object()]: exiting function
[21:08:08:487] INFO [action useable_object()]: starting function
[21:08:08:487] INFO [action useable_object()]: start action for: telescope_instructions
[21:08:08:487] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:487] INFO [action useable_object()]: exiting function
[21:08:08:487] INFO [action useable_object()]: starting function
[21:08:08:487] INFO [action useable_object()]: start action for: door_audry_telescope
...
But I just saw something suspicious in one code block ... I will dive deeper into that now.
Posted By: pegamode
Re: Engine crash after level_load - 02/01/16 20:22
Hmm ... I noticed that though I removed all actions in WED I added one by script. And there was a bug in this action that I just fixed.
Unfortunately it was not the one that leads into the crash :-(
Posted By: Wjbender
Re: Engine crash after level_load - 02/01/16 21:32
I'll now do the "delete-undo" trick and check the logfile again.
And here it is:
[21:08:07:534] INFO [void print_all_running_tasks()]: starting function
[21:08:07:534] INFO [void print_all_running_tasks()]: calculateFreeGPUmemory
[21:08:07:534] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:534] INFO [void print_all_running_tasks()]: mouse_key_observer
[21:08:07:534] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: animate_mouse_map
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: garbageObserver
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: fmod_eventUpdater
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: fmod_trackMemoryUsage
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: inv_arrows
[21:08:07:550] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:550] INFO [void print_all_running_tasks()]: internalClock
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: scheduleHandler
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: load_new_room
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: mouse_key_observer
[21:08:07:566] INFO [void print_all_running_tasks()]: entity is NULL
[21:08:07:566] INFO [void print_all_running_tasks()]: exiting function
[21:08:07:566] INFO [void load_new_room (STRING* lName, STRING* entPosName)]: loading level: telescope_room.wmb
LevelInit at 44.897
15 entities 0 cameras 0 lights 0 sounds 2 paths
1 lmaps 1 textures 7 meshes 7 submeshes 104 faces...ok
LevelReady at 45.528
[21:08:08:206] INFO [void load_new_room (STRING* lName, STRING* entPosName)]: level was loaded.
[21:08:08:456] INFO [action useable_object()]: starting function
[21:08:08:456] INFO [action useable_object()]: start action for: telescope_controls
[21:08:08:456] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:456] INFO [action useable_object()]: exiting function
[21:08:08:456] INFO [action useable_object()]: starting function
[21:08:08:456] INFO [action useable_object()]: start action for: telescope_right_button
[21:08:08:472] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:472] INFO [action useable_object()]: exiting function
[21:08:08:472] INFO [action useable_object()]: starting function
[21:08:08:472] INFO [action useable_object()]: start action for: telescope_left_button
[21:08:08:472] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:472] INFO [action useable_object()]: exiting function
[21:08:08:472] INFO [action useable_object()]: starting function
[21:08:08:472] INFO [action useable_object()]: start action for: telescope_coinslot
[21:08:08:472] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:472] INFO [action useable_object()]: exiting function
[21:08:08:487] INFO [action useable_object()]: starting function
[21:08:08:487] INFO [action useable_object()]: start action for: telescope_instructions
[21:08:08:487] INFO [action useable_object()]: added object to inRoomObjects vector.
[21:08:08:487] INFO [action useable_object()]: exiting function
[21:08:08:487] INFO [action useable_object()]: starting function
[21:08:08:487] INFO [action useable_object()]: start action for: door_audry_telescope
...
But I just saw something suspicious in one code block ... I will dive deeper into that now.
why is there one more entity if you undid the delete?
this one showed 15 the previous 14
Posted By: FBL
Re: Engine crash after level_load - 02/01/16 21:58
Any active view entities?
While it is unclear whether those could be the culprit in any way, some other user (I think Superku) had such strange crashes with view entities , and also our crashy crashy project used view entities. That's more helpless guessing than anything else, though, and might be completely unrelated in the end.
Posted By: FBL
Re: Engine crash after level_load - 02/01/16 22:10
I'll now do the "delete-undo" trick and check the logfile again.
And here it is:
But I just saw something suspicious in one code block ... I will dive deeper into that now.
So did it just not crash or did it also exit properly ("normal exit at...")?
Posted By: pegamode
Re: Engine crash after level_load - 02/03/16 10:32
why is there one more entity if you undid the delete?
this one showed 15 the previous 14
Sorry, I tried both (removing and adding a model) ... seems I just posted the wrong log. But the result in both cases is exactly the same.
In both cases the crash is gone and I get a "normal exit".
Posted By: pegamode
Re: Engine crash after level_load - 02/14/16 21:23
I broke down the code as much as I could and found out the following:
In one of my c-files I used the following code:
VIEW* ShaftView = {
bmap = "#64x64x14";
material = mtlShaftDepth;
flags = SHOW|NOFLAG1;
}
When I remove the SHOW flag like this:
VIEW* ShaftView = {
bmap = "#64x64x14";
material = mtlShaftDepth;
flags = NOFLAG1;
}
the engine crash is gone.
Is it wrong to use the SHOW flag in this case???
Posted By: jcl
Re: Engine crash after level_load - 02/15/16 07:51
This does indeed give a hint. Is the material a shader? Maybe it is not yet initialized right at start?
Posted By: pegamode
Re: Engine crash after level_load - 02/15/16 08:13
Here's the whole code:
function mtlShaft_event();
MATERIAL* mtlShaftDepth = {
effect = "shaftDepth.fx";
event = mtlShaft_event;
flags = ENABLE_VIEW;
}
VIEW* ShaftView = {
bmap = "#64x64x14";
material = mtlShaftDepth;
flags = SHOW|NOFLAG1;
}
MATERIAL* mtlShaft = {
effect = "lShaft.fx";
}
void mtlShaft_event() {
float sTexAdj[16] = { 0.5, 0.0, 0.0, 0.0, 0.0,-0.5, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0 };
if(ShaftView.bmap) {
sTexAdj[12] = 0.5 + (0.5/(float)ShaftView.bmap.width);
sTexAdj[13] = 0.5 + (0.5/(float)ShaftView.bmap.width);
}
mat_set(mtlShaft.matrix,matViewProj);
mat_multiply(mtlShaft.matrix,sTexAdj);
}
The code's a contribution of another forum member as far as I remember.
Posted By: jcl
Re: Engine crash after level_load - 02/16/16 13:08
I see nothing suspicious at first glace. Can you reduce the project to a minimum that just produces the crash, and send it to Support? We'll look into it.
Posted By: pegamode
Re: Engine crash after level_load - 02/16/16 15:25
Ok ... I will try to reduce it as much as possible and come back to you.
Posted By: pegamode
Re: Engine crash after level_load - 02/20/16 15:34
Sorry, it took me quite a time to break it down this far.
You can download the projekt here:
http://www.meteormess.de/files/crash_test.rarHere's all info I was able to find out:
1) change line "flags = SHOW | NOFLAG1;" to any other flag-combination (only SHOW or only NOTFLAG1" => crash is gone
2) change "video_mode" to 8 => crash is gone
3) modify a random model with prefix "telescope_" (models located in the telescope_room) => crash is gone
4) load only one of the rooms (green_tentacles_room or telescope_room) => crash is gone
I don't see any relations between all those stuff ... I hope you can help.
Posted By: Superku
Re: Engine crash after level_load - 02/20/16 15:38
Posted By: pegamode
Re: Engine crash after level_load - 02/20/16 16:18
Yes, you're right. The crash might be based on the same. At least I hope so :-)
Posted By: HellThunder
Re: Engine crash after level_load - 02/20/16 16:40
Works for my by using
while(proc_status(level_load)) wait(1);
after calling the level_load function.
Posted By: Superku
Re: Engine crash after level_load - 02/20/16 17:30
That's the point and symptom of this crash/ test though, the crash oftentimes disappears (or gets delayed to some other moment) when you change something in the code or a level.
Btw. that line of coding, i.e. waiting for level_load to finish, does not change anything as the function only resumes after level_load has been completed. Only in old WDL or old C-Script times level_load was executed one frame later/ at the end of the frame, thus you needed to wait there.
Posted By: Wjbender
Re: Engine crash after level_load - 02/22/16 09:58
i could not get the script/project to crash on the trial version , but i did get a crash 3 times from the acknex startup on running before the script executed : error in litec at line 25 typedef charbyte
Posted By: pegamode
Re: Engine crash after level_load - 02/22/16 13:12
Hi JCL,
have you been able to reproduce the crash on your system with the test-project I provided?
Kind regards,
Sascha.
Posted By: jcl
Re: Engine crash after level_load - 02/23/16 12:05
Not yet, but I'll do that in the next time. If you're using models with mixed textured and empty skins, it can indeed be the same bug that was recently found.
Posted By: pegamode
Re: Engine crash after level_load - 02/23/16 13:29
Thanks ... we're going to start our beta phase soon and it would be important to get rid of those sporadic crashes first.
Posted By: pegamode
Re: Engine crash after level_load - 02/26/16 07:30
Our models have either textured skins or none, but not a mixture.
Is there any update on this issue?
Posted By: jcl
Re: Engine crash after level_load - 02/26/16 07:59
I could indeed reproduce the crash. It is not caused by the mixed skin bug, but by invalid memory content. Either you have a damaged model or image, or something else in your project is writing into memory.
I've given it to a colleague for looking closer into it next week. In any case you do not need to wait for the skin bug fix since that is not the problem of your project.
Posted By: pegamode
Re: Engine crash after level_load - 02/26/16 08:21
Ok. Thanks for the quick response and for taking a look into it.
Posted By: jcl
Re: Engine crash after level_load - 02/26/16 16:50
I just got confirmation that it's not a wrong model, but in fact an engine bug. More details next week.
Posted By: pegamode
Re: Engine crash after level_load - 02/26/16 18:00
Thanks for the information ... I'm looking forward to more details next week.
Posted By: jcl
Re: Engine crash after level_load - 02/29/16 10:05
The crash was caused by a texture being overwritten under certain, very rare circumstances. That's why it was so hard to reproduce since it disappeared when arbitrary models are added to the level.
As this is the second serious bug found in the last time, we'll produce a quick interim update within the next 2 weeks.
Posted By: Superku
Re: Engine crash after level_load - 02/29/16 10:15
Good news!
Posted By: pegamode
Re: Engine crash after level_load - 02/29/16 14:54
That's great news ... thank you very much.