More than One "File_Open_Read" - Problem?

Is it problematic to have more than one external file open at the same time?

I am using file_open_read to gather data for a saved level, then using another file_open_read to gather data for some specific objects in that level (which are saved in other files).

So while the first file is open, it will then open the second file to get info on the object, then it will close the second file and continue reading from the first.

In general it works, but I've noticed that errors seem to creep into the data once several of these objects have been saved and need to be loaded.

I am having trouble tracking down the source of these errors, but I just realized that having two files open may be a possible cause. The files have different names, and I'm using different variables to store the file handles.

I've noticed that this seems to be fine when I load the level after I restart the game, but if I'm loading the level from another level (returning to the previously saved level) the errors are more likely to creep in.

Is it possible for you to close the first file, open the second file, read and store the values you need in temporary areas, then reopen the initial file and continue reading? See if that works!

Thanks!

I will definitely try that, it just seems like the wrong approach to keep two files open like that anyway.

You can have as many files open as you want(*), in fact, by default your application will already touch a bunch of files just from being opened. It's not uncommon for larger applications to have 50+ open file handles at the same time, so no, that's definitely not the source of your problems.

I would argue you should see about finding the bug in your code that is causing the data corruption. Chances are it will also show up with just one file open, just with a different configuration. And if that's not enough, keep in mind that NTFS is a horrible, horrible filesystem that uses one big lock so every filesystem access is extremely expensive. Definitely not something you want to do in real time.

@WretchedSid,
Yes, that's kind of what I was afraid of.

Every time I think I've figured out the cause, it turns out I'm wrong.

The annoying thing is that you have to play for quite some time for the problem to appear, so for a long time I thought it was working fine. I tested it and it worked, so hey, I kept adding new features etc...

Now that I have actual players, playing the game, the problem shows up.

Can you think of anything that might cause a local variable to be assigned strange numbers that I am not assigning to them? I'm thinking it might be some kind of memory leak, that kicks in after playing for some time...

I don't want to bog anyone down with my actual script, which is quite long and complicated, but what's happening is that when the game reads from a file, and it reaches the end of the file, it's supposed to read a zero "0".

However, sometimes, instead of a zero, it reads some strange random number. I have no idea why these numbers show up, instead of the zero.

, have you tried error(myValueString); ?
Also when the weird number shows up, in your text file it still shows 0 right?

Okay, I am pretty sure I have a memory leak. When I play the game, each time it loads a new level, I can see the memory going down to a certain point, then it builds back up as the level gets loaded.

However, each time it does this, it drops down to a point slightly higher than the last time, so each level starts to use up more and more memory. Once it reaches a certain level, the crashes and errors start occurring.

This strange variable bug is just one of a few bugs that occur, but it is the one that occurs most often, so I thought I would deal with it first. However, now I can see that it just depends on what the game is doing once the memory reaches that high level.

@Reconnoiter - is error(myValueString) an A8 feature? I have A7 and the manual does not mention anything about it...

The text file does not show the strange numbers. It's happening when the function reaches the end of the file, so it should return 0 and end the function. This works fine, until, as I've found out, the memory use gets too high.

I will search for memory leaks here on the forum. If anyone knows about finding/detecting/isolating them, please let me know where to look.

Oops forgot you use A7, error() maybe is a A8 feature (search for "error" without quotes).

A few ideas where to look:
- since when do you have this leak? Try remembering the things you have added, especially pointer related and ent_remove / ptr_remove related and array etc.
- check if any pointer you use is not NULL (especially entity, panel and bmap pointers)
- check your arrays if you do not access anything outside of the array's limit/size (e.g. myArray[-1] , silly example I know ).
- check if you dont do anything complicated in your entity events (or if you forgot to do a wait(1); before e.g. doing a c_scan or spawn particles or such in an entity event )

I think I found my problem. Every time I create a certain map entity, the nexus goes up a bit. I can't believe I did not notice this.

I use ent_purge when removing the entity, but this does not seem to reduce the nexus. When it is created again, the nexus goes back further each time.

I think by using a model, instead of a map entity, this problem will be solved.

Or maybe there is a way to reduce the nexus when removing a map entity, so it will not build up?

Yes, replacing my map entity with a model does not increase the nexus on each ent_create.

Dooley, I think you are chasing the wrong issue here. A memory leak doesn't lead to memory corruption. Leaking memory isn't good and should be fixed, but it's not the cause of your garbage data. That usually comes from writing into corrupt pointers and overriding existing data.

Possibly. However, the memory leak is causing the game to crash too. If I can fix it, at least I can move on to the next possible cause...

Oh man, you were right. I did a great job of reducing the amount of memory the game uses. Memory usage seems to stabilize much lower than the previous version of the game.

However, the data corruption is still happening.

Okay, I am opening up a file, reading from it, and storing the information in variables. Then, it re-writes the file with updated information and saves it again. So I guess this may be what you described as "overriding existing data."

Can you tell me if there is a right or wrong way to do this? If it's as simple as just not over-writing a file, I may be able to come up with a better way to do it.

Okay I found a potential cause of corruption.

For some reason, when I wanted to check if a file existed, I was using

if(file_open_read(filename))

instead of

if(file_exists(filename))

In most cases this actually worked okay, since the function was looking for an existing file. However, in one case, it would only work when the file did not exist.

In this case, if the file did exist, I guess that file would actually remain open for reading, and it never got closed. I am about to test the game with this corrected ... wish me luck!

Download process hacker or any other tool and check for open (file) handles.
You will spot 99% of all not closed handles instantly

And yes, your above code was leaking a handle on ANY valid file because you didnt save the handle/return value...

However, in theory, this IS the way to go to check if a file exists IF you want to read it as well!

The reason is that otherwise you might end up with a race condition: You check if the file exists and then open the file. Between the check and you opening the file, the file gets removed and you end up trying to open a file that doesn't exist (which returns an invalid handle).

The right way to do it is to open the file, which checks if it exists anyway, and then to check the returned file handle. And as Chaos pointed out, don't forget to close the files once you are done reading them, or you'll leak file handles.

Okay, thank you!

So I'm imagining something like this:

This is what I'm doing currently (after fixing it), do you think this will cause errors?

This version is more terse and does exactly what you want.

Thank You!