|
1 registered members (PeroPero),
788
guests, and 6
spiders. |
|
Key:
Admin,
Global Mod,
Mod
|
|
|
limit reading & writing to file
#468227
09/26/17 13:35
09/26/17 13:35
|
Joined: Dec 2011
Posts: 1,823
Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Howdy, I am trying to append some text from a very big data file to an other big data file. I am using file_asc_read and file_asc_write for this. The appending goes fine however it eventually stops at line ~13982. I think it reached a limit or such. When I append the text another time it ends at line ~27963 (about twice as long). Anyway to bypass this limit? Here is the code that does the copying:
var _sizeReadFile = file_length(_handleReadFile); //get size
while (_sizeReadFile > 0) {
var _byte = file_asc_read (_handleReadFile); //read byte
file_asc_write (_handleSave, _byte); //save byte
_sizeReadFile -= 1;
}
Thanks
|
|
|
Re: limit reading & writing to file
[Re: Reconnoiter]
#468234
09/26/17 18:38
09/26/17 18:38
|
Joined: Jun 2007
Posts: 1,337
Hiporope and its pain
txesmi
Serious User
|
Serious User
Joined: Jun 2007
Posts: 1,337
Hiporope and its pain
|
Hi, you can copy the content to a buffer and save it. Here goes an untested example:
long _size = 0;
char *_file = file_load ( "myfile.txt", NULL, &_size );
char *_fileNew = sys_malloc ( _size * 2 );
memcpy ( _fileNew, _file, _size );
memcpy ( _fileNew + _size, _file, _size );
file_save ( "mynewfile.txt", _fileNew, _size * 2 );
file_load ( NULL, _file, NULL );
sys_free ( _fileNew );
|
|
|
Re: limit reading & writing to file
[Re: txesmi]
#468260
09/27/17 11:50
09/27/17 11:50
|
Joined: Dec 2011
Posts: 1,823
Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Hey txesmi that does copy the whole data ty, but how do I append it to the original file? (it now overwrites, something like memcat would be nice if that exists :S)
Last edited by Reconnoiter; 09/27/17 11:52.
|
|
|
Re: limit reading & writing to file
[Re: Reconnoiter]
#468262
09/27/17 12:03
09/27/17 12:03
|
Joined: Nov 2007
Posts: 2,568
Germany, BW, Stuttgart
MasterQ32
Expert
|
Expert
Joined: Nov 2007
Posts: 2,568
Germany, BW, Stuttgart
|
use the stdio.h file API to access files. you can open a file with
FILE * f = fopen("filename", "a");
to append data to the end of the file. also try to prevent single-byte reads and read as much data as possible in a single fread call
|
|
|
Re: limit reading & writing to file
[Re: MasterQ32]
#468267
09/27/17 15:02
09/27/17 15:02
|
Joined: Dec 2011
Posts: 1,823
Netherlands
Reconnoiter
OP
Serious User
|
OP
Serious User
Joined: Dec 2011
Posts: 1,823
Netherlands
|
Hi, tnx for that link & tip. I almost got it working now, there is still a small bug in the code with big data files; as in that it copies & appends everything perfectly but then where it should stop it doesn't and instead appends puts some extra crap. Also the error("reading error") triggers (see code below) so I am probably setting some size variable wrong.
FILE * _fWrite = fopen(_chr(tmpWrite_str), "a");
if (_fWrite != NULL) {
// error("works");
FILE * _fRead = fopen(_chr(tmpRead_str), "r");
if (_fRead != NULL) {
long _lSize;
char* _buffer;
size_t _result;
// obtain file size
fseek (_fRead, 0, SEEK_END);
_lSize = ftell (_fRead);
rewind (_fRead);
// allocate memory to contain the whole file
_buffer = (char*) malloc (sizeof(char)*_lSize);
if (_buffer == NULL) error("memory error");
// copy the file into the buffer
_result = fread (_buffer, 1, _lSize, _fRead);
if (_result != _lSize) error("reading error");
// append buffer to write file
fwrite (_buffer, 1, _lSize, _fWrite);
fclose (_fRead);
free (_buffer);
}
fclose (_fWrite);
}
// else error("error");
|
|
|
Moderated by mk_1, Perro, rayp, Realspawn, Rei_Ayanami, rvL_eXile, Spirit, Superku, Tobias, TSG_Torsten, VeT
|
