Unicode issue

Posted By: BES

Unicode issue - 09/29/08 18:08

I know that 3DGS doesn't currently support Unicode, but before I purchased the engine, I emailed Conitec about this and they assured me that it was possible. Does anyone know how this is possible? The application I'm developing require the use of the asian language. I'd like to know if this is possible or did I choose the wrong engine?
Posted By: Michael_Schwarz

Re: Unicode issue - 09/29/08 18:46

Originally Posted By: BES
I know that 3DGS doesn't currently support Unicode


It does

Quote:
,but before I purchased the engine, I emailed Conitec about this and they assured me that it was possible.


see?

Quote:
Does anyone know how this is possible? The application I'm developing require the use of the asian language.


Well, first you need to use TrueType fonts and not Bitmap fonts. (before you ask: Look under "FONTS" in the manual) Second, the font must be Unicode compatible (Like Arial Unicode for example) and third, the customer needs to have that font installed, if necessary make the installer/setup program install it.

Quote:
I'd like to know if this is possible or did I choose the wrong engine?


As you can see, you didn't, as it is possible.
Posted By: Quad

Re: Unicode issue - 09/29/08 21:10

you know just wanted to post .


and some chinese characters:

project thread: http://www.coniserver.net/ubb7/ubbthreads.php?ubb=showflat&Number=221090#Post221090
Posted By: BES

Re: Unicode issue - 09/30/08 01:49

I'm glad to hear that 3DGS is unicode ready. Quadraxas, your screenshot looks real cool. Do you have some code examples of how you got the asian characters on the screen. I've installed Arial Unicode MS, but can't anything to show up. Thanks.
Posted By: DavidLancaster

Re: Unicode issue - 09/30/08 02:00

Hey guys, myself and Simon have been trying to implement unicode.

Here is how we are defining the font:

FONT* GlobalFont = "Arial Unicode MS#16";

We are then reading text from a unicode .txt file and storing the data into a string. But when I display the string on screen the text isn't in Japanese it's just normal Ariel font. Any ideas how to implement it?
Posted By: Quad

Re: Unicode issue - 09/30/08 05:30

@BES: refer to the thread link i posted. The Chinese project is not mine.(bomber's i guess.)

@david: i cant help about japanese chars, this time. Maybe someone else...
Posted By: Michael_Schwarz

Re: Unicode issue - 09/30/08 12:51

Originally Posted By: DavidLancaster
Hey guys, myself and Simon have been trying to implement unicode.

Here is how we are defining the font:

FONT* GlobalFont = "Arial Unicode MS#16";

We are then reading text from a unicode .txt file and storing the data into a string. But when I display the string on screen the text isn't in Japanese it's just normal Ariel font. Any ideas how to implement it?


my guess is, that the textfile uses a specific chinese font for Western PC's (like I have some Thai fonts you can use to type thai text on western keyboards). In that case you'd have to use that specific font.

Try to open the file in notepad and set the font from notepad to arial unicode and if you see normal characters you know that you are using the wrong font type the text was written in.
Posted By: DavidLancaster

Re: Unicode issue - 10/01/08 05:24

Hi Michael,

Thanks for your help. I've set the font to Arial Unicode MS and the text appears as it should, in japanese (in notepad). The problem is either the font in gamestudio or that strings can't hold unicode text??? I load the text into a 3DGS string, display it on screen with the font Arial Unicode MS, and it displays incorrectly.

Any ideas?

David
Posted By: DavidLancaster

Re: Unicode issue - 10/01/08 05:30

This is how it looks:



Here is my code:

Code:
var filehandle_n;
STRING* strUNICODE = "                                                        ";

TEXT* testing_text =
{
	pos_x = 0; pos_y = 0; red = 0; green = 0; blue = 0; layer = 99;
	font = GlobalFont; string ("                              "); flags = FILTER | VISIBLE;
}

function testing_startup()
{
	str_cpy(gString,"LoadThisString.txt");
	filehandle_n = file_open_read(gString);
	if (filehandle_n != 0)
	{
		file_str_read(filehandle_n,strUNICODE);
	}
	file_close(filehandle_n);
	str_cpy((testing_text.pstring)[0],strUNICODE);
}

Posted By: Quad

Re: Unicode issue - 10/01/08 09:18

try copying your text file into a TEXT*. if this does work, then it has something to do with file_str_read.
Posted By: DavidLancaster

Re: Unicode issue - 10/01/08 09:52

Thanks Quadraxax. I tried it and I get the same result, it doesn't work.
Posted By: Pappenheimer

Re: Unicode issue - 10/01/08 11:17

Just an idea, didn't test it:
What, if you use "file_asc_read" instead of "file_str_read", and then "str_for_asc"?
Posted By: DavidLancaster

Re: Unicode issue - 10/01/08 13:38

Thanks Pappenheimer.

file_asc_read (handle)
Reads a byte (0..255) out of the file with the given handle

The problem with that is that unicode fonts extent past 255 bytes.
Posted By: DeepReflection

Re: Unicode issue - 10/01/08 15:55

@ David Yes, and No! the binary representation in the text file, I'm pretty sure still will be in some byte/char/8bit length, the problem is how this is encoded. As fast as there are string/char functions and so on involved You must know how it was implemented. I was tempted to make a hasty comment, but the sanity hold me off, and I locked UTF up a bit on Wikipedia UTF crazy It's a "bit" complicated, least to say.

As a fast and dirty trick, I would open up your text file with a HEX editor of some kind, try other applications like Wordpad or MS Office, Open Office, Google translator function on Internet usw. Check the differences and or put these different code into your test for GameStudio until it gets right. Then you know at least what application you have to use to create textfiles for the game.

Your example reminds me when I tried to translate a Japanese text file, I had to try out some applications before the cut and pasted text was considered anything else then (/%&&/%¤"#% by Google translator. smile
Posted By: Michael_Schwarz

Re: Unicode issue - 10/01/08 16:17

that could be indeed a problem.

Most windows applications save textfiles with ISO-8859-1 or ANSI encoding, perhaps what the engine needs is a file with UTF-8 encoding(preferably without signature). Have you tried re-saving the file in notepad by selecting UTF-8 encoding?

Or maybe even the other way around, I dont recall how it was exactly right now. In any case, it couldn't hurt to try different file encodings.
Posted By: BES

Re: Unicode issue - 10/02/08 17:20

David, I know that Dreamweaver saves documents in UTF-8 format. Perhaps you can use that text editor to get the properly formatted document.
Posted By: BES

Re: Unicode issue - 10/02/08 18:08

This is Mark @ Brethren- tried saving the file as UTF-8 and using a unicode font in A7, but no go. I have a few words in japanese and a few in english and the english displays fine, but not the japanese. I've tried it with only japanese too, though it shouldn't matter, and it's still garbled. Simon thinks that the engine is only grabbing a byte at a time when rendering instead of double byte (which is consistent since the garbled japanese is twice as many characters as the input japanese text). Really curious how others got the text to display in the screenshots above!
Posted By: bomber

Re: Unicode issue - 10/03/08 09:41

I'm not sure why ur having this problem, this is the code in my game(it has Chinese in it)
string strChat0[1000];
string strChat1[1000];
string strChat2[1000];
string strChat3[1000];
string strChat4[1000];
string strChat5[1000];
string strChat6[1000];
string strChat7[1000];
string strChat8[1000];
font newfont4me = "Arial",1,20;
text txtChat
{
pos_x = 680;
pos_y = 500;
layer = 30;
strings = 9;
font newfont4me;
flags=visible;
string = strChat8, strChat7, strChat6, strChat5, strChat4,
strChat3, strChat2, strChat1, strChat0;
red=255;
blue=0;
green=0;
}
But it works just fine maybe it's cause I have a Chinese version of Windows^
Posted By: BES

Re: Unicode issue - 10/06/08 18:25

Thanks so much for the reply bomber! Are you loading your text from an external file, or are you able to type in Chinese strings in SED with the Chinese version of windows?
Posted By: learner

Re: Unicode issue - 10/08/08 08:39

I am sorry that I am new to 3DGS.

My problem is that when I type the Chinese characters using MS IDE, it turns into some strange symbols when executing. I have set the default to the Chinese character. When I type, I can see the Chinese displayed properly. The problem only occurs when I run the script.

What do I need to do? Do I need to define the Chinese font? Id so, can you show me an example?

Many thanks.
Posted By: bomber

Re: Unicode issue - 10/11/08 11:56

Originally Posted By: BES
Thanks so much for the reply bomber! Are you loading your text from an external file, or are you able to type in Chinese strings in SED with the Chinese version of windows?

Both, the magic and weapons names are from external files, the chat and reminder text are in Chinese strings in SED
Posted By: DavidLancaster

Re: Unicode issue - 10/14/08 07:48

I'm beginning to believe that this isn't possible with 3DGS...no one can provide a working example. Would anyone be interested in being paid to provide a working script which successfully implements a unicode font into 3d game studio?
Posted By: bomber

Re: Unicode issue - 10/18/08 09:13

ok I'll try!
Posted By: DavidLancaster

Re: Unicode issue - 10/21/08 06:26

Please let us know how you get on Bomber. Thank you so much!
Posted By: DeepReflection

Re: Unicode issue - 10/28/08 23:54

Hi,

I tried and I thought this should have done the trick really,

This is the output from Notepad.




Now from this text code I extracted out of the saved file with hexedit, translated that to decimal, fiddled with a char array and put that to screen. Bad result, abc was shown nicely though whistle

Some notes:

First notepad embedding some switchcode in the file, I presume this tells a similar application that open this file to use SimHei UTF font. Game Studio seems not to care for this.

Secondly notice that each advanced character is built of three bytes with a start byte (character) that tells the application to sum the next two to the actual character you want to show.

Actually I think JCL or someone of the team have to step in and tell what command codes we have to inject to get the wanted characters shown.


Code:
////////////////////////////////////////////////////////////////////
#include <acknex.h>
#include <default.c>
////////////////////////////////////////////////////////////////////

// Font with asian characters
FONT* some_font = "SimHei#25b"; // true type, WhatEver, bold, size = 25

STRING* aloha = str_create("#100");

STRING* aloha = ""; // Nullify string, TEXT* crack with non initialized STRING*

// Small text saved with notepad and SimHei font
//               switch?    a, b, c,       asian,      asian,       a, b, c,
// HEX output   ef,bb,bf,	 61,62,63,    e4,bb,a7,	   e4,bc,bf,     61,62,63,  
// Decimal    239,187,191,  97,98,99,  228,187,167,  228,188,191,   97,98,99,

char test[30] = { 239,187,191,97,98,99,228,187,167,228,188,191,97,98,99};

char endtest[30] = { '|',' ','A','n','y',' ','L','u','c','k','?'};

TEXT* greetings_txt =
{
    pos_x = 150;
    pos_y = 250;
    font = some_font;
    string (aloha);
    flags = VISIBLE;
}

////////////////////////////////////////////////////////////////////
function main()
{
    str_cpy(aloha, test); 
    video_mode = 6;
    screen_color.blue = 150;
    wait (-3);
    str_cat(aloha, endtest); 
}






Posted By: jcl

Re: Unicode issue - 10/29/08 09:36

The method above has nothing to do with unicode. It just displays ASCII characters.

Although Unicode is not officially supported by Gamestudio, maybe there are some tricks to display unicode text nevertheless - if so, I don't know them. However, you can of course display foreign characters by using a foreign character ASCII Truetype font.
Posted By: DeepReflection

Re: Unicode issue - 10/29/08 17:33

Oki, that mean GameStudio functions only handle single byte characters between 0-255 (8 bit ASCII), correct?

And anyone that want to use a TrueType font with special characters have to use the country specific one or fork a own one and place the wanted characters in position 0-255 in that font library?
Posted By: jcl

Re: Unicode issue - 10/30/08 12:36

Strings can contain 8 bit or 16 bit characters. The latter may be a backdoor for some unicode tricks by users that I don't know. However, the DirectX functions that display the TEXT object are indeed set up to interpret strings as single byte ASCII characters. Therefore, you'll need a country specific font for displaying Greek, Cyrillic or other special characters.
Posted By: BES

Re: Unicode issue - 10/31/08 16:57

Hi jcl. Thanks for chiming in on the conversation. Can you direct me to the DirectX functions that are responsible for rendering the TEXT object? Perhaps I may be able to look at the code and see if there's a workaround. If Bomber can show chinese, we at least know that strings can contain 16 characters. Thanks.
© 2024 lite-C Forums