inkey and Unicode

Posted By: Superku

inkey and Unicode - 01/31/18 11:07

Hello!
I've been trying to implement Unicode keyboard input in my regular lite-C Pure game.
Using on_message I receive WM_CHAR events, including characters sent by a Japanese IME thing for testing. However, it seems like I cannot convert those to actual Unicode characters (the lParam is just 1 which probably is not right for any subsequent ToUnicode calls...?).
I assume this might be because IsWindowUnicode(hWnd) returns 0 and WM_CHAR messages are transformed to ANSI before they are sent to the application.
I've thought about creating my own hWnd with RegisterClassExW (which is not supported in GStudio8/include/windows.h, only RegisterClassEx, so maybe via a DLL) but I couldn't figure that one out yet. Would that be the right way?

Alternatively, I've tried to call
SetWindowsHookEx(WH_KEYBOARD_LL, HookCallback, NULL, 0);
in a DLL or directly in lite-C, then do the Unicode translation there and send the Unicode characters to the engine.
However, it seems like the callback function is not triggered correctly, in particular no response to/ from IME input. WH_KEYBOARD_LL may or may not be the wrong HookID but I did not have luck with any other mode.

Do you have any advice?
Thanks for reading.
Posted By: Kartoffel

Re: inkey and Unicode - 01/31/18 21:42

I gave it a shot and jap. IME does indeed not trigger KEYDOWN events. You might need some other functions to retrieve IME text since it's not really a key press/release but a string of characters that's sent to the window.
Posted By: Superku

Re: inkey and Unicode - 02/03/18 17:26

Yes, the hook is destroying the inputs. (EDIT: Oh, and thanks for giving it a shot!)

I've created a new Unicode hWnd window with the same/ default on_message engine message loop. That one is receiving the same messages as the default engine window, so the issue must be somewhere else (in the message loop itself?).

I don't think I can do this on my own.
Mr. Lotter, do you have any advice?

Thanks.
Posted By: Superku

Re: inkey and Unicode - 02/19/18 12:56

Anyone got an idea how to get Unicode input running in A8? confused
Posted By: Superku

Re: inkey and Unicode - 04/12/18 13:32

Mr. Lotter, can you do something about this, if necessary, release an official/ inoffical interim version with Unicode support? That would be amazing!

The issue in short: on_messages receives only ANSI WM_CHAR input (for IME input which is received as a sequence of multiple WM_CHAR messages, which is correct, this means they oftentimes all have the same values although it's different characters).
Posted By: FBL

Re: inkey and Unicode - 04/17/18 22:30

So you never actually catch a WM_UNICHAR message, only WM_CHAR?

https://msdn.microsoft.com/de-de/library/windows/desktop/ms646288(v=vs.85).aspx

See comment concerning wParam:
Quote:
The Unicode DefWindowProc posts a WM_CHAR message with the same parameters and the ANSI DefWindowProc function posts either one or two WM_CHAR messages with the corresponding ANSI character(s).
Sounds like what is happening to you.

Then you could build your own inkeyw based on custom message handling.

Code:
#include <acknex.h>
#include <windows.h>

LRESULT CALLBACK OriginalHandler(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam);

LRESULT CALLBACK MyMessageHandler(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
   if (WM_UNICHAR == message)
   { 
       printf("OMG it's Unicode!");
       return 0;
   }
   return OriginalHandler(hwnd,message,wParam,lParam);   	  
}

function main() 
{
   OriginalHandler = on_message;	// store the original message loop
   on_message = MyMessageHandler; // and replace it by your own
   ...
}

Posted By: Superku

Re: inkey and Unicode - 04/18/18 16:56

Thanks for the suggestion, I actually hadn't found WM_UNICHAR yet, always only looked at WM_CHAR or non-filtered input. :-X
However, I cannot seem to trigger WM_UNICHAR at all.
I found WM_IME_CHAR which gets fired N times before N characters from an IME arrive, however they are converted to ANSI as far as I can tell before I get my dirty fingers on them. Even when I CreateWindowExW() a new Unicode window (IsWindowUnicode() returns 1 then) I cannot extract or combine WM_CHAR events into unicode digits. confused
Posted By: FBL

Re: inkey and Unicode - 04/18/18 17:20

So if you add your own lessage handling and WM_IME_CHAR message fires, what does wParam contain then? Is this value already truncated?
Posted By: Superku

Re: inkey and Unicode - 04/18/18 17:38

wParam contains a bunch of values, most often 63. Currently I'm only receiving a single WM_CHAR event for each Unicode character - I thought I could trigger multiple calls some time ago -, all of them in the 8 bit range.
Posted By: FBL

Re: inkey and Unicode - 04/18/18 18:49

Hm... 63 is a '?' .... supsicious somehow.
All values for wparam are always in 8 bit range? Strange.

According to the msdn you may receive both single and double byte characters, but this is not Unicode then and requires conversion - which is not trivial at all.
Posted By: FBL

Re: inkey and Unicode - 04/19/18 08:17

This is somewhat but not really working.
The WM_IME_COMPOSITION message carries the last IME character modified. By evaluating lParam it is possible to filter when there's actually something to be done, as this message fires more or less all the time you fiddle around with IME thingy.

Problems:
I think the result is DBCS and not real Unicode
Only one character can be caught this way. All others are still a nasty "?"

A message logger is included (some spam stuff like mouse move events excluded).
I also tried fiddling with PeekMessageW as it was done in some Visual Basic stuff, but this does not seem to help here, although the "?" problem seemed similar.

There is no visual output. Play around and check messages.txt afterwards.

Code:
#include <acknex.h>
#include <windows.h>

#define WM_INPUT 0x0099
#define WM_UNICHAR 0x0109
#define WM_IME_CHAR 0x0286 

long WINAPI PeekMessageW(long lpMsg,long hWnd,long wMsgFilterMin,long wMsgFilterMax,long wRemoveMsg);
#define PRAGMA_API PeekMessageW;user32!PeekMessageW

WPARAM unichar = 0;
WPARAM resultchar = 0;
MSG msg; 
var file;
STRING* msgstr = "#300";

LRESULT CALLBACK OriginalHandler(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam);

LRESULT CALLBACK MyMessageHandler(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
	if(message >= WM_INPUT && message != WM_MOUSEMOVE && message != WM_ENTERIDLE && message != WM_NCMOUSEMOVE)
	{
		str_printf(msgstr, "message : <0x%x>, WPARAM : <%d>, LPARAM <0x%x>.rn", message, wParam, lParam);
		file_str_write(file, msgstr);
	}

	if (WM_IME_COMPOSITION == message)
	{
		unichar = 0;
   	//if (PeekMessageW(&msg, hwnd, WM_IME_CHAR, WM_IME_CHAR, PM_NOREMOVE))
   	
   	if (lParam == (GCS_RESULTREADSTR|GCS_RESULTREADCLAUSE|GCS_RESULTSTR|GCS_RESULTCLAUSE))
   	{
				//next message will be char transfer, so grab char of last ime change
   			unichar = wParam;
   			str_printf (msgstr, "IME CHAR cached: %d.rn", unichar);	
				file_str_write(file, msgstr);
		}
	}
	
   if (WM_CHAR == message)
   {
			str_printf (msgstr, "WM_CHAR received - real: %d cached: %d.rn", wParam, unichar);	
			file_str_write(file, msgstr);
	   	if (unichar != 0) 
	   	{
	   		resultchar = unichar; 
	   		unichar = 0;
			}
			else
			{
				resultchar = wParam;
			}
		//now do something useful with resultchar
   }
   return OriginalHandler(hwnd,message,wParam,lParam);   	  
}

function main() 
{
	file = file_open_write("messages.txt");
   OriginalHandler = on_message;	// store the original message loop
   on_message = MyMessageHandler; // and replace it by your own
}



Sample message log:
Code:
message : <0x210>, WPARAM : <262146>, LPARAM <0x14105c>.
message : <0x31f>, WPARAM : <1>, LPARAM <0x0>.
message : <0x100>, WPARAM : <81>, LPARAM <0x100001>.
message : <0x102>, WPARAM : <113>, LPARAM <0x100001>.
WM_CHAR received - real: 113 cached: 0.
message : <0x100>, WPARAM : <87>, LPARAM <0x110001>.
message : <0x102>, WPARAM : <119>, LPARAM <0x110001>.
WM_CHAR received - real: 119 cached: 0.
message : <0x101>, WPARAM : <81>, LPARAM <0xc0100001>.
message : <0x101>, WPARAM : <87>, LPARAM <0xc0110001>.
message : <0x100>, WPARAM : <69>, LPARAM <0x120001>.
message : <0x102>, WPARAM : <101>, LPARAM <0x120001>.
WM_CHAR received - real: 101 cached: 0.
message : <0x101>, WPARAM : <69>, LPARAM <0xc0120001>.
message : <0x2a2>, WPARAM : <0>, LPARAM <0x0>.
message : <0x11f>, WPARAM : <-2139094504>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2013069312>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2138046448>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2138046449>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2138046450>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2013069312>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2139094924>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2139094916>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2138046454>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2138046455>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2139094918>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2139094920>, LPARAM <0xd8350887>.
message : <0x11f>, WPARAM : <-2139094922>, LPARAM <0xd8350887>.
message : <0x215>, WPARAM : <0>, LPARAM <0x0>.
message : <0x11f>, WPARAM : <-65536>, LPARAM <0x0>.
message : <0x2a2>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <8>, LPARAM <0x0>.
message : <0x10d>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <269>, LPARAM <0x0>.
message : <0xc055>, WPARAM : <1>, LPARAM <0x19ee08>.
message : <0x288>, WPARAM : <6>, LPARAM <0x19ee08>.
message : <0x282>, WPARAM : <5>, LPARAM <0x1>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0xc055>, WPARAM : <1>, LPARAM <0x19f888>.
message : <0x288>, WPARAM : <6>, LPARAM <0x19f888>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0xb0>, WPARAM : <1701960>, LPARAM <0x19f84c>.
message : <0xb0>, WPARAM : <1701960>, LPARAM <0x19f84c>.
message : <0x282>, WPARAM : <4>, LPARAM <0x1>.
message : <0x100>, WPARAM : <229>, LPARAM <0x1c0001>.
message : <0x10f>, WPARAM : <113>, LPARAM <0x1e00>.
IME CHAR cached: 113.
message : <0x286>, WPARAM : <113>, LPARAM <0x1>.
message : <0x286>, WPARAM : <119>, LPARAM <0x1>.
message : <0x286>, WPARAM : <101>, LPARAM <0x1>.
message : <0x10e>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <270>, LPARAM <0x0>.
message : <0x102>, WPARAM : <113>, LPARAM <0x1>.
WM_CHAR received - real: 113 cached: 113.
message : <0x102>, WPARAM : <119>, LPARAM <0x1>.
WM_CHAR received - real: 119 cached: 0.
message : <0x102>, WPARAM : <101>, LPARAM <0x1>.
WM_CHAR received - real: 101 cached: 0.
message : <0x101>, WPARAM : <13>, LPARAM <0xc01c0001>.
message : <0x10d>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <269>, LPARAM <0x0>.
message : <0xc055>, WPARAM : <1>, LPARAM <0x19ee08>.
message : <0x288>, WPARAM : <6>, LPARAM <0x19ee08>.
message : <0x282>, WPARAM : <5>, LPARAM <0x1>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0xc055>, WPARAM : <1>, LPARAM <0x19f888>.
message : <0x288>, WPARAM : <6>, LPARAM <0x19f888>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0xb0>, WPARAM : <1701960>, LPARAM <0x19f84c>.
message : <0xb0>, WPARAM : <1701960>, LPARAM <0x19f84c>.
message : <0x282>, WPARAM : <4>, LPARAM <0x1>.
message : <0x100>, WPARAM : <229>, LPARAM <0x1c0001>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1e00>.
IME CHAR cached: 51842.
message : <0x286>, WPARAM : <63>, LPARAM <0x1>.
message : <0x10e>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <270>, LPARAM <0x0>.
message : <0x102>, WPARAM : <63>, LPARAM <0x1>.
WM_CHAR received - real: 63 cached: 51842.
message : <0x101>, WPARAM : <13>, LPARAM <0xc01c0001>.
message : <0x10d>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <269>, LPARAM <0x0>.
message : <0xc055>, WPARAM : <1>, LPARAM <0x19ee08>.
message : <0x288>, WPARAM : <6>, LPARAM <0x19ee08>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <5>, LPARAM <0x1>.
message : <0xc055>, WPARAM : <1>, LPARAM <0x19f888>.
message : <0x288>, WPARAM : <6>, LPARAM <0x19f888>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1b9>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0xb0>, WPARAM : <1701960>, LPARAM <0x19f84c>.
message : <0xb0>, WPARAM : <1701960>, LPARAM <0x19f84c>.
message : <0x282>, WPARAM : <4>, LPARAM <0x1>.
message : <0x100>, WPARAM : <229>, LPARAM <0x1c0001>.
message : <0x10f>, WPARAM : <51842>, LPARAM <0x1e00>.
IME CHAR cached: 51842.
message : <0x286>, WPARAM : <63>, LPARAM <0x1>.
message : <0x286>, WPARAM : <63>, LPARAM <0x1>.
message : <0x10e>, WPARAM : <0>, LPARAM <0x0>.
message : <0x282>, WPARAM : <15>, LPARAM <0xa790bff>.
message : <0x282>, WPARAM : <270>, LPARAM <0x0>.
message : <0x102>, WPARAM : <63>, LPARAM <0x1>.
WM_CHAR received - real: 63 cached: 51842.
message : <0x102>, WPARAM : <63>, LPARAM <0x1>.
WM_CHAR received - real: 63 cached: 0.
message : <0x101>, WPARAM : <13>, LPARAM <0xc01c0001>.
message : <0x2a2>, WPARAM : <0>, LPARAM <0x0>.
message : <0x281>, WPARAM : <0>, LPARAM <0xc000000f>.
message : <0x281>, WPARAM : <1>, LPARAM <0xc000000f>.
message : <0x2a2>, WPARAM : <0>, LPARAM <0x0>.
message : <0xa1>, WPARAM : <20>, LPARAM <0x1610524>.
message : <0x215>, WPARAM : <0>, LPARAM <0x0>.
message : <0x112>, WPARAM : <61536>, LPARAM <0x1610524>.
message : <0x2a2>, WPARAM : <0>, LPARAM <0x0>.
message : <0x281>, WPARAM : <0>, LPARAM <0xc000000f>.
message : <0x102>, WPARAM : <32>, LPARAM <0x0>.
WM_CHAR received - real: 32 cached: 0.



character list, which does not seem to match to what I clicked on IME keyboard:
http://thinkingstiff.com/characters.html
Posted By: Superku

Re: inkey and Unicode - 04/19/18 09:36

Thanks for giving it a go again.

I've been using the following console for a few years now, it has been invaluable to me:

Click to reveal..
Code:
// made by Rackscha, adapted by Superku

#ifndef Console_h
	#define Console_h
	#include<windows.h>;

	long WINAPI WriteConsole(int Handle, char* Buffer, int CharsToWrite, int* CharsWritten, int reserved);
	long WINAPI CreateConsoleScreenBuffer(long dwDesiredAccess, long dwShareMode, long *lpSecurityAttributes, long dwFlags, long lpScreenBufferData);
	long WINAPI SetConsoleActiveScreenBuffer(long hConsoleOutput);
	long GConsoleBuffer;
	int consoleInitialized = 0;
	int consolePrintTrue = 1;
	int consolePrintTarget = 2; // write in both acklog and console

	void consoleInit()
	{
		if(consoleInitialized) return;
		consoleInitialized = 1;
		AllocConsole();
		GConsoleBuffer = CreateConsoleScreenBuffer(GENERIC_WRITE, FILE_SHARE_READ, 0, CONSOLE_TEXTMODE_BUFFER, 0);
		SetConsoleActiveScreenBuffer(GConsoleBuffer);	
	}

	void cdiag(char* AText)
	{
		if(!consolePrintTrue) return;
		if(consolePrintTarget) diag(AText);
		if(consolePrintTarget == 0 || consolePrintTarget == 2)
		{
			if(!consoleInitialized) consoleInit();
			WriteConsole(GConsoleBuffer, AText, str_len(AText), NULL, 0);
		}
	}
	
	#define cprintf0(str) cdiag(_chr(str))
	#define cprintf1(str,arg1) cdiag(_chr(str_printf(NULL,str,arg1)))
	#define cprintf2(str,arg1,arg2) cdiag(_chr(str_printf(NULL,str,arg1,arg2)))
	#define cprintf3(str,arg1,arg2,arg3) cdiag(_chr(str_printf(NULL,str,arg1,arg2,arg3)))
	#define cprintf4(str,arg1,arg2,arg3,arg4) cdiag(_chr(str_printf(NULL,str,arg1,arg2,arg3,arg4)))
	#define cprintf5(str,arg1,arg2,arg3,arg4,arg5) cdiag(_chr(str_printf(NULL,str,arg1,arg2,arg3,arg4,arg5)))
	#define cprintf6(str,arg1,arg2,arg3,arg4,arg5,arg6) cdiag(_chr(str_printf(NULL,str,arg1,arg2,arg3,arg4,arg5,arg6)))
	#define cprintf7(str,arg1,arg2,arg3,arg4,arg5,arg6,arg7) cdiag(_chr(str_printf(NULL,str,arg1,arg2,arg3,arg4,arg5,arg6,arg7)))
#endif


Then logging is as simple as writing
Code:
cprintf4("nMyMessageHandler WM_CHAR at frame %d: msg(%d) w(%d) l(%p)",(int)total_frames,(int)message,(int)wParam,lParam);


Screenshot: https://i.gyazo.com/61f642d7b3d7e767e66e543ecd1fa9c4.png

I've had no luck using PeekMessageW either so far but I'm gonna check out WM_IME_COMPOSITION and related content later today.
© 2024 lite-C Forums