I've noticed that if you set the Windows DPI to larger than normal, it also upscales the engine window. So if I request an 800x600 window, I actually get something much bigger and with bigger and blurrier pixels.

It's worse if I try to fullscreen the game, because while my screen is 1920x1080, I can only run Gamestudio games at 720p unless I specifically turn the desktop scaling down to normal. Other games do not have this problem.

Is there a workaround for this or do I have to tell my players to turn off desktop scaling?

If that problem is caused by the engine it really should be fixed.
With the current development of screens (1440p, 4k, etc. getting popular) more and more people use desktop scaling.

The display is controlled by DirectX, not by engine. But I'll check if we can do something about that.