It's perfectly fine the way it is, that's game dev.
I bet txesmi has found the reason for why it behaves like it does.

In case you need deterministic/ framerate independent outcome (which you normally do NOT need), use a lockstep or fixed step approach and blend the visuals between old and new pos. I do/ did that for some multiplayer stuff, that's it.