That's exactly the problem. Relativity itself does only tell you that if you're moving, your clock is slower. Then you tell me that I am moving, so my clock is slower. Of course both are right.
The thing which causes a permanent "shift" of time, if you so will, is changing the inertial system (however that is done). That effect is hard to explain in few words but in principle that's the answer. That question is called "twin paradox" (see my link above).

edit: @evilsob: the problem with the gps satellites is not their greater speed but their constant acceleration; this is not a question of special relativity but of general relativity and much much harder.