That's correct (although the one traveling is the younger one in the end). The thing is that the one traveling away and then back changes his inertial system (e.g. by accelerating) which in the end cuts down his time consumption. The acceleration itself doesn't have an effect, though, as you're accelerating and decelerating. If you could change your inertial system without acceleration this would have the same effect.

This would allow us to build a time machine. *want*