Okay,
I have done some thinking after taking a little break. I think I came up with a much easier way to handle this. If you are thinking of helping me solve this, don't worry about it just yet. I will update again later...

See my games here: https://www.indiedb.com/company/michael-klaus-schmidt