I had a look at the code as well, the pointer is the same as on creation. As an experiment (not sure if it makes sense) I checked the bytes before the pointer as well (negative byte-offset), in case something else overflows and overwrites that panel memory. Those bytes have the same value before ptr_remove.
There's not much going on between panel creation and removal either which could damage memory inbetween.
@Quad: In what way could wait damage a local pointer? I've never had issues with that (despite all the other issues one can have with wait).