here is a theoretical idea for you wich you could try
out and if it works okay it could possibly make things
simpler...

if you are placing these buildings on top of terrain
you could determine the chunk of terrain your entity
are placed upon , then you could alter that chunk
of terrain by merging the entity mesh with the terrain
chunk's mesh ,then get rid of the entity and only
keep track of where that entity is (what terrain chunk it is on)

this way the terrain chunks should manage the added
geometry on it's own ..

The only problem with this idea is ,I dont know how
you would keep your skins correct and you would
actually add more geometry to the terrain chunk
than was initially planned for each chunk upon
creation ..

enough from me ..

jb