I don't think you'd want to remove the whole address. I think if you leave an otherwise blank address, but with Country=UK, that might work.
Did you trying saving the bills once you messed with the address stuff? I have a vague recollection that messing with address stuff might take a Save to take effect versus switching forms where the fields update right away.
If you remove the address, except Country=UK, then save and go back into edit mode, does it update?