The wrapper used for the ahah replacement on the remove button doesn't span whole element being replaced. (Unlike the wrapper for the upload button, which does). Thus doing things like sending an upload that results in a re-size message, a followed by a "remove" will leave the message on screen.
Upload button uses the full wrapper -> $element['#id'] .'-ahah-wrapper'
Remove button just currently just uses-> $element['#id'] .'-wrapper'
Adding the "ahah" cleans up the problem..