Would appear that if the Custom Contact View is filtered to only allow selection of users in a particular group and a user is later removed from this group, one has to remember to unflag the user otherwise they will still show up in the contact form. The module should be able to cleanup itself whereas a user removed from the group which is a selectable set (for toggling) then all flags for the user (as pertinent to this module) should be removed.