temporary uploaded files not deleted

When I use imagefield to upload image , I found that if I select a image and press upload, then it will upload to my server. it's a correct action.
However, if I uploaded a wrong image, then I replace another one image and press upload button. the wrong image that I have uploaded still exist in my server.

I guess if a hacker create a script to replace replace ad replace image. then many image will upload to my server. also, some user actually upload the wrong image , that also use of the server space.