There aren't very many references to differences in performance (as in the speed of the web server displaying content to users) between the different image-handling methods. Beyond the few factoids below, the purpose of this page is to act as a collection point for people wanting to write or point to more detailed information on this aspect of comparing the various modules.

Direct HTML references to image files

Just inserting standard <img> tags in line with your content is a very-low overhead method for your site to handle images.

Image module

From File-to-node relationship: it is theoretically possible that, for sites with lots of images (say hundreds of thousands), using Image module to store each image as individual nodes could result in a performance hit from the extra database overhead.

Public vs Private file handling setting

Choosing the Private option, while giving Drupal control over access to files, can cause problems for some image-handling modules as well as possibly impacting a busy site's performance. Google for further information.