H2 block titles should disappear if no title specified

Solution is to edit block.tpl.php as follows:

  <div class="block block-<?php print $block->module; ?>" id="block-<?php print $block->module; ?>-<?php print $block->delta; ?>">
  
    <?php if($block->subject) : ?>
    <h2 class="title">
      <?php print $block->subject; ?>
    </h2>
    <?php endif; ?>
    <div class="content">
      <?php print $block->content; ?>
    </div>
  </div>

Note the:

    <?php if($block->subject) : ?>
    <h2 class="title">
      <?php print $block->subject; ?>
    </h2>
    <?php endif; ?>

OK, this may sound dumb, but I'm a Drupal newbie, so please bear with me. :P

I'm having this problem too, and I understand your solution, but where do I find block.tpl.php? I can't seem to find it! Thanks.

block.tpl.php is located in the theme's directory; it came with the theme.

I believe another solution is to put the string <none> within the block title field, if you don't want to show the title.

Added jwolf's code in #2 to 6.x-1.x-dev, and will also be available in 6.x-1.4 release.