How can I remove History from profile.

Maybe a bit late, but it does work that way!

Open the user.module file in the Module>>user folder and comment out line 458 to 466 (including the '}')
(tested with Durpal 5.2)

this is correct way http://drupal.org/node/35728
(use profile module in core)
1) create in template.php function phptemplate_user_profile

function phptemplate_user_profile($user, $fields = array()) {
return _phptemplate_callback('user_profile', array('user' => $user, 'fields' => $fields));
}

2) create file user_profile.tpl.php and output what you want,
test:

<div>
    <h3>$USER:</h3>
    <?php foreach($user as $k=>$v)
        if (is_array($v)) foreach($v as $kk=>$vv) print "--$kk=$vv<br />\n";
        else print "$k=$v<br />\n";
        ?>

    <h3>$FIELDS:</h3>
    <?php foreach($fields as $k=>$v) {
        if (is_array($v)) {
            foreach($v as $kk=>$vv) {
                if (is_array($vv)) foreach($vv as $kkk=>$vvv) print "-- -- $kkk=$vvv<br />\n";
                else print "-- $kk=$vv<br />\n";
            }
        }
        else print "$k=$v<br />\n";
        }
        ?>
</div>

You should not correct drupal's output by changing drupal's code.
If you want to change functionality instead of appearance, you should choose to modify drupal's output in a custom module. In the example, you would have a module called mywebsite. Into the file mywebsite.module you put the following code:

/**
 * Implementation of hook user to do some customization of this website
 */
function mywebsite_user($type, &$edit, &$user, $category = NULL) {
  if ($type == 'view') {
    // turn off history by returning an empty array having the same key as the one defined in user_user()
    $items['history'] = array();
    return array(t('History') => $items);
  }
}

Put the file into /sites/all/modules/custom/mywebsite directory. Put an according mywebsite.info into the directory, too. Activate the module as usual. More information in module developer guide.

There appears to be a module that can do this: http://drupal.org/project/myaccount_alter

"Gets rid of unwanted sections on the 'My Account' page such as the 'History' section or anything added there by a contributed module.

"Install, then go to the myaccount_alter settings page and check off the sections you would like to disable."

I haven't used it, but was searching site on similar keywords and thought this might be a good lead for you. I note the module was updated 2008-Feb-16 (rel. 5.x-1.2), which indicates its a living project.

There is a hook_profile_alter to modify this, just implement it in a module:

function mymodule_profile_alter(&$account, &$fields) {  
    foreach ($fields AS $category => $items) {
        if($category == 'History') {
            unset($fields[$category]);
        }
      }  
}

Write your own custom module and put this code in it:

/**
* Implementation of hook_profile_alter
*/
function custom_profile_alter(&$account) {
unset($account->content['summary']['#title']);
unset($account->content['summary']['member_for']);
}

http://drupal.org/project/remove_history

I accomplished it by doing this:

1. Move the file user-profile-category.tpl.php from modules/user to your sites theme folder.
2. Edit the file that is now in your theme folder and comment out everything using the <!-- --> html tags.
3. Clear your cache.

If there is a better way for doing this in D7 please let me know.

in D7 history is a field on the user entity

goto admin/config/people/accounts/display set the format of the history field to 'hidden'

If you want to just hide the header ("History"), this will do:

.profile h3 {
  display: none;
}

Of course, make sure that you don't have any other h3's in there.

In Drupal 7 it can also be removed through code using hook_user_view_alter (in a similar fashion as the hook_profile_alter implementations that are explained above).

In a custom module add:

/**
 * Implements hook_user_view_alter().
 */
function YOURCUSTOMMODULE_user_view_alter(&$build) {
  unset($build['summary']);
}

I tried to hide the History section using Features (exporting the User view mode) but that doesnt work.

There is no need to manipulate the Drupal core code or to add anothe code. Just go to following path of your site and set the History field's dispaly setting from "Visible" to "Hidden". That's it!

/admin/config/people/accounts/display