The main menu does not disappear when the option on theme's Settings page is unchecked, even after clearing the cache. The only way I was able to get it to go away was to change this line in page.tpl.php from:
if ($mainmenu && !empty($main_menu)):
Obviously deleting the code would have worked also, but I use this same template file on other sites that DO use the main menu, so that solution wasn't desirable...
This problem occurred on 7.x-3.0-Beta11