Changing the default language on a Drupal site can cause a variety of issues. Most importantly it changes the assumption in Drupal about all the content and configuration that does not have a language explicitly assigned. Such as views, the site name, contact form configuration, etc. Most people who want to change their default language only want to do so to change how the language negotiation works, and they have no intention to translate / swap all configuration/content that does not have language assigned to the new language.
Many of us observed repeated issues from people who just wanted to change the language negotiation configuration by changing the default site language and broke their translation processes due to the changed assumptions about their language. This is a very common problem. There is a contributed module written to work around this problem by introducing another configurable language negotiation piece: http://drupal.org/project/fallback_language_negotation. It is not a very well known module but solves the problem nicely, allowing the site to fallback to a language in the negotiation process without actually changing the default language.
Change the current default language provider and instead of tying it to one specific language, allow an administrator to select either the site's default language or any other active site language.
BEFORE (configure link on the default language row went to the page to *actually* change the site default language):
AFTER part 1: negotiation page
AFTER part 2: configuration page for selected language
Now the configure link in the new selected language row, see part 1 image above, goes here and a language can be selected without changing the actual site default language
#1803638: Improve default language change process (ui and help text) This is a follow up and not handled in this issue because: (from comment #50) there are some things (like site name, maybe others like stuff to do with views or contact forms) that right now, dont have language selectors/cannot be translated. It would be good to try and get some improvement in those areas... before feature freeze. But, more to the point: since it's not known how much progress we can get in that and other areas of translation, it's not useful to describe exactly (in the help text) the current state since hopefully that will change between now and feature freeze. And thus, that should be a clean up task.