Changing module weights can break CB settings form.

Because you are relying on form_altering to add settings, changing module weights in UI can disable module settings. For example, panels breadcrumbs for taxonomy relies on altering taxonomy form element of admin settings, and when weight of taxonomy_breadcrumbs > panels_breadcrumbs, form_alter won't work because you are changing sub-element of non-existing array.
If we really want to make it independant from weights, in each custom_breadcrumb submodule form_alter() should create separate form element instead of trying to change existing one.