I added a new content type but did not want some of the fieldgroups to be shown as vertical tabs so I unchecked "Include new fieldsets in vertical tabs by default." This had the unexpected result of turning off vertical tabs for all of my custom fieldgroups in all content types.

The problem is that this checkbox does not set the form display style for the fieldgroup but rather overrides it, so if you're lazy and rely on the checkbox to convert new fieldgroups to vertical tabs and later decide to change the behavior, you're in for a surprise.

If this can't be changed to actually set the new fieldgroup's display to vertical tabs, then I'd suggest ditching this feature.