Is there a way to add a prefix to a field item that's in a multi-group so that it is added when a new instance of the group is created?

Lets say I have fields 1 ,2 and 3 in a multigroup and would like a heading to show up before field 1 and field 4. It would look like this:

Heading 1
Field Label 1
Field 1 input

Field Label 2
Field 2 input

Heading 2
Field Label 3
Field 3 input

I am able to use hook_form_alter in my custom module to add a prefix for specific items, but that doesn't apply when another item is added. I couldn't find a good example to after_build to follow. Any suggestions would be much appreciated.