Looking at the following code from OG, it's clear that it is up to implementations of hook_og_ui_get_group_admin() to determine access to the menu items they are returning:

/**
* Implements hook_og_ui_get_group_admin()
*/
function og_ui_og_ui_get_group_admin($group_type, $gid) {
  $items = array();
  $default_access = og_is_group_default_access($group_type, $gid);
  if (og_user_access($group_type, $gid, 'add user')) {
    $items['add_people'] = array(
      'title' => t('Add people'),
      'description' => t('Add group members.'),
      // The final URL will be "group/$group_type/$gid/admin/people/add-user".
      // @see og_ui_group_admin_overview().
      'href' => 'admin/people/add-user',
    );
  }

This is not made clear in the documentation -- leading to implementation errors such as #2065317: user can see group admin item, but can't access the page.