Is ist possible to add custom links to the contextual links to the panels page?

try module 'Custom Contextual Links'

Hm. At the moment there isn't currently a way to inject contextual links into a panel. Probably the best way to do this would be to add something via hook_ctools_render_alter() that can be added to the $info file and that could then be added to the links in the build process which happens just after this alter in ctools_context_handler_render_handler(). The patch to do this is probably not terribly difficult to write.

It probably looks something like this:

  if (!empty($info['contextual_links'])) {
    $links += $info['contextual_links'];
  }

And a little extra code to ensure that $links is actually an array in case there aren't actually provided by the task handler (which in practice is rare).

This is totally a hack, but after several failed approaches at altering the contextual links on the page, I ended up finding that hook_page_alter() (yuk!) solves this problem. In my situation, I wanted to move the content of the tabs (local tasks) to the contextual links, as shown.

Even though I accomplished this with the following code, it'd still be nice to have a more direct way of handling this.

My module was called "vincia_custom":

/**
 * Implements hook_page_alter().
 *
 * Move links in tabs to contextual links.
 */
function vincia_custom_page_alter(&$page) {
  $page['#pre_render'][]  = 'vincia_custom_page_prerender';
}

/**
 * Prerender callback for the entire page.
 *
 * Move links in tabs to contextual links.
 */
function vincia_custom_page_prerender($page) {
  if (!path_is_admin($_GET['q']) && isset($page['content']['system_main']['contextual_links'])) {
    if ($tabs = menu_primary_local_tasks()) {
      // Remove the first tab, assumed to be the "View" link.
      unset($tabs[0]);

      $links = array();
      foreach ($tabs as $tab) {
        $links[] = $tab['#link'];
      }
      $links = array_merge($links, $page['content']['system_main']['contextual_links']['#links']);
      $page['content']['system_main']['contextual_links']['#links'] = $links;
    }
  }
  return $page;
}

After that, just remove the $tabs from being printed in page.tpl.php, and you've got a replacement for tabs on the front-end.

You might want to check how this is done in Open Atrium they made https://drupal.org/project/contextual_tabs.

There is also the following patches that provide this ability when using the Panels Page Manager:
#1400884-14: Custom menu items in pane dropdown links
#2108413: Add edit link for custom content panes
#1948278: Get an edit link for the view panes in Page Manger

Hope that helps

Drupal 7 is no longer supported, closing.