I am just posting this incase somebody runs into this issue. I have solved it for myself which I will explain.

Problem

When editing an existing Panel page or creating a new one.

  1. No layouts would showed, the dropdown for "Category" under layout was empty. Unless I am enabled "Panopoly Theme" module then its layouts would show
  2. When I clicked "add content" for pane the options in lefthand section were very limited. For instance these didn't show up: exiting node, custom content, page elements, etc.

This was not happening for Mini Panels. I enabled the module just to check.

Cause

function panels_common_get_allowed_layout_object($module_name) {
  $allowed_layouts = unserialize(variable_get($module_name . "_allowed_layouts", serialize('')));

This variable was bringing back limited layout selection(only panopoly theme).

I didn't step debug the "add content" problem but it is seems it was the same issue.
May have been from here:

function panels_common_get_allowed_types($module, $contexts = array(), $has_content = FALSE, $default_defaults = array(), $default_allowed_types = array()) {
  // Get a list of all types that are available
  $default_types = variable_get($module . '_default', $default_defaults);
  $allowed_types = variable_get($module . '_allowed_types', $default_allowed_types);

So there seems to have been a problem when the variables where set and then panels kept getting these from variables.
Clearing the cache didn't help.

Work around

I updated to dev versions of both Panels and Ctools and it didn't help(maybe b/c variable was set with versions)
I was able to fix this by running 2 drush commands

drush vset panels_page_allowed_layouts 0
drush vset panels_page_allowed_types 0

This forced panels to reload variables and then it acted as expected. So I don't know why the vars where set with those values in the first place.
Also I tested this solution on a version of the site that didn't have the dev version of ctools and panels and it still worked.

UPDATE:
I tried to reproduce this on new site and couldn't
On the new site is never the above variables are never set?!

Comments

Panopoly ships limiting you to only its layouts. Were those layouts not available? If that is the case, that would be the problem.

It's not really Panels' fault if that variable gets set (even if it's by code) and then the environment changes but the variable does not change with it. There is really no way for Panels to know.

merlinofchaos, ok so should Panopoly Theme make sure it deletes that variable when it is disabled?

UPDATE: I think part of the problem is the Panopoly Theme module wasn't dependent on Panopoly Core(it is now) so I was using outside of the Panopoly distro. But even though the dependency wasn't there it wasn't really written to be used outside of the distro.