variable_config ads substantial performance hit

See screenshot from newrelic, which is the breakdown of a node view page (handled by page manager). variable_config adds an extra 5% onto the time. I think the problem is caused by the unconditional call to variable_config_fetch_all_variables() in entity_load(). variable_config_fetch_all_variables results in a query of all field and field instance data.

I think variable_config may need to store its own cache detailing which fields have config data and use that as a more direct method of determining which field instance data needs to be loaded rather than querying all drupal field instances to find this out.

Or, at least, the return from variable_config_fetch_all_variables should be cached in drupal cache tables.