Currently, the only way to register libraries on the theme level is to use hook_libraries_info_alter(). This is problematic for two reasons (in case you want to register libraries in a theme) drupal_alter() highly depends on when it is invoked in the lifecycle of a Drupal request. If the theme layer is not initialized at the time when libraries_info() is invoked this causese the theme alter hooks to be ignored. Also, if the theme changes (e.g. because we are in the admin theme, or because we are using a theme key altering module [e.g. a context module plugin that changes the theme key based on certain conditions]) we may have invalid data (libraries) in the cache that should not be there. The same thing may happen the other way round => libraries_info() is invoked on the backend => cache is populated without invoking hook_libraries_info_alter() for the frontend theme (the frontend theme would normally register a library) => Next time we read the info from the cache on the frontend, the library that the frontend theme is trying to load is not there because it hasn't been loaded into the cache.
Therefore, I suggest that we do per-theme-key caching of libraries and directly invoke hook_libraries_info() for all themes in the theme trail of the theme that is currently active. I can provide a patch for that if you guys agree.