CSS compressor conflicts with modules that require theme change on the fly

If you enable the CSS compressor then theme changing modules like taxonomy_theme (and perhaps sections but haven't verified) breaks. I think this is a core bug since the compressed css is carried from page to page whether the new theme is in effect or not.
One soultion could be, when enabled, CSS compressor module compresses all current css configured and then use the appropriate compressed file as required instead of keeping the same file throughout.