Drupal Commons overall Performance?

I've noticed my site taking extremely long times to load. I even switched servers to speed it up. Some testing reveals group pages with only a few users and maybe 50 postings taking 9.5 seconds (wait time) for initial load and topping out at 13.5 seconds all complete. That's some time...to wait.

I thought I must have a module messed up, but I"m not so sure about that now. All other Drupal sites on this server are flying. So, I took Firebug's NET testing to your "http://commons.acquia.com/groups/commons-feedback" sample site.

I'm seeing 1.5ish second wait times there on initial load - AND I assume you guys are running some nice PHP and Mysql caching.

So what I'm asking, is there a bit of a performance drag with all these modules and for commons in general (that you may be working on), or are your happy, in general, with it's performance comparatively? IE Is it ALL my problem, or did I inherit a bit of a problem?

And if there are performance issues (and I understand a large amount of modules brings it's own overhead issues)...what's the biggest things (most time saving) I can do to mitigate them (on a shared server without PHP and Mysql caching)?

For instance, Number 1 - kill the Shoutbox. 2 - kill the recent members block. Etc.

Thanks.