l10n_server testing fails due to potx not being there/not enabled

If you look at this simple CSS patch testing for example, it should obviously pass all tests (we have no tests to check for CSS stuff): http://qa.drupal.org/pifr/test/56648 However, it fails in all kinds of funny ways due to potx module not being there (undefined functions, certain tests files not found, etc). Our setup code starts up with the following:

  public function setUp() {    // Set up required modules for l10n_community.
    parent::setUp('locale', 'potx', 'l10n_server', 'l10n_community', 'l10n_drupal');

However, the potx module is only a requirement for the l10n_drupal submodule by now, so l10n_community (the module being tested) does not have it as a dependency. Generally the issue is that the test needs to be under one module, but the modules being tested do not have a requirements relationship by their nature. So I assume the testbot will not set up all requirements in advance. How can we solve this?

Currently, contrib patch testing is pretty useless for us, since all tests will fail.