Contrib projects to be included in beta stage of automated testing for modules

autoassignrole - has tests

Edit by Dave: confirmed testing enabled

XML sitemap - 100% test suite pass (and technically is already enabled for testing on drupal.org)
Admin role - 100% test suite pass
Poormanscron - 100% test suite pass
Password change confirm - 100% test suite pass
Examples - would be great to make sure they're current with development

Edit by Dave: confirmed testing enabled

Contact - backport/experimental replacement for core contact.module

Edit by Dave: confirmed testing enabled

http://drupal.org/project/admin_menu - has tests
http://drupal.org/project/mollom - has extensive tests

Edit by Dave: confirmed testing enabled

Privatemsg - has tests (Drupal 6 and 7)

Edit by Dave: confirmed testing enabled

Content Access has tests
Rules has tests (many more in d7)
Entity API has tests (has two different modules to test, does that work?)
Faces API has tests
Profile2 has tests

Edit by Dave: confirmed testing enabled

Can we add Mollom, please? We have many tests for both D6 and D7.

http://drupal.org/project/image - has tests

Edit by Dave: confirmed testing enabled

http://drupal.org/project/weather - has tests

Edit by Dave: confirmed testing enabled

http://drupal.org/project/linkchecker also have tests and may show many coder bugs. I have for sure found many false alarms in past and they are not fixed as I know.

Edit by Dave: confirmed testing enabled

http://drupal.org/project/services - has no tests but we're getting ready to start rolling a stable release and this would prove really helpful with handling bug fix patches.

Edit by Dave: confirmed testing enabled

http://drupal.org/project/encrypt - has tests
http://drupal.org/project/proxy - has tests (though failing at the moment)

Edit by Dave: confirmed testing enabled

http://drupal.org/project/media -- still in development; has tests

Not sure how many you want here, but
http://drupal.org/project/porterstemmer has tests
http://drupal.org/project/search_by_page has tests, and several of them depend on other contrib modules

http://drupal.org/project/potx - has extensive tests
http://drupal.org/project/l10n_server - has extensive tests

talk - has tests (DRUPAL-6--1 branch)

captcha - has tests
geshifilter - has tests

http://drupal.org/project/nd - has tests

http://drupal.org/project/storm - has 1 basic test, with more to come soon.

http://drupal.org/project/languageicons - No tests (yet), but I certainly wouldn't mind being included in this beta. =)

The issue description has been updated, and seems to imply the contrib testing bot is now live. Do existing patches that were already set to 'needs review' need prodding in some way for the bot to notice them? -- they still say 'ignored'. Eg: http://drupal.org/node/412288

For Proxy (listed above as being enabled), I am not able to get the testing to start for patches and it still says ignored; for example:
http://drupal.org/node/699180#comment-2548690

Also, will there be an easy way to find the branch testing, like a link on the project page?

Thanks for the great work on this. It is very awesome!

Are there any plans to add more contrib projects to the beta stage now that the testbot is back up and running?

Thanks.

@zzolo You have to make a commit to the desired branch to get it 'enabled' for testing.

http://drupal.org/project/og_statistics - has tests for every feature.

Cweagans and I are going to have several sprints on blogapi module. I'm not the module mantainer, however cweagans asked me to do this request because he was not going to be able to do for the next days as commented in the issue:#686832: Split connection methods into abstracted backends

So, our interest is to have the blogapi module included in the contrib testbot.

Thanks!

Please add role_change_notify to the test, as it now has tests.

openlayers 2.x branch now has a number of tests.

Pathauto's HEAD finally also has some proper tests, so it would be good to have that included in this as well. :)

Pathauto HEAD and 6.x* branches are already included - http://qa.drupal.org/pifr/test/31923

comment_notify - has tests that pass on 6.x branch. The 7.x tests for comment notify are currently broken.

API module's tests will cleanly pass when #735724: Remove globalToString() echo is resolved. However, I'm guessing an automated 6.x checkout of Grammar Parser will fail due to the "Looking for 6.x support?" note at http://drupal.org/project/grammar_parser.

Clock - has tests for 6 and 7.

Context - has tests

Ah, sorry can you remove the context project, we depend on a lot of contrib modules, and as simpletest gets the wrong versions a lot of our tests fail when they really shouldn't. Apologies for time wasted.

Can you please also add ecommerce http://drupal.org/project/ecommerce

securepages_prevent_hijack - has tests

Millennium Integration ... 6.x-2.x-dev version only. See http://drupal.org/cvs?commit=356414

Thanks to you and Kieran at the QA sprint, I now have a meager handful of tests for my module. More to come!

Better Exposed Filters - has tests

Question: So... by just marking an issue as "needs review" in my issue queue, that enough to get tested? Nothing looks different when I do (see http://drupal.org/node/468388#comment-2874104 )

**EDIT** nevermind, found it =) [Thanks @berdir]

Please add Google Analytics, too.

Could someone give me a small hint, please - why the patch in http://drupal.org/node/249701#comment-2979452 is "ignored" and not tested against 6.x-2.x?

l10n_server which is included in the beta stage has issues with how the testing framework is set up vs. the tests being run: #802034: l10n_server testing fails due to potx not being there/not enabled

Version Control API - has tests
Vote up/down - has tests, few and a little broken but including it will let me fix it quicker ;-)

RE #48: I think the project issue testing module might have a bug left over from the past where it used to ignore patches with a -D6 suffix. You might want to file an issue on the Project Issue Testing project.

subscribe

Rules Transformers has some few tests, but I'll add more, as the project advances.

Libraries API now comes with tests! D7 version only, for now.

Multicron has tests.

smartcrop - has tests

http://drupal.org/project/edge comes with tests, too

http://drupal.org/project/translation could use the testbot, too :)

Hi, http://drupal.org/project/user_delete (an attempt to backport D7) account cancellation module is including tests now.

Thanks!

http://drupal.org/project/piwik has tests.

http://drupal.org/project/embeddable has tests.

http://drupal.org/project/skinr has tests. :)

Boombatower, I kindly ask you to include http://drupal.org/project/rtsg in the testing framework :)

Modules that would love to be integrated in the next round..

http://drupal.org/project/userpoints
http://drupal.org/project/userpoints_nc
eventually http://drupal.org/project/userpoints_contrib can be also included (due to the D7 upgrade on the way, but not sure if d6 has any testcase)

Chessboard Renderer - has tests
88 passes, 0 fails, and 0 exceptions

Please add css_injector.

uuid has tests now. Thanks!

Nodequeue has tests: http://drupal.org/node/468136#comment-3816526 - Thanks!

Properties - has tests

Thanks :)

This is really great! Thanks for all the hard work on this. It really adds lots of value to developing on d.o. I am curious if there is a discussion, or what the status is, on getting automated Coder reviews for the contrib space as well (as I know it is available for core)?

Thanks @boombatower for all the great work. If there's some way I can help, please let me know.

Not sure if you're still accepting submissions here, but Date Popup Authored has tests.

For some time, I'm having serious trouble with the testbot in two contributed projects:

1. Libraries API: Testbot throws an "Detect invalid PHP syntax" for the HEAD branch, but the review log details state a very weird error message:
   

   [21:12:08] Invoking operation [syntax]...
   [21:12:08] No such module [example_info_file.libraries].
   [21:12:08] Checked [17 of 17] relevant file(s) for syntax.
   [21:12:08] Encountered error on [syntax], details:
   array (
     '@reason' => 'No such module [example_info_file.libraries]',
   )
   

   The XML-RPC output also lists that example_info_file.libraries module, which does not really exist (an .info file with that name exists for testing purposes, but no module).

   Tests are passing successfully when being run locally.

2. Skinr: Testbot passes on HEAD, but a current patch against HEAD does not pass, and after adding extra debug() output to the patch, the test results show a weird list of enabled modules -- neither the Skinr module itself nor the required Node module does not seem to be installed in the test environment.

   Here, also, tests are passing successfully when being run locally.

   Not sure whether this is related, but the XML-RPC output for that test shows that the testbot added a bogus additional dependency on the Skinr DRUPAL-7--1 branch (which is empty though).

I'm sorry for posting this in this issue, but I have absolutely no idea what the proper place for reporting such issues is. In particular, it's very hard to figure out whether this is an issue with SimpleTest, PIFR, PIFT, or some other component of the testing framework. In order to go out of beta with contributed module testing, we definitely need a single dedicated project issue queue to handle such issues.

I have a very similiar issue with Userpoints. Sometimes, the testbot runs "Rules Scheduler" tests additionally to our own, producing unrelated test failures. See #945622-2: 'Reason' column: Description gets replaced by truncated operation when you edit point transactions created by other modules for an example.

Can you please add

- Reference (tests on D7 only)
- Taxonews (tests on both D6 and D7)
- G2 (tests on D5 and D7, not D6)

better title.

And now with 100% less repetitiveness and redundancy.

http://drupal.org/project/fontyourface - has tests
http://drupal.org/project/myspace_sync - has tests

field-collection - has tests.

In the time you read this comment I'll be submitted the testcase for the Mailhandler project.

Thanks in advance!

Entity cache - has a test stolen from core.

Enabled Modules (http://drupal.org/project/enabled_modules).

Tests to be added soon.

http://drupal.org/project/commerce - has hella-tests, still adding more... poor pcambra's lappy takes an hour and a half to run 'em all.

http://drupal.org/project/cck_signup has tests (in 7.x only).

http://drupal.org/project/ubercart - has about 1000 tests on each of the D6 and D7 branches, and more coming soon. Thanks!

Are the tests being run taken directly from the repository, from the latest development snapshot, or from the latest tagged release of the contrib project?

@TR Tests are run based on the version specified in the individual issues. It also runs 'branch' tests when you commit code to a branch, but those results are not displayed on d.org. They're only used to determine if it can continue to run tests on issues or if it needs to postpone tests.

If I'm not mistaken, it's not working if you have dependencies, right? i.e. Commerce wouldn't work, so I'm guessing Ubercart won't either?

Yes, if your module has dependencies on other projects, testing does not currently work very well. You can work around it to use testing as a 'does this patch apply or not', but not currently as a reliable testing framework.

I do hope we'll be able to get loading dependencies to work in the next month or two (in the near term). Fixing the criticals on PIFR is my biggest community priority. It's just that there are a lot of them. And boombatower already made this work (I think) in the mainline (master), but getting that deployed is still a pain.

BTW: I filed #1093886: Allow 'pift enable project testing' permission for project owners/enabled users to hopefully let project owners manage this themselves.

Title has a couple of tests and we expect to have more in the near future ;)

restws has tests.

User relationships has some tests, more to come (at least in D7)

drupal.org/project/views please

kenorb: Thank you for impacting a busy development day by having this enabled on my module without my knowledge or permission.

In case you didn't realize, PIFR will automatically mark issues 'needs work' if the patch causes tests to fail. That means if the tests aren't perfect, perfectly valid patches get thrown into the 'needs work' graveyard and nobody knows about it. Thanks to you, I'm now stuck trying to figure out what pifr just screwed up for me.

For the love of all that is holy if this framework is enabled on a module, make ABSOLUTELY SURE the primary maintainer of the module consents, because if things aren't set up right, it is destructive.

Not to mention the test bot won't really work for Views since it has a dependency on Ctools. :-/

I think that this issue is thought to be used by maintainers to request automated testing on projects they maintain.
kenorb should not have asked to enable them for Views, as he doesn't even have write access to Views repository.

/me was already long ago adequately lashed and repentant :-)

@merlinofchaos: Sorry, if that cause any problem. I could help with test fails, I'm currently working on some of them.
I found only one issue with Views test
#1101332: Call to a member function set_display() on a non-object in tests/views_exposed_form.test on line 34
which I can solve.

Secondly enabling modules for test bot should be part of project configuration, not by posting module names in the issue. Especially when people don't know exactly how this test bot works, like me.

The CAS module has test routines. Please enable testing (for D7).

I have a project used as a practice space for making and reviewing patches. Would like to enable testing so that participants can also experience what that's like.

Current content is the module. Already has test routines. Please enable D7 testing.

Can you please enable MongoDB ? It already has some tests in some branches.

Please enable testing for CAS Attributes.

Thanks!

I'm looking forward to being able to enable testing myself as a module maintainer in the future!

CAS Attributes enabled

Media Feeds - has tests

Media Fields enabled.

I'm not sure that simpletest can handle modules with dependencies which I imagine Media Feeds has.

Thanks. Yes, it doesn't work. rfay pointed me to #102102: Parse project .info files: present module list and dependency information in IRC.

May I get Relation? We do not have dependencies. We work with everyone but we do not depend on them.

Taxonomy access has the beginnings of tests in the 7.x branch, and they'll be backported to 6.x eventually.

Relation and taxonomy_access are turned on.

taxonomy menu now has tests.

Taxonomy Menu enabled.

Please add Classified Ads, which already has D6 tests which will be ported to D7.

Classified Ads enabled.
The D7 version has lots of files[] in the .info which are not necessary (they're only needed for files that contain classes).

Can you please add http://drupal.org/project/ip2country ?

This project has 200 tests that run green for D6, D7, and D8 (!!!).

ip2country enabled.

Access by Term has tests...
Thanks...

Access by term enabled.

Please enable feeds_namespaces. It has dependencies, but the unit tests should work regardless.

feeds_namespaces enabled.

Please enable http://drupal.org/project/commentcloser. It has no dependencies, and the unit tests should work 100%.

done

Please enable http://drupal.org/project/birthdays. Has tests.

Birthdays added

Is there still a problem with modules that have dependencies, since #1241704: Deploy Project Dependency, Project and PIFT on d.o is marked as fixed?

If so are there any extra steps for modules that have dependencies before joining automated tests?
Thanks

@rootatwc if you need anything enabled for testing, just say the word (until maintainers can do it themselves)

Great!
thanks boombatower for the fast response:)

@rfray http://drupal.org/project/newsletter has some tests and i am currently writing more.

Entity translation has tests.

enabled entity translation

civicrm_activity_ical - has tests

civicrm_activity_ical enabled

The mongodb tests always fail because there is no /visible/ MongoDB server to which they can connect. I wonder if it really makes sense to have them run in automated mode at all except for making sure the patches are well-formed.

Now, is there not a MongoDB server in the d.o. infra somewhere, which could be accessed by these tests ?

Nothing I know of. You might want to add a condition to your test to skip tests if it can't connect to a server.

Hooray! We no longer have to do this. Every project owner now has the capability to turn on testing. You do it by editing the project, issues tab "Enable automated testing".

See http://jthorson.doesdrupal.com/node/34 for more information.

Also please note that issues on the testbots and testing can always be filed in the Testbot queue.

I couldn't find documentation for this functionality in the contrib module maintainers guide, so I added http://drupal.org/node/1449736.

@smokris++

Thanks so much.