Disable tests for the module? [#1393776]

Comment #1

MiSc CreditAttribution: MiSc commented 12 January 2012 at 07:36

Priority:

Normal

» Critical

Change status to critical, because of all tests fails.

Log in or register to post comments

Comment #2

fgm

French

Paris, France

CreditAttribution: fgm commented 12 January 2012 at 09:45

Priority:

Critical

» Normal

Critical is for when a module causes a site fail, which is not the case, and major when the code itself is not working, which is not the case either, so downgrading.

Some tests should pass, as at least in one version the test setup checks whether a mongodb server is available and disables further checks when it is not, while some of the tests do not need a server and can thus pass even on qa.d.o.

I agree, however, that with so little tests being actually usable without a mongo instance, it might make more sense to disables automated testing on qa.d.o. until such a server becomes available on the infra. If other maintainers agree, I will disable testing.

Log in or register to post comments

Comment #3

MiSc CreditAttribution: MiSc commented 12 January 2012 at 09:56

Priority:

Normal

» Critical

I think it should be critical:

Critical bugs either render a system unusable (not being able to create content or upgrade between versions, blocks not displaying, and the like), or expose security vulnerabilities. These bugs are to be fixed immediately.

In Drupal 7 this also applies to test failures in the primary environment: we have a policy of a 100% pass rate for tests, so failing tests block all other development. The primary testing environment is currently MySQL on Linux. Tests failing in other environments should be marked as major.

I have not seen any tests for this module that works with the qa.

Log in or register to post comments

Comment #4

fgm

French

Paris, France

CreditAttribution: fgm commented 12 January 2012 at 10:03

By exactly the words you are quoting, this is not critical, since there is no way qa.d.o. can not be construed to be the "primary" environment for the mongodb driver, and disabling testing as you suggest is does not actually fix anything. How about submitting a patch to fix tests and/or code so they can behave properly, instead ?

Log in or register to post comments

Comment #5

MiSc CreditAttribution: MiSc commented 12 January 2012 at 10:17

But how would testing on qa.d.o. be relevant for this module? There are not going to be a mongodb instance there. Writing test that works that are not dependent on mongodb seems pointless for me for this module. I am not try to criticize here, just understand why you have choose to have qa-tests on this module. Surely I could learn to write tests and patch the tests here, but first I need to understand why.

The testing on qa.d.o. is on a mysql environment. And the test fails. And therefore it looks critical to me. I think it should not be tested in a mysql environment in the first place, because it is misleading.

Log in or register to post comments

Comment #6

fgm

French

Paris, France

CreditAttribution: fgm commented 12 January 2012 at 10:33

A few tests are actually signficant: for the watchdog component, for instance, one of the tests applies to the module itself, not its ability to log something into mongodb.

But beyond that, tests should not fail when a server is not available, but behave properly and handle the error, which they do not currently do well, so it is better IMHO to have failing tests, showing that there is indeed at least a problem with tests, and probably with the overally connection handling, than not having tests and make believe all is rosy in non optimal conditions.