[TESTING BROKEN] FileFieldValidateTestCase fails randomly

Posted by int on December 24, 2009 at 11:17am

10 followers

Issue Summary

The automatic test FileFieldValidateTestCase fail random..

In some patch that don't touch in FileField, or only in javascript, that nothing related with filefield have to be re-tested because the simpletest fail in FileFieldValidateTestCase

Error message:

FileFieldValidateTestCase	101	1	7
Message	Group	Filename	Line	Function	Status
array_flip(): Can only flip STRING and INTEGER values!	Warning	entity.inc	108	DrupalDefaultEntityController->load()	
Trying to get property of non-object	Notice	file.test	374	FileFieldValidateTestCase->testRequired()	
Undefined property: stdClass::$uri	Notice	file.test	146	FileFieldTestCase->assertFileExists()	
File exists after uploading to the required multiple value field.	Other	file.test	375	FileFieldValidateTestCase->testRequired()	
Undefined property: stdClass::$fid	Notice	file.test	154	FileFieldTestCase->assertFileEntryExists()	
array_flip(): Can only flip STRING and INTEGER values!	Warning	entity.inc	108	DrupalDefaultEntityController->load()	
Trying to get property of non-object	Notice	file.test	156	FileFieldTestCase->assertFileEntryExists()	
Undefined property: stdClass::$uri	Notice	file.test	156	FileFieldTestCase->assertFileEntryExists()	

Comments

#1

Posted by int on December 24, 2009 at 5:44pm

Like:
http://qa.drupal.org/pifr/test/22976
http://qa.drupal.org/pifr/test/23248
http://qa.drupal.org/pifr/test/23258
http://qa.drupal.org/pifr/test/23282
http://qa.drupal.org/pifr/test/23286
http://drupal.org/node/582622#comment-2405160

Two testing servers and one give OK in testing HEAD, and other no.
http://qa.drupal.org/pifr/test/32
http://qa.drupal.org/pifr/test/33

#2

Posted by int on December 24, 2009 at 11:24am

Title:

The automatic test FileFieldValidateTestCase fail random

» The automatic test FileFieldValidateTestCase fail randomly

#3

Posted by int on December 24, 2009 at 11:31am

I don't know if this is true, but automatic test should never be created with random content. The automatic test should be reproduced, exacly the same variables.

e.g. Instead be one variable with random content, should be 3 variables with static content, with that 3 variables with almost possible text.

#4

Posted by Dave Reid on December 24, 2009 at 8:26pm

Priority:

normal

» critical

Test bot has been disabled until we can figure this out.
http://qa.drupal.org/node/68

#5

Posted by beejeebus on December 24, 2009 at 9:44pm

subscribe. nasty stuff.

#6

Posted by sun on December 24, 2009 at 10:21pm

Title:	The automatic test FileFieldValidateTestCase fail randomly	» [TESTING BROKEN] FileFieldValidateTestCase fails randomly
Component:	field system	» file.module

Also investigated this today... I ran this test multiple times on my local machine (Windows), always passed 100%. AFAICS, the test does not seem to do anything scary.

What I didn't test is running tests in concurrency. Did anyone try this?

#7

Posted by arhak on December 24, 2009 at 10:38pm

also http://qa.drupal.org/pifr/test/23068

and a different failure is at http://qa.drupal.org/pifr/test/23044 (my own patch not related with FileField at all)
this case HAD (because now it passed) 2 fails and 14 exceptions, which might help to alight the issue
- 1 fail was "File exists after uploading to the required multiple value field." with 7 exception around like the other tests failing
- the other fail was similar but for for the single value field (also with 7 exceptions around it, the same exception's messages it got for the multiple value field)

#8

Posted by Damien Tournoud on December 25, 2009 at 5:19pm

The probable culprit is FileFieldRevisionTestCase::testRevisions(). Files are shared between the test instances, and this test actually physically removes files.

The only correct fix is to copy all the test files to the new files directory in DrupalWebTestCase::setUp().

#9

Posted by Damien Tournoud on December 25, 2009 at 5:38pm

Status:

active

» needs review

If I'm right, this should fix it.

Attachment	Size	Status	Test result	Operations
667264-random-file-related-failures.patch	5.89 KB	Idle	Unable to apply patch 667264-random-file-related-failures.patch	View details

#10

Posted by beejeebus on December 25, 2009 at 5:59pm

Status:

needs review

» active

from #drupal:

(16:56:04) beejeebus: anyone got any clues about reproducing that?
(17:17:07) DamZ: beejeebus: I think I know what's going on
(17:17:33) DamZ: beejeebus: it's probable that FileFieldRevisionTestCase::testRevisions() is running at the same time as the failing one
(17:18:07) DamZ: beejeebus: the files are shared, and FileFieldRevisionTestCase::testRevisions() is removing the file needed by FileFieldValidateTestCase
(17:55:13) beejeebus: DamZ: oh, its a concurrency issue?
(17:56:10) DamZ: beejeebus: yeah, of course
(17:56:58) beejeebus: DamZ: oh, that should be easy to fix then

#11

Posted by Damien Tournoud on December 25, 2009 at 6:27pm

Status:

active

» needs review

Why changing the status?

#12

Posted by cburschka on December 26, 2009 at 12:24am

This is fairly ironic - unpatched filefield works fine for me, but the patched version runs into the similar errors as the test server.

Note the memory size failure claiming a memory limit of 64M, when the memory limit is actually set to 128M. This is unusual...

File field validation tests 92 passes, 10 fails, and 53 exceptions

Fail       PHP Fatal  database.inc                   1956                      
    Allowed memory size of 67108864 bytes exhausted (tried to allocate 48 bytes)
--
Exception  Notice     file.test                      29                        
    Trying to get property of non-object
Exception  Notice     file.test                      29                        
    Trying to get property of non-object
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      410                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      411                       
    File exists after uploading a file ( bytes) under the max limit (1M).
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      417                       
    Node save failed when file ( bytes) exceeded the max upload size (1M).
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      410                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      411                       
    File exists after uploading a file ( bytes) under the max limit (1024K).
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      417                       
    Node save failed when file ( bytes) exceeded the max upload size (1024K).
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      410                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      411                       
    File exists after uploading a file ( bytes) under the max limit (1048576).
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      417                       
    Node save failed when file ( bytes) exceeded the max upload size (1048576).
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      426                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      427                       
    File exists after uploading a file ( bytes) with no max limit.
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      29                        
    Trying to get property of non-object
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      453                       
    Undefined index:  und
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      454                       
    File exists after uploading a file with no extension checking.
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      463                       
    Node save failed when file uploaded with the wrong extension.
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      471                       
    Undefined index:  und
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      472                       
    File exists after uploading a file with extension checking.
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri

#13

Posted by Damien Tournoud on December 26, 2009 at 1:26am

Nice catch! I forgot to reset the copiedTestFiles indicator, so when several tests of the same class are run, files are only copied the first time.

This actually proves that the random errors in the test bot are actually due to a missing file.

Attachment	Size	Status	Test result	Operations
667264-random-file-related-failures.patch	6.31 KB	Idle	Unable to apply patch 667264-random-file-related-failures_0.patch	View details

#14

Posted by cburschka on December 26, 2009 at 2:08am

I can run the FileFieldValidateTestCase class with 102 passes and 0 fails, but I am not convinced the test completes, because there are three occurrences of

Fail       PHP Fatal  database.inc                   1956                      
    Allowed memory size of 67108864 bytes exhausted (tried to allocate 48 bytes)

Oddly, these failures do not get counted. Multiple different checks show that PHP's memory limit is set to 256M (and in fact any deliberately induced crashes elsewhere show a limit of 256MB was exceeded). This is weird.

#15

Posted by cburschka on December 26, 2009 at 11:59am

Okay, so here's what happens.

I have this file in the folder for whatever reason:

Array
(
    [sites/clean/files/simpletest/mpletest734835/error.log] => stdClass Object
        (
            [uri] => sites/clean/files/simpletest/mpletest734835/error.log
            [filename] => error.log
            [name] => error
        )

It contains this:

[23-Dec-2009 10:50:49] PHP Fatal error:  Allowed memory size of 67108864 bytes exhausted (tried to allocate 48 bytes) in /home/arancaytar/Documents/devel/drupal/core/CLEAN/includes/database/database.inc on line 1956

I've cleaned out and reinstalled my site now and hope that the error goes away.

Edit:

All is well. 102 passes, no failures, no exceptions, and the phantom fails are gone.

#16

Posted by bleen18 on December 26, 2009 at 5:54pm

superscribe

#17

Posted by cburschka on December 26, 2009 at 11:08pm

Note that this is no longer the only test case that fails right now: #654854: Clean up conditions in comment_reply(), avoid node_view() when unnecessary.

#18

Posted by bleen18 on December 26, 2009 at 11:11pm

@Arancaytar

#658314: $page variable in node.tpl.php is buggy

#19

Posted by Damien Tournoud on December 27, 2009 at 2:56pm

In fact, there is no reason to pregenerate the files. The generation is fairly quick and those files are only used in a couple of tests. So I suggest we simply remove the pregeneration and generate the test files on-demand, the first time DrupalWebTestCase::drupalGetTestFiles() is called.

This should be more robust, and would avoid Arancaytar's issue where a phantom error.log was copied over and over to the new test environment.

Attachment	Size	Status	Test result	Operations
667264-random-file-related-failures.patch	7.03 KB	Idle	Unable to apply patch 667264-random-file-related-failures_1.patch	View details

#20

Posted by int on December 28, 2009 at 11:49am

We should commit the #19 path, or remove the breaking test. The testbot can't be so long stopped.

#21

Posted by Dries on December 28, 2009 at 12:08pm

Status:

needs review

» fixed

Committed to CVS HEAD. Bring back the testbots p-uh-lease! Thanks DamZ.

#22

Posted by Damien Tournoud on December 28, 2009 at 1:29pm

Status:

fixed

» needs review

This should fix the remaining test failures. Some tests were doing unholy things with the test files.

Attachment	Size	Status	Test result	Operations
667264-fix-remaining-failures.patch	2.58 KB	Idle	Unable to apply patch 667264-fix-remaining-failures.patch	View details

#23

Posted by int on December 28, 2009 at 1:30pm

Status:

needs review

» needs work

http://qa.drupal.org/pifr/test/32
Fail:
FileScanDirectoryTest (Found two, expected javascript files.)
ImageEffectsUnitTest (Division by zero)
ImageToolkitUnitTest (Division by zero)

#24

Posted by int on December 28, 2009 at 1:30pm

Status:

needs work

» needs review

cross post

#25

Posted by cburschka on December 28, 2009 at 1:31pm

Cross-post.

#26

Posted by Damien Tournoud on December 28, 2009 at 1:57pm

I ran the tests manually on qa-1 (my test slave). 100% pass.

#27

Posted by Dries on December 28, 2009 at 2:00pm

Status:

needs review

» fixed

Committed the patch in #22. Good thing we're making progress. Thanks @DamZ. :)

#28

Posted by int on December 28, 2009 at 11:45pm

Fixed issues should not be tested. should be automaticly removed from the queue.

#29

Posted by bleen18 on December 28, 2009 at 8:41pm

Title:	[TESTING BROKEN] FileFieldValidateTestCase fails randomly	» [TESTING BROKEN] FileFieldValidateTestCase (and FieldInfoTestCase?) fails randomly
Status:	fixed	» active

I believe that this error is still occurring but now with "FieldInfoTestCase" ...

the patch in comment #3 of #658148: The "access user profiles" permission is poorly named is nothing more then a find/replace on the string "access user profiles" but it is failing inexplicably on "FieldInfoTestCase"

I'm not 100% certain, but I suspect that its related to this issue ... thoughts?

Title:	[TESTING BROKEN] FileFieldValidateTestCase (and FieldInfoTestCase?) fails randomly	» [TESTING BROKEN] FileFieldValidateTestCase fails randomly
Status:	active	» fixed

The error doesn't appear to be happening anymore. Even if it is, this would be an unrelated error because FieldInfoTestCase is not copying any files anywhere.

If a non-file-copying tests is failing erratically again, let's open a new issue.

#33

Posted by System Message on January 14, 2010 at 4:00pm

Status:

fixed

» closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Project:	Drupal core
Version:	7.x-dev
Component:	file.module
Category:	bug report
Priority:	critical
Assigned:	Unassigned
Status:	closed (fixed)

[TESTING BROKEN] FileFieldValidateTestCase fails randomly

Jump to:

Issue Summary

Comments

#1

#2

#3

#4

#5

#6

#7

#8

#9

#10

#11

#12

#13

#14

#15

#16

#17

#18

#19

#20

#21

#22

#23

#24

#25

#26

#27

#28

#29

#30

#31

#32

#33