Correct regex for singular assertions
| Project: | Project Issue File Review |
| Version: | 6.x-1.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | critical |
| Assigned: | boombatower |
| Status: | closed |
Justin Randell noticed this evening that the Menu item creation/deletion tests had a failure in them. So I spent a few minutes rolling back patches until it stopped failing. It turns out, this patch causes it.
The failure was:
"Menu item was reset" in menu.test line 317 (MenuTestCase->resetMenuItem())
Testing bot, however, did not catch this. :( I marked the patch back for retesting at :30, it passed at :57, and was committed at :18. The only thing committed in this ~20 minute window was http://drupal.org/node/67349#comment-1122071 which is a one-line change to SQLite, nowhere near the menu system.
We need to get to the bottom of this, because if we can't trust testing bot to come back with accurate results, it's not going to actually save core maintainers any time.
#1
Ok, my findings so far is that something is weird in the testing kingdom:
(1)
the very last test case is never reported (User blocks), because of a bug in the regexp in pifr_review_format_results()--> strike that: I was wrong(2) there are two different results for "Comment on a node page",
(3) "Menu item Creation / Deletion" is nowhere to be found
#2
Subscribing as it was my issue that somehow got a false ok from testing bot... :)
#3
Here's another one it wasn't catching: http://drupal.org/node/338239#comment-1127512. This is not good. :(
#4
Seems like the test failures are not reported in the testing results, so it looks like they all passed. The user/login patch didn't show the result for the menu item creation/update test.
#5
Yet another one: http://drupal.org/node/340052#comment-1131402 Gives the green light with only 105 passes. I'm inclined to say the whole test suite did not run. ;)
#6
Queried the results and found that the following 196 patches seem to be not reporting/running all the tests. All of which were run on slave 8 except: simpletest_tune.patch, 303660_HEAD_input_format_8.patch. Not sure if there is an issue with the slave or PIFR. The fact that slave 8 is reporting them all is almost uncanny.
Exact query for clarification:
SELECT filename FROM pifr_file WHERE pass + fail + exception < 6000 AND message = '' AND status > 3;+-----------------------------------------------------------+| javascript_unit_testing.patch |
| uploadjs.diff |
| error_class_with_empty_message.patch |
| headers_0.patch |
| theme_col.patch |
| menu_translate.patch |
| welcome_intro_change_0.patch |
| drupal_cron_cleanup.patch |
| node_types_rebuild.patch |
| node_revision_text.patch |
| install_default_settings_md5.patch |
| issue-doxygen-install.patch |
| custom_date_00.patch |
| system_perm_move.patch |
| dirindex.patch |
| taxonomy_autocomplete_1.patch |
| drupal.run-tests-php-windows.patch |
| simpletest_help.patch |
| node_access_10.patch |
| remove-ie-mac-css.patch |
| cron-timing-4.patch |
| language_0.patch |
| cron-timing-131536-26.patch |
| javascript_unit_testing_0.patch |
| cron-timing-131536.patch |
| language_244162.patch |
| cron-timing-131536-34.patch |
| 261284.patch |
| locale_270226.patch |
| 261284_0.patch |
| jquery_fix_D7.patch |
| update_block_cache_mode-235673-11.patch |
| locale_32.patch |
| list_0.patch |
| welcome_4_0.patch |
| 115540_1.patch |
| drupal_form_multistep_troubles.patch |
| install-notify-update.patch |
| 221712-browser-language_0.patch |
| 183678.patch |
| trigger-issue-241013_7.x.patch |
| issue-theme_comment.patch |
| issue-theme_comment_0.patch |
| cgi-headers-175855-D7.patch |
| content-length-header-297141-D7.patch |
| actions-synchronize-319404-D7_0.patch |
| custom_date_01.patch |
| issue-doxygen-install-2.patch |
| issue-297141.patch |
| issue-297141_0.patch |
| issue-297141_1.patch |
| tngdb-pdo_pgsql-blob-1223989019.patch |
| disabled_checkbox_tests.patch |
| issue-261284-15.patch |
| update.sh_.patch |
| mw_212.patch |
| 219584_0.patch |
| drupal-menu-submit-7.x_0.patch |
| admin_content_node_11.patch |
| table_select_empty.patch |
| drupal.update-themes_0.patch |
| show-tables.patch |
| access-callback-harden-D7-319360-7.patch |
| menu.module_7.x.mlid_.patch |
| drupal.run-tests-php-windows_0.patch |
| file_276280.patch |
| simpletest_tune.patch |
| mime_extension_mapping.d7_0.patch |
| batch_with_timer-1226226432.patch |
| drupal.status-report-export_2.patch |
| mime_extension_mapping.d7_1.patch |
| 332725-rip-locale-inc-callback_0.patch |
| node_types_rebuild_0.patch |
| preprocess-rename-file-to-template_file.d7_1.patch |
| time_limit_174617_7.x_0.patch |
| search_36.patch |
| mail.test-299138-13.patch |
| drupal_default_country_0.patch |
| js_filter_xss_3.patch |
| dbtng-getFieldTypeMap-1226499767.patch |
| 212130_D7_0.patch |
| settings_d6_d7_3.patch |
| dbtng-db_select_temporary-1226655467.patch |
| issue-57178-9.patch |
| issue-173037-8.patch |
| poll_choice_js-331708.patch |
| 247071_show_attachments_in_teaser_3.patch |
| system-action-watchdog-202032-4.patch |
| 247071_show_attachments_in_teaser_4.patch |
| 247071_show_attachments_in_teaser_5.patch |
| 220559-language-switcher-with-test.patch |
| 165786_aggregator_rename_description_column_to_body.patch |
| d6-body-class-array-04.patch |
| comment_approval_4.patch |
| drupal_js-302251.patch |
| cache_get_multiple_1_0.patch |
| 335185-remove-no-js-css-D7.patch |
| 335185-remove-no-js-css-D7_0.patch |
| dirindex_0.patch |
| batch_with_timer-1226896443.patch |
| 335237-modules-uninstalled-filter-D7.patch |
| 320303-modules-uninstalled-menu-D7.patch |
| parent_theme_pattern_clobbering.patch |
| multiple_load_18.patch |
| 303660_HEAD_input_format_8.patch |
| 335035-disabled-controls_0.patch |
| preprocess-rename-file-to-template_file-D7.patch |
| custom_url_rewrite_outbound-2.patch |
| issue-334950-6.patch |
| password_checker_fix_3.patch |
| password_checker_fix_4.patch |
| aggregator_pluggable_5.patch |
| 290246_edit_own_comments_permission_0.patch |
| 290246_edit_own_comments_permission.patch |
| multiple_load_21.patch |
| t-docs.patch |
| parent_theme_pattern_clobbering.b-head.patch |
| 247071_show_attachments_in_teaser_6.patch |
| drupal.status-report-export_3.patch |
| teaser_undefined_value.patch |
| multiple_load2.patch |
| multiple_load_19.patch |
| comment_approval_5.patch |
| dbtng-aggregator-1227258722.patch |
| cache_get_multiple_1_0_0.patch |
| taxonomy_count_nodes_HEAD_0.patch |
| taxonomy_count_nodes.patch |
| taxonomy_term_count_nodes.patch |
| drupal_nodeapi_signatures.patch |
| thai-vowel-7.x-dev.patch |
| 290246_edit_own_comments_permission_1.patch |
| 27234_help_text_picture_size.patch |
| 183678-3.patch |
| 183678-4.patch |
| drupal_310863_auto_locale.patch |
| drupal_310863_auto_locale_0.patch |
| book.revision-log.patch |
| simpletest_help_2.patch |
| file_30520_7.patch |
| 222855.patch |
| hook_url_rewrite_1.patch |
| file_276280_0.patch |
| 325827-date-default-timezone-set-D7.patch |
| subselect_from_1.patch |
| menu-access-callback-load-337907.patch |
| dbtng-PDO_CASE_LOWER-1227407521.patch |
| 335185-remove-no-js-css-D7_2.patch |
| parent_theme_pattern_clobbering.c-head.patch |
| HEAD_parent_theme_pattern_clobbering.c.patch |
| parent_theme_pattern_clobbering-c-HEAD.patch |
| theme-registry-fix-d7.patch |
| parent_theme_pattern_clobbering-e-HEAD.patch |
| recent_content_block_02.patch |
| grant-op-278675-1.patch |
| grant-op-278675-6x-2.patch |
| node_access_11.patch |
| node_278675.patch |
| multiple_load_28.patch |
| test-menu-callback-registry-337907-4.patch |
| d7-strict_1.patch |
| multiple_load_25.patch |
| taxonomy_term_count_nodes_1.patch |
| assertFieldByXPath-3.patch |
| file_create_url-16.patch |
| dbtng-block-1227763112.patch |
| blob-locales_source_hash-1227760668_0.patch |
| blob-menu_router-arguments-1227761227.patch |
| multiple_load_2_0.patch |
| file.inc__11.patch |
| mw_221.patch |
| drupal_goto-parse_url_1.patch |
| t-docs_0.patch |
| 183678-7.patch |
| aggregator_pluggable_7_1.patch |
| mw_224.patch |
| drupal_openid-prociders_user_friendly_link_try2.diff |
| 337926-followup.patch |
| dbtng-menu.inc-1227961848.patch |
| 337926-followup_1.patch |
| 296624-fix-do-search_0.patch |
| mw_225.patch |
| 340568-fix-distinct-field-sqlite.patch |
| anon-sessions-5.patch |
| 340557-drupal-is-frontpage-D7.patch |
| hook_help.patch |
| cache_registry_parse_1.patch |
| localize.actions.settings_0.patch |
| taxonomy_term_count_nodes_3.patch |
| page_cache_0.patch |
| 235673-block-caching-fix-D7.patch |
| 295983.patch |
| streamwrapper-D7-21.patch |
| transactions_2.patch |
| drupal_web_test_case.php-330582-48.patch |
| drupal_web_test_case.php-330582-49.patch |
| aggregator_pluggable_9.patch |
+-----------------------------------------------------------+
#7
Reset those patches:
UPDATE pifr_file SET status = 1, test_count = 0 WHERE pass + fail + exception < 6000 AND message = '' AND status > 3;#8
Marking as fixed since no one has reported back any bad results in 7 days.
Feel free to re-open if found.
#9
Seems to still be occurring.
http://drupal.org/node/345632#comment-1151029
#10
#11
Node preview 18 passes, 1 fail, and 0 exceptionsThe issue is that "fail" is singular here, and the regexp can't catch that.
#12
Waiting for confirmation from testing.
#13
#14
Committed after confirmation that it works.
We should be able to trust the results!
#15
Automatically closed -- issue fixed for two weeks with no activity.