Originally part of #2171683: Remove all Simpletest overrides and rely on native multi-site functionality instead:

Simpletest only has an option to keep test assertion results, but does not allow to retain the test site + its database tables yet.

Retaining the complete test artifacts allows developers to perform advanced debugging.

Attached patch is extracted from the parent issue and rolled against HEAD, but it will be easier to re-extract the changes from the patch in the parent issue once that has landed.

This patch does not really make sense without the test site directory changes from the parent issue.

CommentFileSizeAuthor
#4 drupal8.test-keep-artifacts.4.patch10.31 KBsun
drupal8.test-keep-artifacts.0.patch9.66 KBsun
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Related issues: +#2175397: Move test environment clearing from TestBase::tearDown() into TestBase::run()
jibran’s picture

jibran
he/him
Primary language Urdu
Location Sydney, Australia
CreditAttribution: jibran commented
Status: Postponed » Needs review

NR for testing.

Status: Needs review » Needs work

The last submitted patch, drupal8.test-keep-artifacts.0.patch, failed testing.

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Status: Needs work » Needs review
FileSize
drupal8.test-keep-artifacts.4.patch10.31 KB
amateescu’s picture

amateescu CreditAttribution: amateescu commented
Status: Needs review » Needs work 
+++ b/core/modules/simpletest/config/schema/simpletest.schema.yml
@@ -4,6 +4,13 @@ simpletest.settings:
+    clear:
+      type: mapping
+      label: 'Clean-up options'
+      mapping:
+        artifacts:
+          type: boolean
+          label: 'Clear test artifacts'
     clear_results:
       type: boolean
       label: 'Clear results after each complete test suite run'

+++ b/core/modules/simpletest/lib/Drupal/simpletest/Form/SimpletestSettingsForm.php
@@ -108,8 +114,9 @@ public function validateForm(array &$form, array &$form_state) {
+      ->set('clear.artifacts', !$form_state['values']['simpletest_clear_artifacts'])
+      ->set('clear_results', $form_state['values']['simpletest_clear_results'])

Why are we keeping 'clear_results' separate instead of moving it under 'clear'?

amateescu’s picture

amateescu CreditAttribution: amateescu commented

Also, are there any security considerations that we need to take into account if we leave full-blown site installations around in the file system?

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Status: Needs work » Needs review

re: #5: Because that is out of scope here → #2175151: Rename simpletest.settings:clear_results into clear.results

re: #6:

Regarding security, the full corresponding chapter in the issue summary of #2171683: Remove all Simpletest overrides and rely on native multi-site functionality instead applies and explains the situation. In short: The sites cannot be accessed without having the valid private key of the corresponding site.

amateescu’s picture

amateescu CreditAttribution: amateescu commented
Related issues: +#2175151: Rename simpletest.settings:clear_results into clear.results

Ok then, let's add that issue as related.

sun’s picture

sun
Primary language German
Location Karlsruhe
CreditAttribution: sun commented
Parent issue: #2171683: Remove all Simpletest overrides and rely on native multi-site functionality instead » #2215035: [meta] Simpletest UI cleanups
Related issues: +#2171683: Remove all Simpletest overrides and rely on native multi-site functionality instead

alansaviolobo queued 4: drupal8.test-keep-artifacts.4.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 4: drupal8.test-keep-artifacts.4.patch, failed testing.

mgifford’s picture

mgifford
he/him
Primary language English
CreditAttribution: mgifford commented
Assigned: sun » Unassigned

Unassigning so someone else can take it on.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Project: Drupal core » SimpleTest
Version: 8.9.x-dev » 8.x-3.x-dev
Component: simpletest.module » Code

Triaging issues in simpletest.module to determine if they should be in the Simpletest Project or core.

This looks like it a simpletest module issue, changing the project.