Document git config import workflow

I think one potential issue with proposed git workflow in the issue summary is the UI clears the staging directory due to the code below:

+/**
+ * Form submission handler for config_admin_import_form().
+ */
+function config_admin_import_form_submit($form, &$form_state) {
+  if (!lock()->lockMayBeAvailable(CONFIG_IMPORT_LOCK)) {
+    drupal_set_message(t('Another request may be synchronizing configuration already.'));
+  }
+  else if (config_import()) {
+    // Once a sync completes, we empty the staging directory. This prevents
+    // changes from being accidentally overwritten by stray files getting
+    // imported later.
+    $source_storage = drupal_container()->get('config.storage.staging');
+    foreach ($source_storage->listAll() as $name) {
+      $source_storage->delete($name);
+    }
+
+    drupal_flush_all_caches();
+
+    drupal_set_message(t('The configuration was imported successfully.'));
+  }
+  else {
+    drupal_set_message(t('The import failed due to an error. Any errors have been logged.'), 'error');
+  }
+}

If the directory is under VCS we shouldn't be clearing it here. Here's why we're doing the clearing - http://drupal.org/node/1697256#comment-6641656

PROBLEM I actually skipped this one cos I have no idea how to stage a deletion. Copying all files from dev_active -> prod_staging obviously won't work, because the old YML file will still be sitting there so it'll appear as though there's no change. Seems like we need to clear out the staging dir after import.

I'm not sure how we can handle the conflicting requirements of handling VCS vs FTP controlled deployment. Perhaps the automated deletion should be configurable?

I precisely stated in #1697256-106: Create a UI for importing new configuration that this must not happen. It's beyond my understanding how and why the pasted code of #2 was able to get committed.

That turns this into a critical bug, since Drupal causes data loss and touches data it is not allowed to touch.

That turns this into a critical bug, since Drupal causes data loss and touches data it is not allowed to touch.

I am having a hard time why this is critical or why this causes data loss. In theory anything arriving in the staging directory came from somewhere else, either through being pulled in via VCS or file-synced manually. Once config is imported its use is done, they have no purpose beyond that action. So what conceptually prevents them from being deleted? In a VCS situation you can just pull again, they're not lost. In the meantime, if they are left in the staging directory, there is a much greater possibility of data being lost for real. Namely:

1) View foo is changed on source and synced to staging on target
2) View is imported on target
3) View is changed on target
4) View bar is changed on source and synced to staging on target
5) View is imported on target
6) Since view foo is still sitting there, it also gets synced and overwrites local changes.

xjm had this happen to her in her testing at #1697256-107: Create a UI for importing new configuration.

All that said, David Strauss brought up that he would like to see a way for this to work in situations where staging is not writeable by the web server user. We discussed some possibilities that would avoid the situation outlined above (mostly centered around keeping timestamps of files and doing hash comparisons) but didn't really come to any conclusions.

Gave this some thought. I'm on board with making this configurable behavior, as long as the default is to clear the directory. We can add a Settings tab to the import page and put it in there. As I think about it, the kinds of conflict described in #4 is less likely in situations where your deployments are more controlled (aka when you're using a proper deployment mechanism with VCS and a standard workflow.) Also, if we get #1515312: Add snapshots of last loaded config for tracking whether config has changed in, it will further mitigate these conflicts.

Compromising again until there's a reproducible bug here.

FYI, we have snapshots and a diff UI now.

Testing out the Git workflow with the way I usually set up Drupal sites and I needed to add an extra line to my .gitignore file so I could commit the configuration files but not the whole files directory.

# Ignore paths that contain user-generated content.
sites/*/files/*
sites/*/private/*

# Don't ignore config paths.
!sites/*/files/config_*

Here is the process I tested:

1. Followed the instructions from Building a Drupal site with Git to setup my repository.
2. Created a .gitignore file from the example.gitignore, and included the extra unignore statement from #10
3. Installed Drupal
4. Added, committed and pushed files:

   git add .gitignore
   git add sites/default/files/config_*
   git commit -m "Added .gitignore and configuration files"
   git push
       

5. Clone out the repository again for development.
6. Copy settings.php file from Production site to Development site, adjust database config and flip $config_directories['active']['path'] and $config_directories['staging']['path'] variables.
7. Copy Development site's sites/default/files/config_xyz/active/* files to sites/default/files/config_xyz/staging/*
8. Development site: add, commit and push changes:

   git add sites/default/files/config_*/staging
   git commit -m "Added staging configuration files"
   git push
       

9. Pull files on Production site
10. Note: Here I needed to delete all Production configuration files inside sites/default/files/config_*/staging, I was getting an import error in the UI
11. Make a configuration change on Development, add, commit and push code.
12. Pull files on Production site
13. Navigate to admin/config/development/sync, and Import all
14. Tada! It works!

It might work but after step 13 what files are left in your production staging directory? I'm guessing not what you would expect...

Thanks funkym for getting started on this. A couple of things. It would be useful to have `ls` output for the directory in question both before and after the process. Also, I'm confused why your commiting .gitignore to the repo. Should that go in your your home directory?

Testing.

Curios, Why did you need to do step 10? that sounds like a bug.

Could the path value be anything for staging/active?
$config_directories['active']['path'] and $config_directories['staging']['path']

Am I correct in assuming that these directories could in fact be anything?

Also, I am a little confused as to why they cannot be set in the UI.

config.info is now config.info.yaml

Is there anything here that's not covered by #2121751: [META] Making configuration synchronisation work and other issues?

@catch - the issue summary of that issue does not mention Git. We really need to sort out what gets committed where in the use case where a site uses Git and can be deployed to 3 environments (dev/stage/prod). We can do that here or in the other issue, if the scope there is expandable.

I have been trying to group the workflow-related issues in #1703168: [Meta] Ensure that configuration system functionality matches expected workflows for users and devs

Lets add a page to the configuration document in the handbook that explains a git based workflow.

FYI, we documented this workfkow pretty thoroughly when using a typical dev => stage => prod workflow. See https://www.acquia.com/blog/moving-your-drupal-8-configuration-local-ser...

I apologize, but I do not consider a video to be documentation.

Closing as duplicate of #1703168: [Meta] Ensure that configuration system functionality matches expected workflows for users and devs and the fact that active config is in the db and D8 export files which can be put into git.

This still needs documentation. The referenced issue was about ensuring that the workflow matched what was intended and it was closed because we no longer intend to have it work that way.

Closing because this because the workflow discussed is not longer the way CMI works and people are using CMI successfully today

So we need to document how people are using it today and what the recommended workflow is.
If there is documentation then please link to it and close this issue.

http://drupal.stackexchange.com/questions/85635/what-is-the-suggested-wo... has some information.

Feel free to pick whatever you want from the answers there and create/update d.o documentation.

@frob - there is a blog post linked from that acquia page which is text based. i dont like video based docs either.

documentation++

beejeebus++

This does not seem like a Drupal Core issue to me. It seems like a "add docs to drupal.org" issue. Accordingly, moving to Documentation project. If there are docs to update that are files in Drupal Core, please update the issue summary and move back to Core.

Looks a lot of new tools now exists and having at least a stub page would be great