Document exporting domain settings

This is a job for Domain CTools. That's what the module is for.

Ok, to really do this, we have a problem. {node_access} must have numeric keys in the gid column, so we can't get rid of those.

That means that the option is making {domain}.domain_id a foreign key to an exportable-sensitive table (or export value).

That's a pretty big architectural change in the core module.

domain_export

domain_id => serial
subdomain => machine-name foreign key to {domain}.subdomain

On save / load of an exportable, the domain_id would be checked against this table and updated in {domain} as appropriate.

It would also encourage a wholesale switch from using $domain as an array to making it a real object.

Could also convert domain_id to a UUID with an algorithm based on the subdomain string.

Lots to unpack here.

1) Exportables are generally a Good Thing. Among other things, they can allow for scripted updates across servers (or different domains on different servers). They can also reduce some database calls if we bypass some of the current loading procedures.

2) domain_id is the foreign key for all joins to {domain}, plus the key for {node_access}.gid. In the core module suite, that includes 7 tables right now. And integers make better indexes than varchars, so I'd prefer to keep them that way. (Though I could be argued out of that position by a DBA.)

3) We actually don't need a {domain}.machine_name column, since {domain}.subdomain works just fine for that. (I've already tested it. {domain}.subdomain is a unique string containing legal characters.)

4) The problem, as I understand it, is the auto_increment on the serial {domain}.domain_id keys. We have to prevent those from incrementing (i.e. changing uncontrollably) when we do an import. I think there may be several ways around that issue, including a {domain_export} mapping table that stores subdomain and generates a domain_id that can be used to update the foreign keys in other tables. Or we could come up with a reliable UUID and use that for domain_id.

5) It hadn't occurred to me to start with Domain Conf, but that makes sense. Problem is, we have to solve the domain_id issue first or none of this will work.

Very productive IRC chat with pounard, hejrocker, EclipseGC.

Given that UUID INTs are unreliable, the "best" solution may be an id mapping table "owned" by exportables that can update the domain_id key in other tables.

This solution is outlined in #2.

In essence, domain_id is always treated as a foreign key. The primary key is owned by the export.

This is only going in to 7.x.3. The object discussion is separate, yes.

Right now I have sample loader code that takes hook_default_domain() data and injects it into hook_domain_bootstrap_lookup(), effectively letting domains in code overwrite domains in the db.

For the main module suite, I would want to make most things exportable. Right now, I'm just focused on domains and then aliases.

We should possibly start a new Git branch for this. I haven't had time to look at it recently.

It could, but I don't think domains need to be entities. Too much overhead and not enough reward.

The trick here (for CTools) is that we have to make a new table to store the numeric ID / machine name of the domains, since so many things (including {node_access} require numeric keys.

The other problem is that CTools really wants you to use it's own admin tools, and I just finished overhauling the admin UI for 7.x.3. So I don't really want to change that.

1) We'd have to alter that behavior to look for default domains first.

2) I don't quite understand.

3) I think we may end up with a separate 'export' interface.

See #1142558: domains as entities for the discussion of domains as entities. Nutshell: I'm against it.

This will need to go in the main module, I think.

Here's a start of a patch. Needs lots of work. This could use some eyeballs, too.

All this first patch does is update the db schema to allow for exports from a 'machine_name' while still using numeric ids for the keys (which are required by {node_access}.

TODO

* Change the CRUD hooks and forms to the new schema.
* Integrate with CTools.
* Unknown.

And a patch that keeps current tests passing! Hooray tests.

Minor spin-off patch: #1207278: Allow default setting from domain form

Current patch against head.

OK, here's a patch that (at least) starts the capacity by allowing Bulk Export to export domain records.

There's still much to be done:

-- How to integrate into the existing CRUD UI?
-- How to load exported domain information?
-- How to load extra data (like aliases)?

My temptation is simply to use exported data to overwrite what's in the current database, but leave the rest of the functionality intact.

Input welcome.

So it occurred to me that we have to store 'machine_name' in both tables, since 'subdomain' is volatile (that is, subject to edits).

This version allows that. It also gets the Export UI partially working. We may end up with some custom handler code.

Summary: I'm not crazy about CTools trying to take over the module CRUD interface. It's more than I want or need, especially since it makes numerous assumptions about data structure.

I am not inclined to rewrite everything to support CTools, so I wonder if we can implement a simple Import/Export routine that can be supported by Features directly?

And a quick poll, which behaviors are most important?

1) Exporting domain configuration from one site to another?
2) Bypassing the database by loading settings from code?
3) Knowing if individual domain records have been altered (overridden) from the exported version?
4) Other.

I think that is the exact use-case that people want. The question is, do we try to use CTools are go straight for Features exportables.

I tried CTools and am now interested in trying Features.

My problem is with the UI and data model that CTools wants to force on its exportables (overridden, disabled, etc.), It doesn't really fit what we need.

I also have 7.x.3 admin working exactly as I want and have NO desire to rewire the UI or move it all to a massive class handler because Ctools expects it.

That said, if _someone else_ submitted a patch for that behavior, I'd be thrilled.

But my initial review says that CTools doesn't quite fit if it's entirely up to me to do the work.

Well, the current patch is in position to implement the default CTools handling of states, so maybe take a swing at that integration.

Updated patch against 7.x-3.x.

I'll be honest, I have no clue what CTools is trying to do here. Someone else is going to have to either write this patch or pay for my time.

In regards to #32, no, actually. Exports of Fields and settings, for instance, don't follow that pattern. The problem we're facing is not the same as Panels / Views, but more akin to exporting configuration.

Given that a) domains are not object or entities (and likely won't be), and b) there is a TON of ancillary data (aliases, settings) that we may want to export as part of a domain definition, it looks like direct Features export support might be more fruitful.

The big problems to solve are:

-- How to export the data (in what format).
-- How to load the data (which is very tricky, given the bootstrap process).
-- How to tell if the data has been altered.

Yeah, I made more progress in one day using Features natively.

Some new decisions to make:

-- How to track domain status?
-- How to handle additional feature elements (like aliases)?
-- To export all domains or allow selection?

To test. Apply the patch and drop the file into the domain directory as domain.features.inc

Slept on it for a night. I see a very basic battle plan that gets most people the ability to export / update from code using Features.

Instead of per-domain exports, I think we really want to export the entire data set, and offer the option to also export ancillary data:

-- Domains (export all domain records)
-- Domain configurations (export all Domain Conf values)
-- Domain aliases (export all Domain aliases)
-- Domain theme settings (export all Domain themes)

On activation of a Domain feature, the target database will be updated with the values from the feature. During normal operations, settings will _not_ be loaded from code, which makes bootstrap easier, since we don't have to change it.

This will, at least, allow the export / import / reset of domain configurations from site to site. It will also allow the creation of install profiles / feature packages.

As a side note, Domain Views will need to start using machine_name keys.

I've been making steady progress on this, and the Features approach is "better" here because it gives more direct control over the output with less code refactoring.

Hope to have this wrapped up this week.

Here's the current state. This is looking very good.

Features:

-- Export all or selected Domains to a Feature.
-- Export all or selected Aliases to a Feature.
-- Reload / reset Domains and Aliases from a Feature.

TODO:

-- Domain Conf
-- Domain Theme
-- Tests

To test, apply the patch and rename the new files properly (and put them in the right folders).

Note that the patch includes a database update.

No. But I'm very tempted to commit so people can test. Let me take a last look.

Tests still pass. I'm going to commit what I have so people can poke at it.

Still needs tests for this functionality. Also needs eyeballs on:

-- What happens when exporting / importing to a new site that hasn't been configured.
-- How differences between source site and target site are reconciled (if they are).
-- Is the UI confusing?

Work-to-date committed.
Note that a database update is included.

Tagging.

The big problem in D6 is the whole {domain_export} table. But the approach should still work.

I'd look at committing a patch, but don't intend to work on the D6 version at all. That branch is closed to new features.

Switching the primary key could be problematic, but that's really just a "watch out for this" warning and not a barrier. I simply don't have time to backport the code to D6. But it should work.

It looks like the checks for "Overridden" are not coded correctly, since they don't check against the entries in the database.

Needs to be updated...

Better patch that fixes some errors and checks overridden status better.

Also makes the Rebuild functions destructive.

Likely to be committed unless someone objects.

Committed.

Still needs documentation.

Yes, it does. That is (currently) the intended behavior.

I am, frankly, unclear on the concept of rebuild. Features documentation gives little or no guidance on the issue.

Here's my logic, which is still flexible at this point.

1. Domains in features can live side-by-side with Domains in the database, but in that case, the feature is marked as Overridden. I think this would be a different case if we were exporting one domain per feature, but we aren't. We are exporting the entire domain configuration as a single bundle.
2. Unlike Views, we aren't exporting to code in order to avoid hitting the database during page execution. We're exporting configuration to be replicated across environments.
3. It seems logical to me to use a destructive rebuild -- since you are "restoring" the feature from scratch. If you didn't want to do a destructive rebuild, you would re-export the feature, yes?
4. You have to deliberately wipe/rebuild the feature by using the revert process. So there is a built-in safety mechanism.

Attached is a screen shot (annotated) of a Feature that is "overridden".

So I think your scenario is looking at the situation backwards. Here's what I would expect:

1. Create the domain feature on a test site.
2. Check into VCS (version control system)
3. Deploy the feature to production.
4. People add / change domains on the live site.
5. Re-export the feature from live. (Or from a clone of the live db).
6. Check in the revised feature to dev / VCS.

I think this is more of a Features management problem than anything else.

Now, if there are automated processes that continually reset the Features on a site, that would be a problem. The solution (currently) would be to add step 3a) Disable the Feature to prevent rebuild.

But I'm not seeing anything in normal site operations that force-reverts domain features.

One last note of clarification:

A Domain feature is more like a Content Type export than anything else. If the Content Type changes, you either have to revert it or re-export it.

So it's not a case of "Content types in Features living side-by-side with native content types." It's more like "The Foo content type is defined in a Feature and has been altered. What would you like to do about it?"

Well, I'm open to debate here.

If people expect non-destructive Features, we should support that. But then how would you ever Revert the Feature to its base version?

We could, presumably, make it an option in the Feature export.

How do you handle Content Types in such a scenario? Are they locked down and never edited on live?

What I think you really need is better domain workflow tools, like those suggested in these two posts:

• #1303616: Allow alias registry for auto-creation
• http://drupal.org/node/1096988

I'm to the point now that I load up all the domains on the dev side, and use some aliasing tricks to ensure that the domains work on all environments.

The trick in that case would be to enable the Domain feature, and then disable it.

From what you describe, though, I don't know that exporting domains to Features is feasible in your use-case.

You might be better off creating a small "feature" module that auto-creates the secondary domain on install -- assuming you are using install profiles or similar. Domain Access will create the primary domain on installation, and your small module could just create the second one.

But the way I would fix the client bug ATM is how we have to handle features for things like content types and changes to Views:

1. Client spots bug and asks for fix
2. We sync live db to dev
3. Fix the feature on dev
4. Export the feature to code
5. Upload to live

Personally, I think Features forces this workflow unless you lock down the live site -- which you can also do by not giving anyone the 'administer domains' permission.

@joachim It was such a PITA that I gave up.

Documentation stub needs work here: http://drupal.org/node/1331868