When exporting groups of rules into a Feature, by category, the generated array export needs to be sorted in a predictable manner.

This works as expected the first time you export, but if you modify a Rule, it gets moved to the top of the list in the web interface, then when you re-create the Feature... the order is screwed up in the file, making it impossible to track what technically should be single-line changes in an export configuration.

I hope this hasnt already been reported, i searched around and couldn't find anything related to this behavior.

Comments

jwilson3’s picture

This issue is actually being experienced on multiple levels of the exported rules, here is an example to explain the first scenario:

Lets say you have a feature called 'test_feature' and you want to add two rules to it.

1) Edit the rules, and add the category 'test_feature'
2) Go Create the feature, and add in the rules in the test_feature category.
3) The resulting feature creates a file test_feature.features.inc with an implementation of hook_rules_default(), containing the two exported rules:

  return array(
    'rules' => array(
      'test_feature_1' => array( ... ),
      'test_feature_2' => array( ... ),
    ),
  );

4) You unzip the feature and install it in your site.
5) You remove the 2 hand-made rules, in favor of the newly created ones from the feature.
6) You edit the Second rule, make some minor change in a text string or something.
7) Note that the modified rule is moved to the top of the list in the web interface, and now has a status of Modified.
8) Re-create the feature, to include these modifications in your rule, expecting perhaps a one-line change...
9) Download / unzip the new feature only to find out that the entire rule list is reorganized, with the SECOND rule, before the FIRST.

  return array(
    'rules' => array(
      'test_feature_2' => array( ... ),
      'test_feature_1' => array( ... ),
    ),
  );
jwilson3’s picture

Deleted, double-post.

jwilson3’s picture

Even if this first scenario was fixed, there is still a second issue that are causing trouble. The order of array keys inside each exported rule's actions also easily get rearranged.

On one export the order is like this:

        '#actions' => array(
          '0' => array(
            '#weight' => -1,
            '#info' => array( ... ),
            '#name' => 'name_of_an_action_provided_by_module_x',
            '#settings' => array( ... ),
          ),

But on a subsequent export, after some modifications from the web interface, the order changes:

        '#actions' => array(
          '0' => array(
            '#name' => 'name_of_an_action_provided_by_module_x',
            '#settings' => array( ... ),
            '#info' => array( ... ),
            '#weight' => -1,
          ),
jwilson3’s picture

These may seem like trivial issues but they create DIFFs and commit records that are really really impossible to read/understand/walk through. If there were a way to sort alphabetically these things that dont depend on array key order, we could make predictable exports, and easily track changes between subsequent exports.

damienmckenna’s picture

How about using a ksort() before exporting the arrays? That way they always get exported in the same order?

damienmckenna’s picture

Status: Active » Needs review
StatusFileSize
new1.86 KB

How does this work?

jwilson3’s picture

StatusFileSize
new2.56 KB

@DamienMcKenna

So your patch fixes the internal sort of each element inside the first level of each rule that is exported. But this doent sort the overall listing of rules. For some reason, when you edit test_feature_2 rule, it is still getting inserted before test_feature_1 in the export. In function rules_categories_features_export_render() I put a print_r($export); between the call to rules_export_items, and features_var_export() and sure enough, only the changed rule got printed, which means to me somehow features_var_export() is integrating the changed elements into the pre-existing elements somewhere else.

Here is a further enhancement on your patch that handles scenario 2, where the #actions and #conditions of specific rules would appear out of order.

Status: Needs review » Needs work

The last submitted patch, rules-n928178_7.patch, failed testing.

jwilson3’s picture

StatusFileSize
new2.66 KB

Oops. maybe this patch will actually work.

jwilson3’s picture

Status: Needs work » Needs review

retest.

fago’s picture

Title: Export order » Always export in the same order
Status: Needs review » Needs work

Makes sense.

       rules_item_type_invoke($item_type, 'export', array($item_name, &$export[$item_type][$item_name], &$export, $module));
+      ksort($export[$item_type]);

Let's fix that in the respective rules export API function directly.

+
+/**
+ * Implementation of hook_default_variables().
+ */
+function rules_default_variables() {

Please keep anything unrelated out of this issue/patch.

>How about using a ksort() before exporting the arrays? That way they always get exported in the same order?
I wonder whether we really need this? Does the order of action in a rule really change once it has been created?

jwilson3’s picture

Working on a new version...

> Does the order of action in a rule really change once it has been created?

I haven't seen the order of (multiple) actions gets changed between exports, but if you look at the patch we are not sorting the actions themselves, which are always exported in natural order with string keys as ascending integers from zero (eg $rule['#actions']['0'], $rule['#actions']['1'])

We are instead ordering the internal elements in each action. #3 explains the situation that happens quite frequently. For instance, I often see that the #weight variable gets pushed to the first array index, without any explanation, or anything having been changed in the interface page for that action at all.

fago’s picture

Well, that may only happen if you edit the action. But if we want to ensure this is consist, we need to do it properly. Conditions might be nested in a tree due to logical operations, thus the only way to fix is a recursive function or a loop that goes through the whole tree and applies the sorting to all elements.

It might be easiest though to go without #3 for now. If people edit the Rule, it is marked as overridden. No big deal imho.

damienmckenna’s picture

Fago, the problem occurs when you then export the updated Rule, it can frequently happen that elements are presented in a different order, so trying to manually DIFF & identify changes becomes much more difficult. If all elements were correctly sorted we'd be saved some aggravation.

fago’s picture

I see. To fix up the whole tree, we could also fix the order of the element keys when the element is edited. That way we do not write a recursive function or such.

jwilson3’s picture

Good point about recursively applying. and admittedly, i havent tested any of the logical operands in chaining conditions to see how those would be affected by export.

I'm on the fence about making the element order consistent for the entire module (eg when the element is edited), instead of special casing it just for exports, because running recursive ksorts might be unnecessarily more processor intensive, causing longer page loads while administering Rules from the interface. Whereas the sorting on export is an infrequent task that is expected to take a little while to complete.

fago’s picture

> ...because running recursive ksorts might be unnecessarily more processor intensive, causing longer page loads while administering Rules from the interface

The point is, when doing it on edit we would not need the recursive operation. Sorting *always* just the currently edited hierarchy level should do it and make the exports "stable".

jwilson3’s picture

Version: 6.x-1.3 » 6.x-1.4
Status: Needs work » Needs review
StatusFileSize
new1.62 KB

I stumbled back across then once I updated to 6.x-1.4 and the patch from #9 no longer applied, so I cleaned it up (based on feedback from #11) and updated it to apply against 6.x-1.4.

After applying this patch, edited rules are now back to exporting in the correct order. :)

Granted there are additional concerns about recursive chains etc that remain untested, but since this isn't a 'destructive' patch, is there any opposition to getting this enhancement into the codebase?

jwilson3’s picture

Version: 6.x-1.4 » 6.x-1.x-dev
StatusFileSize
new2.03 KB

First attempt at using git for the patch, applied against -dev

fago’s picture

Status: Needs review » Needs work

>Granted there are additional concerns about recursive chains etc that remain untested, but since this isn't a 'destructive' patch, is there any opposition to getting this enhancement into the codebase?

Yes, it's an in-complete fix as it doesn't respect rules makes use of hierarchies. I'm not happy with committing fixes that solve the problem only half-way.

So we've to ways to fix it properly:
a) Run a recursive ksort routine on export.
b) Run a non-recurisive sorting routine every time an element is edited, just caring about the element itself and not about the properties of its children.

Both ways should solve the problem and would be fine to me.

mitchell’s picture

Status: Needs work » Postponed

Postponed till someone starts work again.

mitchell’s picture

Component: Feature module support » Provided Module Integrations

Updated component.

kenorb’s picture

Component: Provided Module Integrations » Rules Core
Issue summary: View changes
Status: Postponed » Closed (outdated)

D6 is now not supported. If the issue persist in 7.x or 8.x, please re-open.

kenorb’s picture

Component: Rules Core » Module Integrations