See also #1264716: Install fails when using automated install

Column 'module' cannot be null ) [query_string] => INSERT INTO {role_permission} (rid, permission, module) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2) [args] => Array ( [:db_insert_placeholder_0] => 3 [:db_insert_placeholder_1] => create commerce_customer_profile entities of bundle shipping [:db_insert_placeholder_2] => ) ) 

This is caused if you enable multiple features at the same time (during an automated install) and if one feature (A) defines the user role and another feature (B) defines the permission. The problem is caused by the way the enabling of features is handled. The only solution we found so far is to add the following code to feature B

function my_feature_B_enable() {
  if (!variable_get('my_feature_A_rebuilt', 0)) {
    features_rebuild(array('my_feature_A' => array('user_role')));
    variable_set('my_feature_A_rebuilt', 1);
  }
}

The variable_get/set is there to avoid rebuilding it multiple times, one time is enough.

The same problem can occur with taxonomy and permissions.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jhedstrom’s picture

I've found that this can even happen with a single feature, if the feature is exporting a taxonomy vocabulary, and the corresponding permission. This seems to only happen when installing the feature as part of an installation profile.

Related issue: #1063204: Adding a new permission causes integrity constraint violation.

attiks’s picture

We had the same problem when enabling a bunch of features at the same time.

recidive’s picture

Tracking down this issue, it looks like this is a problem with module list not being updated, when e.g. you enable a feature module that creates a taxonomy vocabulary or add a new field and at the same time you try to set the permissions they add.

When user_permission_get_modules() is called in _user_features_get_permissions() it doesn't return the modules/permissions for the permissions we want to set in the feature module.

This problem seems to happen only when installing a drupal site using the automated install (e.g. by using drush si or aegir), and as other says, it can also happen when enabling multiple features at once. I wonder what's fundamentally whith installing a site using automated install comparing to install it using the installer (install.php), as the latter seems to work fine.

I see there are a static cache and a persistent one in module_implements(), which may be related to this problem.

recidive’s picture

Adding a call to cache_clear_all('module_implements', 'cache_bootstrap'); and drupal_static_reset('module_implements'); to the top of user_permission_features_rebuild() function doesn't help with the problem, so I guess the problem is that the modules aren't getting enabled before features module is trying to add permissions.

Is there any way we can ensure required modules are enabled before permissions are set through features?

hefox’s picture

hefox’s picture

Status: Active » Needs review
FileSize
704 bytes

Does this help?

hefox’s picture

FileSize
712 bytes

Oops

attiks’s picture

not tested, but the problem with continue is that the permissions will not be set or am i missing something?

recidive’s picture

@attiks, yes, I tested the patch in #7 and the permissions were not set. @hefox said it would by chance be called again at later time, when the permission existed, but this haven't worked.

jhedstrom’s picture

The patch in #7 resolves the issue I described in #1. I haven't tested this on the 2 feature scenario described in the opening comment.

recidive’s picture

recidive’s picture

Status: Needs review » Needs work
febbraro’s picture

Status: Needs work » Closed (duplicate)

Sounds like

#1) Same problem as described in #1187858: Node permissions fail to import
#2) Can be solved with proper module dependencies and #1195432: Use dependency information when importing

recidive’s picture

I tested the patch in #1195432 and it doesn't fix the issues described here. Actually, after updating to latest dev version of features module, with changes done in 13-14/9 now I'm getting a different problem with a taxonomy related permission. Also a few warnings at the top.

$ drush si martplug
You are about to DROP your 'martplug_beta' database and then CREATE a new one. Do you want to continue? (y/n): y
Starting Drupal installation. This takes a few seconds ...                                                                              [ok]
Missing argument 1 for features_enable() features.module:814                                                                            [warning]
array_keys(): The first argument should be an array features.module:816                                                                 [warning]
Invalid argument supplied for foreach() features.module:772                                                                             [warning]
WD php: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'module' cannot be null: INSERT INTO                 [error]
{role_permission} (rid, permission, module) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2); Array
(
    [:db_insert_placeholder_0] => 3
    [:db_insert_placeholder_1] => delete terms in 1
    [:db_insert_placeholder_2] => 
)
 in user_role_grant_permissions() (line 3028 of /Users/henrique/Webroot/martplug-1.0-beta1/modules/user/user.module).
Cannot modify header information - headers already sent by (output started at /Users/henrique/drush/includes/drush.inc:917)             [warning]
bootstrap.inc:1207
PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'module' cannot be null: INSERT INTO {role_permission} (rid, permission, module) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2); Array
(
    [:db_insert_placeholder_0] => 3
    [:db_insert_placeholder_1] => delete terms in 1
    [:db_insert_placeholder_2] => 
)
 in user_role_grant_permissions() (line 3028 of /Users/henrique/Webroot/martplug-1.0-beta1/modules/user/user.module).
Drush command terminated abnormally due to an unrecoverable error.                                                                      [error]
Error: Declaration of DrushMakeProject_Profile::buildPath() should be compatible with that of DrushMakeProject::buildPath() in
/Users/henrique/.drush/drush_make/drush_make.project.inc, line 371

We need urgently to figure out why features behave different from when installing with installer (install.php) and automatic install (drush si/aegir).

smk-ka’s picture

@recidive
Could you try adding module_implements(NULL, FALSE, TRUE); to the modifications added by the patch in #1187858: Node permissions fail to import, i.e., at the top of user_permission_features_rebuild()? The methods you tried in #4 are not actually clearing the module_implements static cache (use the $reset parameter instead). This should then clear both the node types as well as the module_implements cache, which might help with this issue.

recidive’s picture

@smk-ka: I added the module_implements(NULL, FALSE, TRUE) call just above the node_types_rebuild() added by the patch in #1187858: Node permissions fail to import and am getting the same error described in #14.

If you want to, please ping me on IRC so we can work to fix this problem. Thanks!

recidive’s picture

@smk-ka: Is there any other thing I could try? I'm getting crazy here, 3 weeks stuck on this issue. Need to get this sorted out to move my install profile forward.

It's very odd how it works fine with install.php but not with drush si/aegir, maybe we should look what they do different that may be causing this problem, I could not find anything.

I tried moving the problematic permissions to a separate feature module but this haven't worked. My features were kit compliant, but now they are starting to look awkward.

recidive’s picture

Status: Closed (duplicate) » Needs work

Re-opening now that #1187858: Node permissions fail to import got fixed and this one still remains.

jhedstrom’s picture

The fix from #7 still resolves the issue, so I'm re-rolling to apply cleanly to the latest features release.

neochief’s picture

I've been fighting with this bug hardly a lot recently and I think I came to the proper fix for this issue. For those of you who don't care about details, just try the attached patch. Rest read below. The following scenario outlines problem with importing permissions, but it's just one of the potential problems, caused by just the same bug.

This beast fails only if you install the site in non-interactive mode, i.e. though drush. I'm attaching simple install profile with which you can reproduce the bug by running drush site-install featurebug.

The devil sits inside of several Features cache functions. They all being reset after emach module is installed in interactive mode of installer (through browser), since each batch operation is kind of new page view for the module.

However, for drush it's just single page view and there are situations, when single feature may be enabled in queue before everything else, including core module includes and therefore only part of core and contrib feature includes will be loaded intill the end of installation process (since first feature static caching all the existing module includes).

To illustrate this, if you install my test install profile Drupal will build following queue of modules to install:

Array
(
    [user] => 0
    [filter] => 0
    [node] => 0
    [field_sql_storage] => -8
    [field] => -9
    [text] => -10
    [features] => 0
    [menu] => -1
    [a] => -2                     <<<< Dummy feature which triggers building of cache.
    [block] => -3
    [book] => -4
    [color] => -7
    [comment] => -11
    [contextual] => -12
    [dashboard] => -19
    [dblog] => -20
    [help] => -32
    [file] => -33
    [image] => -34
    [options] => -35
    [list] => -36
    [number] => -37
    [path] => -38
    [taxonomy] => -39    <<<< Core module, needed to properly import vocabularies and permissions.
    [search] => -40
    [shortcut] => -41
    [field_ui] => -42
    [rdf] => -43
    [php] => -44
    [t] => -45                 <<<<< Feature with vocabularies.
    [p] => -46                <<<<< Feature with permissions.
    [featurebug] => -47
)

Note, that all of the dependencies in profile are set correct, it's not a bug of installation profile, you can check yourself.

So, here's what happens:
1. When feature A is installed, taxonomy is not enabled yet. Therefor in features_include, it's default implementations isn't loaded and cache of implementations is built without it.
2. After taxonomy is enabled, nothing is changed, since there are no cache resets.
3. When feature T is installed, since there's no taxonomy include which implements import available, it doesn't create any vocabulary.
4. When feature P is enabled, it tires to find who implements permission "delete terms in 1" (1 is vid of a vocabulary) before import. Since vocabulary isn't created, core taxonomy module doesn't report that permission belong to it. This ends up with empty placeholder $module for that db_insert into permissions table.

The solution is to clear those static caches after you've enabled a module (not necessarily a feature, since the module itself can provide some hooks for interactions with existing features).

Again, the reason why everything works just fine when you enable features in interface or install site in browser, is because each feature is enabled one per a page view and caches being cleaned automatically.

neochief’s picture

Status: Needs work » Needs review
Haza’s picture

+++ b/features.moduleundefined
@@ -278,6 +278,7 @@ function features_help($path, $arg) {
   if ($modules = array_intersect(array_keys(features_get_features()), $modules)) {

You are looking for modules that are features. We should not look on that, but we need to find modules that are exportables into features instead.

+++ b/features.moduleundefined
@@ -351,7 +353,7 @@ function features_include_defaults($components = NULL, $reset = FALSE) {
-    $components = $include_components;
+    $components = array_keys($include_components);

Isn't that just a bug fix ?

Haza’s picture

Status: Needs review » Needs work
FileSize
667 bytes

Here is an other approach to solve that. Please have a look.

hefox’s picture

Looks like a sorta duplicate over at #1537838: Upgrading to 7.x-1.0-rc2 (from rc1) breaks taxonomy creation, yea? functionally similar patches, though think like theirs a bit better for where it does it.

bojanz’s picture

I like #1537838: Upgrading to 7.x-1.0-rc2 (from rc1) breaks taxonomy creation better as well.

Once that is in, I think we want to add a:

  // Make sure the file list is rebuilt now that a module is disabled.
  features_include(TRUE);

to features_modules_disabled() so that it's functionally equivalent to this patch.

Do we want to do that here, or in a new issue?

bojanz’s picture

Title: Enabling multiple features at the same time may fail » Rebuild the file list properly when a feature is enabled or disabled
Priority: Major » Normal
Status: Needs work » Needs review
FileSize
706 bytes

Here it is.
Added the call to features_modules_disabled(), moved the features_modules_enabled() call so that it doesn't fire if no features were enabled (that's just wasteful).

bojanz’s picture

Status: Needs review » Closed (duplicate)

Okay, included that fix in #1530386: Avoid unnecessary cache rebuilds and improve installation performance, which provides a more complete fixup.

neochief’s picture

Status: Closed (duplicate) » Active

Issue described in #20 is still relevant.

pfrenssen’s picture

I'm also affected by this issue. Here are rerolls of the patches in #20 and #23.

pfrenssen’s picture

OK neither seem to solve the issue I'm having. I'm getting a 'Column module can not be null' error message when updating a row in the role_permission table when reverting a feature (similar to the one in #14).

bojanz’s picture

That means that the module providing the permission is no longer active.

mpotter’s picture

Status: Active » Needs work

This still needs some work. The patches in #29 cause features_include(TRUE) to be run twice. While I'm interested in solving the issue clarified by neochief in #20, I also don't want to kill the performance of Features during installs and/or drush. So we just can't start throwing around cache-clears without really understanding what needs to be done.

pfrenssen’s picture

Agreed, I was quite pressed for time and made the rerolls to be able to test them.

pfrenssen’s picture

@bojanz, thanks for the hint, it turned out to not be an inactive module but a vocabulary that was deleted while the feature was switched off. Subsequently enabling the feature caused Drupal to try to set a non-existing permission, hence the error. In the end it was a simple drush fu that removed the outdated permission from the feature and fixed the problem.

This could be avoided by checking if the permissions still exist in user_permission_features_rebuild() before setting them with user_role_change_permissions() but that is a different issue.

pfrenssen’s picture

FileSize
2.31 KB
1.87 KB

I have finally found the time to look into this on a snowy afternoon :) I started by trying the remaining problems mentioned in this issue and analysed the use cases that would warrant cache clears, wrote test scripts for each case and fixed all the problems that were unearthed by the test cases.

  • User permissions do not revert reliably when the user roles are defined in a different feature, or in the same feature and these features are enables at the same time.
  • Vocabularies are not created on feature enable.
  • User roles are not created on feature enable.

Cache clears are necessary in three cases during features_modules_enabled():

  1. One of the enabled modules provides a new Features component.
  2. One of the natively supported modules was previously disabled and has now been enabled.
  3. One of the enabled modules contains a new feature.

Requirements: a clean Drupal install and some modules

I have three test cases. To prepare for them, create a clean Drupal install with the following modules in their latest dev releases:
$ drush dl ctools diff features features_extra --dev

Test case 1: install profile

This is adapted from neochief's install profile. I have extended it with a user role exported in a separate feature to test the use case mentioned in the original report.

  1. Download featurebug.tar.gz and unpack it in the profiles/ folder.
  2. Run the following script:
#!/bin/bash -e

# Install Drupal.
drush si featurebug --account-name=drupal --account-pass=drupal

# Enable modules.
drush en ctools diff features -y

# Populate the component cache.
drush eval "features_get_components();"

# Enable a feature and a module that defines a component.
#drush en testfeature -y

# Show overridden state of features.
drush fl

Result: The "p" feature containing user permissions has not reverted and is displayed as overridden.

Test case 2: all components in one feature

  1. Download testfeature.tar_.gz and unpack it in sites/all/modules/.
  2. Remove the featurebug install profile (because of some duplicated feature exports).
  3. Run the following script:
#!/bin/bash -e

# Install Drupal.
drush si minimal --account-name=drupal --account-pass=drupal

# Enable modules.
drush en ctools diff features -y

# Populate the component cache.
drush eval "features_get_components();"

# Enable a feature and a module that defines a component.
drush en testfeature -y

# Show overridden state of feature.
drush fl
drush fd testfeature

Result: No overrides, works fine.

Test case 3: enable a module with an additional component

This enables Feature Extra Block and its included test feature which provide an additional component that allows to export blocks.

  1. Run the following script:
#!/bin/bash -e

# Install Drupal.
drush si minimal --account-name=drupal --account-pass=drupal

# Enable modules.
drush en ctools diff features -y

# Populate the component cache.
drush eval "features_get_components();"

# Enable a feature and a module that defines a component.
drush en fe_block features_extra_test -y

# Show overridden state of feature.
drush fl
drush fd features_extra_test

Result: none of the components could be reverted.

pfrenssen’s picture

Status: Needs work » Needs review
FileSize
4.36 KB

With this patch all three test cases work fine. It has the following changes:

  • Added all three cases where the static caches should be cleared to features_modules_enabled()
  • Cleared the static cache of module_implements('features_api') when clearing features_get_components(). When module_invoke_all('features_api') is called this uses a cached list of modules that implement this hook, which excludes any newly enabled components.
  • Implemented missing hook_features_enable_features() hooks which cause components not to be reverted when their features are enabled.
  • When a single feature contains both user roles and permissions the permissions would revert first, as user_permission comes before user_role alphabetically. This prevents the permissions from reverting. So try to revert the roles first upon enabling the feature.
     /**
    + * Implements hook_features_enable_feature().
    + */
    +function user_permission_features_enable_feature($module) {
    +  // If one feature contains both user roles and permissions, make sure the
    +  // roles are reverted first so the permissions can be applied on them.
    +  user_role_features_rebuild($module);
    +  user_permission_features_rebuild($module);
    +}
  • Add dependencies on features that export user roles to features that export permissions, so the roles are always reverted first:
    --- a/includes/features.user.inc
    +++ b/includes/features.user.inc
    @@ -36,6 +36,16 @@ function user_permission_features_export($data, &$export, $module_name = '') {
         }
       }
     
    +  // Include features that provide user roles as dependencies.
    +  foreach (features_get_component_map('user_role') as $role_map) {
    +    foreach ($role_map as $feature) {
    +      // Do not allow a module to depend on itself.
    +      if ($feature != $module_name) {
    +        $export['dependencies'][$feature] = $feature;
    +      }
    +    }
    +  }
    +
       return array();
     }
BrockBoland’s picture

I'm having an issue similar to the one described in #20: when using drush to enable a Feature containing Workbench Moderation states (added by a patch from #1314508: Make states/transitions exportable with the Features module but that's not important), which depends on Workbench Moderation so that is also enabled, the states are not created. Did some digging, and the cache features_api (used in features_get_components()) is not refreshed, so it doesn't know that Workbench Moderation defines a couple component types during a revert.

Anyway: I just tried out the patch in #36 and it did indeed fix this problem for me, but since I was having no issues with users or taxonomies, I cannot give the rubber stamp of RTBC.

kristiaanvandeneynde’s picture

The patch works nicely in most use cases, but when stumbling upon this issue, I've noticed one more static variable that's screwing us over: $include_components in features_include_defaults().

I left the irrelevant parts out below:

function features_include_defaults($components = NULL, $reset = FALSE) {
 static $include_components;

 // THIS CONTAINS AN OLD 'CACHE' THAT SHOULD BE RESET TOO
 if (!isset($include_components) || $reset) {
    $include_components = features_get_components();
    // ...
 }

 // WE COULD PROVIDE AN EMPTY ARRAY HERE TO 'SKIP' THE LOOP BELOW
 if (isset($components)) {
    $components = is_array($components) ? $components : array($components);
 }

 // ...

 foreach ($components as $component) {
    if (isset($include_components[$component]) && (!isset($included[$component]) || $reset)) {
     // WILL NOT RUN BECAUSE THE FIRST ISSET FAILS WHEN USING AN OLD 'CACHE'
    }
 }
}

Basically, Pieter's solution from #36 makes sure that features_get_components() is refreshed, but there is still an old copy of f_g_c()'s return value being kept in the static $include_components.

I attached a patch that fixes this, although 90% of the credit should still go to pfrenssen.

P.S.: Dag Pieter :)

kristiaanvandeneynde’s picture

The only line I added is this by the way:

features_include_defaults(array(), TRUE);
RunePhilosof’s picture

The problem reported by kristiaanvandeneynde in #38 could also be corrected by doing this:

 function features_get_default($component, $module_name = NULL, $alter = TRUE, $reset = FALSE) {
   static $cache = array();
   features_include();
-  features_include_defaults($component);
+  features_include_defaults($component, $reset);
illuminatedgeek’s picture

#38 seems to work, so I'm rerolling for the final 1.0 release.

hefox’s picture

Status: Needs review » Needs work

Ugh. Bloated patch.

There's various issues about this role and permission though the title of this issue is currently not about it, though the description is...

I'm really not happy with the permission/role stuff here, I rather go the approach of having a way to specific order of components to revert in by default.

pfrenssen’s picture

Yeah I'm also not satisfied at all with this approach. This patch was what was needed to fix all the problems in the test cases provided in comment #35, but it is doing too much at once. I am more in favor of #1572578: Rethink the way features are enabled / disabled which would make much of this patch obsolete, and the other things that are in there should be split up into different issues.

attiks’s picture

Patch no longer applies and the fix from #1572578: Rethink the way features are enabled / disabled does not fix this, still getting db errors because module is NULL

attiks’s picture

In case it might help somebody, rerolled patch from #41 attached, but not working (for me)

meichr’s picture

Issue summary: View changes

Patch in #45 works fine for me (Features-7.x-2.0).
Had the problem that with 'drush si' enabling a single feature containing a vocabulary and two permissions on it (edit, delete) led to two Warnings that the permission would not exist.
Even with moving the two permissions to a separate feature, defined as dependent on the vocabulary feature didn't help without the patch in #45 (explanation in #20).
As a workaround this patch is perfect for my installation profile until features implements a new strategy of enabling features or applies this patch. I assume this will not be the case before features-8.x-1.x-dev due to being late in Drupal 7 now and the complete change of features in Drupal 8.
But thanks for this awesome module to the maintainers for all their work on it and tanks for this patch to my pre-posters.
Meichr.

meichr’s picture

So, though the patch in #45 works so far, using the Internationalization (i18n) module together with Taxonomy has the effect, that the multilingual setting for individual vocabularies is not imported by an installation profile with 'drush si'.
The i18n module adds the form item 'i18n_mode' to the vocabulary settings form. The value is properly exported to the feature together with the other form items, but not imported. After installation the feature form tells that the feature containing the taxonomy settings is "Overridden".
My current workaround is either using the Features form to revert the feature or 'drush fr ' after installation.
As I'm not very well into the code of the features module I don't know yet what enhancement to the patch would solve this one.

meichr’s picture

The problem described in #47 is a dependency problem between the i18n_taxonomy sub module (of i18n) and the feature module containing the taxonomy component. The order must be that i18n_taxonomy is enabled first, before the feature can be rebuilt (enabled).

To add a dependency to i18_taxonomy into the feature module I have provided a patch for the i18n_taxonomy sub module at the issue queue of the i18n module: Add Features support for Taxonomy translation sub module.

bzitzow’s picture

Re-Rolled the patch from #45 against features version 7.x-2.0 release

joseph.olstad’s picture

I observed this issue on features version 7.x-2.9

Patch 49 had copy-paste issues.

I took 49 and cleaned it up and rerolled it.

I haven't yet tested this thoroughly though

See patch:

joseph.olstad’s picture

Version: 7.x-1.x-dev » 7.x-2.x-dev
Status: Needs work » Needs review
joseph.olstad’s picture

Priority: Normal » Minor
Status: Needs review » Needs work

The patch doesn't work. Same issue after applying it.

"Cannot insert the value NULL into column 'module' "

Happily however, simply clearing cache and re-enabling the module and another feature revert does the job as a workaround.

joseph.olstad’s picture

Category: Bug report » Support request
Status: Needs work » Active

After testing and debugging a little more thoroughly I discovered that the error I observed:

drush en new_custom_module -y;
Cannot insert the value NULL into column 'module', table 'role_permission'; column does not [error]
allow nulls. UPDATE fails.

only occurred when I was using drupal core 7.51

I then restored my db prior to core upgrade, rolled back to drupal core 7.50 and the issue I observed went away after repeating the same module installation that failed under 7.51.

So, looks like this is not directly related to the 'features' module, might be due to some changes in 7.51

however please note, I am not using a vanilla core, I have about 8 hand picked patches that I use and it could be that my patches to core that I am normally using could be incompatible with the new core 7.51.

joseph.olstad’s picture

doesn't appear to be a features module issue specifically , this issue could confuse some people so close it.

Followup for what I observed in related issue.

Likely that what the others above observed has been fixed as I've never seen this problem before except today and I use features 'a lot'

joseph.olstad’s picture

Status: Closed (outdated) » Fixed

Ok, wierd, I tested 7.51 again and fixed this issue by upgrading my defaultconfig module to the latest dev (same as latest release I believe)

So I found a fix,
upgrade your defaultconfig module if you use it and want to use 7.51 core!

joseph.olstad’s picture

so others above likely using the defaultconfig module with the 'features' module as well.

joseph.olstad’s picture

Turns out I had features conflicting on base fields and they were out of sync, so not sure if the patch above works or not, in my case my features needed to be updated to not conflict with shared field settings and once decorked that it was working as designed.

Sorry for the confusion

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.