Module installation fails on big sites

Thanks for the bug report. Upping to major as it makes a mess of a site.

I will spend some time thinking about how best we can address this problem.

Maybe you can provide a standalone php script to initialize the new UUID fields?
Or don't try to do so much during install, but provide an "initialize" interface that will finalize the install using batch API?

I'm getting a hard fail and totally borked database on UUID install as well

I've come up with a simple script for use with drush that uses the batch processing to get around timeout limits that are observed when installing this module on extremely large websites.

This example could lead to an improved uuid.install for future versions. However for now, this script is to be used as follows:

drush --root=/path/to/drupal/site php-script uuid_install_script_for_extremely_large_sites-1728310-4.php

see code:

if (!db_field_exists('users', 'uuid')) {
$operations_install = array(
    array('module_enable', array(array('uuid'), TRUE)),
   );

  $t_args = array('%variable' => 'module_enable(\'uuid\',TRUE)');
  $title_install = t('Creating uuid schema to be able to process for extremely large sites by calling %variable in batch', $t_args);

  $batch = array(
    'title' => $title_install,
    'operations' => $operations_install,
    'file' => drupal_get_path('inc', 'module'),
    /* 'finished' => 'module_exists(\'uuid\')',*/
  );
  batch_set($batch);
  $batch =& batch_get();
  $batch['progressive'] = FALSE;
  drush_backend_batch_process();
  drush_print(print_r($batch,1));
  watchdog('batch', '$batch=' . print_r($batch,1) );
  watchdog('batch','batch finished');
} else {
  drush_print("uuid already installed");
}

the reasoning is, uuid_sync_all is the function that times out on extremely large sites, but simply calling module_enable in a batch operation should (according to my theory) do all the work necessary in a batch function so it (hopefully) won't time out. So, this operation if on extremely large sites might take several hours or even days depending on the server performance and site configuration and size of site , the operation should in theory process the whole operation. I haven't actually tested this with extremely large sites but such a test could be done using the devel content generator to prepare a site.

I developed the above script on drupal core 7.34 and tested it against only 4 nodes so it does need to be tested with extremely large sites.

Any feedback is welcome. Perhaps this isn't the solution however I have used the batch processing in the past but not exactly in this way although similarly.

perhaps drush enable module already does this sort of batch wrapper it'self, I'd be interested to hear feedback on this as it's probably not a unique situation, there could be other situations where extremely large schemas need some sort of batch processing (maybe not in this manner but in a similar manner).

Thanks.

the script from comment #4 probably won't resolve memory limit issues the enable module is clumped one batch, what probably should happen is to break down the uuid_sync_all function (called during uuid_install() ) into several batch operations so that memory limits aren't exceeded when processing insanely huge amounts of entities. what could be done is to batch queue up a thousand entities at a time or so .

What the script from comment#4 does do is prevent timeout errors, generally batch jobs won't time out, of course they face the same memory limits that any php operation does but have the ability to break down operations into smaller chunks.

with that said, feel free to test the script with insane datasets and find out what the limits might be with a given configuration and total number of entities.

Just updated menu_token which now has a dependency on uuid, so had to retroactively install this module, hitting 3 million individual node_revision entries isn't fun.

Perhaps this could be optimised to a single query per database table?

Hi @MustangGB, so you used the script from comment #4? As for optimisation, try xhprof, it will give you a graphical representation of your bottlenecks with latency between calls and the bottleneck in red. xhprof was designed by facebook to optimise their php code, we have used it in Drupal to optimise core and several contrib modules. I've not done this yet for uuid_install() but if you are interested you could run xhprof and it will point out a lot of problem areas. As for 3 million node_revisions, good thing the install only needs to happen once!

I'd remove trying to generate missing uuids from installation tasks (hook_install(), hook_modules_installed() implementations) and maybe show some warning/error message about needs to create missing uuids, something like Node module does when content access permissions need to be rebuilt.

Creation of missing uuids may be performed via administration UI then (UUID module allows to do this via admin/config/system/uuid), but it should be done via batch. So there is problem with the _uuid_sync_table() function which tries to fetch all records which do not have assigned uuid and then loop through them and update each record - it should process records in batches. This function isn't called directly, it's called in the hook_uuid_sync() implementation (which is invoked by by the uuid_sync_all()) - so there is problem with hook_uuid_sync(), it should allow batching somehow...

The only implementation of the hook_uuid_sync() I saw is uuid_uuid_sync() (the UUID module own implementation), I searched through number of contrib modules which have integration with the UUID but haven't found any other implementations of this hook, so not sure what exactly may be performed here (it would be nice to see such examples)...

This patch adds a queue called 'uuid_sync' for generating uuids in tables. Generating uuids for a table is moved from _uuid_sync_table() to UuidSyncQueue::process() and a limit of processing 50 items per cycle is added. This means that when an item from the queue 'uuid_sync' is processed, uuids will be generated for a maximum of 50 records. If indeed 50 records were processed, there may be more records to generate uuids for, so the item will be put back in the queue in that case. During a single cron run, multiple items may be processed from the queue. During my testing, for about 13.500 records an uuid was generated on a single cron run.

Upon install, the module will still add uuid columns to tables (which can also take a while), but the task of generating uuids will only happen on cron runs.

This solution isn't perfect as you'll end up with a situation where some entities will have uuids and others won't yet (until enough cron runs have completed). I don't know the drawbacks of that situation. But this is at least a start.

Why don't you on install just initialise necessary tables, and when an entity is requested, only then generate UUID?
It will work basically the same as cache. First request will check UUID existence, if it doesn't exist - create it, otherwise just return.

#10 works well for me as a workaround. I think if it were to be committed it should have a hook_requirements() warning and maybe a UI to batch process the queue. I'm currently running drush queue-run uuid_sync on ~150 000 nodes.

Anyone could get this done for me ?

I ended up with a corrupted db because lack of warning. I had to recover it manually but I have probably now a little mess to sort out. I am having 0.25million nodes and close to 2million comments and 0,4million private messages.

Any professional help appreciated
Thank you