Hello,

I'm working on some updates for the Juicebox HTML5 Responsive Image Galleries module (which integrates with views) and have a targeted question about the proper use of hook_update_n when re-structuring existing user configuration data that's been populated through the views APIs.

This module manages custom views display settings through the views APIs (views_plugin_style::options_form), and therefore I do not directly manage the way this settings data is dealt with in the database. From what I can see, views just stores custom display data in a "schemaless" structure as serialized data arrays (inside the views_display tables). Furthermore, the structure/hierarchy of this serialized data directly mirrors that of any fieldsets used on the configuration form itself (unfortunately it seems '#tree' = TRUE must be enforced on the form fieldsets).

One of my updates changes the way some configuration options are nested in fieldsets on the configuration forms, which therefore impacts the way this data is structured in the serialized data arrays noted above. So in order to support existing user data during the update it's looking like I need to leverage hook_update_n in such a way that directly changes these serialized data arrays but without using any views APIs (as that seems to be very bad practice within hook_update_n). I'm slightly nervous about this though as those data structures are core views concepts (not concepts of my module) and if I do something wrong I could inadvertently damage entire view displays for hundreds of my users.

So as a sanity check, I'm hoping someone could help me confirm if I'm approaching this the right way. I can share some example code that I'm testing, but my strategy is:

  1. Fetch a bunch of rows that contain the individual view display data directly with db_query (inside the views_display table).
  2. Loop though these rows and unserialize the info in the "display_options" column (that may contain my module's config settings).
  3. Loop though the unseralized data and search for indicators that this is a views display that my module handles. If I find cases where my module is involved, then:
    1. Restructure the data array to align with the the new nested fieldset structure that will be used on the configuration form itself.
    2. Serialize the new data array.
    3. Write the updated seralized data back to the database with db_update.

Does this all seem correct, or is there other logic that might be missing to address additional storage locations where this views data might be placed?

Any info and guidance is hugely appreciated!

Comments

rjacobs’s picture

rjacobs commented

Here's an example chunk of hook_update_n code that I've been testing to specifically update the Juicebox view displays. Note that _juicebox_update_7100_process_settings() is the function that actually changes the settings structure (just re-nests some parts of the settings array).

// Next check all juicebox view displays and processes the settings for each.
if ($sandbox['view_displays_processed'] < $sandbox['view_displays_total']) {
  $result = db_query_range('SELECT display_options, vid, id FROM {views_display}', $sandbox['view_displays_processed'], $view_display_rows_per_pass);
  $view_displays = $result->fetchAll();
  if (!empty($view_displays)) {
    // Loop through the returned views displays looking for Juicebox displays.
    foreach ($view_displays as $view_display) {
      $data = unserialize($view_display->display_options);
      if (!empty($data['style_plugin']) && !empty($data['style_options'])) {
        if ($data['style_plugin'] == 'juicebox') {
          $current_settings = $data['style_options'];
          $updated_settings = _juicebox_update_7100_process_settings($current_settings);
          // If any updates were found go ahead and update the relevant
          // display in the database.
          if ($current_settings != $updated_settings) {
            $data['style_options'] = $updated_settings;
            $data = serialize($data);
            db_update('views_display')
              ->fields(array('display_options' => $data))
              ->condition('vid', $view_display->vid, '=')
              ->condition('id', $view_display->id, '=')
              ->execute();
            $sandbox['view_display_updates']++;
          }
        }
      }
      $sandbox['view_displays_processed']++;
    }
  }
}
merlinofchaos’s picture

merlinofchaos commented
Status: Active » Closed (won't fix)

Typically the recommended way is to edit a view through the UI, and export it into code. Then, when you make changes, you re-export it.

This kind of interaction might be possible, if you're willing to become very familiar with the data, but you should consider it unsupported.

rjacobs’s picture

rjacobs commented

Thanks merlinofchaos,

I do want to reply, but I won't change the status back in any way. I know the views queues are extremely busy and I'm really just grateful to get a reply from someone who is a true expert on the concepts involved.

Typically the recommended way is to edit a view through the UI, and export it into code. Then, when you make changes, you re-export it.

I have to admit that I'm a bit confused about how exportable concepts would come into play in this scenario. Perhaps you are suggesting that I should recommend that my users export their views before running the update (thus triggering the hook_update_n logic in question) so they can restore just in case something goes wrong?

This kind of interaction might be possible, if you're willing to become very familiar with the data, but you should consider it unsupported.

I suppose this generally means that updates to configuration data structures created views plugins for existing views (such as those introduced by other modules) was never conceptualized external to the UI. This generally makes sense, and I suppose it also means that any "nested" config form structures used by a plugin should ideally never change, or if they do change users should be expected to rebuild their views.

Anyway, you did mention that this "interaction might be possible" so long as I'm careful with the serialized data. That is encouraging. I suppose I should interpret to mean that if I do implement a hook_update_n function as described I just have to be very careful to only change the sparse parts of the config data that relates to my module/plugin.

No matter what, maybe my code snippet above will be useful to someone else hitting the same situation someday.

Cheers!

merlinofchaos’s picture

merlinofchaos commented

When I say export, I mean that they should be placed in a module using the hook_default_views, and then removed from the database entirely using the 'revert' command in the Views UI.

When your users update those views, they should then be re-updated, put into the code, and then, again, reverted. There are drush commands to ease this experience, as well as the bulk export module to allow you to point and click and produce a few files. You can also write or borrow a short bit of code to keep views in an individual file (which can be handy when you have a lot of Views and it makes it much easier to manage exporting just one or two.

If you *are* going to use an update hook -- and I suspect that doing so is going to ultimately be more work than the recommended workflow -- what you probably want to do is:

$view = views_get_view('viewname');

Then change options like so (Note: The variable name display_options is from memory and may be wrong):

$view->display['display_id']->display_options['optionname'] = 'value';
// ... and so on
$view->save();

Note that this isn't always the best way to change options. If you do $view->init_display() and $view->set_display('display_id') you can then use $view->display_handler->set_option() which will make sure that options are set according to their correct default status. Still, if you're just going to update the view, that's probably the one time I might ever tell someone that touching the options without going through set_option() is probably safe.

Naturally I would test the hell out of your script on a dev site first. :)

rjacobs’s picture

rjacobs commented

Thank you!

I think I see what you are saying about the use of hook_views_default_views, etc. The reason that I've not been exploring that is that my "Juicebox" contrib module only implements a views display plugin that users can use in their own views (it only extends the views_plugin_style class). The module does not actually implement any full view definitions. The relevant code for how the module integrates with views is fairly basic and shown in this plugin file source. Because of this I was not seeing the export/revert concept as applicable, or at least not something I would centralize in an update process. The bottom line is that any views that leverage my module are totally varied in structure, unknown to me, and built from scratch by Drupal site builders (they only depend on my module for the display settings).

In terms of using an update hook, I had actually initially drafted/tested a version that does exactly what you recommended (using views_get_view(), $view->save() and so on). Then I started reading about how API functions, functions that may call hooks, and functions that may depend on schema definitions were discouraged in hook_update_n (as the schema cannot be relied upon and not all module functions are available). The use of $view->save for example seemed like it would fall into this "discourage" category within a hook_update_n. This is why I started down the agreeably ugly road of only using core functions like db_query and db_update as shown above.

Even though what I have so far does seem to work in testing, it feels a bit bloated and ugly. Also, as it's something hundreds of user would run when updating to a new release, across a huge variety of setups, I wanted to be sure I was understand all the moving parts as best I can.

merlinofchaos’s picture

merlinofchaos commented

When you have a display plugin and your data changes structure, I typically handle that in the init() or construct() method.

You need to devise a way to detect if the display's settings need to be updated; do that in the method, and directly change $this->options. Just remember that you need to ensure that your option_definition() matches so Views doesn't throw away data. So if you do something like $this->options['version'] be sure that exists in option_definition() so that it doesn't change accidentally, even if that setting is never actually visible in the UI.

Now, you need to be sure that this isn't slow, since this will happen every time until the new view is saved. If it's really ugly, you might need to put a warning in the watchdog log or something to let the administrator know that the view needs to be re-saved in order to stop re-running the init hook.

If you look in Views' own init methods, you'll see a couple of places where we do this in order to try and keep things as compatible as possible moving forward.

However, this is actually critical because your update will not catch users who have exported views into code, as I spoke about above. And that means you're going to get issue you reports that you can't fix.

rjacobs’s picture

rjacobs commented

These are good points, and I feel more comfortable about the variables involved now.

Initially the idea of running some checks at "runtime" to address changes in display settings felt like bad practice, but I'm now seeing how this situation may be an exception. That fact that any exported views would be somewhat immune to any update logic is a hugely useful insight. Also the idea of embedding some form of version indicator within the settings (to use as a pre-check before triggering any special, and potentially expensive, update logic) is an interesting idea.

I'm going to dig into the specifics of this approach and see what I can do. One additional trick is that my module also implements a field formatter, so I'll need to see if there is also a good way to process those settings in a similar manner (when initializing/loading the display settings at runtime)... but that's a whole other story.

Thanks so much for taking some time to assist me with this, I really appreciate it.

rjacobs’s picture

rjacobs commented
Title: Use of hook_update_n to alter seralized data structures managed by views » Methods to alter display data structures when views integration modules are updates (hook_update_n vs runtime checks, etc.)

Tweaking the title in case this is of interest to other module developers out there who integrate with views.