Allow extra data to be added to submissions in result displays

Fixed a typo in the original patch.

I'd prefer to avoid adding any new hooks for this functionality.

@@ -228,6 +228,8 @@ function theme_webform_results_table($variables) {
   // to sort the results.
   $header = theme('webform_results_table_header', array('node' => $node));
 
+  $extra_data_results = module_invoke_all('webform_results_extra_data', $submissions);

We definitely can't be putting hook invocations inside of theme functions.

I'd prefer to avoid adding any new hooks for this functionality.

Do you have any specific solution in mind?
I don't know where else the data for this functionality should come from (in a pluggable way). Currently there is nothing like "some additional data for webform-results without adding components". IMHO providing such an interface is usually done by creating a hook (at least in Drupal). But either way either it means redifining an existing interface or it means creating a new one.

We definitely can't be putting hook invocations inside of theme functions.

The invocation can easily be moved to webform_results_table() and passed to the the theme-function as parameter.

Here is a patch without hook-invocations in the theme function. I guess I won't work on documenting this new hook until we have agreed on an approach.

i need this too, but i use the 4.x version. shouldnt this go to 4.x and then backported?

here is a patch for 4.x

and lets make webform_results_get_extra_data() only return data for submission requested ;)

sorry, here is the interdiff

and final patch for today..fix some header bugs

Related: #1830370: Hook to add additional submission information in downloads

Currently there is nothing like "some additional data for webform-results without adding components".

There is already a hook for hook_webform_submission_load($submissions), that could be used to extend current $submission objects. However, there isn't any way to actually do anything with the extra data that's been collected. Rather than introducing a new hook that loads data in a different way, I think it would be better to use the existing hook for loading data directly into the $submission objects.

This does have a lot of overlap with #1830370: Hook to add additional submission information in downloads, which basically makes what you need possible for CSV downloads, but not for the table view.

@quicksketch: The interesting part of the request is "without adding actual components". So why do I think hook_webform_submission_load() wouldn't work for that?

• In hook_webform_submission_load() there isn't (and can't be) any information about what the submission object will be used to. So there is no way to add data only to the CSV and not to the submission page or the overview …
• You'd have to implement at least one dummy component that doesn't display anything in the form - and remove the empty wrapper-divs with a hook_form_webform_client_form_alter(). I don't think there is even a way to hide this component from node/%/webform except yet another form_alter.

So there is no way to add data only to the CSV and not to the submission page or the overview …

You can now do this for CSVs, because of the new hooks add in #1830370: Hook to add additional submission information in downloads.

/**
 * Respond to the loading of Webform submissions.
 *
 * @param $submissions
 *   An array of Webform submissions that are being loaded, keyed by the
 *   submission ID. Modifications to the submissions are done by reference.
 */
function hook_webform_submission_load(&$submissions) {
  foreach ($submissions as $sid => $submission) {
    $submissions[$sid]->new_property = 'foo';
  }
}

/**
 * Add additional fields to submission data downloads.
 *
 * @return
 *   Keys and titles for default submission information.
 *
 * @see hook_webform_results_download_submission_information_data()
 */
function hook_webform_results_download_submission_information_info() {
  return array(
    'new_property' => t('New export property'),
  );
}

/**
 * Return values for submission data download fields.
 *
 * @param $token
 *   The name of the token being replaced.
 * @param $submission
 *   The data for an individual submission from webform_get_submissions().
 * @param $options
 *   A list of options that define the output format. These are generally passed
 *   through from the GUI interface.
 * @param $serial_start
 *   The starting position for the Serial column in the output.
 * @param $row_count
 *   The number of the row being generated.
 *
 * @return
 *   Value for requested submission information field.
 *
 * @see hook_webform_results_download_submission_information_info()
 */
function hook_webform_results_download_submission_information_data($token, $submission, array $options, $serial_start, $row_count) {
  switch ($token) {
    case 'new_property':
      return $submission->new_property
  }
}

You could also add information in certain other places, such as the submission display and analytics display, because those areas have similar alters you can use. However, there isn't an alter hook for the table display.

Essentially I think what we have here is specific hooks for individual things. The patches in this issue added more general hooks that could be used in a number of different places. I'm not sure one approach is necessarily better than the other, but I'm leaning towards keeping down the path we're on, where there is only one loading mechanism and then each individual interface or display must be individually altered to include additional information if needed.

Sounds nice! Until I can migrate to 7.x-4.x I'll have to stick to this patches here though.

I've just started to port Webform tracking to webform-7.x-4.x. Sadly the proposed solution is highly unscalable. The hook is called once per property for each submission. I guess I'll make a patch for that.

The proposed patch also adds the additional data to the results table (not only CSV exports). This functionality is still lacking in 7.x-4.x.

Here is the patch that makes hook_webform_results_download_submission_information_data() called only once per submission.

oh, thanks for this patch, i never got the time to roll it myself

#17 just improved the given hooks. I've still had to extend the submission information hooks to also work with the webform results table. As there is some API changes involved I really would like to get quicksketch involved.

Here is a patch that extends hook_webform_results_download_submission_information_*() to also work for the results table. This one works on top of #17!

On top of that I would even suggest to make the components use the very same framework too. This would remove the corresponding code from theme-functions completely.

Here is a combined patch for #16 and #19.

Here is a backport for 7.x-3.x. Note that it only works if the patch for #1327186: Use BatchAPI to Export very large data sets to CSV/Excel is applied first. We'll use and maintain those patches until we've finally migrated to 7.x-4.x with campaignion.

This does have a lot of overlap with #1830370: Hook to add additional submission information in downloads, which basically makes what you need possible for CSV downloads, but not for the table view.

Do you have an idea for including the additional submission data also in the table view?

Argh. While I only briefly looked at this, it seems clear that #25 introduces a substantial API change. I don't see how we can consider this for 7.x-4.x, and I don't really foresee a 7.x-5.x branch, since I suspect that a year from now all our efforts will be put into the D8 port.

Also, regarding the results table, the plan is to use views for this, even though it will require creating a dynamic view to contain the component columns.

With regret, I am going to close, but I'd be happy to reconsider some API change that is in addition to the current hooks, rather than instead of them, and one that limits itself to the download. I wish we had made these changes prior to 7.x-4.0. :(

Also, regarding the results table, the plan is to use views for this, even though it will require creating a dynamic view to contain the component columns.

Using views for the result table would be great. Do you have an idea how to add additional data to the results table in the meantime?

While I only briefly looked at this, it seems clear that #25 introduces a substantial API change.

Depends on what you call substantial. #24 is the patch that was actually proposed #25 is only the backport for webform 7.x-3.x. IMHO The only backwards incompatible API change is the removal of theme_webform_results_table_header. The other changes only extend hooks by making additional data available for them.

I wish we had made these changes prior to 7.x-4.0.

We did. The patch has been ready months before 7.x-4.0.

Also, regarding the results table, the plan is to use views for this, even though it will require creating a dynamic view to contain the component columns.

Good luck with that. Until then (I suppose it will be years from now) there are the patches in this issue.

Alas the patches were never RTBC, and with something like 400 open issues at the time, I was unaware of them. This isn't your fault, torotil, and neither is it mine. It is the fault of the community of 400,000+ websites that benefit from thousands of hours of volunteer time and don't contribute anything back to the issue queue. /soapbox

7.x-3.x is only receiving critical bug fixes, and I hope to not have any more -3.x releases, absent someone else takes over maintainership of that branch. I also would not want to widen the API gap between 3.x and 4.x.

If you want to put this forward, the work on hard-coded tables -> views sure needs help.

@Chipie -- You can build a view for each webform (assuming you only have a small number of them), using the existing views integration. Views can retrieve submission results.

Alas the patches were never RTBC, and with something like 400 open issues at the time, I was unaware of them. This isn't your fault, torotil, and neither is it mine.

True, true. Having pretty much all my work on webform being overlooked until it's too late and then having it postponed until 8.x-x.x … really makes me want to fork webform sometimes. -- Except that I don't have the time for that at the moment. Then again: All the patches that I have to maintain this way actually constitute something like a fork with the downside of not using proper version control.

Let's work together on 7.x-4.x. As I understand it, this issue helps with performance, but is this a deal-breaker for you? Maybe there is a way to extend the API to be forward compatible?

I too have all-but-given up on posting patches for certain projects -- drupal 7 core in particular -- since I know they will be ignored. I have been going through the old backlog of issues and trying to give at least a yes-let's-do-it or sorry-no decision on each one. There are still 250 open issues. And all the time I spend managing the queue is time I'm not writing, reviewing, or committing patches. :(

@Dan -- Thank you for you solution, but I'm looking for a generic approach, because the webforms are created by the user and we don't want to create an custom view for each webform. I will take a look at the theme function for the table.

@Chipie -- I am close to having the Results Table in views. This would be one view which configures itself for the webform, whose nid is passed as an argument to the view. This may help you out. See the related issue.

but I'd be happy to reconsider some API change that is in addition to the current hooks, rather than instead of them, and one that limits itself to the download. I wish we had made these changes prior to 7.x-4.0. :(

I've taken another look at #24 and I think I could also implement it without hook-changes - by defining two additional hooks: hook_webform_submission_information_info() and hook_webform_submission_information_data() .

Keeping theme_webform_results_table_header() would need some hack. All themes implementing this function would also redefine (and override) the original submission information.

This will take some time though.

but is this a deal-breaker for you?

Yes it is.

I'm planning on deprecating the existing submission table (the one without the data) and results table (the one with the data). I'd rather see you work with the views-based replacements, which probably won't require any extra hooks.

Would it help you if I posted a preliminary patch for comment? So far I have:

1) A results header that mimics the items-per-page controls that webform uses.
2) Expanding a single data column that represents all the components into multiple columns at view display time.
2.5) Drupal alter of view after modification. Allows someone to add more than one column per component.
3) A serial number (with "(draft)") link to view the submission.
4) Click-sorting the the data columns by raw value (string)
5) Update Views support from version 2 to version 3

Left to do:
1) Click-sorting by a cast of the field for number components
2) Click-sorting on the first question in a grid.
3) NOT click-sorting components for which it doesn't make sense
4) Allow selection of views and legacy tables

Maybe:
1) Revise the classes to conform as closely as possible to existing classes. Markup will still be views-ish.

The component expansion was accomplished with hook_views_pre_view. I would have preferred to use a display_extender, but it doesn't get run early enough. I did wanted to modify the view definition, rather than clone parts of an already-constructed view.

Because this will just be a default view, you can modify it as you see fit. This also lays the ground for someone using a Views Bulk Operations view for the submissions or results. This could then be used for things like bulk-resending of e-mails, setting a "Mark as Read" component, and so forth.

@DanChadwik - I really need some solution for this in webform 7.x-X.x. Will the views based results and download happen only for D8 or for D7 too?

Here is may first stab at a backwards-compatible patch.

This is 7.x-4.x. I'm not actively working on 8.x. :)

Changing to duplicate. Please carry on further discussion in the related hard-coded tables issue.

Here is an updated patch that works with #21 from #1327186: Use BatchAPI to Export very large data sets to CSV/Excel.

Fix a notice caused by the patch.

Another re-roll of the patch in #39 for 7.x-4.x . I still need this until webform_tracking is migrated to the views-based output.

Re-rolled the wrong patch. This is a re-roll of #24. #39 was only a proof-of-concept.

You can mess with tables like this, in this example I move a few columns to the start

function webformmanager_menu_alter(&$items) {
  $items['node/%webform_menu/webform-results/table']['page callback'] = 'webform_results_table_new';
}

function webform_results_table_new($node, $pager_count = 0) {
  if($node->nid==16650 || $node->nid==16654 || $node->nid==16655){
    //foreach($node->webform['components'] as $key => $value){
    //  $node->webform['components'][$key]['name'].=' ('.$key.')';
    //}
    $shiftitems = array();
    $components = array();
    if($node->nid==16650){
      $shiftitems = array(37, 35, 7, 27, 28, 29, 36);
    }
    elseif($node->nid==16654){
      $shiftitems = array(31, 35, 34, 26, 28, 29, 27);
    }
    elseif($node->nid==16655){
      $shiftitems = array(50, 35, 7, 48, 27, 47, 28, 29);
    }
    if(!empty($shiftitems)){
      foreach($shiftitems as $val){
        $movecomponent = $node->webform['components'][$val];
        $components[$val] = $node->webform['components'][$val];
        unset($node->webform['components'][$val]);
      }
      foreach($node->webform['components'] as $key => $val){
        $components[$key] = $val;
      }
      $node->webform['components'] = $components;
    }
  }
  // drupal_set_message('<pre>'.htmlspecialchars(print_r($node->webform['components'],1)).'</pre>');
  return webform_results_table($node, $pager_count);
}

Yeah. I still need this. - re-roll to 4.15.

… and re-roll against 7.x-4.x.

Re-roll.

See #41; please use that issue or make a new one.

I have several contrib modules that depend on this patch. Until they are modified (which wasn’t a priority so far) I’m maintaining this fork. This has nothing to do with #41 or any other issue. Only with the fact that years after this patches were posted here a worse solution was merged into 7.x-4.x while this issue went unnoticed. Leaving me in the current situation. cheers ;)

If there is a better solution, I would like to get that in. It needs to happen in a new focused issue.

I finally had time to split parts of this into a backwards compatible API extension: #3086038: Extend submission information handling.

Re-roll against 7.x-4.x. Sorry, still stuck with this patch.

I’m finally able to migrate to module versions of webform_tracking and webform_paymethod_select that work without this patch — or more with more performance with the patch from #3086038: Extend submission information handling.

The only part of this patch that is not also in #3086038 is the change that makes the results table (non-views) be based on the hooks as well. I’m posting the remaining changes here as patch for being used on top of #3086308 in case anyone else is interested in these changes.

Another tiny update to the patch: Make use of the new hook_webform_results_download_submission_information_data_row() and its performance improvements.

… and another update.