This is a bigger one.
Problem description:
I have one webform with lots of submissions and a hidden field, which I fill somewhere. I always want to view only webform submissions where that field has one certain value - i.e. only a fraction of all submissions.
The issue is: all submissions are retrieved from the database, after which almost all of them are discarded again (using _webform_report_test_filters()). This is huge resource overhead.
It is actually possible to handle many cases of filtering with the SQL statement, which is much faster.
You currently retrieve all submission data with (a convoluted version of)
SELECT <fields>
FROM {webform_submitted_data} d
INNER JOIN {webform_submissions} s ON d.sid = s.sid
INNER JOIN {users} u ON s.uid = u.uid
WHERE d.nid = %d
i.e. retrieve all submissions for a certain webform node.
But you can make a grouped query returning only the records from {webform_submissions} that you want - and then join {webform_submitted_data} on that one, to retrieve data for only those wanted submissions.
Example: say you have 2 filter criteria in your webform report
- some value (hidden webform element with cid 1) must be equal to 1234
- some date (element with cid 5) must be newer than 1-1-2010
Your SQL, outputting only data for submissions where those 2 criteria are matched, becomes
SELECT <submission fields>, d.cid, d.data FROM
(SELECT <submission fields>
FROM {webform_submitted_data} d
INNER JOIN {webform_submissions} s ON d.sid = s.sid
INNER JOIN {users} u ON s.uid = u.uid
WHERE d.nid = %d
AND ((d.cid = 1 AND d.data = '1234') OR
(d.cid = 5 AND CAST(d.data AS DATE) >= '2010-01-01'))
GROUP BY <submission fields>
HAVING count(d.cid) = 2) s
INNER JOIN webform_submitted_data d ON s.sid = d.sid
(I spotted the trick in notifications.module, which does something similar when matching node inserts/updates against user-specified criteria.)
My patch is basically
- a way to construct this SQL
- preceded by a huge concatenation of boring switch/case statements, to determine which filter criteria are fit to include in the SQL - and to mangle fields/date in the right shape.
- noting that all the filters which are used in the SQL, don't need to be checked by _webform_report_test_filters() any more.
I didn't change any logic in _webform_report_test_filters(), on purpose; only took care that not all logic will run any more. This way, we can disable all the filtering of the SQL statement, without harm (i.e. things will still be filtered as they are now).
Plus, selective stripping of now-unused filter functionality from _webform_report_test_filters() is just too tiring...
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | 1138264-2.patch | 20.05 KB | roderik |
| webform_report-performance2.patch | 20.05 KB | roderik |
Comments
Comment #1
roderikOh, plus one thing...
I had to get rid of the message
''There are no submissions for the selected webform. Either the form
has not yet been completed by anyone, or the results have been cleared. This will not
prevent you from creating this report, but this message will be displayed on the report
page until someone submits the selected webform.'
...because with the filtered SQL, I don't know of a way to distinguish between "no results for the filtered report" and "no results at all for this webform". If you want to add it back, ideas are appreciated.
Comment #2
roderik*sigh* I was doing some last minute editing while uploading this patch, and made a silly quoting mistake.