This is my first significant contribution to any open source project. I have wanted to contribute to Drupal for a long time, in gratitude for all the awesomeness others have put into it already. By necessity, I began working on fixing and extending the vote generation code, but then I continued working on it until it was basically feature-complete for all of VotingAPI's capabilities (as of 7.x-2.11).
Here are the features it now supports from the admin UI:
- independent criteria to delete and generate votes:
- entity type
- node types (when entity type is node)
- vote tags (multi-axis voting categories)
- percent, points, and option vote value types fully supported, for all possible values
- easy presets for some of the most common voting widgets
- custom settings accept an arbitrary list of vote values, to support ANY VotingAPI widget
- powerful statistical models generate very realistic votes with a useful diversity of results
CAVEATS:
- This code cannot generate votes for "rated while editing" widgets offered by the Fivestar module, which are stored in a field on the entity, not with VotingAPI.
- I attempted to update Drush support for the new features, but I have not tested ANY of this with Drush. I was only focused on the admin UI and statistical models. So if you prefer using Drush to generate votes, you might experience some bugs with this code. Drush users, please test and let me know!
- I tried to avoid changing any public function signatures, but there are some changes to the option array structures, STRICTLY limited to generation and deletion of fake votes. So if any of your scripts call existing functions to generate or delete fake votes, you might have to make some changes. This should not affect most users.
- The one edge-case widget type I haven't fully supported is one which would accept an infinite number of possible values, such as a slider widget that produces floating point values. I don't believe any such widget exists currently, although the VotingAPI would support that. But I HAVE made it possible to generate votes with any FINITE number of values, including floating point values, which should be sufficient for testing such a widget, if it ever exists.
Other than the caveats described above, the new vote generation features work very well. I included fairly robust validation, error handling, and user feedback. I tested with the Fivestar module and the Rate module, under many different widget configurations. The results are very realistic, based on statistical models that produce Gaussian distributed user biases, and voting averages that converge on specific random target values (instead of always 50%).
I have tried to follow the Drupal coding standards, but I hope you'll forgive me if I haven't conformed entirely to the "Drupal way". I haven't written any documentation for it, but the code is very well commented, and the admin UI has fairly clear descriptions for all inputs, so hopefully it's all self-explanatory.
Attached are drop-in replacements for votingapi.drush.inc
and votingapi.admin.inc
, modified from the latest stable release, VotingAPI 7.x-2.11. (These two files are unchanged in the current dev snapshot.) No database changes are required, although you might have to clear the admin menu cache to see the new vote generation UI. As with any update, backup first. Then just copy these two files into your sites/___/modules/votingapi
folder, overwriting the originals. The admin UI can be found at SITE/admin/config/development/generate/votingapi
I would avoid installing these files on a production site until it's tested a lot more.
Comment | File | Size | Author |
---|---|---|---|
#3 | votingapi_devel_generate_feature_complete-2001090-3.patch | 37.44 KB | galundin |
#1 | VotingAPI_7.x-2.11_generate_overhaul.zip | 9.07 KB | galundin |
VotingAPI_7.x-2.11_generate_overhaul.zip | 9.06 KB | galundin |
Comments
Comment #1
galundin CreditAttribution: galundin commentedBug: my validation was rejecting negative values, which are used by Up/Down.
I don't see a way to delete the original attachment, but here is the corrected version:
Comment #2
galundin CreditAttribution: galundin commentedI did some more testing today with a fresh installation. Here's a quick walk-through of the minimum steps to see what this code does:
votingapi_vote
table in the database, but they're not valid and not recognized by any widgets that I know of.votingapi.drush.inc
andvotingapi.admin.inc
into the votingapi folder, overwriting the original versionsIf you have any difficulty with any of these steps, please do let me know and I'll be happy to help. I would also really appreciate any error reports or suggestions for improvements.
Comment #3
galundin CreditAttribution: galundin commentedTo be more consistent with community standards, here is the same contribution in the form of a patch.
When applied to 7.x-2.x, the result is the same as the attachment in comment #1.
Comment #4
interdruper CreditAttribution: interdruper commentedThank you for this really powerful code for generating random votes. I have tested it under D7 with VotingAPI 7.x-2.11 and Fivestar 7.x-2.0-alpha2 (in 5-star format) and everything works fine, except this minor bug:
I guess that just a simple validation after calculating votes will resolve this minor bug.
Finally, I encourage you to port the code to an extra and separate module for VotingAPI, so it can be tested by the community in the usual way.
Comment #5
galundin CreditAttribution: galundin commentedHello, interdruper. Thanks for the feedback!
I repeated the test you described, but I wasn't able to reproduce that bug. I added a Fivestar field to the Article node type, "rated while viewing", with five stars, and a vote tag of "
vote
". I generated 50 users and 50 Article nodes. I deleted all votes and then generated new votes with the specified criteria, using (percent: 20,40,60,80,100
) values. A total of 1991 votes were generated. Then I used phpMyAdmin to examine the contents of thevotingapi_vote
table. I looked through all 1991 votes, and the "value" field was always correctly set to one of (20,40,60,80,100) for every vote.The code that generates these votes first produces Gaussian distributed values (floating point) and then quantizes them into valid values:
The
quantization_threshold
is the midpoint between two consecutive values. So it's basically rounding to the nearest valid value, and it can only return a valid value. In this case, the valid values specified were (20,40,60,80,100), so it cannot return any other value.The
votingapi_cache
table contains the calculated results of the aggregate function ("average
" for percent value types), and those do have floating point values. Perhaps that is what you observed? Or maybe a Views listing of the same aggregate data? If not, can you please tell me where you found these invalid values?Regarding your suggestion to release this as a separate module: Voting API already provides a feature to generate votes, but it is very buggy and incomplete. To me, it makes sense to include this "fix" and improvement to this existing feature in Voting API. But I don't know all of the community protocol, so if others suggest that it should be a separate module, then I will do that. I would also like to hear from the maintainer, torotil, about this.
Comment #6
interdruper CreditAttribution: interdruper commentedI reviewed again my test, and you are right, the votes are generated correctly, the non-integer values come from the aggregate field ('Content: Rating') exported to Views by Voting Module. Sorry for the non-accurate bug report.
About porting your code to a different module, the functionality is already implemented by Voting Module indeed, but in a elementary mode and far from the power of your code. And also with a buggy behavior. I could not use it for my tests nor I found any hint/patch in the queue issue, that why to find your code was a gift for my tests.
As usual, the best for the community will be a cooperation between the module maintainer and you to incorporate this functionality to the module. But since it is an important addition (many lines of code), perhaps the maintainer has no time to integrate it, even with your collaboration. In this case the best will be to create a new module with Voting API dependency; in this case you could add more functionality and receive the feedback of the community more effectively.
I think that making available your code as a patch is not accurate, since not only it fixes the bugs in the vote generation functionality of the module, but also extend it in an important manner. If finally you could not agree with the module maintainer to integrate it into a new VotingAPI release, imho it would be better to have it as a separate module. Thus your code will get more visibility and will be more useful for the community than as a patch.
Comment #7
galundin CreditAttribution: galundin commentedThanks for the update on the bug report.
Thanks also for your input on releasing this as a separate module. Those are some good points. You are right, it is a very big patch that addresses multiple issues, which could make it difficult to review properly. On the other hand, it is only intended for development testing, not production use. I sent an email to the maintainer of Voting API to get his input too. I will make a decision after I hear back from torotil.
Comment #8
torotil CreditAttribution: torotil commentedFirst I'd like to thank you for your patch!
That's quite some piece of work. It's quite a big patch that's why I haven't had the time to take a closer look until now. To keep up the quality I'd rather split it into a series of smaller patches.
A few things that I've noticed so far (apart from that):
Comment #9
torotil CreditAttribution: torotil commentedComment #10
galundin CreditAttribution: galundin commentedThanks for the feedback. I want to be sure I understand your suggestions.
> What's the reason for bundles being limited to node types?
I had to research bundles just now. I guess I didn't understand that concept before. So the node type is actually a node-specific form of a bundle type? Instead of
node_type_get_names()
, is there a corresponding function that would provide the same kind of information, but generalized to all entity types, instead of just node? I looked for this, and I think it would befield_info_bundles($entity_type)
. Correct me if I'm wrong about that. Otherwise, I'll go ahead and extend support to the bundle types of all entity types.In the options array, I'll replace
node_types
withbundle_types
. In the UI, I'll set the input label to use the name of the selected entity type, to avoid confusing users. E.g. "SELECT NODE TYPES" or "SELECT USER TYPES", instead of "SELECT BUNDLE TYPES". In the case when there is only one bundle type I will hide that input. Does that sound okay?> There is still some places where node_type and entity_type are mixed.
I don't know what you mean by this. Can you please refer to a specific line of code that has this problem?
> I'd prefer to modularize votingapi_generate_votes() somehow.
When you say modularize, do you mean that it should be possible for other modules to extend the vote generation with custom functionality?
> I'd rather split it into a series of smaller patches.
That's understandable, but I have to be honest, it seems like that would be a somewhat challenging and error-prone process. I'm still open to consider it, if you can suggest a feasible approach to segmentation. But perhaps interdruper's suggestion to roll it into a separate module could provide a simpler path forward, to allow it to be tested and improved in a more holistic manner. Perhaps then, after its quality reaches an acceptable level, we might find a way to integrate that module into a future release of Voting API. Does that sound reasonable?
Comment #11
torotil CreditAttribution: torotil commentedBundles
Bundles are defined in the implementations of hook_entity_info() and you can get a list of all entity_types and bundles via entity_get_info(). field_info_bundle() is a wrapper around entity_get_info() that's perhaps more usable in this case.
node_types vs. entity_types
>> There is still some places where node_type and entity_type are mixed.
> I don't know what you mean by this. Can you please refer to a specific line of code that has this problem?
It seems that I'm unable to find the the lines of code in the patch that I was referring to at the moment.
modularize votingapi_generate_votes()
I've meant to move parts of the code into utility functions (ie. handling of the vote_values argument, deletion of old votes, …) even better if they are reusable. Perhaps some of those routines could be shared between votingapi.admin.inc and votingapi.drush.inc .
Splitting
Segmenting it could be done by file (admin section vs. drush command), by command (only one drush command at a time) and by command arguments (a patch with a simplified command with less options first and add options with a second patch). I think this could be done fairly quickly and it would allow me to take a closer look at each section of the code. That's essentially what I would do if I would apply this patch as it is: take part of the patch, apply it, read it, improve it, commit it to the tree, repeat …
I think votingapi as well as this patch's code would benefit a lot from including this patch.
Comment #12
galundin CreditAttribution: galundin commentedThanks for the detailed guidance. I guess you're right, that should be easy enough. It would make sense to approach the modularization and segmentation at the same time. Basically just refactoring, submitting one piece at a time. The challenge would be to keep cohesive functionality in each patch. So I'll start with the back-end (drush code), and preserve the interface from the old UI, at least temporarily.
When I break out useful chunks into utility functions, do you think some of those should be exposed as part of the API? Specifically, being able to delete votes by criteria could be useful to other modules. I guess we can work out the interface for each function whenever it's determined to be generally useful.
I was thinking, since content generation is actually part of the Devel module, and since the new vote generation code is fairly complex, it might make sense to separate it into a new file,
votingapi.devel.inc
. What do you think about that?When I submit each patch, should I post them to this issue, or create a new issue for each one?
Comment #13
torotil CreditAttribution: torotil commentedIf you want I can give you commit access to votingapi then we can do this in a separate feature branch. Maybe we can also coordinate the effort via email or jabber …
> Specifically, being able to delete votes by criteria could be useful to other modules. I guess we can work out the interface for each function whenever it's determined to be generally useful.
The reason why there no utility function doing this is that votingapi_delete_votes() doesn't only delete votes but also invokes hook_votingapi_delete(). This means deletion cannot be done in a single DELETE database query.
If you're interested in improving the API you can also take a look at the 3.x branch where I've started to work on an object oriented API …
Comment #14
galundin CreditAttribution: galundin commented> I can give you commit access to votingapi
I'm still learning to work with Git, so I'm somewhat hesitant about that. I know you can revert any mistakes I might make, but is it possible to restrict my access to a specific branch, so I don't have to worry about accidentally committing to the wrong branch or something?
I'll have to study the 3.x branch so I can rewrite this in an easily portable way...
> we can also coordinate the effort via email
Sure, I'll send you an email with my contact info.
Comment #14.0
galundin CreditAttribution: galundin commentedadded link to admin UI
Comment #15
rooby CreditAttribution: rooby commentedOne important thing that seems to be missing is a number of votes to generate.
Preferably number of votes per node, but even total number of votes would be better than nothing.
Comment #16
galundin CreditAttribution: galundin commentedThe way I wrote it, each user has an 80% chance of voting on each node. So you can control the number of votes generated by the number of users in the system. I know that's not ideal, but it's still much more useful than what you get without this patch.
These days I'm doing most everything on the client-side, with a very thin layer of authentication and validation on the back-end. Voting calculations still belong on the server, but I prefer a more light-weight solution for that. So I'm not able to support any revisions to this patch.
Comment #17
ressa CreditAttribution: ressa commentedIt is a pity this code hasn't been committed yet, since it works great with 2.11.
Thanks for the great work @galundin!
Comment #18
TR CreditAttribution: TR commented