Hello everbody,

a few weeks ago I started a new project which needed extensive voting mechanisms. I found VotingAPI (from eaton), which -basically- had a few of those features but also lacked quite a bunch. I also didn't like the overall implementation with Views and the way that Fivestar (or other modules) were needed to make it work the way it is supposed to. Enough with the introduction... Here is what I've been working on (so far):

Vote is a module that defines a new entity type (vote) and stores its votes as such entities. Votes can be cast on _ANY_ entity type (even on "vote" itself, hehe). Vote also comes with "functions" and "voting types":

A function basically provides a handler which defines how and where a function stores its result values. Each function can have its own db table to store the results in order to allow more complex functions to break the INTEGER barrier which the default result table uses for its results.

A voting type basically combines a group of functions. For example a voting type "percent" would combine the functions "average" and "count" while "points" would also use the "sum" function.

Voting types and functions both use a set of classes as handlers to provide custom option forms and advanced (and individual) functionality for each function / voting type. The functions handler -for example- makes it possible to store function result values in seperate db tables with different table structures.

Votes are being organized by "voting pools" (comparabale with node types). A voting pool could for example be "overall" or "fish". Each voting pool uses one of the voting types (for example a voting pool named "overall" could be of type "percent") and thus is allowed to use whatever functions that voting type provides. Since "overall" is of type "percent" it would -for example- be allowed to use "count" and "average". In the administrative UI the user is now able to check / uncheck those functions and (if checked) chose from the custom settings that those functions provide with their handler classes (the UI is similiar to the Text Formats UI in core).

Adding new votes causes the module to add (or update) an entry to a "schedule" table in the database which handles the recalculation of results. Those entries contain the information about the "constellation" that was changed. A constellation is a pool and the entity_type and entity_id that we voted on. When cron is run (if recalculation is set to cron) or on hook_exit (if set to real-time calculation) the schedule is being worked (beginning with the oldest entry) using a looping mechanism and some more functionality to make sure the page doesn't time out during recalculation. When a "constellation" of votes ("pool", "entity_type", "entity_id") have been recalculated the entry is being removed from the schedule.

All the stuff that I just talked about works already. I still need to implement Views, Rules and Fields (I will provide a set of widgets and maybe even ask the developers of Fivestar to allow me to reuse theirs). Also I want to provide node-bound voting so that the node type settings can be set to allow votes on every node of that type without having to abuse the formatters as AJAX input forms (like Fivestar does it right now). You have to think about it as something like "frontend input fields" that are defined by the node type settings somehow. I didn't really think that through though ;).

Here you can download the current development snapshot:

https://github.com/sumsi/Vote/

A word of warning: The entire code is COMPLETLY undocumentated (will add that when I am finished). Since there is no frontend to actually add votes yet you will have to use the code snippet that I put into "vote_init()" to play around a bit.

Attached to this post you find a few screenshots of the administrative UI!

Thanks for your time!

CommentFileSizeAuthor
#23 vote.tgz21.27 KBfubhy
#21 vote.tgz20.17 KBfubhy
#20 vote.tgz20.16 KBfubhy
#1 6.jpg83.55 KBfubhy
5.jpg54.63 KBfubhy
4.jpg67.06 KBfubhy
3.jpg53.94 KBfubhy
2.jpg56.38 KBfubhy
1.jpg106.52 KBfubhy

Comments

fubhy’s picture

StatusFileSize
new83.55 KB

Sorry, I forgot one screenshot ;)

fubhy’s picture

Category: feature » task
dawehner’s picture

The code itself seems to be written very clear, but some documentation really would help here.

fubhy’s picture

The documentation is following asap (laying my hands on that right now). The basic data handling of this module seems to be nearly finished now so I am going to document that and afterwards start working on the frontend (node, fields, views and rules implementation).

fubhy’s picture

Documentation for vote.module is done (which covers most of the functions that are of interest ;) ) .... except some extended explanation on the hooks, but thats going into vote.api.php anyhow!

UPDATE:

I created a drupal.org sandbox and a associated git repository.

SANDBOX: http://drupal.org/sandbox/sumsi/1074206
GIT: http://drupalcode.org/sandbox/sumsi/1074206.git

dawehner’s picture

Status: Active » Reviewed & tested by the community

Code documentation now. Well written and there are not many code style problems anymore.

cweagans’s picture

I would vote yes, as well.

fubhy has noted in IRC that a lot of people are mentioning duplication of functionality. I'd like to point out that this approach is very very different from how Voting API does things and that it will be a more flexible solution. If we discouraged this type of innovation, we'd still have flexinode. :)

fubhy’s picture

I just spoke with eaton. He had to leave again and we couldn't finish our conversation but will look into this and it seems like we are gonna end up with some sort of collaboration.

Concerning development: The admin UI also supports result overview pages for every function. Custom functions can reuse the functionality of the default result pages to create custom overviews. (I already did that for the score overview for pools of type "options"). All in all its going really well!

I set up a fresh drupal installation on one of my servers and installed this module. If you want to play with it just text me on IRC (sumsi). I will give you the URL + Login.

fubhy’s picture

Status: Fixed » Reviewed & tested by the community

Removed.

joachim’s picture

> I also didn't like the overall implementation with Views and the way that Fivestar (or other modules) were needed to make it work the way it is supposed to.

Part of the strength of VotingAPI IMO is that the UI is pluggable -- you can make star ratings, +1/-1 widgets, etc etc.

The bottom line is that if this module's db handling is indeed better, users of voting api will want to migrate. The best way forward would be for this to be a new branch of VotingAPI with an upgrade path.

fubhy’s picture

Thanks joachim! Also, you really brought something up there. How would we migrate the old data (which is stored slightly different than it used to be stored with VotingAPI) into "Vote". Pretty easily I say! It should be quite easy to create a batch process that fetches all data from the old VotingAPI (votes) table and groups them into pools (and automaticaly creates those pools -> pools = tags). The only thing that could get us into trouble there would be the fact that with the old VotingAPI one tag name could be used for multiple value types ("Vote" has a unique relationship between "pool" and "value type"). This would mean that we would have to also group the old votes by value type and make sure that we assign a new pool name in case a tag is being used for multiple vote types. I think I am going to create that migration tool later tonight. Should be done by tomorrow if nothing gets in my way :).

fubhy’s picture

Aaaah, nevermind. It doesn't even lead into a problem that a tag could server multiple value types in the old VotingAPI. I can just name the pools that I generate from the old data with "%tag-%value_type" and throw a notice to the user that he might want to change the machine-name after the migration (machine-name changing is supported by "Vote"). I guess that makes it even easier. Shouldn't be a problem to figure that out today.

Okay.. One more update: We basically just need one simple function for this:

  • Get all tag <-> value type pairs from the old vote table and save them as pools with machine-name = "tag-value_type" and all functions enabled with default settings
  • Get all votes from the old vote table and insert them into the new vote table using "tag-value_type" as pool name
  • Schedule all vote content for recalculation (already part of the program)
  • Done

Because of the new scheduling / recalculation concept I dont need to port the results at all (just the votes) and when the migration is finished the user would just have to press the "Schedule everything for recalculation" button on the "Settings" page and wait for the vote results to be recalculated. Easy!

joachim’s picture

Are you maintaining compatibility with the votingAPI UI plugins?

fubhy’s picture

You mean Fivestar and such? I will create a few widgets for fields on my own. However I wont adopt the -in my opinion- hacky approach of using the output of the formatter as an AJAX input "widget".

Fields are going to be used for casting vote DURING creation of a node / comment, etc ONLY! However, I also have a solution / idea for casting votes on the frontend. It will be possible to attach votes to entites directly (for example a node type can be configured to allow frontend voting like -at the moment- the fivestar formatters do). Those will be configured on the node type settings (under the "edit" inside of the vertical tabs). In the future that functionality will be applied to user profiles, taxonomy vocabularies and so on.

For an example of similiar functionality you can use the Flags module. Flags can also be attached to nodes directly (by configuring them for a node type for example).

joachim’s picture

> Fields are going to be used for casting vote DURING creation of a node / comment, etc ONLY!

Huh? That really doesn't satisfy the use case of a voting system then! You need users to vote on things they can't edit -- that's part of the point.

I strongly suggest you take a deeper look at the votingAPI ecosystem. Parts of it may be hacky (IIRC at least one of the voting widgets is a complete pain to theme), but the principle of providing a commong data storage system and then a variety of voting widgets is sound and should be preserved.

In my opinion, anything that seeks to improve the data storage of voting shouldn't be breaking the API provided to the widgets, but slotting in seamlessly. If the API between storage and UI is itself hacky, that can be improved at another iteration.

fubhy’s picture

REVOKED

I just asked the Fivestar guys - Awesome: No more hacky field formatters for frontend voting in the D7 branch. So my entire previous statement is revoked: I will try to wrap my head around their code and see how we can make it work together. But from the glimps I took right now its really not hard at all! Actually it should make a big part of the Fivestar code so much easier. We will see, we will see.

fubhy’s picture

Hello again!

Done:

  • Support for alternate databases (vote storage / result querying now pluggable),
  • Migrating votes from VotingAPI to Vote by simply clicking a button in the admin UI
  • Talked to ericduran (Fivestar)

I just finished the vote migration from votingapi -> vote (its a sub-module) and rewrote the functions which are involved in creating the results queries (fetching vote data in order to calculate results) to be pluggable (support for alternate db's like mongodb). However, since votes are entities the alternate databases need to support the basic entity storage features like entity_load() aswell (I have no clue if the mongodb modules does that).

Also, I spoke with ericduran (maintainer of Fivestar). Like eaton, he is interested and will look into this. He also mentioned, that he is planning a rewrite of Fivestar (I hope this is not top-secret :P). We basically share the same ideas. I am really looking forward to working on this when things arent as busy as now (before DrupalCon).

marvil07’s picture

Nice to see so much activity around this.

As the maintainer of vote up down, I am really interested on this too, so please feel free to ping me on IRC or by contact form to coordinate if you want ;-)

I was also planning to do a fivestar widget, but joining forces will be just plain awesomeness!

fubhy’s picture

Eric had plans for reworking the Fivestar concept. We didn't talk much because he had to leave but it sounded really great and we should definatly coordinate this because I sense that there might be a way to create a whole voting suite with an API for storing votes and calculating results (this one) plus an API / frontend for handling widgets for votes with a) Fields support and b) direct entity support for voting on the frontend. It seems like your "vote up/down" could be a part (a widget) inside of this suite.

fubhy’s picture

StatusFileSize
new20.16 KB

Hello everyone. I've worked on this for some more hours and ended up with something that is actually worth sharing. Remember: This is only the API, no Fields / Widgets to actually use in the frontend. In order to add votes you simply need to create a pool and play with the vote_init() function in order to store some random votes. Recalculation is set to real-time (can be set to cron too) so you will be able to see the results immediatly (remember to activate (checkboxes) functions for the pool).

The storage layer is now completly pluggable (the default storage method already is a sub-module).

fubhy’s picture

StatusFileSize
new20.17 KB

There was a minor bug with the definition hooks... Fixed that in the attached file.

fubhy’s picture

I am now implementing two different types of DrupalQueueInterface implementaions. One for the case in which the caching is set to "real-time" and one for the case in which the caching is set to "cron".

The first one doesnt use the database to queue the updated constellations for caching (and thus reduces the ammount of required insert queries when adding a new vote) the second one uses the database for queueing the constellations for recalculation during cron.

Cron will always check for db queue entries in case the recalculation method is being set to "real-time" even though there are still items in the db queue. However, if a constellation is being recalculated by the real-time queuing method (because it overlaps with one of the items in the db queue) those items are being removed from the cron queue.

Makes sense in my head.. :P

fubhy’s picture

StatusFileSize
new21.27 KB

Update

Implemented DrupalQueueInterface and started with implementing views.

bojanz’s picture

Very interesting project. Keeping a close eye on it.

fubhy’s picture

Views implementation done and changed the vote result storage so that it now stores the result values seperatly for each function (vote_result_count, vote_result_sum, ...).

Shadlington’s picture

Subbed!

fubhy’s picture

I started a discussion on the fivestar project page to talk about the next layer (frontend) for this stuff.

http://drupal.org/node/1098638

eaton’s picture

Just a heads up that fubhy and I are talking a bit about the potential for turning this into a 3.x branch of VotingAPI. Further bulletins as events warrant...

sun’s picture

Stumbled over this issue by accident, which is not good.

1) Can we close #1098638: Discussing a possible future of Drupal voting in favor of this issue/discussion?

2) Can we move this issue to VotingAPI's issue queue?

3) While it sounds like differences and API design details have been discussed privately, I'd really love to see a comparison to current VotingAPI to make more sense of this effort - that would tremendously help everyone, before participating in discussions and looking at the code. It seems like such a document could be titled "VotingAPI 3.x", and outline the issues/problems/todos/targets the community can work on.

4) Related to 3), there's also #1105768: Discussing Vote and VotingAPI

5) Although I generally prefer and recommend to join forces on existing projects, it sounds like vote.module introduces a new, formal, first-class entity type "vote" including fundamental Field API integration, so moving developments into a new project/module namespace "vote" makes sense from a architectural standpoint. If @eaton is happy with this effort, process, collaboration, direction, and ultimately, duplication, then I'd suggest to elevate the current sandbox project now.

fubhy’s picture

Hey sun,

Regarding 1) Yes I think we should close the fivestar issue for now since first of all we need to figure out how the API itself will work in the future. Discussions about frontend implementations like fivestar can follow. I might have been a bit too fast there.

Regarding 2) This issue is basically the project application so it should not be moved.

Regarding 3) This approach of voting is based on the ideas that eaton introduced with VotingAPI but every single line is newly written code. I could surely write a documentation to show the differences between both APIs and also provide some pro con from my point of view (although both, you and eaton, see one of my big "Pros" as a "Con" -> Pools).

Regarding 4) Yes, that post was created in order to talk about the architecture of the code without abusing this issue.

Regarding 5) The approach is fundamentally different, thats right. However its still just an API that doesn't provide any Field API functionality (the entity type "vote" doesn't use Fields either). The API that I wrote does all the stuff that eaton's API does while also providing a administrative interface for creating vote pools and configuring them and other major enhancements like:

1. Queueing of newly added votes with DrupalQueueInterface (either in a static or as a "reliable" (db) queue).

2. Storing vote results in seperate tables in order to allow custom result functions to handle data differently ("score" for example needs an additional db column).

3. Definition hooks for everything:
- vote storage,
- vote types (defaults: "percent", "points", "options"),
- functions (defaults: "sum", "score", "average", "count")
- result queries (for optimizing the performance by reusing queries across different functions - e.g. sum, average and count can use one query together)

4. Custom and extendable configuration options for pools and for functions, provided by the module that introduces them.

5. Potential support for external database storage (mongodb, etc).

6. Votes are entities

7. Pools are bundles for entities and can be configured on the admin UI. Functions can be activated / deactivated for each pool seperatly. (This is what you guys evaluated as a con... I see it as a pro. The argument that other modules should have full control over the settings is reasonable and supported by me but this can still be achieved by providing functionality for this in the background. (Take "Fields" module as an example... "field_create_field" could be "vote_create_pool" for us.

laura s’s picture

Status: Reviewed & tested by the community » Fixed

This has been RTBC for quite some time now. Open questions now seem be around not quality of work, but rather where to put the work, about which @eaton and @fubhy seem to be in active discussion. Approved. :)

joshcanhelp’s picture

Status: Reviewed & tested by the community » Fixed

Just to throw it out there... I'd be happy to help with anything I can with this project. I can develop (I'd call me "intermediate") or test or whatever is needed. I'm launching a site this month (personal project) that needs a coherent, easy-to-work with voting system and Fivestar is not the easiest to work work. It would probably be easier to help this get built than it would be to try and hammer another module into submission!

Let me know!

Status: Fixed » Closed (fixed)
Issue tags: -module, -voting, -vote, -project, -votingapi, -drupal 7, -votes

Automatically closed -- issue fixed for 2 weeks with no activity.

avpaderno’s picture

Component: new project application » module
Status: Closed (fixed) » Fixed
Issue tags: -module, -voting, -vote, -project, -votingapi, -drupal 7, -votes

I am giving credits to the users who participated in this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.