This issue is a part of Projects quality initiative. The goal is to implement a simple version of project reviews (and maybe ratings) still on Drupal 6, keeping it as simple as possible for upgrade to D7 later.

Steps:

1. Create new node type "review"

Fields:

  • title
  • body
    - text area labeled "Review"
    - required
  • project
    - node reference to point to the project the review is about
    -- this would be pre-populated based on a final argument to node/add/project-review, e.g. http://drupal.org/node/add/project-review/views or something
  • core version
    - use the same taxonomy that project_release nodes use for the core version (to target the review to at least a version of core)
    -required
  • project version
    - node reference to point to a specific release node of the project (required or optional needs to be decided)

2. Create view for reviews listings

View of review nodes that takes the project ID as an argument, e.g. at node/N/reviews
- displayed as a separate page with a link to it on the right sidebar.

Something simple as this:



3. Create block on the project node with

- total number of reviews
- link to reviews listing
- "Add review" link which takes you to node/add/review/[project-short-name]

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

tvn’s picture

Issue tags: +projects quality

tags

webchick’s picture

Priority: Normal » Major

The goal is to implement a simple version of project reviews (and maybe ratings) still on Drupal 6, keeping it as simple as possible for upgrade to D7 later.

YAY!! :D

I'm elevating this to at least major, possibly critical, as this is the #1 requested feature from our larger community, and a huge site builder improvement. So very excited to see some movement on this.

Random notes (I am not up on various discussions, and am at a sprint and don't have time to read them atm so apoloigies if this re-iterates some stuff):

1) We absolutely need to have ratings, alongside reviews. The "standard" for this sort of thing is mobile App stores. As close to this model as we can get this, the better off we'll be in terms of accessibility/approachability for both technical and non-technical users.

2) That means none of this "rating across N axes" stuff. It's way too complex, and it adds huge additional mental burden for a reviewer. I want to be able to do "Rating: 4-star, Review: Worked for me, but ran into a few bugs. [more details]" I do not want to have to feel like I need to spend 1.5 hours carefully analysing the issue queue activity, documentation, etc. of a project in order to give those fair ratings. I'm just not going to rate anything in that case, and I feel I have useful feedback to offer, so that would be a shame. As a reader, that's also the information I want; a general idea if this project is worth downloading or not, and if it's better or worse than another, similar-sounding module.

3) I would highly recommend only tying this feature to modules with stable releases. Rationale:

a) People aren't terrified to put their code up on Drupal.org before it's completely 100% perfect. They can go ahead and have the same (low-ish) barrier to contribute as they have now and gradually over time get the module into shape and then expose it to the community for review.

b) It encourages end users to go after modules with stable releases instead of modules with only unstable/beta/dev releases.

c) It encourages module developers who want people to use their stuff to actually roll stable releases so people do so and not get stuck in beta-13 hell. :)

webchick’s picture

Oh. And also? Some people are jerks. We will need to make sure to link to the dcoc from the review form and also have a lightweight moderation process around this. I'd recommend a "Was this helpful yes/no" on each review and a View that shows the most unhelpful comments. I think it will be trivial to find 4-5 people to reload that page every once in awhile and unpublish/delete accordingly.

Senpai’s picture

Issue tags: +drupal.org D7

Scope Creep Warning: Please keep this initial build *very* tightly controlled by a Feature so that there's not a significant amount of D7 Upgrade work to do. Architect it to accommodate all of these cool ideas, yes, but don't build any more functionality than you have to prior to the jump.

dww’s picture

Title: Implement project reviews and ratings » Implement project reviews
Category: task » feature

A) I'm -100 to a generic 5-star rating. It's utterly meaningless, as I've written many times in many places. If multiple axis rating is too much of a burden, then let's not do it at all. Just because people expect 5 stars doesn't mean we should give it to them. "Be like Apple" only gets us so far, especially when our terrain is so very different from Apple's (as I've also explained many times). Also, at least in my original proposals, the multi-axis stuff was optional. You don't *have* to rate it on all questions. You can post a review with no ratings. Or a review and only answer 1 or 2 questions (e.g. "works as designed" and "ease of use" or whatever you're familiar with). But if you're going to say "4-star" IMHO you should have to say WTF you're referring to. ;) I'm not totally set on the exact questions, but having 5-stars floating completely disconnected from a metric you're rating is a big fail to me. I'll probably be overruled on this, but I'm going to keep fighting for now.

B) Therefore, I think we should split ratings into a separate issue so that the reviews aspect doesn't get bogged down. We could at least implement a review node type now, and hash out a workable ratings solution in the meanwhile. I think the review part is fairly uncontroversial, so let's move that forward. We can always add the ratings field(s) retroactively, and then folks can go back and edit their reviews to add ratings if they care. In fact, #50605: Add user ratings for projects is already open with a lot of discussion. I think it's legit to keep working on ratings there instead of starting a whole new thread about it. I'll reply there in a second.

C) Agreed with comment/point 2.3. (OT: this is why I label my points inside a comment with letters not numbers -- gets really confusing otherwise.) ;) However, it's a bit tricky to do that cleanly. You might have reviews on a site without releases at all. Also, what happens if you have 6.x-1.3 but only 7.x-1.0-beta3? Do you get to add reviews that point to this project or not? Are you saying we should restrict the choices for the project version selector to official releases, only? This sounds easy but is a bit complicated in practice.

D) Totally agreed with comment #3. I think up/down voting on reviews is a clear win, even if generic ratings on the projects themselves is not. I also like the idea of a link to the dcoc (although that should be site-specific submission guidelines, not something in the feature itself). And I also like a management view of the least helpful reviews for troll-fighting.

Thanks,
-Derek

webchick’s picture

It's not "Be like Apple." It's be like Apple, Android, Amazon, WordPress plugins, Mozilla/Chrome extensions, and every single other well-known rating system on the entire planet that anyone's familiar with. I don't think 5-star ratings are "useless" at all, and the popularity of these other platforms proves this. The goal of 5-star ratings is "Do other people generally like this or not?" "WTF you're referring to" is in the review. I'm completely fine with ratings being required to have reviews. But making ratings both harder to create and harder to discern the answer to that basic question gets a huge -100 from me. I'm also more than willing to keep fighting. ;)

However, does this feedback/argument actually belong over at #50605: Add user ratings for projects?

webchick’s picture

Sorry, just read the whole comment instead of getting annoyed at the first part. ;) I'll comment over there later.

tvn’s picture

I've updated issue summary to reflect only reviews.

"Only tying this feature to modules with stable releases." - sounds like a great idea, though I see from dww's comment that it won't be easy to implement. We are going for only what's easy to build and easy to port to D7 right now. :)

"Was this review helpful? yes/no" is in plans (see http://drupal.org/node/1580230) however this issue is about the very basic version of reviews.

tvn’s picture

Issue summary: View changes

update to remove ratings

webchick’s picture

"You might have reviews on a site without releases at all. Also, what happens if you have 6.x-1.3 but only 7.x-1.0-beta3? Do you get to add reviews that point to this project or not? Are you saying we should restrict the choices for the project version selector to official releases, only? "

Well I think even in the "simplest thing that could possibly work" edition (and I'm on board with starting with that), it's necessary to select the version to which the review applies, because this is incredibly useful metadata in the reader discerning whether or not the review is valid to their situation. For example, on Amazon.com, reviews are for the 1st edition of a book, the 2nd edition of a book, or whatever. So I would make "Release" a field on the review form, and treat it like we treat that field in the issue queue where if a project has no releases the field just isn't displayed, but if it is there, it's required.

Then in Drupal.org customizations, we would put a hook_form_alter() (and possibly hook_menu_alter()) with our own custom logic there to further filter the list. But to answer your question, in a situation with 6.x-1.3 but only 7.x-1.0-beta3, you could post a review against 6.x-1.3 only.

webchick’s picture

And, as promised, rant about the ratings system posted over here. ;) http://drupal.org/node/50605#comment-6136756

chx’s picture

I am going to abandon every module of mine I am not paid to maintain the minute this goes in effect. Same for ratings.

chx’s picture

So, it's hard enough to keep up with issues and now I need to moderate reviews too? Will I get to delete every nonflattering one keeping the occassional flattering too? If I don't what motivates to continue when my hard work is dissed without base? Are we adding paid modules too? Why dont we leave these things to an app store which we are not? Do you see reviews on github? Why not? Guess.

Edit: I unfollowed this issue as well. I had my say, you do whatever you want, but #11 applies. I won't be alone. Poor Relation module :(

rerooting’s picture

Just throwing this out there, I don't think project reviews are a very good idea. Quality is something that is decided by reviewing code, checking issue queues for open bugs, testing modules - generally assessing the health and quality of a module and whether it fits in with your use case. Have you ever looked at the android market reviews? They basically become the issue queue for inexperienced users.

Inexperienced d.o. users will use the reviews as a place to report bugs and give bad reviews to good modules because of competency issues. I predict that, if this is implemented, views will level out after a year with a 2-3/5 star rating and a bunch of shoddy reviews because the greatest amount of reviewers will be people who can't figure out how to use it and will just go on d.o and angrily give it a 1 star rating (ok maybe this is an exaggeration but, really!).

I've given this thought over the years, and at times thought 'gee it would be nice to have reviews' but I don't think that burdening maintainers with moderating these comments is a good idea, especially considering that we are somewhat strapped for capacity on that front as it is!

Additionally, leaving discussions of the quality of a module and its architectural strategy to the issue queues is more productive - it may lead to patches or strategic changes for the next version.

Lastly, theres the issue of critical mass - many of the best modules I have found lately are fairly new or 'undiscovered' and thus have installs numbering in the hundreds or less. These will be left without reviews, and folks, when searching based on reviews or ratings, tend to overlook items without reviews or ratings, that or distrust them. Will I have to rate/review every up and coming module just to ensure that folks will use it, help test it and help it to grow? Well... yes... but it'd be a pain!

I think it's best to invite more proactive involvement with the community on d.o - have a problem? Report it. Can't figure out what your supposed to do? Support or feature request for documentation. Found a bug? Report &/or patch it. That's how we roll!

rerooting’s picture

Ok, I'm being a bit of a drama queen about this, haha.

The most important additions, if both reviews and ratings must happen are:

  1. Reviews should have 'this review was helpful' +1/-1 widget (user driven moderation!)
  2. Review form should make it very obvious that bugs, support requests, etc. can and should be submitted in the issue queue
  3. Ratings should be of specific categories - documentation, stability, etc. possibly with one average overall rating (forces users to decide what about the project they are reviewing!)
  4. More priority/focus is placed on project statuses, with more options (i.e. Needs Documentation)

If possible, reviews should be skipped outright and we should just pass to the ratings. If reviews are implemented though, #1 is very important!

Mark Trapp’s picture

Came in here through the Twitter announcement, and I don't know how much my opinion matters since I only maintain two small modules on Drupal.org, but just to echo chx in #11, implementing reviews would be reason enough to mark them as abandoned and move development to GitHub.

Drupal.org users are not my customers: I give back and maintain code when I can because that's the right thing to do, not to make sure the modules are popular. Moving to GitHub ensures I can continue to do that without having to deal with reviews.

Perhaps this review system can be opt-in on a per project basis instead?

Garrett Albright’s picture

While feedback is a great thing to have from a developer's perspective, I don't think the model put in place by the Apple App Store is a good goal. So much illiteracy. So many people posting one-star reviews because of stupid reasons or doing stuff like "add feature X and I'll give it five stars" as if holding a hostage.

I think that perhaps review writing permissions should be linked with a role granted to people who have a clue. My initial thought was that only those who themselves have contributed modules can review others', but on further consideration feedback from "site builders" who can't code themselves would be useful as well. But allowing just everyone to post these things may easily cause more noise than signal, just like App Store reviews.

Alternatively, do away with numerical ratings altogether, as others have said, and go with just text reviews with "Helpful? Yes/No" links and have downvoted reviews be hidden by default.

RobW’s picture

I agree with most of the sentiment here. It would be great to have a way to see at a glance which modules are considered best in their category, and by itself # of reported installations doesn't do this so well. But reviews and a single 5 star rating won't help either -- a single star rating isn't specific enough, and reviews have a huge potential for misuse (creating a less-savvy issue queue, moderation difficulties, all mentioned above).

Drupal's consumer base isn't the same as Apple's, it isn't an app store for casual walled garden product users. We have our own strengths to play to, and we might not be able to simplify everything to a single button. "As simple as it can be while being as complex as it needs to be" to paraphrase (I think) Milton Glaser.

The problem here is how do people know how much the community likes a module when browsing through so many that do similar things? There are more options than reviews and 5 star ratings, e.g.:

  • Plus one model -- only records how many people find a module useful, no punitive reactions.
  • # of times installed -- Doesn't work when you aggregate all Drupal modules together, but if they were divided into categories, or were part of a larger "feature compare" table, could be good information.
Garrett Albright’s picture

Via chx on the Twitters, a relevant article about the personalities that come into play when it comes to app store reviews. Some NSFW language.

webchick’s picture

Well, I'm certainly open to tweaks on this basic idea (e.g. opt-in/out of ratings/reviews on a per-project basis), but the current situation we have is absolutely untenable for the future of our project.

People with extensive Drupal experience are capable of gauging the health of a project like http://drupal.org/project/ctools vs. a project like http://drupal.org/project/flexinode based on a variety of factors: the author and how good of a reputation they have in the community, the number of 'fixed' vs. 'active' issues in the issue queue and general issue queue activity, a quick code review to check for things like coding standards compliance, utilization of Drupal API functions and best practices.

An end user, particularly one new to Drupal, has absolutely none of those capabilities at hand. To them, all of these projects have exactly equal importance, equal quality, equal likelihood of being the solution they need or of destroying their site. The only clue they have is more people use module X than module Y, but that says absolutely nothing about how others are using the module and whether or not it fits their needs.

As a result of this, evaluating between multiple competing or overlapping modules takes hours, if not days. And as a result of that, the community overall has taken an extremely hardline view on duplication/collaboration that drives away would-be contributors in droves. Both of these factors together are doing nothing less than destroying our project's future.

So there's a very good reason that this idea was ranked #1 of all the possibile improvements we could make to Drupal.org. We need to implement it. Let's discuss how to do so in a way that gives end users the tools they need to be funneled towards only the best projects, while at the same time respecting our developer community.

RobW’s picture

"Evaluating between multiple competing or overlapping modules takes hours," and there should be information brought to the fore to help make that decision, I totally agree. But how do reviews fix this? How do they give the same information as "the author and how good of a reputation they have in the community, the number of 'fixed' vs. 'active' issues in the issue queue and general issue queue activity, a quick code review to check for things like coding standards compliance, utilization of Drupal API functions and best practices"? Will reviews let a user know whether Address Field and Geo Field modules together are a better fit for their specific project than the Geolocation Module?

We need to fix the "how do I choose between modules" problem. I think that's what's behind the "we need reviews" upvotes on improvements to Drupal.org. If there are compelling reasons that show reviews can do this, then let's implement them. But I don't think they can do it alone, and especially not by introducing them as an analog to App store reviews.

Choosing a Drupal project is more complex than picking a completed, tested, stand alone app from the app store. Are there good examples of an ecosystem like Drupal that uses project reviews?

I don't want to naysay without offering alternatives to the problem. IMO, exposing metrics we have that already mean something, making them more approachable and accessible, would be a partial solution to the problem. Metrics like:

  • Authors and contributors, and their other projects (e.g., "Webchick, also maintains Drupal Core and Quiz. 1800 commits on 26 projects")
  • Commit activity (maybe in a graph)
  • Issue Queue activity (same as above)
  • Full versions available or only devs/betas
  • # of installs, in relation to projects in the same category
  • And maybe plus ones or "watches", a positive only single point response.

All of these could be aggregated into a little "Project Health" section. If other's think this is a good idea, I would be happy to mock it up in another issue.

I hope I am not coming off as confrontational. I remember starting out in Drupal three years ago, and how much I had to research outside of D.O. to figure out that I needed Pathauto, Global Redirect, and Views. I want to contribute to making future user's lives easier, for sure, but also not abandon or hobble the center of Drupal, the developers who put so much into the projects we all use, and without undercutting or dividing work between our awesome issue queue and a new project feedback space.

webchick’s picture

Choosing a Drupal project is more complex than picking a completed, tested, stand alone app from the app store. Are there good examples of an ecosystem like Drupal that uses project reviews?

Over at #50605-62: Add user ratings for projects I detailed four examples of other projects directly in Drupal's use case that use ratings/reviews/some combination thereof to help end users with extension choices, including one from the community (of distribution authors) itself.

And I don't think reviews alone fix this issue, apart from detailing more about how a module is used which can be helpful, which is why I see reviews and ratings as coupled. I didn't have any say in this issue being split into two confusing parallel threads, unfrotunately.

However, I don't think adding even more metrics that an end user has to be able to parse and judge each one as to whether or not they are good or bad is the answer here, although that seems to be the road we've gone down so far with exposing usage stats, downloads, and new sparklines stuff the Project team is working on. I think the key is giving them one single metric, which they can use as overall guidance, and then dig down into more details if they want it.

The original e-mail from Dries on this topic way back in 2006 or something had a simple red / yellow / green indicator based on a number of different factors like you've outlined there. But that feature request has been open (somewhere in the Project queue I think) for over half a decade now, and therefore seems to be a great deal more complicated to design and implement than a tried and true ranking system that people are already familiar with on other sites, and that already has very familiar patterns for dealing with gaming/abuse.

RobW’s picture

Neither Wordpress, Mozilla Addons, or Joomla Extensions have a public issue tracker like our issue queue. And development on all of them is either private or centered around Github. That's a big difference, and gives rise to some important considerations that people have raised here. Github has an issue tracker, and certainly doesn't have reviews, but it's users are developers. We can't say "look at the Wordpress Plugin Directory, that's obvs the right way" or "look at Github, that's obvs the right way", because Drupal is in a middle ground between the two.

"I think the key is giving them one single metric, which they can use as overall guidance, and then dig down into more details if they want it." -- completely agree. I'm not saying we should add more metrics, I'm saying we should make the ones that the savvy users check now in order to make decisions, the ones we all agree matter, more accessible to newer users. Aggregating them into a single-glance Project Health meter with more specific info below for those who want it seems like a great idea. Coupled with the issue queue this could be much more powerful than the Wordpress/Mozilla/Joomla ratings + five star system.

And I'm not arguing blanketly against some sort of non-issue queue user feedback and sharing area. "Will reviews let a user know whether Address Field and Geo Field modules together are a better fit for their specific project than the Geolocation Module?" Thinking more about this situation, I can see something like reviews being very useful. A user writing "I used X and Y to build a feature that does Z" would be fantastic. But that doesn't fit under "Reviews" exactly, it's more like "User Experiences".

Drupal contributors, our current user base, the people who make the stuff that less savvy users want to use, are worried about adding reviews. Seriously, the issue queue and collective development/contribution workflow is the greatest thing about Drupal. I and some others think that reviews would impact it negatively. I think we need to take a closer look at the problems we're trying to solve, throw out options, and show on paper first that whatever solution we decide to implement will affect the whole system as positively as possible.

RobW’s picture

I just read through a lot of the other Projects Quality Initiative posts, and a ton of these issues go back to 2006-9. I even saw a couple of version changes from 4.7 -> 5.x. Agh, Webchick, I feel your pain now. Things need to happen and not just get stuck in the endless "no, not perfect yet..." cycle.

Reviews I think are too problematic. User experiences, or that idea phrased some other way, I think could be useful. Anything we do needs to direct bug reports and participation back toward the issue queue.

rerooting’s picture

A few ideas:

  • Provide some kind of tool to streamline the building of comparison charts, especially where project information can be easily updated. These can be useful but can end up outdated really quickly. They need their own space!
  • Enhance 'related modules' to also include 'recomended companion modules' - solves geofield/addressfield usecase (dont forget geocoder!). Allow users to suggest them along with +1s (this could be a toggleable project specific feature - imagine how many views companion modules there are!) Related modules tends to include either other projects the maintainer/company has worked on or funded, or similar ones that are older/usually deprecated. Inviting a folksonomy of interrelated projects would be great.
  • Project macros! Like the issue number macros, but for projects. They could show Title, status, last commit, reccomended branch. This would make discussions about modules that are a bit older way more useful. To make it less complex, one could leave sandboxes out of the picture and just use machine names, such as [p:ctools]
  • In general, an overhaul of the documentation sections to allow users to explore modules in various ways could be really helpful.
  • 'Use cases' or 'Real-World Examples' could be specific 'recipies' for in-demand capability - location maps, slideshows, etc. Each would be related back to the modules that are used, say using relation. There are many of these kinds of articles already in the documentation, though some are outdated.
  • Integration of some of the articles from the Drupal Planet feed with the Documentation section might be nice as well. Some of these articles are really helpful!
RobW’s picture

@rerooting

Some of these ideas are really good, but suggestions like overhauling documentation are definitely out of scope of this implementation issue. "'Use cases' or 'Real-World Examples' could be specific 'recipies' for in-demand capability": Features are going to be allowed on D.O. soon, so I think that can be combined with "related modules".

Just to keep things constructive, let's keep alternate suggestions directly related to reviews and ratings, and with at least the same ease of implementation.

tvn’s picture

Some ideas in #24 are really good but they should be discussed in other issues/g.d.o topics, some of them have issues opened already. This issue is specifically about implementation of the *very simple* version of the project reviews while d.o is still on Drupal 6.

As webchick pointed out in her comments, request for reviews/evaluation system for projects is around for a long time already and is top 1 on various community votes, so we should start doing something about it sooner than later. Of course reviews alone will not fix all our problems, they are just one step in making it easier to choose projects. I hear concerns raised in this thread but I am not sure that adding reviews will only and obligatory have such a negative impact. I went though reviews for a couple of modules on http://drupalmodules.com right now and haven't seen anything too scary, for example referred earlier Views project has very positive reviews and pretty high ratings also.

I think we should accept that reviews system is not ideal while needed, start with basics, see how it works and adjust it when/how will be necessary. We also can and should discuss various improvements of the system, I do like some of the proposed ones:
- 'Was this review helpful?' widget and hiding not helpful reviews (again this is in plans just not the part of this issue)
- Possibility to flag review as a spam/insulting etc. to quickly report them to moderators
- "Review form should make it very obvious that bugs, support requests, etc. can and should be submitted in the issue queue"
- Impose some limitations on who can write a review e.g. user should be registered for at least 2 weeks, user should have at least 5 posts/comments etc.
- List reviews written by user on his profile page
- Remove possibility to post reviews if user's reviews got X down votes as unhelpful or were flagged as spam/insultive
etc.

RobW’s picture

Those are some good, well thought out checks and balances. If they were implemented with reviews, I think we would avoid a lot of the concerns here.

rerooting’s picture

+1 on #26. These should be fairly easy to implement with rules, flag, etc.

Apologies on #24, I will investigate extant efforts and join in on other conversations or start new ones

Raysunil’s picture

on the link http://drupal.org/node/1580230 describe the field
optional rating fields:
works as intended
well maintained
ease of use
documentation
But on http://drupal.org/node/1637534 link there is no describtion about the field. Also in http://drupal.org/files/project-reviews-ratings-mockup-v1.png , there is rating section. However, in http://drupal.org/node/1637534 , nothing has been maintion i need a clarification.

tvn’s picture

SSDsunil, http://drupal.org/node/1580230 is a general plan of the Projects quality initiative. But we won't implement everything listed there right now. This issue (http://drupal.org/node/1637534) is about implementing basic version of reviews for projects (without ratings).
At first we were discussing the mock-up with ratings section, but later discussion about ratings moved to #50605: Add user ratings for projects and mock-up changed to http://drupal.org/files/project-reviews-ratings-mockup-v2.png

eigentor’s picture

As I read on the project team week notes volunteers are missing to implement this.
I wanna help out with this. Having figured out the d.o. sites workflow I would like to throw in my hat and help out. Could start from July 16th.
I guess I have to speak to tvn?

tvn’s picture

eigentor, the dev site is now ready: http://reviews-drupal.redesign.devdrupal.org/
Let us know if you have questions or need some help.

Raysunil’s picture

FileSize
3.06 KB

Here is an attached module which is in vary basic stage,we need some time to finesh it, we think we can done by the end of the week . Please review and give your feedback.

cweagans’s picture

People with extensive Drupal experience are capable of gauging the health of a project like http://drupal.org/project/ctools vs. a project like http://drupal.org/project/flexinode based on a variety of factors: the author and how good of a reputation they have in the community, the number of 'fixed' vs. 'active' issues in the issue queue and general issue queue activity, a quick code review to check for things like coding standards compliance, utilization of Drupal API functions and best practices.

I'm a little late to the game here, but it sounds like we should take what experienced Drupal developers do and automate it, essentially displaying a sort of "project health" gauge. I don't want to derail this conversation, but I'm really in the same boat as chx and dww: a 5 star rating system with comments really is not going to be that helpful. Earlier in the issue, somebody made the point "add x feature and I'll give it 5 stars". A lot of people generally don't know how to objectively review a piece of code, so let's let a computer do it instead.

There are already a lot of things that we could take into account:

- Amount of time between most recent commit and now (more = worse)
- Ratio of closed to open issues
- Recent maintainer activity
- Number of people using the project
- Number of automated test assertions, and whether or not they are all passing

I mean, there are a lot of things that we could take into account.

I feel very strongly that we should not make this a user-driven review because people will do crazy things with it. The rating system may work okay for the App store, but I don't think it fits here.

cweagans’s picture

One more thing: there's no reason that we can't *display* the project rating as a five star widget, but I think it should be calculated from data, not completely user driven.

If we really must get users involved, I would really like to see their votes count for only a portion of the final project rating.

eigentor’s picture

Assigned: Unassigned » eigentor

I'm on it now. Content Type generated. Got stuck with the view yesterday, am only a semi-pro with arguments and so the arguments dot not want to work correctly.

Assigned the issue to me, since this is only about implementing the outlined structure. Discussing the if and how of project reviews has its own issue I guess.

eigentor’s picture

Got the view with arguments basically working now.
http://reviews-drupal.redesign.devdrupal.org/admin/build/views/edit/proj...
At least when entering the arguments manually on the Views admin page :P
Currently wrestling to get the block to display on the project page and to get the Page view that lists all the reviews for a given project, which would be project/webform/reviews

eigentor’s picture

Status: Needs review » Active

Some more progress on this.

Page View for reviews

The Block

- total number of reviews
- link to reviews listing
- "Add review" link which takes you to node/add/review/[project-short-name]

I do not get to display on the project page itself, but it displays on the reviews view page instead. Not quite sure why.
The block itself is only a proof of concept. I did it as a view, since it was easiest for me to get the links. It also has a template views-view-fields--project-reviews--block-2.tpl.php, because I did not know how do display the node count in a different way.
This is crappy coding since I am a terrible coder. Someone who knows what he's doing should probably custom-code this block, should not be too hard.
The links to the reviews page and to create a new review come out weird, since a drupal.org is prefixed befort the paths, I don't know why this happens.

As for the prepopulation of the node reference field with the project from which the review is created: This also needs to be done by a decent coder. I only would know how to do it with http://drupal.org/project/nodereference_url/ and I guess we do not have that on d.o.

Fields of the content type

  • To make the body field required, I had to add it as a custom text field.
  • I made the Project reference also required
  • I think I could not choose to make Core version required or not, since the vocabulary itself is always required anyway, I think in D6 it can be only chosen per Vocabulary
  • I guess the project version field would be populated from a view that preselects the available release nodes for that project. The Autocomplete behaves strange and only offers five to ten results without being able to scroll down. http://screencast.com/t/pi1Gzbez
  • The Autocomplete for the project version needs some clever Pre-selection. There are by far too many releases to get the user to understand which one to choose. Also you have to start with the project name, e.g. "views 7.x" instead of just entering the release number
eigentor’s picture

Status: Active » Needs review
tvn’s picture

Status: Active » Needs review

eigentor, thanks for working on this, looking good. I am sorry, was too swamped to comment earlier. Now that DrupalCon Munich is next week we could discuss this further there, what do you think? Perhaps let's setup a BOF on project reviews and ratings? We still need some discussion on how we are going to implement this so that both site builders and module maintainers are happy.

eigentor’s picture

Yeah good idea.Lets get together over this in Munich.

tvn’s picture

webchick’s picture

Notes from the BoF session:

1) We need to add a checkbox to the project node to allow maintainers to turn on/off this feature. Upon launch, this would default to false, so that we could roll it out selectively on only a few projects, but after a couple of weeks we'd default to on. Project maintainers who didn't want this feature simply turn it off, like they can their issue queues.

2) Only allow reviews against *stable* releases. This both incentivizes people who want lots of people to use their modules to roll stable releases, and prevents the natural progression of "still in dev, not ready yet" from causing pre-emptive reviews.

3) On the modules listing page, after a month or so of this feature being rolled out, change the listing page to sort by rating instead of usage (or possibly some combination; Neil mentioned an algorithm).

Specific implementation notes:

- The permissions to add a review aren't currently on for auth. users on Drupal.org so it's impossible to test.
- There's a block/link missing from the project page to add a review.
- Once you submit a review, you end up on a "lonely node" with no connection back to the project page and/or other reviews.
- On the review add form, change Version from an autocomplete field to a drop-down field, which defaults to the latest recommended release.
- Ditch (or if it's necessary, hide and auto-populate) the "Core compatibility" field.
- On the reviews listing page, allow reviews to be filtered by major branch (6.x-1.x, 7.x-1.x).

webchick’s picture

This BoF was attended by webchick, jthorson, tvn, eignitor, dman, sutharsan, drumm, Andrew Suth, and Cary Gordon for awhile. :)

RobW’s picture

Great compromises in there, Webchick. +1 for sorting by a combination of rating and installs.

RobW’s picture

For future reference, here's an article about a private company's experiences with five star and histogram ratings. I'm not super into their splatter graph solution, but the foundational ideas are worth a read.

goodfil.ms/blog/posts/2012/08/22/why-ratings-systems-dont-work/

TL;DR: don't just give your audience information -- give them meaningful information that is directly applicable to the choice at hand.

Garrett Albright’s picture

- On the review add form, change Version from an autocomplete field to a drop-down field, which defaults to the latest recommended release.

What releases are available for review? Only the currently-recommended ones, I hope?

RobW’s picture

Yes, that's already noted in a comment above.

drumm’s picture

Version: 6.x-1.x-dev » 7.x-2.x-dev

Let's do this on D7.

eigentor’s picture

@drumm so I best start from scratch in D7.
I've made Screenshots of the configuration so far, so I do not have to rethink everything.

Please destroy the old D6 site and create a D7 one to work on.

tvn’s picture

Status: Needs review » Active

eigentor, D7 d.o is in heavy development right now, issue pages get some finishing touches. So it'll be couple more weeks before D7 dev sites are stable enough for you to work freely. We'll create one as soon as we can.

eigentor’s picture

O.K.

drumm’s picture

Issue tags: -drupal.org D7

Features should not be tagged for the Drupal.org D7 upgrade.

fuzzy76’s picture

Are there any next steps for this?

dww’s picture

I think the next steps (which thankfully can happen in parallel):

1) Upgrade d.o to D7
2) Figure out WTF we're doing about d.o governance: #1929526: Drupal.org Software Working Group Charter (DRAFT)

I don't think this issue is going to go anywhere useful until both of those are complete. I guess d.o doesn't have to be launched and running D7, but the D7 port needs to be basically done before we are going to be able to give this proper attention. Someone could in theory be working on this in parallel, but I know that myself and the other Project* maintainers are all focused on the D7 port, not new features like this.

-Derek

fuzzy76’s picture

I guess #55 is pretty much fulfilled, atleast very soon. :D

tvn’s picture

Community vote on ideas for Drupal.org 2014 roadmap is open this week. If you want to see this feature implemented - vote here: https://groups.drupal.org/node/312828

tvn’s picture

Issue summary: View changes

updating image

drumm’s picture

Status: Active » Closed (duplicate)

While this issue has lots of good information, I’m going to have to leave #50605: Add user ratings for projects open since it is the older issue.

dww’s picture

Status: Closed (duplicate) » Active

I think you're conflating reviews with ratings. Reviews as elaborated here are to be text-based comments where people can share their experiences about using a given project. Ratings are about distilling your thoughts/feelings into some kind of quantifiable metric(s) (e.g. N-stars, etc). They're definitely not the same thing, and although both are potentially relevant to project quality stuff, and you can envision a scenario where you have to add a review to provide a rating (e.g. basically how Yelp works), they're not the same thing, both have their own pros/cons, and someone might want one, the other, or both. I don't think these two issues are duplicate at all.

eigentor’s picture

Assigned: eigentor » Unassigned

Uh oh this was still assigned to me. Major misleading, haven't been working on this for years.