A suggestion for reducing new-duplicated modules : require RFC or statement of intent posting

Overall, I think this is a good idea. From what I can tell, AjK does a Herculean effort on the CVS applications to try to prevent duplicate projects. However, yes, as you point out, this only counts for the first module someone intends to contribute. There's no peer review or moderation for additional projects, and that's led to quite a few dups (to the great dismay of AjK, I should add).

The biggest complaint I have with this proposal is that if we go with this: "I don't think we need a technical enforcement angle" then your proposal, while good intentioned, is basically useless. ;) Speaking from experience, if you give project maintainers any freedom at all and document things how you'd really like them to be, the docs will be ignored. Period. Unless we enforce some kind of moderation queue on new project nodes, you might as well be talking to yourself here. As the person most likely to have to write some code to facilitate such a moderation queue, I have a personal interest in *not* suggesting this, but I'm just being honest...

The secondary concern I have is who's going to review this moderation queue? However, given the interest in the Hall of Shame, AjK's own incredible efforts, and the general community-wide aversion to Yet Another Duplicated Module, I bet we'd have no trouble keeping watch over this queue and processing it in a timely manner.

I won't spend time thinking about or writing up technical ideas for the enforcement angle, yet. Let's see what others have to say, first...

Well, an easier first step to a moderation queue for project nodes would be a big fat red message on http://drupal.org/node/add/project-project telling people to 1) search if a similar project already exists that could be extended upon and 2) create the "statement of intent" at least one week prior to module creation.

And a huge +1 to the proposal in general. This could also lead to some general discussion prior to the project creation, advise could be given etc, forces could be joined etc (currently, when starting a new project and not blogging about it or mailing drupal-dev, a maintainer is usually on his own until folks get to know about the thing, which can take quite some time. And the "new projects" feed isn't really that informative either as many people don't put a extensive description there at the time they create the project node).

Sure, we can try that. Adding (unlikely to be read) text to node/add/project-project can be done with no code at all via the admin UI over at: http://drupal.org/admin/content/types/project-project

So, once we get a few more d.o admins to sign off on this, all we really need is:

1) a place where we want these proposals posted (presumably a new forum, unless there's a better suggestion)
2) relevant docs located and updated
3) specific text for the message we want on the project add page
4) notification sent to the devel@d.o list, the next issue of the CVS users' newsletter, etc.

We can see how it goes for a while and then consider adding teeth...

Also, I want to bump this over to the "redesign" component so that those folks notice this issue and consider if/how they want to work this into their plans. ;)

Cheers,
-Derek

I'm slightly concerned about this - I thought there was no policy about duplicate modules. I had talked to greggles about it, and he said that Dries said "leave 'em alone". I tend to agree that duplicate modules are bad (witness the whole Friendslist fiasco with mercmobily, which would have been a perfect case scenario for this approach).

As one of the CVS app approvers, I have to echo what Morbus said. We try to discourage dupes but when Dries himself says competition is good and don't deny them for it, all we can do is encourage people to work together.

As for friendlist, nothing short of an outright ban would have prevented that. He knew he was duplicating a project which itself was one of several dupes and he did it anyway. His justification is that his is better and that it would have been too hard to fix one of the existing ones. I'm not going to get into the argument over whether that was true or not but the point is that, as long as people come into module dev with that mindset, nothing short of requiring modules not be dupes is going to fix the problem. And if Dries is against that idea, we're rather stuck.

Having said that, though, I think having people announce their intentions for new modules would be beneficial to help those who simply aren't aware that the functionality exists. So that will cut down on the unintentional duplication. But it's not a final solution to the problem. Unfortunately, I don't know what is. :(

Michelle

With regards to the whole process of reviewing CVS account applications, the basic process I currently follow is:-

• Insist on a download link within the application for review (yes, I review them all now). The review is NOT about coding standards or functionality. The purpose of the review currently is to verify that the potential contributor has the basic security understanding in place (correct use of db abstraction, filtering output, etc). However, reviewing is made much easier if basics of the coding standards are followed. No download link now always results in a decline with a referral to http://drupal.org/node/59.
  This was put in place in an attempt to stem the rising tide of security related issues (and almost all either SQL injection or XSS issues, i.e. the basics) the Security Team has to deal with.
• With regards to duplication the basic policy (if you can call it that) is that duplications are discouraged but not forbidden. This often results in something of a fine line walk when reviewing. This policy was born out of a conversation between Dries, Gerhard and myself more than 12months ago. Given the explosive growth of Contrib in that time period maybe this is back on topic. I will say that back then, Gerhard and I had proposed the removal of CVS account holders to be granted the automatic right to create a project node and replaced with a more community based moderation system (I was and still am happy to manage that). However, after much talk it was decided back then that that would hinder Drupal's growth. Again, "back then" there just were not as many people interested or concerned about the situation for a community solution to actually function within any reasonable time scale that would satisfy the applicant.

However, we have moved on since then. We now have more then six times as many Drupal Contrib modules than when that discussion took place. The existence of this issue proves we also have people interested and involved in this process of project creation. So may be it is about time this discussion was revisited.

I do get asked quite often in IRC as to why I reach the decisions I make and so far people have accepted what I have said at face value. Derek, thanks for the kind remarks. I believe Michelle and I an ok job "at the front door" of Contrib.

I'd like to hear both suggestions/comments regarding both the initial CVS application process and for the project node create process but I think it's the later case that's currently causing the most issues/concerns. I'm all ears as we Brits say :)

This is a tough situation. I can see both sides of the argument:

a) We have an insane amount of modules. Users are confused on which to use. When you have modules that duplicate another in functionality, it makes the choice that much harder. Users don't care about the underlying code (unless it's so horribly buggy it breaks functionality). They see two modules that have the same feature list and want to know what to pick. They get frustrated because there is no rating system, no reviews, nothing to really tell them what to pick. Experienced Drupallers can get a feel for the module by looking at the queue and the maintainer and maybe diving into the code. But we should not be expecting the average user, especially new ones, to do this. The rating/review system that's being tossed around and hopefully coming with the redesign will help this to some extent, but it's still better to prevent duplication to begin with.

b) Modules get abandoned or have maintainers that don't do a good job. The one that gets to the gate first isn't necessarily the best. If we strictly follow a policy of no dupes, we'd still be using flexinode. Refusing to let people re-invent the wheel hinders innovation and we'd be missing out on potentially cool new modules just because a similar one already exists.

Trouble is, how do we know when "b" is more important than "a"? I think most of us agree that CCK blows away flexinode and it was worth the pain and confusion in 4.7 to get where we are today. But is friendlist going to be so much better than user relationships that it's worth the pain and confusion it's causing now? That remains to be seen. UR has a committed maintainer and so does FL. What I see happening there is we will continue to have two modules that do basically the same thing where the main difference is under the hood where users don't see it. The question is, in the long run, will this help the community or hurt it?

I don't have the answer to this. Maybe it's a matter of getting ratings and letting competition work. Maybe it's a matter of cracking down harder and forbidding dupes. As usual, the answer is likely somewhere in the middle. Perhaps taking it on a case by case basis? Using the suggestion upstream of new modules going through an RFC? But who decides from there? Using the UR/FL case again, there are vocal people on both sides. If the argument had been in an RFC rather than after the fact when it was too late, how could we tell which side was right? Who decides?

Sorry this post was more questions than answers. I wish I had a solution, but all I have are concerns.

Michelle

Regarding "who decides" when it comes to two camps fighting it out. Experience has shown me (imho) that in this case I would allow both in. Reasons being, both modules would appear to have dedicated and committed maintainers who for love nor money are not going to work together for the better good. In that case it boils down to whoever does the best job wins the day. What we lack is the ability to judge the "winner" of such a competition. Abandoned modules are, imho, worse than dupes as unless they are absolutley perfect will gather dust and pull down the image of Drupal as a whole as it's bug issue queue grows. I would rather see these removed rather than marked "abandoned". But then, that could leave many high and dry with no alternative. Yet another problem to solve.

Given that Drupal has so many modules these days, if people would prefer an RFC approach then my solution would simply be:-

• The ability to create project nodes would be restricted to d.o webmasters
• We would have a forum where project proposals could be announced and discussed.
• As for the actual "decision" after discussion, that could lie with the CVS team (Gerhard, Zen, Michelle and myself, may be others with an interest such as Greegles, etc)
• Once a decision is reached the project node would be created and then passed over to the applicant.

I think this is basically what dman was getting at but doesn't require any new code, just a new discussion forum and a change in project node handling permissions. After all, dww spent a lot of energy putting together the much improved system we have today. I don't see it needs any more code.

But I hope that future potential for rating/review of modules will help the problems of end users actually deciding what to go with.

#9 sounds reasonable. Rather than a forum, we could make use of http://groups.drupal.org/contributed-module-ideas which is basically what that group is for, anyway, though less formalized.

Michelle

Where the discussion actually takes place isn't that important. Although I feel, imho, it should be on d.o directly rather than another site, even if it's a brother site like g.d.o

However, what's most important for any of this to work is interest in running this scheme.

Let me take you back to the conversation that took place a long time ago between Dries, Gerhard and myself and enlighten those that don't know what took place in the dusty dark back corridors of email inboxes (as opposed to the now more common public discussion like this one).

Basically, Gerhard and myself felt that there was some need for some type of moderation/approval to try and kerb dupes and, well, plain non-sensical modules that creep in after someone gets their CVS account. Dries was against this. Now, Dries didn't have to swing the heavy baseball bat of his position to win the argument. Dries position was simply this:-

1. At the time of the original discussion there were only around 350 contrib modules and dups could be handled in a simpler easier manner. Not many dups were happening but there were enough for Gerhard and myself to raise it. Dries felt differently on this issue.
2. more importantly given the rate of project node creation and the potential number of people that would be involved in discussing/approving wouldn't scale. There simply was not enough potential people back then who either cared enough or had the time to engage in such a system. This would have represented or would have become a serious bottleneck to the additions of new functionality. So free reign over project node creation continued at the sacrifice of a few problematic dups/users.

That second reasoning, at the time, couldn't be countered as it simply was a fact.

Now, there have been presented here some alternatives. I welcome more ideas and input. However, whatever you think, point 2 above is crucial to making anything work. If you believe the community is in stronger position to make it or some other scheme workable then now is the time to make it known. But at least you all now know why the current situation exists and that should make it easier for people to come up with workable viable alternatives (or not as the case may be).

I was thinking a bit more about this and I think neither that group nor a forum is a good place for this. A better place would be an issue queue. I think it should be a dedicated issue queue, not tacked onto webmasters/infra. This gives us the advantage of a centralized place here on d.o and also has the advantage of statuses. Once a decision is made, it could be marked "fixed" or "won't fix" and there would be a clear record of it.

As for manpower, that's always the tough one. I'm not very good at keeping up with the CVS apps as it is. Keeping up with the discussion of all the projects that get added on a regular basis would be even harder. There's also the question of expertise. If it's an area where I am familiar with the modules that already exist, it's easy for me to weigh in. But modules outside of that area are a lot harder to judge. We already get complaints that CVS apps aren't processed fast enough. I'd hate to see what would happen if we get a backlog of people with accounts wanting to add modules. So we would definitely need to be sure there's enough people dedicated to discussing new module proposals before implementing anything.

Michelle

The number of complaints about speed of CVS approval hasn't been bad at all. I usually manage to get through most during week days. It's when some applies Saturday morning and they don't get processed till Monday (combined with teh expectation of an instant response) that a few have complained ;)

But, as you state, point 2 in #11 eleven above is crucial to a workable system. I'm not entirely sure how many new modules are committed each day but I would be surprised if it were more than people apply for a new CVS account. But to be workable, enough people need to step up and say they'll partake in the issue queue (yes, agreed, dedicated issue queue much better than forum).

So now we let the World turn and other posters on this thread voice their desire to step up to the podium and wish us well (and their time) :)

If I can offer a different scenario:

Currently:

Company X creates the Acme Node Whammy Widget module for a client. It represents some pretty cool work, and Company X wants to contribute it to the community. This genericized Node Whammy Widget module gets contributed, and others can benefit from it or not. If it's awesome, then the community is a winner. If it's not so good, well, the code is there and maybe someone else will want to make it better ("whammier").

Proposed:

Company X creates the Acme Node Whammy Widget module for a client. It represents some pretty cool work, and Company X wants to contribute it to the community. However, before this genericized Node Whammy Widget module can be contributed, Company X has to go to a forum and "pitch" the module to the community, making module contribution something like trying to get a movie made in Hollywood.

Result:

A) People love the idea and Company X gets to contribute.

B) People hate the idea and Company X has to defend itself before being able to contribute work already done.

C) Company Z already has a module called Node Whammer Hammer, and therefore has "squatting rights" thus preventing Company X from contributing based on a "no duplicates" rule.

D) Company Z already has a module called Node Whammer Hammer and Company X is urged to "collaborate" with Company Z on a module project.

At best, the barrier to entry for contributing is raised. At worst (and perhaps very likely in frequent instances) competing modules are not judged based on quality but on who got there first.

Personally I think this is a lose/lose proposition.

@laura s:
Not quite. Your first assumption is correct but as it stands right now, (C) and (D) are also part of the current equation depending upon which way around you hold the hammer. Unfortunately the (w)hammer is heavy and not everyone is sure which way around it should be held.

It's this that's the current problem/issue.

I am against any restrictions that prevents developers to create similar modules.
I support any idea that helps users to find similar modules.

The really good thing with duplicates that they are not duplicates. They are mostly just similar modules.

example:
http://groups.drupal.org/node/18219
"Terms of Use" and "Legal" is just as duplicate as "ubercart" and "e-commerce". They have an absolutely different approach to solve the same thing. When I had to create a "Terms of Use Confirmation" function on a site I evaluated both modules. The functionality is correct in both modules, I would not say that one is better than the other. And which one I choose? I created a new module because my way of solving this problem does not correspond with those two, and I can't merge my type of solution with any of those.

Brainstorming...users can create new project pages, but they are not published by default. Admins go through and review the unpublished projects, review for duplication/GPL-violations/etc and have a chance to work with the maintainers before said projects are published/available to the community. That way every module/theme gets its own review process. I'd volunteer to help with a "review group".

Well, we have http://drupal.org/taxonomy/term/14 which is a running list of all the module project nodes as they're created. AFAIK, CVS maintainers are monitoring that (I often see people like ajk and greggles showing up in the issue queues of new modules and pointing out duplicates and such). So more people who have this particular itch could certainly do the same.

Personally, though, I would much rather volunteers' dump time and energy into making modules more findable than trying to impose restrictions on contributors. This would cut down both duplicate modules and duplicate efforts.

As much as I find dupes annoying, I don't think we'll get anywhere imposing restrictions. Sometimes the person just didn't realize a module already existed but more often they just want to do their own thing and don't care if it causes problems. If they don't want to collaborate, trying to force them is just going to blow up.

I'm hoping the redesign will include reviews and rating. Dries wants to let dupes compete, fine. That will give us a way to tell the winner. Because what we have right now is just confused users not knowing what modules to use.

So I'd say mark this won't fix and just deal with it until we have the tools we need to make sense of the dupes.

Michelle

#21 Michelle - agreed. In my opinion there are benefits to duplicated modules which outweigh the drawbacks so we should get busy dealing with the duplication rather than trying to stop it.

We have two major problems with duplicates:

1) "duplication makes modules it harder to find the right module" This can be solved by documenting the differences and a redesign to make it easier to find the right module.

How to fix: Document, or help with redesign.

2) "duplicated modules waste energy when people could be collaborating" However, in my experience most duplicates are created either because of a non-resolvable conflict with the existing module or because the module was trivial to make. Encouraging people to get over non-resolvable conflicts is unlikely to work. Encouraging people to abandon trivial modules is usually successful.

How to fix: keep track of new modules (as webchick suggested in #20) and file issues encouraging people people to join with another project or document the differences (see some examples of how I've done this in the past with varying degrees of success).

I think that competition is a good thing. In the end, it's the end-users who will benefit from it. Also, more choices mean more freedom to pick the one that best suits your own site/s. We also cannot force people to collaborate if they dont want to, and stopping them from contributing is IMO, not right. I think what is lacking are reviews and ratings of the projects. So I'm also hoping that the redesign will include that.

+1 on "won't fix"

wrote almost the same time as greggles...marking back to "by design"