Description, Rationale

Settings as shown on a User Name field
Relationships or other joins in Views often create "duplicate" results. For example, a node with a field that has multiple values may show up in the View once per value in the multi-value field. It's frustrating, and the "DISTINCT" option in the Views UI does not actually solve the problem because the result row is technically distinct.

This module aims to give a simple GUI method to remove or aggregate these "duplicate" rows. For any given field, including "Global: Text" fields, you can optionally mark the field as filtered ("Filter Repeats") or aggregated ("Aggregate Repeats"). All rows with the same value in that field will either be removed as duplicates (filtered), or aggregated in-line.

I created this module because I found myself constantly filtering duplicates with per-view/Entity hook_views_post_execute() implementations (we have a lot of custom entities with relationships that create "duplicate"-looking rows, but I need the relationship for some kind of contextual filter, etc.) With this module we can now easily do the same effect from within the Views GUI. This module is used on our upcoming production site, so it will be maintained actively.

Project Info

Drupal version: 7.x
Project page: http://drupal.org/sandbox/dansandj/1781290
Git repository: http://drupalcode.org/sandbox/dansandj/1781290.git
Pareview report: http://ventral.org/pareview/httpgitdrupalorgsandboxdansandj1781290git

Author's Drupal Work

Tips for ensuring a smooth review recommends giving some examples of my Drupal experience, so here's a short list:

Co-maintainer, now taking over maintenance of Local tasks blocks.

Accepted issues with patches:

Pending issues with patches:

Also, not a published issue but sent up a patch to fix an XSS issue in Monster Menus (attributed in commit b39f78e).

CommentFileSizeAuthor
views_distinct_settings.png32.4 KBjay.dansand

Comments

ro-no-lo’s picture

ro-no-lo commented

Hi,

your module sounds like a very much needed module. I know these duplicates much to well. Atm. I will only have a look at your code. Your code is very well formated and so forth. So I have only some minor things you may wanna consider.

.module
#71 you have a spell error "ouput". same in #72
#185 seems to access a undefined variable $field_name (at least I hav'nt found an extract() or double $$ statement)
#40 has a unused variable $handler
#104 $views_distinct_action is unused as well

I find the colon in most of your normal // comments unusual, but I guess that's not something the Drupal coding standards "forbid". Codewise your module is very well prepared.

lolandese’s picture

lolandese commented

Views tricks from the Drupal Watchdog:
"If you're using a field with multiple values, in many cases Views has a version of that field with a 'delta' value. Configure the Operator value 'Is equal to' to '0' -- checking to see if there's an item in the field -- and it should return one row as a result."

This Views configuration focuses on the field that causes the duplicate, whereas this module detects a duplicate value of a choosen field independently from the field causing it. A link on the project page to the mentioned trick would give a choice on whether to install the module or use a certain configuration of Views. Possibly with some directions on when to use the module and when the Views configuration.

For sure this modules is a solution many might have desperately searched for. Not everybody finds a blog post on the Drupal Watchdog. No idea if it is documented anywhere else.

jay.dansand’s picture

jay.dansand commented

@ronan.orb:
Thanks! You're right; when I ripped out and centralized the action parsing code in _views_distinct_get_view_actions() I inadvertently left $field_name in place. I never noticed because it only impacted the aggregator_separator, which defaulted to the value I was expecting. Should be all cleaned up now, including the "ouput" typos and the 2 unused variables.

@lolandese:
Thanks for the heads up, no amount of my Googling had unearthed http://drupalwatchdog.com/2/1/little-views-tricks-2. I'm looking into the trick, and once I pin down the specific field types for which it does and doesn't work (seems like just any arbitrary field handler won't work) I'll happily add the documentation for folks to make their own educated choice about when to use which method.

amysteen’s picture

amysteen commented

@lolandese You just ended my 2-day frustration headache. Thank you.... this little tidbit of information should definitely be pushed into views documentation!

hernani’s picture

hernani commented

I did not found any errors regarding coding standards.
I installed and used the module successfully.

The module seems to have some troubles to work with pagers, as they assume the number of results is the one grabbed from the database. That means if pagers are used, some pages will not display the expected number of results.

jay.dansand’s picture

jay.dansand commented

I've tried a couple of times to fix the pager, but I haven't been able to determine a practical and efficient method. The pager is built based on $view->total_rows, which I'm updating, but even rebuilding the pager (using $view->query->pager->total_items = $view->total_rows followed by $view->query->pager->update_page_info()) only adjusts the number of "pages" displayed by the pager, not the number of rows. The actual rows of results are already pulled, and the SQL has a LIMIT clause based on the desired rows.

So, the only way I can think of to fix the pager would be to eliminate/aggregate duplicate rows, increase the SQL LIMIT and re-run the query, eliminate/aggregate rows, and re-run the query with a higher limit until finally you end up with enough remaining rows to fill the requested pager count. The end output would be unpredictable too, since the module can only eliminate/aggregate rows it has been exposed to, so if a "duplicate" row exists at LIMIT + 1, then that row will also display on the next page, which seems like an error.

Until the pager can be fixed, I've added the note about odd pager behavior to the project page. Unfortunately for now, this is a "can't fix" problem. If everything else looks good, would you feel comfortable marking the project as "reviewed by the community"?

jay.dansand’s picture

jay.dansand commented
Issue summary: View changes

Updated status of LESS CSS Preprocessor issue.

windmaomao’s picture

windmaomao commented

I tried this module with views datasource (views_datasource), it doesn't seem working.

and i found $view->style_plugin->rendered_fields is empty from style views_plugin_style_json defined by views_datasource. So now i wonder what is rendered_fields ? and what style plugin should return rendered_fields ?

thanks.

jay.dansand’s picture

jay.dansand commented

This seems like a compatibility problem with Views Datasource, maybe because they haven't made a stable release for 7.x yet. I've created issue #1836542: Compatibility with Views Datasource in Views Distinct's queue, and I'll try to get to the bottom of it when time permits. Does Views Distinct still work when "Use the rendered output of this field" is unchecked in the field? Please use issue #1836542: Compatibility with Views Datasource to respond. Thanks!

scott weston’s picture

scott weston commented
Status: Needs review » Reviewed & tested by the community

Reviewed code and tested against views base installation. I created two views to test and both worked as expected.

klausi’s picture

klausi
he/him||they/them
Primary language German
Location 🇦🇹 Vienna
commented

We are currently quite busy with all the project applications and I can only review projects with a review bonus. Please help me reviewing and I'll take a look at your project right away :-)

jay.dansand’s picture

jay.dansand commented
Priority: Normal » Critical

This project has been marked "reviewed & tested by the community" for about a month, and was waiting for review for 4 months before that. Based on Application Review Times from Review process for Full Project Applications: What to Expect I'm elevating this to Critical Priority.

jthorson’s picture

jthorson commented
Status: Reviewed & tested by the community » Fixed

Yup ... it doesn't take much of a peek at the code to realize the applicant knows his stuff. It's a shame this sat as long as it did. Apologies for the delay, and thanks for sticking with it!

Thanks for your contribution, jay.dansand!

I updated your account to let you promote this to a full project and also create new projects as either a sandbox or a "full" project.

Here are some recommended readings to help with excellent maintainership:

You can find lots more contributors chatting on IRC in #drupal-contribute. So, come hang out and get involved!

Thanks, also, for your patience with the review process. Anyone is welcome to participate in the review process. Please consider reviewing other projects that are pending review. I encourage you to learn more about that process and join the group of reviewers.

Thanks to the dedicated reviewer(s) as well.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

(not verified) commented
Issue summary: View changes

Updated status of submitted issues/patches.