Commitlog should provide RSS feeds of each view

Testing Issue #1042574: No RSS feeds on commit views

Ok, I tried to start this, but it is no so really close to be finished.

Please take a look to FIXMEs and TODOs in the patch :-)

Let's iterate over this again.

Here a patch that actually works for commitlog_global_commit.

To implement this, I had to use a fake path on feed display,and then overwrite it on run-time(I end up there after asking merlinofchaos).

So, this still needs some work, but the basic case is working! :-)

What's new:

• Theme the rss item for operations.
• decreasing the number of FIXMEs
• Add a feed display to commitlog_global_commits

PS: Please note that as mentioned by mikey_p this need to be done for each view set. I am excluding individual commit view since I do not see any use of one-item/non-changeable rss feed. We also need to change views on other projects that are using views_set feature to overwrite behaviour: aka change versioncontrol_project views. (git backend only changes individual commit, so not change required there)

Ok, this took me a while, but I have a re-viewable patch now!

What's new:

• Make sure we have the needed fields, aka no more TODOs on row plugin handler.
• Add a views argument handler for repo_id to be able to set the title propertly.
• Rework commitlog_operations_page() to use our custom views_embed_view, versioncontrol_render_commitlog_view.
• Add feeds to commitlog_repository_commits and commitlog_user_commits views.

What needs to be done before committing this:

• Attach feeds to displays manually before rendering. I am not attaching them through views, since in that way I can not overwrite the path for the feed. That's needed because we want to support different feed displays of views on the same path(based on the view set). Please note that this problem appeared because there is not a way to declare a feed without an associated path in views.

I just realized that this is a deployment blocker, so marking as it as mentioned on #1042574-3: No RSS feeds on commit views

\o/, it was easier than I though, thanks to hefox for helping me with the last bit!

I am working on the related issue at versioncontrol_project #1056568: Provide rss on commitlog views

See the comments in the other issue

Bringing back and answering the comment on #1056568-3: Provide rss on commitlog views (here is the main patch, the one on the other issue is just a follow-up after this will be applied, so responding here)

First and most important, I think you disabled vfv query aggregation. Because I'm seeing a bunch of these queries being run in sequence:

SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (960909);
# Time: 2011-02-12T21:24:57
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:38330 []
# Query_time: 0.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (959434);
# Time: 2011-02-12T21:24:59
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:38330 []
# Query_time: 0.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (959714);
# Time: 2011-02-12T21:25:01
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:38330 []
# Query_time: 1.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (959889);
# Time: 2011-02-12T21:25:03
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:38330 []
# Query_time: 0.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (960404);
# Time: 2011-02-12T21:25:05
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:45330 []
# Query_time: 0.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT data, item_id FROM queue q WHERE expire = 0 AND name = "versioncontrol_git_repo_activity_stream" ORDER BY created ASC LIMIT 0, 1;
# Time: 2011-02-12T21:25:07
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:38330 []
# Query_time: 0.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (960799);
# Time: 2011-02-12T21:25:09
# User@Host: git_dev[git_dev] @ stagingvm.drupal.org:38330 []
# Query_time: 1.000000  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 0
use git_dev;
SELECT versioncontrol_item_revisions.item_revision_id AS item_revision_id,
   versioncontrol_repositories.name AS versioncontrol_repositories_name,
   versioncontrol_item_revisions.path AS versioncontrol_item_revisions_path,
   versioncontrol_repositories.vcs AS versioncontrol_repositories_vcs,
   versioncontrol_repositories.repo_id AS versioncontrol_repositories_repo_id,
   versioncontrol_item_revisions.line_changes_added AS versioncontrol_item_revisions_line_changes_added,
   versioncontrol_item_revisions.line_changes_removed AS versioncontrol_item_revisions_line_changes_removed,
   versioncontrol_operations.vc_op_id AS versioncontrol_operations_vc_op_id
FROM versioncontrol_item_revisions versioncontrol_item_revisions
INNER JOIN versioncontrol_repositories versioncontrol_repositories ON versioncontrol_item_revisions.repo_id = versioncontrol_repositories.repo_id
INNER JOIN versioncontrol_operations versioncontrol_operations ON versioncontrol_item_revisions.vc_op_id = versioncontrol_operations.vc_op_id
WHERE versioncontrol_operations.vc_op_id IN (960869);

Not really, that queries are now executed by the views render method on the rss plugin. And I see what is the problem, I am not using fields on the plugin, so I am rendering the individual commit view for ech operation at render level, that's the problem.

But, that should not be modifying the default display, only the feed display. If that's not the case please confirm it.

and things are really noticeably slower.

Second, the only output that's coming out for anything is

<?xml version="1.0" encoding="utf-8" ?><rss version="2.0" xml:base="http://git-dev.drupal.org/versioncontrol/garbage/path/343333" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title></title>
    <link>http://git-dev.drupal.org/versioncontrol/garbage/path/343333</link>
    <description></description>
    <language>en</language>
          </channel>
</rss>

Mmm.. I am looking the right feed as output for feed display, so maybe you need to clear views cache or something like that?

What's the garbage path for?

Please take a look to the menu callback changes. There you can see why. In summary:

• Views do not allow create feed displays without assigning a path.
• We are dynamically choosing which view to use on a menu callback, and attaching a feed related with that page, so the feed display is also dynamically picked.
• We can not rely on that paths, so instead I am overriding the path at display time.

• Re-work render to use fields and let map which fields we want on each RSS item value.
• Let operation revision, person and attribution views field handlers output plain text instead of html markup based on an field handler option.

This patch avoid the design problem I had done in the last patch by rendering by hand a individual commit view on each row :-/

BTW the row plugin has become really a generic plugin, but there is not one like that on views AFAIK.

What is missing here:

• Update all default views accordingly with the row plugin handler changes.

Ok, finally the review-able version!

What's new here:

• Remove no more used versioncontrol_operation_load_multiple().
• Let choose the link shown on each row as another views field and consecuently remove the query additions.
• Let choose the format output for operation revision views handler.
• Update commitlog_global_commits, commitlog_repository_commits and commitlog_user_commits feed displays.

Naturally, now we need to update the patch for versioncontrol_project.

Hard code-freeze left this out.

I see no drupal_add_feed() for node/{project-node}/commits

Also I'd like to note that we need some solution for redirecting from old-style URLs like http://drupal.org/cvs?rss=true&nid=3060&branch=HEAD - probably we need to update all out subscriptions in all out aggregators ... hard work but I see no other way

I see no drupal_add_feed() for node/{project-node}/commits

Yep, that's because those views are on versioncontrol_project, see #1056568: Provide rss on commitlog views for the patch that do that.

Also I'd like to note that we need some solution for redirecting from old-style URLs like http://drupal.org/cvs?rss=true&nid=3060&branch=HEAD - probably we need to update all out subscriptions in all out aggregators ... hard work but I see no other way

That should be handled on #780342: Handle linkrot from CVS commit browser.

Re-roll, after #1066160: Group entity loading in views handlers this is not applying fine.

This is a dependency of #1072762: Change for git.

Subscribe.

sdboyer: It would be great to receive your review here.

Pretty close here.

+++ b/commitlog/commitlog.module
@@ -97,53 +126,109 @@ function commitlog_ctools_plugin_directory($module, $plugin) {
+    drupal_set_header('Content-Type: application/rss+xml; charset=utf-8');

Is this the standard way of doing things? Seems like views/the display should take care of it. Is it not because of the wonky way we're directly rendering the view?

Nit:

+++ b/commitlog/includes/views/default_views/commitlog_global_commits.view.php
@@ -306,9 +306,481 @@ $handler->override_option('access', array(
+$handler->override_option('title', 'VCS commit messages');

Just 'All commit messages' or 'All commits' would be more consistent with what we've got elsewhere.

Nit:

+++ b/includes/views/handlers/versioncontrol_handler_field_operation_person.inc
@@ -7,6 +7,22 @@
+  function options_form(&$form, &$form_state) {
+    parent::options_form($form, $form_state);
+    $form['plain_text_output'] = array(
+      '#type' => 'checkbox',
+      '#title' => t('Plain text output'),
+      '#default_value' => $this->options['plain_text_output'],
+      '#description' => t('Avoid markup and links on this field output.'),
+    );
+  }

It feels to me like needing this option points to poor architecture on our part. I'm not really in a Views mindset while writing this, though...

+++ b/includes/views/plugins/versioncontrol_plugin_row_operation_rss.inc
@@ -0,0 +1,126 @@
+  function render($row) {
+    global $base_url;
+    static $row_index;
+    if (!isset($row_index)) {
+      $row_index = 0;
+    }
+    $vc_op_id  = $row->{$this->field_alias};
+
+    $item = new stdClass();
+    $item->title = $this->get_field($row_index, $this->options['title_field']);
+    $item->link = url($this->get_field($row_index, $this->options['link_field']), array('absolute' => TRUE));
+    $item->description = $this->get_field($row_index, $this->options['description_field']);
+    $item->elements = array(
+      array('key' => 'pubDate', 'value' => $this->get_field($row_index, $this->options['date_field'])),
+      array(
+        'key' => 'dc:creator',
+        'value' => $this->get_field($row_index, $this->options['creator_field']),
+        'namespace' => array('xmlns:dc' => 'http://purl.org/dc/elements/1.1/'),
+      ),
+      array(
+        'key' => 'guid',
+        'value' => t('VCS Operation !vc_op_id at !base_url', array('!vc_op_id' => $vc_op_id, '!base_url' => $base_url)),
+        'attributes' => array('isPermaLink' => 'false')
+      ),
+    );
+
+    $row_index++;
+
+    foreach ($item->elements as $element) {
+      if (isset($element['namespace'])) {
+        $this->view->style_plugin->namespaces = array_merge($this->view->style_plugin->namespaces, $element['namespace']);
+      }
+    }
+
+    return theme($this->theme_functions(), $this->view, $this->options, $item, $this->field_alias);
+  }

Could you point me to the reference implementation you used for this so I can compare?

Overall, very nearly ready to go in. As with the other patch, I'd like this pushed into a topic branch. If you have the full local commit history and could push that up (rather than just a single squashed commit), that would be handy.

Powered by Dreditor.

Is this the standard way of doing things? Seems like views/the display should take care of it. Is it not because of the wonky way we're directly rendering the view?

I think so, I actually asked for it on #drupal-views for suggestions.

It feels to me like needing this option points to poor architecture on our part. I'm not really in a Views mindset while writing this, though...

WFM, it's not that bad ;-)

Could you point me to the reference implementation you used for this so I can compare?

Sure, it's actually the only one rss row plugin on views ;-)

Overall, very nearly ready to go in. As with the other patch, I'd like this pushed into a topic branch. If you have the full local commit history and could push that up (rather than just a single squashed commit), that would be handy.

So, I think it has been so much time and so many changes, that at some point I just used one of the patches, so I do not have history anymore. Anyway I like independent commits but only when they are stable, if not it's a complete pain to use git bisect, so that's why I think it's ok for this patch to get in as one commit.

Attaching ta patch with the nitpick changed.

Subscriiiiiibe.

subscribing

subscribe

There are feeds example for drupal project http://drupalcode.org/project/drupal.git/rss , 8.x Branch http://drupalcode.org/project/drupal.git/rss/refs/heads/8.x.

Subscribing.

There are feeds example for drupal project http://drupalcode.org/project/drupal.git/rss , 8.x Branch http://drupalcode.org/project/drupal.git/rss/refs/heads/8.x.

Yep, gitweb produces feeds, but the issue here is about getting drupal side feeds for commits, getting the data from versioncontrol information.

tagging

subscribing

So, I think this is mature enough, so after re-checking it on top of current 6.x-2.x I am pushing this, so one release blocker left :-)

I know 12 files changed, 1745 insertions(+), 77 deletions(-) is a lot for one commit, but as explained before, this patch comes from CVS times, and it has changed too much from the beginning, and after the change of remotes (from my old-manual-cvsimport-git-repo to the shiny d.o migrated CVS one) I lost track of the branch. So I hope to not do this again, now on git times(tm).

So where are this feeds could be found now?

EDIT http://drupal.org/node/3060/commits still has no feeds attached

@andypost: fixed does not mean deployed on d.o :-/

We can tag for that.

subscribing

After some ugliness with another drupal.org deployment earlier this week, I'm holding off on deploying new vcapi stuff for a few more days.

Did this ever get deployed? This is a blocker for getting http://drupal.org/project/activitystream_drupalcode working again.

If it still has this tag then it's not deployed.

Changing state again. AFAIK(see drumm comment) that tag is independent from the issue status.

hey all,

I'm interested in seeing this issue committed. What needs to happen to see this move forward? Anything I can test or do?

Peace,
Jeremy

This is still unavailable, e.g. there's no indication on http://drupal.org/node/943786/commits or http://drupal.org/user/108450/track/code of an RSS feed being available.

As mentioned on comment 35:

@andypost: fixed does not mean deployed on d.o :-/

and on comment 42:

Changing state again. AFAIK(see drumm comment) that tag is independent from the issue status.

It has just been deployed!

Of interest to this thread: #1703452: Provide argument titles on provided default views.