Companion issue to #2714529: Add source and destination IDs to the data returned by getMessageIterator() .

Once that lands, drush mmsg doesn't need to print the source_ids_hash column.

Patch coming soon, stay tuned.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dww created an issue. See original summary.

dww’s picture

Title: If getMessageIterator() includes useful IDs, don't print the source_ids_hash column. » If getMessageIterator() includes useful IDs, don't print the source_ids_hash column in drush mmsg command
Issue summary: View changes

Clarified title and summary that this is about the drush mmsg command.

dww’s picture

Assigned: dww » Unassigned
Status: Active » Needs review
FileSize
700 bytes

Postponed might be a more appropriate status here, although the way I wrote this it doesn't depend on #2960209 landing first, and will happily work whether you're running a version of core that includes that fix/patch or not.

heddn’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Can we just load/print the source ids if they don't exist? And needs tests so needs work.

dww’s picture

Can we just load/print the source ids if they don't exist?

Sort of. I suppose we could stop using \Drupal\migrate\Plugin\migrate\id_map\Sql::getMessageIterator() entirely and write our own version that does the same query. But that seems evil and wrong, and will break if the migration's id_map isn't being stored in SQL (not sure if that even works, but presumably if this is all so class-tastically flexible, it's intended to work).

And needs tests so needs work.

Can you be more specific? What exactly would you want tested here? It's a trivial fix to the UI. Please clarify and I'll see what I can do.

Thanks,
-Derek

dww’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests

Upon closer inspection, I see no existing test coverage for any of the drush commands. I have no idea where to start with that. Seems unfair to hold up this trivial fix for adding entirely new test plumbing. If you want to open a new issue for "add test coverage for migrate_tools drush commands" I'll try to help, but for now, I'm resetting the status on this. Hope you agree.

Thanks,
-Derek

mikeryan’s picture

If #2714529: Add source and destination IDs to the data returned by getMessageIterator() gets in, a change to migrate_tools shouldn't be necessary - that patch will remove the hash value from the getMessageIterator() return.

dww’s picture

Except we can't always remove the hash from getMessageIterator(). See #2714529-18: Add source and destination IDs to the data returned by getMessageIterator() and #19.

That said, patch #20 would break this, since we would no longer be able to assume consistent column aliases in that case. :( Not entirely sure the cleanest solution to that. Input welcome.

heddn’s picture

Status: Needs review » Needs work

With drush 9 released, shouldn't we support it too?

dww’s picture

Maybe, although I've never been able to get Drush 9 working on any site where I've tried it, so I'm the wrong one to ask. ;)

Plus, it seems like the parent issue is stalled upstream, so I don't know when/if it's worth trying to fix drush to benefit from it.

Perhaps this is a more appropriate status.

heddn’s picture

benjifisher’s picture

alison’s picture

With #2714529: Add source and destination IDs to the data returned by getMessageIterator() , I'm quite content with the drush mmsg command output (see example below) -- honestly, it's still nice to have the info, don't y'all think?

Perhaps a new feature could be to allow including/excluding columns in mmsg output...?

.............................
Example mmsg output: (I removed characters from the hashes so that it's easier to read here) (maybe that's why ppl want this column gone haha... but, I maintain that it's nice to have all the details!)

$ drush mmsg cwd_node_announcement
 src_nid   dest_nid   source_ids_hash                                                   level   message                                                          
 361       361        a72023a31f42db5d759e1b42846261a2a6  2       DOMDocument::loadHTML(): Tag aside invalid in Entity, line: 11   
 361       361        a72023a31f42db5d759e1b42846261a2a6  2       DOMDocument::loadHTML(): Tag article invalid in Entity, line: 17 
 361       361        a72023a31f42db5d759e1b42846261a2a6  2       DOMDocument::loadHTML(): Tag section invalid in Entity, line: 18 
 361       361        a72023a31f42db5d759e1b42846261a2a6  2       DOMDocument::loadHTML(): Tag aside invalid in Entity, line: 47   
 2631                 bef59f390c51b9b6ce3febdd764150af2f  1       Referenced file does not exist
benjifisher’s picture

Status: Postponed » Needs work

Now that #2714529: Add source and destination IDs to the data returned by getMessageIterator() has gone in, we can un-postpone this issue. I am not sure whether it should be NW or NR, but I guess NW for drush 9 support (Comment #9).

heddn’s picture

NW seems fine since this needs to work w/ Drush 9 (and 10)

heddn’s picture

What's need here? Now the core patch has landed, is this a simple fix?

dww’s picture

It was once a simple fix. ;) Then Drush released 2 more major versions, and apparently we're trying to support all of them.

I honestly haven't looked at any of this code in ages. Not sure why patch #3 is drush-version-dependent. Not sure what else needs to happen here, if anything.

#13 claims the current UI is fine (including the incomprehensible hash value). I disagree, but I don't really care that much.

So, either "Won't fix" if we believe #13 that this UI is good as-is.

Or figure out what, if anything, needs to change about #3 to work with All The Drushes(tm).

There's also the question of "needs tests", but as I pointed out in #6 we had (still have?) 0 test coverage of the drush commands, so I don't think we should be adding all new test plumbing just for this.

YMMV... ;)

Cheers,
-Derek

heddn’s picture

Status: Needs work » Needs review
FileSize
5.05 KB

Let's see if this works for folks.

dww’s picture

+++ b/src/Commands/MigrateToolsCommands.php
@@ -499,8 +499,7 @@ class MigrateToolsCommands extends DrushCommands {
-   *
-   * @option csv Export messages as a CSV

@@ -515,15 +514,16 @@ class MigrateToolsCommands extends DrushCommands {
-    'csv' => FALSE,

@@ -537,29 +537,58 @@ class MigrateToolsCommands extends DrushCommands {
-    if ($options['csv']) {
-      fputcsv(STDOUT, array_keys($table[0]));
-      foreach ($table as $row) {
-        fputcsv(STDOUT, $row);
-      }
-      return NULL;
-    }

Removing the CSV option seems out of scope. Is that included here by accident?

heddn’s picture

I can remove that part into another issue. Drush itself supports the CSV feature natively with more features. But it can be another issue.

heddn’s picture

FileSize
1.15 KB
4.42 KB
heddn’s picture

FileSize
4.84 KB

Needed a re-roll.

heddn’s picture

I'll give this a few more days to wait for community feedback. Then I'll commit it if no one has adverse feedback.

+++ b/tests/src/Functional/DrushCommandsTest.php
@@ -131,4 +129,40 @@ class DrushCommandsTest extends BrowserTestBase {
+    $this->drush('mmsg', ['fruit_terms'], ['format' => 'csv']);
+    $expected = [
+      [
+        'level' => '1',
+        'message' => 'You picked a bad one.',
+        'source_ids' => 'Apple',
+        'destination_ids' => '1',
+      ],
+    ];
+    $expected = <<<EOT

This first $expected isn't needed. Can be fixed on commit.

  • heddn committed db08cf6 on 8.x-4.x
    Issue #2960221 by heddn, dww: If getMessageIterator() includes useful...
heddn’s picture

Status: Needs review » Fixed

Thanks @dww for you contributions.

dww’s picture

Thanks! Sorry I didn't get a chance to actually test this again for real. Appreciate you taking the time to finish this off.

Cheers,
-Derek

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.