I'm not sure what to suggest as a fix though... it could be simply that removing source records from a registered migration is not supported! ;)

Comments

mikeryan’s picture

Title: removing records from CSV causes unimported count to be negative » removing records from source causes unimported count to be negative
Category: bug » support
Status: Active » Postponed (maintainer needs more info)

The unimported count is the count of available source records, minus the row count of the map table (which includes all successfully imported rows, failed rows, and ignored rows - i.e., all rows that have been attempted to be imported). Thus, if data that has previously been imported and recorded in the map table is removed from the source, the count can go negative. I'm not sure what Migrate can do in a general way about this...

joachim’s picture

Category: support » feature
Status: Postponed (maintainer needs more info) » Active

Maybe show a 0 if the result of (available source records, minus the row count of the map table) turns out negative? I appreciate the changing the source records is basically mucking about that Migrate can't be expected to understand, but a 0 (or maybe a '??') would make more sense than a negative.

jonathan_hunt’s picture

I had hoped drush ma would report on matching/non-matching ids between the migration source and the migrate_map table, but drush migrate-audit reports on the field mappings only. It would be great if there was a way to show a diff on source ids in the incoming file and the migrate_map.

andrewmacpherson’s picture

I encountered a similar problem, but it doesn't seem due to removal of source records.

In my case, I had just written a new migration and registered it for the first time, using Migration::registerMigration() during hook_flush_caches(). The migration was a class extending DrupalNode6Migration.

The Migrate UI showed zero source records available, even though I knew the source DB had 354 records. (No records were removed from the source; I was using a Drupal 6 DB dump for dev purposes, not connected to a live Drupal site.)

Nevertheless, drush mi succeeded in importing them, and the migrate UI showed minus-354 unimported. This was repeatable, so I made some screenshots during an import/rollback.

EDIT: I forgot to highlight the relevant row in the screenshots. We're looking at the migration called NodeContentPage, about 1/3 of the way down the list :-)

mikeryan’s picture

Component: Code » Documentation
Category: feature » support
Status: Active » Fixed

Having a negative unimported count is now documented in the FAQ. Because the actual negative count does impart some useful information (how many more you've imported than are currently present in the source), I'd rather keep it there than obscure it with a 0 or N/A - the best approach is to document what it means.

@andrewmacpherson - Having an incorrect source count of 0 is unrelated to this issue. It seems something is wrong with the count query, you might look at your MigrateSourceSQL() call and make sure that the third argument (if present) is either NULL or a valid query object returning a count.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.