I'm getting this error for each row I import:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'destid2' in 'field list': INSERT INTO {migrate_map_greavesproduct} (sourceid1, sourceid2, needs_update, destid1, destid2) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4_0); Array ( [:db_insert_placeholder_0] => 1 [:db_insert_placeholder_1] => 11300033020 [:db_insert_placeholder_2] => 0 [:db_insert_placeholder_3] => 22 [:db_insert_placeholder_4_0] => 22 ) (sites/all/modules/contrib/migrate/plugins/sources/sqlmap.inc:317)

This is a column that's missing from the map table for this particular migration. My map is declared thus:

    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'product_id' => array(
          'type' => 'varchar',
          'length' => 255,
          'not null' => TRUE,
          'description' => "Product mapping ID from the CSV",
        ),
        'gs_ref' => array(
          'type' => 'varchar',
          'length' => 255,
          'not null' => TRUE,
          'description' => "The product code",
        ),
      ),
      // Need to specify the entity type.
      MigrateDestinationEntityAPI::getKeySchema('commerce_product')
    );

...which all looks fine to me :)

I've checked the return value of MigrateDestinationEntityAPI::getKeySchema('commerce_product'), and it produces two keys, the product ID and revision ID. But there's only one destination column in the map table.

I've looked at other destination classes in Migrate, and none that I can see return more than 1 key in their getKeySchema().

Might this be a bug in Migrate that we're hitting?

Comments

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

I had that, and as I remember did a complete uninstall, removed all the map tables, and started over again and got past it. I think it may be a bad hook_update_N() in Migrate.

joachim’s picture

joachim commented
Status: Active » Closed (works as designed)

I'm not sure that's it, as my project's git log shows I updated to Migrate 2.4 long before I started work on my own migration classes. Also, the migrate_map_FOO tables aren't declared in hook_schema() at all, but built in MigrateSQLMap.

However, what I think must have happened in that I set my MigrateSQLMap with a dummy destination and then later changed it to 'MigrateDestinationEntityAPI::getKeySchema('commerce_product')'.

MigrateSQLMap only checks for tables not existing; it doesn't check if the map table is still accurate.

Which is obviously why in your case reinstalling worked.

Closing here. There's this bug report on Migrate: #1175304: warn if map/message tables are out of sync with key schema.

killtheliterate’s picture

killtheliterate commented

I circumvented this by creating the superfluos "destid2" column in the database.

killtheliterate’s picture

killtheliterate commented

rfay, did you uninstall only Migrate, or commerce as well?

killtheliterate’s picture

killtheliterate commented

After uninstalling migrate and my custom classes, the problem persists. Again, simply creating the destid2 column resolves this, but it makes me feel like there is a code goblin in there.

killtheliterate’s picture

killtheliterate commented
Status: Closed (works as designed) » Active
rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

I unstalled commerce_migrate_ubercart, commerce_migrate, and migrate. Then I made sure none of the migrate tables still existed.

killtheliterate’s picture

killtheliterate commented

Hmm. I'm not using commerce_migrate_ubercart at all, and everything else, I've uninstalled and double checked that the db was clean of any migrate tables.

What MigrateDestination are you using?

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

I just did a standard Commerce Migrate Ubercart migration.

killtheliterate’s picture

killtheliterate commented

I wonder if a dependency wound up in the now uncoupled ubercart migration module?

I've rolled back migrate and commerce_migrate to before that release, and the destid2 column error stopped happening.

jackbravo’s picture

jackbravo commented
Status: Active » Fixed

Then maybe this issue can be closed.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

stella’s picture

stella commented
Status: Closed (fixed) » Active

re-opening this. This is still happening for me with the latest version of commerce_migrate. I don't think rolling back to the release prior to the decoupling of commerce_migrate_ubercart is a solution.

chrisolof’s picture

chrisolof
he/him
Primary language English
commented

Happened to me too on 7.x-1.1 with migrate 7.x-2.5. To get past it I uninstalled all migrate modules and then re-enabled them. Now the product import works.

chrisolof’s picture

chrisolof
he/him
Primary language English
commented
Issue summary: View changes

added clarification

daletrexel’s picture

daletrexel
Primary language English
Location Minnesota, USA
commented

I just experienced the exact same problems and temporary solutions, and I found a partial answer. I've been setting up a commerce migration from CSV, and I was getting the same "Column not found: 1054 Unknown column 'destid2' in 'field list'" error. I also confirmed that manually adding the 'destid2' column to the migrate_map_ map table allowed the migrations to run (and roll back) smoothly. It didn't seem like a proper long-term solution.

The solution that DID work was to change the destination key schema in my map.

As in the example above, I initially used the generic method:
MigrateDestinationNode::getKeySchema('commerce_product','product')
which resulted in only the destid1 column in the map table.

The problem was resolved when I switched to the commerce product method:
MigrateDestinationCommerceProduct::getKeySchema('commerce_product')
resulted in both destid1 and destid2 columns in the map table.

There are plenty of commerce migration examples out there that use the former syntax (including examples that come with Commerce Kickstart), so I'm not sure why it's a problem for some and not others. Like the example above, I have two source key fields in my map (the previous system's ID and SKUs), but tests with just one or the other consistently resulted in the missing destid2 column, which makes me think that's a red herring. It's only when I switch to using MigrateDestinationCommerceProduct::getKeySchema() that I've consistently gotten my migration to work and provide the destid2 column that seems to be required.

So, I guess the question is whether it is a problem that MigrateDestinationNode::getKeySchema() sometimes fails with commerce migrations, even when passed the commerce-specific parameters. Is it OK to use the generic method when you are doing a Commerce migration if it happens to work, or should you always use MigrateDestinationCommerceProduct::getKeySchema() out of principle? At a minimum, it seems that some additional documentation could help reduce confusion.

I've been testing with the latest migrate (7.x-2.6 via git), commerce_migrate 7.x-1.1, and Commerce Kickstart 7.x -2.12

daletrexel’s picture

daletrexel
Primary language English
Location Minnesota, USA
commented

On further reflection, I see that I overlooked something rather important in my previous comment. I'll leave it posted as is, rather than editing it, to illustrate how one error can lead to a problem that is difficult for a novice (like me) to diagnose.

The problem with my previous comment was that I confused my initial use of MigrateDestinationNode::getKeySchema() with MigrateDestinationEntityAPI::getKeySchema(), which is mentioned in the initial bug report. The latter is actually much closer to what I ended up successfully using, MigrateDestinationCommerceProduct::getKeySchema(), which on further testing and reading into the API, appears to be interchangable with MigrateDestinationEntityAPI::getKeySchema() (at least for their getKeySchema() methods).

It was late when I filed my previous comment. I'd spent most of the day trying to learn and understand migrate, first by building a very simple CSV page node migration, and then converting that to a simple product variant migration. The page migration used MigrateDestinationNode::getKeySchema(), and I overlooked the importance of updating this when converting it to a commerce migration. I was further confused by the fact that simply changing the destination getKeySchema() method and a cache clear was not enough -- the map tables don't dynamically adapt, but have to be dropped and re-added. I ended up doing this by disabling and re-enabling my commerce migration class.

Looking back at the comments in this bug report today with a fresh eye, I see that the crux of the issue was resolved by the second comment. The subsequent comments seem to suggest that the problem, its cause, and its resolution, remain difficult for people to understand. You really have to dig into the migrate modle, key schemas and what's going on with the associated map table to understand this bug. And if you are like me, a novice building a migration, making and correcting mistakes along the way, it's very likely that you'll end up in this scenario where your existing map table no longer works for your migration's destination class (especially if you're building up to a commerce migration) -- you don't fully understand why, and you don't know what all is required to fix the problem.

The biggest problem seems to be a lack of testing to confirm that the map table matches your destination key schema. The user is given no feedback that anything is wrong until they try to run a migration, and by then, it's too late. (In my case, I could not roll back the product variants created by the failed migration, but had to delete them manually.) It would be great if the map table's structure were compared to the destination key schema sometime earlier -- at cache clear, migration registration, or checking status (via UI or drush ms). I believe this is what joachim is getting at in reference to bug #1175304

My inclination is to say that this bug can probably be closed (again), but see if there are ways the documentation could be improved to better explain the implications of using the right destination key schemas (and how to fix problems created by using the wrong destination key schema in the first place).

Sorry for the long essays in a bug report!

nanak’s picture

nanak
Primary language French
Location Sarlat-la-Canéda
commented

For what it worth, the correct destination is:
The destination should be set to

$this->destination = new MigrateDestinationCommerceProduct('commerce_product', 'product');
MigrateDestinationEntityAPI::getKeySchema('commerce_product');

where 'product' is your product type.

You should then have the destid1 bound to the product_id, and destid2 bound to the revision_id

quietone’s picture

quietone commented
Status: Active » Closed (outdated)

Drupal 7 is End of Life.