Jump to:
| Project: | Migrate |
| Version: | 7.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed (won't fix) |
Issue Summary
I've been adding migrate support to a host of UberCart related things:
#620812: Support importing UberCart product fields with migrate.module
#625866: Support importing uc address records with migrate.module
#626380: Add migrate.module support for uc_order (orders, order product details, etc)
The bulk of this work is mostly cut+paste coding to copy nearly identical functions around, and then fill in data that's already declared in hook_schema(). :(
Instead of this:
<?php
function uc_order_migrate_types() {
return array(
'uc_order' => t('UberCart order'),
'uc_order_product' => t('UberCart order product'),
);
}
function uc_order_migrate_fields_uc_order($type) {
return array(
'order_id' => t('UC Order: Order ID'),
'uid' => t('UC Order: User ID'),
'order_status' => t('UC Order: Order status'),
'order_total' => t('UC Order: Order total'),
'primary_email' => t('UC Order: Primary email'),
'delivery_first_name' => t('UC Order: Delivery: First name'),
'delivery_last_name' => t('UC Order: Delivery: Last name'),
'delivery_phone' => t('UC Order: Delivery: Phone'),
'delivery_company' => t('UC Order: Delivery: Company'),
'delivery_street1' => t('UC Order: Delivery: Street 1'),
'delivery_street2' => t('UC Order: Delivery: Street 2'),
'delivery_city' => t('UC Order: Delivery: City'),
'delivery_zone' => t('UC Order: Delivery: Zone code'),
'delivery_postal_code' => t('UC Order: Delivery: Postal Code'),
'delivery_country' => t('UC Order: Delivery: Country'),
'billing_first_name' => t('UC Order: Billing: First name'),
'billing_last_name' => t('UC Order: Billing: Last name'),
'billing_phone' => t('UC Order: Billing: Phone'),
'billing_company' => t('UC Order: Billing: Company'),
'billing_street1' => t('UC Order: Billing: Street 1'),
'billing_street2' => t('UC Order: Billing: Street 2'),
'billing_city' => t('UC Order: Billing: City'),
'billing_zone' => t('UC Order: Billing: Zone code'),
'billing_postal_code' => t('UC Order: Billing: Postal Code'),
'billing_country' => t('UC Order: Billing: Country'),
'payment_method' => t('UC Order: Payment method'),
'data' => t('UC Order: Data (serialized array of extra info)'),
'created' => t('UC Order: Created timestamp'),
'modified' => t('UC Order: Last modified timestamp'),
'host' => t('UC Order: Host IP address'),
);
}
...
function uc_order_migrate_fields_uc_order_product($type) {
return array(
'order_product_id' => t('UC Order: Product: Order Product ID'),
'order_id' => t('UC Order: Product: Order ID'),
'nid' => t('UC Order: Product: Node ID'),
'title' => t('UC Order: Product: Title'),
'manufacturer' => t('UC Order: Product: Manufacturer'),
'model' => t('UC Order: Product: Module (SKU)'),
'qty' => t('UC Order: Product: Quantity'),
'cost' => t('UC Order: Product: Cost'),
'price' => t('UC Order: Product: Price'),
'weight' => t('UC Order: Product: Weight'),
'data' => t('UC Order: Product: Data (serialized array of extra info)'),
);
}
...
?>It'd be incredibly slick to be able to just say something like this:
<?php
function uc_order_migrate_type_tables() {
return array(
'uc_order' => array(
'name' => t('UberCart order'),
'table' => 'uc_orders',
'module' => 'uc_order', // If undefined, defaults to the module implementing hook_migrate_types()
),
'uc_order_product' => array(
'name' => t('UberCart order product'),
'table' => 'uc_order_products',
),
);
}
?>and then migrate.module can just invoke hook_schema for the appropriate module, harvest the data for the declared table (including field names, types, descriptions, etc), and do the right thing. In fact, via drupal_write_record(), we could almost completely automate these functions the same way:
hook_migrate_import_[type]()
hook_migrate_delete_[type]()
...
We could still support the current approach for people who like doing it manually. But, in cases where you basically just want a direct drupal_write_record mapping into the Schema API table definition, this would save a *LOT* of busy work and duplicate effort.
Thoughts?
-Derek
Comments
#1
Totally see the use case for something like this, in fact this could probably be handled as it's own module. migrate_tables?
However, most of the migrations we have now try to go though existing APIs in the module itself (user_add, node_save), instead of directly inputting to the database. This is useful because things like permissions, hooks, default settings, validations, etc all get a chance to work.
Still, I do think this could be useful functionality.
#2
In a migrate 2 context, this might be a code-generating drush command...
#3
Not schema API related, but here is what I did on my current project. My sources are generally single table migrations and I pull all fields in that table. Here is what I put in one migration class and in the base class that all migrations extend (welcom.inc). Note that I am using MSSQL source which can't pull Fields from the PDO object.
.../custom/welcom_migrate/content_profile.inc | 2 +-
.../all/modules/custom/welcom_migrate/welcom.inc | 12 ++++++++++++
2 files changed, 13 insertions(+), 1 deletions(-)
diff --git docroot/sites/all/modules/custom/welcom_migrate/content_profile.inc docroot/sites/all/modules/custom/welcom_migrate/content_profile.inc
index de7ae7c..48a9fb7 100644
--- docroot/sites/all/modules/custom/welcom_migrate/content_profile.inc
+++ docroot/sites/all/modules/custom/welcom_migrate/content_profile.inc
@@ -22,7 +22,7 @@ class ProfileWelcomMigration extends WelcomMigration {
);
$query = 'SELECT * FROM PROFILE ORDER BY Username ASC';
$count_query = 'SELECT COUNT(*) FROM PROFILE';
- $fields = array(); // TODO
+ $fields = $this->getFields('PROFILE');
$this->source = new MigrateSourceMSSQL($this->db, $query, $count_query, $fields);
$this->destination = new MigrateDestinationNode('profile');
diff --git docroot/sites/all/modules/custom/welcom_migrate/welcom.inc docroot/sites/all/modules/custom/welcom_migrate/welcom.inc
index e3e9f9e..9870496 100644
--- docroot/sites/all/modules/custom/welcom_migrate/welcom.inc
+++ docroot/sites/all/modules/custom/welcom_migrate/welcom.inc
@@ -56,4 +56,16 @@ abstract class WelcomMigration extends Migration {
// Some tables have dirty username columns.This SQL cleans them.
$this->clean_username = "RIGHT(CreatedByUsernameFK, (LEN(CreatedByUsernameFK) - CHARINDEX('\', CreatedByUsernameFK))) AS clean_username";
}
+
+ function getFields($tablename) {
+ $sql = "SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.Columns where TABLE_NAME = '$tablename'";
+ mssql_connect($this->db['servername'], $this->db['username'], $this->db['password']);
+ mssql_select_db($this->db['database']);
+ $result = mssql_query($sql);
+ while ($row = mssql_fetch_array($result)) {
+ // Note: a string value is required.
+ $cols[$row['COLUMN_NAME']] = '';
+ }
+ return $cols;
+ }
}
#4
I think this basically got done over in #1004812: Generic table destination plugin. I'll leave it to others to mark this as a duplicate.
#5
Yes, I agree - #1004812: Generic table destination plugin addresses this need.