By cpearson on

To: Best Help Available,

My team has just been tasked with the the final go-live transition and data load from a previous NON-Drupal website to, obviously, our new Drupal built site. We are unfortunately coming in at the end of the build and getting the site "cold" from a previous external developer that will be moving on as we take over admin of the site internally and in perpetuity.

Old site has common functions like:

Consumer - Dealer / Distributor lookup.
&
Authorized Dealer / Distributor sign-on for wholesale pricing.

The site will and does support retail purchasing visitors (AD-HOC) and our Dealer Network folks (ACCOUNTS), segregating informational aspects, proprietary data and differing online sales paths for each. MySQL drives the old site and data is updated from our ERP system with nightly refreshes.

The new Drupal site is supposed to offer the same functions, and does so with minor issues unrelated to this post that are within the domain of our developer. However, due to some staffing deltas with the developer they have lost some "magic" in-so-far as the **refresh** and **reload** capacities that will be needed to sync the sites when we do our live turn-up.

My team and I have poked around but we are not Drupal developers, yet we have to get this work done and we're not going to get much in the way of help due to the developer moving off the project.

"Location" module is in play for the "Dealer Lookup"; "Users" will be handling both AD-HOC and ACCOUNT visitors. We have initial, now stale, data loaded into the needed tables but again the "magic" left when developer had staffing issues. We've had good success with BRUTE FORCE table changes the the "Location" etal. tables, "content_type_dealer_location" (custom table?) and "node" etal. tables to get refreshes and inserts for new data. However, the success has not been complete and we don't get good results from our [gmap] strings even though they are identical to the [gmap] strings that are present in the roll back versions of the stated tables.

As I have now dug further into Drupal, I'm hoping to get some help with IMPORT features that I just don't know are available. We did the BRUTE FORCE work in 2 days but I fear we are missing some minor relations between tables that are causing our problems.

I don't have time to lurk around on the forums and "learn" or "find" the answer. I would hope to get directed to some great import tool.

Thx

⋅ Categories: Drupal 6.x

Comments

johnhanley’s picture

johnhanley commented

Importing directly into the database is not recommend unless you're intimately familiar with the data structure.

I have performed dozens of data migrations (including 8,000+ records just last weekend) and swear by Node Import.

Mapping the old data structure to Drupal can be tedious and time-consuming. Preparing a quality CSV file is more than half the battle. At the bare minimum you'll want a title, body and creation date (in Unix timestamp format, assuming preserving the original creation date is important.) Node Import also has the ability to map data to CCK fields and taxonomy terms so plan ahead.

I usually import in batches (500-1000 records) and backup the database incrementally. Be prepare to make several attempts before getting it right. Node Import is slow and a cycles hog so I would recommend performing the import in maintenance mode.

Good luck!

cpearson’s picture

cpearson commented

Great directions to node_import.

We've just now cracked the "magic" and had the wrong "format" value so our [gmap] strings weren't being treated right. ~10,000's of multi-table inserts to refresh location, content_xxx and node etal. while still preserving the previous state of node and node_revisions (so our roll back state can revert to pre-screwed with).

Straight PERL and MySQL to MySQL direct... Takes about 4 mins. And results are pristine.

Seems like we jumped right to intimate!

Thx