Mass import from Salesforce to Drupal

Hi
I would like to evaluate your module. I will be working on salesforce Drupal integrationin an upcoming project. Will be good to have this handy.

Many thanks
Jay

I think this would also be useful for a project we are working on and I'd be glad to help test or whatever is needed.

I think this is a great idea... our current thinking is that salesforce would be the authoritative/master source for data, it makes sense to be able to do this. Having this tool would be great for people who have a salesforce org and want to add drupal and all its benefits as a community/customer front end.

This is a candidate for inclusion in salesforce API core once it is RTBC.

A patch to update the import to use the overhauled fieldmap system

Surely it is going to waste resources on web server if there are thousands (or tens of thousands) of records to import ? Isn't it better to use the workflow/trigger approach and push data from salesforce when changed in salesforce ? I can see that it might be useful to have a way to perform an initial load of the drupal objects from salesforce, but after this it is better to make the updates to drupal objects event driven - e.g. if a change is made in salesforce, that change needs to be pushed into drupal, and if a change is made in drupal, that change needs to be pushed into salesforce. This would ensure the two are kept in sync and the data transfer and sync would be efficiently using system resources.

I've used this and the import works really well on initial import. I need to further sync nodes on update from SF however (only one way, from SF --> Drupal). Looks like #12 helps with that. But this patch works as is, IMO.

Hello all,

New to drupal so apologies if I am doing something silly.

I have installed this plugin and when importing all looks good until I get the following error:

# Could not find import function sf_Document_import
# Updated node

Please note a few things. I am actually using Open Atrium which as you may or may not know is a quite heavily customised version of Drupal for Intranet use.

What I am essentially wanting to do is sync from Salesforce into Drupal/Open Atrium so that I can make documents availible to users of our intranet without having to log in.

Any help appreciated. If you need more info please just ask.

Many Thanks

Hi Arron, thanks for the quick response.

I have tried 2 mappings. Documents and contacts each with a few simple field mappings just for testing purposes.

Modules installed are simply salesforce module (incluiding sf_user, sf_prematch,sf_node,sf_import & salesforce_api) and ldap auth (which works but I am using Drupal/Open atrium local admin account)

We are running salesforce enterprise

I havent updated versions so havent read the README for them. I will have a look.

Is there a basic mapping I should try first? or limits on the mapping? I have uploaded our wdsl file and have some custom fields.

Many Thanks

Gavin

Hello Arron

I am using the latest version of sf_import

I am using the latest version of salesforce module (just got this server going yesterday)

Just to confirm the field maps. I went into fieldmaps, match the objects and select the fileds. When saved this field map is availible in the import tab.

Gavin

EDIT: Apologies. I may have wasted your time. It was a problem with field mapping. I have created new fields in Drupal and it appears to work.

Apologies for this.

While testing the module I found a mistake.

line 193 $map['index'] should instead be $map['fieldmap']

189  $context['results'][] = ($created ? 'Created' : 'Updated') . ' ' . l('node ' . $nid, 'node/' . $node->nid);
190    if($nid) {
191      $context['sandbox']['imported']++;
192      if($map['automatic']) {
193        salesforce_api_id_save('node', $nid, $record['Id'], $map['fieldmap']);
194      }
195    }

I created a patch for the emptyvoid's change.

Anyone have a tar of the latest?

Is this working?

Can the import be set to sync on cron, or at least daily?

I think this module is awesome and very very useful. I use it all the time on a 6.x-2.0-alpha2 site with no problems.
Yesterday I tried on 6.x-2.x-dev, but with all the new changes it didn't work. Has anyone else tried?
I really think that it should be included in the salesforce module especially if "At its essence, Salesforce module treats SalesForce like a data silo.".

Any thoughts?

Sure thing, here is a tar of my current version of this feature. It is currently running against the:

; Information added by drupal.org packaging script on 2010-04-03
version = "6.x-2.x-dev"
core = "6.x"
project = "salesforce"
datestamp = "1270253676"

Release and has no problems, I successfully imported 10,000 records with it.. looking to extend it with module or better yet write a new module sf_sync to trigger an update, compare periodically on cron run. I read a few issue posts about the notion of queuing transactions. I think it would be a good idea for performance reasons to limit cron syncs to specific times of the day (1am - 5am) and only run syncs on low load times. I guess technically you could setup a slave server to asyncronously issue create, update, and deletes to a dedicated database server and off load the sync so it isn't running on your primary HTTP Drupal instance. Anyway here is my current version of the import module, thanks to @aaronbauman for the original source.

After further investigation I just realized that the import as the name implies only imports business objects from Salesforce into Drupal as nodes. The import doesn't map a bridge between the imported nodes and the matching business object in Salesforce.

After a little research I have determined that the "map bridge" is created in the following table "salesforce_object_map", so I am going to work on extending the import module to optionally enable the setup of the "mapping bridge" during the one time import.

My use cases is running a one time import of (say 10,000 account business objects) but I want the nodes in Drupal to be bridged to the Account objects in Salesforce) so if a field value is updated in Drupal I'd like it to already know that the node is bridged to a object in salseforce and push the update (more on that later with the sf_sync module).

Similarly if you create node reference CCK fields to associate content types like account and contact or account and recordType the current import process checks if a "node map" exists (but how would it if you never created the map on import?) So when importing an account you (currently) will never get the recordType value to associate to your imported record type nodes. :( Sure I could manually add the mapping for 3-20 records, but as my example implies as would go mad having to enter 10,000 mappings by hand.. ack!

Pretty sweet once I get the GUI and methods extended then I'll provide a working version here. This will definitely be useful with the sf_sync module I plan to write next week. I'll post my notes and a .gz of my current code as I make progress.

I have attached an updated version of the sf_import module this update fixes several errors that were generated in the code. Also as my previous post states, #29 when importing the system will not link the node to the Salesforce business object unless when defining the field map you check the "automate" option. By checking the "automate" option for a Drupal/Salesforce business object it directs Drupal to push new nodes into Salesforce and send updates from Drupal into salesforce automatically. There may be cases where you want to import the nodes into Drupal and link each to an object in Salesforce, but don't want the nodes to force an update to Salesforce.

I have added an additional option to "link" nodes to the associated business object without having to enable the "automate" option within field mappings.

Pardon my ignorance, I'm interested in trying this out, where can I find the latest version of the module?

Cheers,
Daniel

I don't see that this fix made it into the dev code. Specifically that part where if none of the automatic settings are selected but "Link" on the mass import page is selected that no links are created.

It looks like the function call to salesforce_api_id_save() was moved from sf_import.module into sf_user.module and sf_node.module. Because $extra array (which contains the extra-linked field value) is not passed to sf_node_import() and sf_user_import(), the value of the "Link" field on the import form is not respected.

We could do one of the following:

1. Pass a new function parameter set to TRUE to force a link
2. Pass the $extra array as a new parameter to each function. This allows for expansion of the contents of $extra

I vote for #2. Either way I can do a patch quickly.

I think an $options array is a good idea for now, if you can put together a patch that would be great!

Attached is a tested patch to allow the node and user "Link" checkbox to work during a mass import.

Thank you for this - committed: http://drupalcode.org/project/salesforce.git/commit/bf8dd68