This collection of patches provides a UI and schema for local storage of SalesForce Object Definitions (those returned from a describeSObject SOAP call). The UI is implemented as a MENU_LOCAL_TASK in the Salesforce settings.

The motivation for this is to form a basis for a data caching layer and to alleviate some of the bottleneck and overhead around SOAP describeSObjects and describeGlobal calls. The local object definition provides a faster and more readily available way to access this data.

I'd really appreciate any feedback and am very open to refactoring any of this work.
Here are a few issues this brings up:

  1. How can we best use Drupal's caching system instead of yet another salesforce data table?
  2. Per-object configuration for SalesForce Object Definitions. What kinds of options do we need?
  3. Caching layer: this is a huge piece of pie to bite off. What's the best place to start?
  4. Refactoring existing code. Once everyone is satisfied w/ the implementation, where can we leverage the local SObject definition?

I've started on a preliminary data caching system, cron-ing new data FROM SalesForce TO Drupal, but I'm not so happy with it. I'm sure some folks w/ more experience than me have some ideas, but if not I'll open a discussion topic in the Enterprise CRM Integration group soon...

PS. the Drupal upload bot has subtly renamed my .patch files, but it should still be obvious which patch applies to what.

CommentFileSizeAuthor
salesforce_api.module.patch7.7 KBaaronbauman
salesforce_api.install.patch1.42 KBaaronbauman
salesforce_api.admin_.inc_.patch4.31 KBaaronbauman
salesforce.class_.inc_.patch8.43 KBaaronbauman

Comments

Bevan’s picture

Bevan commented

See cache_set() and cache_get().

We shouldn't need to create tables for this I don't think.

Before taking this to far, when/how do we determine to clear cache? Every cron run? But do we really need every object definition cached?

Possibly we should be looking for any change when we get new data from salesforce, and if there are changes, checking if they affect any mappings.

aaronbauman’s picture

aaronbauman
he/him
commented
Status: Needs review » Needs work

Caching is motivated by the need to reduce not only the network traffic but also the number of API calls.
If Drupal makes even two SalesForce calls instead of one, the user will notice a significantly delayed page request.

We definitely don't need *every* object definition cached, only the ones with corresponding field mappings.

cache_set and cache_get sound good to me -- i'm marking this needs work for now.

joshk’s picture

joshk commented

Caching is motivated by the need to reduce not only the network traffic but also the number of API calls.

Yes, this is a big issue for getting a 2.0 release out. Salesforce meters the number of API calls allowed, and constantly refreshing the object definition burns through those needlessly.

My preference would be for a first step to have this stored via cache_set with a CACHE_PERMANENT value, and then cleared with a sf_api_cron() implementation.

Next would be adding a setting on the admin screen to change how long to keep the data for (which can replace the CACHE_PERMANENT with a time-based value) and a "refresh now" button, in case someone adds custom fields to their salesforce object schema, and wants to map them to drupal fields immediately.

joshk’s picture

joshk commented

I will be picking this up shortly to implement a solution using drupal's core cache table. However, I think a lot of aaronbauman's work here carries over, just no need for a custom caching backend.

Bevan’s picture

Bevan commented
Title: SalesForce Object definition local per-object storage » Cache object definitions
Priority: Normal » Critical
aaronbauman’s picture

aaronbauman
he/him
commented
Assigned: aaronbauman » Unassigned

I'm a little short on free time at the moment, if anyone else wants to run w/ this.

joshk’s picture

joshk commented
Title: Cache object definitions » Create solid data structure
Assigned: Unassigned » joshk

Ok, taking this on. Goals:

  • Store mappings in SQL, not variables
  • Cache object definitions
joshk’s picture

joshk commented

This has been committed. I need more testers with more diverse use-cases to beat on it a little bit and produce bugs if they can!

joshk’s picture

joshk commented
Status: Needs work » Closed (fixed)
rj’s picture

rj commented
Status: Closed (fixed) » Active

Hi, I don't think these patches have been committed against DRUPAL-6--2. I do not see any of these features in the current release (specifically, the Object Setup tab), nor do I see the changes when I look at the diff log. Ex: http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/salesforce/...

I tried patching these files against the most recent release and had several failures too. However, I did get the "Object Setup" tab, which resulted in the following error:

call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'salesforce_api_admin_object'

joshk’s picture

joshk commented
Status: Active » Closed (fixed)

These patches are not going to be applied, as they're lacking in many ways.

The issue of a lack of persistent data storage has been resolved though, so this issue is closed.

I will open a new issue for the "Object Setup" feature, which looks nice.