fen's sandbox: Civichimp

Synchronize Mailchimp mailing lists with CiviCRM groups

Finally committed a new revision, refactored but still not ready for prime time. I'm very open to collaboration in the form of comments, suggestions, code, testing or anything that can help this move forward. I'm building it for Democracy Now! and want them to have a solid platform asap.

Goals

Support member management in CiviCRM syncing groups to Mailchimp lists for mass-mailing.

• Update matching Mailchimp lists when contact(s) added or removed from an associated CiviCRM group.
• Update CiviCRM groups when subscription changes occur in Mailchimp using either:
• A batch, drush-based cron job (used in current implementation)
• The Mailchimp Webhooks API
• Support bulk import of contacts and synchronizing lists.
• Support a "dry run" mode that will log operations but not execute them
• Documentation!

A D6 version in development. I'm open to creating a D7 version, too, and since I'm using CiviCRM v3 APIs, this should be relatively easy. But I need to get the D6 version stable first.

A list of some of the issues still to be worked on follows.

Opt-Out

In CiviCRM, a person may still be a member of several groups (regular or Smart) but have opted out of receiving any emails by checking the "Do Not Email" option. CiviMail checks the "Do Not Email" option before preparing a message to send to a user - but this is not currently checked before syncing to Mailchimp. (This will be in the next update.)

On the flip side, When someone unsubscribes from Mailchimp, should that set the "Do Not Email" flag in CiviCRM? I think so, and will add this also in the next update.

Caching and Optimization

Currently the module operates an a single contact at a time and makes many calls via the Mailchimp API that retrieve data that could easily be cached. This and Opt-Out capability must be in the next release.

Email Address Changes

If Bob switches from bob@aol.com to bob@yahoo.com in Mailchimp, how can this information be transmitted back to CiviCRM so that the primary email address in CiviCRM also changes (and a new contact is not made instead)? Similarly, is such a change is made in CiviCRM, it must be propagated to the correct contact in Mailchimp; the civichimp_pre_hook() may be also to solve this, but this is not yet tested nor implemented.

Smart Groups

The GroupContact API returns all the (regular) CiviCRM groups a contact is a member of but doesn't return Smart Groups. Essentially you have to build each smart group - using the where_clause in civcirm_saved_search and tacking on AND contact_id = <whatever> - and check if a user is inside it. So a rather heavyweight search loop across all the smart groups is needed to find that in order to be able to update Mailchimp when any details for a contact changes.

There is a "Make it Happen" (CiviCRM project fundraising) suggestion for Smart Group Optimization that would help move this task forward. I am trying to revive this. It's not clear yet if all cases would be covered, such as when users are manually added/removed from a smart group, nor how child groups would be handled.

Associating Groups with Lists

The current mechanism used is by exact match of unique group/list names, which is a bit fragile. And admin panel similar to the one in the Mailchimp module would help here. Note that CiviCRM groups and Mailchimp lists will not be synced until an association is made.

Bounce reporting, Opens and Click-thoughs

I have no idea how to get any of this back to CiviCRM. Current plan is to rely on the Mailchimp interface here.

Civichimp was inspired by code posted by @poieo at http://drupal.org/node/272044#comment-4213300 and draws some techniques from the Mailchimp module by @levelos