Unless we create a child class per migration class for the migration destination, we get the same count number on every migration because the cache id is __METHOD__, that always output MigrateSourceMSSQL::count

This patch use the md5 of MigrateSourceMSSQL::countQuery as the cache ID for the count query.

CommentFileSizeAuthor
#2 source_cache-1201762-2.patch14.82 KBmikeryan
0001-Use-a-better-cache-id-on-MigrateSourceMSSQL-count.patch1.42 KBmarvil07

Comments

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Title: Use a better cache id on MigrateSourceMSSQL::count() » Implement default source count caching
Category: bug » feature
Status: Needs review » Needs work

I've had in the back of my mind implementing source count caching directly in MigrateSource, rather than dealing with it (or not) in each source plugin. Think I'll give it a shot, making sure I take care of this problem as I do it...

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Needs work » Needs review
StatusFileSize
new source_cache-1201762-2.patch14.82 KB

OK, the tricky part was not breaking any existing code (i.e., if anyone has implemented their own source or list classes). I haven't committed yet, please give it a try and see if it works for you on MS SQL. Notes:

  • Caching is disabled by default, for a given source pass array('cache_counts' => TRUE) to the $options argument on the constructor. See the BeerNode example.
  • Sources are now expected to implement computeCount() rather than count($refresh = TRUE) - the latter is now implemented in MigrateSource and handles the caching itself. Existing source classes implementing count() will work just as they always had, but are encouraged to switch to computeCount() to take advantage of the caching.
  • The MigrateList class has a similar switch from count($refresh = TRUE) to computeCount() - neither method is explicitly declared for the moment so nobody on either side of the divide should break.
mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Oops, don't have time to redo the patch right now, but "abstract public function computeCount();" in source.inc should be commented out, otherwise source classes outside of Migrate itself will break...

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Version: 7.x-2.x-dev » 6.x-2.x-dev
Status: Needs review » Patch (to be ported)

Committed for D7, still needs to be backported to D6.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Patch (to be ported) » Fixed

Committed for D6.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.