I think other hosting modules could do with persistent cache tables, but this is for #1875440: static and persistent caching for _hosting_package_instances_load() and will probably be useful for other functions in hosting_package that hit the db.

Comments

I guess the same question would apply here - do we really need a whole cache system for packages? Does it really improve performance?

Patches were blocked elsewhere because of the performance of the package system apparently being slow. So that can't be true and it not be true that caching is a good idea.

What do you mean patches blocked? Which patches?

Ah, I'll try to remember to track down which one it was this weekend! people were complaining that this functionality is "already very slow".

If packages table are slow to access, that may be because of stale entries. See #1034520: Cleanup zombie entries in hosting_package_* tables?.

Ah, and #1067688: bulk operation forms are too slow with thousands of packages maybe the issue you were refering to. As mentionned there, I feel some indexes and fixes to the code may be better than the blunt caching approach...

Caching is a pretty standard way to help speed things up in the broader Drupal community, actually the broader programming community in general.

You're probably right about garbage collection and faster code improving the speed of the lookups, but that doesn't make caching queries somehow a bad idea - for users that use a non-db implementation of their cache it means they could avoid hitting the database altogether which can only be a good thing.

I'm not proposing this as a magic bullet, I'm more saying that if Aegir had been migrated to D7 successfully "packages" would probably be entities by now and it would be trivial to add support for static and persistent caching for lookups through the contrib Entity Cache. Since we're dealing with D6 the "normal" procedure is to setup a cache table ourselves and manually wrap our loads in cache_get() and invalidate it during writes.

I don't see why caching is "blunt", it's a perfectly legitimate way to improve performance in many circumstances. Other that your personal taste, is there a good reason *not* to do it here? Assuming of course that any patches are actually benchmarked before they are committed..

wait, $reset is already documented in that patch for hosting_package_instances_load()

+ * @param boolean $reset
+ * If reset is TRUE, the cache of objects matching the current $param will be
+ * rebuilt and fresh results will be returned. Defaults to FALSE.

do you just want me to add a @see hosting_packages_instances_load() for the _hosting_package_instances_load() comment?

Here we go. This is pretty difficult to misinterpret :) the comments in my patch are now the best in the whole file even if i do say so myself :P

Status:Active» Needs work

Awesome, thanks! Now can we have some benchmarks? :)

Is there a "standard" aegir benchmarking database(s) + file system I could grab from somewhere? I don't have a 2.x version of Aegir in prod anywhere so I don't have anything "heavy" to run tests on. I'd really want to be trying to simulate something that has seen at least a year of active usage rather than something I build out and stick one or two platforms on. I also don't want to spend hours creating dummy sites and shit if possible.

I am not sure... I guess you could run the test suite in vagrant to bootstrap a site. It already sets up a few sites and platforms for you, and then you can throw apache bench or siege at the site creation AJAX hooks or something.. that's where a lot of the pain is.

Version:6.x-2.x-dev» 7.x-3.x-dev
Issue summary:View changes

New features need to be implemented in Aegir 3.x, then we can consider back-porting to Aegir 2.x.