Starting with 2.1 (2.2 for stable), MongoDB allows collections to have a Time To Live to be set with ensureIndex() to be automatically expired.

This could allow actual server based cleaning without any PHP code.

http://docs.mongodb.org/manual/tutorial/expire-data/

CommentFileSizeAuthor
#7 1802114--mongodb_cache--take-advantage-of-TTL-collections-7.patch3.05 KBamontero

Comments

fgm’s picture

fgm
Primary language French
Location Paris, France
commented

Come to think of it, some might even be interested in them for the watchdog: they are an alternative to capped collections in some scenarii.

fgm’s picture

fgm
Primary language French
Location Paris, France
commented
Issue summary: View changes

The 2.1/2.2 implementation allowed only a single duration for a given index. Since 2.3/2.4 it is possible to specify expireAfterSeconds as 0, and insert the expiration timestamp in the documents themselves so the deletion thread will delete documents at their requested expiration, which seems just ideal.

http://docs.mongodb.org/v2.4/tutorial/expire-data/#expire-documents-at-a...

fgm’s picture

fgm
Primary language French
Location Paris, France
commented

After experimenting with it, using the latter option seems especially interesting, as it removes the need for an explicit flushing operation, which is performed by the MongoDB server in the background.

Some remarks before committing to this, though:

  • the expiration is performed by the TTLMonitor, which only runs every 60 seconds, so shorter expiration times will not be covered
  • the TTLMonitor can be disabled from the command line --setParameter ttlMonitorEnabled=false or by a command db.adminCommand({setParameter:1, ttlMonitorEnabled:false}), so its availability should be checked in the requirements

This does not seem to be an actual problem, though.

Anonymous’s picture

Anonymous (not verified) commented

I'm actually doing this for sessions in a hacked version of the MongoDB Sessions module & it works great:

$expires_at = REQUEST_TIME + 200000;
  $fields = array(
    'uid' => (int) $user->uid,
    'cache' => isset($user->cache) ? (int) $user->cache : 0,
    'hostname' => ip_address(),
    'session' => $value,
    'timestamp' => REQUEST_TIME,
    'expires_at' => new MongoDate($expires_at),
  );

It would be great to have this in the module. With this, I have ini_set('session.gc_probability', 0); in settings.php, i.e., no PHP garbage collection needed. +1

See also: https://api.drupal.org/api/drupal/core!vendor!symfony!http-foundation!Se...

fgm’s picture

fgm
Primary language French
Location Paris, France
commented

@rhclayto interested in rolling a patch for it ?

amontero’s picture

amontero
Location Barcelona
commented

Recently, I've successfully moved my cache_form bin from MySQL to MongoDB.
As of now, I'm periodically calling cache_clear_all(NULL,'cache_form',FALSE) to trim expired items from the cache_form collection, but I had first to create an index for the 'expire' field, since the process imposed a heavy load on my server (hint: IMHO, it should be created always on install/first access).
So, I've also become interested in TTL-expiring collections, but for cache bins. The above mentioned index was created with TTL, but it's not working. Reading the code above, I've figured that a NumberLong timestamp 'expire' field is not a valid field type for TTL indexes to work.

Since indexing the 'expire' field would be beneficial alone, I would like to take a stab on it and also benefit from TTL expiration. This way, MongoDB would behave more similar to other cache backends such as Redis, where expiration is a built-in feature.

AFAICT, there are two paths:
- Change the 'expire' (and 'created') fields' type to date. I can't see the implications of doing it, if it can break compatibility or would need field type conversion. However, the most orthodox way, IMO.
- Leave current 'expire' field as is and add an 'expire_at' field as @rhclayto says in the comment above. Seems easier but duplicating fields sounds suboptimal.

Advice on how to proceed would be appreciated.

amontero’s picture

amontero
Location Barcelona
commented
Status: Active » Needs review
StatusFileSize
new 1802114--mongodb_cache--take-advantage-of-TTL-collections-7.patch3.05 KB

First stab at using TTL fields for both 'expire' and 'created' cache item fields. Field conversion is not necessary, since code provides a underlying field type check, to convert from MongoDate (milliseconds since epoch) to timestamp (seconds since epoch).
Tested with mixed NumberLong/MongoDate cache items in the same bin and worked. All tests pass, also.
Attaching patch for review and getting feedback.

fgm’s picture

fgm
Primary language French
Location Paris, France
commented

For review on https://www.drupal.org/project/mongodb/issues/1802114