activity with custom token values = historical vs. realtime

Issue Summary

This took a while to figure out and root around the complexities, so I just wanted to log it somewhere, for us to handle when the API documentation is written. In short, this discussion is about token values, as called for by activity.module, as *implemented* by activity contribs, and whether it is real or historical.

The problem I was having was a single token, [user-wearing], being replaced with the realtime value from hook_token_values() and NOT the value that is saved inside activity.module's tables. This caused a problem because [user-wearing] was a historical bit of data, not realtime - what was saved in activity.module's table should always be used, not the real-time value returned from my implementation of hook_token_values().

<Morbus> greggles: i'm running into a problem, traced to token_get_values(). i am always passing in an $object, which contains the tokens I want to use. however, it always seems to be defaulting to use the values from the hook_token_values(), which will be wrong. i'm having a tough time tracking the logic properly.
<greggles> Morbus: in general token_get_values expects a $node or a $user as the object and then it parses the tokens out of that object for you. Morbus: if you've already built up all the tokens you want, then all you really need is strtr
<Morbus> greggles: what seems to be happening is that if (isset($tokens[$type][$id])) { is always gonna be run, even when $tokens is empty. tokens is empty on the first call, even if i'm passing in an $object. since the id check fails (since $tokens doesn't exist), it calls the hook, and ignores the data in my $object.
<greggles> Morbus: $object is not meant to hold your tokens, so it's understandable that it's not behaving as you expect
<Morbus> and right, my $object/$id is a serialized 'default' from _token_get_id. greggles: yeah, this rounds back to where it seems activity.module is using token wrong.
<Morbus> greggles: so you're not convinced that checking for an id in an empty $tokens list is wrong?
<greggles> correct about checking for an id in an empty $tokens
<greggles> Morbus: static $tokens;
<Morbus> right. which is empty the first time it's used.
<greggles> Morbus: that's a fairly naive caching system, so that tokens for a given object are only generated once
<Morbus> so you *always* call hook_token_values, even if $object is passed in.
<greggles> yes, but... object is not supposed to be some set of tokens. object is supposed to be a big nasty collection of data and it is the token module's job to dig into it and get the individual tokens out of it
<Morbus> so what if it isn't? why does the logic change? if i'm passing in a node, why call the hook and not base it on the passed node?
<greggles> Morbus: $tmp_tokens = module_invoke_all('token_values', $type, $object, $options); the object gets passed out to the hook implementations to dig into and find their tokens
<Morbus> greggles: so, it seems to me, to work around activity's wrongness (of sending in token values for $object), would be to sit on hook_token_values and return the live data when $object isn't passed, but to return $object itself if $object IS passed.
<greggles> personally, I'd rather see activity module patched to use token properly. if you need a quick fix, you could do something with your own hook_token_values implementation, probably
<Morbus> ok, yeah, it's a relatively quick fix as an end-user for me.
<Morbus> return $object ? $object : array( 'user-wearing'
<Morbus> in hook_token_values;
<Morbus> greggles: activity saves, in its tables, a serialized array of all token values associated with a particular message. it is this set of token_values that it passes to token_replace. how would you rather it be done? there's no "data" save that singular oneliner.
<greggles> token is built to work with things like $nodes and $users. so, data is stored wherever appropriate. loaded into the standard form of that $message object, and then the whole $message gets passed into token_get_values which lets the hook_token_values implementations parse out the tokens
<Morbus> but the problem is that activity is a historical thing. it can't even use the realtime info. it has to use historical data. which is why it saves that data in the table.
<greggles> sure
<Morbus> hook_token_values is geared toward realtime, not historical.
<greggles> but it should store the data in the table in whatever form makes the most sense for activity - not in the form that makes sense for token
<Morbus> sure. what activity stores is something like "this is a [token-placeholder]", "[serialized=array('token-placeholder' => 'value')]"
<greggles> yeah. but then when the data needs to get displayed later it shouldn't be passed to token_get_values, but strtr
<Morbus> well, not necessarily! because there are different contexts in which an item is stored. like, it has to decide if [author-all] is "you" (i'm viewing it) vs. someone else viewing it (in which case, it displays the username). so, it does a mishmash of both historical and real-time. [realtime-author-all] is wearing [historical-value]
<greggles> yeah
<Morbus> so it can't really do the str_replace and etc., because it, as a central api, can never tell which token is historical and which token is realtime. so, it's up to me, as a token-value hooker for activity, to know when to return what
<greggles> yes
<Morbus> in my case, i know my user-wearing is always historical, so i do the return $object ? $object : array(. ok, so, with alllLL that said, are we back to "it's not really a bug, per se - it's up to the token-value implementer to fix"
<greggles> yes =)
* greggles wipes brow
<Morbus> great. thanks for the discussion :D
<Morbus> mind if i pastebin this discusson in an issue for my own reference? <greggles> haha - np