Help/support on the notification API and writting new subscription kinds

I was away for a few day but I greatly appreciate your contribution and help.
I will review and do my homework and get back if something resists me.

Thanks a lot again.

PS : and if I achieve something I'll try to get something back in a tutorial form maybe.

Top PS : please try to read it all first before answering since writing the stuff made me thing about what I do not understand and I explained it at the end, especially in II) .

Vocabulary : I decided to call a "subscription instance" a particular subscription and parameter set a user want to be notified about.
Like a triplet :
- subscription type(i.e. : descendant of that term)
- params set (i.e : the term the user is interested in)
- user (i.e. : the user wanting to be notified about node pertaining to a term or its descendant)

For the record I am now trying to do a VERY simple notification : no parameters, no various configurations in the node page or else, simply a hard-coded logic : if subscribed elect some nodes. Else There is too many things to get rights with too little feedback and I can't figure them out properly.

First, let's see the pieces we need for that new type of subscription:
1. We need to define that subscription type, using hook_notifications('subscription types'):
$types['vocabulary'] = array(
'event_type' => 'node', // This will be for node events (insert, update, comment...)
'title' => t('Vocabulary'),
'access' => 'subscribe to vocabulary', // New permission must be created somewhere else
'user page' => 'user/%user/notifications/vocabulary', // Menu item for a page that must be defined somewhere else
'fields' => array('vocabulary'), // These are the condition fields this subscription needs, in this case just the vid
'description' => t('Subscribe to all content tagged with a term of a vocabulary.'),
);

what is 'user page' => 'user/%user/notifications/vocabulary', ? the admin page for that particular notification ? that would appear in the "notification" part of admin right ?

I assume you took one of my use cases as example and that in 'fields' => array('vocabulary') you just decided to give a name to the sole parameter for a "notification" when a node relating to a particular vocabulary is updated right ?

If I needed 3 parameters zip, zap and zoum I would write 'fields' => array('zip',"zap",'zoum'), is that right ?

If I don't need any parameter or special configuration page, can I set both fields to null ? or omit them.

(I ask this because when I create a new subscription kind in that hook I can't see it anywhere in nodes or admin notification page so I don't know if I did something wrong or if It is just that I need do to more things ...)

2. We need to define how that 'vocabulary' field behaves (field type, translating values for display, using it in autocomplete, etc...). For that we implement hook_notifications('subscription fields');

$fields['vocabulary'] = array(
'name' => t(Vocabulary'),
'field' => 'vocabulary',
'type' => 'int',
// This one should return an array of vid => name. Will be used when displaying the subscription for naming the value
// and for a drop down field when creating a subscription with this field
'options callback' => 'notifications_vocabulary_names',
);

Here we are talking about what API for the keys ? the schema api ? and thus in the 'field' => 'vocabulary', part what is "vocabulary" there ? the name of a column in the db ?

Again if I take no parameters in my notification I can I skip implementing this call right ? Or should I still return something ? (and what)

3. Now we need to define how to query for this type of subscription when a node event happens. That is hooks_notifications('query', 'event', 'node', $event) (event type, event object)

$node = event->objects['node']; // This object will have been loaded by notifications_content
$vids = mycustomfunction_get_vids_for_a_node($node); // We will get an array of vids for the node
$query[]['fields']['vocabulary'] = $vids;
return $query; // With these values, the query builder will build the query to match this conditions

With this, the Notifications engine will have enough information to build and display this type of subscriptions. If we want to display these options for nodes as node links, we can add some more. These are extensions handled by notifications_content module, not by notifications itself.

This is where I am at a completely loss "define how to query for this type of subscription" ? What do you mean ? Query the database ? but for what ?
Do you mean BUILD a query that will say if the node is to be notified about or not ?
But you seem to only build a PART of a query that is completely hidden ?
What will this query be run against ? to retrieve what ?

Does that mean that I can't have LOGICS to implement my notification but I am only allowed to express it into parameters/conditions of that mysterious query ?

If this is the place where I am asked to tell if the node should be notified about I don't really get why I am concerned about "queries" (see below in II to get more of what I finally understand)
Since the notification engine knows all about it I could expect to get : the node, the event, the array of params if any and be asked if the node is to be notified about or not.
in my case that would be mycustomfunction_does_the_node_belong_to_that_vocabulary or mycustomfunction_does_the_node_tags_fall_under_that_term (meaning including its descendants in the vocabulary) that would return a boolean ...

OR does that mean I need to tell the notification engine how to query/retrieve (but retrieve what ?) for MY custom notification for THIS particular node ?

To query enabled subscriptions of this type for a node, we need to implement hook_notifications('query', 'node', $node). Will be the same as the one in 3.

The mystery thickens there ... is this the SAME hook ? different OP ? and again what are we SEEKING through this "query" ?
I can't help myself think that since code duplication is evil, having to do the same thing twice is suspicious, unless there is case you don't have too.

To display link options for node we implement hook_notifications('node options', $account, $node)

// First we check these subscriptions are enabled for this content type (notifications_content module)
if (notifications_content_type_enabled($node->type, 'vocabulary')) {
// For each vocabulary linked to the node, we need to return one of this:
foreach ($vocabularies as $vocabulary) {
$options[] = array(
'name' => t('Vocabulary @name'),
'type' => 'vocabulary',
'fields' => array('vocabulary' => $vocabulary->vid),
);
}
}
// This will produce, with the help of notifications_ui module, one link for each vocabulary the node is in like "Subscribe to Vocabulary 1", "Unsubscribe to vocabulary 2", etc...

This should be enough for basic subscriptions of this type. The 'names' parameter is obsoleted (we use now the 'fields' information instead to format subscription names.

This is for the case my custom subscription could lead to several actual subscription with different params for the node right (in the particular case you picked up vocabularies for the node) ? In case I have no parameters (and thus no variant), can I slip that hook ?
Especially since my simple subscription is not a "per node" one, more like "subscribe to that set of content" in the profile page of the user that translates in some vocabularies internally but the user don't see that (and in my case it translates in ALL content nodes EXCEPT the one that belong to one particular vocabulary UNLESS they below to another particular one, but all this logic is "set in stone" and not customizable by the end user).

In next version, 4.x I'm looking into simplifying and better documenting all these hooks. I guess the notifications hooks should be split into some different hooks.

Let me know whether this helps / works and also how you think we can improve the documentation.

Thanks a lot for you time, I'll try to explain what puzzle me still :

I understand more things about the API but still I have NO IDEA about WHERE I tell the notification API to notify for a node.

Is it possible to do such things :
* notify at random for a node for instance (some node get notified about some don't depending on a roll of dice, crazy but why not)
* notify for the nodes that fall under a term AND its descendant in taxonomy

or it is NOT possible because I have to limit myself on that query thing you describe in (3) that only combines predefined kind of items ?

On a more general things I'll try to describe the kind of API I was expecting and what I was not expecting at all
I was not expecting :
* to fully understand the underlying storage of the notifications
* to dabble in the "queries" of the notification engine,
thus tying myself to a particular implementation of the notification engine.

What I expected is :

I) to describe my new custom notification
- Name and such
- what kind of even I am interested in
- optionally params and type of the params
- optionally a custom form on the administration API
- optionally a custom part of the node form to manage that subscription

II) receive a call (hook) giving me
- the event and its date (node/comment/whatever I registered for)
- the params in an array (thus I don't need to access the underlying stuff and know about it so it can change freely)
and return to that hook and simple boolean telling if the (node/comment/whatever I registered for) if to be notified about to people who registered for that subscription with those params.

Writings this I start to understand a bit more of the problem : in (3)(your notes) what we query for are the SUBSCRIPTION INSTANCES, the actual "registered" subscriptions no ?
Like "what instances of a my_custom_subscriptions" MATCH that node ? is that right ?
If that is right that means that subscriptions like the one I NEED to do for taxonomies are not possible because they require a PER instance processing.
(I need to check for EACH subscription instances if the terms in the node are taxonomical descendants of the term in param in the subscription instance )
Or at least I would need to return SEVERAL queries to retrieve nodes, in that case, one for the subscription to the exact term in the node, one to subscription to its parents, and so on till the top vocabulary.
Or am I wrong again ?

II) that's it
I understand there is more possible hooks for messaging, templating and stuff but for the core of the problem that should be enough no ? ...

Just to explain about my background, I a professional programmer, with a background in pascal, python, UML, unix/BSD and windows, moderately fluent in PHP and having under my hand among other things the "pro drupal development" book from J.K. VanDyk, I am not the most bright of developers, far from it, but still not quite the "average joe" in "programming". But I am overwhelmed by all the implicit things I face : the term "fields" is used in a billion place with different meaning, I often asked to know about the underlying database schema in full to understand the processes, many times I have options but nowhere to find all the POSSIBLE options, and stuff like that.
Maybe I am the only one feeling this :).

Thanks a lot for your help, I started notes already about a "tutorial" thing so I really want to get through this if my brain allow me and try to help others on this module.

Have a nice easter anyway :)

AS usual a good night sleep goes a long way for solving understanding problems.

I think I got how the notification are basically structured and I'll try to experiment with that and do a report here about what I understood of the relatonship betwen a subscription type, a "subscription instance" and a notification, basically I think I got, a last, the schema that is in the dock.

I'll try to do a summary and then map it to the technical hooks as much as I understand them.

And thanks a lot for your prompt answer I did not refresh the page before posting the previous message.
I'll read it carefully ASAP.

Thanks a lot.

The main problem is that I did not even get the basics of the notification logic so when you talk about querying I don't have the slightest idea about WHAT (table) we query.
Too many assumption and implicit in our mutual explanations make things cloudier than they should I think.
I'll try to explain what I understood so far and this will be the basis of my "documentation complement"

======================================================================================================

The terms

subscription or "subscription kind" or "subscription type"

A capability to subscript to something provided by a module.
for instance : subscribe to a particular node update, subscribe to new nodes filled under a particular taxonomy term or subscribe to nodes whose title start with the letter T.
a "subscription type" can provide for parameters than allow the subscriber to specify the node id he wants to subscribe to, or the taxonomy term_id, or the letter at the start of the title of a node he is interested about ...
They are mostly defined in code in modules specifics to each subscription type.

"subscription instance"

A subscription instance is a particular/specific realization of a subscription type ONCE a subscriber subscribed to it with a GIVEN set of parameters.
That goes into a set of tables that store all triplets (subscribers, subscription type, parameter(s)) that represent all subscriptions

"Event"

An event is a change (creation, update, deletion etc) in a drupal element (mostly a node, but could be a comment, or maybe a user) that COULD trigger a notification.
And Event will trigger the framework module to look for possible "subscription instances" that could be concerned by the Event. "subscription type" modules provides hooks to respond to event they registered for in their definition hook ('subscription types' op in hook_notifications if I am not mistaken).

"notifications"

The signal that a particular node for a particular "subscription instance" (pertaining to a particular "subscription type")
notifications are stored in a table that is processed by the framework to build the actual messages that are send to the user, depending on its preferences regarding frequency and such. De-duping and grouping will also be handled by the framework as well as templating. A particular "subscription type" module can influence that, but that's another subject.

The process

I

The framework provides several means to subscribe to something : through the user profile, on the node itself, in the administration UI, etc .. Those have to be supplied by the module defining a "subscription type" in various hooks.
(hooks and point and expected things to be described here)

II

When a subscriber (a user most of the time), register for a particular subscription that is offered to him through the UI a "susbcription instance" is inserted into the system (a set of tables).
(give a reference to the already existing description of this tables here)

III

When a node is updated or created particular hooks of "subscription types" in modules that defines them are called to ask them to retrieve the "subscription instances" that they know about.

They have to build a query that retrieve the particular triplets (subscribers, subscription type, parameter(s)) that represents the subscriptions they manage that are concerned by the changes in the node.
(explain the various hook and op that provide that her)

IV

The result of this query will be used by the framework to create "notifications".
(reference to the notification and messaging documentation here)

Schematic reference http://drupal.org/node/284657 and http://www.schoolofeverything.com/files/notifications_overview.pdf
======================================================================================================

Is that a good or acceptable description of the general logic of the framework that I should build upon "my" understanding of it ? (and hopefully others)

I re-read the documentation at http://drupal.org/node/253102 and terms are not defined and some are really confusing like "notification conditions" that have nothing to do with any "programmatic logic" but are only the "values of parameters" associated with a particular "subscription instance" (in my own parlance). And that's what thrown me out of the loop to understand the framework.

Now of course I have to understand all the details and I'm still a long way to have my subscription lol.

I separated answers to some particular items you mentioned for clarity and separation of concerns

Just about the last one I think we have some misunderstanding: Your module defines a subscription like 'Users can subscribe to taxonomy terms' but from that point *any user* can subscribe to *any taxonomy term* (though you also have access permissions and callbacks to define whether a particular user can subscribe to a particular term)

So once a node is updated, there's not a simple boolean. There's a query condition like "let's query for subscriptions for which 'tid' == (any of the terms of the node).

More similar to what you describe is the Notifications Custom module in this package: http://drupal.org/project/notifications_extra

With this one you can define a subscription like 'A subscription to the taxonomy term xx'. So users get a simple checkbox and you don't need to do anything else but to define the 'subscription field' in the notifications callback.

Have you tried this one?

yes I tried that module but is really is "scratch one particular itch thing"
- you can't subscribe to a vocabulary in full
- unless I am mistaken it does not understand the concept of taxonomy per see, only of "terms" meaning it does not allow to subscribe to a term AND its descendants
it is really "tag" oriented more than taxonomy oriented and that's a show-stopper.

More over it does not allow you to define custom subscriptions that are anything else than an implicit "AND" between particular "subscription conditions". For instance you can't say I want to build an ad-hoc subscription to THAT term OR that term OR THAT term, because the module will only give you the node that are tagged with EVERY terms :(. So instead of mix and matching the terms you are only narrowing the condition.

The only thing that is like "I" need in that module is that its a "global subscription" that gives only a checkbox on the user profile page, this is indeed all I need and I want to emulate in my subscription.

I further narrowed by needs to be able to test and get some feedback more quickly (before self desperation ;) ).

I will first build a subscription as such :

- a "global subscription" on the node event
- that only appears in the user profile as a checkbox (subscribed or not)
- At event time I want to check for a particular, for now hard-coded, set of conditions on the node and in case of the conditions say so, have notifications send

that means that I need to either build the query (in (3)hooks_notifications('query', 'event', 'node', $event) (event type, event object) ) that retrieve ALL "active subscription instances" for that particular subscription type OR return "nothing" ( an empty query ? a special query with no result set like "where 1=2" ? ).

It is the most simple kind of subscription I can think of, no parameters, no multi-select + parameters queries to build with the query langage, etc ...

Questions :
* in

 $types['vocabulary'] = array(
        'event_type' => 'node',  // This will be for node events (insert, update, comment...)
        'title' => t('Vocabulary'),
        'access' => 'subscribe to vocabulary', // New permission must be created somewhere else
        'user page' => 'user/%user/notifications/vocabulary', // Menu item for a page that must be defined somewhere else
        'fields' => array('vocabulary'), // These are the condition fields this subscription needs, in this case just the vid
        'description' => t('Subscribe to all content tagged with a term of a vocabulary.'),
      );

'user page' => is the page that is added as a tab in the user profile right ? well in fact it is a special "menu hook" for the router right .?

In that particular case I only need to implement :

* hook_notifications('subscription types') (leaving the params fields set to none or omitting them ?)
and
hooks_notifications('query', 'event', 'node', $event) (event type, event object)
building a query (i need to dig up what this query really look like because it is hard to build the where close without seeing it ) in case my node match my hard-coded conditions and doing nothing in other cases ?

and that' all ?

I am fighting the beast with these assumptions right now ...

I am still alive (not that kicking ;) )

I made small progresses, tamed having the same kind of system we have in the custom_notification module for selecting/de-selecting subscription on the user profile page, and inscribing the "subscription instance" in the 'subscription ' table.

I set myself for a very simple kind of subscription with NO parameters.
Either the user as subscribed or not.

First

So i am to the point where I need to answer to the 'query' event of the hook_notifications($op, &$arg0 = NULL, $arg1 = NULL, $arg2 = NULL) hook.
I dug a bit into the code and it seems to me that this case in not expected in the query builder.

Am I wrong ?
Meaning I expect to test my condition in code and say to the query builder "no param / basic query for subscribers with no fields to set" if the condition on the node match.

Is that possible ?

Or do I need in that case to use the second option you mention :B. Your module returns no query conditions and it does its own query after the event is triggered, that is hook_notifications('event queued', $event). This one would be much harder and you really need to know about the storage for this. While for the other one you need to know nothing about the module storage.

Second

I want my module to manage two "subscriptions kind" (complementary ones with no parameter, like one half of the content and the other half of the content).
It seemed doable to pass several subscriptions to the "case 'subscription types':" in hook_notifications but when the query event is called I see no way to know for what subscription kind the notification framework is querying me for (is preparing the query for).
I did not see where the hook will let me know the type or name of the subscription I set up in the 'subscription types' op ( like 'name' =>'globalSubsNoMini' => array('globalSubsNoMini' or 'type' =>'globalSubsNoMini', ....)).

Again, am I making a wrong assumption and must one module target only on subscription kind ?

Third

Or course an apparent solution to this would be to have one "subscription kind" with ONE field "subType" that gets a different predefined value for each of the subscriptions I want to build and build the query accordingly with "OR-ed" fields if the node match several condition that are represented by the subtypes.

How do I make such a "OR" in the "query" op, is that possible ? the code I saw seems to imply AND and only target equality of "fields=values".

Is that another "B" case or did I miss something ?

Thanks a lot for you time.

PS: I am in a situation where it is hard to have a debugger working reliably so I can only try to READ the code and put traces in it. Hence some things that stay a bit unsure when a debug step-by-step trace would help for the answer.

PS: congrats for the 4.0 release.

Still making progress (I think) and I took the third route, I made two fields that I keep invisible and that I set depending on the subscriptions kind I want.
I present "virtual" subscription kind A and B to the user and internally manage it as my "Main subscription" with field A or B flagged (and possibly both).

Since I don't need nor want those fields to appear in the UI anywhere I did not fill the "field" op event hook, is that ok ?

I am now trying to integrate with the rest of the framework especially the administration screens notably at admin/messaging/subscriptions/admin

I have troubles with the "Type" and "Description" columns in that screen because they seem to query the "fields" event hook, I am not sure I understand why the field event is called exactly.
Partially filling the event by describing the fields as exposed in the documentation even brings me to have one of the fields value displayed in the screen, witch looks odds.

How I can hook up into that process and give MY description and type name ?
And what is the logic that is followed in that query ? I suppose it is related to wanting to show to WHAT a subscription is related (subscription to that author or this topic) but it does not seems to support the case where there is no field or a non user facing field.

thanks a lot for your time.