since nearly all of the other feature requests are now officially postponed, i wanted to open a new thread to start gathering ideas about the signup core/plugin interface, how it should work, what it needs to support, etc. here are some of my initial thoughts on the subject:

  1. plugins could take the form of ".inc" files in the signups directory. we could ship them in signups/contrib, and sites that wanted to enable a given plugin could just copy/symlink the appopriate .inc file into place. this is how flexinode works to allow sites to define their own fields for their arbitrary node types, and it seems to work pretty well.
  2. plugins need to be able to register that they want to add new data (forms, tables, etc) to various existing tabs:
    • default node view
    • node/X/signups admin tab
    • admin/signup main control tab
    • admin/settings/signup page
    • user page

    ideally, for each of the additional output added to any of these tabs, the plugin would either have a way to programatically declare the "weight", or there should be a setting somewhere to configure it, so that admins can control the order and look of the signups data, buttons, links, etc.

    something like this could be accomplished by making sure the existing methods that generate each of these pages incrementally builds up a final $output string (which they mostly do already). there could be a unique hook name for each of the tabs/pages of the "core" module, and any plugin which implemented a given hook would have that hook called, and the return value of the hook would be concatenated onto the $output string (preferably with an order based on weight, as i mentioned above).

    to facilitate all the existing output being properly weighted, we might even want to have the default/core functionality all implemented through the same hook system, living in a file called "signup-core.inc" or something. that could also serve as an example for plugin authors looking for how to do things.

  3. plugins need to be able to add their own independent tabs to existing (core) pages/tabs:
    • node view (for another signup-related tab visible to all users, but not the admin interface)
    • adding subtabs to the node/X/signups admin tab (maybe this tab should be renamed "admin signups" or something)
    • new subtabs under admin/signup
    • new subtabs under admin/settings/signup
    • new subtabs of the user page?

    again, the core function responsible for generating each of these parent pages or tabs would have a corresponding hook name that each plugin could implement. the hook would be responsible for declaring/name the new tab, and implementing the callback function to generate the tab once it has been clicked.

  4. plugins should probably be able to add their own columns to some of the existing tables
    • table for viewing all users signed up for a given node (read-only) on the default node view
    • the user-detail table on the node/X/signups admin tab
    • the event-detail table on the admin/signups overview tab

    this might get tricky if we want the plugin's hook to actually be able to modify the main page's SQL query (which would be nice to allow it all to be sorted). alternatively, the plugin could be responsible for filling in its own data on a row by row basis, but that seems more difficult and ineffecient.

  5. we need to figure out if we're going to force all plugins to implement their own versions of the various DB tables, or if plugins which need to use the DB can modify the schema of the existing tables. probably modifying existing table schemas will be too complicated, so while inefficient, we might say that each plugin which wants to keep track of data has to use its own tables. this definitely has implications for the SQL query generation code and related hooks. i must admit i'm mostly an SQL novice, so this is one of the areas of the plugin/core API i'm most unsure about...
  6. in general, when i've said "hook" so far, i really mean a family of functions implemented in the .inc file. for example, all hooks could have the form: "signup_plugin_{plugin-name}_{parent-page}_{hook-type}_{hook-action}".
    • "{plugin-name}": should be obvious -- the name of the plugin. for example:
    • "{parent-page}": the parent page the plugin wants to modify (e.g. page it wants to add a new tab to)
      • "nodeview" (node/X)
      • "nodesignups" (node/X/signups)
      • "user" (/user/X)
      • "admin" (admin/signup)
      • "settings" (admin/settings/signup)
    • "{hook-type}": the kind of plugin hook described above
      • "add_data" (add more output to an existing page/tab)
      • "add_tab" (add a whole new tab to an existing page/tab)
      • "add_field" (add a new field in an existing core table)
      • "cron" (stuff your plugin wants to accomplish via cron)
      • "add_perm" (add a new value to be included in the array returned by the signup_perm() drupal core hook)
    • "{hook-action}": the action the hook needs to implement, depends on the {hook-type}
      • "name" (provides the string to use as the name of the new tab, the header for a new field, etc. also, this hook might be responsible for checking user/role access rights, and returning NULL if there's nothing to add for the given user trying to view the page...)
      • "output" (method to generate and return the new output (data/tab/page))
      • "weight" (provides the weight of the new data on the page)
      • "db_select" (what to concatenate to the SQL query for this new field)?
      • "config" (how the plugin hook interacts with settings)?
      • "validate" (validate some kind of user-input related to the hook)

      i'm just making stuff up here, and this isn't meant to be an exhaustive list. i'm just fleshing out the kinds of things we might need to allow hooks for.

    • so, for example, the hook that declared the name of a new tab off the admin/signup page for a plugin called "reply" would be a function called "signup_plugin_reply_admin_add_tab_name()", which
      would live in a file called "modules/signup/plugin_reply.inc"
    • some of the hook naming convention stuff might need to change if we consider a single plugin that wants to add two separate tabs to the same parent (e.g. if we move all the conflict stuff to a plugin, it'll want to add two additional tabs to the admin/signup page, not just one). therefore, we might need to add yet another "field" to the hook naming scheme, and
      have a hook that returns an array of strings to use for this field so that plugins can declare as many of each kind of hook as they want.

some closing questions as food for thought:

  • is this going to make signup more difficult to maintain and extend than just adding the various new features being suggested? ;)
  • should we deal with different plugins trying to interact with each other?
    • should we just say "the core defines the possible {parent-page} values, and plugins either have to use those existing pages or add their own tabs to one of them
    • or should we attempt to allow 1 plugin to add a subtab to a tab or page created by another plugin? (this seems awfully complicated, and probably not necessary).
  • are there features folks currently want to see in signup that wouldn't fit into this hook model? i'm pretty sure if all this plumbing existed, i could easily add all the things i want to add to signup as new plugins. is that true of the other ideas people have had?
  • i'm guessing an overhaul this major would require working on the CVS head (4.7) codebase, right? is there something more lightweight we could do in 4.6 to allow some initial plugins without quite so much development overhead?
  • how do other modules handle this problem?
    • some of the ideas here are heavily borrowed from flexinode...
    • are there other good examples to look at?
    • are there drupal standards for the names of any of these sorts of things? can we make the names more sane by using other (already existing) terminology, conventions, etc?

where's this drupal conference you mentioned in the other threads? perhaps i should try to attend, so we could talk about this in person. ;) i'm in the sf bay (oakland), in case any of the other core signup folks are near here and we could organize some face-to-face time to hash out more of these details and start trying to implement something like this... (assuming people think a system sort of like this is a good idea).

thanks,
-derek

Comments

__Tango’s picture

__Tango commented

I've only had time to skim this post, but i wanted to mention that although i'm not sure where hunmonk is, I live in the bay area too (mtn view) and am planning on attending the berkeley meet-up next thursday (Feb 16: http://drupal.org/node/47489). if you're going to be there, we can talk about this.

...alex...

dww’s picture

dww
we/he/they
commented

my brother works at cnmat (the location of the upcoming bay-area meetup), and i know the host of the event through him. unfortunately, i won't be able to attend that one. :( but, hopefully next time.

i suspect hunmonk and other developers will hash this stuff out at the drupal convention later this week, and will come up with a more coherent plan than what i sketched out here. i was just trying to get my initial thoughts down to see what other folks thought. now that i know about the drupal convention and that that's the sort of place where these kinds of APIs usually get designed, i just hope hunmonk at least reads over this thread as a 1st draft proposal to start from. ;)

-derek

hunmonk’s picture

hunmonk commented

__Tango: i'm at drupalcon now, and will be meeting w/ the co-developer of signup module while i'm here, and we'll be discussing this issue. i'm afraid i won't make it to the other gathering.

hunmonk’s picture

hunmonk commented

i've been talking to other devs and thinking about this challenge. haven't come up w/ a clear answer yet, but want to let all know that it's being looked at. i'm currently debating whether i'd like to simply make signup's functions into available api's for other modules (which seems to be the defacto approach for handling the challenges that the module is facing), or taking a whole new approach which might better address the challenges, and also be useful going forward for other modules facing the same challenges.

some ideas i've been toying with:

  1. an SQL query builder -- this seems like it would make life a lot easier for core modules and plugins to work together at the query level. this approach was rejected in core in favor of the db_rewrite_sql approach, which seems a little messy to me.
  2. db table management API -- accessible by plugins, which would allow them to make changes to the core module's table structure, or add new tables if necessary. i'm imagining some plugins would need to store info in the db, but in many cases it would be more efficient to add to existing core tables.
  3. plugin management API -- i'd much rather see some kind of UI to manage plugins, as opposed to moving files around.

writing that code and moving signup module to that approach would require a fair amount of work, and at this point i'm not sure it would be worth it.

it's looking to me like we could either:

  1. use the 'just expose APIs' approach, which would be less time consuming and more in line w/ current standards, but a more limited solution, i think
  2. try to build out some basic engines that would better support a core/plugin approach, which would be more time consuming, less in line w/ current standards, but potentially much more robust

thoughts?

dww’s picture

dww
we/he/they
commented
Version: 4.6.x-1.x-dev » 5.x-2.x-dev

i brought this up as one of the main topics of discussion during the "development" break-out session at the bay area drupal meeting on thursday, and got some good feedback. alex (__tango) was there, along with neil (drumm), earl (merlinofchaos), adrian (amazon), and some other random folks. it seems like many of the things i wanted to do as a plugin could be very easily accomplished by just adding new contrib modules that work with signup.module. all the new tabs and blocks and such that i've written could be done as signup/contrib/signup_[foo].module files: very small modules that require signup, but extend the functionality in some way.

it also sounded like some of the 4.7 forms API stuff will help a lot in terms of collecting/displaying additional data, and that plugins could possibly alter existing pages more easily, without any (or at least only tiny) changes to the core signup module. i haven't look at the forms API at all yet, though i did finally start playing with a 4.7 test site and it seems pretty slick in many respects. the collapsable sets of fields are really nice.

at the same time, it seems like a lot of modules could benefit from more support in drupal core for module plugins, sub-modules, etc. in general, if the answer to the question "how do i modify xxx.module to add more functionality i need" is usually "you could just contribute your own version of the module that does what you want", that's going to tend to cause more module forking, something that both hurts the community as a whole and the individual module authors. i'd much rather it was easier to write sub-modules that interact with an existing module but allow more control to intercept and re-implement existing hooks, DB queries, menus, blocks, etc. instead of putting a bunch of work into signup to have a nice submodule hook system, it'd be much better in the end to have more support in core, if we can come up with a reasonable way to do it. plus, then people could extend/alter existing modules (tango was talking about needing to do similar stuff to the profile module) without needing coordination or even permission from the "core" module developers.

however, even if there were more support in core drupal for submodules like this, that's going to be quite a while before it's really done, accepted, and released. in 4.7, it'd probably help signup to have more of an internal API for a few of the key aspects of interacting with submodules (we'd have to hash that out in more detail -- basically all the stuff that's not easy/obvious to implement with the forms API, the update/install API, and possibly CCK/views). in 4.7, we'll add some contrib submodules that extend the functionality, and we can get more experience with the sorts of problems/issues that creates.

an SQL query builder -- this seems like it would make life a lot easier for core modules and plugins to work together at the query level. this approach was rejected in core in favor of the db_rewrite_sql approach, which seems a little messy to me.

i think it'd be great to have a more sophisticated SQL query re-writing interface, but that does seem like a ton of work. i didn't get the full picture, since i haven't looked at the 4.7 forms API, CCK, and views, but it sounds like the combination of those 3 might go a long way towards solving some of the problems i raised (at least that's what merlin and adrian were arguing). for example, adding new columns or rows to tables the core signup module is producing might be possible. i'd need to investigate more to see if that's really true, but it's another angle to consider.

db table management API -- accessible by plugins, which would allow them to make changes to the core module's table structure, or add new tables if necessary. i'm imagining some plugins would need to store info in the db, but in many cases it would be more efficient to add to existing core tables.

according to drumm, it sounds like there's a new install/update API in 4.7 core that's supposed to finally solve the problem of DB schema definitions and changes. so, i think in terms of this aspect, 4.7 might already have all we need. the plugin/contrib module could just use this interface directly, and if you install a given contrib sub-module that wants to add another field to signup_log, it should all work. that's all you really have in mind, right? 4.7 is also supposed to (though i'm not sure if it's actually in the 4.7 code yet) also have better support for managing dependencies between modules.

i'm moving this issue to "cvs" status, since it's clear we're not going to want to massively re-organize signup and add a new API in 4.6. we can still consider if any of the add-on functionality i've already done could be accomplished as contrib sub-modules in 4.6 without any real changes to the core 4.6 signup code. even without the 4.7 install/upgrade API, i could just document any manual changes to the DB schema needed when installing any given contrib module(s). 4.6 drupal admins are used to modifying their DB when installing new modules, anyway.

another angle to think about signups that someone (i think adrian) raised was that ultimately, it's another instance of the node relationship problem. if users were nodes (which i hear there's talk of moving towards), then a "signup" is just making a node relationship (between a user "node" and some other node), plus a bunch of metadata about the relationship. so, we should keep our eyes on the various node-relationship projects and see if someday we can't fold signup into that realm.

i'm glad you're still thinking about this issue. i hope 4.7 will make many of these problems easier to solve, and if we can get a reasonable proposal together for changes to core in 4.8 to make submodules/plugins easier for all drupal modules, i think that'd be great.

thanks,
-derek

hunmonk’s picture

hunmonk commented

it also sounded like some of the 4.7 forms API stuff will help a lot in terms of collecting/displaying additional data, and that plugins could possibly alter existing pages more easily

it's great for altering form output--you can pretty much do anything you want. but drupal output is certainly more than forms :) currently, modules have some control over core drupal output through hooks, but it's limited. adrian has discussed implementing the same basic approach he used w/ form API to the rest of drupal core output, which would be fantastic for contrib module authors. we'll have to keep an eye on that.

it seems like many of the things i wanted to do as a plugin could be very easily accomplished by just adding new contrib modules that work with signup.module. all the new tabs and blocks and such that i've written could be done as signup/contrib/signup_[foo].module files

this is certainly a valid approach, and was the first one i thought of. the combination of exposing singup core APIs and writing submodules would solve many issues that we're wrestling with, and probably wouldn't be all that time consuming compared to a more ground-up approach. i can't help but think that it still leaves some basic issues unaddressed, but even so it might be the next best step.

at the same time, it seems like a lot of modules could benefit from more support in drupal core for module plugins, sub-modules, etc.

i couldn't agree more here. the tricky part is that core development is not under our control :) it can certainly be influenced, but no amount of lobbying will guarantee any change. because of this, my general approach to feature addition in drupal is to add it in contrib first, if at all possible. this has the double benefit of making it available sooner for people, and providing a good testing ground for possible future inclusion in core. more than one current core feature started out this way. given that, my inclination is to write a contrib module that provides tools for other modules to more easily use a core/plugin model.

that's going to be quite a while before it's really done, accepted, and released. in 4.7, it'd probably help signup to have more of an internal API for a few of the key aspects of interacting with submodules

really, there's no reason why we can't both expose the APIs, and provide some general external tools.

i think it'd be great to have a more sophisticated SQL query re-writing interface, but that does seem like a ton of work

not when there's already been some code written for it :) it's possible that there may be enough existing code lying around that we can cobble together a set of tools more easily than i initially thought. i've already seen query building code in moshe's sandbox.

according to drumm, it sounds like there's a new install/update API in 4.7 core that's supposed to finally solve the problem of DB schema definitions and changes

i haven't tried out the new install hook yet, so i don't really know if it'll work for our needs. it sounds like it would...

it's clear we're not going to want to massively re-organize signup and add a new API in 4.6

from a development perspective it's clear. however, there is a fair amount of interest in continuing development for the 4.6 version of signup, which might be enough to motivate me to do the same kind of work there. this is something i'll have to think more about.

so, we should keep our eyes on the various node-relationship projects and see if someday we can't fold signup into that realm.

agreed. but for now i think we should either:

  1. do development with existing tools
  2. write our own stuff if we deem it's necessary

of course i'm tremendously impatient, so i would say this... ;)

dww’s picture

dww
we/he/they
commented
Version: 5.x-2.x-dev » 4.6.x-1.x-dev

it's great for altering form output--you can pretty much do anything you want. but drupal output is certainly more than forms :)

adrian (or merlin, i forget now exactly who it was) said something like "the forms API is a misnomer, it should really be called the page rendering API", and i got the impression it was able to do more than just forms. but, he was somewhat vague at times, and not completely clear to differentiate his opinions about how things should be, vs. how things really are in 4.7. ;)

given that, my inclination is to write a contrib module that provides tools for other modules to more easily use a core/plugin model

interesting, i hadn't thought of that. you mean something like writing a completely stand-alone "plugin.module" that uses the existing module hook API to provide some additional APIs that module authors could use to make their modules more plugin friendly, right? yeah, if we can find a way to do what we want to do to signup generically enough that this could be done to other modules, i'm all for this. basically, put the plugin API as a separate module, and then port signup to that. if other contrib modules could benefit from it, they could use it, too, and if we gather enough steam, it might be folded into drupal core some day.

really, there's no reason why we can't both expose the APIs, and provide some general external tools.

right, that's what i meant. regardless of whatever other APIs/hooks come into existance, there's no reason not to make signup's core functionality more API-driven, so that both the core signup and any contrib signup modules can share more code.

not when there's already been some code written for it :) it's possible that there may be enough existing code lying around that we can cobble together a set of tools more easily than i initially thought. i've already seen query building code in moshe's sandbox.

ok, that's great. if this is mostly done already, i'm happy to use it and/or help get it ready for use.

from a development perspective it's clear. however, there is a fair amount of interest in continuing development for the 4.6 version of signup, which might be enough to motivate me to do the same kind of work there. this is something i'll have to think more about.

ok, sure. i'm moving the version back to "4.6" then. ;) personally, a) i'm only using 4.6 for the site i actually care about and b) i've already got it all working how i want via local modifications to signup.module in my own local cvs repository. i don't see myself switching my whole site over to 4.7 anytime soon, so i'd be interested in continued development on 4.6. furthermore, if many of the things i've done already could be easily re-factored into seperate contrib sub-modules in 4.6 (which i think they can), i'd be willing to do that work, just to a) make my changes/enhancements available to other folks who want/need them and b) help move this process along for longer-term signup development.

-derek

duaelfr’s picture

duaelfr
he/him
Primary language French
Location Montpellier, France
commented
Status: Active » Closed (won't fix)

This version of Signup is not supported anymore. The issue is closed for this reason.
Please upgrade to a supported version and feel free to reopen the issue on the new version if applicable.

This issue has been automagically closed by a script.

simon georges’s picture

simon georges commented
Status: Closed (won't fix) » Active

Reverting recent closing.