attached patch is a working deletion API for core. the deletion API will provide a standardized method through which deletions from the database can be managed more completely. it offers several hooks, both prior to deletion, which allow outside modules to display messages on a confirmation screen, process the data prior to deletion, or abort deletion altogether. currently the API works with all node/comment deletions, and can be easily expanded to deletion of pretty much any kind of content. couple of things to note:

  1. i'm not attached to any of the naming conventions i've used -- if if somebody comes up with names that feel more intuitive, i'm more than willing to adjust things.
  2. the hook examples in comment module are just that -- examples to illustrate some of the useful features of the API with regards to module input, and are not meant for inclusion in core in their current state.
  3. there is an excellent demo site here: http://deleteapi.xcarnated.com

    the site includes an overview of the API, links to the code that makes it work, and serves as a test ground to see it in action. please visit it if you want more details than can be gleaned from the patch itself... :)

Comments

webchick’s picture

Ooooo!!

Subscribing! Will definitely test this later.

Arto’s picture

Subscribing, and echoing the above sentiment...

fgm’s picture

This is certainly interesting, but besides the pre/post hook, it seems to essentially reinvent long-lived transactions in a specific case, by calling them packages. Shouldn't this be abstracted yet further, to take advantage of engine-level support for such ?

chx’s picture

MySQL 4.1 does not support transactions.

mfb’s picture

This would be very useful for ecommerce, with regards to products. I would note that similar APIs (with pre and post hooks) are needed elsewhere in ecommerce, e.g. transaction workflow and payment (for deletion, cancelation, completion, etc.)

hunmonk’s picture

@fgm: as i was building this, i immediately began to see that it could be leveraged for transactions -- but most drupal installations aren't running a transaction storage engine, and i felt that the code to properly determine that somebody was running a transaction engine would be clunky, so i abandoned that idea.

i would even have settled for a situation where after the queries were compiled they could be submitted in a batch -- but mysql_query() does not support batch query submissions. :( i think this is about as close as we can reasonably get given the limitations...

hunmonk’s picture

StatusFileSize
new39.52 KB

keeping up w/ HEAD.

hunmonk’s picture

StatusFileSize
new39.95 KB

keeping up w/ HEAD. i had over 100 revisions to the trashbin patch -- does this patch have the same stamina?? ;)

davidtrainer’s picture

I just watched your lightning talk on this - nice job. I do agree with and wanted to record the one small UI gripe that someone brought up. If the deletion is going to be aborted, that logic ought to come before the confirm dialog, which shouldn't appear at all. Anything the user "confirms" should actually happen.

dmitrig01’s picture

/me must subscribe...
hunmnok++

hunmonk’s picture

StatusFileSize
new39.99 KB

this version of the patch implements the previous request -- abortions are now first class operations. the module requesting the abort can pass both a message and a redirect destination to the API.

the only minor UI issue with this implementation is that on multiple deletions, if you're deleting multiple items, and some will be aborted and others won't, the aborted items will still be listed on the confirm screen even though the API posts a warning that it will be aborted. this is because the main information for the confirm form is generated when the confirm page is requested, and all items are in there. i *could* write around this, but i believe that it will unnecessarily complicate the API -- it's pretty obvious what's happening given the warning message, so i'd like to leave it as is.

again, please note that the stuff in comment module is in there to serve as a working example of the API hooks, and is not intended for core.

test site is still here: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new38.73 KB

pulling out some cruft per Steven.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new38.77 KB

broken by confirm form fix/language patch. updated patch attached.

test site: http://deleteapi.xcarnated.com/

somes’s picture

Does this allow easier user interface deletion with AJAX - might a pop out div be used with the new system to confirm that the node should be indeed deleted

hunmonk’s picture

@Somes: this is a back-end API, not front end user candy -- AJAX delete would be feature creep for this patch. please file another issue to discuss that very valid feature request. :)

hunmonk’s picture

StatusFileSize
new38.77 KB

keeping up w/ HEAD.

hunmonk’s picture

StatusFileSize
new37.72 KB

broken by menu module updates. updated patch attached.

hunmonk’s picture

StatusFileSize
new37.72 KB

attached patch fixes a PHP 5 issue where array_merge_recursive() results in NULL if NULL is merged into another array.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new37.81 KB

keeping up with HEAD.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new37.79 KB

broken by the watchdog translation patch. attached patch corrects.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new37.78 KB

broken by the menu translation patch -- updated patch attached.

keeping up can be murder sometimes... *wipes brow* :)

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new37.79 KB

keeping up w/ HEAD.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new37.74 KB

broken by the fapi 3 patch -- updated patch attached.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

StatusFileSize
new37.78 KB

broken by fapi changes. updated patch attached.

test site: http://deleteapi.xcarnated.com/

hunmonk’s picture

it's now two weeks before the code freeze, and this issue has yet to receive any feedback from a core committer. if anybody cares about improving drupal's very limited deletion cycle capabilities, now is the time to speak up! :)

patch applies to current HEAD, works as advertised, and there's a test site w/ exhaustive documentation. i need feedback on this patch ASAP, so i have time to make any necessary corrections before the freeze.

i have carried the deletion torch for two major release cycles, with no real progress in core -- this will be my last attempt to help this area of drupal... :)

test site: http://deleteapi.xcarnated.com

merlinofchaos’s picture

Priority: Normal » Critical

subscribing. Upping the priority a bit, becuase I think this is good functionality.

hunmonk’s picture

StatusFileSize
new37.62 KB

attached patch makes some minor documentation corrections -- all functionality is the same.

i've taken the time to make sure the documentation/examples on the test site are all in order as well:

test site: http://deleteapi.xcarnated.com

Stefan Nagtegaal’s picture

Status: Needs review » Reviewed & tested by the community

This is very nice.. Itested this very heavily, and I dare to set this RTBC..
There are no errors showing up, and everything works smoothly and without any problems...

Dries pls commit this or leave your comment here.
Hunmonk is keeping this patch up for ages now and the functionality is really good..

hunmonk’s picture

StatusFileSize
new33.52 KB

well if we're going to be bold, let me pull out the example hooks i had in comment module, since they should probably be part of another patch!

attached patch is exactly the same, with the example comment hooks removed.

i'm hesitant to leave this RTBC, but what the hell... ;)

hunmonk’s picture

StatusFileSize
new33.4 KB

minor conflict w/ the multiple menu patch. attached patch corrects, and has been tested as working.

test site: http://deleteapi.xcarnated.com

drewish’s picture

Status: Reviewed & tested by the community » Needs work

- I don't think it's safe to hijack confirm_form() to delete specific.
- The form for confirming the deletion of comments via admin/content/comment has T as a button title.
- The UI at admin/content/node is very confusing when part of a deletion is aborted. The user is told that their node cannot be deleted but still offered a button to try it anyway.

drewish’s picture

The deleting multiple comments from /admin/content/comment redirects to itself so you get the menu tabs and help text which is confusing. I checked a clean copy of HEAD and this is an issue there as well.

hunmonk’s picture

Status: Needs work » Needs review
StatusFileSize
new39.04 KB

I don't think it's safe to hijack confirm_form() to delete specific.

i disagree with that assessment. the confirm_form isn't being hijacked at all -- i've simply repurposed it with a structure similar to the recent l() and url() patches, and gave it sensible defaults. since the confirm form is most often used for delete confirmation, it makes sense for it to have a default setup for deletions -- the core code for delete confirm forms gets much cleaner this way. and the confirm form can still be used in all the ways that it used to be...

The form for confirming the deletion of comments via admin/content/comment has T as a button title.

fixed.

The UI at admin/content/node is very confusing when part of a deletion is aborted. The user is told that their node cannot be deleted but still offered a button to try it anyway

also fixed. i struggled with how to handle this -- it's tricky b/c you have multiple packages, but you want everything displayed through one confirm form. i tackled it by a new feature in the API -- allowing package-specific confirm form elements to be passed in. this way, if the package is aborted, then those elements can be tossed instead of merged into the confirm form.

along the way, i greatly improved the redirection logic for the API, resulting in much more sensible fallbacks in the case of aborted deletions.

The deleting multiple comments from /admin/content/comment redirects to itself so you get the menu tabs and help text which is confusing. I checked a clean copy of HEAD and this is an issue there as well.

since this is a current problem in HEAD, i'd like to vote that we tackle it with another issue... :)

new patch attached. let's keep banging away!

NOTE: http://drupal.org/node/141131 broke comment forms in HEAD! i've adjusted the test site to accomodate this until the issue is fixed. if you want to try this patch locally, you can comment out the call to setrawcookie() in comment.module to work around the issue.

test site: http://deleteapi.xcarnated.com

hunmonk’s picture

StatusFileSize
new39.04 KB

minor typo that was preventing comment nodeapi deletions from being properly checked.

also, i've put a small example module on the test site which better demonstrates custom message and abort features of the API -- if a node has more than two comments, deletion will not be allowed. if a comment subject has the word 'abort' in it, it also cannot be deleted. messages regarding the number of comments deleted with a node are posted to both the confirm form, and as a user message after deletion.

i'll try to load up that example module so that folks can install it locally if they'd like.

and as always...

test site: http://deleteapi.xcarnated.com

hunmonk’s picture

StatusFileSize
new1.83 KB

ok, attached is the example module i described in the previous comment.

hunmonk’s picture

StatusFileSize
new1.86 KB

whoop. that last example module i posted had a bug in it. this one should do much better. :)

hunmonk’s picture

StatusFileSize
new39.2 KB

drewish didn't like the way the abort messages at admin/content/node and admin/content/comment where displayed both on the confirm form, and after the deletions were executed. after further thought, i agreed that it was potentially confusing, so the following patch corrects that UI issue.

it's actually quite slick now that confirm form elements can be added per package -- you simply include the hidden confirm form elements in that way, and if a package is aborted, then it's not even considered on the confirm screen at all!

test site: http://deleteapi.xcarnated.com

drewish’s picture

hunmonk, I'm really impressed by what you've gotten done here. The interface is great. I've started trying to look through the code. Noticed a few things but I'll give it a closer look later.

+++ modules/comment/comment.module	2007-05-19 05:34:57 +0000
@@ -232,7 +232,8 @@ function comment_menu() {
 
   $items['comment/delete'] = array(
     'title' => 'Delete comment',
-    'page callback' => 'comment_delete',
+    'page callback' => 'drupal_get_form',
+    'page arguments' => array('comment_delete', 2),
     'access arguments' => array('administer comments'),
     'type' => MENU_CALLBACK,
   );

should that be arg(2)? since it should be a comment id?

+  // array_filter() returns only elements with TRUE values, array_keys() returns the nids.
+  $nids = array_filter($edit['nodes']);

I don't think you call array_keys().

The parameter format of drupal_delete() is a little odd. i'd almost rather see all the options passed in as an array rather than a variable number of parameters.

I'm also going to voice my opposition to making confirm_form() default to confirming a deletion.

hunmonk’s picture

StatusFileSize
new39.2 KB

should that be arg(2)? since it should be a comment id?

i'm pretty sure that's the correct way to pass arg 2 in the new menu system...

I don't think you call array_keys().

whoop, i removed it from the code and not the comment :) attached patch corrects.

The parameter format of drupal_delete() is a little odd. i'd almost rather see all the options passed in as an array rather than a variable number of parameters.

i'm not sure how viable that is, since the function needs to be so flexible. leaving the variable params makes it very easy to pass in the query stuff. also, $id and $op are always present, while the other args change depending on $op. i'm pretty sure what i have is the cleanest approach, but i'm open to hearing better ones.

I'm also going to voice my opposition to making confirm_form() default to confirming a deletion.

it wouldn't be that hard to make another function, delete_confirm_form(), and make that do what i'm making confirm_form() do now. i'm open to adding that, but i really don't think it's necessary -- as i outlined above, most uses of confirm_form() are for delete confirmation, and the function is still flexible enough to do everythign it used to do.

test site: http://deleteapi.xcarnated.com

hunmonk’s picture

StatusFileSize
new52.17 KB

gah, last patch wasn't quite right -- here's the one w/ the fix...

hunmonk’s picture

StatusFileSize
new39.17 KB

i swear this is the right one... :/

test site: http://deleteapi.xcarnated.com

chx’s picture

Status: Needs review » Needs work
  1. Additional arguments are passed to the function, depending on the operation to be performed: Give full function call examples, it's very confusing otherwise.
  2. I really would like to see a research into this api crossed with batch, not necessarily for D6, but onwards.
  3. drupal_delete('', 'set package', $package_id); No need. Just $delete_id = $package_id will do.
  4. if (count($package_info) > 3) { 'Once the number three, being the third number, be reached, then, lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.' is this the reason? or...?
  5. module_invoke_all('delete_execute', $package_info); make this a drupal_alter('delete', $package_info).
hunmonk’s picture

Status: Needs work » Needs review
StatusFileSize
new40.48 KB

great review chx! all of your concerns are addressed in the attached patch. i'll need to post a new .tgz of the example module with that change to drupal_alter(). to follow shortly...

test site: http://deleteapi.xcarnated.com

hunmonk’s picture

StatusFileSize
new1.89 KB

... and the new example module...

hunmonk’s picture

StatusFileSize
new40.49 KB

after some more thought on drewish's suggestion to not hijack the confirm_form() function for the deletion API, i've rearranged a few things to keep that function more general. i did keep the options array, because it seems more in keeping with the current trend to eliminate functions with eight billion arguments. :)

the submit handler for DAPI has been moved into the api itself, and a more specific function name has been given to the callback. the _only_ thing left in the confirm form that's specific to deletion is that the 'yes' button defaults to 'Delete'. the original default was 'Confirm', which i don't think was all that useful anyways.

patch attached.

http://deleteapi.xcarnated.com

hunmonk’s picture

StatusFileSize
new40.43 KB

broken by the schema patch. update patch attached.

david strauss’s picture

@chx (from a long ways back) MySQL 4.1 supports transactions fine. It's MyISAM that doesn't.

chx’s picture

Sure. I know. TO be more on topic, what else is needed to get this in for D6?

david strauss’s picture

Sorry, but I think this patch is a bad idea. It reinvents a wheel carefully rounded, inflated, and balanced by the database world. Worse yet, it uses new terminology (like "packages") to refer to concepts long familiar to database developers. It would be much better to add foreign key support to the schema API and support code to emulate foreign key CASCADE DELETEs for MyISAM tables. (They would run slower than native CASCADE DELETEs but work.)

There might be justification for allowing modules to attach messages to delete forms that will affect their data, but modules can already do that though hook_form_alter().

david strauss’s picture

After playing with the demo, deletion API seems to also implement an analogue of RESTRICT DELETE on foreign keys.

david strauss’s picture

I should re-characterize my discontent. I'd love this patch if:
* Terminology were changed to make it consistent with standard database operations (foreign keys and transactions)
* It tried to use native database operations where possible
* (Maybe) it integrated with Schema API to identify deletion dependencies for RESTRICT and CASCADE

I'm sorry to be so hard on this patch, but the more we do in PHP, the slower, more chatty, and less reliable operation becomes with the database.

nedjo’s picture

I like the idea of a deletion API and appreciate the thought and effort that's gone into this patch.

However, I share some of David's concerns. A deletion API should be a step toward a unified, schema-based data API. This patch doesn't read as such a step.

Is this the approach we will also want to be taking for loading, creating, and updating objects?

I've tried to sketch out some of the steps toward a unified data API in the patch

http://drupal.org/node/113435

(which includes a minimalist deletion API, also triggered by a drupal_delete() function)

and the issue

http://drupal.org/node/145684

I suspect that little of the complex handling implemented in this patch will be needed when we have consistent schema-aware data APIs.

hunmonk’s picture

I should re-characterize my discontent. I'd love this patch if:
* Terminology were changed to make it consistent with standard database operations (foreign keys and transactions)
* It tried to use native database operations where possible
* (Maybe) it integrated with Schema API to identify deletion dependencies for RESTRICT and CASCADE

in response:

  • i have no problem changing the terminology -- we can call packages 'widgets' -- i don't care :) if you'll lay out more clear terms instead of the ones i currently have, i'm happy to change them. i consider this to be an easily fixable problem.
  • why are we discussing database level transaction functionality, when we can't support it?? afaik, we're still going to support MyISAM tables, so doing transaction level stuff at the database isn't an option. i'm pretty sure most people with a MySQL install are using MyISAM.
  • the central point of this API is to provide application level support for deletions -- if we don't have it in some form, then the drupal's deletion cycle is blind to any information that outside (non-core) modules have regarding that deletion.

I suspect that little of the complex handling implemented in this patch will be needed when we have consistent schema-aware data

i haven't looked at the data API closely yet, so i won't comment on the idea that the approach obsoletes this one. what i do know is this patch works now, solves deletion related problems now, and has been heavily tested. afaik, there's no guarantee that the data API will go in next cycle, either, and we can benefit from this now.

david strauss’s picture

@hunmonk It's important to discuss foreign keys and transactions because those are built into InnoDB and PostgreSQL, and they run far faster and more reliably than any equivalent in PHP. We're really going to hurt ourselves if we only use what MyISAM supports natively. What we can do is either emulate the functionality for MyISAM in PHP or provide a safe fallback.

This patch -> database equivalent
* Deleting associated items automatically -> Foreign key with CASCADE
* Restricting deletion when associated items exist -> Foreign key with RESTRICT
* Responding in a custom way to deletions -> Triggers

If we set this system up in a DB-centric way, we can map as many operations as possible to the native DB system. As this code is structured now, it's impossible to offload tasks to the database.

hunmonk’s picture

@David Strauss:
i'm *pretty* sure that a large majority of people are using MySQL with MyISAM tables for their installation. if this is the case, then it's doubtful that we would drop support for MyISAM anytime soon. And from what i recall from my research into database engines, MyISAM is actually the right choice most of the time for a website -- it's not transaction-safe, but it's very fast on reads, which is what most websites need.

if we support both systems, then we've got a new problem: more code to maintain. if you strip out the code comments from this patch, it weighs in pretty light, and it works for all systems. that's a pretty big advantage. i'll agree that it's slower, but so what? in almost all site scenarios, deletions are a fairly rare occurrance -- so performance isn't as much of a consideration as it would be for something like a function that is called every page load.

so far i'm not convinced that moving things to the database layer would be an overall win.

hunmonk’s picture

StatusFileSize
new40.43 KB

keeping up w/ HEAD.

chx’s picture

No patch can save the world, not even a Drupal core patch. If it works and does something useful on its own, then it is good to go. One of the worst things you can do is elaborate on other, equivalent approaches and suggest complex extensions to the patch. Additional features can be added later. A scalpel is often better than the Swiss Knife.

This patch is a good one, various goodness based on schema can happen. And yes, it should happen later but for now we have a working Deletion API here and no code for a schema based deletion API.

drewish’s picture

I agree, with chx. it'd be great to have this done at the DB layer but right now we've got nothing and hunmonk has come up with a solution at the application layer. As he points out deletions are rare so efficiencies aren't as critical as they might be if this affected viewing.

david strauss’s picture

The recent posters make some good points. Having something to influence or deny deletion is definitely a good idea, and it is certainly a fairly rare operation. We can revisit DB integration with this if/when we drop support for MyISAM or MyISAM gains InnoDB's features. (The latter will happen somewhere in the 5.x series.)

nedjo’s picture

That this patch currently works counts for a lot. But in itself it's not enough.

A scalpel is often better than the Swiss Knife.

I'm concerned that this is the Swiss knife more than the scalpel.

From the same patch reviwing guidelines chx cited, http://drupal.org/patch/review:

Does it make good use of our systems or does it re-invent the wheel?

The Schema API is part of Drupal core. Yes, it's new. Yes, we haven't yet scratched the surface of what it will enable us to do.

But at the least, it will be our best way of answering questions like "what relationships does this item have?". Foreign key attributes were removed from the initial patch, but they're the next logical step.

This patch invents a custom way of mapping those relationships. At this point, that feels like a step backward rather than forward.

Can you see the changes being used elsewhere too?

Drupal is full of one-off data APIs. We have a different set of APIs for each major data type. The way to fix that is not to do another one-off API. A deletion API is the wrong goal. What we need is a unified data API, one that includes deletion.

A new API generates momentum. We introduce it because we want it used. Hence we should introduce a deletion API because it's a step toward a unified, schema-based data API. If it rather blocks then enables that path, it's the wrong decision, however pleasing or handy it may appear on its internal merits.

I'm not saying this patch shouldn't go in. Just that I have two concerns that haven't yet been addressed and IMO should be first:

1. How will parallel update, insert, and load methods work? Can the same basic logic apply there? For each of those methods, we need to answer basically the same questions: what are the relationships of this item? how can I (load, update, insert) those related items?

2. How will this API be changed ASAP to use the schema API instead of its own custom approach? How is it schema-ready, even if it doesn't yet use the schema API? Has it isolated the 'what are the relationships of this item' question in such a way that it will be easy to point at the schema API instead? How does it avoid promoting a plethora of new code in Drupal core and contrib that uses obsolete methods?

I don't mean we need working code. We just need to be confident these questions have been answered, or at least considered and thought through.

chx’s picture

nedjo, could we stop talking schema? This is a working patch which would be quite good for the Drupal 6 cycle. If and when Schema will have relationships -- it does not have that yet -- we can do more. Not now. Please.

hunmonk’s picture

from a conversation i just had w/ chx:

hunmonk: nedjo keeps mentioning that the deletion API needs to work w/ the schema API, but i'm not clear why that's an issue

chx: what David and Nedjo wants is the following:
in the future -- it's not even in yet -- we will add relationship to schema -- foreign keys. then you can know how to delete an object
while advanced databases can use cascading deletes. mysql needs supporting php code

hunmonk: but that has almost nothing to do w/ the deletion API. the deletion API's main goal is to open the deletion cycle to all modules. how the deletions are done is a fairly minor consideration in the API as far as i'm concerned. the important point is that modules have access to the relevant deletion information, and can take action with it. am i off base here??

chx: I answered. I recommend you closing the issue and opening a new one. this is clouded beyond recognition

hunmonk: well from your answer, it seems like the only adjustment that would be needed is a switch, which would perform the deletions based upon what the underlying database can do. we can't even put that switch in until the schema API supports that stuff

chx: yeah

i couldn't agree more with his assessment -- we're losing the focus of this patch. again, this API is about opening drupal's very closed deletion cycle to outside modules. the reasons and benefits are clearly laid out on the test site. the actual final method of deletion is a very minor part of this code, and can be changed _if and when_ we have other, better ways of doing it.

i'll be closing this issue shortly and reposting so we can get a fresh start here...

hunmonk’s picture

Status: Needs review » Closed (fixed)

closing. duplicate of http://drupal.org/node/147723