File this under 'things I really should have noticed two days ago before the freeze.'

Basically, quite a few modules, like upload, inline, links, insert_view, freelinking, and so on do processing of the node's body for formatting. The important thing to note is that these modules *alter* content rather than adding to it.

When the big node rendering refactoring went in, we added an #after_render mechanism to drupal_render. We thought that this would be sufficient for such uses, but it is not by any means. Why? Those modules need the current node and its context to do their filtering properly, something they get with nodeapi view but NOT with the #after_build rendering construct. This was made to work with upload.module, but only just barely: I've been working on porting three more modules over to the 5.0 APIs, and they will not work with this mechanism.

The following patch fixes the problem by adding one hook: hook_node_content_alter($node, $text, $teaser, $page). It's very simple: after all the rendering is done, it gives modules a chance to alter JUST the rendered text while using the complete node context. This one hook solves the problem, greatly simplifies things for implementing modules, AND eliminates the now-unecessary call to hook_nodeapi_alter, which was previously used just for inserting references to #after_build, AND lets themes call node_content_alter($node, $node->content['particular_field']) to get the properly filtered text of a single field in the node, something that was previously impossible as well.

Seeing as how the code freeze is a day passed already, I hold little hope for this fix making it in. As the guy who wrote the patch that *broke* this functionality, though, I'd implore the powers that be to consider this an emergency. Without the fix, we are breaking a number of mission-critical modules for drupal site builders.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

eaton’s picture

FileSize
4.58 KB

A slightly modified version. Since nodeapi 'alter' already went in, I've preserved it. This *only* adds the additional hook_node_content_alter() function. If there's a better name for it I'm all ears.

eafarris’s picture

Thanks for this, Jeff.

It looks like freelinking is still working for me in HEAD, how can I see this bug? Freelinking does not require knowing the "node and its context." Though that would be nice, the filtering system never gave filters this information before. Filters just get a piece of text and return a piece of text, no context.

What am I missing here?

eaton’s picture

My apologies -- looking at an earlier piece of code, I had mistakenly thought that freelinking used the nodeapi view operation, rather than the filter operation. :-) The problem is that other modules like inline *do* require that context, and without this patch, they would become crippled in the same way that filter operations are -- they have no context.

profix898’s picture

Yes, this is a critical patch. I'm also maintaining a module (reptag) using _nodeapi view to modify a node depending on its context. But is there any reason to introduce a new hook? Why dont you add another operation to hook_nodeapi, e.g. view_context or view_alter?
Your '... it lets themes call node_content_alter($node, $node->content['particular_field']) ...' is a nice idea, but its additional functionality.
However, I'm all for a patch like this :)

eaton’s picture

The reasons lie in the relatively complicated inner workings of nodeapi. Only the first parameter of a nodeapi call -- the node itself -- is passed in by reference. Thus, it would be impossible to pass in $node, $op, then $text as the $text is what needs to be altered.

We COULD pass in $text as the first parameter and node as one of the later ones, but that would be counter to the defined interfaces of nodeapi and cause a lot of needless confusion.

We COULD define a third nodeapi operation for node rendering, one that lets modules alter $node->body or $node->teaser after all the rendering is done. That, though, runs counter to the new rendering model wherein the $node->body and $node->teaser are used just to hold the post-rendering html.

If we were to do that, it might make sense to ditch the nodeapi 'alter' call's current purpose -- rearranging elements, etc -- and use it purely for this purpose. However, that removes the secondary phrase of the rendering that we'd added so that modules could modify eahc others' additions. This option, of the above, is probably the cleanest, although it trades alteration-of-the-content-array for alteration-of-the-rendered-body.

This solution doesn't break the already-defined features in 5.0's APIs, and solves a second problem for themes while we're at it. I can understand if the solution in the curent version of the pathc is not favored for that reason, but I'm hoping and praying that it gets in. I got sucked into FormAPI work shortly after doing the nodeapi refactoring, and didn't spot this issue until actually porting a module.

eaton’s picture

FileSize
4.76 KB

Fixed an error on priview with submit.module.

drumm’s picture

I'll leave this for Dries to review, but I think it may be an okay solution. It is god that this is not altering any existing APIs.

profix898’s picture

FileSize
4.4 KB

Just installed the patch and it works nice. I really hope this one makes it in ...
(Replaced 'AS' with 'as' since thats the usual case in core)
+1

@Eaton: Thanks for explanation. I actually didnt have time to track all core changes over summer. But after reading the update modules guide and a quick look through the code it really makes sense.

Dries’s picture

This used to work with Drupal 4.7, not? Why is it broken now? Why can't we make it work like in Drupal 4.7?

eaton’s picture

In Drupal 4.7, $node->body was assembled as pure text, the way forms were in 4.6. In Drupal 5.0, $node->content is assembled as a structured array, and modules implementing nodeapi op view can add their elements to $node->content. Later, that's rendered into the text that node_view() returns.

Modules like inline need access to the entire post-render text, not just the array of data, to do their substitution work. They *can* use the #after_render callback via the drupal_render() method, but that only gives them the text, not the node object that contains context for the substitution (like the nid, the file array, and so on).

To make it operate like 4.7, we can do one of two things: roll back the node rendering patch, or apply this patch.

profix898’s picture

Bringing this issue into developer's scope again. Since it implies an API change (or extension actually) it should be discussed early after the code-freeze IMO to give contrib developers time to adopt.

drumm’s picture

Status: Needs review » Needs work
+    if (count(previews)) {

Thats not right.

drupal_render() is called a few times in the context of node content. Does node_content_alter() need to be called elsewhere? If not, what is the advantage of putting it in a function?

eaton’s picture

If other modules want to pull out a sub-chunk of the node for display in a sidebar block (for example), they would need to call that function to properly process the chunk for display. That was the only real reason for putting it into a separate function.

eaton’s picture

FileSize
4.28 KB

previews/$previews typo fixed.

moshe weitzman’s picture

can we not include a copy of #node as context for modules to do their #after_render? thats consistent with node_form().

i'm not so fond of this. we have too many hooks into the view process already. the pipeline goes:

hook_view()
nodeapi('view')
nodeapi('alter')
field specific theme functions
#after_render

lets try not to add another :(

eaton’s picture

Moshe,

If you think that would be sufficient, I'd be willingto roll up a patch that includes $node, $teaser, and $page in flags on the main $node->content[] array. While it would still leave most themes locked out of the game as far as printing subsets of the content array, you're correct that it would make it possible for modules to work around the problem.

In retrospect, removing nodeapi's 'view' and 'alter' operations, and removing #after_render, in favor of two operations (alter the content array, or alter the rendered node) would have been a much better course of action. I really, really regret that wasn't something that I realized before the freeze.

eaton’s picture

It just occurred to me, though. Wouldn't that mean that we're setting $node->content['#node'] to $node? Does PHP handle those sorts of circular references properly..?

moshe weitzman’s picture

I was thinking that the value of #node would be the object cominhg from node_load() ... your idea to include additional flags is good.

personally, i think it is fine to simplify the API as you describe. very few modules have upgraded at all, and even fewer those who need these specialized operations.

eaton’s picture

FileSize
4.36 KB

Well, this definitely changes APIs more than the previous one did, but it does so in a 'reasonable' fashion. It's untested, but an example of what Moshe's suggestion might yield:

The node rendering pipeline, with this patch, goes:

  1. hook_view()
  2. nodeapi 'alter' (where the raw content array can be altered)
  3. node body/teaser is rendered
  4. nodeapi 'view' (where the post-rendering text can be altered, just like the 4.7 era nodeapi 'view')

This will still force some updates on the part of other modules, but it also makes a relatively clear distinction betwen 'the operation you should use to do post-render filtering,' and 'the operation you should use to add to or modify the node's contents.'

Thoughts?

eaton’s picture

Status: Needs work » Needs review
FileSize
4.88 KB

Here's a slightly updated version of the patch that keeps things simpler and fixes this problem. Please -- really, please! consider the importance of this issue as we go forward with conversions and updates of contrib.

Changes:

1) nodeapi view operates in the standard 'HEAD' way -- modules can add chunks to the $node->content array.
2) nodeapi 'alter' is now fired after the node is rendered from that content array. thus, 'alter' serves as a last-step chance for modules to directly alter a node's text. It makes sense and is much more useful there than as a duplicate of the 'view' op.
3) The unecessary 'after_render' marker in drupal_render() is removed. This just slowed down other rendering operations unecessarily and added another 'magic secret flag' to our already complex structure.

eaton’s picture

FileSize
4.66 KB

Fixed a minor array/object bug that was keeping upload.module from recognizing files during previews.

eaton’s picture

At chx's request, a very simple nutshell explanation of why the patch is necessary:

In Drupal 5.0, node content is built as a structured array, then rendered into text after all modules have contributed their chunks to that array. This works great, except for the few modules that actually DO need to modify the entire fully-rendered contents of the node (like inline.module, premium.module, and upload.module). 5.0 currently includes a rudimentary workaround for it (the #after_render flag), but this only gives modules access to the *text* of the node, cutting them off from important contextual information in the node object.

This patch fixes that by removing the relatively useless #after_render flag, and simplifying the node processing into into a 'view' op (for assembling portions of the node), and an 'alter' op (for modules that need to hack at the text after it's been completely rendered).

dopry’s picture

I've applied the patch to a current head. It applies cleanly. Uploads previews work properly with clean urls off. Previews won't work with clean urls on because of the .htaccess in the files directory. You have to comment out RewriteEngine Off and add Options +FollowSymlinks . It would be nice to see a sample implementation for inline or premium since they are the use case and upload works before and after this patch. Better op names would be nice as well, 'build view'/'process view' or something like that. We've got alteritus. Anyway +1 for more context when manipulating output content.

moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

i reviewed the patch and it does what it says. i think renaming the operation is not useful so late in release cycle. also, there may indeed be a problem with .htaccess but thats outside the scope of this patch.

eaton’s picture

*poke poke*

Hoping this can make it in. The patch is much cleaner than previous iterations and will make life much, much, much easier.

As in, not broken. :-)

drumm’s picture

Status: Reviewed & tested by the community » Needs work

Lets leave #after_render in form.inc. That would be one less API change. (It can probably be removed for the next release cycle.)

Does hook_nodeapi('alter') need to be moved? How will this affect other modules implementing this hook?

eaton’s picture

We could leave #after_render in, but it's only used in one location as a workaround for something that this patch fixes better. The modules that would be affected by the change -- those that might be using #after_render OR the 'alter' operation -- have not yet been ported to 5.0s APIs.

The 'alter' op moved so that it could be executed after the rendering rather than before. It was a do-nothing operation.

We could, in theory at least, avoid any changes by adding a FOURTH operation. But this is a very, very edge case. It's an important edge case, but one that the majority of modules (including all the modules that have been converted so far) don't care about. This is an API simplification.

If you think that there's no other alternative, I can look into changing it, but I feel strongly that this is a necessary simplification.

eaton’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
3.99 KB

I've attached a new version that leavse #after_rendre intact. If anyone does end up using it, it's still there.

I'm tentatively marking this RTBC again, as there was 1 request (leave #after_render) and 1 question (is moving 'alter' necessary).

aries’s picture

Thanks for this patch, Jeff!

+1 to be comitted.

chx’s picture

Aries can't write a proper support message :/ He contacted me over paging module which I have rewritten during the 4.7 cycle for Merlin and I seriously think that without this patch paging will be (almost) impossible during this cycle.

eaton’s picture

Oi, you're correct -- I'd forgotten about paging.module. Thanks for the reminder, chx. That one is very important.

chx’s picture

FileSize
4.15 KB

Rerolled with #after_render removed.

Dries’s picture

Status: Reviewed & tested by the community » Fixed

Committed to CVS HEAD. Thanks.

Anonymous’s picture

Status: Fixed » Closed (fixed)