Change Image Attach file selection into autocomplete

At around 17,000 image nodes, the loading of an edit form with 'image attach' enabled exceeds 300 seconds on a dedicated, pretty powerful server with multiple cores and 24 GB of RAM.

That's more than five minutes where the complete (!) browser does not respond to any user interaction. Website users don't understand why their browsers hang or even crash when editing pages. Nobody ways 300+ seconds just to edit a node. Together with crashing some (probably older) browsers, and losing all entered data, this imho qualifies as a critical bug as defined in Priority levels of issues: "Critical bugs either render a system unusable (not being able to create content or upgrade between versions, blocks not displaying, and the like) [...]".

Apparently we have another Catch 22; developers don't use 'image' module anymore, so there won't be further development; on the other hand, we can't migrate either, since there is no migration path for 'Image Attachment' and no solution for re-using image nodes; since developers don't use 'image' module anymore...

It'd be interesting what those currently listed 71,974 users of 'Image' module will do when D8 is being released, and what they think about marketing blurb like "Why Choose Drupal? Use Drupal to build everything from personal blogs to enterprise applications." Yeah right, it just says you can "build everything", not "you can keep running anything" - beyond a fancy fire-and-forget demo site.

This is very, very sad for those of us that were wasting thousands of hours building sites with Drupal; but thank you (really) for the honest answer; this kind of honesty would suite those well who still advocate Drupal. At least I understand why we cancelled our membership with Drupal Assocication - a problem of 71,974 users that apparently nobody can solve on its own should be something to care about, instead of spending the member's money on generating more dishonest marketing blurb, to lead even more people into the trap of this dead end system (which I have been using for over seven years, with increasing problems to update with every major release).

Please don't take this personal; I am fully aware that everybody has to eat and pay their rent. It's just frustration that even "popular" FLOSS is not a bit more future-proof than proprietary software (Borland Paradox - abandoned; Microsoft Frontpage - abandoned...) and there's nothing I can do about it ;-(

Should we give this issue a "won't fix"?

Open Source is only future-proof for you and your use case, as long as you are able (technically, financially, etc.) to completely handle all relevant code, sure. However, everybody knows that even a skilled programmer can not solve all problems on it's own, and this is the point where Open Source works - or fails. If the community process fails - in case of the 'Image' module it clearly does - some other solution has to be found.

One possible solution would be to hope for a bigshot company with the ability to sponsor development work; sadly, there are not enough companies like this, or those are already drained off with other sponsored work (there are numerous other de-facto abandoned modules in contrib). And that's where the community has to jump in, and imho it should be the Drupal Association's task to identify and to take care of problems like this. Of course they can not support every abandoned module, but 'Image' is not just "some" module; it was for years the de-facto standard for image handling in Drupal, and neither Imagefield nor D7's Field API 'image' is currently capable to fully replace it (e.g. re-use of image nodes).

In other popular community-driven cases - e.g. MediaWiki - the Wikimedia Foundation has made this step years ago; MediaWiki is still clean GPL software and has sporadic code contributions from the community members, but the essential pieces are maintained by employed developers; it does not fully depend on people free to go away at any time. If I donate to the Foundation, I know this and I expect this. But yes, obviously the Drupal Association is not the Wikimedia Foundation, and Drupal is not MediaWiki. The last time I checked, commons.wikimedia.org was able to maintain and deliver 13,162,720 media files. It's really, really creepy if Drupal starts to die with 10k image nodes, or with some 40k taxonomy terms.

Again, these scalability issues happen all around in Drupal, and obviously neither single developers, nor companies can resolve them. That's why - imho - either something like the DA has to jump in, or Drupal has to significantly revoke what it promises (stuff like "Standards Compliant - Accessible - Modular - Stable - Free" and the like).

According to the annual report of the DA for 2001, they paid $411,377 for "staff overhead", $84,008 for "website development", and $23,567 for "graphics", so some people appear to get paid for their work already. I don't know why they can buy "graphics" for $23,567 but not sponsor to fix complex issues in core or broken essentials from contrib. At least that would be a reason to renew our (organizational) membership.

Anyway, this is off-topic, and I can not speak for the 71,974 users of 'Image' module.

I've already suggested the Chipin approach about 14 months ago, but there was not much response; joachim was "sceptical", also Chipin is a pure Flash application and money runs through Paypal. Merlin has so far raised $10,669 from 216 contributors through Chipin to "Get Views in Core" (468,406 users), but he estimates to "likely require more than a hundred thousand dollars".

Another approach would be Flattr; for example, the Java application 'jdownloader' has received "2,323 Flattrs from 1,095 persons" (not possible to tell what this is in cash); Flattrs are accepted by W3C, Wikipedia, Amnesty, WordPress, and others; according to some 2nd hand reports, it is possible to earn €500 ($650) per month; depending on where and how you live, this might be a good monthly income, or a couple of drinks in a strip joint.

However, I can neither put a Chipin nor a Flattr widget an anyone else's project page, and I'm even refused my own full project status (something we unsuccessfully tried to start sponsoring over a year ago) ;-/

@joachim: Collecting money means that someone promises to be able to solve a problem if he gets paid; it requires some kind of account (bank/Paypal), and this account should be owned by the same person that made the promise, not by someone else, and dedfinitely not by someone who can not solve the problem. Trust is essential when trying to raise money directly, and it needs to be clearly communicated what happens with the raised money if the goal is not reached.

If there is a middle-man (curator/Treuhänder), it needs to be someone well-known and highly trustworthy with a good reputation in the community (like Dries), or an organizational body (non-profit/gemeinnützig because of things like tax deductibility) that is trusted to spend the colleced money in the interest of the donators; that's why I think an existing organization - the DA - should take core of this.

@Damien (#5): I'm no programmer, so I don't know how big the effort actually is to exchange UI widgets from "select list" to "autocomplete", so the following is pure amateurish guessing: In the code of nodereference_url.module is a fallback to autocomplete defined like this:

    // Fallback to autocomplete.
    elseif ($fallback == 'autocomplete') {
      $element = array(
        '#type' => 'nodereference_autocomplete',
        '#default_value' => isset($items[$delta]) ? $items[$delta] : NULL,
        '#value_callback' => 'nodereference_autocomplete_value',
        '#element_validate' => array('nodereference_url_autocomplete_validate'),
      );

In ./image/contrib/image_attach/image_attach.module I find this:

        // Only show selection box of image nodes if the user may attach some,
        // or if there are existings ones that may be removed.
        if ($may_attach_existing || $has_existing_images) {
          $form['image_attach']['iids'] = array(
            '#type' => 'select',
            $value => empty($node->iids) ? NULL : $node->iids,
            '#multiple' => TRUE,
            '#size' => 6,
            // Title, options and description are set just below.
          );

If these are even the right pieces of code, they look pretty different to me (e.g. one is $element, the other is $form[]), and it does not suffice to replace '#type' => 'select' with '#type' => 'nodereference_autocomplete' since D6 does not not know such on form element (or because of something else). When searching in "Pro Drupal Development" (VanDyke, 2008), there is not even "nodereference" in the index; so some kind of "nodereference_autocomplete" would introduce a new dependency on CCK, or would require to code a "nodereference_autocomplete" form element from scratch - and that's nothing a non-developer can fix, right?

As Damien noted in #5, this hopefully is an "easy fix", so I'm awarding €100 for a patch to change Image Attach file selection into autocomplete.

Two simple rules: (a) The patch needs to be RTBC, and (b) it needs to work for me (meaning: it needs to fix the performance issues when Image Attachments are enabled ;). This bounty is open as long as I am using D6 and 'Image' module, and ideally it could outline how 'Image Attach' would work on D7.

Thanks, Joachim, for your thoughful explanation. I was afraid of such a reply.

> you'd be better off exploring upgrading your sites to Drupal 7

Yes, I spent quite some time with this, but upgrading is simply not possible at the moment, and I have growing doubts that it ever will be. Even if the 'Image' module & 'Image Attach' problem would be solved, we'd still have quite a number of unreplacable and essential modules on our sites, starting with the input format when creating content and ending with access control when reading content. And even if we'd consider dropping basically all image content, breaking all text formatting on every node, and killing all access control to private content, even then the current migration paths for CCK are failing because of numerous long-standing issues. The problem behind this is that companies are currently running a lot faster away from Drupal than new developers grow back, as we know since last Drupalcon. As you said, it it's way more hours of work to port all the missing pieces than we or any other individual or small company can afford. Until this is resolved one one way or another, it's either Drupal 6 or no Drupal at all.

Should we "won't fix" this?

> If we replace b. with an autocomplete, we'd get a single-valued autocomplete. You'd only be able to
> attach a single image at a time. To attach more, you would need to save the form, and edit the node again.

Hm, regarding the usability that doesn't sound too bad; it's exactly the behaviour I'm used to with multi-value image attachments (even if this might be designed to behave differently). Getting a single-value autocomplete field for multi-value CCK fields is something I'm used to from CCK + Nodereference URL when I'm using the autocomplete widget as fallback behaviour (necessary for being able to edit the nodereference value, but makes it impossible to add a 2nd value to the CCK field).

> AFAIK not very much as changed with input formats from 6-7 apart from the hook being split up, so that
> should be a fairly simple upgrade.

Depending what you consider "a fairly simple upgrade", it's "an easy fix" like changing Image Attach file selection into autocomplete; last year someone tried a port and failed (btw, mediawiki_filter is an offspring from Origo which shut down a couple of weeks ago; but this project, the [failed] full project status application on d.o, and the [failed] sponsoring attempts from past year are off-topic for our 'Image Attach' issue).