By JohnForsythe on

I just finished adding a new feature to DrupalModules.com. It's an insanely fast module search tool called Module Finder. It searches for results as you're typing, executing queries in as little as 12ms, and updating the results live! If a module has already been rated, it even includes a bar graph right beside the listing.

My goal was to make finding Drupal Modules faster than ever before, and I think I've succeeded. I built the search engine from the ground up. What you see here is the 3rd generation of this effort (it's roughly 15x faster than the last iteration). The front end is powered by jQuery. I used the jQuery update module to get the latest version running on Drupal 5. Other than that, there are no contributed modules involved, it's all custom code.

Try it out, I hope you like it :)

I've tried my best to keep it cross browser compatible, and it's been tested in FF2, IE6, IE7, and Opera successfully. Let me know how it works for you.

--
John Forsythe
Need reliable Drupal hosting?

⋅ Categories: Drupal 4.7.x, Drupal 5.x, Drupal 6.x

Comments

ultimateboy’s picture

ultimateboy commented

Absolutely stunning.

Great work, as usual, John.

Thanks for the great service you have given to the Drupal community.

--matt
http://www.monarch-digital.com

-- matt tucker

modul’s picture

modul commented

Is the "Module finder" technique also available for download? Maybe I overlooked, but I couldn't find a download link. Or is it "just" custom code for your own site?

JohnForsythe’s picture

JohnForsythe commented

It's all custom code, designed specifically for my data set, but tailoring it to other sites may not be that difficult. I am definitely thinking about creating a public release, if enough people are interested. We'll see what kind of response this thread gets.

--
John Forsythe

ebrittwebb’s picture

ebrittwebb commented

Yes, please do make it available. I can see how this would be fabulous for all kinds of node searches!!

kyle_mathews’s picture

kyle_mathews commented

Yes, and as an aspiring javascript expert, I'd love to see what tricks you're using.
Kyle Mathews

AxelBernhardt’s picture

AxelBernhardt commented

I am also very interested in this module. There is also a project on my list to use it there. So this nice thing might grow a bit further ...

Axel

iamwhoiam’s picture

iamwhoiam commented

Very good job, using it extensively now! Much quicker than drupal.org's search method

It'd be amazing to get a decent Ajax search engine for drupal.

BostonDave’s picture

BostonDave commented

Just found this thread and love your site - also adding a YES in the "would be interested in the public code" if it helps.

spjsche’s picture

spjsche commented

What a wonderful service, and making the search facility available would be excellent.

Thanks for your time.

Stephen

Walt Esquivel’s picture

Walt Esquivel commented

Great job John! Your "Module Finder" functionality will definitely help out a lot of people.

Thank you so much for your dedication and hard work!

Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing

Fayna’s picture

Fayna commented

I would love to see this available to the public also!

I'm sure once others see it on your site they will ask you to make it public anyway :) I think it would greatly benefit the Drupal community.

I'd also think it would be awesome if you committed your custom code of NodeReview to the module here on Drupal.org.

If you decide to release some of your custom code to the public, why not slap some Google Adsense ads on your website?

JohnForsythe’s picture

JohnForsythe commented

I plan to keep DrupalModules.com advertising free. The server is expensive, though, and donations are welcome.

It sounds like people really want the search functionality, so I will make an attempt to create a public module. I also have an initial patch for NodeReview coming, maybe later tonight.

--
John Forsythe

Fayna’s picture

Fayna commented

That's totally understandable. :) Sounds great John!

JohnForsythe’s picture

JohnForsythe commented

I just made some performance enhancements to the Javascript, and replaced the opacity fadeout with a simple spinner/throbber instead. You should find it scary-fast now ;)

You might need to refresh the page to get the latest code.

--
John Forsythe

jason.fisher’s picture

jason.fisher commented

Absolutely spectacular. You've made my day.

sean_s_oreilly’s picture

sean_s_oreilly commented

This was something that particularly new users need, to be able to rate modules so that new users can see which ones to install and play with first.

esllou’s picture

esllou commented

would you be interested in producing a module based on this functionality? I for one would be willing to make a donation towards making this possible.

It would be very useful in a couple of places:

1. /blog or /video listings
2. integrated with views with teaser/full node listings

JohnForsythe’s picture

JohnForsythe commented

If you mean the search functionality, I am working on getting it into contributable form. Donations are gladly accepted, and will definitely help. I could also set up a chip-in widget if someone wants to suggest a reasonable target.

--
John Forsythe

esllou’s picture

esllou commented

what sort of time scale are we looking at? By the end of the month? I think the Chip-in is a good idea. I have no idea what amount is usual in these situations but perhaps somewhere in the $150-200 region???

It would need to be a module that was applicable in a few quite generic environments, such as:

1. search box widget installed at top of blog, video, audio index pages (/blog, /blog/1, video/2)

2. at the top of views (with full nodes or teasers showing)

zilla’s picture

zilla commented

would love to see this for d6!

joshua_cohen’s picture

joshua_cohen commented

I can't wait to see this module.

zilla’s picture

zilla commented

okay, drupalmodules is exactly what i've been waiting for and i'm thrilled to see somebody do it...

now what would be way cool: a mod developer looking for sponsorship or funding to develop (or port to d6 for example) could put up a chip-in request and drupalmodules could track all activity - most funded, most wanted (e.g. reverse bounty: people chipin to get a dev to port a module)

does that seem feasible?

joep.hendrix’s picture

joep.hendrix commented

piece of functionality! Well done!

Furthermore, I really like the concept of your site. Maybe this could replace the so very much wanted drupal.org modules rating system (mentioned here and here.

Keep up the good work and yes, please share your module on drupal.org.

-----------------------------------------
Joep
CompuBase, Drupal websites and design

-----------------------------------------
Joep
CompuBase, Dutch Drupal full service agency

mokargas’s picture

mokargas commented

Is this available yet? I'm even prepared to pay for it :)

Ric’s picture

Ric commented

Hi,

excellent functionality. any news on plans to share the search module?

thanks and congratulations

B747’s picture

B747 commented

I plan to keep DrupalModules.com advertising free

And there was me thinking I saw an Acquia ad on every single page on the site. Must just be my imagination...

Discover new modules, effectively search help and support with Heebie Drupal UberSearch. A must for any Drupal user - we promise you'll love it!

JohnForsythe’s picture

JohnForsythe commented

Sometimes plans change. Acquia's sponsorship means that I can keep the site running without having to worry about begging for donations anymore. It also helps promote the company Dries co-founded; the same company who was the primary sponsor of the last two Drupalcons. I'm happy with the decision, and I'm sure my users will be, too.

--
John Forsythe

Jonah Ellison’s picture

Jonah Ellison commented

I'm impressed by the speed and would love to see the code behind this.

DrupalDummy-1’s picture

DrupalDummy-1 commented

I am impressed. Please make it available to us. Thank you.

Please release Drupal 6 version also.

modul’s picture

modul commented

Not a bad word about the Module Finder site. It's a great tool and a hugely welcome addition to drupal.org. But there's one thing worrying me a bit, i.e. to what extent the speed would be transferable to other sites. I mean: is it so speedy because of its code, or is it so speedy because of the heavy artillery behind it in the form of server hardware? It would be nice to test it in a "poor" server environment. Any information on this?

JohnForsythe’s picture

JohnForsythe commented

The server is not all that beefy. Two single-core 2.66ghz xeons, 1 gig of ram. The main thing is knowing how to optimize your server. I run XCache (makes PHP super fast by caching opcodes), and I've done a lot of tweaking to the MySQL configuration (cache size, etc) to get maximum performance. The code itself is also extremely optimized for the task. The main thing holding back performance at this point is the physical distance between between the user and the server (ie: ping time/lag), and the speed of the user's connection (dial-up can get swamped with too many requests). I pay extra for a "performance network" connection to help minimize lag.

So what does all that mean for someone who would want to run this on a low cost shared host? I don't know. It really depends on how your ISP has set up their system. If they're running separate MySQL servers over a laggy internal network (*cough dreamhost*), you're probably going to get terrible performance. If your server has MySQL running locally, and caching is enabled, you're probably good to go. Even better if the server has opcode caching enabled, like XCache or APC.

The level/speed of interactiveness can also be tweaked. Currently, it only waits for a 100ms pause in typing before sending a new search request. This is why you get results almost immediately after you type something. But it also means that someone typing into the form can generate up to 10 requests per second (some ISPs would really hate you for this). If you bumped the delay-before-making-a-new-search-request up to 500ms, you'd eliminate about 90% of the server thrash, with only a small sacrifice in interactiveness.

--
John Forsythe

modul’s picture

modul commented

Thanks, John. That was a really instructive and to-the-point reply!

JohnForsythe’s picture

JohnForsythe commented

For those of you wondering about getting this turned into a public module..

After doing some work adapting this code to another site, I came to a realization: While it's not difficult for me to go in and hand-tweak the code for a specific site, getting it to the point where someone without extensive Drupal, PHP, jQuery, and MySQL knowledge could get any use out of it would take a lot of work.

Here's what I think would be a minimum for a public release:

- Point and click adding of search fields, with CCK support
- Ability to search the result set of a View
- Easily tweakable parameters (like search delay speed) via an admin interface
- Drupal 6 support
- Adding all this without sacrificing performance

This stuff isn't necessarily hard, it's just a ton of work. It's probably a month long project. There's three things keeping this from happening:

- CCK for Drupal 6 needs to be finalized before I can start coding around it.
- Same goes for Views2.
- I don't have a month I can give up right now

This doesn't mean it won't happen, it's just not going to happen right now.

--
John Forsythe

baobab’s picture

baobab commented

Hello, is there any option to see a javascript of the live search?

zilla’s picture

zilla commented

it's really an incredible searhc tool and i could just imagine how this might be used on a content rich site to find content by typing in assorted elements (title, other variables, mixing it up)- with preview and pass throughs to full nodes etc..

but i guess it might make more sense to put up some kind of project page if you're serious abotu this, at least to ge a discussion going and form a test group, get contributions and so on..

........................................................................
i love to waste time: http://twitter.com/passingnotes

baobab’s picture

baobab commented

Withdrawing my question about javascript.

I'm wondering how its possible to trim Views if requested by XMLHTTPRequest to deliver only its own results without all blocks around. The problem is that page mirrors itself completely into the <div> where only search results were supposed to be loaded.

It was discussed in here: http://drupal.org/node/115949,
the conclusion was that its possible with Views 2, but for Views 1 there is no patched solution yet.

Possible other solutions are close, but are not applicable for exposed search (on my opinion):
http://drupal.org/node/210926 - ajaxViews provide only block views,

http://drupal.org/project/views_datasource - allows to choose views datatype, however I didn't get yet how to make this to be part of an initial view.

there are
http://drupal.org/project/dynamic_views - but its not actually what I'm searching for, (and it doesn't work for me either when placed exposed filters into the block, as advised here http://drupal.org/node/213069 for drupal5)

Anybody any thoughts regarding solving it? If I find a solution will write here.

paganwinter’s picture

paganwinter commented

Subscribing...
Any more word on this...?

bokswagen’s picture

bokswagen commented

nicely done!

This is the kind of search I've been looking for and hopefully you can help me. I want to implement this on my site.

I am very interested in the search function of your site. Hope the module pushes thru!

paganwinter’s picture

paganwinter commented

bump...
if anyones still listening...