Hi everyone,

I have started to port the module to Drupal 6.x. I'll be posting the progress of this task here. :)

It is almost done, and I'll probably commit here and release a BETA pretty soon. This is the first thing I'm porting from D5 to D6, so I might forgot something, so I would appreciate any feedback that anyone could provide.

It has not been as complex as I initially thought. The hardest part has been sphinxsearch_watchdog(), used in XMLPipe generator for main index processing. Said function is required as we could have been restarted DB connection, and that breaks implementation of dblog_warchdog() because it makes use of db_set_active() to unconditionally enable main DB connection to store the watchdog records, which is something not really necessary here.

Well, I'll bump this issue when this port has been commited to Drupal CVS.

Cheers

PS: The issue tracker is forcing me to select a valid project version. Since I haven't branched yet, there's no 6.x-1.x option to select. Ok, I'll change that as soon as the work is committed to proper branch.

Comments

markus_petrux’s picture

markus_petrux commented
Title: Porting sphinxsearch module to D6 (and Battle Plan) » Porting sphinxsearch module to D6

Ok, committed, branched for D6 and created a development snapshot (release 6.x-1.x-dev) hopefully available from project page.

I feel it's ready, but I don't have any site in production running D6. I would appreciate any feedback if someone else could test this package.

Once it can be confirmed current functionality works in D6, then I'll start adding a few more features.

Tasks that may run in parallell:
- #328271: Implementation of hook_sphinxsearch_api().
- #327816: Ability to execute XMLPipe generator from PHP CLI.
- #319528: Ability to see popular search terms.
- #328059: Cache sphinx search results (specially when faceted search support is enabled).
- #332077: Theme whole search page.

Where do we go?

- We need to index more node related attributes (language, workflow flags, ...), CCK fields integration, and also allow external modules add their own stuff.

- Once we are able to choose what we wish to index, the advanced search form and guided search blocks become too cluttered, with too many different things in just one UI, and that will confuse everyone, so we need a method to customize the advanced search form. Here, I'm thinking in that we could have admin defined "profiles". Each profile would be composed of name and description (for admin purposes, but also for the end user), and a list of enabled attributes, predefined default values, predefined access control filters, etc. Admins could assign access permissions to this profiles. And so, this information could be used to provide different paths (local tasks, ala core search with node, users, etc.). So we could have a search page for "stories", "content users", "projects", "products", "jobs", "issues", etc.

- Implement additional blocks that may offer "related content" capabilities based on the power of sphinx in resolving queries by terms fast, etc.

- Multisite search? Search services integration? Though, I'm not sure yet what needs to be done here, and how. We could provide search services, and someone else (that we need to autorize, or not) would be able to send us search requests, or something like that... I think there's something done in this direction, I need to explore this stuff.

markus_petrux’s picture

markus_petrux commented
Version: 5.x-1.x-dev » 6.x-1.x-dev

Just wanted to mention that I'm busy working on other projects, and that's a good thing because are focussed on D6 and CCK, which is something that I need to learn in order to think about methods to extend Sphinx index attributes. Another thing that's needed is a way to index/filter workflow related attributes, and content language. For all this, I would like to design a method that's as easy and powerfull as possible.

I hope to get something done in one or two weeks.

If no one requires a stable D6 release with current funtionallity, then I'll keep working on this and release when the method to extend attributes is implemented.

Because of lack of time, I would not initially add more features to the D5 branch. We all need to focus in D6 and on...

markus_petrux’s picture

markus_petrux commented

I have sometime this weekend, and I hope to get things fixed, at least to make sure the module works. Then, will come the time to start adding features, etc.

Initially, I do not plan to create a release until all features are in place and working, that will save me time that would have to go on providing upgrade paths, etc. If someone wishes a working stable version before all features are added, please ping. ;)

Here's a few bugs reported for current D6 branch that I marked as dups of this one. Thanks to all who reported them as this helps me focus on the things that don't work as they should.

markus_petrux’s picture

markus_petrux commented

Well, a new development snapshot will be generated with a few more fixes related to this port.

Since usage statistics have been enabled, I know now that there are one or two out there giving this module a try, so thanks.

I'm sorry for the delay in comming here, but daily job took preference and I had to be more focussed on something else. So, what one wishes to do, and what it is actually able to, cannot always match.

That being said, I think this starts to get a bit more usable now. Let's see if I can start adding new stuff here soon... the battle plan is more or less described in #1, which I'm going to update now.

markus_petrux’s picture

markus_petrux commented
Title: Porting sphinxsearch module to D6 » Porting sphinxsearch module to D6 (and Battle Plan)
roberto.ch’s picture

roberto.ch commented
Component: Code » Sphinx search indexer
Assigned: markus_petrux » Unassigned
Category: task » support

how you authorize the indexer for requesting pages from drupal6-site?

[ROOT@psrLAMP01:/opt/sphinx] bin/indexer --all
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file '/opt/sphinx/etc/sphinx.conf'...
indexing index 'index_platforms_main0'...
ERROR: index 'index_platforms_main0': xmlpipe: expected '<document>', got 'Not authorized.
'.
total 0 docs, 0 bytes
total 0.010 sec, 0.00 bytes/sec, 0.00 docs/sec
indexing index 'index_platforms_main1'...
ERROR: index 'index_platforms_main1': xmlpipe: expected '<document>', got 'Not authorized.
'.
total 0 docs, 0 bytes
total 0.010 sec, 0.00 bytes/sec, 0.00 docs/sec
indexing index 'index_platforms_delta'...
ERROR: index 'index_platforms_delta': xmlpipe: expected '<document>', got 'Not authorized.
'.
total 0 docs, 0 bytes
total 0.010 sec, 0.00 bytes/sec, 0.00 docs/sec
distributed index 'index_platforms_join' can not be directly indexed; skipping.

saluti
roberto

markus_petrux’s picture

markus_petrux commented
Component: Sphinx search indexer » Code
Assigned: Unassigned » markus_petrux
Category: support » task

Please, open a separate issue. Thanks.

markus_petrux’s picture

markus_petrux commented
Title: Porting sphinxsearch module to D6 » Porting sphinxsearch module to D6 (and Battle Plan)

Well, I'm very sorry for the huge delay in working on this project, but circumstancies have left very little time for this.

Now I'm working on a project where performance for searching is really required again, and I plan to check the issues queue to at least leave a stable release that can be used on production.

In regards to plans for the future... I guess things have changed a lot since then, and it seems to me the way to go is doing something around Search API, and also thinking in D7 and beyond. Anyone wishing to work on this direction, please contact me for dev access to this package. Thanks!

markus_petrux’s picture

markus_petrux commented
Status: Active » Fixed

Well, after spending some time on the queue, probably we can consider the module is happily ported to D6, finally.

We may use the queue for additional issues that may arise.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.