Greetings,

I found that CCK module is going to be used as a node building system instead of flexinode soon. But, as I know there is still no solution to perform a custom search over created content.

Flexinode had its own search, then it was moved into flexisearch.module ... but neither of all existing search solutions provide an ability to create a *custom* search form for each content type with a *custom* query.

For instance, I have a content type "book" with the following fields: author, publisher, year, weight. This content type is used for e-commerce package as a product. So, it also has a price.

The only thing I need is to create the following search: find all books with the specified author, publisher, with the year within specified range and the price within specified range ... and it is important that the search form must be well designed but not as it is made in flexisearch.module.
I'm sure many people came across with such problem too.

I may be missing something. If so, please tell me is there an ability to perform such search?

If no, how about to discuss here the way to implement this ? I have a couple of ideas as for custom flexinodes searching ... they may be applied to CCK as well.

Thank you.

Comments

edrupalec’s picture

Duke,
do you have this working already? is it possible to see it?

if it's working, I would stick with flexisearch for the time being.

Drupal ecommerce, at http://www.drupalecommerce.com
http://www.drupalecommerce.com/troubleshooting
http://www.drupalecommerce.com/modulesexplained
http://www.drupalecommerce.com/47vs46
http://www.drupalecommerce.com/howto

ardas’s picture

I have developed an advsearch.module for Drupal 4.6.
It is still in development and has several bugs, but the main thing is an idea.

I didn't migrate to 4.7 yet and I would like to conjointly rework it if necessary and then release for community.
The main idea is the following:

advsearch gives a searchapi hook with two operations: 'form': any module can handle it to return a specific form; 'query': the module must return a query according to the created form.
you can set up which node types will have a search tab on the advanced search page, their titles and the order of this tabs. Each tab will display its form (searchapi('form')) and runs its query ((searchapi('query')))

The only thing you need to do in your module is to handle hook_searchapi, design form and create a query. It is not an 'interactive search builder', it is only for developers, but it gives an ability to create custom form and query.

I think that this is a good idea for CKK and Flexinode modules to add hook_searchapi implementation.

I can send you this module for review on email if you are interested.

OpenChimp’s picture

This is a great topic, and don't know why there isn't more discussion about it. I have thought about this before, but haven't had the time or need to implement it.

CCK is awesome, and I'm so glad it's being brought into core. It allows you to create complex data types, but these data types do need a search function, and CCK already stores all the data needed to generate a form for searching on various fields, and Forms API provides a means of generating forms.

What would be needed is:

  • CCK_SearchForms - this would serve to communicate the structure of a CCK object to the forms API
    • Multiple Interpretations - sometime a search form needs to provide and ANY interface and sometime it provides an ALL interface. This option would be required to produce the desired forms.
    • Some complex CCK objects are multi leveled, which may require multi-page/step forms, which can be submitted multiple times to create sub components of the object
    • Form Management - restrict the view of certain fields on the form. Customize field styles
      • Multiple Selection Fields - Select whether the form element should display as a series of checkboxes or multi-select box.
      • Select from a List Fields - Select whether the form element should display as a series of radioboxes or a dropdown
  • advsearch - provides the hooks into the searchAPI and placement of the search page with tabs feature.

I'm fairly new to drupal. If anyone takes interest in this project, and think it should be posted elsewhere, please do, and let me know if I can help.

ardas’s picture

Interesting idea. Lots of time passed from the day I created this advsearch module for Drupal 4.6 and now I understand that it is depricated. I found a way how to customize search for different content types. I'm using Flexinode in this example but it doesn't matter. Here is a description of my approach:

Assume, I've created a new content type "melody" and my new search module name for melodies is "band_melody".
1. Create function band_melody_form_alter($form_id, &$form). For $form_id == 'search_form' && arg(1) == 'band_melody' alter the search form as you want it to be.
2. Create band_melody_search_validate() to validate your new fields.
3. Customize output using theme_band_melody_search_form().
4. Implement hook_search() to create a search logic.
5. Create band_melody_search_page() to customize search results page.

Using this approach, you will need one module to customize search of one content type.

----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

ardas’s picture

So, CCK_SearchForms will be a separate module which will be able to generate forms for CCK content types. May be they can be even used to customize node page, what do you think ?
One important thing - this new module should give an ability to customize forms layout via theme_ function because it is usually required to customize search forms to make them beautifull.

Not sure now about advsearch module will be required at all.
----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

jaxxed’s picture

it would be nice if some more arg/url functionality was used to allow links to quicksearch results without the need for the filter form part

i.e. ... book/author/dickens

This would help out the profile_as_node group

fluitfries’s picture

I just wanted to give my feedback on the actual solution that is being worked on for the entire community.

1) It is starting to sound like the CCK_SearchForms is duplicating features that are possible via Views. For example, the idea of customizing the node page, show/hide certain fields, and providing themeing. I feel like all of this is already possible through Views, right?

2) We could use a simple 1-10 selection menu next to each CCK field of a CCK node that tells CCK_SearchForms module how important data from that field is to the search. If a search finds more than 1 hit in a CCK node, the CCK fields marked with higher priority are listed higher in the search results. Make sense?

These are just quick thoughts- sorry if I misunderstood anything that was said.

jaxxed’s picture

I would assume that cck_search(forms) would likely not conflict with views as it would likely use views to implement any search features. It is more likely that new functionality would be written to provide and interface to using views filtering in cck.

Naturally cck already sets up the definitions required for views implementations.

fluitfries’s picture

Hi there,

I'd just like to know how I can get my CCK types included into drupal's search module. My search works fine for nodes like "story", but the bulk of my content are CCK types, and these don't seem to be searched at all!

What do I need to do?

Thanks!

ardas’s picture

Hi,

This is what I'm trying to figure out now as I also need to combine CCK and Core Search.

First of all, check what exactly data are stored in 'node_revisions' table in 'body' column for your CCK nodes. Core search module searches content based on this column. I don't know yet if CCK nodes produce any body automatically, but I hope they should.

----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

fluitfries’s picture

My Nodes are more like Records than Pages, so there is nothing listed in that field of the node_revisions table. :(

I have hundreds of nodes that I imported and the only data in these nodes lies in custom fields that were added to that node type. I use the views module to list and view those nodes, sort of like you would with Excel and a spreadsheet.

I have need to search the various fields on these nodes in order to pull up only certain records.

Hopefully that clarifies things!

ardas’s picture

My solution is the following:

axiom 1 - Body field is necessary for search module to perform text search.
axiom 2 - CCK nodes may have various data fields (search module doesn't know anything about them).
axiom 3 - CCK module doesn't produce any bodies for his nodes automatically (as Flexinode did for instance).

So, we should produce bodies for these nodes ourselves. More then, we can decide what exactly we need for each type of node to make it searchable. Node bodies will be used only for search and should no influence on output because they are overwritten in content_view() hook:

  $node->body = implode('', _content_field_invoke('view', $node, FALSE, $page));
  $node->teaser = implode('', _content_field_invoke('view', $node, TRUE, $page));
  $node->readmore = (strlen($node->teaser) < strlen($node->body));
}

We can implement hook_nodeapi() 'insert' and 'update' operations in a module which is specific to your project to produce certain bodies depending on node types.
It will be something like this:

function mymodule_nodeapi($node, $op, $arg = 0) {
  switch ($op) {
    case 'insert':
    case 'update':
      if ( .... check type of the $node....) {
        mymodule_write_body($node); // will update node body with a necessary values.
      }
      break;
  }
}

NOTE: you can just update body, you don't need to worry about creation because hook_nodeapi() fires after the node has been already saved into a table.

The advantage of such approach is that we can produce only necessary text for body field which may consist of only needed CCK field values. Flexinode generated bodies automatically, so they were filled up with unnecessary values.

What do you think about all these ?
----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

fluitfries’s picture

This definitely sounds like a good solution. I admit that this will be my first undertaking of creating a custom module but I will give it a go. I understand what your are proposing: a module that identifies the type of CCK node, selects the designated CCK fields, and copies those values into the body of the node. The search module then picks up this newly populated body for those nodes, which causes the nodes to be included in the search results.

Not knowing much about how to code for drupal, I have a few questions that may or not answer themselves as I get going on this;
1) How will the body of the node get updated to reflect changes to the CCK field? Will I have to manually "run" the module every time I update the CCK node?
2) Is this a better solution than making a duplicate of the search.module and just telling it to look in additional database fields for it's search data?

Thanks very much for your help, I hope your solution works for me and others!

scb’s picture

1) No, you won't have to call that manually each time. Just implement the function mymodule_write_body($node); like Duke said. The hook nodeapi will be called automatically each time the target node is modified.

2)Yeah, it might be better to biuld another custom module, but that solution would be less maintenable... On the other hand, this body thing just looks like a hack... don't know.

ardas’s picture

1) I even thought that we need to build some good and customizable module to populate body fields automatically. I think that this module should provide a good settings page listing all node types with their fields. A user can choose which fields he would like write into body field... or something like that. May be there is something like this already developed :) Have you ever try to search for it ?

2) This is a good question. Sometimes, we need a very specific search algorythm (for instance: between dates, or greater than any field value, etc.) In this case we need to create our own module which will use searchapi to provide such search (kind of very advanced search page :) BUT ! It will work for this type of content only ! It will not give you an ability to perform side wide search.

So, my solution is usefull if you want to search through ALL content just using text search technics. Creating your specific module is good only to implement not trivial logic. I have such module and if you want I can send it to you.

----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

scb’s picture

I have just tested this, and it won't work. The search module does the search based on the search index. When the site is indexed for searching, the node_update_index hook calls

node_invoke_nodeapi($node, 'view', false, false)

which rewrites the body as described by Duke above. So, writing a new body content on node creation doesn´t work

The way to go here is implement the hook update_index. I have tested it and it works:

/**
 * Implementation of hook_nodeapi()
 *
 */
function yourmodule_nodeapi(&$node, $op, $teaser, $page) {
	switch ($op) {

		//Update search index with CCK fields' contents
		case 'update index':
			//Check it is the CCK node type you want
			// You may use print_r($node) or the contemplate module 
			// to view variable fields' names and content types' names
			if($node->type=='yourCCKtype') { 

				//Select field data to be indexed.
				//These are just example fields on one of my CCK types
				$data = '';
				$data.= $node->field_description[0]['value'];
				$data.= $node->field_technical_information[0]['value'];
				$data.= $node->field_client['value']['title'];
				
				//drupal_set_message("Index updated for $node->title: $data");
				
				return $data;
			}
			break;
	}
}


scb’s picture

For simple searches, the CCK module should include a checkbox for each field that declares that field as 'searchable'.
Then include this 'update index' hook call and index all fields marked as searchable.

For more complex searches, the easiest solution is to build custom modules... but it would be nice to have a bunch of 'complex search functions' that could be reused (like searching between dates, etc.) I don't know if those should be part of the search module or not.

I don't know how hook_search() works, but I think it could be used for tweaking search results?

ardas’s picture

After you write a code to produce bodies you should update all your existing nodes and run cron to index them (I forgot to tell about it before).
This is how search.module works - you need either to wait till next cron run or run it manually. After that all your content will be indexed. So, it is not enough to just create a module - you need to invoke 'update' operation of nodeapi hook for bodies to be generated and then reindex them... Sorry about that.

I should read more about 'update index' operation ... it looks like a much better solution because you don't need to update everything and bother to run cron.

+100000 !

----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

ardas’s picture

I'm afraid that populating body fields is also important. This is because body is used when displaying search results. So, even if you provide search index you may see just titles without any descriptions which is bad. Of course, you can override search results output but default one may be not good enough... What do you think about it?

----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

scb’s picture

You can't skip indexing your site. In fact update index is called whenever search module is indexing the site.
It just tells search module how to build the index for that node, in case you don't like the stadard way, which is indexing all that is "viewable" (i.e. output by the view hook). I don't know how does CCK handle the indexing of its nodes, though. I'll look into it.

Regarding display of bodies, as you noted before, content_view hook already does this:

  $node->body = implode('', _content_field_invoke('view', $node, FALSE, $page));
  $node->teaser = implode('', _content_field_invoke('view', $node, TRUE, $page));
  $node->readmore = (strlen($node->teaser) < strlen($node->body));

which is called in the standard node_search hook, during search results display:

        // Allow modules to change $node->body before viewing.
        node_invoke_nodeapi($node, 'view', false, false);

So, search won't display empty bodies anyway.

ardas’s picture

Agree ! We should not skip indexing.

I think that content_view() hook only displays bodies but not store them into the database.

You are right about node_invoke_nodeapi($node, 'view', false, false) but looks like it is called after do_search() function, so it is only needed to override node's output, but if this node has no bodies in the database which may be indexed by the default indexing or provides their own indexing (like you proposed) it won't be found by do_search() at all. So, node_invoke_nodeapi($node, 'view', false, false) will not help because it is called only fo found nodes... This is how I understood node_search() function logic.

Just have a look into your node_revision table in the database. Are there any bodies for CCK nodes? I've never tried before - I will start using CCK only in the next project :)
----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

scb’s picture

Yes, but remember, do_search() performs the search based on the indexes, not the nodes' content!
It only queries tables search_index, search_total, and search_dataset. CCK nodes' bodies are not present in the database, you're right, but they are present in the indexes tables, because they are put there by the update index hook (well, not exactly the bodies, but the fields that were chosen to be put there in the update index hook).
I'm checking it now, but I think CCK nodes' content are indexed anyway, even if you do not implement update index...
I also think the field node->body is never used by the search engine.

EDIT: I have just checked it out. Node module's node_update_index() hook calls this for every node before indexing:
1) hook_view() ->this includes automatically all viewable content in the index, by filling body field 'on the fly'
2) nodeapi($node, 'view', ...) -> lets other modules add/modify the body field filled in (1)
3) nodeapi($node, 'update index', ...) -> lets other modules add extra custom fields (not in view)
4) add node title (between h1 tags, which give it more weight in search) to body and perform indexing.

Database body and teaser fields are never set for CCK nodes, but search should work anyway out of the box (at least, for all viewable fields). Problem is, if you want to customize which fields are searchable you have to rewrite the body, because altering update index would only add more fields, but not modify existent ones (for example, one might want to remove field labels from search, which are part of the output of cck view hook). But by setting the body field in database (during node update) won't work, because node->body will be reset in CCK's view hook (step 1 above), during search indexing.

So, if you want to add custom fields to the search, update index is the cleaner option, but if you need the standard searchable fields to be modified, rewrite the body through nodeapi view hook (but this will modify the actual output of node's view! and this is a problem...).

ardas’s picture

Correctly.
The only question now - should we create cck_search.module or it is better to propose this feature for content.module itself ?
----------------
Regards,
Dmitry Kresin, ARDAS group (www.ardas.dp.ua)

marcoBauli’s picture

Howdy,

i read in this issue here about this solution that could help with CCK teasers and with the possibility to use CCK nodes' teasers as meta descriptions for the Nodewords.module (thanks sergio for landing this post there ;), but i cannot quite understand how to "use" the code proposed here nor if it is a working solution or just an early stage proposal..

Also, i would be interested too in making indexed not the whole body of my nodes, but only a couple of main fields. You guys think this is possible?

Any hint or help greatly apreciated please!

scb’s picture

You have to create a module which implements the hook nodeapi function.
For the cases 'insert' and 'update', you have to write to the node's body and teaser fields in the database the data you want to be there. Those fields are in the node_revisions table.

Example (NOT tested):

function yourmodule_nodeapi($node, $op, $arg = 0) {
  switch ($op) {
    case 'insert':
    case 'update':
      if ($node->type=="yourcontenttype") {
        // update node body with necessary values.
        $body = "hello world"; //Build your body contents based on node's cck fields
        $teaser = "hello world"; //Build your teaser contents based on node's cck fields
        db_query("UPDATE {node_revisions} SET body = '%s', teaser = '%s' WHERE vid=%d",$body,$teaser,$node->vid);
      }
      break;
  }
}

Notes: I'm not sure about the vid field, but it is the table's primary key so it should be enough to reference the right node. In order to know the names for the cck fields to build the body and teaser with, you may use the contemplate module, or a print_r($node) somewhere.

About your second question (modifying index of cck nodes) we're still discussing that point. It seems that there's a pairing between a node's view and it's indexing. You may add new data to the index easily, but I think you can't modify data present in the node's view, which is included in the index by default, and cannot be removed. There might be a solution for this (without patching node.module), but I don't know...

scb’s picture

I have asked about this, and proposed a workaround here:
http://drupal.org/node/89428

Duke, it is node.module that needs to be modified, I'm afraid...

dewolfe001’s picture

I tripped over this problem after we began importing massive amounts of material into CCK nodes, then none of it showed up via search.
When search indexes content via its cron job call, it does:

  // Update word index
  foreach (module_list() as $module) {
    module_invoke($module, 'update_index');
  }

This means that ALL modules that have a [module]_update_index function can hook into this call and generate more indexing material. You could modify node.module to create a step-through of the CCK; or doing adding it to content.module or doing a cck_indexer.module could do it too where this is one of its few functions.

Other thoughts: apply a Google search to the visible end-product pages and side-step Drupal's search. Or, if I had access to the Google API (no longer available), I could do a search, get an XML record set, cook out node id#s then apply taxonomy filter vs. the set of nodes returned. Though I think that violates Google's TOS (moot point as you can't get new keys)

- Mike

chase_elliott’s picture

Is there a consensus here about the best way to include CCK info in Drupal core search?

I have a news site where I've created a CCK type (content_news_story) that is the main node type for our site, so, every time a new news story goes up it's a CCK node, not the standard story node.

Problem is the site search misses these entirely, which I know is the main issue here.

What is the simplest way to get Drupal to search/index my content_news_story body? Doing a Google search seems to pick up the pages fine, but core does not.

Thanks much to anyone who can help with this... It's killing me.

FreeFox’s picture

I'm searching for a solution too. Is there one already?

Thanks
Jan

ardas’s picture

I tried CCK 5.0 and it looked like the search problem was partially solved. Here is the summary:

1. node.module implements update_index hook and indexes every not indexed node.

2. $node = node_load($node->nid);

3. // Build the node body.
$node = node_build_content($node, FALSE, FALSE);
$node->body = drupal_render($node->content);

4. // Fetch extra data normally not visible
$extra = node_invoke_nodeapi($node, 'update index');
foreach ($extra as $t) {
$text .= $t;
}

So, search index will be populated for every CCK node and will consist of all data from $node->content field. As I know CCK node puts all its fields into $node->content. That is why all CCK nodes can be searched by any field value.

Here is a piece of comments from search.module:
"... However, note that the search system already indexes all visible output of a node, i.e. everything displayed normally by hook_view() and hook_nodeapi('view'). This is usually sufficient..."

I'm not sure, that CCK_Search.module is so necessary now in Drupal 5.0, unless it can provide an ability to fully customize search index for CCK node and even disable it for certain node types. If you implement update index hook for you specific CCK node type it will be only added to the index built by core node.module but not overrides it!

What do you think about it?
----------------
Regards,
Dmitry Kresin, ARDAS group (ARDAS group)

lias’s picture

It is another alternative to drupal search.

http://drupal.org/project/Modules/category/89

lsabug

danielb’s picture

So how come my drupal 5.4 doesn't search CCK fields? Were you wrong, or did I stuff it up?

danielb’s picture

I am looking in the search_index table, and I see no mention of anything other than Title, Body, taxonomy... useless :(

can't I just 'join' the content_type_xx table on somehow in teh search.module indexing function?

danielb’s picture

I needed to get the search module to search upon fields in my content type 'bio'

in search.module, (v 1.209.2.5 2007/07/26 19:16:48) about line 452 find this:

function search_index($sid, $type, $text) {
$minimum_word_size = variable_get('minimum_word_size', 3);

then add this:

$extra_fields;
$bio = db_fetch_array(db_query('SELECT * FROM {content_type_bio} WHERE nid = %d', $sid));
foreach($bio as $bio_field) {
$extra_fields .= " ".$bio_field;
}
$text .= $extra_fields;

Next time you edit those nodes, and run cron, the extra fields will be in the search index. Huzzah!

danielb’s picture

you could even slap some
or etc.. tags around any field values you consider to be extra important to give them more importance etc...

technivant’s picture

I tried several of the suggestions in this post, but ultimately the only thing that allowed me to index ALL of my CCK fields for a given content type was to disable the Contemplate that I was previously using for it. I can get away without it and it helps me get my CCK fields indexed in their entirety.

nimazuk’s picture

Thank you!
This worked for me on D 5.7!

N.Mehrabany
وب سایت عکاسی و طراحی وب
CSS Formatter & Optimizer

Nima

Amalanthe’s picture

For me, on a production site, when the cck field is a node reference, it can't we found in search results.
I try, try again to fix this problem, but I have for the moment no solution.
When cck field is not a node reference, there is no problem for finding it in search results.

ardas’s picture

This is how node.module performs search indexing:

   ..............
    $node = node_load($node->nid);

    // Build the node body.
    $node = node_build_content($node, FALSE, FALSE);
    $node->body = drupal_render($node->content);

    // Allow modules to modify the fully-built node.
    node_invoke_nodeapi($node, 'alter');

    $text = '<h1>'. check_plain($node->title) .'</h1>'. $node->body;
    ............

This means that after node is loaded, node_build_content() will content_view() to apply all CCK formatters to their fields. All rendered content will be put to $node->body and included into index. This should work....

I propose you to do some debugging here. Try to output $node->body into a log to find out is there a proper value for your node reference field?

----------------
Regards,
Dmitry Kresin, ARDAS group - Web site development, Drupal services, Software development, IT outsourcing.

paganwinter’s picture

For me, using CCK 3.x, this doesnt seem to be working. I have set the Search Display to Title(link) for my node reference fields, but they're still not being indexed.
But the hook_nodeapi snippet did the trick for me.

paganwinter’s picture

Subscribing...

Gogowitsch’s picture

Your requirements are very likely fulfilled with the Faceted search module (or shouldn't it be spelled Facetted?)