Boolean mode results may be less relevant than normal mode

What do you mean by user content types? User profiles? The Similar Entries node ID argument that is used for retrieving similar content requires a node ID. We are certainly interested in supporting all entity types and allowing the argument handler to be limited by entity type. Also, we plan on porting 6.x-1.x to 7.x-1.x as an alternative to a Views dependent Similar Entries. In fact, I'll be working on that in the coming days.

See #1439244: Port 6.x-1.x to 7.x-1.x for information on the port to D7. I ported the whole module to D7 today and will be committing the changes shortly and releasing a development snapshot. Please test and let me know how it works for you : )

Ahh... forgot to check that box. It's up now and I look forward to hearing feedback. I did as much testing as possible and it was stable on my end. There are a lot of versions on the home page now - two for each D6 and D7 - but I've heard a lot of support from users for all of the above, so I plan on keeping it that way for the foreseeable future.

Which version are you using? 7.x-2.0-beta1 is incompatible with Views 3 because Views 3 was released after that version. The compatibility issues have since been fixed. I recommend the development snapshot. I just committed a bug fix for an issue with the default view that prevented the block from being displayed on certain node types.
SQLSTATE[42S22]: Column not found: 1054 Unknown column 'score' in 'order clause'
This message was an error that was commonly seen in beta1 as well. It will still occur if the FULLTEXT search is not being executed. That is, if you modify the contextual filter in such a way that prevents it from running the query then the 'score' field will not exist. Without the FULLTEXT search the view is useless anyways, so generally if you get this error it means the view is not going to return results regardless. If you're not using beta1 any more please update the issue version so I know where to look. After the release of Views 3 beta1 began giving similar errors when trying to edit Similar Entries arguments, filters, sorts, and fields.

Can you give more information on how exactly you modified the contextual filter?

Here's a patch that will be committed to prevent that error message in the event that the query is removed.

Committed.

But still need more information. Please try the development snapshot. Make sure that you completely uninstall and reinstall the module.

What do you mean there's no block info? If you use the development snapshot that error should go away as I mentioned. But the fact that you're seeing that error means you won't get any kind of accurate results anyways because the 'score' field is only active when the FULLTEXT search is actually in the query.

changed "When the filter value is NOT available" to "Display a summary ""

Herein lies the issue. This argument can be, but should not be changed in this manner; it will only cause a list of results that are not actually relevant. The argument is not passed to the view directly, so it needs to get the content ID from URL in order to work properly. Essentially, the summary displays a list of links that are out of order which is fairly useless. While the preview will always work since you're inputing an argument directly, the actual block view must have an argument from the URL to work.

The current development snapshot sets the configuration options on the contextual filter in this manner. But 7.x-2.x-beta3 has known issues with the default block view which is why I suggested the development snapshot. I can't duplicate this issue in my own environment with the current snapshot (in fact, I am able to use the 'Display a Summary' option just fine, though it is completely inaccurate and irrelevant as I mentioned), so I'll need verification that this is still an issue with the current code. I made several commits to fix precisely these issues last night. Also, be sure to clear the Views cache or possibly reinstall the module to ensure the new settings are applied and the default view is reset. Views caches the default view definition which has changed significantly since beta3.

Please try the development snapshot. Uninstall and reinstall the module and let me know if that fixes those issues. It should do so, and you should only need to use the default argument to get content ID from URL.

Disabled default action and argument type in the nid contextual filter to prevent misconfiguration.
http://drupalcode.org/project/similar.git/commit/dd4ca74

That's interesting.
There is certainly a difference in the queries between 6.x and 7.x. Namely, the 7.x branches use the BOOLEAN MODE operator to enable the ability to use operators - give node titles more weight, for example. So, does it seem like some of the module's results are irrelevant? One notable difference with BOOLEAN MODE is that it doesn't discount words that appear in > 50% of nodes, whereas performing the query without BOOLEAN MODE does not add any relevance to words that appear in > 50% of nodes.

Thanks for the report on this. Certainly the module needs to return authentically relevant results or it's useless. But that just gave me an idea. It's clearly possible that we can provide an option for enabling or disabling boolean mode - the advantages of normal mode being the elimination of terms that appear in > 50% of nodes, and the advantages of boolean mode being the ability to increase the relevance of certain fields of a node.

I'm going to work on putting together a patch that will provide an option for each mode. I'd love to hear your results on the relevance of each mode's results.

Okay. This patch adds an option for enabling boolean mode searches. The option can be found in the argument handler (contextual filter). I haven't decided whether this should be enabled or disabled by default, but your results may influence that decision. I'm going to make sure everything is working before I commit it, but I'll let you know.

Lol forgot the patch.

Committed. The development snapshot should be updated shortly with this patch applied, but I'm not setting this to fixed. Instead, I'll wait for results.

The new option is in the contextual filter as I mentioned. I intend to add more options to the BOOLEAN MODE options as well.

The development snapshot probably didn't update yet then. Simply reverting my test view has been updating the argument handler for me, so it just must not be the right code. Otherwise, you may even have to reinstall CTools, which I've had to do before. Sometimes the snapshots take a while to update with new commits - certainly a lot longer than an official release does, but I know that at a minimum the new options should be in the argument's interface if the module was updated. The options are at the bottom of the Similar Entries: Nid argument - the same location where some other arguments had resided previously.

I actually added several new options to the Similar Entries: Nid argument tonight (this morning? lol). With boolean mode there are options for adjusting the relevance of the node title in various ways and adding boolean logical operators to a customizable list of words in the query. The latter feature is for advanced use, but it's useful for excluding common words from the boolean search where the normal search automatically filters out those common words. Also improved the support for including content fields in the query.

Uh oh! I may have to set up a new Drupal installation to test the default view again because it has always been clicked for me. So you weren't getting results for the view before you clicked it? Theoretically, the Similar Entries: Nid argument handler should set that option within Views even if the argument is never edited by the user:

    $options['default_action'] = array('default' => 'default');
    $options['default_argument_type'] = array('default' => 'node');

That code sets the default action to Provide default value and Content ID from URL, but it's certainly possible that Views is not getting this information. I think it's time to set up a clean environment again so I'll do that. The last thing I want is for a critical option to be left blank when a new user installs the module, thus preventing it from working until they post an issue in the queue and figure out the problem.

Thanks a ton for doing these tests. It's better to have a real-world test on a site with real nodes that use real words rather than my test site that has real nodes with fake words. It's difficult to determine exactly how accurate results are when the content is fake, and accuracy of results is an absolute priority for this module. I'd love to know which is more relevant, how much worse one is than the other, and any particular settings that affect accuracy either positively or negatively. I suspect that normal mode may return more relevant results, but I hope boolean mode isn't too inaccurate. I will be doing more work on the boolean mode queries later today to try to improve the relevance of its results by weighting destination node titles significantly.

FYI: I've added even more options to the Similar Entries: Nid argument handler and updated the query to improve the results returned from boolean mode searches. In the development snapshot, there will now be options for increasing or decreasing the relevance of destination node titles, bodies, and fields in addition to the relevance adjustments that already existing for the current node's title field. This option essentially allows the user to manipulate the relevance score by changing the math that's performed in the query. This should allow the user more control over the quality of results when using boolean mode. I think this is about the limit of options that can be made available for improving relevance. Let me know how it works out. Thanks!
#1263898: Include Other CCK fields & also Taxonomy (probably Content Taxonomy Support)

damn... without boolean mode they're relevant and with boolean mode they're useless? This must be due to the nature of the search. Essentially we're using every word of the current node as a search word. And without discounting words that appear in > 50% of nodes it's probably matching so many common words that results become irrelevant.

damn... without boolean mode they're relevant and with boolean mode they're useless? This must be due to the nature of the search. Essentially we're using every word of the current node as a search word. And without discounting words that appear in > 50% of nodes it's probably matching so many common words that results become irrelevant.

Well, boolean mode is currently disabled by default. Whether it will be taken out completely is yet to be seen. There may be instances where users may want to use it. So, the patch in #20 needs to be ported to 6.x-2.x and boolean mode should be disabled in that version as well. I think it's still possible to increase the weight of titles vs bodies while in normal mode as well, so I'll be putting that in both versions after some changes to the interface.

Alright, so this patch provides options for adjusting the relevance of all fields in both normal and boolean mode queries. It performs mathematical calculations on the scores during the query and averages those scores so that scores won't fluctuate too much. Also, boolean mode is disabled by default. This is being committed to the development snapshot.

And here's the patch for the 6.x branch. This ports all the new options discussed here and in the other issue. It also indexes the database different, with each field having its own index. This is what allows us to apply math to specific fields.

It should be unchecked by default and it will definitely stay that way because of the results of this test. The updated code leaves boolean mode off and warns the user that it may negatively impact the accuracy of results. But boolean mode will stay in for the few cases where users might want to use the view on nodes with a small amount of text or where finite customization is necessary.

Thanks again for helping to test this. Without a site with real content (only test content with lorem ipsum text) it's tough to determine exactly how accurate the queries are. I think I've figured out the right interface and queries for this module now, though.

The 6.x-2.x branch's queries are now essentially identical to the 7.x-2.x branch.

Thanks for the update!