Under "Advanced configuration" on the "More Like This" new block configuration page (admin/settings/apachesolr_mlt/configure_block), it doesn't make sense (to me) to have the default value of the "Minimum document frequency" field be 1. It seems to me, you need to have at least two documents containing the field in question before you can compare the open node to anything other than itself.

This patch changes the default, pre-fill from 1 to 2.

CommentFileSizeAuthor
apachesolr-mindocs.patch984 byteshorncologne

Comments

mikejoconnor’s picture

mikejoconnor commented

The mindf is actually the minimum number of times the word appears in the document, not the minimum number of documents the word is in. Poor naming.

I think it's best to leave it at one, with the idea that you may only use a keyword once in a document, yet it is still relevant. I would like other opinions on these settings, I haven't had a chance to test them.

pwolanin’s picture

pwolanin commented
Status: Active » Closed (won't fix)

let's leave as is in the absence of compelling argument for change.

horncologne’s picture

horncologne commented
Status: Closed (won't fix) » Active

Thanks for the clarification, Mike. If the functionality is as you say, then I see that 1 is fine.

The name, however, led me to the wrong conclusion. I will continue to suggest a patch that makes this clear.

- jam.

horncologne’s picture

horncologne commented
Status: Active » Needs review

Now I am confused, Mike!

This is line 214-224 of apachsolr_mlt.module:

  $form['advanced']['mlt_mintf'] = array(
    '#type' => 'textfield',
    '#title' => t('Minimum Term Frequency'),
    '#description' => t('A word must appear this many times in any given document before the document is considered relevant for comparison.'),
    '#default_value' => isset($block['mlt_mintf']) ? (int) $block['mlt_mintf'] : 1,
  );
  $form['advanced']['mlt_mindf'] = array(
    '#type' => 'textfield',
    '#title' => t('Minimum Document Frequency'),
    '#description' => t('A word must occur in at least this many documents before it will be used for similarity comparison.'),
    '#default_value' => isset($block['mlt_mindf']) ? (int) $block['mlt_mindf'] : 1,

You seem to be describing the functionality of mintf rather than mindf - can you please make sure that we are talking about the same thing?

Seems to me like the minimum mintf == 1 is perfectly good as a default.

But if the minimum mindf == 1 that means (correct me if I am wrong) that the given term must appear in at least one node/document for it to be used for comparison. Since the node being displayed has that term, the worst case scenario would be apachesolr_mlt comparing the content to itself ... right? That's why I think this should be two as a default/minimum. A term must appear in at least one *other* document as well or this is meaningless.

- jam.

pwolanin’s picture

pwolanin commented

docs:
http://wiki.apache.org/solr/MoreLikeThis
http://wiki.apache.org/solr/MoreLikeThisHandler

horncologne’s picture

horncologne commented

Good stuff! Now that I have RTFM'd ...

mlt.mintf - Minimum Term Frequency - the frequency below which terms will be ignored in the source doc.

mlt.mindf - Minimum Document Frequency - the frequency at which words will be ignored which do not occur in at least this many docs.

... I still don't see how it makes any sense to compare a word that appears in only one doc. That comparison must be with itself, right?

If a word appears in two documents, then you have something to compare and similar content to recommend.

/me clarifies: The world won't end over this one, I just think it makes sense one way and it probably works just fine another way, too ... I'll let go at this point.

Cheers,

- jam.

pwolanin’s picture

pwolanin commented

I'd agree that making mindf 2 is possibly more sensible, but having at 1 does have an effect. The mlt handler is, I believe, doing a kind of term vector distance calculation, so if the term occurs only in one document it will increase the relative distance of that document from all other documents.

pwolanin’s picture

pwolanin commented
Status: Needs review » Closed (works as designed)