Mapper for Taxonomy

I would love to see some sort of taxonomy mapping on feed imports. Articles for stlouisreview.com are imported using FeedAPI, and when an article is imported, it is automatically assigned a main section term for sorting the article from one vocabulary, then is assigned free tags using another freetagging vocabulary.

That's what, I'm guessing this issue is about? Namely, when importing feed items into nodes, taxonomy terms can be mapped from the feed items to the node?

yes geerlinguy

Its allready done and working, see the patch in #1.

Alex i will fix the coding styles and stuff.

About the "tags requirement", i guess you got it wrong. A Tax. / Vacab does not need to be a "tags" one, to work with the feed api.
The only point where tags and "non tags" vocabs are different in that mapper is:
- If the vocab is not a "tags" one, and the feed-entry has a term, which does not yet exist as a term in drupal it will NOT be created and NOT be attached to the feed-entry-node
- When its a tag, the term is auto-created and attached

But this is only for the terms which are not yet present in the drupal-vocab. All the others will work like you expect it.

Ah, good! Cool!

subscribing

Just wondering - has this patch been applied to HEAD yet? If not, can I safely apply the patch to my production site? I'd like to get this functionality working a.s.a.p., but can live without it for a while.

[Edit: For those interested in testing - grab the file in reply number 1 above, rename it taxonomy.inc, then place it inside the 'mappers' folder in your feeds module folder.]

I have not looked at the code but have tried #1. I'm able to specify vocabularies as targets in a node processor, but no term gets created and even when the terms already exists, no term-node association gets created.

@geerlingguy tried doing that but I do not seem to see any difference. I assume there should be a new option in the mapping section. Can you confirm where the change is seen?

Cheers,
Nick

If you go to site building > feeds > list, then edit the feed you're using, you can go to the 'Node processor' and then select a Source and Target - and for whatever node type you're using, you should see the taxonomies associated with that node type in the list, like "Taxonomy: Tags."

Hi,

I have just tried that and it doesn't seem to work for me. I have a taxonomy, just for testing it is associated with every node type. I have created a new feed type, but when I go to Node processor I do not see any TAXONOMY related name listed.

I can see category in the source but nothing in the destination/target. This may well be me doing something wrong. I can however see the taxonomy in any other page contant I have created.

Nick

Okay, got it now, I ahd renamed it ,inc.php instead of just .inc.

I'm able to verify: no terms are getting matched, even though the .inc file seems to allow me to at least make the initial mapping.

As I currently need this function I did a complete rewrite.

I did check the code with the coder module and rewrote anything to comply to the coding standards. Hopefully I did not forget something ;)

function taxonomy_feeds_set_target(&$node, $vid, $terms) {

  // Load target vocabulary to check, if it has the "tags" flag.
  $vocab = taxonomy_vocabulary_load($vid);
  if ($vocab === FALSE) {
    watchdog('FeedsTaxonomy', 'The selected vocabulary with the id %d could not been loaded.', array($vid), WATCHDOG_WARNING);
    return;
  }

We now do have an error message if the vocabulary gone missing for any reason.

  // Unset any previous tags so we can update.
  if ($vocab->tags) {
    unset($node->taxonomy['tags'][$vocab->vid]);
  }
  else {
    unset($node->taxonomy[$vocab->vid][$term_found->tid]);
  }

I unset any tags so we can do a clean refresh.

  // Return if there are no or empty terms.
  if (!is_array($terms) || empty($terms)) {
    return;
  }

  if ($vocab->tags) {
    // Simply add a comma separated list to the nodefor a "tags" vocabulary.
    $node->taxonomy['tags'][$vocab->vid] = implode(',', $terms);
  }

A "tags" vocab is added quiet easily. See taxonomy.module from line 654 to 686.

  else {
    foreach ($terms as $term_name) {
      // Check if a term already exists.
      if ($terms_found = taxonomy_get_term_by_name_vid($term_name, $vocab->vid)) {
        // If any terms are found add them to the node's taxonomy by found tid.
        foreach ($terms_found AS $term_found) {
          $node->taxonomy[$vocab->vid][$term_found->tid] = $term_found->tid;
          if (!$vocab->multiple) {
            break;
          }
        }
        // If the vocab is not for multiple tags break after the first hit.
        if (!$vocab->multiple) {
          break;
        }
      }
    }
  }
}

The adding of terms in a none tags vocab is quiet easy too. See taxonomy.module 688 to 704. By taxonomy_get_term_by_name_vid all terms of the current vocab that match are selected and added to the terms array of the node. I try to add any terms that are found. This is just for the strange case a vocab contains a the same tag more than once. Drupal can do this so I just add this. If the vocab is not for multiple tags we quit after we found the first tag.

This means if we want a feed to bring in any tags use a "tags" vocab. If you want to filter the tags a feed can use import into a "normal" vocab and maybe activate multiple tags. By this you can make sure only existing tags from a vocab will be used.

So far so good but there is a problem. taxonomy.module in line 693 needs a vid for the node. This of course is not the vocab id but the version id of the node! If nothing is set we can't update any tags because they always refer to vid 0.

I will look into this as I believe this is a problem of feeds in common. But I can't find the invocation of the hook right now. So I set this issue to "needs work". But initial import should work fine and maybe somebody can already test and review it.

@alex_b If you read this please tell me if I am missing something or if the $node->vid is not properly set. Feel free to contact me by email if this is better than using this issue as its basically off topic. :)

This now depends on #634886: Add vid to node proccess functions or the vid has to be added in this mapper.

KarsT are you seriuos? Are you really making a complete rewrite, missing 50% of special cases and really think, this is the best approach? I mean, why not adjust or bugfix the code we had? Beside you reuse my code and talk about a completely rewrite..whatever. This is how you piss of people the big way.

Beside the mapper works perfectly, people have the wrong understanding what a "tag" taxonomy is, and what a normal taxonomy is (by theory). So thats why for some terms are not created... because its not a "tags" taxonomy.

The mapper works and is live for some weeks now.

This is NOT tested with content_taxomomy, but this is a completely different thing. You have to use the taxonomy module and the the vocab you want to use in the mapper, that this is assigned to the content_type XYZ. So before it works in the mapper, you must be able to add terms in the vocab when you edit / add this content type.

When it does, you will see the taxonomies for the mapping.

@geerlingguy: Can you please provide more informations:

- What kind of vocab?
- Are there Tags in the Feed, what is the format of those?(are there terms passed to the mapping, so are terms actually correctly extracted)

!This module does not extract the terms!. So ther terms must be in an array in the feed-item, if they are not, they are not matched
!This module does not analyze the content of the feed and creates tags! This is not the extractor module!

This modules adds terms which ARE assosiated(so existing ones) with the feed to the node.

Alex would you clear things out here so its clear on which code base we will work, so i know i need to fix anything here. I was away for a week and could not follow this due that.

Hi Eugen I never ment to piss you of. I send you a PM about this. :)

Thanks for the detailed post - the terms in the feed are all pre-existing terms in my 'Section' vocabulary. It's not a 'Tags' type vocabulary, but is 'Multiple select' and 'Required.'

In the feed, they are inside the 'category' tag (separate from the 'content' tag) inside a feed item:

<category term="local news" start="2009-11-16T12:36:08" end="2010-02-24"><![CDATA[local news]]></category>

Are you saying (it seems so, but I think I'm confused) that this mapper will only work with 'tag' type vocabs—i.e. vocabularies with the 'Tags' checkbox checked on the Edit vocabulary page?

Well cleared things up with Kars-T. Sorry for maybe heating stuff up a little, maybe its just my overall frustration about the contribution-procedure in Drupal in general.

Kars-T`s implementation fixes 2 bugs introduced in my version and its better to read. I hope he adds the patch for removing the "unset()" part, due it will produce problems with extractor.

So the first version should be removed or ignored in this thread, further patches (or rewrites :) ) should be done by a patch against 623424_feeds_taxonomy_mapper-d6-v01.patch or the newer one.

In general will need some unit-tests, as the taxonomy implementation and "API" is quite hard to cope with, so we need to get through all cases:
1. All new terms ( not a single exists ) and vocab is NOT a "tags" one -> dont create and dont assign terms
2. All new terms ( not a single exists ) and vocab is a "tags" one -> create and assgin all terms
3. All existing terms and vocab is a NOT "tags" one -> assgin all terms ( no creation)
4. All existing terms and vocab is a "tags" one -> assgin all terms ( no creation)
5. No terms at all and vocab is a NOT "tags" one -> no terms are assigned. What about the current ones
6. No terms at all and vocab is a "tags" one -> no terms are assigned. What about the current ones
7. New terms and existing terms and vocab is NOT a "tags" one-> assign the existing terms, drop the new terms
8. New terms and existing terms and vocab is a "tags" one-> assign the existing terms AND assign / create the new terms

For everbody looking at the code or writing test please be awared, that the way "tags" and normal terms are processed by the taxonomy api is different. Tags are processed as a comma-seperated list and splitted up later in the deeper code-space of taxonomy while NOT "tags" are processed as an array or object of terms.

@geerlingguy: there are 2 bugs in my code, so actually it should not work. I have refactored the code and there are to vars which have not been renamed..plus one special case does not get handled. So please use the version created by Kars-T with the patch he will provide.

New version of the patch to include the suggestions Eugen made :)

I did remove the taxonomy reset because Eugen pointed out that it would destroy any terms that where added by other mapping modules for example a scraper.
And the variabable "$vid" was renamed to "vocab_id" so its clear its _not_ the "version_id" of a node.

Cool - I'll test Kars-T's latest patch to see if that sorts the terms.

subscribing

New version with the paths relative to the feeds directory. I really hope I didn't mess this up again. I had fights over this with the testing bot for days till I figured it out ;)

I just had the case that I wanted to import a single field from a feed into a taxonomy e.g. Author Name so I can easily aggregate over them. But this won't succeed as my parser gave a string to the function and we only accept arrays.

  // Return if there are no terms.
  if (empty($terms)) {
    return;
  }

  // Cast a given single string to an array so we can use it.
  if (!is_array($terms)) {
    $terms = array($terms);
  }

I did change the checks a bit.

Two things:
I don't know if I should add some sanatiy checks like trim and a max lenght?
And what if the parser gives an object? Should this be handled as well?

subscribing

Sweet!

Great thanks an lot! :D

Subscribing, waiting for alpha9 with mapper. Great progression, thanks!
Greetings, Martijn