| Project: | Feeds |
| Version: | 7.x-2.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | critical |
| Assigned: | Unassigned |
| Status: | needs review |
| Issue tags: | D7 stable release blocker, Needs tests |
Issue Summary
Problem/Motivation
There is D6 fatal PHP code in this module; It calls taxonomy_node_get_terms(), which just flatly does not exist in D7.
To create the failure: Create a mapping which maps a taxonomy term from the feed definition node to a term field on the target node as in this screenshot:
Then run a feeds import, perhaps with "drush cron".
You end up with an error like
Error: Call to undefined function taxonomy_node_get_terms() in
sites/all/modules/feeds/mappers/taxonomy.inc, line 30Proposed resolution
Several contributors have collaborated to provide a patch, rerolled in #82, and which includes a test.
Remaining tasks
Module maintainers and others should review and test the patch to make sure it does what it is supposed to and nothing else.
Original report by dasjo
taxonomy_feeds_get_source uses taxonomy_node_get_terms() which doesn't work anymore with drupal 7:
An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: /batch?id=9&op=do StatusText: OK ResponseText: Fatal error: Call to undefined function taxonomy_node_get_terms() in /feeds/mappers/taxonomy.inc on line 31
this happens as taxonomies are stored in fields for drupal 7
http://drupal.org/node/909968
Comments
#1
I've smacked into the same issue. I'm unable to assign taxonomy terms from feeds to feed items.
This error occurs when attempting to import, after having mapped the correct term fields.
#2
Needs fixing.
#3
Sub. Function taxonomy_node_get_term simply doesn't exist anymore in D7, I don't know why.
#4
See also #978242: Document the changes to taxonomy.module properly to avoid confusion about missing $node->taxonomy property.
#5
Subscribing. This is a critical issue, as it's one of the greatest feature of feeds.
#6
The error still exists in Drupal 7 final, when trying to import items and automatically assign terms from the parent feed to them. It's easy to replicate. The error is:
An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows.
#7
Subscribing. Same problem.
#8
Getting the same issue... Subscribing.
#9
Subscribing.
#10
Subscribing
#11
Subscribing
#12
Probably not the best solution, but you can add the taxonomy_node_get_terms re-factored for D7 to a custom module or template.php. Again this is NOT the best solution, but it will get you by for now.
function taxonomy_node_get_terms($node, $key = 'tid') {
static $terms;
if (!isset($terms[$node->vid][$key])) {
$query = db_select('taxonomy_index', 'r');
$t_alias = $query->join('taxonomy_term_data', 't', 'r.tid = t.tid');
$v_alias = $query->join('taxonomy_vocabulary', 'v', 't.vid = v.vid');
$query->fields( $t_alias );
$query->condition("r.nid", $node->nid);
$result = $query->execute();
$terms[$node->vid][$key] = array();
foreach ($result as $term) {
$terms[$node->vid][$key][$term->$key] = $term;
}
}
return $terms[$node->vid][$key];
}
#13
Thanks desmondmorris.
I managed to get the batch process to finish but when I check the node created by the feed, the taxonomy had not been updated. Did your new node manage to save the taxonomy term?
#14
sovarn, yes my new nodes did save the taxonomy term as expected. You must make sure that:
1. you have a taxonomy term associated to your feed
2. your feed item has a term_reference field
3. you have mapped your feed term to your feed item term in the node processor of your feed importer
#15
I got it working.
You have to have the feeder node to be published. This is because it will will only appear in the taxonomy-index table if the node is published.
Also putting the code in template.php didn't work as cron does not find the function if you want it to be updated.
Thanks again.
#16
sovarn, good catch about cron. glad I could help.
#17
Subscribing. I would love to see this fixed, though i do understand that Feeds is only alpha3 at this time.
#18
For a quick hacked fix, put the function in #12 into the .module file or your own module and it will work fine.
#19
I have taken a shot at a patch for this issue. I renamed the function from #12 to "taxonomy_feeds_node_get_terms" and added it to feeds/mappers/taxonomy.inc. I then changed the call to taxonomy_node_get_terms to taxonomy_feeds_node_get_terms in the taxonomy_feeds_get_source function, also in taxonomy.inc.
#20
I forgot to change the status to "needs review"
#21
Subscribing
#22
same here - sub
#23
Subscribing
#24
#19 work great !
I notice that if the node feed is not publish the term is not transfer to the node feed item. It's not a big deal !
For now the issue look solve ! Maybe apply the fix to DEV version... Thanks !
#26
#19 works fine.
#27
+1 subscribing
#28
Why would such a handy function be removed?
#29
can we please commit this?
#30
subscribing
#31
subscribing
#32
Can anybody explain why taxonomy_node_get_terms() does not exist any more? Or why the taxonomy_index table only matches published nodes? It seems that the taxonomy functionality in D7 is vastly reduced from what it was in D6 or D5, but with no good reason. This is a definite WTF.
#33
Scott: You can find an explanation for the changes in Drupal 7 in #978242: Document the changes to taxonomy.module properly to avoid confusion about missing $node->taxonomy property.
#34
+++ b/mappers/taxonomy.incundefined@@ -106,6 +106,27 @@ function taxonomy_feeds_set_target($source, $entity, $target, $terms) {
+ $query->condition("r.nid", $node->nid);
+ $result = $query->execute();
+ $terms[$node->vid][$key] = array();
+ foreach ($result as $term) {
+ $terms[$node->vid][$key][$term->$key] = $term;
+ }
+ }
+ return $terms[$node->vid][$key];
If we are retrieving terms by a node's ID, but caching them by VID that could be confusing. Also, we should probably be only loading the tid column and then running taxonomy_term_load_multiple().
In addition, the taxonomy_index table is not maintained for any unpublished nodes. Will that become a problem here?
Powered by Dreditor.
#35
#36
sub
#37
Marked #1140346: Taxonomy mapping not functioning in Feeds 7.x and #1190886: Possible to add taxonomy with node processor importing to an article? as duplicates of this issue.
#38
#20 did work at one point. Anyone have any luck?
#39
#20 did work at one point. Anyone have any luck?
#40
Why did you change the priority and assignment?
#41
#42
This patch follows guidelines from #34:
- nid is used as a key in static cache variable
- code that gets tids from all taxonomy fields is completely changed; it fixes issue with unpublished nodes now
#43
Previously added tids with the same key get wiped here:
$tids += array_map('_taxonomy_extract_tid', field_get_items('node', $node, $field_name));Same patch as in #42 attached but with above line replaced with:
$tids = array_merge($tids, array_map('_taxonomy_extract_tid', field_get_items('node', $node, $field_name)));#44
Subscribing coming from http://drupal.org/node/988856#comment-4731528
More reading gives me this quote: "Feeds already does a mapping and need to support mapping incoming terms to taxonomy reference fields". See: http://drupal.org/node/978242#comment-3748214 under feeds.
Term reference fields are supported by this patch, right?
greetings, Martijn
#45
Subscribing. Same problem.
#46
I have tested patch #43 to work for this issue. Only issue I see is the extra whitespace that needs to be removed. But hopefully this can be done when committed.
#47
This still really could use a simpletest to confirm its behavior. :/
#48
Maybe provide a
feeds_taxonomy_node_get_terms()haha :-)
<?php
/**
* The function missing from D7
* Taxonomy node get terms
*/
function feeds_taxonomy_node_get_terms($node, $key = 'tid') {
static $terms;
if (!isset($terms[$node->vid][$key])) {
$query = db_select('taxonomy_index', 'r');
$t_alias = $query->join('taxonomy_term_data', 't', 'r.tid = t.tid');
$v_alias = $query->join('taxonomy_vocabulary', 'v', 't.vid = v.vid');
$query->fields( $t_alias );
$query->condition("r.nid", $node->nid);
$result = $query->execute();
$terms[$node->vid][$key] = array();
foreach ($result as $term) {
$terms[$node->vid][$key][$term->$key] = $term;
}
}
return $terms[$node->vid][$key];
}
?>
#49
Subscribing
#50
I'm also having this problem which is preventing me from re-implimenting D5.9 Feed Parser based solution as a "Feeds" solution on D7.8 using Feeds 7.x-2.0-alpha4 and CHAOS 7.x-1.0-rc1.
My "Feed" and "Feed Item" content types both have the same custom "Term reference" field type with identical title and field names. Every time a import attempt occurs the progress bar briefly appears saying initialising and then the above message appears.
About to try 7.x-2.x-dev 2011-Aug-14
Note: If you are trying to have RSS or XML (etc.) "Feed Item" nodes inherit a taxonomy term from the parent "Feed" importer node and seeing "An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows." then this Issue page appears to be the most relevant of several similar sounding Feeds project issues created ;¬)
#51
No change with 7.x-2.x-dev 2011-Aug-14
#52
#43 worked for me, thanks for that!
#53
subscribe
#54
Subscribe
#55
subscribing.
#56
Subscribing
#57
Subscribing.
#58
[off-topic] Applebee, you don't need to add a comment in order to subscribe anymore. There is now a "Follow" button at the top right of the page. [/off-topic]
#59
The patch in #43 worked for me.
#60
#43 works
Confirmed
#61
#43 works
Thx
#62
+++ b/mappers/taxonomy.incundefined@@ -106,6 +106,48 @@ function taxonomy_feeds_set_target($source, $entity, $target, $terms) {
+ static $terms;
Should this be using drupal_static() in Drupal 7?
+++ b/mappers/taxonomy.incundefined@@ -106,6 +106,48 @@ function taxonomy_feeds_set_target($source, $entity, $target, $terms) {
+ if ($field['type'] == 'taxonomy_term_reference' && field_info_instance('node', $field_name, $node->type)) {
+ $tids = array_merge($tids, array_map('_taxonomy_extract_tid', field_get_items('node', $node, $field_name)));
+ }
+ }
field_get_items() can return a non-array value. Is this condition covered here or will calling array_map() throw an error?
+++ b/mappers/taxonomy.incundefined@@ -106,6 +106,48 @@ function taxonomy_feeds_set_target($source, $entity, $target, $terms) {
+ }
+ ¶
+ // Load terms and cache them in static var.
Trailing whitespace.
+++ b/mappers/taxonomy.incundefined@@ -106,6 +106,48 @@ function taxonomy_feeds_set_target($source, $entity, $target, $terms) {
+ * tid from array item returned by field_get_items().
+ * ¶
+ * @param $item tid information in a form of single element array (key == 'tid', value == tid we're looking for)
+ * ¶
+ * @return tid extracted from $item.
+ * ¶
+ * @see taxonomy_feeds_node_get_terms()
Trailing whitespace.
#63
Rerolled patch from #43 with suggestions from #62.
#64
Better still, rerolled patch from #43 with suggestions from #62 and a test.
Tweaked testInheritTaxonomy() in feeds_mapper_taxonomy.test to work for D7.
Commented out testRSSCategoriesToTaxonomy() as that also needs to be rewritten for D7.
#65
#64 failed on a manual patch with this:
patch -p1 < feeds__taxonomy_node_terms_fix_959984_64.patch
patching file feeds.info
patching file mappers/taxonomy.inc
patching file tests/feeds_mapper_taxonomy.test
Hunk #2 FAILED at 118.
1 out of 3 hunks FAILED -- saving rejects to file tests/feeds_mapper_taxonomy.test.rej
first couple of lines of the rej file are:
*************** class FeedsMapperTaxonomyTestCase extends FeedsMapperTestCase {
*** 65,102 ****
* Test inheriting taxonomy from the feed node.
*/
Was I doing something wrong?
I reverted the changes and applied #63 successfully; testing it now.
#66
Make sure you are in the root of the module folder e.g. /feeds when you apply the patch.
#67
I think I'm in the correct folder; the first two files look to be patched correctly, it's the test/feeds_mapper_taxonomy.test that doesn't seem to be patching right. I'll attaching the rejects file as a txt file.
#68
Ah. Looks like you're applying it to the 2.0-alpha4. You need to apply it to the latest 7.x-2.x-dev release.
#69
This issue hasn't been solved after more than one year. Will a stable release of Feeds module finally fix this error? I really don't want to apply a patch.
#70
it's still "needs work" as stated in #62, so you can help finalize it or just wait
#71
The patch of #64 went through, but had no effect. I am still not able to inherit the tags from a feed to a feed item while the import of the feed items themself this time did not cause an error message.
#72
@flailingmaster: Did the patch work for you?
@lyy9981: Please test the patch on a development site, if you can. It needs community feedback/review.
@maerys: Can you provide an export of your feed importer and/or provide details of your feed node -> feed node item set up?
#73
@kidrobot oh, of course, sorry.
Feed importer
$feeds_importer = new stdClass;$feeds_importer->disabled = FALSE; /* Edit this to true to make a default feeds_importer disabled initially */
$feeds_importer->api_version = 1;
$feeds_importer->id = 'feed';
$feeds_importer->config = array(
'name' => 'Feed',
'description' => 'Import RSS or Atom feeds, create nodes from feed items.',
'fetcher' => array(
'plugin_key' => 'FeedsHTTPFetcher',
'config' => array(
'auto_detect_feeds' => 1,
'use_pubsubhubbub' => 0,
'designated_hub' => '',
),
),
'parser' => array(
'plugin_key' => 'FeedsSyndicationParser',
'config' => array(),
),
'processor' => array(
'plugin_key' => 'FeedsNodeProcessor',
'config' => array(
'content_type' => 'feed_item',
'update_existing' => '0',
'expire' => '-1',
'mappings' => array(
0 => array(
'source' => 'title',
'target' => 'title',
'unique' => 0,
),
1 => array(
'source' => 'timestamp',
'target' => 'created',
'unique' => FALSE,
),
2 => array(
'source' => 'url',
'target' => 'url',
'unique' => 1,
),
3 => array(
'source' => 'guid',
'target' => 'guid',
'unique' => 1,
),
4 => array(
'source' => 'description',
'target' => 'field_feed_item_description',
'unique' => FALSE,
),
5 => array(
'source' => 'parent:taxonomy:tags',
'target' => 'field_organisation',
'unique' => FALSE,
),
),
'input_format' => 'filtered_html',
'author' => 0,
),
),
'content_type' => 'feed',
'update' => 0,
'import_period' => '1800',
'expire_period' => 3600,
'import_on_create' => 1,
'process_in_background' => FALSE,
);
Both content types »feed item« and »feed« have just three fields: title, body (feed item)/newsfeed (feed) and the taxonomy field based on the same vocabulary (called »organisation« , which just has to options: »personal« or »institution«). Every feed item that is generated upon a feed should inherit the particular taxonomy tag. If a feed is tagged with organisation all the feed items automatically should have this tag as well.
Doesn't sound so difficult but it seems to be tricky. But before i patched there was an error. After patching I was able to import the feed items without error, but the tags do not come up.
Is there any other information you need?
#74
@maerys, it looks like your Mapping for Node Processor settings are not right.
On admin/structure/feeds/feed/mapping you should not have any source mapping to the term reference field Organization because this will prevent the feed_item from inheriting the term from the feed node.
Can you remove that mapping so your settings look more like the attached screenshot? Then try the import again.
#75
Subscribe
#76
eloiv, please see comment #58 and the follow button on top of this page.
#77
+1
#78
Hello @kidrobot. Thanks for answering.
Unfortunately there's a "but"... As I wrote import itself now works after patching, but the transfer of the taxonomy tag of the feed to the feed items still doesn't do its job.
#79
@maerys, taxonomy terms will not be mapped from the feed node to the feed item until you remove this part of from your feed import as this makes the feed item inherit the terms from the source item instead of the feed node.
5 => array('source' => 'parent:taxonomy:tags',
'target' => 'field_organisation',
'unique' => FALSE,
),
Can you remove the above from the code in your module or remove the mapping using the UI on the Mapping page?
#80
ok. it's been removed, so »parent« means source and not feed node. but how do the feed item inherit the taxonomy of its feed node?
EDIT: ah, got it. there is this (new) option [http://cl.ly/3b2R1u0g3S3l0b3T382b] which I have never seen before at the mapping page and now everything works fine. thanks for your patience and great work!
#81
Bumping, and tagging, as this is advertised functionality that doesn't work at all.
This just came on my radar, I'll give it a full review soon.
Will someone to update the issue summary?
#82
Here's #64 by @kidrobot rerolled for current code. Now I'll try it out.
Edit: I tried it out, nodes were created successfully with the taxonomy term I wanted to apply. I'm happy. I didn't look inside the patch.
#83
And updated the issue summary.
#84
Adding a graphic for the issue summary.
#85
Patched #82 to dev and tests pass.
#86
Patch in 82 is failing against dev right now.
#87
I'm having trouble applying this patch. I did a fresh pull, and when I git apply --check on master, I get:
error: patch failed: feeds.info:23
error: feeds.info: patch does not apply
error: patch failed: mappers/taxonomy.inc:27
error: mappers/taxonomy.inc: patch does not apply
error: patch failed: tests/feeds_mapper_taxonomy.test:20
error: tests/feeds_mapper_taxonomy.test: patch does not apply
If I checkout 7.x-2.0-alpha4, I get:
error: patch failed: tests/feeds_mapper_taxonomy.test:65
error: tests/feeds_mapper_taxonomy.test: patch does not apply
Any hints?
#88
@killtheliterate, @handsofaten: Try the 7.x-2.x branch. You should see something like this.
~/Drupal/Patches $ git clone --branch 7.x-2.x http://git.drupal.org/project/feeds.gitCloning into feeds...
remote: Counting objects: 4755, done.
remote: Compressing objects: 100% (2016/2016), done.
remote: Total 4755 (delta 3594), reused 3662 (delta 2717)
Receiving objects: 100% (4755/4755), 1.70 MiB | 355 KiB/s, done.
Resolving deltas: 100% (3594/3594), done.
~/Drupal/Patches $ cp feeds.taxonomy_node_get_terms_fail_959984_82.patch feeds/
~/Drupal/Patches $ cd feeds
~/Drupal/Patches/feeds $ git apply -v --check feeds.taxonomy_node_get_terms_fail_959984_82.patch
Checking patch feeds.info...
Checking patch mappers/taxonomy.inc...
Checking patch tests/feeds_mapper_taxonomy.test...
~/Drupal/Patches/feeds $ git apply -v feeds.taxonomy_node_get_terms_fail_959984_82.patch
Checking patch feeds.info...
Checking patch mappers/taxonomy.inc...
Checking patch tests/feeds_mapper_taxonomy.test...
Applied patch feeds.info cleanly.
Applied patch mappers/taxonomy.inc cleanly.
Applied patch tests/feeds_mapper_taxonomy.test cleanly.
#89
Thanks, kidrobot. Seems to be working.
#90
IMHO, taxonomy_feeds_node_get_terms() is something that all modules should be able to (re)use. Would there be a way to make it less feeds-specific? In an ideal world this would be in core, but I assume adding functions in core is not going to happen...
#91
Thanks @Kidrobot. Now, I'ma see if this patched module will import into terms.