Posted by dotman on November 2, 2011 at 6:01pm
18 followers
| Project: | Feeds |
| Version: | 7.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | needs work |
| Issue tags: | Needs tests |
Issue Summary
I'm importing around 12000 nodes, and everything is coming in fine except there is no summary/teaser created for the body_summary field. the problem is, all my articles have an inline image at the top, so just setting the teaser to trim won't work since the images still shows.
Is there anyway to auto generate, or at least map a field to the summary so it get's populated?
If this has been addressed, please tell which version since I reluctant to move since I've run so many patches.
I'd be willing to contrib/pay someone for a patch if interested as well.
Thanks.
Comments
#1
Currently the body field is treated as any other "text_with_summary" field. In mappers/field.inc it treats it as a generic text field and only populates the value column. Which is a good stop gap measure. But a better mapper for text_with_summary fields would deal with both the value and summary columns.
To implement this,
1. add another mapper in feeds/mappers. link.inc might be good starting points since it has multiple columns in their field storage. the file would be named 'text_with_summary.inc'.
2. after this is done, in your mapping you should see body:text and body:summary. Map whatever source to both of these.
3. use feeds tamper to deal with the summary field. use regexes to remove images. or strip tags to get rid of all html in the summary. also truncate to your favorite length.
Since text_with_summary field is in core, this functionality should be part of the feeds module rather than a contrib module.
#2
Here is a patch. It basically moves the text field mappers to mappers/text.inc. I believe all the fields in mappers/fields.inc should be treated this way for consistency. Thus list.inc, number.inc,etc would be other candidates for their own .inc files.
#3
Hye John, thanks. have you tested this successfully? So upon doing a re-import, i would now map the content for body to body:text, and any custom field for the summary to body:summary? just want to be clear.
thanks.
#4
yeah. I tested it. Thats also how the import would go also. In my tests I was mapping the same field to body:text and body:summary, but two seperate fields should work the same.
#5
The patch works great for the body but it also seems to affect any other text fields defaulting them with a ":summary" suffix and I'm not able to import them via feeds. i.e., field_twitter_account:summary and field_reference_id:summary. I changed the field_reference_id to an int field and the import succeeded. Any ideas?
#6
The patch in #2 works fine, but it does not set the input format of the body field.
I somehow did not manage to create a proper patch, so I just post the updated function which adds the text format (replace your func in text.inc with this one):
function text_with_summary_feeds_set_target($source, $entity, $target, $value) {
if (empty($value)) {
return;
}
if (isset($source->importer->processor->config['input_format'])) {
$format = $source->importer->processor->config['input_format'];
}
// Handle non-multiple value fields.
if (!is_array($value)) {
$value = array($value);
}
// Iterate over all values.
$i = 0;
list($field_name, $sub_field) = explode(':', $target);
$info = field_info_field($field_name);
foreach ($value as $v) {
if (!is_array($v) && !is_object($v)) {
if (isset($entity->{$field_name}['und'][$i]['value'])) {
$field['und'][$i]['value'] = $entity->{$field_name}['und'][$i]['value'];
}
if (isset($entity->{$field_name}['und'][$i]['summary'])) {
$field['und'][$i]['summary'] = $entity->{$field_name}['und'][$i]['summary'];
}
if ($sub_field == 'text') {
$field['und'][$i]['value'] = $v;
}
elseif ($sub_field == 'summary') {
$field['und'][$i]['summary'] = $v;
}
elseif ($sub_field == 'summary_truncated') {
$field['und'][$i]['summary'] = text_summary($v); // ($text, $format = NULL, $size = NULL)
}
if (isset($format)) {
$field['und'][$i]['format'] = $format;
}
}
if ($info['cardinality'] == 1) {
break;
}
$i++;
}
$entity->{$field_name} = $field;
}
#7
Please roll a patch with this code so it can be reviewed and tested by the community.
Thanks,
Jed
#8
#9
Setting status to 'active' until a patch is provided.
#10
hey,
I manually added some lines to the patch from #2. I tried to apply it to a fresh clone from GIT, and it works correctly.
However - maybe you give me a hint, how to automatically create the patch, here is what I tried:
git clone --branch 7.x-2.x http://git.drupal.org/project/feeds.gitcd feedsgit apply -v text_with_summary_mapper-1329626-2.patchgit diff > add_input_format-1329626-10.patchThanks, and best
#11
Is there any reason you dont use the migrate module?
#12
I use Feeds when I want to import content regularly. I would see Migrate module as solution for one time migrations (or did I miss something?)
#13
as mentioned in #5 simple text fields are not working, with the patch posted before. I could fix it easily and updated the patch,
please test..
#14
I cannot see why this is critical at all.
#15
patch #13 works fine for me, thanks a lot!
#16
It would be nice to have some tests here.
#17
#13 working well here.
#18
This should be implemented using the new configuration api for mappers.
#19
This is going to conflict with #1588938: Allow selection of filter for each text field imported. You can see where I'm going with this. I've already split the field.inc into number.inc and text.inc.
#20
I have got this patch working on 7.x-2.0-alpha5 however not on 7.x-2.0-alpha6 I am unable to find the field.inc file am I missing something?
#21
Sorry, figured it out managed to get the patch in text.inc on alpha 6 and it looks like its working.
#22
Can someone post a patch that applies to the latest dev, please? Lakes, sounds like you got it working (if you dont know how to patch, maybe post your text.inc file so I can roll it) Thanks.Patch below...
#23
Here's a quick patch that (hopefully) applies to the latest dev.
#24
Patch in #23 confuses mapping when there are more than one text with summary fields.
I'm using XML Expression parser and when I add text/teaser fields for both my text-with-summary fields, only one body text field appears in the XML Expression Parser settings field.
#25
I think issue #962912: Mapping to node summary deals with the exact same problem.
#26
I've tried the patch in #23 and I had one problem with it: mapping to plain text fields no longer worked. That is because I believe the callback for plain text fields is wrong. It should not be
field_feeds_set_target_text, buttext_feeds_set_target.(I have no time at the moment to post a new patch.)