Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 PST on 31 March 2024, to get $100 off your ticket.
Feeds saves a hash of source row data and skips processing the hash is the same. However, I'd like to be able to force an update for a particular batch job. Even if the source row hasn't changed, I may have changed the processing behavior with hook_feeds_after_parse() or Feeds Tamper module settings.
Comments
Comment #1
twistor CreditAttribution: twistor commentedIf you have changed the way the items get parsed then the hash of the items will change.
As a note, if you want to re-run an import you can delete the last two mappings and re-add them in the opposite order. Not ideal, I know, but can be useful.
Still a valid feature request.
Comment #2
geek-merlin>If you have changed the way the items get parsed then the hash of the items will change.
Source code says: hash comes from source data and unchanged hash saves us from forther processing - which is a good thing in production.
I think this is an important DX issue and iterating a feed is a PITA without that so raising prio (feel free to readjust if i'm too self centered).
Here is a straightforward implementation which worksforme, please test.
As soon as this gets committed i will simplify my pending patch in #1152940: Feeds term import with hierarchy and weight where we needed the same for a two-pass import.
Comment #3
twistor CreditAttribution: twistor commented#1152940: Feeds term import with hierarchy and weight got committed. Can you re-work this patch to remove the Term specific functionality?
Comment #4
geek-merlin> Can you re-work this patch to remove the Term specific functionality?
I don't see term specific functionality in #2...?
Comment #5
twistor CreditAttribution: twistor commentedFeedsTermProcessor has its own version of this patch added in #1152940: Feeds term import with hierarchy and weight. That code should be re-factored to use this patch.
Comment #6
geek-merlinPatch in minisprint branch:
* Implemented 1364116: Option to skip hash check on re-import - i reworked that to stay compatible for most scenarios by letting getHash return a fake hash so other importers can inherit that.
* Reverted force-update config in #1152940 as skip-hash-check #1364116 implements that
* also this fixes #1698076: Force update for TermProcessor should default to FALSE.
Comment #7
geek-merlinComment #8
geek-merlinComment #9
twistor CreditAttribution: twistor commentedThis should actually be 2 settings. Forced updating should be set on a per-source basis.
Comment #10
twistor CreditAttribution: twistor commentedComment #11
geek-merlinI don't agree moving the "force update" setting from "importer" to "source".
At least i don't understand your background.
For me this setting is an extension to the "update existing content" setting and so global to the processor.
Having this per source is a different issue to me which i now see no use case.
Comment #12
twistor CreditAttribution: twistor commented@axel, I concede the point. My thought is that it's most useful for debuggin/developing, rather than a long term solution. I agree though, that it's more of an extension to "Update existing". So, then why don't we put it in that list :)
Comment #13
geek-merlin> it's more of an extension to "Update existing". So, then why don't we put it in that list :)
I already thought about that but "skip hash check" is orthogonal to the "update / replace" setting.
Incorporating in the list would multiply that out to 5 options like this reworded)
old wording:
what do you think about this?
Comment #14
twistor CreditAttribution: twistor commentedCommitted to 7.x
http://drupalcode.org/project/feeds.git/commit/71093af
http://drupalcode.org/project/feeds.git/commit/71093af
Missed you on the second one Axel.
Comment #15
slefevre1 CreditAttribution: slefevre1 commentedI think it would be handy to have this option on the import page of each feed. We ran into a situation where we had a feed for a content type that was in a features module. We added some fields to the feature, but our feeds ran from cron before we updated the mapping, and feeds considered the nodes to have been updated even though it didn't care about the new fields when it ran.
It would have been convenient to run the feed on its /import page, checked the box to ignore hashes, and then just go on our merry way. We don't want the hashes skipped on each import, just when we screw things up :P
Comment #16
osopolarHere is the 6.x backport. It's only for FeedsNodeProcessor because the other processors aren't using hashes at all.
Comment #18
osopolarCan't see how the patch causes the test failure - I guess there is something else causing the test to fail. Maybe Changing the version to current dev helps.
Comment #22
geek-merlin@osopolar: the patch is frozen to beta and probably won't apply (which is a good feature ;-)
you might want to re-upload it with dev version set !
Comment #23
osopolarPatch for dev attached. The patch in #17 applied well, but there it seems that there is something wrong with the locale (language) mapper mapper. Let's see if the problem goes away by itself.
Comment #24
moonray CreditAttribution: moonray commentedWorks. The line numbers just changed slightly, but the patch still applies.
Comment #26
twistor CreditAttribution: twistor commentedLooks fine.
Not too much work going on in 6.x these days.