I have very large CSV file which imports during nights only and over 3 days. Next day I set it up it stuck. Manual run of import cront job showed similar issue to #1029102 , i.e. Warning: fopen(public://feeds/FeedsHTTPFetcherResult1295145004_1) [function.fopen]: failed to open stream: "DrupalPublicStreamWrapper::stream_open" call failed in FeedsFetcherResult->sanitizeFile()

This is due to the fact that file is marked as temporal by feeds fetcher plugin and removed after 6 hours by Drupal.

Proper solution might be managing all files used by feeds and never mark these temporary and instead delete these when feed is imported.

CommentFileSizeAuthor
#1 feeds-redownload_remote_file-1801680.patch1009 bytespavel.karoukin

Comments

pavel.karoukin’s picture

pavel.karoukin commented
Title: Large file import - temporary file gets removed before feed finish import and never re-downloaded causing fatal error » [PATCH] Large file import - temporary file gets removed before feed finish import and never re-downloaded causing fatal error
Status: Active » Needs review
StatusFileSize
new feeds-redownload_remote_file-1801680.patch1009 bytes

I've found easy solution - re-download remote file if local file was removed. Patch attached. This potentially might cause problems if feed file will change between downloads. Not sure so. Discussion is welcome.

twistor’s picture

twistor commented
Status: Needs review » Closed (duplicate)

Can we move this over to #1029102: Importing Large CSV document (downloaded and processed in the background)?