In the table feeds_importer, every feeds import has a corresponding entry. When one edits a feed importer thru drupal admin, and set the "Expire nodes ", to whatever value "never; 3 days" etc, the entry in the table doesn't get updated. It stays as "s:13:"expire_period";i:3600;". Unless you manually set a different value in the database.

Please help to look at the code to see what's wrong.

Comments

404’s picture

404 CreditAttribution: 404 commented
Title: feeds doens't save "Expire nodes " value to database » set imported nodes to be never expired but they get expired
Priority: Critical » Normal

Sorry, the previous bug report is wrong. By looking at the export of the feed import, I got a better idea,

Feeds correctly updates the value of "expire node".

What didn't change is this:

  'content_type' => '',
  'update' => 0,
  'import_period' => '2419200',
  'expire_period' => 3600,
  'import_on_create' => 1,
  'process_in_background' => 0,
);

What does expire_period mean? And where can one set its value. Does it matter in my case?

But still my feed module behaves oddly. I have a csv exported from another drupal. It doesn't get updated much.

I set never to expire the node, and replacing existing nodes when import.

If I do the initial import, everything is fine. After some time (several hours), i go to import pane, it says 30% items deleted. Nobody tried to delete them. When I click import (at this time, the csv source from another drupal didn't get update), feeds will delete some nodes and create some new ones. Right after that moment, if I click the import, feeds only reports nodes are updated no new nodes created. But just wait a few hours, feeds automatically delete some nodes imported.

I set the unique id in mapping, too.

Feeds is not supposed to delete my imported nodes since i set them to never expire.

Here is the code my feed importer export 

$feeds_importer = new stdClass;
$feeds_importer->disabled = FALSE; /* Edit this to true to make a default feeds_importer disabled initially */
$feeds_importer->api_version = 1;
$feeds_importer->id = 'dhg_book_feeds';
$feeds_importer->config = array(
  'fetcher' => array(
    'plugin_key' => 'FeedsHTTPFetcher',
    'config' => array(
      'auto_detect_feeds' => FALSE,
      'use_pubsubhubbub' => FALSE,
      'designated_hub' => '',
    ),
  ),
  'parser' => array(
    'plugin_key' => 'FeedsSyndicationParser',
    'config' => array(),
  ),
  'processor' => array(
    'plugin_key' => 'FeedsNodeProcessor',
    'config' => array(
      'content_type' => 'dhg_book',
      'expire' => '-1',
      'author' => '1',
      'mappings' => array(),
      'update_existing' => '2',
      'input_format' => 'plain_text',
    ),
  ),
  'name' => '导入读后感图书',
  'description' => '',
  'content_type' => '',
  'import_period' => '-1',
  'import_on_create' => 1,
  'process_in_background' => 0,
  'update' => 0,
  'expire_period' => 3600,
);
mvee’s picture

mvee CreditAttribution: mvee commented
Priority: Normal » Critical

I'm experiencing the very same behaviour as described above and I'm trying to figure out why Feeds automatically deletes imported items even when imported nodes are not configured to expire.

Similarities of my use case to the above description are:

1. I'm importing nodes from a CSV file;
2. Nodes are set to be updated, not replaced;
3. Existing nodes are determined based on a custom field which is defined as unique in the mapper settings.

Any clues as to what the 'expire_period' feed importer setting is for? It is defined on line 241 of FeedsImporter.inc as follows:
'expire_period' => 3600, // Expire every hour by default, this is a hidden setting.

Still trying to debug this but any hints would be appreciated.

ShaunDychko’s picture

ShaunDychko CreditAttribution: ShaunDychko commented

Have you tried exporting your feeds config to a module as described at http://drupal.org/node/622698, then manually changing the expire_period to a really long time in the future? (admittedly, this is just a workaround, but maybe it works?)

Dave Reid’s picture

Dave Reid
he/him
Primary language English
Location Nebraska USA
CreditAttribution: Dave Reid commented
Priority: Critical » Major

This is not exactly critical. Please do not abuse the priority field.

SonniesEdge-1’s picture

SonniesEdge-1 CreditAttribution: SonniesEdge-1 commented

Is there a fix available yet for this bug?

404’s picture

404 CreditAttribution: 404 commented

I am running 7.x-2.0-alpha4. I don't have this problem anymore. I think this is fixed.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic commented
Version: 7.x-2.0-alpha3 » 7.x-2.0-alpha4

I have alpha4 installed and Im having this exact behavior. Im importing nodes via a csv parser, and have the expire set to unlimited. Later, I'll find an arbitrary number of my imported nodes have "expired" and the delete tab shows "deleting 10% complete" or 30% sometimes.

I do not have cron configured, so this is happening during some other hook.

A quick search of the feeds module tree reveals that 'expire_period' is likely a red herring artifact from days of yore. expire_period is *only* set. never accessed anywhere.

A little more digging reveals that there is a stray job in the job_scheduler table with feeds_source_clear in there. This may somehow be related to the fact that I couldnt get the "Process in Background" to do anything (even when I had cron setup properly) So I may have hit delete while that was running, and it somehow created this feeds_source_clear job, and then when I went to "Import on Submission" and *not* "Process in background" It may have not cleared out any related Scheduled Jobs. And so I keep importing.. and whenever the Job scheduler happens to run, its finding that old clear job. (Maybe when I clear the cache or something? not running cron, so not sure whats causing it to execute).

Anyhow no time to dig, but its highly possible that this is *still* a bug, and results in deleted content.

tomcatuk’s picture

tomcatuk CreditAttribution: tomcatuk commented

Subbing. Nodes seem to be expired (deleted) regardless of the settings in the node importer.

ejwettstein’s picture

ejwettstein CreditAttribution: ejwettstein commented

I'm having the same problem with importing a CSV file. Everything goes fine and then the items will be deleted shortly after the import is done. The expiration is set to "Never". I am losing data. (I have to keep reimporting it!)

It's fine in one installation and broken in another one. Both are running the same version.

Okay, the log shows Notice: Undefined variable: job in FeedsSource->scheduleClear() (line 312 of C:\www\d7.oprforgs.org\sites\all\modules\feeds\includes\FeedsSource.inc). This is when the items get deleted. (Rolled back?)

ejwettstein’s picture

ejwettstein CreditAttribution: ejwettstein commented

Doh... I uninstalled feeds completely, re-installed and recreated import rules. Everything seems to good now!

martysteer’s picture

martysteer CreditAttribution: martysteer commented

I had this problem too, and noticed the feeds_source_clear record in the job_scheduler table (as mentioned in #7).

I removed this entry using PHPMyAdmin and it seems to have fixed my node expiration on cron problem.

khiminrm’s picture

khiminrm CreditAttribution: khiminrm commented

Subscribe

dmenefee’s picture

dmenefee CreditAttribution: dmenefee commented

The feeds_source_clear record is inserted whenever I delete blog entries that I imported. It appears that the delete functionality isn't invoking the job scheduler properly, and therefore isn't removing the record.

tuccio’s picture

tuccio CreditAttribution: tuccio commented
Version: 7.x-2.0-alpha4 » 7.x-2.0-alpha5

I am experiencing the same issue as #7 and #9:

A feeds_source_clear job shows as scheduled, and at the same time the imported entities get deleted I see a log entry stating:

Notice: Undefined variable: job in FeedsSource->scheduleClear() (line 319 of /var/www/html/sites/all/modules/feeds/includes/FeedsSource.inc).

I believe it all started when I tried to delete part of a failed large import while the "Process in background" option was selected.

Feeds is version 7.x-2.0-alpha5 (with Commerce Feeds 7.x-1.3)

tuccio’s picture

tuccio CreditAttribution: tuccio commented

It appears this is a known bug that was fixed in dev: http://drupal.org/node/1690434

dgtlmoon’s picture

dgtlmoon CreditAttribution: dgtlmoon commented

can you retest with latest dev?

tramm’s picture

tramm CreditAttribution: tramm commented

I'm using latest dev from 2012-Dec-27 and still having the same issue that nodes are deleted despite I set them not to be deleted. I am importing RSS/Atom feed with some interesting feed parameters. I confirm that I have done something like #14 describes (starting deletion manually during something is importing in the background), however I don't know if that can be our culprit.

tramm’s picture

tramm CreditAttribution: tramm commented

I think I have "solved" my problem. It appears that the nodes are being expired according to their alleged creation time and not the time they were imported. If I import an RSS, it has items with creation times several months ago and these items were expired during the first expiration round which is usually one hour after import. So not immediately, which is kind of confusing.

I patched Feeds importer for my use replacing n.created with fi.imported in expiring function of FeedsNodeProcessor.inc and it seems to work as expected.

twistor’s picture

twistor CreditAttribution: twistor commented
Status: Active » Closed (duplicate)

This is a duplicate of #1690434: Notice: Undefined variable: job in FeedsSource->scheduleClear() (line 319 of sites/all/modules/feeds/includes/FeedsSource.inc).

@tramm, The expected behavior when dealing with nodes is to use the created field. So you are correct.