Last updated July 10, 2012. Created by juampy on July 10, 2012.
Log in to edit this page.
Feeds processes data using the Batch API. You can avoid hitting a page time-out when processing large amounts of data by keeping count of the processed elements and setting the batch progress.
The simplest example of implementation can be found at feeds/includes/FeedsBatch.inc:
<?php
class YourFeedsProcessor extends FeedsProcessor {
// Total elements to process per page
const MAX_PER_PAGE = 50;
/**
* Update your process() method within your custom Feeds processor with the following logic.
*/
public function process(FeedsImportBatch $batch, FeedsSource $source) {
// Set counter of processed elements for this page load.
$processed = 0;
// Set counter of all processed items across page loads.
if (!isset($batch->processed)) {
$batch->processed = 0;
}
// Set total elements
if (!$batch->getTotal(FEEDS_PROCESSING)) {
$batch->setTotal(FEEDS_PROCESSING, count($batch->items));
}
// Loop items
while ($item = $batch->shiftItem()) {
// You can replace the following two lines by your custom logic to process nodes.
$object = $this->map($item);
$object->save();
$processed++; // Processed in this page load.
$batch->processed++; // Processed total.
if ($processed > self::MAX_PER_PAGE) {
$batch->setProgress(FEEDS_PROCESSING, $batch->processed);
return;
}
}
$batch->setProgress(FEEDS_PROCESSING, FEEDS_BATCH_COMPLETE);
?>The above code will make the page to reload every 50 items, thus increasing the progress bar and avoiding page time-outs.
Have a look at how this is implemented by the generic FeedsProcessor in Drupal 7 or by FeedsNodeProcessor in Drupal 6.