Batch API and hook_cron

I managed to do this a few weeks ago. It's pretty simple and straightforward once you know how to do it.

This is how I did it:
Attention: This is done using the drupal_queue module for Drupal 6. For Drupal 7, the queue part is similar, but with a slight difference when actually setting the queue up. Dunno if there's a difference for the Batch part.

Take the actual logic out of the batch and place it in separate functions. All the batch then does, is be a wrapper around these functions. Your code will now look like this:

<?php
/**
 * BATCH
 */

// Batch definition
function mymodule_mybatch() {
  $lotsa_data = mymodule_logic_get_data();

  foreach ($lotsa_data as $one_data) {
    $operations[] = array('mymodule_mybatch_first_operation', array($one_data));
    $operations[] = array('mymodule_mybatch_second_operation', array());
  }

  $batch = array(
    'operations' => $operations,
    'title' => t('My batch'),
    'init_message' => t('Initializing'),
    'error_message' => t('Whoopseedaisy'),
    'finished' => 'mymodule_mybatch_finished',
  );

  batch_set($batch);
}



// Batch callbacks
function mymodule_mybatch_first_operation($data, &$context) {
  $my_variable = mymodule_logic_function_one($data);
  $context['message'] = t('First operation');
  $context['sandbox']['my_stuff'] = $my_variable;
}



function mymodule_mybatch_second_operation(&$context) {
  $my_variable = $context['sandbox']['my_stuff'];
  mymodule_logic_function_two($my_variable);
  $context['message'] = t('Second operation');
}



// Batch finish callback
function mymodule_mybatch_finished($success, $results, $operations) {
  $message = t('All done!');
  drupal_set_message($message);
}



/**
 * ACTUAL LOGIC
 */
function mymodule_logic_get_data() {
  // Get some data
  // Return some data
}



function mymodule_logic_function_one($data) {
  // Do some stuff
  // Return something
}



function mymodule_logic_function_two($my_variable) {
  // Do some stuff
}
?>

1. Make wrapper functions around the logic functions you separated, similar to the operation callbacks for the batch. For a queue, they're worker callbacks, and always take the parameter $item (which is the queue item that's being handled).
    
2. Define your queues using hook_cron_queue_info().

   Since queues can't exchange data the way the Batch API does (no $context equivalent), and since you can't assign worker callback results to $item for another queue (defining all $items are done before the queues are run, not while they're running. This should actually be possible if you experiment a bit, but this way's just simpler), put all operations for a single item in one worker callback. You could probably get around this in a couple of ways, but due to the Queue API's nature, it doesn't guarantee stability.

   If you have several operations that don't have anything to do with the results of the previous one, you can set up separates queues for them.
    

3. Finally, actually set your queue up. You can do this in a separate function and call it in hook_cron. For the sake of not cluttering everything up, I'm doing this in hook_cron directly.

 
This code will look like this:

<?php
/**
 * QUEUE
 */

// Queue definition -- implementation of hook_queue_info()
function mymodule_myqueue_queue_info() {
  $queue['myqueue'] = array('worker callback' => 'mymodule_myqueue_worker_one');
  return $queue;
}



// Worker callbacks
function mymodule_myqueue_worker_one($item) {
  $my_variable = mymodule_logic_function_one($item);
  mymodule_logic_function_two($my_variable);
}



// Implementation of hook_cron().
function mymodule_cron() {
  // Create the actual queue
  $myqueue = drupal_queue_get('myqueue');
  $myqueue->createQueue();

  // Get all data for your queue
  $lotsa_data = mymodule_logic_get_data();

  // Fill the queue
  foreach ($lotsa_data as $one_data) {
    $myqueue->createItem($one_data);
  }

  // Run the queue
  drupal_queue_cron_run();
}
?>

 
That's it! All done. Depending on what you need to do, I suggest also implementing a way that checks whether or not an item's been handled already. That way, you won't endlessly queue the same things up for no good reason. If you always need to handle the same single file completely from beginning to end, then you don't need to.