Because ZIP creation and manipulation functionality is not implemented into PHP for most people, I found a method of creating zips with a simple zip class that should work on any platform.

Here is the 'zip.inc' file (taken from the phpMyadmin project) that you should place in the includes directory (optionally, if you don't want users to add files into the includes directory, you can save it in your module, save it in the same folder as your module, etc.):

/* $Id: zip.lib.php,v 2.4 2004/11/03 13:56:52 garvinhicking Exp $ */
// vim: expandtab sw=4 ts=4 sts=4:
/**
 * Zip file creation class.
 * Makes zip files.
 *
 * Based on :
 *
 *  http://www.zend.com/codex.php?id=535&single=1
 *  By Eric Mueller <eric@themepark.com>
 *
 *  http://www.zend.com/codex.php?id=470&single=1
 *  by Denis125 <webmaster@atlant.ru>
 *
 *  a patch from Peter Listiak <mlady@users.sourceforge.net> for last modified
 *  date and time of the compressed file
 *
 * Official ZIP file format: http://www.pkware.com/appnote.txt
 *
 * @access  public
 */
class zipfile
{
    /**
     * Array to store compressed data
     *
     * @var  array    $datasec
     */
    var $datasec      = array();

    /**
     * Central directory
     *
     * @var  array    $ctrl_dir
     */
    var $ctrl_dir     = array();

    /**
     * End of central directory record
     *
     * @var  string   $eof_ctrl_dir
     */
    var $eof_ctrl_dir = "\x50\x4b\x05\x06\x00\x00\x00\x00";

    /**
     * Last offset position
     *
     * @var  integer  $old_offset
     */
    var $old_offset   = 0;


    /**
     * Converts an Unix timestamp to a four byte DOS date and time format (date
     * in high two bytes, time in low two bytes allowing magnitude comparison).
     *
     * @param  integer  the current Unix timestamp
     *
     * @return integer  the current date in a four byte DOS format
     *
     * @access private
     */
    function unix2DosTime($unixtime = 0) {
        $timearray = ($unixtime == 0) ? getdate() : getdate($unixtime);

        if ($timearray['year'] < 1980) {
            $timearray['year']    = 1980;
            $timearray['mon']     = 1;
            $timearray['mday']    = 1;
            $timearray['hours']   = 0;
            $timearray['minutes'] = 0;
            $timearray['seconds'] = 0;
        } // end if

        return (($timearray['year'] - 1980) << 25) | ($timearray['mon'] << 21) | ($timearray['mday'] << 16) |
                ($timearray['hours'] << 11) | ($timearray['minutes'] << 5) | ($timearray['seconds'] >> 1);
    } // end of the 'unix2DosTime()' method


    /**
     * Adds "file" to archive
     *
     * @param  string   file contents
     * @param  string   name of the file in the archive (may contains the path)
     * @param  integer  the current timestamp
     *
     * @access public
     */
    function addFile($data, $name, $time = 0)
    {
        $name     = str_replace('\\', '/', $name);

        $dtime    = dechex($this->unix2DosTime($time));
        $hexdtime = '\x' . $dtime[6] . $dtime[7]
                  . '\x' . $dtime[4] . $dtime[5]
                  . '\x' . $dtime[2] . $dtime[3]
                  . '\x' . $dtime[0] . $dtime[1];
        eval('$hexdtime = "' . $hexdtime . '";');

        $fr   = "\x50\x4b\x03\x04";
        $fr   .= "\x14\x00";            // ver needed to extract
        $fr   .= "\x00\x00";            // gen purpose bit flag
        $fr   .= "\x08\x00";            // compression method
        $fr   .= $hexdtime;             // last mod time and date

        // "local file header" segment
        $unc_len = strlen($data);
        $crc     = crc32($data);
        $zdata   = gzcompress($data);
        $zdata   = substr(substr($zdata, 0, strlen($zdata) - 4), 2); // fix crc bug
        $c_len   = strlen($zdata);
        $fr      .= pack('V', $crc);             // crc32
        $fr      .= pack('V', $c_len);           // compressed filesize
        $fr      .= pack('V', $unc_len);         // uncompressed filesize
        $fr      .= pack('v', strlen($name));    // length of filename
        $fr      .= pack('v', 0);                // extra field length
        $fr      .= $name;

        // "file data" segment
        $fr .= $zdata;

        // "data descriptor" segment (optional but necessary if archive is not
        // served as file)
        // nijel(2004-10-19): this seems not to be needed at all and causes
        // problems in some cases (bug #1037737)
        //$fr .= pack('V', $crc);                 // crc32
        //$fr .= pack('V', $c_len);               // compressed filesize
        //$fr .= pack('V', $unc_len);             // uncompressed filesize

        // add this entry to array
        $this -> datasec[] = $fr;

        // now add to central directory record
        $cdrec = "\x50\x4b\x01\x02";
        $cdrec .= "\x00\x00";                // version made by
        $cdrec .= "\x14\x00";                // version needed to extract
        $cdrec .= "\x00\x00";                // gen purpose bit flag
        $cdrec .= "\x08\x00";                // compression method
        $cdrec .= $hexdtime;                 // last mod time & date
        $cdrec .= pack('V', $crc);           // crc32
        $cdrec .= pack('V', $c_len);         // compressed filesize
        $cdrec .= pack('V', $unc_len);       // uncompressed filesize
        $cdrec .= pack('v', strlen($name) ); // length of filename
        $cdrec .= pack('v', 0 );             // extra field length
        $cdrec .= pack('v', 0 );             // file comment length
        $cdrec .= pack('v', 0 );             // disk number start
        $cdrec .= pack('v', 0 );             // internal file attributes
        $cdrec .= pack('V', 32 );            // external file attributes - 'archive' bit set

        $cdrec .= pack('V', $this -> old_offset ); // relative offset of local header
        $this -> old_offset += strlen($fr);

        $cdrec .= $name;

        // optional extra field, file comment goes here
        // save to central directory
        $this -> ctrl_dir[] = $cdrec;
    } // end of the 'addFile()' method


    /**
     * Dumps out file
     *
     * @return  string  the zipped file
     *
     * @access public
     */
    function file()
    {
        $data    = implode('', $this -> datasec);
        $ctrldir = implode('', $this -> ctrl_dir);

        return
            $data .
            $ctrldir .
            $this -> eof_ctrl_dir .
            pack('v', sizeof($this -> ctrl_dir)) .  // total # of entries "on this disk"
            pack('v', sizeof($this -> ctrl_dir)) .  // total # of entries overall
            pack('V', strlen($ctrldir)) .           // size of central dir
            pack('V', strlen($data)) .              // offset to start of central dir
            "\x00\x00";                             // .zip file comment length
    } // end of the 'file()' method

} // end of the 'zipfile' class

Then, in your module, you can do the following:

require_once("./includes/zip.inc");
$zip = new zipfile();
$files_to_zip = array('mypic.jpg', 'mydoc.doc', 'readme.txt');
foreach ($files_to_zip as $filename) {
	$zip->addFile(file_get_contents(file_create_path($filename)), $filename); // the second parameter specifies the filename in the zip
}
if (!file_save_data($zip->file(), file_create_path('test.zip'))) {
	drupal_set_message(t('The zip could not be created.'), 'error');
}

You can also write the file_save_data function into the class or add more functions that can improve the class or offer more standardized functions.

Michael

Comments

Michael M’s picture

Before using the zipping functionality, check to see if the gzcompress function exists:

if (@function_exists('gzcompress')) {
  // include, process, etc.
}

----
http://eUploads.com

JamieR’s picture

Thanks for posting this - I've implemented it into my fileshare module where it allows the user to compress and download a directory. It works fine for smaller files, but for larger ones it becomes a memory problem. I'm considering trying to re-write the phpmyadmin class to support streaming to disk but am not sure if it's really possible... See below for how I'm using the class. I'm allocating a ridiculous amount of memory to php for now and I'm not happy about it. :)

PLEASE HELP!

Thanks - Jamie.

/**
 * Zip and transfer a directory of files
 * taken from Michael M's post http://drupal.org/node/83253
 *
 * @param $source Directory to zip and download
 * @return FALSE on failure
 */
function _fileshare_zip_transfer($source) {
  ini_set('display_errors', TRUE);
  ini_set('memory_limit', '500M');
  set_time_limit(600);
  $size_limit = 50000000; //50MB
  
  ob_end_clean();
  $to_zip = _recursive_readdir($source);
  $dir_size = $to_zip['size'];
  unset($to_zip['size']);
  if ($dir_size > $size_limit) { // greater than 50MB
    drupal_set_message('Sorry, '.basename($source).' is '._resize_bytes($dir_size).'. Greater than the '._resize_bytes($size_limit).' file limit.');
    return FALSE;
  } else {
    $zip = new zipfile();
    foreach ($to_zip as $path) {
      // the second parameter specifies the filename in the zip
      $data = file_get_contents($path);
      //$data_len += strlen($data);
      $zip->addFile($data, str_replace($source, '', $path));
    }
    $filename = _sanitize_filename(basename($source)).'.zip';
    $filetype = mime_content_type($source);
    $filesize = $data_len*1.25;

    $name = mime_header_encode($filename);
    $headers = array (
      'Content-Type: application/x-zip; name='. $name,
      //'Content-Length: '. $dir_size, // this needs to be exact to work...
      'Content-Disposition: attachment; filename='. $name
    );
    foreach ($headers as $header) {
      // To prevent HTTP header injection, we delete new lines that are
      // not followed by a space or a tab.
      // See http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2
      $header = preg_replace('/\r?\n(?!\t| )/', '', $header);
      header($header);
    }
    print $zip->file();
    exit();
  }
}

/**
 * Recursively reads a directory
 *
 * @param $filepath directory to read
 * @return $file_array an array of file paths
 *         $file_array['size'] contains totaly directory size
 */
function _recursive_readdir($filepath) {
  $file_array = array();
  // Read directory
  $handle = opendir($filepath);
  while (false !== ($file = readdir($handle))) {
    if ($file != "." && $file != "..") {
      if (is_dir($filepath.'/'.$file)) {
        //recursive read directory and append array values
        $return_array = _recursive_readdir($filepath.'/'.$file);
        $file_array['size'] += $return_array['size'];
        unset($return_array['size']);
        $file_array = array_merge($file_array, $return_array);
      } else {
        $file_array['size'] += filesize($filepath.'/'.$file);
        $file_array[] = $filepath.'/'.$file;
      }
    }
  }
  closedir($handle);
  return $file_array;
}
Michael M’s picture

Jamie,

When exactly does there seem to be a memory problem: when the ZIP file in memory becomes too large OR when you add files that are too large?

I will have to test this with large files or a large number of files.

What kind of memory issues are you experiencing? How do you know that there are memory problems (your development PC is hanging, the memory used by PHP sky-rockets, etc.)? Is it a problem when while the file is transfered to the users OR is the memory problem with the creation of the ZIP file? It looks like you are creating a ZIP in memory and then outputting it for download. Maybe you can try to save the zip as a file and forward the user to the file location?

What exactly do you mean "streaming to disk"?

Just brain storming...

----
http://eUploads.com

JamieR’s picture

Hi thanks for the reply!

It's php's memory limit that I'm hitting. I'm setting php's memory temporarily to 500 megs: ini_set('memory_limit', '500M');

And it's working, but it's got our IT guy freaked out. And since I want to include the functionality in the standard release of the fileshare module I would like it to work on most webservers... and I doubt a shared webserver would allow that type of memory allocation.

So to be able to keep php's memory limit set at something like 16M we'll have to read the file in and stream it to disk... or read it in in small bits and use the disk as a cache... then have the user download the temp file and remove the file after the download...

I hope that helps clear things up! :)
Let me know what you think. Thanks - Jamie.

drbitboy’s picture

The following changes allow using zip.lib.php (renamed zipstream...) to write the .zip to stdout on the fly so the data sections of the .zip are not saved in memory, which should greatly reduce the memory requirement of this code for large .zip files.

The only thing you need to add is a call to the ->setDoWrite() method of the zipfile class *immediately* after the creation of the class and before adding any files e.g.

require zipstream.lib.php   // zip.lib.php replacement
  $zf=new zipfile;
  $zf->setDoWrite();
  for ($filenames as $fn) {
    $zf->addfile( file_get_contents($fn), $fn);
  }
  echo $zf->file();   // Echo not required after ->setDoWrite() as ->file() returns empty string

Here are the changes:

2c2
< /* $Id: zip.lib.php,v 2.4 2004/11/03 13:56:52 garvinhicking Exp $ */
---
> /* $Id: zipstream.lib.php,v 2.4 2004/11/03 13:56:52 garvinhicking Exp $ */
27a28,34
>      * Whether to write zip file via echo in one pass and have
>      * have -> file return an empty string, or to save all info 
>      * and have -> file() return entire zip file in a string
>      */
>     var $doWrite = false;
> 
>     /**
55a63,66
>     function setDoWrite() {
>       $this -> doWrite = true;
>     }
> 
96c107
<         $dtime    = dechex($this->unix2DosTime($time));
---
>         $dtime    = substr( "00000000" . dechex($this->unix2DosTime($time)), -8);
133,134c144,149
<         // add this entry to array
<         $this -> datasec[] = $fr;
---
>         // write this entry on the fly, ...
>         if ( $this -> doWrite) {
>           echo $fr;
>         } else {                     // ... OR add this entry to array
>           $this -> datasec[] = $fr;
>         }
165c180
<      * Dumps out file
---
>      * Dumps out file to stdout if ->doWrite==true, else to return value as string
167c182,183
<      * @return  string  the zipped file
---
>      * @return  string  if ->doWrite ==false: the zipped file 
>      *                  else the empty string
173,184c189,212
<         $data    = implode('', $this -> datasec);
<         $ctrldir = implode('', $this -> ctrl_dir);
< 
<         return
<             $data .
<             $ctrldir .
<             $this -> eof_ctrl_dir .
<             pack('v', sizeof($this -> ctrl_dir)) .  // total # of entries "on this disk"
<             pack('v', sizeof($this -> ctrl_dir)) .  // total # of entries overall
<             pack('V', strlen($ctrldir)) .           // size of central dir
<             pack('V', strlen($data)) .              // offset to start of central dir
<             "\x00\x00";                             // .zip file comment length
---
>         if ( $this -> doWrite ) {
>           $ctrldir = implode('', $this -> ctrl_dir);
>           echo $ctrldir;
>           echo $this -> eof_ctrl_dir;
>           echo pack('v', sizeof($this -> ctrl_dir));   // total # of entries "on this disk"
>           echo pack('v', sizeof($this -> ctrl_dir));   // total # of entries overall
>           echo pack('V', strlen($ctrldir));            // size of central dir
>           echo pack('V', $this -> old_offset);         // offset to start of central dir
>           echo "\x00\x00";                             // .zip file comment length
>           return "";
>         } else {
>           $data    = implode('', $this -> datasec);
>           $ctrldir = implode('', $this -> ctrl_dir);
> 
>           return
>               $data .
>               $ctrldir .
>               $this -> eof_ctrl_dir .
>               pack('v', sizeof($this -> ctrl_dir)) .  // total # of entries "on this disk"
>               pack('v', sizeof($this -> ctrl_dir)) .  // total # of entries overall
>               pack('V', strlen($ctrldir)) .           // size of central dir
>               pack('V', strlen($data)) .              // offset to start of central dir
>               "\x00\x00";                             // .zip file comment length
>         }
neillfontes’s picture

Hi drbitboy,

I tried using your solution but I got no luck. I got the file printed on the screen instead of creating a compressed file. Can you provide a sample of its implementation/source code or similar?

I'm having the very same struggle to not blow up the memory during zip file creation.

Thanks,

Neill

mauro_ptt’s picture

Thanks Michael! I'll be trying this out, really really useful!!! thanks a lot