ZoomifyFileProcessor.php fails on large images.

sgriffin - April 30, 2009 - 19:34
Project:Zoomify
Version:6.x-1.x-dev
Component:Code
Category:feature request
Priority:normal
Assigned:kratib
Status:closed
Description

What might cause this and or what might be a work around?
Fatal error: Allowed memory size of 100663296 bytes exhausted (tried to allocate 30700 bytes) in /home/xxx/public_html/sites/all/modules/zoomify/ZoomifyFileProcessor.php on line 110

#1

sgriffin - April 30, 2009 - 19:46

Perhaps a standalone system application would work better. I am researching how to use the linux binary of http://krpano.com/tools/kmaketiles/

#2

kratib - April 30, 2009 - 22:43

I just copied the ZoomifyFileProcessor file as is from its original creator. Can you link to the large image that caused the error.
I think it would be a good idea to allow people to plug their own tiling backend, because each one comes with pros and cons. That's probably the next step for Zoomify.

#3

sgriffin - April 30, 2009 - 22:47

I think I'm going to just thrown in the towel on this after some research. For regular photos, I think its probably fine, but it's going to be not workable for the type of images I want to use.
Looking at the processing script, it seems everything is native php. I think it'd probably a bit more robust if everything was done in image magick, but further research shows that may not work well also near and above gigapixel images.

#4

sgriffin - May 1, 2009 - 11:27

You can try this..

But its a fairly small example in place for the images I intend on using later.

#5

kratib - May 1, 2009 - 01:24

Can you please try ZoomifyImage ? That's how zoomify.module used to convert but I opted for a pure PHP solution to reduce dependencies. Maybe not the best decision in retrospect... But in any case, the whole point of Zoomify is to handle large images so I'd like to resolve this and I will appreciate your help doing so :-)

#6

kratib - May 1, 2009 - 01:31
Assigned to:Anonymous» kratib

The image that you pointed to was successfully converted on my test machine with a PHP memory limit of 64MB. Did this one fail for you? What was your memory setting?

#7

sgriffin - May 1, 2009 - 11:26

Mine is set for 96M.. yes, that was the one that failed.
If you check the site, do not think I have it working, I only have hard coded the path to manually created tiles.

I will try the other way again, I think I was missing the PIL dependency part. I will fix that and get back to you.

#8

kratib - May 1, 2009 - 17:42

The other way (using Python) has been removed from the code. I meant trying it manually to see if it works for you. If it does, I will add to Zoomify the ability to select which backend (PHP or Python) to use for image tiling.

#9

sgriffin - May 2, 2009 - 13:17

I got the other process running, it took about a minute for that particular file on CLI.

What considerations are their for switching processing scripts?

I think using python, could be a great idea, but it does require some very basic linux knowledge to do the old school ./configure make etc for the 4 required software elements.

#10

kratib - May 2, 2009 - 17:26

I'll work on re-enabling that Python code over the weekend. On modern Linux systems I don't think it's necessary to compile anything, because most package managers have the necessary Python + PIL packages. The only problem is that if images can take 1 min or more to process, then it might be a good idea to process them in the background. More code will be needed to account for this. I will start with a basic online approach (where the page waits until the image is processed) then I will move to the more complex background processing approach.

#11

kratib - May 3, 2009 - 00:29

I just committed support for ZoomifyImage as a backend. Please download the latest dev release (12 hours from now) and select the new backend from the Zoomify settings page. Let me know if it works better for you.

#12

sgriffin - May 4, 2009 - 02:32

Hmm I don't think it worked as is for me. The python part, so I installed/compiled
freetype-2.3.9.tar.gz |
jpegsrc.v6b.tar.gz |
zlib-1.2.3.tar.gz |
Imaging-1.1.6.tar.gz to get it working.

In the module I get...

Traceback (most recent call last):
File "/home/pensacol/public_html/sites/all/modules/zoomify/ZoomifyImage/ZoomifyFileProcessor.py", line 20, in ?
import PIL.Image
ImportError: No module named PIL.Image

but I can run "python /home/pensacol/public_html/sites/all/modules/zoomify/ZoomifyImage/ZoomifyFileProcessor.py forsyth.jpg"
successfully.

and I can run the exec( $cmd from root.. just fine.
I believe it must be a path or permission issue. Everything is 755 and in /usr/local/lib/python2.4/site-packages/PIL

#13

sgriffin - May 4, 2009 - 02:59

bingo
dunno what happened but I specified the path to python /usr/local/bin/python and it seems to be running fine.
Not sure what changed recently in my php but it no longer has access to the system path.
Perhaps a path to python would be nice? and maybe a tick to remove the zoomify tab if you use it as the default view, I wonder how/what it was running a python before?

This was generated automatically in a couple minutes
http://pensacolabedbreakfast.com/photography/still-photos/forsyth-park-s...
10376x6285

Great work!

#14

sgriffin - May 4, 2009 - 02:59
Status:active» fixed

#15

kratib - May 4, 2009 - 03:45

Concerning Python, I'll hold off on accessing it through a path because this seems to be an installation issue on your side.

A setting to remove the Zoomify tab has been added. It will be available in the next dev release.

I tried to access the URL you sent but the Zoomify applet showed "Error loading: http://pensacolabedbreakfast.com/system/files/zoomify/552/693".

#16

sgriffin - May 4, 2009 - 13:41

I see my $_env[path] is /sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin which is different than my system path. I'm not sure where this is set, but I know it changed recently to drop /usr/local/bin/
I linked the updated version of python and everything is normal now without my edit to hardcode the path.

I didn't see any permissions for zoomify. Now, I'll have to hunt for which one it maybe and or change to basepath out of private to a public place.
I really wish every module didn't take private file system as dogma, but gave you an option. Its a major performance hit when you need it to try to protect something and everything is affected.

#17

sgriffin - May 4, 2009 - 14:41

I don't think this module is setup to work with a private file system. It only works as user 0.
I don't have the upload module enabled.
Could this be set to output somewhere inside the webroot?
Setting the basepath inside, created corrected the zoomify directory but subsequently errored out with a can not create path (missing output of %path).

Working on seeing if I can alter this.

#18

kratib - May 4, 2009 - 15:10

So you're saying that private paths are not supported by Zoomify? If so, please open a new issue to this effect.

#19

System Message - May 18, 2009 - 15:20
Status:fixed» closed

Automatically closed -- issue fixed for 2 weeks with no activity.

#20

floridag - June 9, 2009 - 14:20

Thanks for the continued work on this module I really like it.

I'm having similar problems but I think it may be related to my shared hosting account on GoDaddy, I'm going to move off GoDaddy once the site is complete but for now I'm thinking it's too much to pre-process the files up there. I tried the new python pre-processor but it still seems to time out. I'm really not going to allow visitors to the site to upload these large images and I'm wondering if maybe I should just process the files locally and then upload them when their finished.

#21

kratib - June 9, 2009 - 15:19

Please open a new feature request to allow people to upload their own tiles instead of letting Zoomify do it.

#22

floridag - June 9, 2009 - 17:50
Category:bug report» feature request
Priority:critical» normal
Status:closed» active

It would be nice if Zoomify could support the offline pre-processing of the images and then have the tiles uploaded into the site. This would work around processing constraints on shared host systems like GoDaddy.

#23

floridag - June 9, 2009 - 17:56
Status:active» closed

Opened this as a separate issue.

 
 

Drupal is a registered trademark of Dries Buytaert.