Drupal 7 port
rapsli - October 1, 2009 - 06:47
| Project: | Fast Gallery |
| Version: | 6.x-4.0-beta7 |
| Component: | Code |
| Category: | task |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Jump to:
Description
My plans for the Drupal 7 version:
http://www.rapsli.ch/drupal/drupal/fast-gallery-architecture-next-version
Looking for ideas and inputs
#1
Modular is good...
Data structure suggestion: (Example Location module). In Location they have a Location table and Location_Instance. The former keeps all unique locations (LID, L_name, L_address, L_GPS, etc), the latter keeps track of all the places where each location might be used, in node instances, users, etc. The secondary table presently used in Fast Gallery keeps multiple rows for each image (for the different exif fields), which is not really best DB practices, because that information is unique to each image so it should be kept right with the directory information. Also, there is no reason to use multiple rows; merely encode into a single field (or at most maybe two or three fields, like GPS, Keys, EXIF, IPTC, etc.)
(1) I rewired the present fast gallery with very minor changes to fast_gallery.install and fast_gallery.module. I put all of the image metadata into the main FG table. I used JSON encoding, because PHP has convenient functions to encode/decode JSON. One could use XML also, I'm sure, but that is a little more "wordy" than JSON. Other Drupal modules use JSON encoding. So I keep all the EXIF/IPTC/GPS data in one encoded field in the same table as the directory, path, image_name.
As I said, very minor changes to the install and module. I even left the old FG_EXIF table so I didn't have to mess with any other fast_gallery code.
(2) Following the Location Module strategy, the present EXIF table would be thrown away. Instead, that table would keep track of the different places a particular image is used, with flags indicating NID or VID (Node/Revision), LID (Location id for GPS), VID1 & VID2 (Vocabulary ids for keys), UID (attaching to a user?), UPC_ID (attaching to Ubercart). When you erase an image from the folder and rescan the DB, the second table indicates all the dependent usages so you could delete associated nodes, locations, vocabulary terms, etc.
I built the second table, but I didn't do any coding to use it, mainly because it is important to think through the hazards and possibilities. For my purposes, I like the ability of Fast Gallery to build all the galleries out of folders of images, and I might be able to live with a bulk delete cascading through my nodes and Locations.
But, what if you wanted to protect an image from deletion? For example, you might have a gallery where people have commented or rated the images. Put a protect flag in the Image Instance table, to keep from deletion. Oops, now we have to be clever about removing files; can't just delete a directory,r ftp a replacement and run rescan. Maybe when rescan discovers the missing file, it can save an imagecache image as a replacement for the original.
Using the second table would enable Fast_Gallery to use nodes, locations, ubercart, and bring in some of the features from Node_Gallery.
http://photokinesis.us
#2
How to use SIDECAR FILES instead of Image EXIF Metatags
I've found the PHP exif functions to work poorly. I never got fast_gallery to read the exif metadata without errors. On the other hand EXIFTOOL is very robust and well-maintained. Plus, I wanted to use the IPTC fields in my image to store keywords and photographer information. So I use exiftool to dump all the image metadata, including camera, IPTC and GPS to a sidecar file. I use JSON encoding which is easy enough to read in PHP.
I had to modify some code in the updateExifDb function in fast_gallery.class.php to read the JSON instead of PHP's EXIF function. (Actually, I rewired the main fast_gallery table to hold the JSON data, but I retained the fg_exif table so that all the regular Fast Gallery code continued to work.
My procedure is to ftp all the image.jpg and image.jpg.json files to the FG image folder. The extra files are not much more additional hassle once I've scripted exiftool to create the JSON files. I'm not sure if I can get EXIFTOOL to run on my hosting, which might be a better mechanism.
Here is a little unix script that dumps the sidecars for a directory full of images:
find . -name "*.jpg" -print | while read filenamedo
exiftool -json -exif:ALL -GPSLatitude -GPSLongitude -c "%.6f" -FocalLengthIn35mmFormat -XMP-photoshop:ALL -iptc:ALL -nikon "${filename}" > "${filename}.json"
done;
See the EXIFTOOL man page for hundreds of options. EXIFTOOL dumps uniquely, which is nice. I didn't dump all metatags, because I mainly care about camera, GPS and IPTC. The few tricky flags above were necessary to get just what I wanted.
Here is the main code modification to the updateExifDb function of fastgallery.class.php necessary for reading in a JSON encoded field:
public function updateExifDb($file, $gid) {
// Get the file extension
$tmp = explode('.', $file);
$fileType = strtolower(end($tmp));
if ($fileType != 'jpg' && $fileType != 'jpeg') {
// Construct the custom thumbnail path
$tmp_ar = explode('.', $file);
array_pop($tmp_ar);
$tmp = implode('.', $tmp_ar);
$file = $tmp . '.screenshot.jpg';
}
// Grab the EXIF data that we're interested in
// Sidecar JSON created from exiftool:
// exiftool -json -iptc:ALL -XMP-photoshop:ALL -nikon -ModifyDate -FocalLengthIn35mmFormat -GPSLatitude -GPSLongitude -c "%.6f" photo.jpg > photo.jpg.json
// First look for sidecar; if no sidecar, continue with original EXIF data:
$file_json = $file . $file_json . ".json";
if (file_exists($file_json)) {
$fh = fopen($file_json, 'r');
$json = fread($fh, filesize($file_json));
fclose($fh);
//remove curly brackets to beware from regex errors
// strips outer json package brackets to return only the inside array.
$json = substr($json, strpos($json,'{')+1, strlen($json));
$json = substr($json, 0, strrpos($json,'}'));
$json = preg_replace('/(^|,)([\\s\\t]*)([^:]*) (([\\s\\t]*)):(([\\s\\t]*))/s', '$1"$3"$4:', trim($json));
// array for php to work with
$ar_json = json_decode('{'.$json.'}', true);
// var_dump($ar_json);
// If you prefer to work with the object:
$obj_json = (object) $ar_json;
// THE FOLLOWING code keys deals with hyphens so PHP can work with it.
// (hyphens (-) in IPCT codes crashes the php; similar problem with colons (:) ).
$byline = "By-line";
$bylinetitle = "By-lineTitle";
$captionabstract = "Caption-Abstract";
$provincestate = "Province-State";
$countryprimarylocationname = "Country-PrimaryLocationName";
$sublocation = "Sub-location";
// Wierd bug: gallery main display errors on next line if we don't have imagecache configured
// Keywords array goes into $keywords
// jsonread only creates arrays for more than one element, so we need to ensure we have an array.
// implode works fine for empty or single element arrays.
$keywords = $obj_json->Keywords;
if (!is_array($keywords)) { $keywords = array($keywords); }
$places = $obj_json->Contact;
if (!is_array($places)) { $places = array($places); }
$json_keywords = implode (", ", $keywords );
$json_places = implode (", ", $places );
if ($obj_json->ObjectName == '') {
$obj_json->ObjectName = $gid . " " . $filename;
}
// Over-write normal fast_callery EXIF array with sidecar-json data:
$param['model'] = $obj_json->Model;
$param['exposureTime'] = $obj_json->ShutterSpeed;
$param['dateTaken'] = $obj_json->SubSecDateTimeOriginal;
...
http://photokinesis.us
#3
The big benefit of fast gallery is the ability to ftp a large number of files, and automatically present them in galleries with imagecache creating the proper sizes of thumbnails and images. It is really clever and convenient to use the directory structure to define galleries, and to use an image called "folder", to provide the gallery thumbnail.
I've been thinking about how great it would be to extend this ability to insert, delete and update a large number of NODES, USERS, VOCABULARIES, LOCATIONS, PRODUCTS, etc. As I said before, mass deletions need to cascade through dependencies, or else, the dependency needs to protect against deletion. I think the only way to do that is make a copy of the image & metadata when it is used somewhere else so that fast_gallery could delete the files in the directory unhindered. Then the Protect Flag could be set to allow the deletion to cascade or prevent it from deleting downstream.
For example: Massively create a bunch of nodes. Using Imagefield, the simplest node is Title, Body and Image. Without addressing the delete issue, Fast_Gallery could easily populate a large number of nodes by mapping Title and Comment to Node:Title and Node:Body. Or, you could dump any or all of the other exif fields into separate divs in Body. With a little more effort, the fast gallery file structure could map into Gallery and Image Nodes, like Node Gallery.
Extending the use of metadata is the key step for applying Fast Gallery updates to other modules. IPTC is well utilized by journalists, including keywords, locations, photographer, Event/Title/Headline/Caption, Categories (these are normally encoded to a standard, topic/subject taxonomy, but if you aren't a Journalist you could invent your own meanings, I guess).
(If you want to use all those other metadata you see why it is so important to change the present data structure that puts the EXIF data in multiple rows for each image. With thousands of images, your searches are so much faster if you keep the metadata in the same row in the "Image" with the image file data.)
Minimal implementations for using Fast Gallery with other Drupal modules:
- Node: Title, Body, image.
- User: Username, (email?), image.
- Location: Name, GPS Latitude, GPS Longitude (with Node)
- Vocabulary: Vname, Tags (with Node). (IPTC defines at least three vocabularies: Keywords, People, maybe Location)
- Ubercart: Product code, price, product category (Also probably various attributes: size, catalog, email, etc)
- Organic Groups: (I don't know)
What does this have to do with your new Architecture?
Mainly, you need to make sure the new modular organization and data structure don't make these extensions difficult. Specifically, I'm arguing that you need to leave space for a big EXIF/IPTC field in your main "Image" table. I'm suggesting you probably need to create a secondary "Image Instance" table to (at least) handle the dependencies for major Drupal modules like Node, User, Vocabulary, Location. If the add-on modules you are thinking of can depend on a common data structure, then Fast Gallery can be extended without needing to rewrite the base tables. (Which I had to do to store full EXIF/IPTC metadata.)
Secondly, I think you need to retain some minimal EXIF data in your base "Image" table. Because the most immediate benefit is extending Fast Gallery to Nodes or Users, you need at least Title, Body and Username fields, even if they are populated by something as simple as the image file name and the username of the Owner or assigned Owner.
#4
@tstermitz: can you provide a schema of how you see the db structure?
I'm only a couple days away of a first running D7 Version that includes only the very basic -> displaying images. I think though, this would be a good start to see if we are going into the right direction. I'll commit it to head, as soons as it's running, then we can see where we are going from there on.
#5
committed an initial version to cvs head.