My client needed to get a large number of images into IMCE v1. So I hacked together a little import module based on the below blog post. It should also work with IMCE v2, see comments in blog post.

http://www.freelock.com/blog/john-locke/2010-02/using-file-field-importe...

The concept is simple. On install the module creates 3 directories in the Drupal files directory. One root directory and 2 sub directories, imce_import/upload & imce_import/complete.

I then created for my client a FTP account with the root/home of imce_import. The client uploads using FTP files into the upload directory. Then goes to the admin/content/imce-import path and selects the IMCE folder they want to import into and clicks run. FYI there is a new required permission "use imce import".

The files are copied to that location and added to the database and then when complete moves the source file from upload to complete.

Feel free to do with this little module as you wish, use it, hack it, create a project from it.
I hope somebody finds it use full.

Please note I have only tested this in v1 but in theory should work in v2.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

nateB’s picture

Great stuff. Hopefully I can get this moving towards D7 too.

NaX’s picture

It should be very easy to port D7 as it is a very simple module, but I personally will not be porting it to D7 as I currently have no need for a D7 port. If this changes I will post it here.

tragic.rich’s picture

Hey NaX,

Really awesome stuff! I have been reading through all the work you have done trying to get filefield_sources to work and now this small module, you are totally gonna save me on my current project! This module looks like exactly what I need but when I go to the imce-import page I get a WSOD :(
Could it be because I am using version 2? IMCE 6.x-2.2

NaX’s picture

FileSize
2.94 KB

Ok, I stole a few min to do a quick test on IMCE v2 and I see where the problem is. Please note that the most likely reason you are getting a WSOD is because your PHP setup is not displaying errors.

The problem was that IMCE renamed its include files from inc/page.inc to inc/imce.page.inc, something I did not pick-up on.

Please try the updated version 2 attached. This version tries to include the file using the IMCE v2 path and if that fails the version 1 path.

Another point to mention is that there is built-in transliteration support but should only work with version 2 and maybe 3, but also with limited testing. If you ever having a problem running the import I suggest you disable transliteration and see if that makes a difference.

I hope that helps.

freelock’s picture

Hi,

Thanks for commenting on my blog!

Just wondering -- have you tried out the new "File attach from server directory" option for filefield_sources? It's been added since I wrote that blog post, and seems to do essentially what it sounds like this issue is aiming to resolve...

Cheers,
John

nateB’s picture

John, the "Server Directory" thing almost works. But it lists a bazillion files, and doesn't give the option to not copy the file to another directory. In my case I'm looking at a pre-existing folder structure from Drupal 6 that I really just don't want to stop using in the migration to D7. The D6 version gives the options and I assume from what's written in the D7 module that it'll eventually get there.

This IMCE FTP import project is really interesting because it allows for a GUI way of managing those additions, and is something that people other than developers could use. I still love the efficiency of yours. If only there were a single button in IMCE to scour all directories in a tree and add certain file types to the database, I'd be the happiest camper at the summer camp for nerds.

I'm posting ads to Craigslist and the like to see if I can hire someone to get this to D7, it's just out of my scope of understanding.

NaX’s picture

@freelock
I worked on the new Attach Feature http://drupal.org/node/438940#comment-1763984
The problem is that it does not fully solve the problem if you are using IMCE. It is great if you are using a File/Image field. IMCE Import is more of a simple purpose built solution that does not require CCK.

@artproduct
Porting this to D7 should be fairly trivial as it is very simple little module. The only thing that could cause problems would be if IMCE D7 version has changed in some way. Before spending any money I would alter the info file to be core = 7.x and test and see what happens. There is also the coder module that is able to assist in porting by reading the code of the module. You could run it through the coder module and see what it outputs without being a developer. Lastly you could offer a small bounty here or in the forums. I will ask here at our office if anybody else has some free time, but we are fairly busy at the moment, so don't count on it.

nateB’s picture

NaX, I appreciate all the tips. The Coder module is entirely new to me and something I'd love to read up on later. I'm not particularly strong in PHP, IA and UI are my skills. Visual stuff, you know. That's why getting image files imported is such a big deal to me.

I offer a bounty of $120, PayPal'd.

nateB’s picture

FileSize
6.33 KB

So I was able to get a near-working version of the IMCE Import for Drupal 7 with the help of a developer.

The creation of directories on install doesn't happen, but moving files and adding them to the database does. Manually adding the directories of the D6 version directly to the sites/nameofsite/files directory is the solution for now.

Anybody want to give it a try? Maybe give a few pointers?

dgcartersa’s picture

FileSize
3.15 KB

Hi there,

I have developed my own D7 port and completed it before #9 was posted. From my testing and comparing, my version address the install directory creation issue and does not error on install.

My version also tackles a bug where the select list on the IMCE Import page changes when a user selects a different active directory in the IMCE browser.

Let me know what you think.

nateB’s picture

Nice stuff, dgcartersa!

austintnacious’s picture

Cool stuff Ladies and Gents!

I know it's early days yet but is this moving towards being a full project?

IT REALLY SHOULD BE!!! GREAT enhancement to IMCE, WYSIWYG editing and serious image content management in Drupal!

I'll still need a D6 version.

nateB’s picture

Unfortunately it seems IMCE import doesn't complete the task the way a direct upload in IMCE does. Files imported through IMCE Import that are referenced by FileField Sources, then removed from the file field in a node, are deleted from their source location. They shouldn't be. Files imported through IMCE Import should behave like IMCE-uploaded files and remain where they're put.

See: http://drupal.org/node/1216908 - IMCE records uploaded files in file_usage table even if they are not in use

NaX’s picture

Version: 6.x-2.x-dev » 7.x-1.x-dev
FileSize
500 bytes

@artproduct
This seems to be a 7.x problem. It also seems the "imce_files" table is no longer in use in 7.x. So we can maybe remove that.
I don't have a D7 site that I can test this, but try out the attached patch and let us know if it works. If it does maybe dgcartersa can update his D7 port.

nateB’s picture

Thanks, NaX.

The error I'm seeing with the patch (where /srv/www/d7-multisite/public_html/ is the Drupal root):

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'fid' cannot be null: INSERT INTO {file_usage} (fid, module, type, id, count) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4); Array ( [:db_insert_placeholder_0] => [:db_insert_placeholder_1] => imce [:db_insert_placeholder_2] => file [:db_insert_placeholder_3] => [:db_insert_placeholder_4] => 1 ) in file_usage_add() (line 663 of /srv/www/d7-multisite/public_html/includes/file.inc).

NaX’s picture

FileSize
591 bytes

@artproduct
It looks like the files table was renamed in D7 to file_managed.
Here is an updated version of previous patch. Give it a try.

nateB’s picture

Thanks. Still getting errors:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://images/photographs/snapshots/2011/10/16/japan-day-15_nb' for key 'uri': INSERT INTO {file_managed} (uid, filename, uri, filemime, filesize, status, timestamp) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6); Array ( [:db_insert_placeholder_0] => 1 [:db_insert_placeholder_1] => japan-day-15_nb_20111016_1238.jpg [:db_insert_placeholder_2] => public://images/photographs/snapshots/2011/10/16/japan-day-15_nboguszewski_20111016_1238.jpg [:db_insert_placeholder_3] => image/jpeg [:db_insert_placeholder_4] => 848563 [:db_insert_placeholder_5] => 1 [:db_insert_placeholder_6] => 1319250806 ) in drupal_write_record() (line 6868 of /srv/www/d7-multisite/public_html/includes/common.inc).

ufku’s picture

Status: Needs review » Closed (fixed)

imce_filefield allows using ftp files in file fields.

MrField’s picture

Title: IMCE FTP Import » IMCE FTP Import for Field Type VIDEO

Hi ufku,
Thanks a lot for the IMCE for FileField-Module! I have still used it for some projects. Unfortunately it works "only" with the file field and not with the field type video. Do you have a version which works in the same way but with the video field (from the video-module)?

Charles Belov’s picture

Title: IMCE FTP Import for Field Type VIDEO » IMCE FTP Import
Issue summary: View changes

Renaming to previous title. If you want a related feature, please open a new issue.