While porting the code over to 4.7 I found a system problem that can make the upload fail (this is both in the cvs and the 4.6 versions). It is not a problem with the code in this module but related to the way that Drupal and php handle file uploads. As near as I can tell, in some situations the mime type on the uploaded csv file does not agree with what the system thinks the mime type should be and it is reported as a failure, even though it really isn't. It looks like it depends on what OS and php version the server is using and the OS and browser type and version used by the client. At any rate, when I tried to upload a csv file it kept reporting that it was unable to find the file even though I could see the file had hit the upload directory.

I found a work-around. Save your data to be uploaded as a csv file, but use a .txt file extension instead of a .csv file extension (or rename the file after it is created). When I did that, the upload worked fine. When I tried to upload the very same file with a .csv extension, it failed every time.

If this works for others, we might want to add some info that the end user can see suggesting that they make that change.

Comments

sanduhrs’s picture

sanduhrs
he/him
Primary language German
Location 🇪🇺 Heidelberg, Germany, Europe
commented

When uploading a .csv file i get Could not find the csv file ''.
The following errors are reported in watchdog:

rename(): open_basedir restriction in effect. File(/csv_1140365041.txt) is not within the allowed path(s): (/var/www/drupal47.xxx.de/httpd:/tmp) in /var/www/drupal47.xxx.de/httpd/modules/user_import/user_import.module on line 277.

and

fgetcsv(): supplied argument is not a valid stream resource in /var/www/drupal47.xxx.de/httpdocs/modules/user_import/user_import.module on line 1144.

After renaming the file to .txt, all went well.
Thanks for the hint.

silurius’s picture

silurius commented

Drupal version: Drupal 4.7 RC (CVS)
PHP version = 4.4.2 cgi
Webserver = Apache/1.3.33 (Unix) DAV/1.0.3 mod_fastcgi/2.4.2 mod_gzip/1.3.26.1a PHP/4.4.2 mod_ssl/2.8.22 OpenSSL/0.9.7e
Database = mysql 5.0.18-standard-log, lock.system=flock
Firefox/1.5.0.1

Getting no error messages, but when I upload a CSV or a CSV file named as TXT, absolutely nothing happens after the upload.

evelien’s picture

evelien commented

same here: nothing is happening. Nothing is written back into the database and there's no error. Anyone?

sanduhrs’s picture

sanduhrs
he/him
Primary language German
Location 🇪🇺 Heidelberg, Germany, Europe
commented

I can confirm this by using Drupal4.7RC2 and user_import-cvs
vg

robert castelo’s picture

robert castelo commented

I'll add an option next week which will allow you to FTP the CVS file to your user_module directory, the import can then be done from that file.

This will sidestep Drupal's current problem with browser uploads, and also be useful in situations where the CVS file is too big for a browser upload anyway.

sanduhrs’s picture

sanduhrs
he/him
Primary language German
Location 🇪🇺 Heidelberg, Germany, Europe
commented

The upload doesn't seem to be the Problem. The File gets uploaded to /tmp, renaming is working too.
But from there, nothing seems to happen.
vg

karens’s picture

karens commented

The upload is not the problem, the problem is that Drupal's file.inc program is deciding the upload is invalid and refusing to pass it back to the module to use. Lots of things have been tried (see more discussion about this at http://drupal.org/node/48202). I finally gave up (and I think that is why Robert is going to try the ftp alternative instead).

In some cases, changing the file extension to .txt worked because file.inc is more forgiving about files with that extension than about files with a .csv extension, but that doesn't always work either. The crux of the problem is that the file you are uploading has not identified itself in the way that file.inc wants to see it, so nothing more can happen.

benthere’s picture

benthere commented

I think the problem in core has been fixed. The mime-magic part of file.inc was removed. See the changelog.

There appears to be a seperate problem with user_import causing uploads to silently fail.

robert castelo’s picture

robert castelo commented

Thanks for the heads up.

I've added a feature that scans for extra files in the module directory, which you upload using FTP, and allows you to use those if available. Also added option for spaces in usernames, and support for the one time loggin in welcome message.

Tonight I'm working on enabling profile info in welcome message, e.g. Dear %profile_firstname. I'll upload the latest version sometime tomorrow.

Once that's done I can get to work on fixing the browser upload feature.

robert castelo’s picture

robert castelo commented

Just uploaded the latest version which includes a few additional features.

I'll work on fixing browser upload next week, unless someone else gets it done first.

benthere’s picture

benthere commented

Thanks a lot Robert. The FTP upload works great. I was just about to give in and enter my memberlist from an Excel spreadsheet manually. Perfect timing. Such a timesaver.

robert castelo’s picture

robert castelo commented

Glad it saved you some time, makes me wince thinking about anyone doing this by hand!

Tip: you can now also use profile fields in your welcome message, so you can ask new members to validate their details ;-)

Zack Rosen’s picture

Zack Rosen commented

I still can't get csv upload to work through the web form. I'd rather not have to rely on the FTP to upload files as the user of the site I am building has never worked with a client.

Am I sunk?

robert castelo’s picture

robert castelo commented

I'm scheduled to work on the user import module in the next two weeks, and uploading the csv file through the browser will be one of the things I'll be taking care off.

It's the main thing holding up releasing this as an official 4.7 release.

robert castelo’s picture

robert castelo commented
Status: Active » Fixed

Fixed in User Import 4.7.x-1.0 version.

Anonymous’s picture

(not verified) commented
Status: Fixed » Closed (fixed)