filepath error with the import function of the flash node module

This is a weird one - I have tried to recreate this bug and I can't! Because I couldn't create the error with my regular test set up I even created a configuration that matches yours, even down to the location of http://localhost/njprod/drupal!

For that install I'm using Drupal 5.7 and flash node 5.x-5.3, both of which I just downloaded from here in case the flash node package was somehow damaged.

Even doing that I can't create the error!

The fact that flash node is presenting a list of files for import suggests it is able to look in the right place to get started, so I really can't figure out why things then go wrong for you. On my set up the resulting path for an import file is http://localhost/njprod/drupal/files/flash/movies/ImportTest.flv

There are a few things you can check, and if you can post the answers it might help us figure this one out.

First, when you go to the import page, where is flash node saying it will look. On my set up it says: The import function will scan the files/flash directory and sub-directories to locate files for import.

What text is displaying in the table of items for import - mine reads movies/ImportTest.flv. Your post suggests you are seeing iles/flash/movies/moviename.flv.

Can you cross check what you have for your site for the file system path (Administer > Site configuration > File system), and also the default flash node path (Administer > Site configuration > Flash node. The default settings would be files for the first and flash for the second.

Do you see any other error messages?

Can you create a flash node with an flv using Create content rather than import?

What if you download flash node again and reinstall it, just in case something did get messed up?

For now I'm stumped so please post again and I'll try to help some more if I can!

--edit--
Just noticed

for all the listed content, not just for the movie folder content

. What other content is being listed? You should only see a list of items that are in the flash directory and sub-directories. If you are seeing other stuff then it means flash node is looking in the wrong place, or starting at the wrong level!

Hi Stuart,

First of all thanks for taking care of my request and replying so quickly. I've been quiet busy this last couple days and probably still will be the upcoming week so might not be able to catch up with your effeciency.

Here's some more details I should have given you on my first post.
I'm running a Drupal 5.7 install updated from 5.5
I actualy reinstalled the module and tried all over again with the latest update before submiting the 1st support request, and I haven't noticed any other error message anywhere so far.

I checked again :
Administer > Site configuration > File system is set up as « files »
Administer > Site configuration > Flash node is set up as « flash »
no slash, no spaces or dots, I would have added by mistake.

Flash node tells me it will look in the files/flash directory. And it does since it can find the files ( it can find it before I create the corrupted path's directory ). I'm sure Flash node is looking in the right place, there's actualy no other content, but I get all the subdirectories as if they were files I could upload. I am not sure if this is normal.

I checked the DB and the file appears in the flashnode table with a file base which should be ok : /njprod/drupal/files
However in the files table I found the movie with the following datas
filename : _flashnode
path : iles/flash/movies/moviename.flv
fiemime : application/octet-stream
filesize : 0

With a 0 ko filesize display I assume the import has somehow failed, however fixing the path directly in the db allows the movie plays.

Finaly, I am able to create a flash node using Create content, it works fine but won't match the needs for my project.

I had to do add a line of php code however to fix the swf tools module as described in this post http://drupal.org/node/197643. Couldn't there be any relationship between this and my issue ?

As for now I just have a few .flv files for testing purpose, I just replace the path manualy in the DB, and look for a way to do it with some SQL, but I'm worried that it could cause other issues later. Especialy if J.Wijering's player somehow gets the filesize from the DB.

Hope I could provide enough informations to find out where this mysterious bug comes from.

Many thanks, again, for your support.

P.S. ...
here's some more informations :

The module gives me the same mistake on a fresh Drupal 5.7 & flashnode install ... but then the subdirectories don't get listed as content I could upload.

The problem occurs wether the file directory is configured as public or private.

I get content listed for import even if it's already in DB ( I thought, from what I understood of the module code, that it wouldn't )

this let me even more confused. Couldn't this have to do with improper server/php settings on my system ? if so what could I check to find out where the problem comes from ?

I'm confused too!

I get all the subdirectories as if they were files I could upload. I am not sure if this is normal.

No, that is not normal or expected behaviour. The module should ignore directories and not present them for import, but then it sounds like this issue was corrected when you re-installed the module.

The base entry in the flash node table is correct for your site, but that string is made from a simple combination, so I think we can ignore that.

I still don't understand why the file paths are being trimmed of the leading 'f'. There are a few regex's in the code to strip and prepare the file paths, so it sounds like they are failing, so the focus has to be there. When I make a similar set up on my test server it works fine, and that's running Windows.

As a test I'll try simulating your set up on my public webspace which is Linux. That way I can at least see if there is an error in the import code for some reason when running live, or on a different platform.

The zero entry in the files table will be because when the table is populated the file path has already "gone wrong", so the system can't find the file to query its size, so you get zero. If we can find out why the path is being corrupted that error should go away automatically!

Thanks for the comprehensive answers to the other questions - leave it with me and I'll investigate some more!

OK, I have created a copy of your config on my webspace, and it works fine, so whatever the issue is it appears to be specific to your configuration.

As a check, I've modified a version of flashnode.module to write out some debug messages. Can you use this file in place of your regular flashnode.module (simply over-write the current version, no need to deactivate or anything). The file is attached below.

Once installed then run the first part of the import and copy and paste the messages that are displayed.

When I run the debug version on my set up I get...

$root = C:\Xampp\htdocs\njprod\drupal
Processing files/flash/Test.flv
Ignore this - already in database
Processing files/flash/movies
Ignore this - directory
Processing files/flash/movies/Butterfly.flv
Adding to array: files/flash/movies/Butterfly.flv
Processing files/flash/movies/ImportTest - Copy.flv
Adding to array: files/flash/movies/ImportTest - Copy.flv
Processing files/flash/movies/ImportTest.flv
Ignore this - already in database
Processing files/flash/temp
Ignore this - directory
Building urls - needle = files/flash/
base_url = http://localhost/njprod/drupal/
file = files/flash/movies/Butterfly.flv
file = files/flash/movies/ImportTest - Copy.flv

This shows what flash node is working with behind the scenes and will hopefully pinpoint where the 'f' goes missing! The first part shows the actual files that are being parsed, while the second part shows what the module is using to construct file paths. I re-read the code and I can't see a reason for this - the regex's look right!

Depending what the above shows we might get a step closer to finding the bug, or it might suggest further debug code is needed.

Thanks!

I had to do add a line of php code however to fix the swf tools module as described in this post http://drupal.org/node/197643. Couldn't there be any relationship between this and my issue ?

In this case, no. This fix is applied when the node is rendered. It doesn't interact with the database. Whatever is going on is somewhere between flash node, Drupal and the server!!

Hi Stuart,

Have you noticed, in my previous post, that the file that are already in the DB get listed too ? I don't know much about php, but from what I understood of the module code it shouldn't get listed. Could you confirm this ?

I ran the debug version and here's the message I get.

$root = C:\Users\beyourownhero\Documents\wamp\www\njprod\drupal\
Processing iles/flash/Comp 1.flv
Adding to array: iles/flash/Comp 1.flv
Processing iles/flash/movies
Adding to array: iles/flash/movies
Processing iles/flash/movies/????.flv
Adding to array: iles/flash/movies/????.flv
Processing iles/flash/movies/movienamehere.flv
Adding to array: iles/flash/movies/movienamehere.flv
Processing iles/flash/temp
Adding to array: iles/flash/temp
Processing iles/flash/test2.flv
Adding to array: iles/flash/test2.flv
Building urls - needle = files/flash/
base_url = http://localhost/njprod/drupal/
file = iles/flash/Comp 1.flv
file = iles/flash/movies
file = iles/flash/movies/????.flv
file = iles/flash/movies/movienamehere.flv
file = iles/flash/temp
file = iles/flash/test2.flv

The ?????.flv file is a problem with chinese language I am currantly dealing with ... Anyways, I get the corrupted path wether the .flv files that are named in chinese or not.

2 of the listed files are already in the DB.

I'm actualy running Drupal on wamp + vista.

I did not set up my root directory under C:\ ... I had not thought of this before, but it might become a problem if the string for the root path got too long. I don't think it has any influence here, but just to make sure, I changed the httpd.conf to set the root direcoty to C:\www\ and tested with a fresh and clean Drupal 5.7 + flash node debug module and got the same issue ...

$root = C:\www\dpltestsite\
Processing iles/flash/Comp 1.flv
Adding to array: iles/flash/Comp 1.flv
Processing iles/flash/test2.flv
Adding to array: iles/flash/test2.flv
Building urls - needle = files/flash/
base_url = http://localhost/dpltestsite/
file = iles/flash/Comp 1.flv
file = iles/flash/test2.flv

the 2 files are already in the DB.

Hope this can help ... let me know if I can check more on my config.

PS : is there any way to add taxonomy to the bulck flashnode import feature ? or should this be a feature request ? don't know if many users would need it though.

OK - the debug comments help. It looks like the error is occurring in the function that builds the list of files as the file list is already corrupted. I'll need to make another debug version tonight and get you to run that to pin point where the corruption comes in.

The damaged filepaths are why the system is reporting files already in the system. It uses the file path to determine if the entry is already in the database. Because the filepath is missing an 'f' it doesn't match existing entries, so they get reported again.

Also, in your debug log I see it is reporting directories and adding them to the list too, which is also wrong.

I'll post again tonight with some new code if you don't mind running it!

<edit>

Ignore this post!

</edit>

OK, I updated the module with some more debug messages. If you could do the same again, use this module, copy the messages and paste the results that would be great.

This version produces more detailed messages during the directory processing stage.

Thanks

Ignore that previous post (for now!) I think I have spotted the source of the problem.

My root path is C:\Xampp\htdocs\njprod\drupal - note no trailing slash.
Your root path is C:\Users\beyourownhero\Documents\wamp\www\njprod\drupal\ - note trailing slash.

The trailing slash is the problem. When flash node processes the files it strips the root path PLUS one more character, to get rid of what would otherwise become a leading slash. Because you already HAVE the slash it strips the first character!

I'll need to look at this tonight to understand why you are getting this (must be a config setting somewhere).

As a test try editing your flashnode.module file. Look for

and change it to

i.e. remove the dot before the second @. This should stop the 'f' being trimmed.

Then try the import again and post back if it works.

I think I know what is going on! The import routine uses the php function realpath() as part of its logic.

The path that is returned varies depending whether you are using PHP4 or PHP5. Under PHP4 the returned path doesn't have a trailing slash. Under PHP5 it does.

I developed flashnode on PHP4, so the logic takes the path plus one character to account for there being no slash. Under PHP5 that means the code inadvertently trims one character too many - the 'f' of files.

The temporary fix outlined in response 10 will fix the symptoms, and I can now update the module to handle this issue properly.

I'll try to fix the code tonight and upload it here as a zip file if you don't mind testing it for me. If it works I'll then commit it and re-release the module.

(It also tells me I need to check my local set up - I was sure I set that up on PHP5!)

OK - I have modified the code and attached a new zip if you don't mind testing it. The import should work properly in this module now.

It still writes out a few diagnostic lines if you could copy and paste them. There are two fixes that could be applied, and the bug you are seeing suggests that root cause ought to crop up in two places, but it doesn't.

I'm puzzled because although my local system is running PHP5 I don't seem to get the "trailing slash", which means it is slightly difficult for me to test the code!!

Either way, there are two fixes and the next set of diagnostic messages will solve the problem and tell me which one I need.

Thanks for your help!

Well, I'm actualy using php5 ... Yet I don't understand why I seem to be the only one to get this bug, I'll read more on the link you provided. Here's the result for the 3rd debug ...

Real path of . is C:\Users\beyourownhero\Documents\wamp\www\njprod\drupal
Real path of ./ is C:\Users\beyourownhero\Documents\wamp\www\njprod\drupal\
Processing directory C:\Users\beyourownhero\Documents\wamp\www\njprod\drupal\files\flash
Processing directory C:\Users\beyourownhero\Documents\wamp\www\njprod\drupal\files\flash/movies
files/flash/Comp 1.flv
files/flash/Comp 1b.flv
files/flash/movies
files/flash/movies/????.flv
files/flash/movies/Comp2.flv
files/flash/temp
files/flash/Comp2b.flv

The module import function works just perfectly now, so I guess all the other errors were related to the messed up filepath. Thanks a lot for support.

Thanks for the debug prints. It means I can implement a simple fix by using realpath('.') to suppress the trailing slash and not have to run a regex.

I've amended both flash node version 5 and version 6 to use the changed code and re-released the packages.

Thank for your help in debugging this!

Good luck with your site, and thanks for your interest in flash node - sorry about the bug :-)

Automatically closed -- issue fixed for two weeks with no activity.