The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.adrinux reports that the tally of how many images are imported may not be correct in all cases.
Problem under investigation.
Scott
The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.adrinux reports that the tally of how many images are imported may not be correct in all cases.
Problem under investigation.
Scott
Comments
Comment #1
adrinux commentedJust got a similar result with a second import.
Preview import listed and report 29 images would be imported.
After import the report was that 11 of 11 images had been imported successfully, despite 28 actually having been imported.
All 29 images were deleted!
So not only is the reported tally incorrect, but one image is missed.
The first import I did it missed image 35 if 39, and reported 18 images imported not 38.
The second import it missed image 18 of 29, and reported 11 images imported not 28.
All very odd.
Comment #2
adrinux commentedand of course the missed image *is* deleted.
Comment #3
adrinux commentedI remembered I'd done a third import before the other two, can't remember what the report was but I can happily say that all 21 of 21 images were imported and deleted, nothing was missed.
Possibly this only happens when more than a certain number of images are imported, perhaps its server related and quite spurious?
Comment #4
syscrusher commentedThe latest CVS version has improved tallying and improved early detection of files that won't import, but I'm not sure I've 100% fixed the problem.
One situation I did find, that could cause this sort of thing, is if you have files that are the same except for their extension and where there is a caption involved. For example, if you have these files:
myfile001.jpg
myfile001.gif
myfile001.html
the HTML file will be "found" as the caption for each of the others, and will be counted twice. There are probably some other weirdnesses like this that I haven't found yet.
Incidentally, the new documentation file TROUBLE.txt documents the above situation and some other common problems people have reported.
Could you please try the latest CVS version in a similar use-case to what you did before that gave the bad tally, and let me know how it works? If it fails to tally properly, it would help if you could give me an "ls -l" (or equivalent) from the import directory as it looked before and after the import run.
Thanks.
Scott
Comment #5
adrinux commentedIndeed, things are definitely improving but still not right.
I'm not using captions so there is no confusion there, and all files have unique names with the same extension.
Just updated today, 30th may. I'll try and give more detail this time.
No command line access, but here is a listing extracted from the cpanel file manager (my ftp client won't let me cut and paste, sorry), the three columns are filename, filesize and permissions:
.ftpquota 0 k 766
sphp_2002_001_tur01.jpg 51 k 766
sphp_2002_002_tur02.jpg 60 k 766
sphp_2002_003_tur03.jpg 49 k 766
sphp_2002_004_tur04.jpg 53 k 766
sphp_2002_005_tur05.jpg 50 k 766
sphp_2002_006_tur06.jpg 57 k 766
sphp_2002_007_tur07.jpg 59 k 766
sphp_2002_008_tur08.jpg 53 k 766
sphp_2002_009_tur09.jpg 81 k 766
sphp_2002_010_benrimes.jpg 53 k 766
sphp_2002_011_benrinnes.jpg 142 k 766
sphp_2002_012_nesbit.jpg 68 k 766
sphp_2002_014_tom.jpg 85 k 766
sphp_2002_015_tom.jpg 104 k 766
sphp_2002_016_tom.jpg 127 k 766
sphp_2002_017_tom.jpg 64 k 766
sphp_2002_018_tom.jpg 90 k 766
sphp_2002_019_tom.jpg 82 k 766
sphp_2002_020_tom.jpg 134 k 766
sphp_2002_021_tom.jpg 100 k 766
sphp_2002_022_tom.jpg 51 k 766
sphp_2002_023_tom.jpg 86 k 766
sphp_2002_024_tom.jpg 59 k 766
sphp_2002_025_tom.jpg 93 k 766
sphp_2002_026_tom.jpg 63 k 766
sphp_2002_027_tom.jpg 75 k 766
sphp_2002_028_tom.jpg 65 k 766
sphp_2002_029_tom.jpg 112 k 766
sphp_2002_030_tom.jpg 104 k 766
sphp_2002_031_tom.jpg 38 k 766
sphp_2002_032_tom.jpg 120 k 766
sphp_2002_033_tom.jpg 156 k 766
sphp_2002_034_tom.jpg 135 k 766
sphp_2002_035_tom.jpg 69 k 766
sphp_2002_036_tom.jpg 142 k 766
sphp_2002_037_tom.jpg 117 k 766
sphp_2002_038_tom.jpg 93 k 766
sphp_2002_039_tom.jpg 110 k 766
sphp_2002_040_tom.jpg 108 k 766
sphp_2002_041_tom.jpg 107 k 766
sphp_2002_042_tom.jpg 87 k 766
sphp_2002_043_tom.jpg 91 k 766
sphp_2002_044_tom.jpg 78 k 766
sphp_2002_045_tom.jpg 76 k 766
sphp_2002_046_tom.jpg 51 k 766
sphp_2002_048_lighthoose.jpg 116 k 766
sphp_2002_049_lighthoose.jpg 196 k 766
sphp_2002_050_lighthoose.jpg 160 k 766
sphp_2002_051_lighthoose.jpg 117 k 766
sphp_2002_052_lighthoose.jpg 114 k 766
sphp_2002_053_lighthoose.jpg 121 k 766
sphp_2002_054_lighthoose.jpg 152 k 766
sphp_2002_055_lighthoose.jpg 93 k 766
sphp_2002_056_lighthoose.jpg 98 k 766
sphp_2002_057_lighthoose.jpg 42 k 766
sphp_2002_058_henk.jpg 83 k 766
sphp_2002_059_henk.jpg 70 k 766
sphp_2002_060_henk.jpg 108 k 766
sphp_2002_061_henk.jpg 158 k 766
sphp_2002_062_henk.jpg 90 k 766
sphp_2002_063_henk.jpg 97 k 766
sphp_2002_064_henk.jpg 98 k 766
sphp_2002_065_henk.jpg 83 k 766
sphp_2002_066_henk.jpg 99 k 766
sphp_2002_067_henk.jpg 169 k 766
sphp_2002_068_henk.jpg 196 k 766
sphp_2002_069_henk.jpg 80 k 766
sphp_2002_070_henk.jpg 85 k 766
sphp_2002_071_henk.jpg 136 k 766
sphp_2002_072_henk.jpg 33 k 766
sphp_2002_073_henk.jpg 135 k 766
sphp_2002_074_henk.jpg 138 k 766
sphp_2002_075_henk.jpg 76 k 766
sphp_2002_076_henk.jpg 71 k 766
sphp_2002_077_henk.jpg 103 k 766
sphp_2002_078_henk.jpg 72 k 766
sphp_2002_079_henk.jpg 73 k 766
sphp_2002_080_henk.jpg 82 k 766
sphp_2002_081_henk.jpg 48 k 766
sphp_2002_082_henk.jpg 66 k 766
sphp_2002_083_henk.jpg 46 k 766
So there are 81 image files listed here, (sphp_2002_013*.jpg and and sphp_2002_047*.jpg don't exist).
So we preview the import, this is cut and pasted from Drupal:
sphp_2002_001_tur01.jpg would be imported.
sphp_2002_002_tur02.jpg would be imported.
sphp_2002_003_tur03.jpg would be imported.
sphp_2002_004_tur04.jpg would be imported.
sphp_2002_005_tur05.jpg would be imported.
sphp_2002_006_tur06.jpg would be imported.
sphp_2002_007_tur07.jpg would be imported.
sphp_2002_008_tur08.jpg would be imported.
sphp_2002_009_tur09.jpg would be imported.
sphp_2002_010_benrimes.jpg would be imported.
sphp_2002_011_benrinnes.jpg would be imported.
sphp_2002_012_nesbit.jpg would be imported.
sphp_2002_014_tom.jpg would be imported.
sphp_2002_015_tom.jpg would be imported.
sphp_2002_016_tom.jpg would be imported.
sphp_2002_017_tom.jpg would be imported.
sphp_2002_018_tom.jpg would be imported.
sphp_2002_019_tom.jpg would be imported.
sphp_2002_020_tom.jpg would be imported.
sphp_2002_021_tom.jpg would be imported.
sphp_2002_022_tom.jpg would be imported.
sphp_2002_023_tom.jpg would be imported.
sphp_2002_024_tom.jpg would be imported.
sphp_2002_025_tom.jpg would be imported.
sphp_2002_026_tom.jpg would be imported.
sphp_2002_027_tom.jpg would be imported.
sphp_2002_028_tom.jpg would be imported.
sphp_2002_029_tom.jpg would be imported.
sphp_2002_030_tom.jpg would be imported.
sphp_2002_031_tom.jpg would be imported.
sphp_2002_032_tom.jpg would be imported.
sphp_2002_033_tom.jpg would be imported.
sphp_2002_034_tom.jpg would be imported.
sphp_2002_035_tom.jpg would be imported.
sphp_2002_036_tom.jpg would be imported.
sphp_2002_037_tom.jpg would be imported.
sphp_2002_038_tom.jpg would be imported.
sphp_2002_039_tom.jpg would be imported.
sphp_2002_040_tom.jpg would be imported.
sphp_2002_041_tom.jpg would be imported.
sphp_2002_042_tom.jpg would be imported.
sphp_2002_043_tom.jpg would be imported.
sphp_2002_044_tom.jpg would be imported.
sphp_2002_045_tom.jpg would be imported.
sphp_2002_046_tom.jpg would be imported.
sphp_2002_048_lighthoose.jpg would be imported.
sphp_2002_049_lighthoose.jpg would be imported.
sphp_2002_050_lighthoose.jpg would be imported.
sphp_2002_051_lighthoose.jpg would be imported.
sphp_2002_052_lighthoose.jpg would be imported.
sphp_2002_053_lighthoose.jpg would be imported.
sphp_2002_054_lighthoose.jpg would be imported.
sphp_2002_055_lighthoose.jpg would be imported.
sphp_2002_056_lighthoose.jpg would be imported.
sphp_2002_057_lighthoose.jpg would be imported.
sphp_2002_058_henk.jpg would be imported.
sphp_2002_059_henk.jpg would be imported.
sphp_2002_060_henk.jpg would be imported.
sphp_2002_061_henk.jpg would be imported.
sphp_2002_062_henk.jpg would be imported.
sphp_2002_063_henk.jpg would be imported.
sphp_2002_064_henk.jpg would be imported.
sphp_2002_065_henk.jpg would be imported.
sphp_2002_066_henk.jpg would be imported.
sphp_2002_067_henk.jpg would be imported.
sphp_2002_068_henk.jpg would be imported.
sphp_2002_069_henk.jpg would be imported.
sphp_2002_070_henk.jpg would be imported.
sphp_2002_071_henk.jpg would be imported.
sphp_2002_072_henk.jpg would be imported.
sphp_2002_073_henk.jpg would be imported.
sphp_2002_074_henk.jpg would be imported.
sphp_2002_075_henk.jpg would be imported.
sphp_2002_076_henk.jpg would be imported.
sphp_2002_077_henk.jpg would be imported.
sphp_2002_078_henk.jpg would be imported.
sphp_2002_079_henk.jpg would be imported.
sphp_2002_080_henk.jpg would be imported.
sphp_2002_081_henk.jpg would be imported.
sphp_2002_082_henk.jpg would be imported.
sphp_2002_083_henk.jpg would be imported.
Found 81 file(s) to import, with 0 captioned from files.
Thus all 81 files are correctly identified for import. No problems at all.
Then run the actual import and Drupal reports:
sphp_2002_001_tur01.jpg imported.
sphp_2002_002_tur02.jpg imported.
sphp_2002_003_tur03.jpg imported.
sphp_2002_004_tur04.jpg imported.
sphp_2002_005_tur05.jpg imported.
sphp_2002_006_tur06.jpg imported.
sphp_2002_007_tur07.jpg imported.
sphp_2002_008_tur08.jpg imported.
sphp_2002_009_tur09.jpg imported.
sphp_2002_010_benrimes.jpg imported.
sphp_2002_011_benrinnes.jpg imported.
sphp_2002_012_nesbit.jpg imported.
sphp_2002_014_tom.jpg imported.
sphp_2002_015_tom.jpg imported.
sphp_2002_016_tom.jpg imported.
sphp_2002_017_tom.jpg imported.
sphp_2002_018_tom.jpg imported.
sphp_2002_019_tom.jpg imported.
sphp_2002_020_tom.jpg imported.
sphp_2002_022_tom.jpg imported.
sphp_2002_023_tom.jpg imported.
sphp_2002_024_tom.jpg imported.
sphp_2002_025_tom.jpg imported.
sphp_2002_026_tom.jpg imported.
sphp_2002_027_tom.jpg imported.
sphp_2002_028_tom.jpg imported.
sphp_2002_029_tom.jpg imported.
sphp_2002_030_tom.jpg imported.
sphp_2002_031_tom.jpg imported.
sphp_2002_032_tom.jpg imported.
sphp_2002_033_tom.jpg imported.
sphp_2002_034_tom.jpg imported.
sphp_2002_036_tom.jpg imported.
sphp_2002_037_tom.jpg imported.
sphp_2002_038_tom.jpg imported.
sphp_2002_039_tom.jpg imported.
sphp_2002_040_tom.jpg imported.
sphp_2002_041_tom.jpg imported.
sphp_2002_042_tom.jpg imported.
sphp_2002_043_tom.jpg imported.
sphp_2002_044_tom.jpg imported.
sphp_2002_046_tom.jpg imported.
sphp_2002_048_lighthoose.jpg imported.
sphp_2002_049_lighthoose.jpg imported.
sphp_2002_050_lighthoose.jpg imported.
sphp_2002_051_lighthoose.jpg imported.
sphp_2002_052_lighthoose.jpg imported.
sphp_2002_053_lighthoose.jpg imported.
sphp_2002_055_lighthoose.jpg imported.
sphp_2002_056_lighthoose.jpg imported.
sphp_2002_058_henk.jpg imported.
sphp_2002_059_henk.jpg imported.
sphp_2002_060_henk.jpg imported.
sphp_2002_061_henk.jpg imported.
sphp_2002_062_henk.jpg imported.
sphp_2002_064_henk.jpg imported.
sphp_2002_065_henk.jpg imported.
sphp_2002_066_henk.jpg imported.
sphp_2002_067_henk.jpg imported.
sphp_2002_068_henk.jpg imported.
sphp_2002_070_henk.jpg imported.
sphp_2002_071_henk.jpg imported.
sphp_2002_072_henk.jpg imported.
sphp_2002_073_henk.jpg imported.
sphp_2002_074_henk.jpg imported.
sphp_2002_075_henk.jpg imported.
sphp_2002_076_henk.jpg imported.
sphp_2002_077_henk.jpg imported.
sphp_2002_079_henk.jpg imported.
sphp_2002_080_henk.jpg imported.
sphp_2002_081_henk.jpg imported.
sphp_2002_082_henk.jpg imported.
sphp_2002_083_henk.jpg imported.
Successfully imported 5 file(s) out of 5, with 0 captioned from files.
Deleted 5 image(s) and associated caption(s) from the import directory.
So, 73 images were imported, numbers 21, 35, 45, 57, 63, 69, and 78 were missed. But there's an interesting observation here, there were 5 images imported after the last missed image! So it looks like there is a connection between the missed images and the tally error. It's like there is a hiccup and the import starts afresh.
And our folder now looks like:
.ftpquota 0 k 766
So all images were deleted.
Checking the logs I see nothing...
Checking the module version just to be sure I updated:
// $Id: image_import.module,v 1.2 2005/05/27 03:06:35 syscrusher Exp $
It's getting late today, I'll try and find the time to do some arithmetic tomorrow, the missed images don't occur exactly every x number of images imported, but I'm wondering if it isn't the amount of data that causes a hiccup and triggers an image to be missed. The images in the first import I did (comment #3 above, and none missed) were smaller, that's what makes me think of data...
Comment #6
syscrusher commentedadrinux wrote:
Adrinux, my hat's off to you for a terrific test report. This helps a great deal. I'm going to look and see if maybe I'm zeroing the count after each error, rather than simply not incrementing it (which would be correct).
Given this error, and the problem reported with "files files files" being in the database, I am seriously inclined to do a major rewrite of the import loop code. This is very annoying -- I'm not used to my code being so buggy. (I'm blaming myself here, not you!)
I'm sorry for the errors. I'll keep working on this.
Scott
Comment #7
syscrusher commentedPlease see issue 23839 for a major update that I hope fixes this bug also.
Scott
Comment #8
syscrusher commentedA variant of the throttle mechanism proposed by adrinux is in the 2005-06-06 version of image_import.module. This ensures that each image gets a timestamp that differs by at least one second from the previous image.
Scott
Comment #9
(not verified) commented