When trying to upload a file I get this:

Error message
PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://2012/04/04//cousin_1.jpg' for key 'uri': INSERT INTO {file_managed} (uid, filename, uri, filemime, filesize, status, timestamp, type, origname) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8); Array ( [:db_insert_placeholder_0] => 1 [:db_insert_placeholder_1] => cousin_1.jpg [:db_insert_placeholder_2] => public://2012/04/04//cousin_1.jpg [:db_insert_placeholder_3] => image/jpeg [:db_insert_placeholder_4] => 2390372 [:db_insert_placeholder_5] => 1 [:db_insert_placeholder_6] => 1333577537 [:db_insert_placeholder_7] => image [:db_insert_placeholder_8] => cousin_1.jpg ) in drupal_write_record() (line 6980 of /Users/gknaddison/Sites/sports/docroot/includes/common.inc).

If I disable fe_paths the problem goes away.

The problem is apparent at file/add using plupload OR via node/add/TYPE using an image and the media selector widget.

Comments

dave reid’s picture

I wonder if this is a problem because file_move() calls file_save() itself.

dave reid’s picture

Maybe it would be beneficial to have the file directory move be processed in using a queue or drupal_register_shutdown_function() that runs at the end of the request.

greggles’s picture

ok, I've begun work on the shutdown function idea. A queue could make sense as well, but I worry that we would need something to process the queue and having images move after 5 minutes (e.g. if we process on cron) could be a problem, right? Like if someone creates a node and has the img src point at one place then 4 minutes later on cron the image moves and the img is now broken.

greggles’s picture

Title: duplicate key constraint » Move files once, in shutdown
Status: Active » Needs review
StatusFileSize
new3.27 KB

I was consistently running into duplicate key problems as described in the OP. To solve that and based on Dave's suggestion I:
* Made the $processed an array of orignames that have been processed so we only process a file once.
* Moved the moving to a shutdown function

The only weird change in here is:

-    $new_uri = $scheme . '://' . $path . '/' . $filename;
+    $new_uri = $scheme . '://' . $path . $filename;

That seems to be necessary to remove an extra forward slash on the end.

I also didn't understand this line ($paths variable is never used) so I removed it:

-    $paths = $path != '' ? explode('/', $path) : NULL;
szantog’s picture

Status: Needs review » Needs work

Hmm.. I applied the patch manually, but I get 500 error, when upload and image. The new diff against the last commit is:
I search, what the problem is, but I've never used drupal_register_shutdown.

diff --git a/fe_paths.module b/fe_paths.module
index 41d8a84..94d8480 100644
--- a/fe_paths.module
+++ b/fe_paths.module
@@ -105,9 +105,14 @@ function fe_paths_file_presave($file) {
   fe_paths_file_process($file);
 }
 
+/**
+ * Actually processes the files.
+ * @see fe_paths_file_presave().
+ * @see fe_paths_move_files().
+ */
 function fe_paths_file_process($file) {
-  // Set a drupal_static variable, to prevent multiple processing.
-  $processed = &drupal_static(__FUNCTION__, FALSE);
+  // Get a static of processed files to prevent infinite processing.
+  $processed = &drupal_static(__FUNCTION__, array());
 
   // Only work on local files, that have not been processed, and that already have a file_entity type.
   if (file_entity_file_is_local($file) && !array_key_exists($file->origname, $processed) && isset($file->type)) {
@@ -131,33 +136,37 @@ function fe_paths_file_process($file) {
 
     $paths = $path != '' ? explode('/', $path) : NULL;
 
-    $new_uri = $scheme . '://' . $path . '/' . $filename;
+    $new_uri = $scheme . '://' . $path . $filename;
     // Move the files, if necessary
     if ($file->uri != $new_uri) {
       $directory = $scheme . '://' . $path;
-      // Create directory, if necessary
-      file_prepare_directory($directory, FILE_CREATE_DIRECTORY);
-      // Set processed here true, to prevent another processing from file_move.
-      $processed = TRUE;
-
-      if (file_move($file, $new_uri)) {
-        // Because of file_move works with a cloned $file object, need to update
-        // this $file object.
-        $file->filename = $filename;
-        $file->uri = $new_uri;
-
-        // Check, if directory is empty, and delete it, if yes
-        $files = file_scan_directory($old_dir_uri, '/.*/');
-        if (empty($files)) {
-          drupal_rmdir($old_dir_uri);
-        }
-      };
+      drupal_register_shutdown_function('fe_paths_move_files', $directory, $file, $new_uri, $filename, $old_dir_uri);
     }
     $processed = TRUE;
   }
 }
 
 /**
+ * A function that can be called in the page shutdown to actually move files. Not safe against recursion.
+ */
+function fe_paths_move_files($directory, $file, $new_uri, $filename, $old_dir_uri) {
+  // Create directory, if necessary.
+  file_prepare_directory($directory, FILE_CREATE_DIRECTORY);
+  if (file_move($file, $new_uri)) {
+    // Because of file_move works with a cloned $file object, need to update
+    // this $file object.
+    $file->filename = $filename;
+    $file->uri = $new_uri;
+
+    // Check, if directory is empty, and delete it, if yes
+    $files = file_scan_directory($old_dir_uri, '/.*/');
+    if (file_exists($old_dir_uri) && empty($files)) {
+      drupal_rmdir($old_dir_uri);
+     }
+   }
+ }
+
+/**
  * Implements hook_token_info().
  */
 function fe_paths_token_info() {
(END)
szantog’s picture

Status: Needs work » Fixed

My fail, wrong manual apply, it seems to work good. Commited.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

ultimike’s picture

It appears that the use of the drupal_register_shutdown_function() is preventing a bunch of the manual image cropping modules from working properly. While the use of the drupal_register_shutdown_function() makes sense, I'm trying to determine if there's a workaround for manual image cropping modules.

I've outlined the issues here: http://drupal.org/node/1647598

I don't think this is an issue with the FE Paths code, I'm just hoping to leverage some expertise to figure out how manual image cropping modules can play nicely with FE Paths.

Thanks in advance,
-mike