On Community sites users change their images often. Drupal renames these images to picture-UID.jpg thus the browser cache gets not updated with the new image and the user does not see any change.

My proposal is to add a timestamp to the user picture on creation or update picture-UID-TIMESTAMP.jpg.

function user_validate_picture(&$form, &$form_state) {
  // If required, validate the uploaded picture.
  $validators = array(
    'file_validate_is_image' => array(),
    'file_validate_image_resolution' => array(variable_get('user_picture_dimensions', '85x85')),
    'file_validate_size' => array(variable_get('user_picture_file_size', '30') * 1024),
  );
  if ($file = file_save_upload('picture_upload', $validators)) {
    // Remove the old picture.
    if (isset($form_state['values']['_account']->picture) && file_exists($form_state['values']['_account']->picture)) {
      file_delete($form_state['values']['_account']->picture);
    }

    // The image was saved using file_save_upload() and was added to the
    // files table as a temporary file. We'll make a copy and let the garbage
    // collector delete the original upload.
    $info = image_get_info($file->filepath);
    $destination = variable_get('user_picture_path', 'pictures') .'/picture-'. $form['#uid'] . time() . '.'. $info['extension'];
    if (file_copy($file, $destination, FILE_EXISTS_REPLACE)) {
      $form_state['values']['picture'] = $file->filepath;
    }
    else {
      form_set_error('picture_upload', t("Failed to upload the picture image; the %directory directory doesn't exist or is not writable.", array('%directory' => variable_get('user_picture_path', 'pictures'))));
    }
  }
}

Comments

derjochenmeyer’s picture

Status: Needs review » Active
Issue tags: -user pictures

Adding a hyphen before time()

    $destination = variable_get('user_picture_path', 'pictures') .'/picture-'. $form['#uid'] . '-' . time() . '.'. $info['extension'];
j.somers’s picture

Status: Active » Needs work

Please provide a patch. You can learn how to create patches at http://drupal.org/patch/create.

derjochenmeyer’s picture

Status: Needs work » Needs review
StatusFileSize
new959 bytes

Here is a first try against D7 HEAD.

Status: Needs review » Needs work

The last submitted patch failed testing.

dave reid’s picture

Issue tags: +user pictures

Adding tag

derjochenmeyer’s picture

StatusFileSize
new959 bytes

Here is a second try against D7 HEAD.

derjochenmeyer’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch failed testing.

derjochenmeyer’s picture

Status: Needs work » Needs review
StatusFileSize
new777 bytes

Here we go again...

stevenpatz’s picture

Status: Needs review » Needs work
StatusFileSize
new730 bytes

Does the test need updated?

Like this?

stevenpatz’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch failed testing.

catch’s picture

We have a constant called REQUEST_TIME which is a bit quicker than calling time() directly.

derjochenmeyer’s picture

StatusFileSize
new783 bytes

using REQUEST_TIME

how can i include a patch for user.test together with this. I tested this patch on a clean D7-dev installation and manual picture upload/replace/delete works.

derjochenmeyer’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch failed testing.

derjochenmeyer’s picture

Status: Needs work » Needs review
StatusFileSize
new1.31 KB

Well next try with updated test (spatz4000 thx for hint)

Edit: REQUEST_TIME in the test will probably not work because obviously it will be a different timestamp :)

What now?

derjochenmeyer’s picture

StatusFileSize
new785 bytes

Manually tested & working patch.

Status: Needs review » Needs work

The last submitted patch failed testing.

derjochenmeyer’s picture

Status: Needs work » Patch (to be ported)
j.somers’s picture

Please, test the patch before submitting. You can clearly see that the patch fails on some tests because (obviously) they are affected by your change. Make sure they work before attaching the patch file to the issue.

j.somers’s picture

Status: Patch (to be ported) » Needs work
derjochenmeyer’s picture

Status: Needs work » Reviewed & tested by the community

Im changing to "reviewed & tested by the community", because I dont know how to rewrite the test in a way that makes sense. Any suggestions?

j.somers’s picture

Status: Reviewed & tested by the community » Needs work

No, reviewed and tested by the community means the patch is OK to be committed to HEAD. This is not the case since the patch doesn't even pass the test bot. The patch needs work and it doesn't really matter if you do it or someone else.

You change the name of the picture file. I don't have the time to look at the test cases right now but I would assume that one of them did not expect this to happen and might thus be checking an incorrect file name. You can easily run these test yourself on your Drupal 7 development copy by enabling the simpletest module.

derjochenmeyer’s picture

many thanks, I'll take a look...

derjochenmeyer’s picture

Status: Needs work » Needs review
StatusFileSize
new1.42 KB

Hope the new test makes sense like this. The only way to get the correct filename is from the database.

Status: Needs review » Needs work

The last submitted patch failed testing.

derjochenmeyer’s picture

Title: Add a timestamp to user pictures to avoid browser cache problems » Add a timestamp to user pictures

I could not find a D7 release for simpletest module.

This one is not working: http://drupal.org/node/236086

Any ideas?

The patch just adds '-' . REQUEST_TIME .

line 731, user.module

-        $destination = file_create_path(variable_get('user_picture_path', 'pictures') . '/picture-' . $account->uid . '.' . $info['extension']);
+        $destination = file_create_path(variable_get('user_picture_path', 'pictures') . '/picture-' . $account->uid . '-' . REQUEST_TIME . '.' . $info['extension']);

i tried to change the test accordingly; line 705, user.test

-    $pic_path = file_directory_path() . '/' . $picture_dir . '/picture-' . $this->user->uid . '.' . $img_info['extension'];
+    $img_name = db_result(db_query("SELECT files.filename FROM {users} users INNER JOIN {files} files ON users.picture = files.fid WHERE users.uid = '%s'", $this->user->uid));
+    $pic_path = file_directory_path() . '/' . $picture_dir . '/' . $img_name;
derjochenmeyer’s picture

Title: Add a timestamp to user pictures » Add a timestamp to user pictures to avoid browser cache problems
catch’s picture

simpletest is in core - just go to admin/build/modules

Instead of db_result(db_query()) you should now use db_query()->fetchField(); in D7.

derjochenmeyer’s picture

StatusFileSize
new2.01 KB

in user.test: For some reason the name of the user picture that gets saved in the files table is different in files.filepath (picture-uid-REQUEST_TIME.ext) and files.filename (default simepleTest name, e.g.: image-test.gif) >> Therefore the only way to get the saved timestamp is to extract it from the saved full path.

derjochenmeyer’s picture

Status: Needs work » Needs review
derjochenmeyer’s picture

Status: Needs review » Reviewed & tested by the community

I really think that simpleTest is awesome, but it could also become a barrier for unexperienced developers. The working patch was written in less than 15min. But it took me almost 2 days to figure out what the test is actually testing, how it works, why it fails, how i could rewrite it that it still makes sense (i hope it does)... :)

webchick’s picture

Status: Reviewed & tested by the community » Needs work

Yeah, SimpleTest is a bit confusing at first, but it saves SO much time on the whole by telling us when we've broken things. :D

A few things. I'm going to be extra nit-picky because it looks like this is one of your first core contributions; I do this only so that you learn best practices early and can apply them throughout your wonderful career as a core contributor! :D

  1. "reviewed & tested by the community" is a status that means "Ok, this thing looks rock-solid and looks ready to become part of Drupal!" The only person who should ever change a patch status to RTBC is someone who is not the original creator of the patch. It's considered bad form to mark your own patch RTBC because it's tantamount to you saying "Ok, I deem my own code perfect," and a) leaves out the peer review process that makes Drupal core so solid, and b) developers are notoriously bad at finding flaws in their own code. ;)
  2. -    $img_info = image_get_info($image->filepath);
    +    $img_info = image_get_info($image->filepath);    
    

    You're introducing some trailing whitespace here (there are invisible spaces at the end of the line). Could you please set your editor to not do that?

  3. +    // for some reason the name of the user picture is different in files.filepath (picture-uid-REQUEST_TIME.ext) and files.filename (default simepleTest name, e.g.: image-test.gif)
    +    // therefore the only way to get the saved timestamp is to extract it from the saved full path
    

    Ok, we can't really commit comments like this to Drupal core. ;)

    a) All comments should wrap at 80 characters.
    b) All comments should begin with a capital letter and end with a period.
    c) But most importantly, we should get to the bottom of why that happened and document it appropriately. ;)

  4. $pic_pathFromDB
    

    The Coding Standards dictate that variable names should:

    a) Not be abbreviated. (e.g. "pic")
    b) Be all lowercase.
    c) Have words separated by underscore.

On IRC, Damien Tournoud also mentioned that the caches should be invalidated automatically since the browser should issue a GET request with a E-Tag and a If-Not-Modified-Since each time the page is refreshed.

So I do think this needs a bit more looking into before it's ready to commit.

derjochenmeyer’s picture

Title: Add a timestamp to user pictures » Add a timestamp to user pictures to avoid browser cache problems
Status: Active » Needs review
Issue tags: +user pictures
StatusFileSize
new1.68 KB

Thanks for the detailed feedback...

  1. I see, sorry :) So there is no need to do anything once the patch passes the tests? "needs review" is good enough? I missed http://drupal.org/node/156119 A "live help" onChange below the dropdown could be very helpfull: "RTBC should only be used In the following cases..."
  2. Done. Is there a way to configure textmate to remove invisible end of line characters automatically? I could not find any documentation searching Google.
  3. My fault, I missed the the DB field description:
    filename 
    Name of the file with no path components. This may differ from the basename of the filepath if the file is renamed to avoid overwriting an existing file.
    
  4. Changed newly introduced variable names, but left the existing ones as they are, e.g. $pic_path
  5. On IRC, Damien Tournoud also mentioned that the caches should be invalidated automatically since the browser should issue a GET request with a E-Tag and a If-Not-Modified-Since each time the page is refreshed.

    I wrote a small custom module that I use on community sites and intranets, because users were complaining that changing the picture has no effect. So it seems that some browsers miss that the picture changed? This patch makes sure the picture is updated, especially when using a custom user account theme with imagecached versions of the user picture (theme_image override).

    Or using this approach: http://www.lullabot.com/articles/imagecache_example_user_profile_pictures

moshe weitzman’s picture

Status: Needs work » Needs review

Don't like this as is. Browser caching of these images is useful. I can see how it is a problem in the case where users change images frequently, but thats the special case, not normal case.

derjochenmeyer’s picture

Of Course the picture still gets saved in the browsers cache. The image still has a static name and as long a user does not change the picture it stayes in the cache. Here is an old proof of concept (still D5) which i presented in Szeged www.iwanttospeak.net ... Like this it only stores an additional information in the filename, which is the timestamp at the time of the upload. If the user stays with the picture forever it wil not change but if he decides to uplaod a differently cropped image only 10 sec later this patch makes sure he actually sees the new image (which he does not in all cases).

Our experience shows that users do change their images, not frequently, but they do. On sites like facebook i get frequent notifications that someone updated their profile image.

Drupal core uses a similar approach of dynamic filenames that indicate a change, e.g. with the css and js aggregation only with an md5 string.

We have a lot of users on different sites that complained about the picture upload having no effect until they did a hard browser reload (shift/cmd/strg+F5). We almost always use imagecache for displaying different versions of the user image.

moshe weitzman’s picture

Thanks for the clarification. I should have recognized that. This patch is useful as is. Perhaps a follow-up patch could fix this for all images by providing an option to file_create_url so that you could be a URL with the file upload timestamp appended.

derjochenmeyer’s picture

file_create_url: do you mean optionally appending something like this? >> path/filename.ext?timestamp

As far as I know the user pictures are the only images that are renamed by drupal?

moshe weitzman’s picture

Different modules do different things with file API. IMCE lets you browse your library of images and change images as needed. It would be nice if those changed images were immediately downloaded.

derjochenmeyer’s picture

Appending a timestamp to the URL of all kinds of images is a different issue, right?

Something like this path/filename.ext?timestamp would work. But how do we make sure that a module changes the files table entry with an updated timestamp and not just the image itself?

damien tournoud’s picture

Status: Needs review » Closed (won't fix)

This does not seem like a good way to do that. HTTP/1.1 has some very fine-grained cache control strategies. If you are using public file downloads, it's a simple matter of configuring your web server to issue an explicit Expire: header. This will avoid the heuristic expiration defined by HTTP/1.1:

Also, if the response does have a Last-Modified time, the heuristic
expiration value SHOULD be no more than some fraction of the interval
since that time. A typical setting of this fraction might be 10%.

(ie. "if your image has not been modified for 10 days, expiration will be set 1 day in the future)

I don't believe that this patch is needed at all.

damien tournoud’s picture

Because Drupal ships with a good example .htaccess, it is just a matter of changing the interval in:

# Requires mod_expires to be enabled.
<IfModule mod_expires.c>
  # Enable expirations.
  ExpiresActive On

  # Cache all files for 2 weeks after access (A).
  ExpiresDefault A1209600

  # Do not cache dynamically generated pages.
  ExpiresByType text/html A1
</IfModule>

You can change it to 1 hour (3600s) or less for picture users by adding some code inside the IfModule block.

moshe weitzman’s picture

Yes, separate issue is fine ... Modules that just change the filesytem directly are bypassing the API and will suffer for it just like the rest of drupal (e.g. not using node_save() and changing a term directly - no action gets fired.)

derjochenmeyer’s picture

Maybe there are better ways to avoid the browser cache problem, but not everyone knows how to use Etags or configure Expire headers. We want that more people use Drupal with less experience needed. I dont know how to configure a webserver this way. From helping out in beginner forums, i know that many drupal users (admins) often don't even know that the browsers cache can be an issue. Endusers dont care at all.

Why does Drupal core not name the aggregated CSS / JS files just drupal.css and drupal.js and uses a cryptic md5 filename instead? Why dont we just configure the webserver? Because it MATTERS that the correct files are provided to render the page every time. It seems the user itself has a lower priority.

From my experience users think this is a bug, because they tried changing their image with no effect.

On one intranet that i built the IT Departement thought Drupal is a buggy system. The first thing the employees tried to do is uploading an image and in some cases changing the picture several times. The IT department of this company couldnt find the "bug" :)

You can change it to 1 hour (3600s) or less for picture users by adding some code inside the IfModule block.

Users uplaod a picture, dont like it (or dont like how its cropped by imagecache) and change it. 3600s will not work. 5s will, but why should i deactivate the caching completely for all users pictures when all i need is an indicator that one file changed?

This patch solves a problem in a very straight forward way. I dont see any disadvantages of this approach.

Related Issues:
#360143: Implement cache-busting query for user pictures
#38171: Improve user picture caching

derjochenmeyer’s picture

Status: Closed (won't fix) » Needs review
sun’s picture

I was bitten by this today (again). The issue is that server directs the browser to cache images via mod_expires and not ask the server again for the same file during that time-frame.

I've researched whether a Files, Location, or Directory directive could work out to special-case the files/pictures directory, but that turned out to be a mess, because only Files is allowed in .htaccess.

Of course, user.module could put another .htaccess file into files/pictures to set a different expiration time or completely disable mod_expires -- but that would not fix the downstream issue with ImageCache.

The only viable solution was (actually) the same change as in this patch (without knowing this issue beforehand) - appending filemtime($picture->filepath).

I never had this issue with other files and images. Only regarding user pictures, innocent end-users do not understand (reasonably), why their user picture does not change after uploading a new one.

So... +1 for this patch.

catch’s picture

Why do we rename user pictures in the first place again?

j.somers’s picture

I believe this was done to explicitly ask for the new file before any cache alternative had been proposed.

catch’s picture

Sorry, I mean why do we rename pictures to picture-UID.jpg instead of just leaving the original filename?

j.somers’s picture

Status: Needs review » Active

After an IRC discussion I believe a better solution than appending the timestamp might be available.

Since the implementation of #357403: Move user picture to managed files there is no longer a need to rename the uploaded picture files to the format picture-uid.ext. I assume this has been done in previous Drupal releases since the only reference to the picture file was the filename and the storage of the filename in the users table.

If we keep the filename provided by the user who uploads the picture, and let the Drupal file API handle file-exists cases we don't need to modify the headers or append a timestamp to the file since it will already be unique. We can simply mark the previous user picture as temporary and have the files garbage collector remove it.

I will attempt to provide a patch later today when I have some time to fiddle around with it.

j.somers’s picture

Status: Active » Needs review
StatusFileSize
new2.15 KB

I attached a simple patch which does what I suggested earlier in comment #51.

Interesting to point out that the previous user picture is removed AFTER the current picture has been uploaded, thus it will not overwrite the file but rename it. I also used pathinfo() to extract the original filename without extension and append the extension retrieved from the image information to make sure that's still correct.

j.somers’s picture

Title: Add a timestamp to user pictures to avoid browser cache problems » Don't rename user pictures
webchick’s picture

We should benchmark this a bit. The nice thing about picture-UID.png is that it doesn't require a hit to the database. (Or at least I think so...)

Can we see before/after a node with 100 comments with user pictures enabled?

sun’s picture

@webchick: Wrong assumption. ;)

      // Multiple comment view.
      $query_count = 'SELECT COUNT(*) FROM {comment} c WHERE c.nid = %d';
      $query = 'SELECT c.cid as cid, c.pid, c.nid, c.subject, c.comment, c.format, c.timestamp, c.name, c.mail, c.homepage, u.uid, u.name AS registered_name, u.signature, u.picture, u.data, c.thread, c.status FROM {comment} c INNER JOIN {users} u ON c.uid = u.uid WHERE c.nid = %d';
webchick’s picture

Ah, great. :) Thanks, sun!

derjochenmeyer’s picture

The good thing about renaming the users picture is thats it looks a bit more "professional"

files/imagecache/user25/pictures/picture-1169-1241862265.jpg

compared to

files/imagecache/user25/pictures/at the beach%20in Ümlaute ßäüö (Mike)_2.jpg

Maybe (?) leaving the filename as it is could cause problems somewhere if special characters are used? User pictures are often the only files that users are allowed to contribute.

(just a personal feeling): I'd prefer renamed versions of the user pictures when building communities.

sun’s picture

@derjochenmeyer: Those characters are - technically - no problem. However, you are free to use http://drupal.org/project/transliteration to intercept all uploaded files and (automatically) convert their filenames to ASCII.

I guess I should have replied earlier: Removing this needless renaming is definitely the proper fix.

Patch looks good. However, bot seems to hang in re-testing (?).

stevenpatz’s picture

Status: Needs review » Active
stevenpatz’s picture

Status: Active » Needs review

Setting to needs review to get a bot test.

catch’s picture

Status: Needs review » Reviewed & tested by the community

Bot's green. RTBC.

dries’s picture

While webchick's assumption in #55 was wrong, it is still an interesting idea ... why wouldn't we do it like that and save some database queries?

I also agree with the comment in #57 -- it feels more clean to rename the picture, plus it avoids people playing games with URLs, etc.

Last but not least, it seems like I might be able to overwrite another user's picture now?

webchick’s picture

Status: Reviewed & tested by the community » Needs review

Still needs review.

j.somers’s picture

Status: Needs review » Needs work

It would indeed be possible to overwrite another user's picture. Didn't think of that.

We could still add the UID to the picture somehow, add it as a suffix to the original filename:

$destination = file_create_path(variable_get('user_picture_path', 'pictures') . '/' . $account->uid . '-' . $path_info['filename'] . '.' . $image_info['extension']);

Renaming the picture is indeed a good idea and it would, in some cases, allow you to save some queries, but I don't know whether or how we could really solve the problem otherwise.

catch’s picture

What's wrong with the _0 thing which upload module does?

derjochenmeyer’s picture

@Dries: #54 would mean that every user picture has also to be converted to a certain filetype. I dont see how those queries could be saved otherwise.

Whats wrong with the original idea to simply add the timestamp :)

  • its a simple modification/enhancement of the existing Drupal procedure.
  • it solves the problem described in #47 without changing the caching procedures for all image files (as with an .htaccess rule).
  • it ensures the users picture filenames to be formal/informational.
derjochenmeyer’s picture

Category: feature » bug
Status: Needs work » Needs review

In my opinion this is a bug that has to be fixed.

And I still think that patch #35 is the easiest solution for this.

Status: Needs review » Needs work

The last submitted patch failed testing.

dalin’s picture

Status: Needs work » Closed (duplicate)

I was about to work on this more only to discover that it was fixed in a duplicate issue
#668058: Resubmitting a user picture does show the first uploaded picture