I faced this peculiarity while using squid as caching proxy to minimize Internet traffic when downloading modules many times during auto-building.

Below are the headers returned for migrate project (and only for it!)

wget -S http://ftp.drupal.org/files/projects/migrate-7.x-2.2.tar.gz
--2011-11-14 13:19:18 GMT--  http://ftp.drupal.org/files/projects/migrate-7.x-2.2.tar.gz
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:3128... connected.
Proxy request sent, awaiting response... 
  HTTP/1.0 200 OK
  Date: Mon, 14 Nov 2011 13:19:22 GMT
  Server: Apache
  Last-Modified: Sun, 18 Sep 2011 23:16:49 GMT
  ETag: "68d48-43e4c3-4ad3f6bbdfa40"
  Accept-Ranges: bytes
  Content-Length: 4449475
  Content-Type: application/x-gzip
  X-Cache: MISS from localhost
  X-Cache-Lookup: MISS from localhost:3128
  Via: 1.0 localhost (squid/3.1.6)
  Proxy-Connection: close
Length: 4449475 (4.2M) [application/x-gzip]

For other modules I get dates in past like

--2011-11-14 13:36:50 GMT-- http://ftp.drupal.org/files/projects/og_views-7.x-1.0.tar.gz
Date: Fri, 11 Nov 2011 10:27:46 GMT
for organic groups

or
--2011-11-14 13:35:52 GMT-- http://ftp.drupal.org/files/projects/views-7.x-3.0-rc1.tar.gz
Date: Mon, 14 Nov 2011 13:12:20 GMT
(the difference is small but it is)

I'm sure the proxy thinks of the content being too fresh because of Date=NOW+/-3 seconds
to take it from cache, and slows down by downloading it from Internet.

Comments

lex0r’s picture

Bump :)
I have to download the biggest module ever - migrate - just because of impossibility to cache it.
Help me, plz!

killes@www.drop.org’s picture

Status: Active » Closed (won't fix)

spoke to the OSL: The ftp servers don't have squid proxies so this is most likely an issue with your ISP.

lex0r’s picture

Status: Closed (won't fix) » Active

spoke to the OSL: The ftp servers don't have squid proxies so this is most likely an issue with your ISP.

That's nice, buy let me clarify the problem again.
I have squid installed in my LAN, not ISP or FTP server provider. And one single file is presumably served
as "the freshest never cacheable stuff!", that prevents squid from caching it properly and resulting in:

  • unnecessary request that breaks the whole idea behind a proxy
  • broken autobuilds when Internet is down because no entry is found in cache
Damien Tournoud’s picture

Category: bug » support

Let me clarify.

  Date: Mon, 14 Nov 2011 13:19:22 GMT

That's the *processing date* on the server.

  Last-Modified: Sun, 18 Sep 2011 23:16:49 GMT

That's the date the object was last modified.

The headers you are showing look correct. You Squid proxy should use Last-Modified (and maybe ETag) to determine the cachability of the object.

killes@www.drop.org’s picture

Status: Active » Fixed

Heh, just wanted to write the same. :p

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

lex0r’s picture

Status: Closed (fixed) » Active

This is my squid setting
refresh_pattern -i \.gz$ 43200 100% 43200 override-lastmod override-expire
Please note that it works in 99% of cases, but only one module spoils the bros. I'm not sure this is a squid problem, but looks like none will help me here, it's a dev forum, not sysadmins' one :)

lex0r’s picture

I believe I know the problem. The issue appeared only on dev releases of media module.
Dev = always fresh headers = no cache.

DSquaredB’s picture

Status: Active » Closed (fixed)

Closing this issue since it has been inactive for quite some time. It can be reopened if the issue is still relevant.