I faced this peculiarity while using squid as caching proxy to minimize Internet traffic when downloading modules many times during auto-building.
Below are the headers returned for migrate project (and only for it!)
wget -S http://ftp.drupal.org/files/projects/migrate-7.x-2.2.tar.gz
--2011-11-14 13:19:18 GMT-- http://ftp.drupal.org/files/projects/migrate-7.x-2.2.tar.gz
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:3128... connected.
Proxy request sent, awaiting response...
HTTP/1.0 200 OK
Date: Mon, 14 Nov 2011 13:19:22 GMT
Server: Apache
Last-Modified: Sun, 18 Sep 2011 23:16:49 GMT
ETag: "68d48-43e4c3-4ad3f6bbdfa40"
Accept-Ranges: bytes
Content-Length: 4449475
Content-Type: application/x-gzip
X-Cache: MISS from localhost
X-Cache-Lookup: MISS from localhost:3128
Via: 1.0 localhost (squid/3.1.6)
Proxy-Connection: close
Length: 4449475 (4.2M) [application/x-gzip]
For other modules I get dates in past like
--2011-11-14 13:36:50 GMT-- http://ftp.drupal.org/files/projects/og_views-7.x-1.0.tar.gz
Date: Fri, 11 Nov 2011 10:27:46 GMT
for organic groups
or
--2011-11-14 13:35:52 GMT-- http://ftp.drupal.org/files/projects/views-7.x-3.0-rc1.tar.gz
Date: Mon, 14 Nov 2011 13:12:20 GMT
(the difference is small but it is)
I'm sure the proxy thinks of the content being too fresh because of Date=NOW+/-3 seconds
to take it from cache, and slows down by downloading it from Internet.
Comments
Comment #1
lex0r CreditAttribution: lex0r commentedBump :)
I have to download the biggest module ever - migrate - just because of impossibility to cache it.
Help me, plz!
Comment #2
killes@www.drop.org CreditAttribution: killes@www.drop.org commentedspoke to the OSL: The ftp servers don't have squid proxies so this is most likely an issue with your ISP.
Comment #3
lex0r CreditAttribution: lex0r commentedThat's nice, buy let me clarify the problem again.
I have squid installed in my LAN, not ISP or FTP server provider. And one single file is presumably served
as "the freshest never cacheable stuff!", that prevents squid from caching it properly and resulting in:
Comment #4
Damien Tournoud CreditAttribution: Damien Tournoud commentedLet me clarify.
That's the *processing date* on the server.
That's the date the object was last modified.
The headers you are showing look correct. You Squid proxy should use
Last-Modified
(and maybeETag
) to determine the cachability of the object.Comment #5
killes@www.drop.org CreditAttribution: killes@www.drop.org commentedHeh, just wanted to write the same. :p
Comment #7
lex0r CreditAttribution: lex0r commentedThis is my squid setting
refresh_pattern -i \.gz$ 43200 100% 43200 override-lastmod override-expire
Please note that it works in 99% of cases, but only one module spoils the bros. I'm not sure this is a squid problem, but looks like none will help me here, it's a dev forum, not sysadmins' one :)
Comment #8
lex0r CreditAttribution: lex0r commentedI believe I know the problem. The issue appeared only on dev releases of media module.
Dev = always fresh headers = no cache.
Comment #9
DSquaredB CreditAttribution: DSquaredB commentedClosing this issue since it has been inactive for quite some time. It can be reopened if the issue is still relevant.