I hate long URLs, but I've just had to work with a feed URL longer than 255 chars, and I was getting two warnings: user warning: Duplicate entry 'http://www.domain.com/rss.cgi?' for key 1 query: INSERT INTO parser_common_syndication (etag, last_modified, url) VALUES ('', '', 'http://www.domain.com/rss.cgi?a=1&b=2&c=3&{$url_longer_than_255_chars}') in /sites/all/modules/feedapi/parser_common_syndication/parser_common_syndication.module on line 237.
That's why I discovered that the url is trimmed to 255 chars when stored into the {parser_common_syndication} table, because its url field is defined as a varchar(255) field.
Thus, the SELECT query (in the _parser_common_syndication_feedapi_get() function) fails, the $db_result var is empty, the $has_etag var is never set to TRUE, the module will try to run the INSERT INTO query again, resulting in those Duplicate entry warnings, since both $result->headers['ETag'] and $result->headers['Last-Modified'] are empty.
What's the solution to this problem?
Well, RFC 2068 does not specify any maximum URL length.
MS Internet Explorer is known to support up to 2083 chars.
While better browsers can handle also much longer URLs.
("yes, Microsoft's web server accepts longer URLs than Microsoft's web browser" - LOL)
If it's not a problem, I think it would be useful to edit the structure of the table, and set the url field at least to varchar(512), varchar(1024) or varchar(2048).
Greetings, Giovanni
ps. moreover, when both $result->headers['ETag'] and $result->headers['Last-Modified'] are empty, the UPDATE query is uselessly executed.
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | csp_long_urls.patch | 4.27 KB | aron novak |
Comments
Comment #1
aron novakYou're right. FeedAPI itself ( {feedapi} ) handles this question correctly (text field).
I see one problematic point here:
url is a primary key: http://bytes.com/groups/mysql/141591-trying-make-text-column-primary-key
Maybe one solution is possible:
instead of storing the URL, let's store its sha1() . Do you think it's acceptable?
Comment #2
aron novakComment #3
Antinoo commentedI've tested the patch with a long URL feed and it works fine.
Setting this as "needs review" only because I'm not "the community". :-)
Thank you, Aron.
Comment #4
aron novakhm. Be confident, you're the community :)
Comment #5
aron novakI just committed this one now.