Feeds should be case-insensitive while parsing headers

subscribing

same here

I was having this problem to. Looking at the headers array it looks like it's just a small typo and 'Content-Type' should be 'content-type'.

Changing this fixed the error for me. I've attached a patch against the latest dev version with this change.

#3
worked!

Change "Content-Type" to "content-type" in line 59.

Thank's

Why has this not been committed yet?

I see 2 reasons: this patch is not against head and no one has marked it RTBC. That is no one has the confidence to mark it as ready.

Lately I've seen a lot of this in the issue queue. An utterly simple patch that many people are likely applying, but no one is editing in the queue saying it works for them or marking it RTBC. We all need to step up on this if we're going to have any release candidates for drupal 7 contrib.

Here's some background on this, which may help.
http://drupal.org/node/156119
http://drupal.org/node/317

@johnbarclay Could you explain a bit what you mean in regards to the patch not being against head. The patch was created against the latest available dev version of Feeds, it was my understanding that it was preferred that patches be created against the latest dev version of a contrib module. Thanks!

You are correct to patch against the latest dev. I marked this as 7.x-2.x-dev.

Guys

I am still getting the error after changing Content-Type to content-type. Here is the error
Notice: Undefined index: content-type in http_request_get_common_syndication() (line 59 of C:\Program Files\BitNami Drupal Stack\apps\drupal\htdocs\modules\feeds\libraries\http_request.inc)

I am using 7.x-2.0-alpha4 version of Feeds module.

Please help

Thanks
M

I tried the 7.x-2.x-dev version and I still get the same error. I am using Drupal core 7.8

That's odd, are you able to post the address of the feed you are importing?

Peronas

Yes the feed link is http://api.eventful.com/rest/events/search?app_key=test_key&page_size=10...

Any help would be appreciated .

Thanks

So it looks like there might be more of an issue here than just a simple typo that initially fixed the problem for some. It seems that there is a lot of inconsistency in the casing when incoming feeds are setting the 'Content-Type' header. I was able to find multiple feeds that set the 'Content-Type' header in a different casing. Some set it as all lowercase, others use initial caps, and then some, such as the feed mnsweeps posted, come through as 'Content-type'.

Because of the way the headers are parsed into the headers array this varying casing is being preserved and causing problems when trying to retrieve the 'Content-Type' index.

According to RFC2616 it looks like the proper casing is 'Content-Type', so there's really not a typo problem in the way the module is accessing the index. We just need to find a way to normalize the incoming 'Content-Type' headers so we can consistently access the index.

Ideally the feeds coming in would set the 'Content-Type' header properly but it seems to be a common enough hiccup that Feeds should be elastic enough to handle it if it's going to rely on the 'Content-Type' value for anything.

Since the headers are being parsed from just a text chunk I'm thinking we could just run a case insensitive replace to make sure 'Content-Type' is cased correctly before it's parsed into the array.

Something like: $header = preg_replace('/content-type/i', 'Content-Type', $header); around line 178 of feeds\libraries\http_request.inc.

I'm currently stuck on a windows machine without git access, but I can roll a new patch in just a few hours.

Peronas

Thanks for detailed information. If you can roll out a quick fix for this I would really appreciate it. I am new to PHP too unfortunately so any help would be awsome !!

Thanks again buddy ! Once you roll out the fix please let me know and I will try it.

Thanks

Peronas

Also I manually changed code to use 'Content-type' and it works now but my problem is there are no records/nodes ingested. The link above returns me an XML which is not a RSS or ATOM feed per the provider. How should I handle this? The provider says I need to use a generalized XML parser. But I am not sure how should I go about that.

Thanks again

peronas

also I would be curious to know how you saw the feed i am using returning it as 'Content-type'. I used Firebug and saw response header as 'Content-Type' but I know I am wrong :-)

Ok attached is a patch against the latest dev, to do just a quick match and replace to make sure the 'Content-Type' header is cased properly.

If you've applied the earlier patch in this issue you'll want to revert it before applying this one. Also if anyone has feedback on possibly a better way to handle this please share!

@mnsweeps To see the headers that feed was sending I just used the devel module to print the headers to the screen before the Feeds module parsed them into the headers array. That way I could see specifically what the Feeds module was receiving.

As for the feed content itself I'm not quite sure what the provider was referring to in terms of a generalized XML parser. The common syndication parser can handle XML feeds, for example: http://drupal.org/planet/rss.xml, but as you noted Feeds doesn't seem to be able to pick up any items from the feed you provided.

In the interest of getting some input from someone who might understand the XML parsing better than me, and to keep this issue focused on the 'Content-Type' header issues, I would encourage you to search/submit an additional issue that's more specific to the importing issue you're having.

Hi,

First let me change the title to better match what the issue is about.

Then the problem is not only on line 59 but on every place where Feeds check headers. Like while checking ETag: Last-Modified, If-Modified-Since, etc ...
Because drupal_http_request() function is doing a strtolower() on every header (common.inc, line 933).
You can check the issue about this update here: #303838: drupal_http_request - case sensitive HTTP header field names
In summary, message header field names are case insensitive (see Message Headers in RFC 2616)

Is this still an issue after #1372074: feeds_http_request does not cache when using drupal_http_request.?

In the issue I link to, headers get normalized to lowercase.

This was fixed in the issue linked to in #19. If anyone is still having this issue, feel free to reopen.