Text fields in iCal feeds not escaped properly

Hi,

I downloaded (with wget) the iCalendar feed from groups.drupal.org and got this (just a snippet of one of the events):

BEGIN:VEVENT
DTSTART;VALUE=DATE-TIME:20060513T140000Z
DTEND;VALUE=DATE-TIME:20060514T210000Z
SUMMARY:DrupalCamp NYC - 13 & 14 of May
DESCRIPTION:On 13 & 14 of May we'll be dorking out at the Polytechnic University Brooklyn Campus. if you are interested in attending or would like to RSVP, add your name to this page.
 
 Polytechnic University Brooklyn Campus
 Six MetroTech Center
 Brooklyn, NY 11201
 http://www.poly.edu/directions/
 
 saturday and sunday schedule
 10a - 1p introduction sessions
 1p - 2p food break
 2p - 6p afternoon session
 
 DrupalCamp NYC will be broken down into three Workshops...
 
 introduction to drupal - (saturday only)
 this is for newbees/novice drupal users who would like to learn about drupal. what is drupal? how to install, configure, and which modules to use. how to utilize the support community. where to find vendors and custom drupal work. adam mordecai, from advomatic, will be spearheading this workshop.
URL;VALUE=URI:http://groups.drupal.org/node/41
UID:http://groups.drupal.org/node/41
END:VEVENT

Checking RFC2445 (Internet Calendaring and Scheduling Core Object Specification, iCalendar, http://www.ietf.org/rfc/rfc2445.txt), especially the following section on escaping human readable content, it seems to me that geodan is on the right track.

The following snippet is the definition of the element #text used to represent text/data associateted with iCalendar properties like SUMMARY, DESCRIPTION and so on. The snippet is taken from the Wikipedia version of RFC2445 as it is more webfriendly. Source: http://en.wikipedia.org/wiki/RFC2445_Syntax_Reference#Text

Text

Used to identify values that contain human readable text.

    * For double quote use: DQUOTE
    * For a comma use: \,
    * For a colon use: ":"
    * For a semicolon use: \;
    * For a back-slash use \\
    * For a new line use \n or \N

It looks like none of these escapes are used by the events module when generating #text fields for the iCalendar feed.

I have not checked if there are other deviations from the iCalendar specification.

This issue is a dublicate of the Google Calendar issue at http://drupal.org/node/58554, but the subject of this issue is more to the point.

Regards,

Christian Larsen

I don't understand too much about iCal so I just applied the patch. :p

Is this committed only to CVS or is it also available in the current stable (4.7.0) version of the Event module?

We need this feature to work. It also seems our calendar feed doesn't work in iCalendar either.

Not sure where else to post this: I run a PHP-based calendar server that also accepts and parses iCalendar feeds. One of my clients is using Drupal, and attempted to attach their feed to my server, which promptly munged much of the HTML in many of the fields. So I did some digging, and discovered the "DQUOTE" referred to in this bug report.

It appears that Drupal is actually using the text string "DQUOTE" in its code, to represent the double-quote character. This is a complete misunderstanding of RFC 2445! The string "DQUOTE" used in RFC 2445 is actually a definition, not a substitution string!

Section 4.1 of the RFC describes "contentlines", and uses a series of definitions to describe the format that text needs to follow. In places where they refer to DQUOTE, it is used like this:

The Common Name parameter follows this format: "CN" "=" param-value.
In the definition that explains how "param-value" is formatted, we see:
param-value = paramtext OR quoted-string
quoted-string = DQUOTE *QSAFE-CHAR DQUOTE
DQUOTE = %x22

So they really mean that a Common Name should look like: CN="John Smith"
A DQUOTE character, followed by one or more QSAFE characters, followed by a DQUOTE character.

The Drupal code for escaping characters says:
* For double quote use: DQUOTE
which is totally incorrect. Double-quotes are escaped as \" in the same manner as single-quotes.

On a related note, RFC 2445 recommends that contentlines "should" be folded, so that they are no more than 75 octets in length, and the Drupal feeds don't appear to do that. Not required, but it's also nice to stick closely to the spec whenever possible.

I have simple fold() and unfold() functions in PHP, which I'd be happy to contribute.

Any chance the DQUOTE issue could be resolved, at least? I'm not a Drupal user or contributor, so I'm not familiar with your development process, but if I could be of help, let me know.

Here's a replacement for the patch file posted earlier. I didn't have access to the original source, so I simply modified the existing patch file. Line counts stay the same. The new patch simply replaces one str_replace() call with another.

Related to other aspects of this discussion, I do NOT use quoted-printable encoding for my calendar feeds (http://www.radicalendar.org), and it handles UTF-8 just fine. There are occasional Hebrew, Spanish, and Japanese feeds on the site, so it gets a good workout.

The question of whether or not to strip HTML tags from the feeds is another matter: if the feed is displayed on another web calendar, the HTML comes through nicely. If the feed is destined for Apple's iCal, the HTML tags are rendered as plain text, which ends up looking ugly. The only good solution may be to offer HTML and plain text versions of the feeds.

And here's a patch for anyone who had already used the earlier patch. A patch patch.

I am a bot confused: Which is the orrect patch? #12? or #12 and #13?

Did anybody try any of these patches?

I have been trying to get two-way ical feeds working and in the process I think I have the ical.inc file working correctly (or at least working so I can export feeds into Google and import them into my own applications).

The patch must have some line ending issues because it looks messier than it is. The changes are:

1) $calname needed to be escaped because it had commas in it (April 1, 2007 to April 2, 2007)

2) Quoted printable was, in fact, not necessary, so I removed it and used the ical escape instead

3) It was not necessary to do strip_tags() so I commented it out (I left it in as a reminder in case there are any more problems later).

4) The ical escaping needed the above-mentioned fix to escape the quote marks rather than use QUOTE

5) The colon needed regular escaping, not the funny stuff with the quotes around it

6) The _ical_tz($tz) function was completely screwed up (which wasn't apparent until I tried to import a feed). It is doing a db_query on a non-existant table. I fixed it to use the timezones.inc file instead.

I tested an event with non-ascii characters and single and double quotes in both the title and the description and was able to get the feed to show up in Google Calendar. I was able to use the import part of the file to import feeds into the Calendar module (a demo is at http://www.elderweb.com/demo/calendar), so I have feeds working in both directions. Sorry, I don't have code to do the import into the event module calendar, but now that it is working someone should be able to do that (or I may try to do it when I have more time).

The previous patch had some date module stuff still in it. Here's the one for the event module.

Thanks! applied

Just thought I'd report that since this patch export to Google works just great for me - thanks Karen!

It's possible this is just iCal4j not behaving properly, but my http://severinghaus.org/projects/icv/">iCalendar validator is complaining about the blank lines that are now showing up. I seem to recall having trouble with the same phenomenon in Sunbird, but that was a year or two ago, and I may be wrong.

I don't see anything in RFC 2445 §4.1 ("Content Lines") saying it's specifically illegal to have blank lines, but the examples don't appear to have any.

In any case, I went ahead and trim()ed the return in ical_escape_text() (@@ 80), and it seems fine now. Any thoughts?