Event feed doesn't work with Google Calendar nor iCal

Here's a copy of the ical feed. As you can see it contains non-English characters and I wonder whether they are correctly encoded and whether that could be why it doesn't work. Any ideas? Anyone who knows the iCal-spec?

BEGIN:VCALENDAR
VERSION:2.0
X-WR-CALNAME:KogVet | September 18, 2006 - October 18, 2006
PRODID:-//strange bird labs//Drupal iCal API//EN
BEGIN:VEVENT
DTSTART;VALUE=DATE-TIME:20061014T070000Z
DTEND;VALUE=DATE-TIME:20061018T123000Z
SUMMARY:NordiCHI 2006
DESCRIPTION:&quot\;NordiCHI is the main Nordic forum for human-computer interaction research. NordiCHI is the meeting place for researchers from academia and industry\, designers\, practitioners\, educators and others from a broad range of traditions and communities.&quot\; 
 
 I år hålls konferensen i Oslo. För studenter är priset €350 + ev workshops. Mer info på http":"//www.nordichi.org/2006
URL;VALUE=URI:http://www.kogvet.whileidle.com/kalender/nordichi_2006
UID:http://www.kogvet.whileidle.com/kalender/nordichi_2006
END:VEVENT
BEGIN:VEVENT
DTSTART;VALUE=DATE-TIME:20061015T130000Z
DTEND;VALUE=DATE-TIME:20061016T190000Z
SUMMARY:Sjöslaget
DESCRIPTION:Åk båt till finland!
URL;VALUE=URI:http://www.kogvet.whileidle.com/kalender/sjoslaget
UID:http://www.kogvet.whileidle.com/kalender/sjoslaget
END:VEVENT
END:VCALENDAR

From:
http://tools.ietf.org/html/rfc2445#page-10

3.4 Encoding Considerations

   This MIME content type can contain 8bit characters, so the use of
   quoted-printable or BASE64 MIME content-transfer-encodings might be
   necessary when iCalendar objects are transferred across protocols
   restricted to the 7bit repertoire. Note that a text valued property
   in the content entity can also have content encoding of special
   characters using a BACKSLASH character (US-ASCII decimal 92)
   escapement technique. This means that content values can end up
   encoded twice.

Thanks Karen! I'll try this!

Sure thing! Thanks Karen and Killes for your time and efforts!

Hoolabaloola! It seems to work! I've confirmed it works with Google Calendar.

Here's the output:

BEGIN:VCALENDAR
VERSION:2.0
X-WR-CALNAME:KogVet | September 19, 2006 - October 19, 2006
PRODID:-//strange bird labs//Drupal iCal API//EN
BEGIN:VEVENT
DTSTART;VALUE=DATE-TIME:20061014T070000Z
DTEND;VALUE=DATE-TIME:20061018T123000Z
SUMMARY;ENCODING=QUOTED-PRINTABLE:NordiCHI 2006
DESCRIPTION;ENCODING=QUOTED-PRINTABLE:&quot\;NordiCHI is the main Nordic forum for human-computer interaction rese=
arch. NordiCHI is the meeting place for researchers from academia and indus=
try, designers, practitioners, educators and others from a broad range of t=
raditions and communities.&quot\;=20=0D=0A=0D=0AI =C3=A5r h=C3=A5lls konferensen i Oslo. F=C3=B6r studenter =C3=A4r priset =
=E2=82=AC350 + ev workshops. Mer info p=C3=A5 http\://www.nordichi.org/2006
URL;VALUE=URI:http://www.kogvet.whileidle.com/kalender/nordichi_2006
UID:http://www.kogvet.whileidle.com/kalender/nordichi_2006
END:VEVENT
BEGIN:VEVENT
DTSTART;VALUE=DATE-TIME:20061015T130000Z
DTEND;VALUE=DATE-TIME:20061016T190000Z
SUMMARY;ENCODING=QUOTED-PRINTABLE:Sj=C3=B6slaget
DESCRIPTION;ENCODING=QUOTED-PRINTABLE:=C3=85k b=C3=A5t till Finland!
URL;VALUE=URI:http://www.kogvet.whileidle.com/kalender/sjoslaget
UID:http://www.kogvet.whileidle.com/kalender/sjoslaget
END:VEVENT
END:VCALENDAR

Thanks a million Karen! Great work! Killes, I'm so happy we can finally harness the full power of this module!

Apparently my initial report was wrong. Google Calendar shows an error message when I try to add this iCal-calendar:
webcal://www.kogvet.se/event/ical/all/all

It may be the character encoding or it may be something else. I stripped the calendar of any non-English chars before and that was when it worked. I didn't realize that until later. Now the character encoding may actually be wrong here, I don't know. Any theories?

Follow-up: I've attached a screenshot of the error message I get at Google Calendar when trying to add the ical feed as a public calendar. The error message is a change because before the patch et c, when the calendar was added there was no error message, but nothing showed either.

I wish I knew what doesn't work. Your text-encoding was definitely needed but like you wrote, there's something else going on here. I've attached a copy of the feed which Google Calendar cannot parse. I tried opening it in Sunbird and Lightning but couldn't, not sure if it was because of the feed itself or them not supporting the format.

Been looking around for a spec and a way to validate the feed but haven't come up with anything. Maybe someone else knows?

Good documentation of the format: http://en.wikipedia.org/wiki/ICalendar

I have a theory... iCal uses CRLF to separate data, but several of our posts have linebreaks in then. Karen, can you BASE64 code these CRLFs so that the file doesn't break the standard?

Here's an example:

DESCRIPTION;ENCODING=QUOTED-PRINTABLE:<p>Nu =C3=A4r det dags f=C3=B6r f=C3=B6retagsseminarium! Denna g=C3=A5ng ha=
r SAKS (Sektionen f=C3=B6r Systemvetenskap) bjudit in Oracle som kommer att=
 prata om vad de g=C3=B6r nu, vad de erbjuder f=C3=B6r tj=C3=A4nster, hur d=
essa fungerar och vad de har f=C3=B6r planer inf=C3=B6r framtiden.</p><p>De=
 kommer sj=C3=A4lvklart =C3=A4ven svara p=C3=A5 fr=C3=A5gor fr=C3=A5n oss s=
tudenter.</p><p>Seminariet kommer att h=C3=A5llas i C1 (I Colloseum, C-huse=
t) den 28 september, kl. 17\:00 och =C3=A4r gratis!</p><p>F=C3=B6r att anm=
=C3=A4la dig ska du g=C3=A5 in p=C3=A5 adressen\:<br />http\://www.saksaren.s=
e/oracle.php</p><p>och anv=C3=A4nd dessa inloggningsuppgifter\:</p><p>User N=
ame\: saks<br />Password\: saksare</p>
URL;VALUE=URI:http://www.kogvet.se/kalender/oracleseminarium
UID:http://www.kogvet.se/kalender/oracleseminarium
END:VEVENT

should be:

DESCRIPTION;ENCODING=QUOTED-PRINTABLE:<p>Nu =C3=A4r det dags f=C3=B6r f=C3=B6retagsseminarium! Denna g=C3=A5ng ha=r SAKS (Sektionen f=C3=B6r Systemvetenskap) bjudit in Oracle som kommer att= prata om vad de g=C3=B6r nu, vad de erbjuder f=C3=B6r tj=C3=A4nster, hur d=essa fungerar och vad de har f=C3=B6r planer inf=C3=B6r framtiden.</p><p>De= kommer sj=C3=A4lvklart =C3=A4ven svara p=C3=A5 fr=C3=A5gor fr=C3=A5n oss s=tudenter.</p><p>Seminariet kommer att h=C3=A5llas i C1 (I Colloseum, C-huse=t) den 28 september, kl. 17\:00 och =C3=A4r gratis!</p><p>F=C3=B6r att anm==C3=A4la dig ska du g=C3=A5 in p=C3=A5 adressen\:<br />http\://www.saksaren.s=e/oracle.php</p><p>och anv=C3=A4nd dessa inloggningsuppgifter\:</p><p>User N=ame\: saks<br />Password\: saksare</p>
URL;VALUE=URI:http://www.kogvet.se/kalender/oracleseminarium
UID:http://www.kogvet.se/kalender/oracleseminarium
END:VEVENT

I tried changing it to HTML to rid if of the additional CRLFs, replacing them with
but it didn't seem to work. I'll need a code solution to this.

Here's an exampel of a wel-formed entry:

    BEGIN:VCALENDAR
    VERSION:2.0
    PRODID:-//ABC Corporation//NONSGML My Product//EN
    BEGIN:VJOURNAL
    DTSTAMP:19970324T120000Z
    UID:uid5@host1.com
    ORGANIZER:MAILTO:jsmith@host.com
    STATUS:DRAFT
    CLASS:PUBLIC
    CATEGORY:Project Report, XYZ, Weekly Meeting
    DESCRIPTION:Project xyz Review Meeting Minutes\n
     Agenda\n1. Review of project version 1.0 requirements.\n2.
    Definition
     of project processes.\n3. Review of project schedule.\n
     Participants: John Smith, Jane Doe, Jim Dandy\n-It was
      decided that the requirements need to be signed off by
      product marketing.\n-Project processes were accepted.\n
     -Project schedule needs to account for scheduled holidays
      and employee vacation time. Check with HR for specific
      dates.\n-New schedule will be distributed by Friday.\n-
     Next weeks meeting is cancelled. No meeting until 3/23.
    END:VJOURNAL
    END:VCALENDAR

A simple string replacement should do the job. I'm gonna try it.

After changing the code back, using the replacement function in ical.inc to replace CRLFs (\r\n) with \\n, it works, at least in Sunbird. It likes the .ics files now, it imports it right but it doesn't recognize or get the character encoding right.

Google Calendar can't even parse it now.

According to my friend who runs iCal on a Mac with a Swedish version of Mac OS X, he can import and subscribe to the ical feed, but the characters do not appear as they are instead he sees the encoded characters instead of the actual. Apparently iCal fails to recognize the encoding.

Lots of posts today! :)

Anyway, line breaks should be okay now that I replaced all "\r\n" with "\\n". Character encoding may be the actual problem here, issues with UTF-8 et c.

Found this which may be of some help:
http://phpicalendar.net/forums/archive/index.php/t-1252.html

Also worth looking into for code and ideas:
http://phpicalendar.net

Strange, it works now with Google Calendar as well but with one problem, it shows characters in BASE64, doesn't get the encoding right either so I think the BASE64 encoding Karen implemented needs tweaking. Tomorrow I'm gonna look at how phpicalendar does it.

After investigating by
1) adding the feed to Google Calendar, seen it work except for the BASE 64 encoding, and also learnt that GCal doesn't update very frequently which is why I haven't been able to see the results of my changes right away
2) and used an application called eventSherpa to try parsing the feeds
3) and comparing the ical feed from my Google Calendar to the one generated by the event module and the RFC and comments found in Wikipedia

I have three theories:

- The RFC says lines should be terminated by CRLF, in escaped chars that's \r\n, the event module terminates the lines with \n, I am now rewriting it to follow the RFC.

- The RFC also says that CRLFs and CRs within fields should be replaced by \n, this is what Google Calendar does and eventSherpa parses this feed correctly

- By comparing a google ics file with the events module ics file I've noticed there are discrepancies what fields are concerned, fields determing time zone and calendar (gregorian or julian) to mention two. Whether these fields are required for an ical feed to be parsed correctly by most calendar applications, I cannot say, but missing fields could be one reason.

- Character encoding doesn't seem to be done with BASE 64, Google uses regular UTF-8 and it works with eventSherpa, which is what the event module does without Karen's code

So there's a lot of detective work left to do. I'll have to read the RFC more carefully now.

I've attached the ics from Google Cal with my calendar with a demo event with non-English chars and linebreaks, this ical feed IS parsed by eventSherpa.

I've also attached an ics from our site (see next post), generated by the event module with my modifications, this feed is NOT parsed by eventSherpa.

Comments and ideas are most welcome!

This is the second file, the one that DOES NOT parse.

Here are the error messages iCal generates with the feed I've got now (the ICS attached above):

2006-09-22 13:53:34.902 iCal[375] Date validation error: DTSTART = '19700329T020000
'
2006-09-22 13:53:34.903 iCal[375] iCalendar recurrence failure FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
line 1:33: unexpected char: 0x0D
2006-09-22 13:53:34.903 iCal[375] Date validation error: DTSTART = '19701025T030000
'
2006-09-22 13:53:34.903 iCal[375] iCalendar recurrence failure FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
line 1:34: unexpected char: 0x0D
2006-09-22 13:53:34.903 iCal[375] Date validation error: DTSTART = '20060928T150000Z
'
2006-09-22 13:53:34.903 iCal[375] Date validation error: DTEND = '20060928T170000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTSTART = '20060928T200000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTEND = '20060928T200000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTSTART = '20061005T170000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTEND = '20061005T170000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTSTART = '20061005T200000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTEND = '20061005T200000Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTSTART = '20061009T081500Z
'
2006-09-22 13:53:34.904 iCal[375] Date validation error: DTEND = '20061009T100000Z
'
2006-09-22 13:53:34.905 iCal[375] Date validation error: DTSTART = '20061010T131500Z
'
2006-09-22 13:53:34.905 iCal[375] Date validation error: DTEND = '20061010T160000Z
'
2006-09-22 13:53:34.905 iCal[375] Date validation error: DTSTART = '20061014T070000Z
'
2006-09-22 13:53:34.905 iCal[375] Date validation error: DTEND = '20061018T123000Z
'
2006-09-22 13:53:34.905 iCal[375] Date validation error: DTSTART = '20061015T130000Z
'
2006-09-22 13:53:34.905 iCal[375] Date validation error: DTEND = '20061016T190000Z
'
2006-09-22 13:53:34.905 iCal[375] Component boundaries mismatch (VCALENDAR VCALENDAR
)

Apple's documentation, may be of use:
http://developer.apple.com/internet/appleapplications/icalendarfiles.html