KarenS, community, I apologize for this interruption. I have spent 2 days trying to troubleshoot this and am at my wit's end.

When importing my google calendar iCal feed, everything works gloriously... except for one event which is multiple "all days".

The date start is Oct 23, 2009 12:00am
the date end is Oct 23, 2009 12:00am

...in reality the event goes until Oct 25. So, "to date" is not being imported correctly. I know this isn't a bug, I just don't know how to do it and I'm on the verge of a breakdown here feeling that I've exhausted all of my searching capabilities to find the answer.

Thanks for any insight.

CommentFileSizeAuthor
#16 date_api_ical.patch2.6 KBjohnmunro
#16 theme.patch716 bytesjohnmunro
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Branjawn’s picture

I have figured out that some events import with DTSTART;VALUE=DATE:20090417 and others have only DTSTART:20081122T000000Z. Could this be messing things up?

Branjawn’s picture

More info, read this somewhere:

The default value type for DTSTART is DATE-TIME, but
20060315 is not a DATE-TIME; you can use DATE values in
DTSTART, but you have to be explicit:

DTSTART;VALUE=DATE:20060315

So, I'm trying to import the same field DTSTART as Date and Datetime? Could that be screwing things up? If so, there is nothing I can do about it right? Because the feed is generated by Google Calendar and I have no control over that...

Branjawn’s picture

Component: Documentation » Code
Category: support » bug
ekes’s picture

OK it sounds like the feed isn't doing anything wrong from what I understand of what you're saying.

Any chance you could put a copy of the feed, or point me to the feed, somewhere? I'll give a test out from there.

johnmunro’s picture

I have a sample for you. For the following valid test ICS:

BEGIN:VCALENDAR
PRODID:Zimbra-Calendar-Provider
VERSION:2.0
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:(GMT+09.00) Osaka / Sapporo / Tokyo
BEGIN:STANDARD
DTSTART:19710101T000000
TZOFFSETTO:+0900
TZOFFSETFROM:+0900
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:FM696
SUMMARY:Herbstferien
DESCRIPTION:...
ORGANIZER;SENT-BY="mailto:wels@domain.jp":mailto:events@domain.jp
X-MS-OLK-SENDER:mailto:wels@domain.jp
DTSTART;VALUE=DATE:20091026
DTEND;VALUE=DATE:20091031
STATUS:CONFIRMED
CLASS:PUBLIC
X-MICROSOFT-CDO-ALLDAYEVENT:TRUE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
TRANSP:OPAQUE
X-MICROSOFT-DISALLOW-COUNTER:TRUE
DTSTAMP:20090909T020325Z
SEQUENCE:50
END:VEVENT
END:VCALENDAR

As per the example given in the 'Map' tab, this gets parsed as:

Array
(
    [title] => Herbstferien
    [description] => ...
    [options] => Array
        (
            [original_author] => 
            [location] => 
            [original_url] => 
            [timestamp] => 1252461805
            [guid] => FM696
            [tags] => 
            [VEVENT] => Array
                (
                    [DATE] => Array
                        (
                            [DTSTART] => Array
                                (
                                    [datetime] => 2009-10-26
                                    [all_day] => 1
                                    [tz] => 
                                )
                            [DTEND] => Array
                                (
                                    [datetime] => 2009-10-26
                                    [all_day] => 1
                                    [tz] => 
                                )
                        )
                    [UID] => FM696
                    [SUMMARY] => Herbstferien
                    [DESCRIPTION] => ...
                    [ORGANIZER] => wels@dsty.ac.jp":mailto:events@domain.jp
                    [X-MS-OLK-SENDER] => mailto:wels@domain.jp
                    [STATUS] => CONFIRMED
                    [CLASS] => PUBLIC
                    [X-MICROSOFT-CDO-ALLDAYEVENT] => TRUE
                    [X-MICROSOFT-CDO-INTENDEDSTATUS] => BUSY
                    [TRANSP] => OPAQUE
                    [X-MICROSOFT-DISALLOW-COUNTER] => TRUE
                    [DTSTAMP] => Array
                        (
                            [datetime] => 2009-09-09 02:03:25
                            [all_day] => 
                            [tz] => UTC
                        )
                    [SEQUENCE] => 50
                    [all_day] => 1
                )
        )
)

As you can see the DTEND array is incorrect. The [datetime] seems to be coming from the DTSTART value as
[datetime] => 2009-10-26
It should be:
[datetime] => 2009-10-31

johnmunro’s picture

Further to the above, changing the date info from:

DTSTART;VALUE=DATE:20091026
DTEND;VALUE=DATE:20091031

to:

DTSTART;VALUE=DATE:20091026T000000
DTEND;VALUE=DATE:20091031T000000

has no effect.

However changing it to:

DTSTART:20091026T000000
DTEND:20091031T000000

works.

But

DTSTART:20091026
DTEND:20091031

does not work.

We need to note that DTSTART;VALUE=DATE:20091026 is valid and that one ics file may have a variety of formats depending on the actual event and (more importantly) the client that created the event.

johnmunro’s picture

The relevant code in parser_ical.module is:

    // Keep iCal timezone information in the feed item so we can create the right date value.
    $date_info = array('DTSTART', 'DTEND', 'RDATE', 'EXDATE', 'DURATION', 'RRULE');
    foreach ($date_info as $key) {
      if (isset($event[$key])) {
        $item->options->VEVENT['DATE'][$key] = $event[$key];
        unset($event[$key]);
      }
    }

This seems perfectly clear that the correct information is being passed back to the caller of the 'parser_ical_feedapi_feed' hook. so the misinterpretation of the DTEND must be in the date_api or feed_api module or feedapi_mapper. Correct?

johnmunro’s picture

Project: iCal feed parser » Date
Version: 6.x-1.1 » 6.x-2.4

It appears the problem occurs before the code in #7 is run. The $event array contains the following at runtime:

DTSTART:
Array ( [datetime] => 2009-10-26 [all_day] => 1 [tz] => ) 
DTEND:
Array ( [datetime] => 2009-10-26 [all_day] => 1 [tz] => ) 

In fact on line 82, $feed_folded has DTEND;VALUE=DATE:20091031 [20]
whereas after
$ical_parsed = date_ical_parse($feed_folded);
$ical_parsed has: [DTEND] => Array ( [datetime] => 2009-10-26 [all_day] => 1 [tz] => )

This narrows the bug to within the date_ical_parse function in the date_api_ical.inc file of date module so moving this bug report to the date module.

johnmunro’s picture

Component: Code » iCal API
johnmunro’s picture

Found it! Problem lies in the logic of the 'if' statement on line 203 of date_api_ical.inc

            if (!empty($subgroup['DTSTART']) && (!empty($subgroup['DTSTART']['all_day']) || 
            (empty($subgroup['DTEND']) && empty($subgroup['RRULE']) && empty($subgroup['RRULE']['COUNT'])))) {
              // Many programs output DTEND for an all day event as the
              // following day, reset this to the same day for internal use.
              $subgroup['all_day'] = TRUE;
              $subgroup['DTEND'] = $subgroup['DTSTART'];
            }

The IF condition is assuming 'all_day' means only one day. Bad assumption. Can someone suggest better condition code?

              // Many programs output DTEND for an all day event as the
              // following day, reset this to the same day for internal use.
johnmunro’s picture

Title: 'To date' not importing » Multiple day VEVENT not working from iCal
Version: 6.x-2.4 » 6.x-2.x-dev
Assigned: Unassigned » johnmunro

I see dev still has the same code so I will do some follow up and get some better code for this case.

The case is where the iCal generator (Google calendar, Zimbra calendar, and others) produces output with DTEND one day following DTSTART. Following some research, this is correct. From http://www.ietf.org/rfc/rfc2445.txt (Page 52)

   The "DTSTART" property for a "VEVENT" specifies the inclusive start
   of the event. For recurring events, it also specifies the very first
   instance in the recurrence set. The "DTEND" property for a "VEVENT"
   calendar component specifies the non-inclusive end of the event.

My interpretation of this is that if there is no time specified then the actual end date is DTEND minus one day. How does one do this in PHP? That is:

            if (!empty($subgroup['DTSTART']) && (!empty($subgroup['DTSTART']['all_day']) ||
            (empty($subgroup['DTEND']) && empty($subgroup['RRULE']) && empty($subgroup['RRULE']['COUNT'])))) {
              // Many programs output DTEND for an all day event as the
              // following day, reset this to the same day for internal use.
              $subgroup['all_day'] = TRUE;
              if (!empty($subgroup['DTEND']) {
                $subgroup['DTEND'] = $subgroup['DTEND'] MINUS ONE DAY;
              }
              else {
                $subgroup['DTEND'] = $subgroup['DTSTART'];
              }
            }

I am a bit of a newb in PHP, sorry. How does one best do $subgroup['DTEND'] = $subgroup['DTEND'] MINUS ONE DAY; taking into account month/year ends, leap years, etc?

I have been following this for more than a year and am really happy to see we are almost there. Thanks Karen for the near perfect iCal solution you have managed on this.

johnmunro’s picture

The logic of the above condition is:

IF a start date is defined 
  AND IF  ( start date is defined as ALL_DAY )   OR   ( there is no definition for the end date )
    THEN make the end date equal the start date
         define the entire event as a single all day event

This should be:

// iCal spec states 'The "DTEND" property for a "VEVENT" calendar component specifies the non-inclusive
// end of the event'
// This is fine for hours such as 11:00~12:00 means one hour, not two but
// for days the current calendar code assumes the end date is inclusive so Oct 30 ~ Oct 31 is interpreted
// as two days, not one. So make an adjustment to account for this difference in logic
IF an end date is defined  AND  end date is defined as ALL_DAY )
    THEN make the end date one day earlier

IF a start date is defined  AND  there is no definition for the end date 
    THEN make the end date equal the start date
         define the entire event as a single all day event

Karen, I would have suggested that the calendar code logic be adjusted to be the same as the iCal logic, but this may cause too many other things to break, not to mention user confusion if the granularity is set to y/m/d with no time. So the above may be the best solution?

Again, how do we make the end date one day earlier taking into account month ends, year ends, leap years, etc?

johnmunro’s picture

Ok, thanks for the rush to help with my basic newb date question, but I believe I have found the solution as:

$subgroup['DTEND']['datetime'] = date( 'Y-m-d', date_convert($subgroup['DTEND']['datetime'], DATE_DATETIME, DATE_UNIX) - 86400);

If there is a faster or more elegant solution let me know, otherwise the solution to this bug seems to be the following code from line 198 of date_api_ical.inc replacing the single if statement with the following two:

            // iCal spec states 'The "DTEND" property for a "VEVENT" calendar component specifies the
            // non-inclusive end of the event'
            // This is fine for hours such as 11:00~12:00 means one hour, not two but
            // for whole days the current calendar code assumes the end date is inclusive so Oct 30 ~ Oct 31 is
            // interpreted as two days, not one. So make an adjustment to account for this difference in logic
            if (!empty($subgroup['DTEND']) && (!empty($subgroup['DTEND']['all_day']))) {
              // make the end date one day earlier
              $subgroup['DTEND']['datetime'] = date( 'Y-m-d', date_convert($subgroup['DTEND']['datetime'], DATE_DATETIME, DATE_UNIX) - 86400);
            }
            // Add a top-level indication for the 'All day' condition.
            // Leave it in the individual date components, too, so it
            // is always available even when you are working with only
            // a portion of the VEVENT array, like in Feed API parsers.
            $subgroup['all_day'] = FALSE;
            // if a start datetime is defined  AND  there is no definition for the end datetime
            //  THEN make the end datetime equal the start datetime
            //  and if it is an all day event define the entire event as a single all day event
            if (!empty($subgroup['DTSTART']) &&
               (empty($subgroup['DTEND']) && empty($subgroup['RRULE']) && empty($subgroup['RRULE']['COUNT']))) {
              $subgroup['DTEND'] = $subgroup['DTSTART'];
              if (!empty($subgroup['DTSTART']['all_day'])) {
                $subgroup['all_day'] = TRUE;
              }
            }

I am not able to know the purpose of $subgroup['all_day'] = TRUE; and whether it should also be applied to the multi-day events or not for the benefit of Calendar, but the above code works for me.

Can someone else please test this too?

Branjawn’s picture

Thanks for all the work being done here to correct the problem.

johnmunro’s picture

In a highly related issue the following code needs to be inserted in theme/theme.inc at line 385 just after $end = date_format($end_date, DATE_FORMAT_DATETIME);

  if (substr($end, -8) == "00:00:00") {
    // it is an all day event so make the end date one day later to satisfy the 'Ongoing' condition
    $end = date_convert(date_convert($end, DATE_DATETIME, DATE_UNIX) + 86400, DATE_UNIX, DATE_DATETIME);
  }

This is not just an iCal issue, but is relevant due to the unbalanced logic of the end date being inclusive for all day events verses non-inclusive for everything else (hours, minutes, seconds)

@Branjawn
Can you try the code from #13 along with this. If it works for you, I'll find out how to roll a patch for Karen's convenience.

johnmunro’s picture

FileSize
716 bytes
2.6 KB

I now have this working well on a production site so attaching patches for the above code for committing.

johnmunro’s picture

Status: Active » Needs review
Branjawn’s picture

So, I totally bailed on using Date, Calendar, FeedAPI, and iCal.
There are so many problems with this right now and I spent weeks troubleshooting. I work on a shared server and cron was giving me hell because of iCal in particular. the next site I have to import a Google Calendar I'll give it another go. I'm no programmer or I would help fix the problems. I respect those of you who can.

scottatdrake’s picture

I applied the patches in #16 and can confirm that it works. Thanks, johnmunro!

Can they be committed?

manos_ws’s picture

I also confirm that the patch from #16 works and should be committed.

Tested on date 6.x-2.x-dev (2011-May-27).

Its a two year's already!

Thanks.

benjifisher’s picture

Status: Needs review » Reviewed & tested by the community

I have not tested this myself, but I am marking it RTBC based on #19 and #20. @scottatdrake, @manos_ws, you should have done that if you wanted to see the patch in #16 committed.

This issue deals with importing an ical feed into Drupal. When exporting an ical feed, we need to make the corresponding change. I just opened an issue to address this: #1282538: ical feed generates wrong DTEND value for multi-day, all-day events.

manos_ws’s picture

Thanks allot benjifisher , the patch is working for 4 months now

Manos

arlinsandbulte’s picture

Assigned: johnmunro » arlinsandbulte

putting on my list to review and commit...

KarenS’s picture

Assigned: arlinsandbulte » KarenS
Status: Reviewed & tested by the community » Needs review

Hold off on this a bit Arlin. I am trying to do some backports of some files from the D7 version, including the date_api.ical.inc file, which will impact this patch.

KarenS’s picture

I couldn't get the patch to apply any more, plus I made a few changes to use the Date API to massage the date.

diff --git a/date_api_ical.inc b/date_api_ical.inc
index 866430d..a2b8eca 100644
--- a/date_api_ical.inc
+++ b/date_api_ical.inc
@@ -215,15 +215,31 @@ function date_ical_parse($icaldatafolded = array()) {
             // even when you are working with only a portion of the VEVENT
             // array, like in Feed API parsers.
             $subgroup['all_day'] = FALSE;
-            if (!empty($subgroup['DTSTART']) && (!empty($subgroup['DTSTART']['all_day']) ||
-              (empty($subgroup['DTEND']) && empty($subgroup['RRULE']) && empty($subgroup['RRULE']['COUNT'])))) {
-              // Many programs output DTEND for an all day event as the
-              // following day, reset this to the same day for internal use.
-              $subgroup['all_day'] = TRUE;
+
+            // iCal spec states 'The "DTEND" property for a "VEVENT" calendar
+            // component specifies the non-inclusive end of the event'. Adjust
+            // multi-day events to remove the extra day because the Date code
+            // assumes the end date is inclusive.
+            if (!empty($subgroup['DTEND']) && (!empty($subgroup['DTEND']['all_day']))) {
+              // Make the end date one day earlier.
+              $date = date_make_date($subgroup['DTEND']['datetime'] . ' 00:00:00', $subgroup['DTEND']['tz']);
+              date_modify($date, '-1 day');
+              $subgroup['DTEND']['datetime'] = date_format($date,  'Y-m-d');
+            }
+            // If a start datetime is defined AND there is no definition for
+            // the end datetime THEN make the end datetime equal the start
+            // datetime and if it is an all day event define the entire event
+            // as a single all day event.
+            if (!empty($subgroup['DTSTART']) &&
+               (empty($subgroup['DTEND']) && empty($subgroup['RRULE']) && empty($subgroup['RRULE']['COUNT']))) {
               $subgroup['DTEND'] = $subgroup['DTSTART'];
             }
             // Add this element to the parent as an array under the component
             // name.
+            if (!empty($subgroup['DTSTART']['all_day'])) {
+              $subgroup['all_day'] = TRUE;
+            }
+            // Add this element to the parent as an array under the
             prev($subgroups);

That seems to parse the ical correctly. I don't understand what the second patch has to do with this. theme_date_time_ago() is not used in the ical processing.

KarenS’s picture

Version: 6.x-2.x-dev » 7.x-2.x-dev
Status: Needs review » Patch (to be ported)

Committed this to D6, except for the patch to the time_ago function, which seems like an unrelated issue.

http://drupalcode.org/project/date.git/commit/4ec58c8

The same thing needs to be committed to D7.

KarenS’s picture

Status: Patch (to be ported) » Fixed
KarenS’s picture

And thanks for all the work!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

jakew’s picture

Status: Closed (fixed) » Active

I'm still having this issue: Date 6.x-2.9, which included the patch committed in #26. I'm using parser_ical 6.x-2.0-beta1 and Feeds 6.x-1.0-beta13 to import a google calendar iCal feed. Multiple-day events with times specified work okay, but a multi-day event with "All day" specified in Google Calendar imports as

Jan 30 2013 - 7:00 pm

when in Google it's entered as:
Thu, January 31 – Thu, February 21

Doesn't seem to have been fixed by the patch... I could also try the dev branch of Date, but I'm not sure that has any further work on this issue in it...

Thanks!

ekidman’s picture

I am also still experiencing this issue. Same versions of date, parser_ical, and feeds as jakew above.

@jakew - did you ever solve this??

When I bring in an ical feed with an event that spans multiple days, with no time specified, it comes in with the end date set the same as the start date.

The VEVENT is formatted as follows:

BEGIN:VEVENT
DTSTART;VALUE=DATE:20130722
DTEND;VALUE=DATE:20130727
SUMMARY:Dawn
UID:50046ac2-8208-474a-b46e-00e4b2500df7
LAST-MODIFIED:20130711T011331Z
SEQUENCE:1374512429
END:VEVENT

And should be imported as an event running from 7/22 to 7/27, but is listed as an all-day event on 7/22 instead. Any help would be greatly appreciated, it's holding us from syncing our calendars correctly at the moment.

ekidman’s picture

I also tried the newest Dev versions of both Date and Feeds for D6 with no luck. Also tried taking the patched date_api_ical.inc file from 6.x-2.4 (the version that was patched above), with no luck either.

I tried all of these on my existing dev site, as well as a fresh drupal install - nothing works. As stated originally, it works if I change the date format from:

DTSTART;VALUE=DATE:20130709
DTEND;VALUE=DATE:20130713

to:

DTSTART:20130709T000000
DTEND:20130713T000000

But this is not a solution as I have no control over how the dates are imported. I've been struggling with this for several days now, and am really hoping someone is able to help - otherwise I have to tell people that it's impossible to reliably import iCal events into Drupal :(

I've tried everything I can think of and have hit a wall at this point.

DamienMcKenna’s picture

Issue summary: View changes
Status: Active » Closed (outdated)

The D6 module is no longer supported and this change was committed to D7. If further work is needed on this functionality please open a new issue. Thank you.