UTC timezone 'Z' not intepreted (by mapper) [#503338]

Howdy all,

Love the iCal feed parser. Got it up and running and importing events from a Google Calendar ICS feed. I have noticed a couple quirky things, one of which I can live with, the other not-so-much. But I'm not sure if it's an ical_parser issue, or a Google calendar issue...

Basically, I've noticed that sometimes the date/times are imported correctly, sometimes there not. I've narrowed it down that Google's ICS file puts the timezone (TZID) in for recurring events, but not for stand-alone events. Hence, when I import, the recurring events' times are correct, but the stand-alones are not (they are 4 hours ahead, which is EST's offset from GMT). My time zone is America/New_York in both my Drupal installation and on my Google calendar.

Example:
Recurring event:
DTSTART;TZID=America/New_York:20090201T093000
DTEND;TZID=America/New_York:20090201T103000
RRULE:FREQ=WEEKLY;WKST=SU;BYDAY=SU
...
Non-recurring event:
DTSTART:20090309T224500Z
DTEND:20090309T234500Z

Is this a problem with ICS, or the parser -- and is there a workaround? I'm not familiar with ICS specs, whether the TZID should be in there or not.

See also http://www.google.com/support/forum/p/Calendar/thread?tid=645a85124aa593..., but there's been no response.

Any help appreciated,
Tom

Comments

Comment #1

ekes commented 27 June 2009 at 09:32

There is actually time zone information in the non-recurring example:-

DTSTART:20090309T224500Z

The Z at the end of that says that the date is UTC (also known as GMT). Which from your description is how we are parsing it :-)

Comment #2

tmdrupal commented 28 June 2009 at 12:47

Thanks -- so are you saying the problem is with the ICS file? Or is there a way to tell the parser something like "if the ICS time zone is UTC, use the site time zone [which, for me, is America/New_York]; if not, use the UTC time zone [which, for me, is also America/New_York]".

Thanks
Tom

Comment #3

ekes commented 28 June 2009 at 16:11

The timezone is UTC. So +0. So to tell it that that is anything else would be hacking it around. It sounds like the feed either doesn't know that it really is in the EST zone, or it's not working when it puts it into UTC. Either way it's wrong in the feed, not the parsing.

Comment #4

tmdrupal commented 29 June 2009 at 13:09

Here's the weird thing: if I open the ICS file in my Outlook, all the dates are fine. They are all EST (America/New_York). Thus, it looks like Outlook and the iCal parser parse the ICS file differently.

Is the iCal parser assigning each node's timezone based on timezone info in each VEVENT:DTSTART tag, or from the X-WR-TIMEZONE tag in the ICS "header" (for lack of a better word). It seems like the former, but it makes more sense to do the former, doesn't it? Since a calendar feed is generally relative to one timezone?

Thanks again
Tom

Comment #5

ekes commented 30 June 2009 at 17:44

The timezone is specified though. IF that Z was not at the end then the parser should look to the other timezone information, site wide or feed specific. However, as there is a Z it is specifically UTC and the parser should not look any further.

Comment #6

dcasey commented 1 July 2009 at 00:30

I am having the exact same issue.

My iCal feed is coming from Google. All repeating events import to Drupal correctly (timezone reads: America/Denver), any non-repeating events are off the GMT/UTC offset, 6am shows 12 noon (timezone reads: UTC).

Viewing the calendar from a shared account online, both dates are correct. Importing the same Google iCal to my computer (iCal on the Mac) imports correctly, as does importing to my Palm Pre.

BEGIN:VCALENDAR
PRODID:-//Google Inc//Google Calendar 70.9054//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:My Calendar
X-WR-TIMEZONE:America/Denver
X-WR-CALDESC:My Calendar description.
BEGIN:VTIMEZONE
TZID:America/Denver
X-LIC-LOCATION:America/Denver
BEGIN:DAYLIGHT
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
TZNAME:MDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART:20090704T120000Z
DTEND:20090704T130000Z
DTSTAMP:20090701T001029Z
UID:ap6sfl4ubqu4rsuan2q6srll0k@google.com
CLASS:PUBLIC
CREATED:20090701T000932Z
DESCRIPTION:
LAST-MODIFIED:20090701T001012Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:6am test (no-repeat)
TRANSP:OPAQUE
END:VEVENT
BEGIN:VEVENT
DTSTART:20090704T120000Z
DTEND:20090704T130000Z
DTSTAMP:20090701T001029Z
UID:op04s0ig9it30ensekckth1mrc@google.com
CLASS:PUBLIC
CREATED:00001231T000000Z
DESCRIPTION:
LAST-MODIFIED:20090701T001000Z
LOCATION:
SEQUENCE:4
STATUS:CONFIRMED
SUMMARY:6am test (repeat)
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR

Comment #7

ekes commented 1 July 2009 at 01:21

OK So I've now followed up on that in google support forum (original link above) - if anyone else knows a good forum to ask I'd be happy.

I guess if it's not going to change, and it is a common problem, a hackish solution could be considered. But only if someone can come up with one that's not going to break parsing of all the feeds that actually do meet the standard and put the time with the Z at the end and mean it.

Comment #8

tmdrupal commented 1 July 2009 at 13:55

@ekes: thanks for looking at the Google support post. I see what you are saying, that since the "Z" is in there, the parser should interpret the time as UTC. The odd thing is that when I import this iCal file into other programs -- Outlook and phpicalendar are the ones I've tried -- they always come over as what I expect: EST. So, either all those parsers are off-spec (or do some assuming), or ical_parser is.

I was looking at some of the phpicalendar code to see what they do. Here is where the d/t is extracted for our viewing pleasure:

function extractDateTime($data, $property, $field) {
	global $tz_array, $phpiCal_config, $calendar_tz;
	$allday =''; #suppress error on returning undef.
	// Check for zulu time.
	$zulu_time = false;
	if (substr($data,-1) == 'Z') $zulu_time = true;
	// Pull out the timezone, or use GMT if zulu time was indicated.
	if (preg_match('/^'.$property.';TZID=/i', $field)) {
		$tz_tmp = explode('=', $field);
		$tz_dt = match_tz($tz_tmp[1]); #echo "$tz_dt<br>";
	} elseif ($zulu_time) {
		$tz_dt = 'GMT';
	}
	
	// Extract date-only values.
	if ((preg_match('/^'.$property.';VALUE=DATE:/i', $field)) || (ereg ('^([0-9]{4})([0-9]{2})([0-9]{2})$', $data)))  {
		// Pull out the date value. Minimum year is 1970.
		ereg ('([0-9]{4})([0-9]{2})([0-9]{2})', $data, $dt_check);
		if ($dt_check[1] < 1970) { 
			$data = '1971'.$dt_check[2].$dt_check[3];
		}		
		# convert to date-time
		$data = $dt_check[1].$dt_check[2].$dt_check[3]."T000000";
		$time = '';
		$allday = $data;
	}
	// Extract date-time values.
	// Pull out the date and time values. Minimum year is 1970.
	preg_match ('/([0-9]{4})([0-9]{2})([0-9]{2})T{0,1}([0-9]{0,2})([0-9]{0,2})/', $data, $regs);
	if (!isset ($regs[1])) return;
	if ($regs[1] < 1970) { 
		$regs[1] = '1971';
	}
	$date = $regs[1] . $regs[2] . $regs[3];
	$time = $regs[4] . $regs[5];
	$unixtime = mktime($regs[4], $regs[5], 0, $regs[2], $regs[3], $regs[1]);
	# chooseOffset checks for Daylight Saving Time
	$server_offset_tmp = chooseOffset($unixtime, $phpiCal_config->timezone);
	if (isset($tz_dt)) {
		$offset_tmp = chooseOffset($unixtime, $tz_dt);
	} elseif (isset($calendar_tz)) {
		$offset_tmp = chooseOffset($unixtime, $calendar_tz);
		$tz_dt = $calendar_tz;
	} else {
		$offset_tmp = $server_offset_tmp;
		$tz_dt = $phpiCal_config->timezone;
	}
	// Set the values.
	$unixtime = calcTime($offset_tmp, $server_offset_tmp, $unixtime);
	#echo "offset_tmp $offset_tmp, server_offset_tmp $server_offset_tmp, $unixtime =".date("Ymd His",$unixtime)." $time<br>";
	$date = date('Ymd', $unixtime);
	if ($allday == '') $time = date('Hi', $unixtime);
		
	// Return the results.
	return array($unixtime, $date, $time, $allday, $tz_dt);
}

You see a few lines into it, it checks for UTC (Zulu) tz. But then when I look at the calendar itself, the timezone is EST, as it should be (from my perspective). Maybe there's a setting somewhere else that offsets this; I'll poke around. Or maybe something jumps out at you.

I think what I will do until (if?) this is resolved is make a dummy php page that downloads my Google ICS file, does a regex replace to strip all the "Z"'s and returns the results as the new feed. I will set the feed URL to this page.

Thanks for your attention,
Tom

Comment #9

tmdrupal commented 4 July 2009 at 18:45

@ekes:

Some valuable more info!

I went and did what I said, create a "proxy" page which removes the "Z" from the end of the non-recurring event date/times, which would make the d/t "local." HOWEVER, after all that, I am thinking the Google iCal feed is correct, and the parser is not reading it correctly. Here's why:

I created a fake Google calendar. You may view it here: http://www.google.com/calendar/embed?src=mot0u1b104d6361vjjshnj8iis%40gr...
View its iCal feed here: http://www.google.com/calendar/ical/mot0u1b104d6361vjjshnj8iis%40group.c...

(The calendar's time zone is America/New_York.) Notice Event B, a recurring event starting 7/9. Time is 1pm EST. The ical says:
DTSTART;TZID=America/New_York:20090709T130000
That looks kosher, right? It is 13:00 (1pm), with no Z on the end, indicating 1pm at the supplied TZID, America/New_York. All good.

Now look at Event A, a single event on 7/8. Time is 9am EST. The ical says:
DTSTART:20090708T130000Z
This says 13:00 (1pm) ZULU/GMT/UTC time. Since Zulu is 4 hours ahead of EST, this translates to 9am EST. So this is actually correct. My event is at 1pm UTC, because that's the same thing as 9am EST. If I suck this into parser_ical, it renders it as 1pm EST.

So, it seems the reason Outlook, phpicalendar, etc. correctly render this date, is because it recognizes the date/time as Z and simply translates it to the local time. It doesn't seem parser_ical is doing that. It seems like it sees the Z, but doesn't translate the time from UTC to local system time. It just take the time that's there (e.g. 20090708T130000Z) and assumes it's local time.

This seems like a bug, doesn't it?

Thanks!
Tom

Comment #10

Craig Gardner commented 8 July 2009 at 04:17

I don't know if the problem is with the parser, but the view. I've done some playing around with this problem and when I display the date and include the time zone information, it shows as UTC for me, which is what it should be. It just doesn't output as the proper time zone.

Comment #11

ekes commented 10 July 2009 at 09:33

Title:	Inconsistent dates	» UTC timezone 'Z' not intepreted
Component:	Miscellaneous	» Code
Category:	support	» bug

Nice work isolating this one. It is a bug. I'll try and get to it ASAP if no one else does.

Comment #12

ekes commented 12 July 2009 at 17:18

Status:

Active

» Postponed (maintainer needs more info)

Hmm... I've just run this in testing and it works if timezone is enabled on the field - it doesn't if timezone isn't enabled. This is correct. The parser is just passing to the mapper the time and the timezone. This should then be mapped into the field, if there is no timezone enabled it is lost there.

Is this the case?

Comment #13

tmdrupal commented 13 July 2009 at 19:53

What do you mean "if timezone is enabled on the field"? On my field, I have the "Time zone handling" set to "Site's time zone." Is this what you mean? I think I had tried changing this to no avail. Or let me know how it should be set up and I'll try that.

Thanks
Tom

Comment #14

ekes commented 14 July 2009 at 07:35

Try it with "Date's time zone". That way when the UTC from the Z gets passed into the date field it is used. With "Site's time zone" it is ignored and the time is assumed to be the sites (when in this case it isn't, it's UTC).

Comment #15

Craig Gardner commented 15 July 2009 at 23:24

I have tried with "Date's time zone" and it still fails.

Comment #16

cameronheikkinen commented 16 July 2009 at 15:04

Same here, still no luck.

Comment #17

ekes commented 16 July 2009 at 15:07

When you edit the node what time zone does the field say the dates are in that (i) come in with TZID=America/New_York (ii) with Z at the end?

Comment #18

jlyon commented 18 July 2009 at 22:16

I'm having the same problem. When I select "Date's time zone" the correct timezone information is saved in the date field (UTC for fields with a Z at the end and the correct TZID value for the other ones), but then the timezone is never converted into my site's timezone for date displays and my calendar view. I think the error is with date.module or a configuration I am mission, but not with ical_parser.

I have my default timezone set to "America/Denver". Am I missing a configuration for date.module somewhere? Can this be done with the date and time formats? Thanks!

Comment #19

jlyon commented 19 July 2009 at 19:04

This patch fixes the date mapper so UTC dates are converted into your site's timezone if you set "if timezone is enabled on the field" to "Time zone handling" set to "Site's time zone." Fixed the problem for me. http://drupal.org/node/451888

Comment #20

cameronheikkinen commented 21 July 2009 at 19:52

What if I can't use that patch?

Comment #21

Craig Gardner commented 30 July 2009 at 01:55

I applied the patch from #451888: TimeZone wrong for UTC iCal data, and it seems to be working for me. Thank you for the info.

Comment #22

equintanilla commented 1 August 2009 at 19:20

Where do I make the changes you outline - specifically, where do I set "if timezone is enabled on the field" to "Time zone handling" set to "Site's time zone."?

many thanks

Comment #23

equintanilla commented 1 August 2009 at 22:19

okay, I tried the patch and found the field settings to change. However when I use No Time zone conversion, it is off by one hour - I think because of Daylight savings time (DST).

Really appreciate the patch!

Comment #24

ekes commented 2 August 2009 at 13:23

Title:	UTC timezone 'Z' not intepreted	» UTC timezone 'Z' not intepreted (by mapper)
Status:	Postponed (maintainer needs more info)	» Closed (works as designed)

ok nice so it is correctly passing on the info that it's UTC and now #451888 the mapper can alter for time zone if required.

Comment #25

philpro commented 12 August 2009 at 04:02

I've applied the patch with no success. I've tried every possible configuration with no success. All of my events are off by 6 hours, which is what I think is roughly the difference of my time zone from UTC less daylight savings time. (I am mountain time). No amount of configuration has impacted the time difference of the imported values. Any thoughts?

Comment #26

philpro commented 6 September 2009 at 00:46

Thought that I had used the patch, but I didn't so now it's working. Opps.

Comment #27

nickbits commented 30 September 2009 at 18:01

Had similar issues but solved with patch on http://drupal.org/node/451888 . Not sure if that is of any use to you guys?

Comment #28

jaharmi commented 13 April 2010 at 14:26

I came across this issue because I'm also getting incorrect times for standalone, non-recurring events imported by Feed API from a shared Google Calendar.

What is the status? I've found this and at least one other thread on the same topic, but given that it's 2010 (and all the threads I've found come from 2009), I'm wondering whether a patch has been rolled in and is not working, or if the patch everyone is talking about was never checked in.

I'm up-to-date with Feed API, Feed API's version of iCal parser, Event/CCK, and Drupal core itself as of the weekend (4/10/2010). My site is configured for the correct time zone and Google Calendar is also set up for the local time zone. Only non-recurring events are experiencing the problem with this setup.

Comment #29

Yorgg commented 29 July 2010 at 12:48

Did you enable the date repeat module?
Refresh the data more often

In my case the UTC timezone 'T' and 'Z' are no intepreted by the CCK module.
This should be by design. Problem seems to be Date module instead though I found this thread more knowledgeable.

UTC timezone 'Z' not intepreted (by mapper)