iCal feed parser + iCalcreator-based export

I also need this functionality and here is a first patch to look at. For now, the patch does not take into account any advanced functionality of ical, but I already had some interesting results with some ical feeds.

2 things to note:

• The patch uses a library available here : http://code.google.com/p/ics-parser/ in MIT license. I hope that it can be included in this plugin, provided as GPL 2
• The patch currently does not work with repeated events, and I believe this will require a patch to Feeds, because Feeds only provides a Date: start and Date: end and no Date: rrule

See http://drupal.org/project/libraries for an initial introduction to why an external library shouldn't be included inside this module. That said, with an appropriate dependency on the libraries module, an iCal parser could be an interesting addition indeed. :)

I was able to get the patch working. I added location back into FeedsIcalParser.inc file and it seemed to work as well. I just had that map into a textfield.

Thanks @guillaumev!

New patch with correct dependency using libraries and also added location.

Note that you will need libraries 2.x to make it work.

There's already a module that does this: parser_ical.

It's usually better for Drupal and the community to contribute to existing modules instead of duplicating functionality and fracturing the codebase.

@kentr Thanks for this, and I agree with you (I had just not seen this module). However, I believe that the date_ical maintainer should probably then remove the line that says in the project description:

Add in an iCal parser for the Feeds module that can parse an iCal feed from another site so it is possible to import as well as export iCal information.

and maybe redirect to the parser ical module for such a functionality...

I agree with you. I find that confusing as well.

Since I need the ical parser as well, I'll support this idea in the date_ical queue.

Thanks for all of your work on this. We have users inputting events into a variety of calendars and I'd love to consolidate them into one place for the end user. Having a lot of trouble finding a ical parser for D7.

@guillaumev The problem with parser_ical is that there's no recommend or dev version for Drupal 7. The only one that's available is an apparently no-longer-maintained, unreleased branch that's only available from the git repo.

I need iCal functionality for my company's website, so I'll be spending work time over the next week or so creating a solution to this. I think I'll end up merging the work done for Drupal 7 on parser_ical into this module, so that all iCal support for Drupal 7 is consolidated in one place (this module). I've already done some work over in #1833362: Use a third party library for generating iCal feeds, so my ultimate version is also going to incorporate the changes I made for that issue.

I've attached a gigantic patch for the 7.x-1.x-dev build which adds the ability to import iCal feeds, using a Feeds plugin. Anyone who's interested, please try it out and tell me what you think.

The vast majority of the work for this patch was already done before I started, as it's really just a heavily modified version of the semi-abandoned Parser iCal module's 7.x-2.x git branch (discussed in depth at #1015128: iCal parser for D7?). I added many tweaks and new features (e.g. alter hooks for various stages of the import process), and re-factored it to fit into Date iCal.

This patch also incorporates all the work Steve Jones and I did on #1833362: Use a third party library for generating iCal feeds, which changed the output mechanism in the Views plugins to use the iCalcreator library instead of the Drupal theme system for formatting iCal output.

Please let me know if there are any additional features you'd like (I don't think RRULEs work, yet) or have any bugs to report. I'm no expert on the iCal format, so anything I haven't personally encountered in the feeds I need to import for my company's site is currently foreign to me.

I've requested to be added as a maintainer for Date iCal, but KarenS hasn't responded, yet, so posting to this issue will have to do for now.

This is super. RRULE is actually really key to our implementation (importing course schedules). Another thing I'd love to have is the ability to map to usernames (userreference field).

Will give this a test shortly!

I'll see about implementing RRULE asap, but I haven't worked with repeating Date fields, so I'm not at all familiar with what actually needs to be done to parse RRULEs into Dates. If you have any good example iCal files that make use of RRULE, I'd appreciate a copy so I can test with them.

Also, does anyone know how to run the tests in the test folder from the patched module? I copied those verbatim from Parser iCal, but I've been testing my code manually because I don't know how to actually execute them.

Here's one of ours: https://www.cs.ubc.ca/calendar/ical/2013-01 -- a good mix of a lot of RRULEs, RDATEs, EXDATEs :)

Yikes, I don't even know what RDATEs and EXDATEs are. But I've got some promising work done on RRULEs already, so I think that functionality should be coming down the pipe some time soon.

The patch is corrupt.

git apply date_ical-add_ical_feeds_parser-1721678-10.patch 
fatal: corrupt patch at line 165

Could you reroll?

@coredumperror Thanks for working on this. I haven't been able to try your patch yet because it is apparently corrupt. I am currently using the parser_ical code from http://drupal.org/node/1015128#comment-6587754, and it's pretty much working except for RRULEs. So if you do manage to get RRULEs working that would be great. I had done some research into possibly fixing it myself, but I'm not too confident in my ability to get it done. RRULEs were working in the D6 version of parser_ical, so you might want to look into how they worked there. When I was investigating it, this code at the beginning of ParserIcalFeedsParser.inc in the D6 module looked like it might be the key

class ParserIcalDateTimeElement extends FeedsDateTimeElement {
  public $repeat_vals = NULL;

I'm ready to start testing, especially as RRULES become available. Is a new patch coming that we can apply?
In any case, thanks for taking over this issue.

(Ah! Well: RDATEs are most often used as individual out-of-pattern instances of an item within the context of an RRULE sequence; EXDATES are excluded items from an RRULE sequence.

The iCal spec is maddening in this particular, however, eg one gotcha is weird requirements re: how you can specify timezones -- although you can use Zulu for RRULES, RDATE and EXDATE require a specific timezone be specified. That's the only way I was able to get them working in all clients, because every one implements iCal *slightly* differently, because programmers.)

Shoot, I knew I should have tried applying my own patch before releasing it... I'm still pretty new at this.

The attached patch should work this time, though I honestly have no clue what a "corrupt patch" error even means, so I'm not sure what was actually wrong with my original patch.

P.S. @dswier - Thank you so much for pointing that ParserIcalDateTimeElement out! I'd run into a bit of a wall with RRULEs, because the Date Repeat Field module implemented repeating dates is a really odd manner, which prevents it from supporting indefinitely repeating dates. This class should help a lot in showing me how to get around that.

RRULEs, EXDATEs, and RDATEs are now supported! I had to work around the fact that Date Repeat Field doesn't support indefinitely repeating RRULEs, which required me to copy-paste one of its functions, just to make a one-line edit to it. Now, rather than just giving up, it treats indefinite repeaters as repeating only 52 times. The Drupal 6 version of Parser iCal module did it like this, probably because there are 52 weeks in a year.

Please let me know if you find any bugs with the existing functionality. I probably won't have much more time to work on Date iCal to add new features, but bug fixes will remain a high priority for the foreseeable future.

I'm quite excited that you got this working! The patch actually applied for me, and importing calendars seems to be working fine. However, I'm still not sure how RRULEs are supposed to get attached to the nodes. I can see you added a mapping target for repeat rule, but unless I'm missing something there does not appear to be an appropriate source mapping to use for it.

git apply date_ical-add_ical_feeds_parser-1721678-20.patch 
date_ical-add_ical_feeds_parser-1721678-20.patch:116: trailing whitespace.
  
date_ical-add_ical_feeds_parser-1721678-20.patch:298: trailing whitespace.
    
date_ical-add_ical_feeds_parser-1721678-20.patch:466: trailing whitespace.
  
date_ical-add_ical_feeds_parser-1721678-20.patch:1044: trailing whitespace.
  
date_ical-add_ical_feeds_parser-1721678-20.patch:1163: trailing whitespace.
  
warning: squelched 274 whitespace errors
warning: 279 lines add whitespace errors.

@dswier: Have you got the Date Repeat Field module installed? It's a submodule of the Date module, so you should just be able to enable it on your site. Once you do, you can create a new date field and set the new "Repeating Date" dropdown in the field settings to Yes.

Wait, nevermind. I misread you. But what I said is potentially useful for others, so I'll leave it.

I think the reason you're not seeing the RRULE source is because you're using the wrong parser. I think it might be a good idea for me to just disable the "Date API iCal parser". Since I haven't worked on that at all, outside of changing a few variable names, I don't even know what it does (it's just a legacy from Parser iCal). In order to get the RRULE source, you need to set your feed importer to use the iCalcreator Parser.

@Isolesen: I dunno what to tell you. I created that patch using git diff, comparing my code to the most recent commit to the 7.x-1.x branch of the Date iCal git repo. If your git doesn't like it, perhaps you should try using patch -p1?

@coredumperror It is whitespace errors. It can apply, but introduces whitespace errors. Those should probably be cleaned up before it is ready to commit.

Alright, my repeating dates are importing after switching to the iCalcreator Parser. If that is the parser you want people to be using I definitely agree that you should not include Date API Ical Parser as an option.

Ok, so after a little more testing, RRULEs are not quite working correctly yet(unless it's something wrong with my setup). My dates in the ical file that do have an RRULE are displaying all the repeats on one day(might be a problem with my calendar view). Looking at the repeats tab for the date node does show the repeats on the correct dates though. Another issue I'm having is that my dates that do not have an RRULE are still getting repeats associated with them(52 repeats, which I believe is the maximum possible number of repeats).

Whoops! I completely forgot to test what happens when my new code parses a VEVENT with no RRULEs. Obviously, it's borked, so I'll get a patch out to fix that ASAP.

However, what do you mean by "dates in the ical file that do have an RRULE are displaying all the repeats on one day"? It's possible that my code missed some essential thing about creating a node with repeating dates, but I need to know exactly how to reproduce that issue, so I can figure out what's wrong.

@Isolesen: How would you suggest that I clean up these whitespace errors that you're plagued with? I created the patch using git... I don't know why it's making a bad patch, so I'm not sure what to do differently to fix it.

Attached is my newest patch, which fixes the repeating dates on VEVENTs which have no RRULE. The fix for this leads me to believe that I may need to do some additional changes to fix the parsing on optional values components like RRULEs, but I don't have time to look into it right now, and I'm about to go on vacation for several weeks. I'll see about getting some better testing done on various edge cases after the holiday.

An explanation of "dates in the ical file that do have an RRULE are displaying all the repeats on one day": I'm displaying the date nodes created by the ical import in a calendar view. In the case of a date having a repeat, all the repeats are listed on the start date on the calendar. So for example if an event was repeating every day of the week, it is listed 7 times on the start date on the calendar. I am somewhat certain that this module is not at fault. It is probably an issue with how I have Date Repeat or Calendar set up, since when I manually create an event node with repeats the same behavior occurs.

@coredumperror: Awesome work, thank you! Re: whitespace errors, check the settings in your code editor, it might be spawning them -- some are better-behaved than others. If you install the dreditor plugin for Chrome/Firefox, you can "review" a patch and it will show you them in red. Or using the coder module it will spit out warnings for you.

Yes you do need to have the iCalcreator library installed. I got a php error saying the library was not found when I tried to import without it.

Also, I figured out what my problem was with the repeats showing on the same calendar day. I had "Display all values in the same row" checked for my date field. Unchecking that made the dates display correctly. So this module is working great for me now, and I have not come across any more bugs.

Things are working a lot better but I'm still seeing some problems where I get error messages related to RRULEs. I've attached a pdf of the error messages and the ics file that was used for the feed. For another feed I didn't have any issues.

added the ics file

@Renee S: You mention the "red" sections in the dreditor review display, but I'm not sure what's actually *bad* about them. Looking at them, they're the places where I intentionally added whitespace to a blank line, to fill out the indentation level in the code block.

[dupe]

Ah, I don't think that's part of the Drupal code standard, that's why dreditor is flagging them. It wants no whitespace on blank lines.

That doesn't actually explain why you're having trouble with the patch, though. I applies cleanly for me and (as far as the comments here seem to say) everyone else. I'm really baffled.

In other news, I'm now officially the owner of this module! I'll start getting my code pushed up to git, so that I can make an official dev release. Then we won't have to worry about my patches being wonky any more!

OK, for those who've had issues with the patches I've made, you can now download the 7.x-2.x-dev release available from the Date iCal project page.

Now that I've gotten that all set up, I can start working on dougm's problem. However, I'm going to close this issue, since I'm now working with the 2.x branch. I'll post a new issue for dougm's problem (#1886580: Importing VEVENTs with RRULEs breaks sometimes). If anyone else discovers any bugs (or you'd like to see new features added), please create a new issue, attached to the 2.x branch.

P.S. If I'm treating this whole process wrong by closing this issue, please yell at me. I've never been a maintainer for a contrib module before.