Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 UTC on 18 March 2024, to get $100 off your ticket.
Has someone already written an iCal processor? If not, how much work would it be?
I am using the calendar module for displaying event type nodes. The calendar module does have an iCal component, but it does not create event nodes.
Comment | File | Size | Author |
---|---|---|---|
#55 | parser_ical 6.x to 5.x patch - parser_ical.d5-patch.20081201.patch | 1.45 KB | ekes |
#52 | parser_ical.module.txt | 4.72 KB | KarenS |
#42 | configure_content_type.jpg | 109 KB | gustav |
#42 | map.jpg | 53.25 KB | gustav |
#37 | parser_ical.module.txt | 4.51 KB | KarenS |
Comments
Comment #1
alex_b CreditAttribution: alex_b commentedHere is the code that we could use for building an iCal parser.
In Date module the iCal logic lives in its own inc file:
http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/date/date_a...
The hard work is done. What's missing is a parser that integrates this iCal parser with feedapi: parser_ical.module !!!
The parser should create parsed items the feedapi way (see parser_simplepie or parser_common_syndication) - the rest can be handled by a Feed Element Mapper mapper implementation for date fields (see feature request here: http://drupal.org/node/215979).
This is not rocket science, it'll take a developer one or two days to get this up. I am unfortunately swamped with work at the moment, but I would love to see this feature happen and I am ready to give advice to whoever steps up to this task.
Comment #2
gustav CreditAttribution: gustav commentedIs anyone working on this already? If not, perhaps I can find some time to take a look at it over the next weekend.
Comment #3
ekes CreditAttribution: ekes commentedI've made something of a start on this - using code from date_ical and the present parsers. It will need some little workrounds to make it work with the date_ical parse to start off with. So I'm claiming this -- *but* if anyone wants to join in, take over, or what ever I'm kinda busy. I just really want it for the continued development of http://drupal.org/node/215979 ;-)
Comment #4
alex_b CreditAttribution: alex_b commentedhooray :)
Comment #5
ekes CreditAttribution: ekes commentedOK, I'm not getting so much time, so this is slow, so I'm going for the release early, release often, and be damned option.
Attached is 'working'.
It has no requirements testing, it has little validity testing, and it's got no install script.
It reuses the parser_common_syndication functions for getting the feed, because they are good, but so has an install issue (you need parser_common_syndications tables already).
It uses the date_api_ical functions for parsing, but has to patch the upload() function as that include the functionality for collecting the feed (which is being done with above). I've mailed karens about the possibility of accepting a patch to separate out the fetching of the ical file and the parsing it but heard nothing yet.
Comment #6
ekes CreditAttribution: ekes commentedOK, I'm not getting so much time, so this is slow, so I'm going for the release early, release often, and be damned option.
Attached is 'working'.
It has no requirements testing, it has little validity testing, and it's got no install script.
It reuses the parser_common_syndication functions for getting the feed, because they are good, but so has an install issue (you need parser_common_syndications tables already).
It uses the date_api_ical functions for parsing, but has to patch the upload() function as that include the functionality for collecting the feed (which is being done with above). I've mailed karens about the possibility of accepting a patch to separate out the fetching of the ical file and the parsing it but heard nothing yet.
Comment #7
Aron NovakFirst, thanks for working on an iCal parser, it's a really exciting use-case of FeedAPI!
Well, I suggest you the following:
- You can safely delete the block after // Do the autodiscovery at this level, pass back the real data // Maybe it's HTML. If it's not HTML, not worth to take a look into the downloaded string
- a simple processor, like this, does not need any tables. You can make it even more simple. The caching is just an option.
- i recommend you to simply copy more code from parser common syndication. The problem is that FeedAPI is not dependent on any parsers, they are totally interchangeable.
Can you supply here a "feed url" suitable for ical parser? I would really like to try out the code :)
I attached a simplified version of the module. I haven't tried it out, but it contains my ideas in code-form. :)
Comment #8
ekes CreditAttribution: ekes commentedCool I'll have a go with it - I'm just looking at the date_mapper again so I'm going to try with some of the iCal output.
I've been using the index of feeds at:
http://icalshare.com/
to come up with interesting and different iCal feeds.
Comment #9
alex_b CreditAttribution: alex_b commentedHey,
Looks like a good start. I created a patch from ekes and aron's posts here. (for adding new files cvs add all new directories and files followed by a cvs diff -u -N from the feedapi directory, let's use a single patch on this queue for our discussion)
It doesn't work though :( :
When I use webcal:// as schema, there is an error that the feed couldn't be retrieved, when I use http://
I get this error:
Call to undefined function date_make_date()
http://skitch.com/alexbarth/8we7/mozilla-firefox
(feed http://ical.mac.com/ghodder/1st%20Year%20multimedia%20timetable.ics )
In regards to patches to date module files, please go ahead and just post them on the date module issue tracker and link to them from here. I am sure Karen will review them. For simplicity, I think the way you took by copying the code into the ical parser module and clearly marking it as patched is a good method for now.
One request: Can we call this module parser_ical ? I don't know where the common is coming from.
Alex
Comment #10
ekes CreditAttribution: ekes commentedCall to undefined function date_make_date()
date_make_date is a date_api version 2 function
Comment #11
ekes CreditAttribution: ekes commentedIt was only there because it was based on parser_common_syndication, so sure. But, on that point...
This isn't a big issue, but I feel I should mention as it feels big in my underlying design brain.
While I agree that the feedapi itself does not want to be dependent on any particular parser. If there are parsers that are coming collectively with feedapi (or in the future just for feedapi) it seems wrong to reuse code by cutting and pasting. There is some good code in the feedapi_common_syndication.
It would be nice to have a
part of the parsers to reuse. Be it just a set of functions and tables.But like I say it's not that important.
Comment #12
Leeteq CreditAttribution: Leeteq commentedSubscribing.
Comment #13
yngvewb CreditAttribution: yngvewb commentedAlso subscribing.
Comment #14
David Goode CreditAttribution: David Goode commentedHere is a minor patch to the parser_common_ical module, but more important a patch and new info file that make it work with Drupal 6, in anticipation of rolling out parser_common_ical with FeedAPI's Drupal 6 version when ready.
David
Comment #15
Mike Sances CreditAttribution: Mike Sances commentedSubscribing
Comment #16
geodaniel CreditAttribution: geodaniel commentedsubscribing
Comment #17
matt_p CreditAttribution: matt_p commentedHi.
I have managed to add events using FeedAPI Node and the iCal parser.
Am I correct in thinking that the start date of an event is not updated with the DTSTART value in the ical feed? How would I go about getting FeedAPI Node to store the start time in a CCK field?
Thanks
Comment #18
KarenS CreditAttribution: KarenS commentedSubscribing, I haven't had time to try this out, but I'm obviously interested :)
Comment #19
tyromind CreditAttribution: tyromind commentedsubscribed
Comment #20
carnive CreditAttribution: carnive commentedhttp://groups.drupal.org/node/13720
This just came out, not sure if it's useful for you guys or not. Author created a module to take a GCal XML feed and display it in a block.
Comment #21
johnmunro CreditAttribution: johnmunro commentedSubscribing
Comment #22
fm CreditAttribution: fm commentedsubscribing
Comment #23
johnmunro CreditAttribution: johnmunro commentedA couple mods to work with current D6.4 & & FeedAPI 1.3 Date 2rc3 to remove errors on refresh.
Changes to parser_common_ical.module following the .toD6 patch
As _parser_common_ical_feedapi_parse is being used for both compatible AND parse, it needs the same data type for the URL:
Line 31 of function parser_common_ical_feedapi_feed
- $url = $args[1]->url;
+ $url = is_object($args[1]) ? $args[1] : FALSE;
To stop errors, $headers must be declared as an array on line 147 in
function _parser_common_ical_download($url) {
$method = 'GET';
$follow = 3;
$data = NULL;
+ $headers = array();
Comment #24
Renee S CreditAttribution: Renee S commentedOk, I've tried the D6 version and nothing happens... it just doesn't work; nothing happens and when I refresh the node it forgets the URL. Could one of y'all who got it working post what you did?? :)
Comment #25
johnmunro CreditAttribution: johnmunro commentedAfter applying the patches, I think I just followed the instructions in the README.TXT
Comment #26
fm CreditAttribution: fm commentedHas the iCal parser been incorporated in to a Dev version yet?
Comment #27
fm CreditAttribution: fm commentedI've pieced together the module and the patches from above, but haven't gotten an iCal feed to work.
I tried a number of feeds. The following URLs are representative of those that I tried:
webcal://flagfootball.meetup.com/100/calendar/ical/Philadelphia+Flag+Foo...
http://flagfootball.meetup.com/100/calendar/ical/Philadelphia+Flag+Footb...
What is the acceptable URL formats for this? webcal? http?
I keep receiving a "Could not refresh feed" message when I refresh the feed. There is no indication of a problem in the watchdog log. But then I could never get the Common Syndication Parser to work. I could live without the Common Syndication Parser because SimplePie Parser worked, but I'd LOVE to have the iCal parser ... that would rock.
Comment #28
happysnowmantech CreditAttribution: happysnowmantech commentedsubscribing
Comment #29
Renee S CreditAttribution: Renee S commentedI found a few bugs in the 6.x module that David G posted. I didn't get it working mostly because the stupid FeedAPI refused to recognize when I'd deleted a feed and re-added it so it and would come back with "0" items, so it was a HUGE pain to test ... I think that's a bug right there in FeedAPI!
The first thing was that the get_url function was grabbing the feed twice for some reason; it wouldn't recognize the BEGIN because it would see BEGINBEGIN.
I came sooo close, though. I would love to have this functionality. The first person to get it working can have my cat.
Comment #30
Shiny CreditAttribution: Shiny commentedsubscribing
Comment #31
alex_b CreditAttribution: alex_b commentedupdating version
Comment #32
KarenS CreditAttribution: KarenS commentedI was going to try this out to see if I can get the FeedAPI ical solution working for D6, but I'm unable to get either of the files in #14 to download and open in my Windows environment. Are those the current working files? Can someone post a patch or text files instead of the tar files?
Comment #33
KarenS CreditAttribution: KarenS commentedNever mind, I figured out a way to get the gz files working and I'm trying to piece together the right code from the above comments and patches.
Comment #34
KarenS CreditAttribution: KarenS commentedI'm getting some of this to work now, just a few general comments:
1) _parser_common_ical_patched_date_ical_parse() in the patch is no longer needed, it's available in the Date API as date_ical_parse().
2) I'm renaming the module to parser_ical instead of parser_common_ical, as noted somewhere above
3) I'm adding a dependency on Date API, which eliminates the need for the redundant function above and for hook_requirements.
4) There is no such ical item as 'CREATED', the right value for the created date is 'DTSTAMP'.
5) Practically all of the ical items are optional, so we have to check isset() on all of them to avoid undefined index errors in D6.
I'm still plugging through it, I'll post an updated patch when I get a little further.
Comment #35
Renee S CreditAttribution: Renee S commentedThis is wonderful, KarenS, I have a huge project that needs this and my skills are limited to tracking down bugs - I can tell what's not working, but not always how to fix it :) Let me know if you need a guinea pig or tester.
Comment #36
KarenS CreditAttribution: KarenS commentedI needed the following patches for other parts of feedapi to get this working without errors: #324001: Undefined property: stdClass::$feed in feedapi_node.module line 84, #325012: Fix E_ALL and E_STRICT warnings.
Here is the iCal Parser module I got working. It works with the code I submitted at #215979: Create a CCK date field mapper to create nodes that have date fields from iCal input.
Comment #37
KarenS CreditAttribution: KarenS commentedI found a way to get rid of the confusing extra date part elements to make the mapping easier. There are some changes in both this module and the one at #215979: Create a CCK date field mapper.
And here are some screen shots.
Comment #38
Renee S CreditAttribution: Renee S commentedWorks for me. Thank you!!
Comment #39
redndahead CreditAttribution: redndahead commentedAny chance of getting instructions on how to use this module to pull in the groups.drupal.org ical feed? I'm trying to review this and I'm haven't been able to pull in a feed.
Comment #40
redndahead CreditAttribution: redndahead commentedI got it figured out. I missed the part about the feedapi_mapper module and that was the sticking point. It's working for me. I've had a couple of small errors come up here and there, but I haven't figured out where the blame lies.
Comment #41
bigjim CreditAttribution: bigjim commentedKaren
I think line 105 is incorrect from the module code in comment #37. The CATEGORIES element of the $event array is a string, hence you need explode() in lieu of implode(). It could be my feed (an ical feed from civicrm on the same Drupal install)
if I'm correct it should be
I'm running Drupal 6.6, civicrm 2.1.1, feedapi 6.x.1.4
Comment #42
gustav CreditAttribution: gustav commentedKaren, thank you very much for putting this together. Unfortunately I can not get it to work. Here is what I did:
- Downloaded latest 6-1 code for feedapi and feedapi_mapper modules
- Overwrote feedapi_mapper/mappers/feedapi_mapper_date.inc with the code from http://drupal.org/node/215979 #36
- Created new module directory parser_ical and put your code from this thread into it. feed I took parser_ical.info from #36 and parser_ical_module from #37
- Went to admin/build/modules and enabled FeedAPI, FeedAPI Node, FeedAPI Mapper, Common Syndication parser, iCal parser
- Went to admin/content/node-type/feed and chose iCal parse, see screenshot.
- Went to admin/content admin/content/node-type/feed/map and could only see what is shown in the screenshot map.jpg. No ical fields to map.
What did I do wrong?
By the way: the parser_ical module does require the FeedAPI Mapper module, doesn't it? This should be added to the dependencies in the .info file.
Comment #43
KarenS CreditAttribution: KarenS commentedOne problem I see in the screenshot in #42 is that you don't have a date field in the node you're mapping things to. Can you paste an export of the content type you're using for your imported nodes? They need to have a date field that will hold the values you're importing. You will probably need the date to allow for both a 'from' and an optional 'to' date, and set it up to have a date-specific timezone, since you don't know what you'll be getting in the ical feed and it could include a from and to date and a timezone.
I don't have more time to work on this right now, but that would be a starting point.
Comment #44
KarenS CreditAttribution: KarenS commentedAlso I found that for some reason I had to create a new content type for the ical parser -- it wouldn't work right to just change the parser on the default content type. So I created a new content type called 'iCal feed' and set that new type up to use the iCal parser, then created my feed with it.
I don't know enough about Feed API to guess why that was, but see if that works for you.
Comment #45
KarenS CreditAttribution: KarenS commentedTo (maybe) help matters, here are the steps I followed. This is copied from the Advanced help in the latest -dev version of the Date module, along with screenshots you can see if the Advanced help module is installed. Also make sure you have the latest -dev code for the Date module, too.
Decide what content type will be used for nodes imported from your iCal feed. It can be a separate content type for each feed, or they can be combined into other content types. Make sure the content type you select has a date field that can be used for the iCal date, plus fields for any other iCal information that will be imported.
Create a new content type for iCal feeds.
Scroll down to the bottom of the content type edit page and look for the place to set up Feed information. Set it up to be a Feed type that uses the iCal parser.
Go to 'Create Content' and create a new node using the iCal feed content type. When you create the node, you will find a place to input the url of the feed and select the content type for the resulting nodes.
Once the feed node has been created, you will see a tab at the top where you can 'map' its data to the fields in your target content type.
Scroll down and look for options->DTSTART: and options->DTEND and map them to the 'From date' and 'To date' fields. Then map any other information from the feed to right fields in the target node.
Once the fields have been mapped, use the 'Remove items' tab at the top of the page to eliminate nodes that may have been created before the fields were mapped, then use the 'Refresh' tab to re-import them.
You can see in the Content screen in the administration area that the nodes have been imported.
Comment #46
gustav CreditAttribution: gustav commentedKaren, thank you very much for all your help. This helped me to get things working.
My problem was that I tried to set up the mapping for the feed content type on admin/content/node-type/feed/map rather than for the individual feed node.
I don't know the FeedAPI framework. Would it be easy to move the setting up of the mapping of the date fields to admin/content/node-type/feed/map so that one only has to set it up once per feed content type rather than once for each feed node?
Setting it up individually for each node also has the difficulty that if one ticks the "Refresh feed on creation" checkbox on the node add form then the ical event nodes are created before the mapping is set up and are thus created without dates.
I agree with jalama that the fix from #41 is needed.
Comment #47
mckien CreditAttribution: mckien commentedThis is really good stuff (thanks, KarenS!), and has given me just what I need to be able to use our corporate calendars on Drupal. The problem I'm having now is with all-day appointments. They are showing up in the Calendar view either on the day after they actually occur, or on both the day they occur and the next day. Here's what is in the ICS file, for example:
So I see the end date is indeed the day after it should be, which I'm sure is the problem. However, there is also an indicator that it is an all-day appointment which I'm pretty sure is not being used. Tried not mapping the end date, and also mapping the end date to the start date (since most of our appointments for this calendar are all day), but doing either of those makes the dates not show up on the Calendar view at all.
I think I'm totally up-to-date on the iCal parser, the Date module (latest dev version) and the Calendar module (also latest dev version).
Any suggestions on what I could do to make this work?
Comment #48
KarenS CreditAttribution: KarenS commentedThe only thing in that ical file that tells me this is an all-day event is the proprietary 'X-MICROSOFT-CDO-ALLDAYEVENT:TRUE', which is not an ical standard. Microsoft not using the standard, what a surprise! I guess we could write in something to handle this, but I really hate to do something so hackish :(
I don't see any other way to do it, though, since the ical file incorrectly says it ends a day later.
This would really be an issue for the Date ical parser, though. I'm pretty sure we can fix it there and not have to make changes in this module, but I'd have to think about that. Let me mull that over...
Comment #49
KarenS CreditAttribution: KarenS commented#46 - I'm no expert on Feed API, but I think it has to work the way it does. You can't map the content type because the content type could be used to create feeds that need different mapping. You have to map the individual feeds to account for the values that specific feed is returning and the place you want that information to go. You could use the same ical feed content type to create several feeds coming from different places and create nodes either in a single content type or in different content types.
You can't do any mapping until it pulls back the feed so it has feed data to use in the mapper, so the two step process that will create fields without dates before they are mapped is unavoidable, but that will only happen the first time you use it. Once the fields are mapped you don't do that again.
Comment #50
gustav CreditAttribution: gustav commentedKaren, thanks for that attempt at explaining this to me. But I still don't understand. Why, on the page admin/content/node-type/ical/map can I map the author fields but not the date fields? Should what you just explained about the date fields apply to the author fields as well?
Comment #51
KarenS CreditAttribution: KarenS commentedI can't tell you anything else. It works the way I did it, that's all I know.
Comment #52
KarenS CreditAttribution: KarenS commentedHere's an updated ical parser module that will allow mapping of other kinds of dates in the date mapper. It also adds in the fix in #41, but does not yet do anything about #47.
Comment #53
alex_b CreditAttribution: alex_b commentedReviewing now, 2 questions:
1) PATCHED FUNCTIONS: _parser_ical_download() is much shorter than _parser_common_syndication_download() - what's the story here?
2) How do you guys feel about rolling the iCal parser into its own project? I'd be very much in favor of this in order to keep projects small and issue queues clean. Anybody wants to maintain this project? If not, I would step up.
Comment #54
alex_b CreditAttribution: alex_b commentedThis is looking pretty good. Ready to be rolled into a project I would say. I will wait for some responses on
#53 - 2) for that.
For convenience and for the meantime I committed the code to my sandbox:
http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/pars...
There were some minor issues I fixed:
* Minor code style issues
* Cleaned up comment for _parser_ical_download()
* Catch 'webcal' URL scheme and substitute it by 'http'
* Remove FEEDAPI_FEED_NOT_UPDATED comment.
This is really great work. Thank you everybody. I can't wait to use this on a project :) I'm sure there is going to be an opportunity soon.
Comment #55
ekes CreditAttribution: ekes commentedAlso tested (with patch) with D5 and FeedAPI 5.x-1.4 working nicely - thanks all :-)
I could make the time to do a proper job of maintaining this one as a standalone, if people want?
Comment #56
ekes CreditAttribution: ekes commentedProject started:- http://drupal.org/project/parser_ical
\o/ really nice thanks everyone that took part. I've got this running on a live site already. Sorry anyone I missed off the commit message.
Setting this issue fixed. New issues can go in the module's queue.
Comment #57
kleinmp CreditAttribution: kleinmp commentedI put a page in the Drupal 'HowTos' using the directions from #45.
http://drupal.org/node/341716
Comment #58
ekes CreditAttribution: ekes commentedExcellent.
I've updated it with the link to the project and added a link from the project page to the HowTo.
Comment #59
alex_b CreditAttribution: alex_b commentedIt is really great to see the iCal parser come true. Thank you to everybody who helped building it. And a special BIG THANKS for ekes for taking on maintainership.
Comment #60
germs12 CreditAttribution: germs12 commentedFirst let me say Thank you to everyone who has worked on this, what a great tool.
Next, I have a question: If I'm using the CCK to build my content type for pulling in iCal feeds which Date type do I use? It appears that CCK allows for Date, Datestamp, and Datetime. All of which are different from the standard that the iCal implementation uses for the DTSTART.
I can successfully pull the feed into Drupal but I can't get the date part to correctly populate. Has anyone done this yet?
Thanks in advance!
Comment #61
germs12 CreditAttribution: germs12 commentedFirst let me say Thank you to everyone who has worked on this, what a great tool.
Next, I have a question: If I'm using the CCK to build my content type for pulling in iCal feeds which Date type do I use? It appears that CCK allows for Date, Datestamp, and Datetime. All of which are different from the standard that the iCal implementation uses for the DTSTART.
I can successfully pull the feed into Drupal but I can't get the date part to correctly populate. Has anyone done this yet?
Thanks in advance!
Comment #62
ekes CreditAttribution: ekes commentedgerms12 I think what your looking for is in the 'how to' mentioned in the comment above #57 It's not the parsing the ical but the mapping. The sections of the 'how to' about the feedapi_mapper and especially the proposed date field mapper are I think want you want to be looking at.
If I misunderstand you then please open a new ticket on the appropriate module.
Comment #63
ekes CreditAttribution: ekes commentedI'll actually close this one to encourage the start of new issues / threads.