($150 bounty) Import Friends for Pro+ accounts

Hi,

Yes, with Janrain Pro you can import information about users'
friends. The 7.x-1.x module supports that feature (called "Request
extended profile data" in the settings). The module, however, has no
support for inviting them at the moment.

Cheers,
George

Hi, any chance we could get this for the 6.x branch as well?

Thanks
Srini

Yes, there are plans to backport the D7 module to D6.

In fact, There is a subtle difference between Janrain Plus and Pro concerning friends lists : With Janrain Plus you can retrieve extended profile data, which provides friends profile urls only for facebook (at the moment at least), whereas Janrain Pro lets us use the get_contacts API call, which covers almost all providers and has more data. See the API docs : https://rpxnow.com/docs#api_auth_info and https://rpxnow.com/docs#api_get_contacts.

I am currently working on a module to retrieve friends lists, it uses extended profile data or get_contacts when available. It uses the Flag module to mark users as friends. I was about to create an issue for this today ! I am willing to submit the module here and see if it can be added as a submodule to rpx or if it should be a separate project. It is a 7.x module for the moment. I'll post the module later today, stay tuned.

@jide: Yes, we'd be very interested to see what you've come up with for your module. I think there's a lot of room to do some very cool stuff here.

In addition to the Flag module (which I like very much), one additional possibility is some integration with the User Relationships (http://drupal.org/project/user_relationships) module. I've worked on the D7 port of that module and it is quite stable and much, much improved feature-wise and bug-wise from the D6 version. It lets you set up any number of two-way or one-way relationships (friend or fan/follower) that can require approval or not.

So using that module it would be possible to create a "Facebook friend" relationship, a separate "Myspace friend" relationship, and a third "Twitter follower" relationship within your Drupal site. All of these users could just be regular "friends" as well.

--Ben

@BenK : Still working on this, this is more complex than what I thought - anyway if we want something solid and done right ;) I'm not so sure about the use of Flag finally. The best approach might be Rules integration, so users can assign fields as they want and eventually use modules as user_relationships or flag or flag_friend or whatever :). I'd be glad to have your opinion on that.

Issues/questions I have :

• How to store friends data ? rpx uses the data key of user, but this has downcomes : It is loaded with user and will be too much data I think. So I began to store friends data in their own table, but I'm thinking about having an entity for that, or to not store data at all and let rules integration do the rest. I'm not even sure storing these data is useful since the key thing is to grab profile urls and find corresponding accounts through authmap table. But some data such as email addresses may be useful to discover user account also.
• Retrieving friends takes time (a user maight have hundreds of friends and the API call can be long), so I had to introduce some AJAX mechanism to handle it. But maybe it should be on cron or something, in background anyway.
• At what frequency should we retrieve friends ? The problem is that if we grab friends only at account linking, when a user adds friends from facebook e.g., these new friends won't be saved afterwards, though it may be nice to see new friends.

Any thought ?

I also had some thoughts and issues with the rpx module itself, I'll create separate issues for these.

Hey jide,

You pose some good questions. I put some thought into the issues you raised and here's what I'm thinking:

The best approach might be Rules integration, so users can assign fields as they want and eventually use modules as user_relationships or flag or flag_friend or whatever :). I'd be glad to have your opinion on that.

>> I like the idea of using Rules integration for this. I could envision a "Load a user's friends/contacts" action that is supplied by the module. After this first action is triggered, we could trigger a subsequent action--either the "Set a data value," "Add list item," or "Create a new entity" action (depending on how the site admin wants the data stored). Additional actions could also be triggered in modules like User Relationships that already have rules integration.

How to store friends data ? rpx uses the data key of user, but this has downcomes : It is loaded with user and will be too much data I think. So I began to store friends data in their own table, but I'm thinking about having an entity for that, or to not store data at all and let rules integration do the rest. I'm not even sure storing these data is useful since the key thing is to grab profile urls and find corresponding accounts through authmap table. But some data such as email addresses may be useful to discover user account also.

>> I think storing some data could be useful, especially for those friends who do not yet have Drupal user accounts (but that you might want to invite to become users using the Invite module, for instance). I potentially like the idea of creating an entity to handle this with some pre-defined fields for common data that is stored. But I think this should be done in a separate sub-module so that the site admin only needs to enable it if he really wants to store data in this way. We would just need to make sure the namespace of this entity doesn't encroach on other modules... we should avoid a generic namespace like "friends" for example.

Retrieving friends takes time (a user maight have hundreds of friends and the API call can be long), so I had to introduce some AJAX mechanism to handle it. But maybe it should be on cron or something, in background anyway.

>> Agreed. Definitely an important consideration. For instance, in the Mailchimp module (http://drupal.org/project/mailchimp), we had to implement some batch processing settings to deal with this problem.

At what frequency should we retrieve friends ? The problem is that if we grab friends only at account linking, when a user adds friends from facebook e.g., these new friends won't be saved afterwards, though it may be nice to see new friends.

>> This problem could largely be solved by Rules integration. The site admin could specify whatever events are needed to achieve the desired frequency. For instance, the site admin could configure the rule to be triggered on the "After saving a new user account" event, the "Linked account was added" event (an event currently supplied by the module), and the "User has logged in event". A site admin could also provide an "Update your friends" link to the individual user using the Flag module. To do this, the site admin would create a Flag called "Update your friends" and make it visible as a link. Because the Flag module automatically provides a Rules event for each flag, we would simply use the "Update your friends has been flagged" event to trigger the "Load a user's friends/contacts" action.

Since George (geokat) is the primary maintainer and coder of this module, I'd love to get his thoughts on all of this. George, what do you think?

And jide, have you made any progress on actually coding any of this?

Cheers,
Ben

Hey Ben,

Since George (geokat) is the primary maintainer and coder of this module, I'd love to get his thoughts on all of this. George, what do you think?

I like the general idea of being able to import friends and maybe even
send invites to them. But at the moment only Google and Windows Live
return email addresses for the contacts.

Also, since importing such a large amount of data may take time, using
cron might be a good idea. The admin could configure the cron job to
go through all the authmap entries updating the friend lists. This
could be done at night, for example, when the server load is minimum.

Another question is presenting the friends data. We could have a panel
accessible to the user, where he would be able to go through all his
imported friends and maybe enter and save their emails (since that
information is only available for Windows Live and Google accounts),
and maybe even send them invites to join the site.

Cheers,
George

Hi Evelyn,

There is no support for the invite feature in the module at the
moment.

We are working on adding this to the D7 version and then backporting
it to the D6 version of the module.

Please stay tuned.

Thanks,
George

Hi George and Ben,

Just to keep you updated, I'm still working (hard) on this. I've taken your (good) thoughts into account, and will support rules, have optional rpx_friend entity, views integration, UR integration (through rules), and provide a mechanism to use cron to refresh friends lists (probably using rules scheduler though).

The toughest part is to support both using entities and not, since things are slightly different in a case and in another. E.g, when not using entities, all friends are retrieved through the API call and data is passed to rules (and to a hook). But when using entities, data retrieved is stored in the DB in the form of entities, which lets us know what friends are new, already existing or to be deleted (user may have removed a friend from his facebook account but it may still be stored in the DB). That leads to the fact that when not using the entity, features are more limited. Taking all these aspects into account makes things a bit more complex ;)

Another thing you might be interested in is that I worked hard on the rules integration, and this part could benefit the rpx module itself : All fields that may be retrieved through API calls (profile, poco etc.) are available in rules, which lets users map those fields using rules. I think this would greatly enhance the rules integration part of the rpx module. Generally speaking, this module may have several things that could be useful to the rpx module.

That's a lot to talk about, and I will share code soon - I've made good progress and I'm almost done, but I definitely want to have a first feature complete version beforehand.

Hope you don't mind if I assign the issue to myself.

I'll keep you posted.

George and Ben,

I have met some new technical difficulties I didn't expect and since I'm missing time to complete the development, I have to postpone the development of the module for the moment.

When making a get_contacts call, the token generated during auth_info is used on the Janrain side to ensure the rpxid is still valid. But if it is a long time since the auth_info call was made, the token is no more valid and thus get_contacts fails. That's unfortunate since it means it won't be possible to refresh friends lists on cron - we'll have to so it right after the user logs in.

I'm currently running out of time to work on this, but I'll share the rules integration part that rpx may use in the rules integration issue.

jide,

Can you post what you've done sometime soon? We can take a look and possibly build upon your work.

And George, what do you think about jide's auth_info token expiration point?

--Ben

Ben,

I'm in a hurry today, but I come back to you next week.

Unfortunately, this will have to be postponed until a future release.

Hi all, I'm still planning to work on this, now that I have time for this again.
About the token expiration problem, I discovered that this is a Janrain setting.

jide,

That's great. The best approach, I think, is to do things incrementally and don't make this project too big from the beginning. If you break it down into smaller chunks and post more frequently, others will be able to help.

--Ben

@BenK,

Definitely, I often tell this myself... So let's share some code. I share the code I've made as is, it is unfinished, has inconsistencies, probably has lots of bugs, may even be unusable, TBH I can't remember where I left the code when I was working on this, and I did not retest this since then. Consider this as a work in progress ;)

First of all, having decided to extensively use rules integration, I quickly came to the conclusion that what exists in rpx_ui.paths.inc is really close to what should be implemented in hook_rules_data_info() to provide fields retrieved by Janrain. Rules could be aware of Janrain dictionaries such as "profile", "merged_poco", "accessCredentials", and so on. I think we could factorize here, using the structured array needed by rules for rpx_core. Moreover, this data is the same for both friends and user profiles, so this could benefit to both friends import and profile fields import. In the module attached, there is a submodule named "rpx_dictionaries" that could be used instead of px_ui.paths.inc with all rpx dictionaries. I realize I did not remove the previous implementation of this, which was a "rpx_friends.spec.inc" include instead of a separate module. One of the things that is still to be done.

Secondly, I have created a "rpx_friend" entity to be used when importing friends - that's the rpx_friend module.

These are the most important aspects of the module.

Again, I'm sorry if this feels like half-done work, but I prefer to share something at this point instead of nothing. Let's see what you think, and discuss, in particular about that idea of a rpx_dictionaries module, which could imply some changes in rpx module itself.

Errr, sorry about the .svn folders.

I decided to rework this from scratch with a much simpler approach, I'll be back with some code later today.

Okay, here is a first patch that adds friends functionality. I used a completely different approach from the first module. It sticks to the rpx module logics and does not rely on Rules anymore. I think this is a much more reasonable approach.

It consists in 2 submodules: rpx_friends and rpx_friend (singular), which is an entity for friends that don't correspond to an existing user.

I had to change some parts of rpx itself in order to share some functions between rpx and rpx_friends.
In order to make rpx_friend entity creation work, you must edit and save settings for the fields in admin/config/people/rpx/mapping/friends for now.
Reviews, comments, anything welcomed !

Supports :

• For existing users:
  • User Relationships
  • User reference
  • Flag
• For non-existing users:
  • A dedicated rpx_friend entity
  • Views integration
  • Basic Rules integration

Todo:

• Asynchronous adding of friends through AJAX
• Update on cron (?)
• Advanced Rules integration

Hmmmm... Damn, I used the 1.x branch. I hope it won't be too problematic.

Seems it will. I'll work on this to catch HEAD.

Hey jide,

The branch has been renamed, so 2.x is actually the continuation of 1.x.

More info here:
http://drupal.org/node/1107082

Sorry about that.

Thanks,
George

subscribing to the D6 backport of import friends features.

Here is an update.
Patch now applies to 7.x-2.x-dev.
I made some progress on features such as friends entities creation/update/deletion, processing on cron, bugfixes...
There is still work to do, but there is good progress.
I would like to rework the API so that user_relationship / flag / user_reference / entity creation stand in submodules and use proper hooks.
Cron processing still needs to be worked a little more and AJAX mechanism is still to be done.
Indentation may use tabs, I'll correct this in a future patch.

Another update. Implemented AJAX support. Still has rough edges.

+1 subscribing to the D6 backport of import friends features.

Okay, we are close to something really functional now.

Notes :

• Refreshing friends on cron has been abandoned for now. Since the auth_info token has to exist when retrieving friends, and since this is in the $_SESSION variable for each user, we can't refresh friends on cron.
• An "invite" link is available in views when using friend entities.
• Each method to relate friends (user_reference, user_relationships, flag, friend entity) now have their own submodule.
• Applies to latest 2.x dev
• It could live in a specific module instead of a submodule of RPX, but changes to RPX module would still be needed.

It would have been *really* nice to have some feedback, especially from BenK and geokat ;)

There was a bug when adding a new mapping for friends. Here is an updated patch.

Some more bugfixes.

It would have been *really* nice to have some feedback, especially from BenK and geokat ;)

Hi jide,

I'm sorry I can't review your patch at the moment. I'm swamped with
the new features I'm working on and the backporting effort. But it
looks really cool.

What is your outlook on porting this code to D6? How difficult do you
think it would be? Does it rely on things that are not available in
D6?

Thanks,
George

Hey jide,

Nice work! I'll be testing your patch shortly. To help George and I, can you re-state what changes to rpx.module are necessary for your new functionality to work?

Looking forward to testing,
Ben

2 Questions:

1.) Has there been any movement on back-porting this to 6.x?

2.) Does the current patch function similar to the Mahalo case shown on Janrain's website? Is it somewhat "plug-n-play"? ref: http://www.janrain.com/products/engage/invite-friends

Thanks!
Kevin

subscribing.

Hi Eve,

No news so far. It's on the immediate roadmap, but I don't have a date
that I can share with you at this time.

Cheers,
George

@jide: I set aside some time this weekend to test your patch in #36, but it won't apply using "git apply". Here's the error I'm getting:

git diff header lacks filename information when removing 1 leading pathname components (line 5)

Can you re-roll this ASAP for the latest 7.x-2.x-dev? If you post a working patch soon, I can test it right away! :-) I'd like to help you get more momentum behind this as it looks very cool.

--Ben

Hi Evelyn,

I don't think this will be implemented in the immediate future. As
for sponsorship, you can try inquiring through Janrain professional
services if you want.

Cheers,
George

Hi @nyleve101, I'll see if I can rework the patch to be applicable with the current branch. I did this for a client and a lot of time elapsed since the patch, so I could not sync the patch with latest commits.