Paypal Recurring Payment

No uc_recurring doesn't and wont include any payment gateways itself, it is all about providing the tools to manage recurring payment and to make it easier for payment gateway to interface with uc_recurring.

But implementing paypal subscriptions is on my personal TODO list and I intend to do some work (if know-one else does) providing patches for the paypal module to interface with this module.

I'm currently using the uc_credit with paypal WPP as a gateway de uc_recurring to handle monthly fees for a site subscription. Client's requirement is to allow users to update their cc information. Since this is not handled by the default uc_recurring fee handler I wrote a custom fee handler that I hope could be integrated to provide additional features to the basic handler. I've attached a simple module I created to handle this.

Ok made a start on paypal support via the Web Payments Standard interfaces (the one where you send the user to paypal's site to enter all the payment details)

I have only done the first part, setting up the subscription. But it has raised an issue.

It will allow you to purchase as many products as you like in the one order but you can only setup one recurring payment in an order - any thoughts on what should happen if a use tries to purchase more then one subscription (should paypal get disabled on the payment page in favour of another gateway that might be able to support multiple recurring payments in the one order), at the moment I have added a warning message that appears if there are more then one recurring fee on an order.

The current patch should allow you to process an order, but it needs some work on the IPN side so that it can properly respond to subscription payments and create a new order for each payment as at the moment its going to add each payment onto the original order.

Just applied the patch. It looks like it works great.

One thing I did notice and haven't looked into yet. I had initially set the recurring fees to last 60 months (I'd prefer to not have it expire, but oh well). When it was set to that I got an 'you have used an invalid link to access the paypal system. I changed it to a 24 month installment and it worked. Not sure why, but I'll look into it.

I'll do some looking around and see if I can figure out how to make the IPN create a new order.

Thanks a bunch for this.

Andrew

I'm wondering if we could re-use the code from uc_paypal_buttons?

Edit: for the ipn stuff. I see he had a comment in for case: 'completed' that looks like the only modification in the ipn code.

Yes we need some error checking for those values, paypal only accepts certain values for the interval between billings for each of the Day/Week/Month/Year selections:
https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=develo...

Regular subscription units of duration. Allowable values:

* D – for days; allowable range for p3 is 1 to 90
* W – for weeks; allowable range for p3 is 1 to 52
* M – for months; allowable range for p3 is 1 to 24
* Y – for years; allowable range for p3 is 1 to 5

But why use recurring payments if you never want it to rebill?

Are you mixing up the number of billings (which can be any number or unlimited) from the interval between rebillings.

This is a monthly subscription. In the recurring payment feature you have to specify the # of installments you want them to pay. I don't see a way to leave it out as it's a required field. So the max I can bill them for is 24 months (which is fine for now). If there's a way around that I'd be happy to hear about it.

In my error it was because I was using 60 months and the max is 24.

there should be no problem with billing each month for 60 months

you need to select
* the payment intervals settings to be first payment 1 month (the product price is what will be billed on checkout)
* regular interval 1 month and
* then from the number of billing periods enter 60 (or tick the box for unlimited billing - if you are using the most recent 2.x-dev where this features been implemented #450214: Unlimited renewals)

One of the problems with the ipn at the moment is that it is using the order number as the reference, but for recurring payments that order number is going to change each renewal - so we need to come up with another reference that we can track the payments to different orders as they are renewed. There is already a reference that paypal creates so that is probably what we need to use instead.

Also another problem with the current IPN code I noticed is that there is no checking that the amount received in paypal is that amount owed on the order, so its actually quite easy for a person to change the information sent to paypal and pay a different amount which would give them the product for cheaper then the price you are charging.

Yeah, I forgot to delete the old uc_recurring from the ubercart folder. Man I need some sleep.

some more work done here on paypal integration that might be useful to this feature: #334394: Integrate PayPal's Recurring Payments API

This works great for me on a single membership / role sale on a recurring basis. Just a thought on the issue raised by univate in comment 3 above; anyone who really needs to could implement the restrict quantity module to limit the number of products that can be bought which might act as a temporary solution until the issue is fully resolved.

Regards

Patrick
UnderDesign | Follow me on Twitter

We are tring to implement uc_recurring payments with Paypal Website Payments Pro. Using the latest patches for uc_recurring and uc_paypal we are able to process the payments but are having difficulty getting IPN to work correctly. We are also running into the issue of multiple products that have recurring payments.

Is there anywork being done on Paypal Website Payments Pro IPN integration and handling multiple products?

Thanks
Josh

The patch in #3 is for Web Payments Standard, there is a link in #11 to a patch that has some work done on Web Payments Pro implementation (although that was based on uc_recurring 1.0)

For the IPN Problem
We have patched together stuff from Patch #3 and stuff from issue 334394. We then made some of our own code changes that got the IPN working with Website Payments Pro. I will post a patch here once we get code cleaned up. The biggest problem was that PayPal does not store the IPN url with recurring payments. I spoke with Paypal tech support and they have entered it as a feature to add but I was given no time table. So now instead of the URL containing the order id we are getting the order id out of fields on the IPN .

To make this work we did 3 things
1. Set Paypal IPN settings to have a static URL "https://www.doman.com/uc_paypal/ipn"
2. On the recurring payment - Creation and Initial Payment, we are using the "CUSTOM" field and the "INVOICE" field to get the order#.
3. On the recurring payment - Scheduled Payments, we are using the "PRODUCT NAME" Field. This is because the "CUSTOM" field and "INVOICE" are not part of the IPN on Scheduled Payments.

I have just done some work on Paypal WPS IPN (patch attached)

I haven't really used WPP in the past so that might be different to WPS which I am sure does use save and the IPN notify url to use on recurring payments, what I have done for WPS in the attached patch is use the order_id just to identify if that payment is the first, after the first payment I use the IPN subscription id which I store in the uc_payment_paypal_ipn database table to find the correct recurring fee to generate a new order for and then process the renewal.

We actually need the recurring payments to be applied to the first order.

It is more like you bought a widget for 5 easy payments of 29.95. verse a recurring subscription that creates a new order for each payment. I do like the idea of using IPN subscription ID for getting what to apply it, that sounds like a much cleaner way of doing it.

How would you recommend applying it to the first order? It seems like a minor change to this if statement in your latest patch:

if ($fee && $order_id != $fee['order_id']) { // if this is not the first payment we need to create a new order in ubercart

I tried the patch but got

File to patch: uc_paypal.module
patching file uc_paypal.module
Hunk #1 FAILED at 16.
Hunk #2 FAILED at 65.
Hunk #3 FAILED at 90.
3 out of 3 hunks FAILED -- saving rejects to file uc_paypal.module.rej

Of course I'm a noob at patching and could have totally blown it.

I see nothing at
/admin/store/settings/products/edit/features

to select a payment method. It looks like there should be selections there but nothing. What am I missing?

Thanks for any guidance.

Phil

Hi Phil,

Have you already patched the paypal module? I think you need to use an unpatched version of the file for univates patch in #16.

Here's what I'd do:

download ubercart
copy the patch file into ubercart/payment/uc_paypal (? think that's right)
I use linux, and this command: patch -p0 < ubercart-uc_paypal.patch

That should work.

@univate... I've had wps recurring up and running for a bit, but have been so busy I haven't been able to poke around and make sure everything works. I've got a bunch of recurring payments being processed next week and let you know how it goes.

Using uc_recurring to sell a 1 month membership role so hopefully the roles get renewed :).

Hi Andrew,

Thanks. I had been struggling with cygwin on windows. Switched over to my linux machine and followed your directions. Works great and everything now appears to be working.

I too am curious as to how payments are processed. This does not appear to use the paypal reccurring feature (or does it) so are the payments still made automatically or does the user have to authorize.

I'll set up a test to play around and see.

Thanks for this module. It's really a great addition

Phil

Some further work on my paypal patch - I have tested the entire payment process and renewal and it appears to work.

I have also had to make a small change to uc_recurring which I commited to give the option so that gateways like paypal can take over the triggering of renewals and instead of uc_recurring.

After enabling the recurring payment feature on a product, I am seeing "Illegal choice credit in Payment method element." logged. This disappears after removing the feature. I'm attempting to use the module with Paypal to create a monthly subscription for access. I used the latest patch against ubercart, with no change in behavior. Are there additional patches for uc_recurring or do I have that if I have used the latest developement release?

Here is another patch to fix a couple of small problems including uc_recurring trying to renew fees, recurring fees are renewed only when an IPN occurs, also requires the most recent version of uc_recurring 2.x-dev

@shadgun this patch only gets Web Payments Standard (WPS) working with recurring fees, you just need uc_recurring 2.x-dev and the curreent patch found here. The error you are getting looks like you are trying to use a 'credit' payment method, possibly Web Payments Pro (WPP)? Also make sure you have the latest verision of ubercart without uc_recurring included (you need to be using the version of uc_recurring from this project)

subscribing

Had the patch installed for a couple days and processed a couple new recurring fee orders. With the new patch the orders are showing up under 'recurring fees' which is great. Have to wait for a bit before I know whether the whole process works, but it looks promising.

Thanks!

You need to configure the recurring product feature from:

Store configuration -> product -> product feature

And select which payment gateways you have installed should be used for recurring payments.

I'm finding a strange occurrence, when I allow users to go through the checkout process for a subscription with recurring fee - as an anonymous user, the whole process works fine with paypal and recurring fee set up on the sandbox side, but when the new user then logs into their account and views 'recurring fee's' under their account details nothing show's, but the order does.

however when the now registered user places the same subscription again, and checks back into account details the recurring fee show's in the list ?

does the recurring fee only show in the list when the first recurring fee is taken ? I didn't think it did.

It seems the 'Next' date for recurring fee's always stays on the current date, I'm testing by doing a daily subscription charge, so I started it today on the 03/07/2009 but the 'Next' field is still showing 03/07/2009 !

Also how do you let the customers delete recurring fee's from their list in 'My Account'. I know they do this officially with their own paypal account - but's it's nice to clear off old recurring fee's that might not be active anymore. I've allowed premissions for it - but I see no option availabe in the 'operations' field?

subscribing

sub

AAAALLRRIGHT! After more then 12 hours struggling to find a clean solution for paypal subscription and role assignment, finally it works! Kudos!

For people still struggling: use the latest DEV version of ubercart and patch of unicate at comment #23.
Here my patched ubercart module, but if you know how to patch, please use latest dev version.

Talked too soon. The subscription payment works, and it is viewable in ubercart.
But when returning to the site from paypal i got a whitescreen.
Also i did this in combination with role assignment in ubercart, and no role has been assigned.

You got my hopes up so high betz! I've been refreshing this page for days.

@AstroMix: did you try it out? With me it works i think. No more white page. Only i really can't test for the moment because of the lack of credit cards :s

Tried it with a sandbox account but the order that was created tells me 'Payment is pending at Paypal'.

And in the automated 'admin comments' i find the following memo's:

17/07/2009
12:19:49 PM	-	The recurring fee for product prem1day failed.
17/07/2009
12:19:50 PM	-	Payment is pending at PayPal: Reason unknown; contact PayPal Customer Service for more information.
17/07/2009
12:19:51 PM	-	Order created through website.
18/07/2009
12:24:10 PM	-	Payment is pending at PayPal: Reason unknown; contact PayPal Customer Service for more information.

If someone can test out with working paypal accounts, that would be great!

I have the same Message- 'Payment is pending at Paypal' - when using the sandbox. I doubt that it works on the the live site, because lm_paypal does complete the transactions in the sandbox.
the message in the paypal sandbox is "waiting for website to complete transaction" or similar.

[EDIT]
Ok I tried with a real second credit card account and the transaction does complete. great!! This is with "website payments standard" enabled.

betz: For some reason I was under the impression that you were working on Website Payments Pro -- is that incorrect? I don't mind testing out the patch if so.

subscribing....

@ AstroMix: no, i didn't write the patches, just shared my patched module so anyone can test it.
Did you test is already?

After some more testing of Betz's patched version there ist one problem remaining: how do users cancel their subscriptions? As I understand the recurring payment process it is initiated by cron and the uc_recurring module. the user's paypal account only grants access. So we do have to cancel the recurring from within our drupal, not from paypal.
I also see some code for cancelling in the module, but there seems to be no user interface (no button under user/*/recurring-fees/operations)
Next Question is: what happens when the user cancels the payment from paypal? drupal doesn't get notified, because there i no API. Does uc_recurring notice when the next payment is not successful, and set the recurring to 0? Otherwise users would end up with multiple subscriptions if they subscibe again.
Thanks for your work on this! It's very much appreciated!

Here is another issue that came up: recurring payments don't contain Taxes. If I set up a global tax, things get quite confusing. It's of course processed correctly in the first order. But then:
- When i set the recurring price to the same as the product price, the tax is not added to the recurring amount, leading to a smaller recurring amount at paypal. But still uc_recurring expects the amount to include the tax, leading to a mismatch of the amounts when recurring payment is made. It seems to me that uc_recurring books the (too small) amount twice in this case :)
- When i set the recurring amount higher than the product price, so to include the tax, recurring payments are processed correctly (expected amount=paypal transfered amount). But: The new orders generated by uc_recurring contain only only one amount (no extra tax), also the invoices are wrong in this case (my whole point of using ubercart for this are the invoices)


And here's another one: When I set the first renewal to +1 day, paypal in fact adds 2 days until the first renewal. This leads to a gap of one day without payment. When setting first renewal to "0 days", things seem to work (paypal reports "[amount] every day" and starts right away).


If someone has time to work on this, and would accept some sponsoring, I'd be glad to help :)

Following the discussion from #483494-10: changes to ubercart to work with uc_recurring I think that the same approach should be taken here. IMO that change that should be done in uc_paypal is providing a hook or maybe drupal_alter(). Like this uc_recurring can implement it's own code, without Ubercart needing to know about it.

I tried the patch from #23 for the ubercart dev bundle, then when I do any checkout using paypal I am getting following error at the paypal page with a return to merchant button.

The link you have used to enter the PayPal system is invalid. Please review the link and try again.

I also tried using ubercart.zip from #32 still getting the same error.
I am unable to find what is the problem, can some one help me.

Well my first customer who signed up has made it through the 1 month recurring period and unfortunately it does appear that they were charged.

I see nothing in the logs that it failed or even tried. Anyone have any ideas where to go from here as far as trouble shooting and getting the payment.

I know I shouldn't have gone with this on a live site yet but that seems to be the way I roll lately

and it's the only all around solution to a drupal shopping cart.

Love what you guys are doing

Phil

Hi Phil,

This patch provides the solution for paypal recurring payments. So we can see that information by logging to your paypal business account. Unfortunately we can not see those payment details in drupal. I am also looking for a way to identify when a recurring payment occur.

Hi Azri,

yes i am using paypal recurring. So hopefully it is working. Not sure it's my partners paypal account so I'll have to watch for it.

Will subscriptions created prior to the patch start working with the patch. I'm thinking not.
but you think future subscriptions will work?

the reason I went with ubercart rather than lm_paypal was to be able to implement the affiliate program. So if ubercart is not seeing recurring payments then the affiliate program will not work. HHHmm

Does it look like this is something that will work when all the kinks are worked out?

Maybe I should switch back to LM-paypal for a bit til this gets resolved. But I sure like ubercart and would love to see this work.

Phil

Univate,

I've been testing this patch for a site I am building currently and there's something I don't understand that I am hoping you could explain:

Why does the patched module process the transaction as a Paypal Subscription? Why not as a regular charge? If the uc_recurring module is meant to manage recurring fees, then why put the subscription management/cancellation in paypal? That's confusing. I would expect the module simply to charge the user's paypal account according to the interval uc_recurring settings for the product and provide a cancel link in their profile, leaving paypal out of the loop except as a simple payment processor for checkout and when the interval comes due.

As it stands, I would have to somehow explain to users that they must visit their paypal account to manage their subscription. That means extra (and potentially confusing) instructions to a user. It also means that they have to leave my site to manage/cancel their subscription. When I join a subscription based site, I expect to manage my subscription from that site.

Additionally, if I understand the code correctly, processing as a subscription means you can only process 1 at a time at checkout. With this limitation, we are unable to use other modules (ie - Discount Framework) to affect the pricing in the cart. For example, we have 3 subscription types and want to offer a discount should you buy more than 1 (Buy 1 for $150, 2 for $250, 3 for $325). Were it a regular charge, would it not then be possible to process multiple recurring fees in a single checkout? One could then use UC Restrict Qty place limits on the quantity as desirable.

uc_recurring is designed to manage recurring payments in drupal/ubercart. It is up to the individual payment gateway modules to interface with the gateway to process/charge the payments. For some gateways that may mean that uc_recurring will trigger the payment to occur when its due or in the case of third-party hosted services like paypal, it might just have to interfaces with their systems to check if a payment has occurred as then triggering events in ubercart to renew a role or create a new order to ship a product.

will the patch for the uc_paypal module get incorporated into the ubercart module?

Subscribing

Have anybody tried to implement Paypal Website Payments Pro recurring module (not a patch) using uc_recurring/uc_paypal hooks? Is it possible? Or it is not currently possible to implement this without hacks/patches? Why?

There has been work on integrating all the ubercart core payment gateway code into this module (#534296: Make uc_recurring completely independant from ubercart, uc_recurring will handle all ubercart core payment methods via inc files). This means we wont have patch ubercart or get anything commited into ubercart and we can be free to continue updating and improving uc_recurring as we need to.

What that means is we will be combining all the paypal website payment standard code thats been written here into one *.inc file which will be part of uc_recurring so everything will just work by downloading uc_recurring

This post is Just to let everyone know where things are going and hopefully we will have something to download test over the next few weeks.

When you say "completely independent" do you mean it won't even use the XML generation, etc. functions from the main ubercart module, or just that the wrappers and hooks will be provided in this module so that you don't have to make any commits to the other modules (but it will still call the main module's code)?

Not requiring the other ubercart modules at all would mean fixing bugs in both places, which would be unpleasant.

No this is still definitely a module that is dependant on ubercart and where possible reuses ubercart's api's.

At the moment we have the problem that uc_recurring is an api for handling recurring payments which calls the payment gateway to handle the actual payments, this requires updating each payment gateway to make is uc_recurring aware. Problem is ubercart bundles up a number of key payment gateways and we need to get our changes/patches into ubercart. This doesn't give us much flexibility to try things out if we have to keep going back to ubercart to update our changes.

This is about trying to make everything easy to get something out there everyone can test by downloading and installing (without patching ubercart)

Univate,

Just wanted to let you know we sure do appreciate your work on this. It is a much needed module.

Phil

I've decided to close off this issue and will start up separate issues for developing paypal wps and paypal wpp gateways

Be useful if you paste a link to that issue, so we can pick up the scent if we Google on in here. =)

Edit: I guess it's this one - #569300: Create Paypal WPP recurring fee handler

we are working around this same issue of recurring payments not including Tax.
has there been any work to resolve this issue in the last 12 months ?

This issue is closed and not directly about taxes. You will have better luck if you post on a tax-specific issue (search first, then create one if it doesn't exist). More detail also means it's more likely that it can be resolved.