I have been working a the uc_recurring_order module (#628066) to fix a number of issues I was having, specifically when using the WPS gateway. I have done a rough fix for the following issues:

  • Recurring options was a blank list on cart and checkout pages.
  • When using WPS the preview form was not being set with the correct _cmd setting so the payment was not being treated as recurring by Paypal.

I and am starting to find my way around the module, but my fixes are a bit rough and ready at the moment so I would like to tidy them up a bit before submitting patch!

So far I have managed to get a recurring payment submitted to PayPal WPS - that is working fine. The payment gets processed and the UC Order gets updated as expected. However, when the payment recurs (next day), the order isn't getting updated with a new payment. The IPN is successfully received, and is listed in the payment log for the order, but the payments pane only lists the initial payment and is not listing the subsequent recurring payments, Order comments or Admin comments fields for the order.

I am assuming that as I am recurring the whole order (using uc_recurring_order) then perhaps something is missing which is telling the main uc_recurring mechanism that this is a recurring order and so it is ignoring subsequent IPN requests?

Any help would be gratefully received and I am happy to put whatever development time is required to get this sorted.

CommentFileSizeAuthor
#26 uc_recurring_order-paypal_wps.patch7.2 KBlookitscook
#25 uc_recurring_order-paypal_wps.patch9.37 KBlookitscook
#24 uc_recurring_order-paypal_wps.patch1.92 KBlookitscook
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

univate’s picture

univate CreditAttribution: univate commented

Firstly what version of ubercart are you using? I have a concern that the following issue: #781010: Patch to Ubercart might break recurring Paypal WPS could cause us some problems with the next ubercart release. So if you are able to test with the latest ubercart dev that would be great as we may need a solution to that.

If we can make paypal work more like other gateways (e.g. test_gateway) that would be good to keep consistency.

As for your main issue, it should be creating a new order not adding the payments under the previous order.

hevmills’s picture

hevmills CreditAttribution: hevmills commented

Apologies - I should have mentioned the version of ubercart: 6.x-2.2. I have now updated to the 6.x-2.x-dev version for testing.

I have sent a manual IPN notification from my Paypal Sandbox account. I can see the IPN arriving and being verified in the main Drupal logs, however there are no additional orders being created. Looking at 'All Orders' I can just see the original order, and nothing else.

I read issue 781010 and agree that it would be wise for each gateway implementation to be consistent and so I am happy to carry out any testing required. I am happy with PHP, but confess that my knowledge of the inner workings of Drupal is a bit sketchy. How can I find out where to start looking in the code to see what happens when the IPN page is called (http://myhost.blah/uc_recurring_hosted/paypal/ipn/3)?

hevmills’s picture

hevmills CreditAttribution: hevmills commented

Thanks for the help so far. I think I may have now got to a genuine problem related to issue #781010: Patch to Ubercart might break recurring Paypal WPS . I can get to the checkout page and fill in my details fine. When I click on to review the order the browser gets stuck in a 302 redirect loop.

On review, the hook_order function is called (uc_recurring_order_order) which then calls uc_recurring_order_process_order which in turn calls uc_recurring_invoke($fee->fee_handler, 'process callback', array($order, &$fee) where $fee->fee_handler is set to 'paypal_wps'

I have not got any further other than to ascertain that if I put in a return before the call to uc_recurring_invoke then the page doesn't go into a redirect loop. Obviously there are lots of errors: 'our order cannot be completed, because we could not process your recurring payment....' when the page comes back, but it doesn't loop.

I will continue to investigate and see if I can lock this down a bit further.

hevmills’s picture

hevmills CreditAttribution: hevmills commented

A bit further on. The redirect loop is being caused by the line drupal_goto('cart/checkout/review'); which is called at line 1015 of /modules/ubercart/payment/uc_paypal/uc_paypal.module

It would appear that the function uc_paypal_wps_form() is being called when the checkout page goes to review. This function then loops through all the implemented order modules, one of which is failing. This failure kicks in drupal_goto('cart/checkout/review'). Rinse and repeat.

The invoked function call which is failing is to uc_recurring_hosted_paypal_wps_process() which does a simple test on the contents of $_POST to set the subscr_id. Isn't the subscr_id set when the PayPal website make an IPN call? I am wondering if this is a problem caused by the fact that WPS is a 'call back system' and these function calls would normally be invoked at the point of checkout? Certainly the $_POST array is empty if I dump the contents of it within the uc_recurring_hosted_paypal_wps_process() function.

In short - if the uc_recurring_hosted_paypal_wps_process() function is being called on the checkout/review page then it shouldn't perform drupal_goto('cart/checkout/review') if part of its processes fail. Is the model wrong here, because I can't see any way that $_POST['subscr_id'] is set outside of a PP wps IPN call?

univate’s picture

univate CreditAttribution: univate commented

Firstly thanks for looking at this paypal issue, I knew that that ubercart patch was going to cause us problems. But this does need to be fixed.

Not sure if you have read the uberart change, but basically they have added a module_invoke in the paypal module to invoke hook_order(). The hok_order() is where uc_recurring was normally doing its work to setup the recurring payment. But in paypal because that hook never got called we had to hack out a solution to setup the recurring stuff separately.

Although that recurring stuff was only looking for the recurring product fees and therefore I don't think it will work by default with the recurring orders.

hevmills’s picture

hevmills CreditAttribution: hevmills commented

No problem - I have a customer who is really putting the heat on me to get this fixed. I am migrating him to Drupal from a custom site I did for him a few years ago. Everything is set to go apart from these issues here so I am more than happy to help get this sorted.

I discovered earlier that the recurring orders doesn't work with wps and have created a fix for this. Would appreciate your thoughts as to the best way to implement it though:

I have created a new version of the uc_recurring_get_recurring_products_in_order() function which fills out an array in the same format, but based on the contents of a recurring order ($order->data), rather than recurring products.

I am wondering where the best place to call this is? At the moment I have modified the uc_recurring_hosted_form_uc_paypal_wps_form_alter() function in uc_recurring_hosted.module with an if else block:

  if(isset($order->data) && $order->data['recurring_option']!=0) {
    $recurring_fees = uc_recurring_store_get_recurring_products_in_order($order); //New function to examine recurring order
  }
  else {
    $recurring_fees = uc_recurring_get_recurring_products_in_order($order); //Original call for recurring products
  }

Would this be the best route, or should I modify the uc_recurring_get_recurring_products_in_order() to call the new code and leave uc_recurring_hosted_form_uc_paypal_wps_form_alter() intact?

I have read the thread about the changes to ubercart. I will re-read more closely and see if I can work it out!

hevmills’s picture

hevmills CreditAttribution: hevmills commented

Following on from #4:

With the patch from #781010: Patch to Ubercart might break recurring Paypal WPS the function uc_recurring_hosted_paypal_wps_process() is called (file: uc_recurring_hosted.module). This checks !empty($_POST['subscr_id']) which obviously does fail($_POST is empty when the review page is being constructed). If I ignore this for the time being and bypass this test the order status is updated:

uc_order_update_status($order->order_id, 'paypal_pending');

This causes the review page to flip back to the checkout page again. If we bypass the call to uc_order_update_status() then all seems well. I am fishing about in the code a bit. Simply by experimentation I have found that bypassing the uc_order_update_status() call means we get to the review page correctly.

If we don't include the patch from #781010: Patch to Ubercart might break recurring Paypal WPS then the uc_recurring_users table doesn't get an entry and consequently there are no recurring orders being configured. When the next Paypal IPN comes in the uc_recurring_users table is checked, found to be empty and to the IPN is logged, but no new order is created.

If we include the patch, _and_ modify the uc_recurring_hosted_paypal_wps_process function:

function uc_recurring_hosted_paypal_wps_process($order, &$fee) {
  // the recurring payment is setup at the time of product purchase
  if (!empty($_POST['subscr_id']) || 1==1) {
    $fee->data['subscr_id'] = $_POST['subscr_id'];
    //uc_order_update_status($order->order_id, 'paypal_pending');
   
    return TRUE;
  }
  return FALSE;
}

Obviously the '|| 1==1' is temporary hack, as is commenting out the update_status call.

This now means that all the uc_recurring tables are updated with the correct data so that when the next IPN call comes in from paypal it does find a recurring payment in the DB.

All that was left was to modify the function uc_recurring_get_fees() to detect a recurring_order rather than a recurring_product and it appears that Paypal_WPS is now working.
The modification to the uc_recurring_get_fees() was just having an alternative sql lookup if the order had a 'recurring_option' set:

 if(isset($order->data['recurring_option'])) $result = db_query("SELECT * FROM {uc_recurring_users} WHERE order_id = %d ",$order->order_id);
      else $result = db_query("SELECT * FROM {uc_recurring_users} WHERE order_product_id IN (". db_placeholders($products) .")", $products);

I will need to tidy up the code, and obviously I need some help making the correct changes to uc_recurring_hosted_paypal_wps_process() !

This may well not be the best way to implement the changes - I have to admit I have no idea _why_ the call to uc_order_update_status($order->order_id, 'paypal_pending') causes the review page to go back to the checkout page in a loop, all I know is that bypassing the call (or changing paypal_pending to 'in_checkout' stops the loop and allows the review page to come up.

I feel I am getting closer to a solution but would certainly be glad of some help / guidance.

geerlingguy’s picture

geerlingguy CreditAttribution: geerlingguy commented
Category: support » bug
Priority: Normal » Critical

Yikes - just upgraded to the latest Ubercart and uc_recurring today, and I ran into this (quite nasty) bug. I am subscribing with great interest, and depending on how the next day goes, I may be diving in headfirst to get it fixed.

Might anyone try creating a patch, so we can get some real movement on this issue?

I'm marking it as critical/bug report, since this module no longer works—at all—with PayPal WPS (at least).

geerlingguy’s picture

geerlingguy CreditAttribution: geerlingguy commented

Well, here's where I am so far:

I changed uc_paypal.module (in ubercart core module) line 1021 from

drupal_goto('cart/checkout/review');

to

drupal_goto('cart/checkout');

This at least sends the user back to the checkout page after an error is thrown.

On the checkout page, the following error is displayed:

Your order cannot be completed, because we could not process your recurring payment. Please review your payment details and contact us to complete your order if the problem persists.

(I have a feeling I might be getting a few contacts tonight :P).

That text is an error thrown by uc_recurring_order_order(), during the order's 'submit' $op; this function throws the error after evaluating uc_recurring_order_process_order() to be false... which it shouldn't be. (This code is in the uc_recurring_order module).

univate’s picture

univate CreditAttribution: univate commented

This is not really a critical issue it this one that needs fixing:
#781010: Patch to Ubercart might break recurring Paypal WPS

This issues is about making paypal work with the uc_recurring_order, at the moment it only works with recurring product fees.

that0n3guy’s picture

that0n3guy CreditAttribution: that0n3guy commented
Title: uc_recurring_order updates » Make Paypal WPS work with uc_recurring_order
Priority: Critical » Normal
Issue tags: +Paypal WPS, +paypal

sub..

ColinMctavish’s picture

ColinMctavish CreditAttribution: ColinMctavish commented

subscribing

katy5289’s picture

katy5289 CreditAttribution: katy5289 commented

What's happening with this issue? I would really like to see this issue fixed. I have a very angry client who would like to have his subscription component working again. Please fix uc_recurring so that it works with PayPal WPS. thank you.

univate’s picture

univate CreditAttribution: univate commented
Component: Code » Gateway Integration

Firstly the goal of this module is to provide recurring functionality to ubercart, not to implement specific gateway functionality.

Secondly this issue is about getting recurring order working with paypal wps - that is where you want an entire order to be duplicated on some interval. As far as I am aware Paypal WPS works with the uc_recurring_products module which is the module you would be wanting to use for site subscriptions.

But as noted on the project page:

Payment methods/gateway supported:

uc_recurring supports various gateways, the following are known supported gateways. Any question about specific gateways should be directed to the maintainers & users of those gateways.
...

@katy5289, I am neither a user or maintainer of paypal modules in ubercart so sorry but I can't help in any way with your issue. If you are not able to fully support your clients then maybe you should refer them to a drupal developer who can debug the specific problem on their site and propose a solution.

cmcintosh’s picture

cmcintosh CreditAttribution: cmcintosh commented

@univate sounds like a typical jackass response. Give me your customer because im not going to put a fully working module out there.

@kathy i have a similar need and am going to be working on this over the next couple of days. I can confirm on ubercart 6.x-2.x-dev and uc_recurring 6.x-2.0-alpha6 that this is a current issue and is related to how uc_recurring is formatting things and not ubercart. I think possible solution to this issue is write a new payment gateway specific to the uc_recurring module needs, referring to this page at paypal: http://bit.ly/oLLRMp about midway down it goes through all the required / optional things for a recurring subscription. At the very least i think adding a hook call into the processing portion for wps would help to allow us to inject the needed valus.

For my notes our links need to include the following at bare minimum:
business - paypal account email
a3 - regular subscription price
p3 - subscription duration, needs to be Integer value
t3 - D (days), W(weeks), M(Months), Y(years)
src(optional, but i think most would want this set to 1) 0 - Sub payments do not recur, 1 - Sub payments recur
no_note - Always need to set it to 1 for Subscriptions.

I will be checking the above first through the paypal module in ubercart and the uc_recurring modules to see exactly where the break down is happening and try to find out if/how to fix it with in the current framework.

The issue with many of the more complicated modules is that there is poor documentation on how they should be working and very little use of qa testing to ensure that the code works as it should.

univate’s picture

univate CreditAttribution: univate commented

@cmcintosh - sorry, but if you think that telling a module maintainer that you have an angry client or demanding the module be fixed will get sympathy from me or any module maintainer...

I have made it clear all along with this module that my goal was to implement recurring functionality for ubercart. There are thousands of payment gateways around the world so the idea that someone can support them all is insanity. The drupal way is allow the community to support their requirements through other modules. I have written support for a few different gateways including some work on Paypal. But I don't use paypal in any active projects

Personally I think ubercart should remove payment gateways from its project so developers can attempt to advance them. I know I have attempted in the past to get improvements paypal into ubercart without any success - #781720: Refactor the Paypal WPS IPN code - that patch would have meant I could remove a lot of duplicate code from this module.

In the interest of helping gateways that are stuck in ubercart get recurring support I created the sub-module "uc_recurring_hosted" as I didn't want to clutter up the core uc_recurring module with any gateways specific code (that includes the gateways I use), I always welcome patches to these external gateways and will usually commit patches directly without any test (since I can't) if they appear reasonable, e.g #376366: Implement subscriptions for Cybersource using Ubercart API

katy5289’s picture

katy5289 CreditAttribution: katy5289 commented

@univate Thank you for coding a new submodule called uc_recurring_hosted to help with solving this issue. I'm not a proficient coder myself and I appreciate all the work you've done so far. I meant to ask nicely that someone would please make it work again. I'm sorry if it didn't come across that way. I lost a client due to this problem.

Could you please update the home page of this project, uc_recurring_order, to indicate that it is broken for PayPal WPS but it is working for WPP? I would really appreciate it.

I would like to be notified when someone adds a comment to this thread. Where do I turn that on? I'm new to the issue queue.

I would like to create a Project Page for PayPal WPS for uc_recurring_order. Is it possible to do this? Could you please help with this?

I would like to help out with testing and also to advertise the need for help with fixing this module/gateway issue. I would like to start a Chipin fund. I could also help to organize a code sprint if needed.

@cmcintosh I agree that this issue is very complicated and needs further discussion and attention and definitely QA testing. Thank you for helping out with this issue. Can you update us with your progress?

katy5289’s picture

katy5289 CreditAttribution: katy5289 commented
Priority: Normal » Critical

I changed the priority to critical since uc_recurring_order does NOT work with PayPal WPS. This module is broken for PayPal WPS.

Use PayPal WPP or Authorize.net as your payment gateway for Ubercart's Recurring Payments and Subscriptions.

Use Membership Suite module by Leighton Whiting available at http://www.moneyscripts.net if you want to use PayPal WPS as your payment gateway. This module looks very good and does not use Ubercart. It's easy to use and was built for membership subscription websites.

cmcintosh’s picture

cmcintosh CreditAttribution: cmcintosh commented

I was able to get the module working with Paypal, however users have to go through paypal to actually cancel orders. I have not had a failed recurring cycle yet so am not sure how paypal will react yet.

pfournier’s picture

pfournier CreditAttribution: pfournier commented

@cmcintosh: what did you do to make it work?

cmcintosh’s picture

cmcintosh CreditAttribution: cmcintosh commented

As i said it will work with Paypal, however canceling of orders must be done via paypal. I believe this is something specific to paypal and not related to this specific module.

parrottvision’s picture

parrottvision CreditAttribution: parrottvision commented

@cmcintosh - So how did you get it to work?

lookitscook’s picture

lookitscook CreditAttribution: lookitscook commented
Version: 6.x-2.x-dev » 7.x-2.0-alpha3

Has any progress been made on this? Paypal WPS is listed as supported on the module homepage, but uc_recurring_order seems to be broken for me with Paypal WPS.

lookitscook’s picture

lookitscook CreditAttribution: lookitscook commented
Status: Active » Needs review
FileSize
uc_recurring_order-paypal_wps.patch1.92 KB

Made a quick patch (based on hevmill's comment) that seems to correctly send recurring order to Paypal WPS. Next up: figure out why the recurring fees (for recurring product fees AND recurring orders) aren't showing up in Drupal.

lookitscook’s picture

lookitscook CreditAttribution: lookitscook commented
FileSize
uc_recurring_order-paypal_wps.patch9.37 KB

This patch includes fix to also create the recurring fee on IPN callback (for recurring order fees). Code for previous patch has also been cleaned up. Use this patch instead.

lookitscook’s picture

lookitscook CreditAttribution: lookitscook commented
FileSize
uc_recurring_order-paypal_wps.patch7.2 KB

Sorry about all the posts. Should have waited till I was done (wish there was a way to remove/edit the attachment on previous posts). Here's a final patch that also includes a fix for canceling an recurring order fee via paypal WPS.