Renewed orders have no line items (e.g. taxes, shipping)

Hey bzbz,

I am experiencing the same issue, with tax information not being carried forward into the 'Feature' that is the recurring fee.

What file are you patching with the above?

-- Marcel

thanks for bringing this up again! I have the same problem on my site. (see http://drupal.org/node/449318#comment-1855086)

subscribe

I would like to fix this in uc_recurring, when cloning the order all we are after is all the user information so that invoices will have all this information.

Do you think we just need to unset all line items so they get generated again?

The one thing I would prefer to avoid is having to manually alter the ubercart database tables. Currently the orders in uc_recurring are all created using ubercart's existing API's/functions which prevent duplicate code and we benefit from all of ubercarts internal systems (e.g. hooks/conditional actions) that get triggered on certain events.

Line items are all added by various modules, based on whats in the order, so are obviously getting added somewhere in the order process in ubercart and probably should be getting regenerated on order creation as the original order could consist of both recurring and non-recurring items while the recurring order is just the recurring items so the tax or shipping would be different. Based on what you have found I would be interested if anyone can confirm that adding the following works:

unset($new_order->line_items);  

The line items that I think would be the most important to test for are:
* tax
* shipping
* discount modules (uc_fee, uc_discount)
* .... any others?

I have change the renewal order to removed all line_items and they should get regenerated now. I tested it on a simple tax of 10% across all orders and this was correctly applied to the original order and the renewal.

This is still an issue with my 6.x-2.x-dev release (latest from 1/3 2010) :(

Taxes is shown - but shipping (and a handling fee I have) is not duplicated.

The original invoice is 311 DKK - and the recurring one (I set it to recur daily for test) is only 219 DKK.

The original invoice lines:
1× Linux starter pack in dvd case with guide lp-starterpack-subscription 0,00 Kr. 219,00 Kr. 219,00 Kr.
* Subscription: Yes - every 6 months
1× Qimo for kids 1.0 1CD-qimod 0,00 Kr. 20,00 Kr. 20,00 Kr.
* Architecture: x86 32-bit

Pakke (Forsikret og med Trace nr): 62,00 Kr.
Handling: 10,00 Kr.
Subtotal excluding Moms: 261,20 Kr.
Moms: 47,80 Kr.
Total: 311,00 Kr.

The recurring invoice lines:
1× Renewal of product Linux starter pack in dvd case with guide lp-starterpack-subscription 0,00 Kr. 219,00 Kr. 219,00 Kr.
Subtotal excluding Moms: 175,20 Kr.
Moms: 43,80 Kr.
Total: 219,00 Kr.

This is a test system (for now) - so I can debug all I want.

If you could point me as to where I should try to dpm($something) or whatever - I'd be happy to try and help to identify/fix the cause this issue.

Probably start by comparing the $order objects, the product may not be getting marked as shippable or something?

when looking at the original order - the products (also the recurring one) has shippable = 1.
in the invoice generated by uc_recurring, the same product only has: recurring_fee = true

uc_recurring started out as a way to add a recurring fees/payment when placing an order for a specific product. So its never really been about placing another order for that same product which is probably what you are doing when dealing with shippable products.

I don't know if we can just start marking recurring fees as shippable just because the product they are tied to is shippable - it might need another option somewhere.

I can't think of a use case, where an item that is shippable - shouldn't have "shipping" fees paid - when recurring.
subscriptions and the likes, would not be shippable anyways, and would thus work as expected.

It seems like it's the logical way it should be - ie. recurring fees should follow the shippable "marking" of the product.

Can you think of a use case, where this isn't so?

I am running the dev version afterall - but one could add an "calculate fees+shipping if the products are shippable" checkbox to the recurring settings perhaps - just to be sure?

also - I have a handling fee - and if people have fees that applied the first time the order was given - they should apply when the order "recurs", don't you think? perhaps shipping and fees should be split up as 2 choices - just like vat (apply vat to: checklist with fees, shipping etc. on the list).

The orders are not recurring - when you add a recurring fee to a product it means some ongoing charge will occur, it doesn't always mean the products is being repurchased again - e.g using uc_recurring to setup a payment plan.

I have already attempted to start creating a module that allows you to renew an entire order: #628066: Create uc_recurring_order module

I think the option to set the price the same as checkout, could be broadened to mean add the product to the recurring order, because at the moment when the renewal happens we just create a new order, add a product with a price and title (based on the recurring fee information).

So the thought is instead if you checked that box, it would create the renewal order and add the actual product that the recurring fee is attached to, to this new order.

I think doing that would be less likely to break existing uses for uc_recurring for current users.

That may the right way to do it.

Currently I have the recurring set on an attribute selection - so the same product can be purchased with, or without recurring fees.

If the original product is added - wouldn't it have to be added without the subscription fee - otherwise the customer would have 2 subscriptions (and if so the new order would say: subscription: no - as an attribute - which would be misleading)?

- and the customer should also be allowed to purchase 2 subscriptions on seperate occations, and have them both run (if they need more than one of the same product or whatever - ie. 2 magazines monthly, instead of just 1 etc.).

It sounds as it it would be best to add the product "copy" as you do now - but just add shipping and fees, like it is done with a real order with the shipping selection being the same as it was last (if any of the returned shipping quotes match exactly) - otherwise just choosing the cheapest.

Adding a product to an order doesn't mean you will end up with multiple subscriptions as recurring fees are only setup on checkout.

We are not adding a product "copy" at the moment, we are essentially creating a new product that doesn't exist anywhere in the system for the purpose of the charging the recurring fee, we are using some of the information from the original product though.

The point is that if we did add the actual product object to the order it would contain the information about it being shippable.

But I am just trying to work out the implications of making that change - ie: it will probably break sites that are trying to us it as a payment plan or subscription that includes an initial shipment with an ongoing subscription.

It would seem odd to have subscription on items that are only shippable at first - but not later (can't think of any - that's why I think it odd :)

But if it is simply wrapped in a config option "use real products, and run 'full checkout' - incl. shipping, fees etc." - it can't break any - as it will be default disabled.

klavs,

If you are selling a paper magazine or newsletter then you'd want to charge the shipping (and certainly taxes) every month.

Thank you.
Alexis

I agree with all the above points - the point I want to make is that historically this module has been about adding a recurring "fee/amount" that gets charged on the schedule thats been setup.

I think if we are instead treating the renewal order, as an order for the same product, then maybe we should have a different product feature, instead of adding a "recurring fee", we add a "recurring product order", which doesn't allow you to set a recurring price it just allows you to set the intervals.

This feature is not high on my priorities, but if someone wanted to come up with a solution I would be willing to commit it here.

I am trying to set up a site with monthly and yearly role subscribtions. Seperate Taxes are needed for quaterly reports and customer invoices. I tried with newest dev versions of ubercart and uc_recurring, but taxes were not added to the recurring amount on paypal (sandbox).
My question now is: do you plan to include this in the recurring product code at some point, or should I rather switch over to the "Recurring Order" Code, which seems to work now? What would you recommend? Which direction will you take?
Thanks for advice!

Hard to say without knowing the specifics of your store, but IMHO you're better off sticking with the regular uc_recurring, because say someone mixes say a shippable product with a subscription, that shippable product will be part of the recurring order.

ju.ri,

You should be able to setup your recurring fee inclusive of the taxes. And create multiple products for the different ways your customers can purchase your services. That's what I would do.

(This is opposed to attempting to use the product price for the recurring fee.)

Your bookkeeping/accounting should be what extracts the taxes...

Thank you.
Alexis

thanks for the advice!
Alexis, I could extract the taxes later for my accounting, but the bigger issue is that my users should be able to download their invoices for each recurring purchase at /user/me/orders, and these should show the tax separately (so it is deductible for them).
Is this possible with recurring payments?
thanks again

I'm a little surprised the taxes aren't carrying over - can you confirm your version of uc_recurring is unsetting the line items as in the OP or comment #7? Are you using the uc core Taxes module?

Hosted gateways like Paypal WPS will need more hacks to make line items work with the uc_recurring API

uc_recurring is designed to generate a new order on renewal and then charging that, for gateways where everything is setup in the beginning (e.g. Paypal WPS) the paypal code will need further hacks to calculate all these details at the beginning instead of when the order gets renewed.

I think you will find that uc_recurring is probably adding the taxes to the order, but what it isn't doing is charging the tax part, because it can't change the amount charged on renewal to what was originally setup in the recurring fee.

@ju.ri, you can create a new feature request here to add this functionality into the paypal specific recurring code (sorry, I don't have any time to spend on this feature myself)

My advice though is if you need more advanced features, use a real gateway that supports more flexible payment options.

Thanks again for advice. I'm testing a setup with WPP Gateway and Credit Card Payment now. Orders and invoices are created correctly with taxes.
[edit:] WPP is not possible in Europe... too bad.

we are working around this same issue of recurring payments not including Tax.

uc_recurring does support adding tax (and other line items) on renewal orders, it works exactly as if the order is created via a normal checkout.

As for whether individual gateways will support this feature will depend on the gateway, for example Paypal WPS is manged outside uc_recurring so when uc_recurring creates a new order and adds the specific line items for that order it can't change the amount paypal charges as that is already set - so you will end up with an order on your site that includes tax, but the amount charged doesn't include tax.

Personally if you want your recurring charges to incorporate these types of features and other charges then my advice is use a payment gateway that just stores credit card details for you and allows your ecommerce system to invoke the correct charge on that card as each renewal.

Personally I think the issues raised here have been resolved (which my fix applied in #9), but this issues has brought up two other issues:
* How do we we mark recurring product as shippable (solution probably is what I said in #16), see new issue for this: #857392: Redefine "Set recurring fee same as product price" to mean "Repurchase product"
* Hosted gateway will need other hacks/modification to support similar features (please open your own issues for gateways that you want to add specific features)

As I understand now is uc_recurring not usable in Europe, because of the missing taxes in the renewals and the wrong handling by paypal. So this is my case big shit. I fought now for days to find a way to setup a membership with übercart and the different modules and problems and now in the last step paypal sucks. Great! We also want to use Paypal WPS, because the other WPP is not available in Europe as previously mentioned.

A simple question now. What should the shops in Europe do if they want to sell subscriptions or roles with a renewal product and the local taxes (German MwSt in my case)?

Is it nessessary to build an own payment gateway with a local payment partner or is PayPal interested to fix this issue? Or is there another solution ?

@broncomania - The issue you have is really with the limitation of Paypal not uc_recurring. Uc recurring calculates taxes at the time of purchase which for a recurring payment will be after the initial checkout. This is added to orders when a renewal is due to occur.

I have made suggestion in the issue queue in the past that someone could certainly spend the time implementing hacks to add taxes to the order in the paypal code and make it work with taxes. Although such a solution will be making assumptions that taxes never change. Since paypal has the limitation that once the recurring payment has setup you cannot change the amount this solution will never be as robust as using a payment gateway that allows you to change the actual amount of the order at renew based on the settings at the time of renewal.

This problem still persists on the D7 branch (7.x-2.0-alpha2). Basically I need the shipping and tax for 2 reasons:

1. Even though nothing is being shipped, tax still applies in the state this is being applied.
2. When sending a blank shipping address to authorize.net, it sets it as a 'suspicious' transaction and bloats up that log file to the point it's hard to wade through.

I think the patch would be straightforward: use the conditional action/rule for the tax and simply use the shipping information as before (because we collect a shipping address in the beginning because there is a physical product shipped). If there are any gotcha's, please let me know.

TAX FIX - uc_recurring.module

uc_taxes_filter_rates will not work to apply tax line items for any order passed 'in_checkout'. And the uc_recurring_create_renewal_order specifically unsets all line items from scratch. Therefore, there are two options here. We can set the initial order status to 'in_checkout'

$new_order = $order;
$new_order->order_id = $new_id;
$new_order->created = time();
$new_order->order_status = 'in_checkout';

Or we could remove the unset($new_order->line_items); line and loop through, unsetting all but the tax line.

I prefer the first solution because it's a lot cleaner and the $order->status in uc_recurring_create_renewal_order is set to pending before the function exits, so it doesn't seem like it would necessarily harm anything.

SHIPPING FIX - uc_authorizenet.module
'customerProfileId' => $profile['customerProfileId'],
'customerPaymentProfileId' => $profile['customerPaymentProfileId'],
'customerShippingAddressId' => $profile['customerAddressId'],

Basically the shipping field was not be added in. And note the mismatch customerAddressId verus customerShippingAddressId. But if you check out the documentation at http://www.authorize.net/support/CIM_XML_guide.pdf (page 37), you'll see that this is required.

Here's a patch for the tax fix suggested in #36

The shipping issue fix in #37 is in ubercart core so I spun a new ticket here #1578482: customerShippingAddressId needs to be added to uc_authorizenet CIM payments.

Marking this issue "needs review" for the patch in #38.

Looking to @univate or @longwave to comment on this one line change

-  $new_order->order_status = 'processing';
+  $new_order->order_status = 'in_checkout';

Quick summary: by starting the duplicated order for a recurring fee as 'in_checkout', tax line items can get added whereas they cannot when an order status is beyond 'in_checkout'. And finally, the order status in this function (uc_recurring_create_renewal_order) is set to 'pending' anyway, so the scope of this change is very limited and shouldn't affect anything else.

Patch is in #38.

I'm bumping up against this as well. Here are my observations/questions:
1. Why does uc_recurring clear the line items for the new order?
2. Do we want taxes to be calculated on the new order using current rules/taxes, or using the tax information at the time the original order was placed? I would think the former, so setting the state to 'in_checkout' makes sense.
3. However, since the renewal order isn't really "in checkout" - I'd propose that uc_recurring create a new status ('renewing'?) with the "in checkout" state.
4. We also have to be sure to then update the status to the first 'post_checkout' state once the payment was submitted for processing.

1. Why does uc_recurring clear the line items for the new order?

What if taxes change? What if there is a shipping component and those change? What if (my case), there was an error in taxes NOT being calculated before due to an ajax issue in the D7 checkout form (see #1373236: Allow multiple modules to react on checkout Ajax events and there was a previous ticket where taxes were not being treated as line items in the initial version of D7 ubercart), and now we need to right that wrong.

2. Do we want taxes to be calculated on the new order using current rules/taxes, or using the tax information at the time the original order was placed? I would think the former, so setting the state to 'in_checkout' makes sense.

I would say recalculate. It takes very little time and ensures that they are correct going forward.

3. However, since the renewal order isn't really "in checkout" - I'd propose that uc_recurring create a new status ('renewing'?) with the "in checkout" state.

To me this is semantics, but ok :) The reason it can't be set to pending or beyond is that ubercart essentially fixes the tax calculation for that order at that point prevents that line item from being added back after all line items were unset just a few lines earlier. If renewal is wedged between in_checkout and pending, then I'm cool with that as long as the tax line item isn't looking for in_checkout versus NOT pending.

4. We also have to be sure to then update the status to the first 'post_checkout' state once the payment was submitted for processing

This already happens uc_recurring_create_renewal_order() with the call to uc_order_update_status($new_id, 'pending'); This returns the renewal order back to the function that called it in order to submit the payment information, which then would trigger it to payment_received.

The small tweak in #38 handles all of these use cases beautifully and has been operating on a production site for almost 20 days now. I would suggest this versus adding more code unless there are cases where my tweak causes some unintentional bugs somewhere else!

bump

I know this is still under discussion, but I still think the in_checkout route is the way to go. I reapplied the patch in #38 after the latest module upgrade.

I can circle back to this later if people want to pick this back up.

Has anyone used the patch in comment #38 in alpha3?

Works for me with latest 7.x-dev.

#38 not worked for me. In my case, recurring profile should be created in paypal using product price + shipping cost both. And initially in paypal profile, it creates recurring profile with product price only and removes other line items like shipping charges etc.
So, for workaround, i modified uc_recurring/modules/uc_recurring_hosted/uc_recurring_hosted.module and edited 1 line in uc_recurring_hosted_form_uc_paypal_wps_form_alter(&$form, $form_state) function: -

- $data['a3'] = sprintf("%0.2f", $recurring_fee['recurring product']->fee_amount == 0 ? $recurring_fee['product']->price : $recurring_fee['recurring product']->fee_amount); // Original script 
+ $data['a3'] = sprintf("%0.2f", $order->order_total); // Modified script for applying order total as recurring amount.

After this changes, i can see recurring profile with total amount in paypal profile. Hope this helps someone else.