Module currently produces invalid notify_url based on $payment_method['instance_id'], e.g. example.com/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps

Which results in 404 when returning from PayPal.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

rszrama’s picture

The problem isn't in the URL encoding or the "return" from PayPal; it's actually that the PayPal WPS form specifies a notify URL containing a payment method instance_id while the menu item in the PayPal module expects a payment method method_id. I'm looking into it to see if the instance_id is necessary, and if so we can just update that menu item accordingly.

rszrama’s picture

Status: Active » Fixed

Ok, I just changed the menu item to accept an instance_id instead of a method_id... meaning it'll load the payment method indicated in the URL along with its settings from the Rules action that enables it. This seems to be closer to the heart of what the IPN processing function is intending to do anyways. Go ahead and update and report back if you still have problems.

Note: I also just committed a bunch of fixes for this module to work with the latest dev version of the Commerce modules. You may need to rebuild your test site if you aren't already running dev.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

ahimsauzi’s picture

I am having hard time finding documentation on what url to use on Paypal side for IPN.
Do I define this via WPS rule or is there a predefined url?

rszrama’s picture

Nope, you shouldn't put anything in at PayPal for the IPN URL. This data is included automatically in the form used to redirect to PayPal.

ahimsauzi’s picture

Great, thanks. I take it that order status will only change upon completion of an order on a publicly accessible server.

ahimsauzi’s picture

Sorry to open this again but in PayPal the two options are Receive IPN messages (Enabled) or Do not receive IPN messages (Disabled). The enabled option requires a properly formatted URL.

Just to make sure, the redirect to PayPal form uses different method (API most likely) to communicate with PayPal and receive the proper notification upon order completion and trigger order status change from pending to complete.

gagoo’s picture

Subscribe

heeed’s picture

Having same problem. This has become a showstopper.

Perform order
Go through paypal (sandbox) correctly
return to site
No payment details displayed in order

Reports of 404 page not found when paypal returns information back to the following address:

/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps

Server is publicly contactable and all modules are upto date

foobarblat’s picture

still broken - getting complaints from paypal

notify_url value sent from drupal commerce is preposterous bogus garbageo

rszrama’s picture

Can you post the notify_url that's getting posted here? I don't see any reason in the code for it to be anything other than a URL to your site.

cindyr’s picture

Same problem. The email from PayPal is:

Please check your server that handles PayPal Instant Payment Notifications (IPN). IPNs sent to the following URL(s) are failing:

http://www.mysite.com/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps
cindyr’s picture

Status: Closed (fixed) » Active

Additional info, watchdog errors:

Type	page not found
Date	Monday, August 6, 2012 - 15:41
User	Anonymous (not verified)
Location	http://www.mysite.com/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps
Referrer	
Message	commerce_paypal/ipn/paypal_wps|commerce_payment_paypal_wps
Severity	warning

and (note the time difference - I only placed one order and was redirected by PayPal back to my site within a few seconds.)

Type	commerce_paypal_wps
Date	Sunday, August 5, 2012 - 16:55
User	admin
Location	http://www.mysite.com/checkout/1/payment/return/9PUQtXEcH4eoYdZU4KasspsFEWWfOEDYVPZT2EobtZM?tx=7A043351DY559160G&st=Completed&amt=1.03&cc=USD&cm=&item_number=
Referrer	
Message	Customer returned from PayPal with the following POST data:

Array
(
)

Severity	notice
andyg5000’s picture

Status: Active » Postponed (maintainer needs more info)

Are you able to load [your site]/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps in a browser or do you get a 404?

Have you tried clearing the menu cache?

cindyr’s picture

All caches cleared, that URL causes a 404.

Stan Turyn’s picture

Subject: PayPal Instant Payment Notification Warning

Dear ...,

Please check your server that handles PayPal Instant Payment Notifications
(IPN). Instant Payment Notifications sent to the following URL(s) are
failing:

http://sitename.com/paypal/ipn

If you do not recognize this URL, you may be using a service provider that
is using IPN on your behalf.

Just received the above from Paypal, 2 days after the site went live. What have I not configured correctly?

rszrama’s picture

Literally nothing, as it should be completely automatic. I'm beginning to wonder if something didn't change in Drupal core that's making this URL inaccessible. I've never had a problem with IPN delivery on my site, but I may be a version behind.

Stan Turyn’s picture

Thanks, Ryan

The site is running Drupal 7.15 and dev version of Commerce

andyg5000’s picture

I just ran a fresh install of the following:

Drupal 7.15
Commerce 7.x-1.3
PayPal 7.x-1.x-dev

After enabling all of the modules including PayPal WPS, I only received the 404 before clearing caches. After caches were cleared, I see activity in watchdog that suggests IPNs are being processed whenever commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps is accessed.

The only strange thing that I can see is the url encoding of "|" to %7C, but I don't see how this would break the menu router or cause any 404's.

I'm happy to help troubleshoot this with anyone in the #drupal-commerce IRC channel. Just shoot me a ping.

cindyr’s picture

Did you use the Commerce Kickstart installation profile? I did not, and I'm wondering if there's something different between the Kickstart installation and Commerce modules. I'm using:

Drupal 7.9
Commerce 7.x-1.3
PayPal 7.x-1.x-dev

I'll try upgrading Drupal to 7.15 and see if that fixes the problem.

andyg5000’s picture

I did not use commerce kickstart. I installed a fresh copy of D7.15 and added commerce and commerce_paypal.

Stan Turyn’s picture

Status: Postponed (maintainer needs more info) » Active

Ok, I'm receiving threatening emails from Paypal after every order with:

Instant Payment Notifications sent to the following URL(s) are
failing: http://sitename.com/paypal/ipn

Why do Paypal say the URL is "http://sitename.com/paypal/ipn" ? Isn't it supposed to be "http://sitename.com/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_pa..."? Or do they just shorten it in their emails?

cindyr’s picture

Stan, what version of Drupal are you using? I just updated from D7.9 to D7.15, and it now works. (Yay!) Something in the Drupal upgrade (or upgrade process) fixed the same Paypal issue for me.

Stan Turyn’s picture

Cindy, it's 7.15 for me too. The thing is, I'm happily receiving payments from Paypal and see no errors in logs/have is no configuration to troubleshoot but they are unhappy with that URL and may disable IPN on my account. I can access "/commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps" no problem but Paypal report "/paypal/ipn" in their emails, which 404s.

rszrama’s picture

I guess the question is why are they expecting /paypal/ipn to exist? Did you add this in your PayPal profile somewhere? It isn't coming from this module.

Stan Turyn’s picture

Status: Active » Postponed (maintainer needs more info)

Thanks a lot, Ryan, you put me on the right path - spoke to Paypal technical support and they re-set IPN preferences for my account. I'm changing the status of this issue back to the way it was before - "postponed".

I do have a question about #5 though - "Nope, you shouldn't put anything in at PayPal for the IPN URL. This data is included automatically in the form used to redirect to PayPal." - does it mean that IPN should be set to "disabled" in Paypal account profile on their site?

rszrama’s picture

Theoretically, it shouldn't matter, but apparently somewhere along the line PayPal started caring. What I don't know is if disabling would also disable the IPNs sent to our custom notify_urls.

andyg5000’s picture

From my experience, changing IPN settings to disabled in on PayPal's end (at least in sandbox) does NOT prevent IPN functionality from working properly.

rszrama’s picture

Status: Postponed (maintainer needs more info) » Closed (cannot reproduce)

I also just confirmed this for my live site using this module - my PayPal account has IPN turned off, but PayPal WPS related IPNs are coming through just fine and are not generating 404 errors at the URL in question. Even in local testing, I am not able to force 404 errors. Without any way to reproduce this error, there's not much we can do. If someone gets a test environment going that consistently fails and is willing to share that for us to debug, we can reopen this.

Until then, I'm just going to mark it closed and reiterate:

You do not have to configure the IPN settings at PayPal at all. In fact, unless you know you need to receive general IPNs from PayPal beyond the IPNs generated specifically by PayPal WPS (such as for some other module using PayPal for recurring transactions), you should leave IPN off in your profile settings.

rszrama’s picture

PayPal offers some troubleshooting tips here:

https://www.x.com/content/ipn-troubleshooting-steps

It could be that your web server firewall settings are blocking remote POSTs, which would result in PayPal attempting to send IPNs that show up in the Drupal logs as being empty. I can't think of any reason for it to show a 404, though, unless some other module on the site is changing the IPN URL... perhaps in a multilingual setup that could be happening if a module is altering path elements or something.

jazzdrive3’s picture

FYI,

On my last two sites, I have had to turn IPNs on in the Paypal account settings, with the http://example.com/commerce_paypal/ipn as the url. Neither of the them worked at all until I changed this. We simply were not getting any IPN notifications. Orders had no payment transactions.

rszrama’s picture

Hmm, honestly that shouldn't matter at all, because PayPal uses whatever IPN URL gets sent with the transaction. If one gets sent with it, then it ignores whatever's in the settings form. : ?

Unless, are you using PayPal for subscription payments?

jazzdrive3’s picture

No, just regular WPS redirect.

It didn't work unless I manually turned it on in Paypal itself. That was the only change.

Stan Turyn’s picture

It's working fine for me with IPNs turned off in Paypal account settings - still get IPNs through Commerce.

agileadam’s picture

Here's what happened to me: We did a site redesign for a client that was previously using a different shopping cart system. PayPal WPS transactions seemed to be going through OK and I was getting back a single POST message in my Drupal watchdog logs. I thought everything was working well until I looked at the Payment tab for an order that was placed using the PayPal WPS payment method. It didn't have any payment record.

Then I decided to switch to my PayPal sandbox URL and checked out with a PayPal sandbox account. It went through, I got 3 messages in the watchdog log, and the order's payment tab had the PayPal transaction information. Success. So, I figured it had to be a setting in my client's PayPal account.

After more investigating/research I figured I'd have a look at the IPN History. I noticed each entry said "Disabled." I figured, after reading the commerce_paypal docs, that this was OK. I also noticed the notify_url was correct. So, why wasn't it working? Well, I had a look at the IPN settings and noticed that the URL was still pointing to their old site. I changed this to be the same address as the notify_url that I was seeing in the IPN history. I also set "Message delivery" to be "Enabled" by turning on IPN. I put through a test transaction and was happy to find 3 messages in the watchdog logs. I then had a look at the payment tab and found the PayPal transaction information there. Yay! Lastly, I checked the commerce_payment_transaction table in the DB and sure enough, there was a row there for the new paypal_wps transaction.

Now, I don't know which setting made it work, or if both changes made it work, but it works. The IPN History status showing "Sent" vs. "Disabled" makes me think it was important for me to turn on IPN. I think it would've worked without fixing the IPN url given that all of the entries seemed to be using the right notification URL. Also, when I view the IPN details of an entry BEFORE the changes I notice no response code and a delivery status of "Disabled." When I view one after the changes, I see a response code of 200 and a delivery status of "Sent."

mikecar’s picture

FileSize
1.03 KB

in case anybody is interested: I run across this issue, and debugged it. the *current* problem lies with paypal, not the module. it seems that sometimes paypal will call back with the wrong url encoding of the ipn url. therefore, the pipe symbol "|", required by commerce_payment, gets mangled to %7C, and the Drupal menu router gags (actually the commerce_payment loaded does). What makes this extremely hard to diagnose is that this behavior is not consistent. sometimes paypal calls with the correct url, sometimes not.
who knows why.

in any case, the attached patch fixes this issue, while not doing something bad (I hope) when the bug does not manifest. in essense, it hijacks the commerce_payment loader, and fixes the url segment. it is generated against the latest version of commerce_paypal, commerce_paypal-7.x-2.x-dev (as of 21-05-2013)

I hope this helps somebody....

Dave Cohen’s picture

I like the look of mikecar's patch, and I've applied it. Some hours later, I got another email from paypal. So, either they are sending an email because requests failed before I applied the patch. Or, the patch isn't fixing the problem. I can't tell which.

The email contains:

Please check your server that handles PayPal Instant Payment Notifications (IPN). IPNs sent to the following URL(s) are failing:

https://www.greatsunflower.org/store/commerce_paypal/ipn/paypal_ec%7Crules_paypal
https://www.greatsunflower.org/store/commerce_paypal/ipn/paypal_wpp%7Ccommerce_payment_paypal_wpp

If you do not recognize this URL, you may be using a service provider that is using IPN on your behalf. Please contact your service provider with the above information. If this problem continues, IPNs may be disabled for your account.

Thank you for your prompt attention to this issue.


Thanks,

PayPal
smurfxx’s picture

Issue summary: View changes

I have the same issue, does the patch resolve the error?
I have the latest paypal module installed, do I have to apply patch?

Why this patch, if working, is not in the newest release of this module?

EDIT: I applied this patch and seems that everything is ok, I forgot also to put the returning url in my paypal account.

dwkitchen’s picture

Status: Closed (cannot reproduce) » Needs review

I am experiencing exactly the same problem and getting the above emails from PayPal.

I have applied the patch and waiting to see if the there are no more issues.

freelylw’s picture

FileSize
30.73 KB

after I finished the payment process in paypal, it stop at the " thanks for your order" page. user has to click on the email address to return to the original website. otherwise it won't automatically return to the original site.

Please support this. I have no way to solve this problem.

Please see attached image of screen-cut for the page where I stop at. Thank you.

stevieb’s picture

I just got the feared mail from paypal

will try the patch

mglaman’s picture

Status: Needs review » Needs work
+++ <html><b>Current File</b></html>
@@ -30,6 +30,16 @@
+    error_log('paypal bug ' . $instance_id);

This should be watchdog()

nvahalik’s picture

Patch updated.

nvahalik’s picture

Status: Needs work » Needs review
rszrama’s picture

Hmm, do you think we should just be doing URL decoding in core instead of this fix in PayPal? Or maybe just add a generic function to the Payment module for loading a payment method instance from a URL?

mglaman’s picture

Core should try to do some URL decoding. It might simplify some edge cases, especially iframe implementations.

nvahalik’s picture

But isn't this essentially a malformed URL coming from PayPal? Do we want to foist that on all of the payment modules?

rszrama’s picture

I don't think it's forcing everyone to use an autoloader that does URL decoding (unless that's a non-intrusive change). At the least we can provide a wildcard autloader that does the decoding similar to this patch so that contributed modules may make use of it as they desire.

ndiSIGN’s picture

Hi,

Looks to me this thread: https://www.drupal.org/node/2263585#comment-11293927 is about the same issue, and with a solution (#9).

But in my case I needed a strange route to come to that solution.

2suns’s picture

Since the installation+configuration of "cURL HTTP Request module" (as #49 points out) fixed this for me instantly
I suspect that the IPN url encoding is being garbled during sending of the payment request to paypal. Paypal then tries to contact that url.

siva_drupal’s picture

Patch file not working for 7.x - 2.3 version. If you guys have some other idea, please update.

LIQUID VISUAL’s picture

I have this problem with stable releases of Commerce and Commerce PayPal. Also using entity translation.

We have started receiving messages from paypal, wonder if something in core is restricting access to URLs?

For first step, pasting in

http://example.com/commerce_paypal/ipn/paypal_wpsIcommerce_payment_paypa...

as IPN url in papal business setup and leaving Paypal IPN notifications turned on. Will try 2) turning IPN off in paypal (and see if necessary reporting of payments continues), and 3) using curl.

Those are the things to try?

Thanks! Working fine, and reporting payments, but we're getting same error messages from paypal.

geraldito’s picture

Had the same issue and could fix it by using cURL HTTP Request (as mentioned by #49). I followed these steps to get it working: https://www.drupal.org/node/2727793#comment-11434455

mglaman’s picture

Status: Needs review » Postponed (maintainer needs more info)

Can subscribers please try the dev release? That has changed to using cURL. Which based on the last comments.. that seems to have resolved most issues.

brylie’s picture

Is there any update on this issue? We are receiving notifications from PayPal to the effect of:

IPNs sent to the following URL(s) are failing:

https://.../commerce_paypal/ipn/paypal_wps%7Ccommerce_payment_paypal_wps

If you do not recognize this URL, you may be using a service provider that is using IPN on your behalf. Please contact your service provider with the above information. If this problem continues, IPNs may be disabled for your account.

brylie’s picture

We are still experiencing this problem. I am hesitant to install the cURL module, as linked above, as it has not had a release since 4 December 2015. In general, we are trying not to install too many modules, as they prevent a smooth upgrade to Drupal 8.

If the issue boils down to escaping the pipe "|" character, why not just replace that character in the URL with something else like underscore?

rszrama’s picture

You wouldn't have to install the cURL module. cURL is usually available via a PHP module that's installed by default in most environments.