RBS Worldpay changes break ecommerce 5.x [#562048]

Comment	File	Size	Author
#48	worldpay.txt	13.51 KB	pwhite

Comment #1

daniel.hunt commented 28 August 2009 at 09:51

(selected paypal as the component because worldpay doesn't exist in the list)

Log in or register to post comments

Comment #2

gordon commented 30 August 2009 at 01:14

Project:	e-Commerce	» Worldpay Payment Integration
Version:	5.x-3.x-dev	»
Component:	paypal	» Code

Passing over to world pay issue queue

Log in or register to post comments

Comment #3

Dublin Drupaller commented 30 August 2009 at 08:54

Assigned:

Unassigned

» Dublin Drupaller

Hi Zoro,

Do you know what changes worldpay have made?

I don't have a live project to test with. If you drop me an email I can look into it.

dub

Log in or register to post comments

Comment #4

daniel.hunt commented 30 August 2009 at 10:12

Hey Dub,

It looks like they've changed their external IP address, or at least, the DNS that it resolves to.
I think I've sorted out a possible solution for it though, but I don't want to paste it here until I find out for certain that it works or not (so I can disclaimer it appropriately ;))

The issue itself stems from the auto-verification during the worldpay callback procedure, where the calling server's ip address is subjected to a reverse dns lookup.
It ensures that all callbacks come from "*.worldpay.com" before processing them, but it appears as though the changes to the ip and/or dns name have resulted in this domain being changed to "x.y.z.rbsworldpay.com". Due to the lack of a reliable sandbox environment with WorldPay (as opposed to PayPal's approach) the best way to test this particular change for me is to await another *real* Worldpay payment. I'll let you know when that happens (we accept PayPal & Worldpay payments, but can't force one or the other due to credit card acceptance issues)

Thanks for the response, and PM (I thought there used to be mail alerts for issue updates?)
Daniel

Log in or register to post comments

Comment #5

Dublin Drupaller commented 30 August 2009 at 10:38

thanks for the quick response Daniel.

I believe Worldpay do have a new sandbox style testing area...akin to paypals sandbox thing...(Alan Burke kindly offered the use of his test area a while back). I'll drop alan an email and check out the possible reverse dns issue you mentioned.

Will report back up here as soon as I've identified what the problem is.

Dub

Log in or register to post comments

Comment #6

daniel.hunt commented 30 August 2009 at 11:10

Well in that case, here's the change I made to the callback function:

  $ip = $_SERVER['REMOTE_ADDR'];
  $fullhost = gethostbyaddr($ip);
  $host = preg_replace("/^.*(\..*\..*)/", "*$1", $fullhost);
  if (in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) {
    ....
  }

Log in or register to post comments

Comment #7

alanburke commented 30 August 2009 at 13:19

Hi all,
Worldpay have been doing some rebranding alright.
They are now RBS worldpay.

I don't have any Ecommerce installation live either,
but Zoro's solution seems about right.

I'm on the way to Druplacon this week,
so I don't have easy access to those sandbox details.
If it is really needed, I guess I can chase up Worldpay from Paris,
and see what they can do.

Alan

Log in or register to post comments

Comment #8

daniel.hunt commented 30 August 2009 at 14:38

Thanks for getting back about this Alan.

We may as well hold off and see if my changes work or not. I'd expect at least 1 WorldPay payment in the next week, if not the next few days.

Log in or register to post comments

Comment #9

pwhite commented 1 September 2009 at 11:34

Zoro, Alan,

I've also been having issues with Worldpay since this date, I've made the same change on the worldpay payment module and will report back if this fixes the issue. My site has a number of orders per day so this should be a good indication.

Many thanks

Log in or register to post comments

Comment #10

Dublin Drupaller commented 1 September 2009 at 13:38

thanks for the update Pete. please report back up here as soon as you know.

dub

Log in or register to post comments

Comment #11

daniel.hunt commented 2 September 2009 at 06:25

I can confirm that this does *not* fix the problem.
*sigh*

We need the sandbox environment to get to the bottom of this I think :(

Log in or register to post comments

Comment #12

pwhite commented 2 September 2009 at 10:13

I can report the same:

line 313 also references worldpay.com

if ($host != '*.worldpay.com'){

I've changed this to rbsworldpay.com

Will let you know what happens.

Log in or register to post comments

Comment #13

daniel.hunt commented 2 September 2009 at 10:18

I've actually heavily modified that particular function for some custom hooks, so the change I made to the main if() should have worked (if it was indeed the correct change) for me.

Been doing some digging too: WorldPay changes

It's possible that some of the ip address resolve to the *.rbsworldpay.com DNS name, and some don't. I wonder if we just accepted a range of IPs instead of doing a reverse lookup, would this suit better.

Log in or register to post comments

Comment #14

pfaocle

He/Him

English

commented 2 September 2009 at 12:25

Experienced a similar problem. Altered lines ~284 and ~309 of contrib/worldpay.module to:

if (in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) {

and

if (!in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) {

respectively and changed the "WorldPay processing URL" setting in Worldpay config to https://secure.wp3.rbsworldpay.com/wcc/purchase - see technical info on Worldpay site.

Have had two successful orders processed since. Using D5.19 and ecommerce 5.x-3.4.

Log in or register to post comments

Comment #15

daniel.hunt commented 2 September 2009 at 10:39

Excellent leafish_paul, thank you for the (hopefully correct) point in the right direction.
Using the code changes I pasted above and changing the WorldPay processing URL seems to have worked for you - I've made the same config change on my own site and will respond here if everything starts behaving again.

Daniel

Log in or register to post comments

Comment #16

alanburke commented 2 September 2009 at 10:43

I have requested access to my sandbox.
I doubt I will get this in the meantime - as it is tied to my work email account which I don't have access to while on Holiday*.

Alan
* And that's the way I like it ...normally

Log in or register to post comments

Comment #17

daniel.hunt commented 2 September 2009 at 10:45

Hmm, rereading the changes page for worldpay and I don't see how just making that one conf change will make a massive difference.

But, if it's all still working for you i'm happy to sit here and hope it's all going to work now

*edit* Thanks Alan - but you're on holiday, don't go over doing it :)

Log in or register to post comments

Comment #18

pfaocle

He/Him

English

commented 2 September 2009 at 10:52

True, this setting will only "bypass the additional webpage containing a manual redirect button"... We have infrequent orders on our client's site (6 since 31 Aug), so we may have been lucky with the IP resolving correctly.

Edit:...although section B on the URL you've posted suggests a reverse IP lookup is fine:

"If your system uses the peer hostname ending in '.outbound.wp3.rbsworldpay.com' to validate whether incoming communications are being sent from RBS WorldPay then these changes will not affect you and you do not need to make any further changes."

Log in or register to post comments

Comment #19

daniel.hunt commented 2 September 2009 at 10:54

Yeah the reverse lookup, and the regex I'm suggesting, should allow for the *.rbsworldpay.com to work.
I'm not sure yet why it isn't

Log in or register to post comments

Comment #20

Dublin Drupaller commented 2 September 2009 at 10:55

just looking at the changes.....

the main one appears to be the following:

Redirecting Shoppers to RBS WorldPay Payment Pages

The measures we are introducing to prevent cross site scripting will no longer allow shoppers to be automatically redirected once they have submitted an order on your website to our payment pages. This change has been enforced as a result of the recent release of Internet Explorer 8 which, because of an incompatibility issue with the improvements we are making, resulted in an error message to shoppers warning about cross site scripting.

Unless you make some minor changes to your systems, shoppers wishing to purchase from your website will now be presented with an additional webpage containing a ‘clickable’ redirect button in order that they can be transferred to the RBS WorldPay payment pages. However, you can avoid this further step in your payment service altogether by replacing the submission URLs used by your system to send purchase token details to our payment service. If you wish to bypass the additional webpage containing a manual redirect button, then please update your website or shopping cart software with the submission URL:

Old URL: https://select.worldpay.com/wcc/purchase

New URL: https://secure.wp3.rbsworldpay.com/wcc/purchase

@leafish_paul #14 your changes look good.

dub

Log in or register to post comments

Comment #21

Dublin Drupaller commented 2 September 2009 at 10:57

if anyone has a live shop with some low value items on it...drop me an email. I don't mind using my credit card to test it while we're waiting for a sandbox.

@ Alan. enjoy your holiday and don't worry about it. I phoned worldpay earlier requesting a sandbox test environment.

Log in or register to post comments

Comment #22

daniel.hunt commented 2 September 2009 at 10:58

@Dub - the changes don't relate to the callback procedure though. Well, not in the way we're having difficulties.

If they do, and I can't see how, fair enough :)

Log in or register to post comments

Comment #23

Dublin Drupaller commented 2 September 2009 at 11:02

@ zoro: yep, you're right. I was just commenting at the changes listed on the worldpay site in general...it didn't mention anything else that would be affecting the callback.

however, leafish_pauls suggested line change I was referring to does effect the callback procedure...it looks for .rbsworldpay.com as well as worldpay.com.

<?php
if (!in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) {
?>

Log in or register to post comments

Comment #24

daniel.hunt commented 2 September 2009 at 11:04

@dub: Yeah I saw that - I presumed he was referring to my suggested code change at #6 though...

Log in or register to post comments

Comment #25

daniel.hunt commented 2 September 2009 at 11:12

I just noticed the "!" in his code.

Surely doing something like that to line 284 would mean that there's definitely a problem somewhere? ie: the match is specifically not saying that the server is a worldpay server?
In general I'm sure there's not a problem with that for a short time period like this, but it does open the door to future attacks.

Am I misreading this somehow?

Log in or register to post comments

Comment #26

pwhite commented 2 September 2009 at 11:14

I've made the changes in #14 and will let you know if it works - my site is http://www.cathtatedirect.com - it sells cards etc, most are less than a few £££.

Log in or register to post comments

Comment #27

Dublin Drupaller commented 2 September 2009 at 11:15

@zoro: sorry. didn't see you had posted that earlier. I also see that you mentioned that didn't work as well.
I think looking at an alternate callback procedure might be called for. I hope to have a sandbox within a day or two, so we can test.

Log in or register to post comments

Comment #28

pwhite commented 2 September 2009 at 11:16

zoro thats saying that if .rbsworldpay.com is not in the hostname then to set it as a payment failed - use it on line 313. On the first mention of rbsworldpay.com you don't need the !.

Log in or register to post comments

Comment #29

daniel.hunt commented 2 September 2009 at 11:19

This is all getting confusing ;)

Log in or register to post comments

Comment #30

Dublin Drupaller commented 2 September 2009 at 11:26

@zoro: do you think the CALLBACK PASSWORD with junior select might be an option?

Callback Password

The Callback Password to verify callback responses with. This value must match the value provided in the WorldPay Customer Management System -> Installation Account -> Configuration Options -> Callback Password field.

not sure how secure that is from spoofing, but, let me know if you've already looked at that.

Log in or register to post comments

Comment #31

Dublin Drupaller commented 2 September 2009 at 11:38

d'oh!

I forgot that I used the Md5 callback password with the newer version of worldpay: http://drupal.org/project/ec_worldpay (drupal 6.x and Drupal eCommerce 4.x)

If pwhite's changes don't work..when I get sandbox access I can backport the MD5 callback option to this version.

Log in or register to post comments

Comment #32

daniel.hunt commented 2 September 2009 at 11:39

I haven't looked into it, no.
It could well be a valid replacement for verification though. Did you get the sandbox account already?

*edit*
Excellent news. Would it be possible to work on a backport before the sandbox account is ready?

Log in or register to post comments

Comment #33

Dublin Drupaller commented 2 September 2009 at 11:46

pwhite @ #28.

Just tried a test order on your shop using my credit card. The payment went through but I got an error message on your site saying that your site couldn't validate the order with the worldpay server..which is a callback issue.

The call back link from worldpay was: https://secure.wp3.rbsworldpay.com

nice shop by the way. off topic: one minor thing though...there's a patch here for copying billing to shipping address during the checkout process. http://drupal.org/files/issues/address-bill-ship-same.patch

Log in or register to post comments

Comment #34

Dublin Drupaller commented 2 September 2009 at 11:53

@zoro #32

nope. haven't got the sandbox yet. will take a day or two according to worldpay support.

I wouldn't like to try backporting without fully testing it (there are dbchanges to be made as well).

An elastoplast solution would be for sites with physical orders to comment out the host check and keep an eye on your worldpay receipts until I get the sandbox and fully test the Md5 option.

The md5 option is a better way of checking than a host check (which probably can be spoofed) anyway.

Will try and get this done asap and post up here.

Log in or register to post comments

Comment #35

pfaocle

He/Him

English

commented 2 September 2009 at 12:25

Re: #28, yeah sorry all - my fault. The first change near line 284 should be without the not:

if (in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) {

Edited my previous comment. Thanks.

Log in or register to post comments

Comment #36

Dublin Drupaller commented 2 September 2009 at 12:53

just to clear up any confusion..

@ pwhite: I just tested your shop. Payment worked, but, your drupal shop displayed an error message indicating that the callback isn't working. Can you indicate which version you are using + what changes you have made?

@leafish_paul: Can you confirm categorically that the following change to the worldpay.module for drupal ecommerce 3.5 worked?

<?php
if (in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) {
?>

Apologies if I'm asking you guys to repeat yourselves, but, I'm getting a little confused.

dub

Log in or register to post comments

Comment #37

pwhite commented 2 September 2009 at 13:01

Dublin,

Many thanks for testing and for the address book patch, I will put that in once we have this sorted!

Now according to the error message the callback is failing at this point (I assume this was the error message you got, I've not changed the default):

(line 313)

  if (!in_array($host, array('*.worldpay.com', '*.rbsworldpay.com'))) { 
    global $conf; 
    $worldpay_failed_name = 'worldpay_server_fail';
    $worldpay_failed_default = t('<h1>Sorry!</h1><p>We were unable to validate your request with the WorldPay(tm) server. Please click through to our %link to try again or to contact us if you are having difficulties.</p>', array('%link'=>l('Products page', 'product')));
    $output .= (isset($conf[$worldpay_failed_name]) ? $conf[$worldpay_failed_name] : $worldpay_failed_default);
  }

I spoke to Worldpay yesterday and they confirmed the change from worldpay.com to rbsworldpay.com and that by checking the host is from rbsworldpay.com should have fixed the problem (clearly it hasn't).

Log in or register to post comments

Comment #38

pwhite commented 2 September 2009 at 13:09

I'm using Drupal 5x with Ecommerce 3.4

Log in or register to post comments

Comment #39

Dublin Drupaller commented 2 September 2009 at 13:18

@pwhite: thanks for that and you're welcome. Feel free to do a refund to my card if you get the chance!

As a quick test...can you change references to the hostnames to this:

if (in_array($host, array('*.worldpay.com', '*.rbsworldpay.com',  '*.outbound.wp3.rbsworldpay.com'))) {

Not sure if it will make a difference (I would have thought the * wildcard would bypass the need to specify outbound.wp3...but, that's what the tech docs recommend.

http://www.rbsworldpay.com/support/gg/index.php?page=news&sub=rebrand&c=IT

Log in or register to post comments

Comment #40

daniel.hunt commented 2 September 2009 at 13:20

The specification of the "*." is only because that's how we're creating the string in the regex - remember that. It's not the actual domain name.

Log in or register to post comments

Comment #41

pwhite commented 2 September 2009 at 13:24

I've made the change in both places. I will check with the person managing the store to see if your order has been sent out, if not they will refund!

Log in or register to post comments

Comment #42

daniel.hunt commented 2 September 2009 at 14:06

I've just discovered that the last few WorldPay payments I received failed (or rather, the callback failed) because I undid my "fix".
Sigh.

So, I've re-applied the fix and I've setup some basic logging for $_POST, $_SERVER and $host. Hopefully when we get another payment I'll have a bit more info to play with.

For anyone interested in testing it out on our site, you can try purchasing a licence from DeviceAtlas. I doubt I could sort out a refund, however, so don't do it just for testing unless you actually want one ;)

Log in or register to post comments

Comment #43

alanburke commented 2 September 2009 at 15:24

Great work Guys
BTW it's a busmans holiday -
I'm at Drupalcon!

Log in or register to post comments

Comment #44

Dublin Drupaller commented 2 September 2009 at 19:09

@alan. I am seriously jealous. just to make you feel even better...the weather in ireland is seriously bad at the moment. it's been lashing rain all blinking day in dublin. wintry cold.

Log in or register to post comments

Comment #45

pwhite commented 3 September 2009 at 09:45

I can confirm that the fix in #39 has fixed the problem. We've had three completed orders over night.

Many thanks

Log in or register to post comments

Comment #46

daniel.hunt commented 3 September 2009 at 09:50

OK this is hugely confusing now.
Can you paste your entire callback function here so we can see what exactly you have? There are so many conflicting suggestions about what could work that I don't know what your regex is :)

Log in or register to post comments

Comment #47

pwhite commented 3 September 2009 at 16:06

Ok line 285

  if (in_array($host, array('*.worldpay.com', '*.rbsworldpay.com',  '*.outbound.wp3.rbsworldpay.com'))) {

line 313

  if (!in_array($host, array('*.worldpay.com', '*.rbsworldpay.com',  '*.outbound.wp3.rbsworldpay.com'))) {

I've updated the the callback https://secure.wp3.rbsworldpay.com

I had a call this morning from the store saying that orders over night had been maked as completed rather than pending (suggesting the callback is working).

Log in or register to post comments

Comment #48

pwhite commented 3 September 2009 at 16:07

Status	File	Size
new	worldpay.txt	13.51 KB

Whole worldpay module attached here:

Log in or register to post comments

Comment #49

Dublin Drupaller commented 3 September 2009 at 16:30

Thanks Pwhite....

I'll draft up a patch for the module based on the changes. I still don't have the sandbox from worldpay and I'm away for the next few days so I'll update the module next week.

Log in or register to post comments

Comment #50

daniel.hunt commented 3 September 2009 at 16:59

Excellent, thanks pwhite

Log in or register to post comments

Comment #51

dakku commented 8 September 2009 at 16:41

subscribing

Log in or register to post comments

Comment #52

pfaocle

He/Him

English

commented 6 November 2009 at 11:49

Just a heads up on further changes to RBS Worldpay's systems: this time IP address changes. I'm unsure how this will affect us just yet, but I will be keeping my eye on this for the one client we have using this module!

More info: http://www.rbsworldpay.com/support/gg/index.php?page=news&sub=pci&c=UK

Log in or register to post comments

Comment #53

Dublin Drupaller commented 7 January 2010 at 09:19

Status:

Active

» Closed (fixed)

Log in or register to post comments

RBS Worldpay changes break ecommerce 5.x

Comments