Due to an underlying bug I discovered in phRets: https://github.com/troydavisson/PHRETS/issues/4, if the listings returned by your MLS (you must be using offset, BTW) are exactly divisible by MAX ROWS, phRets will return an error something like "Miscellaneous Search Error. [Offset] is beyond the number of returnable records.". This is because if your MLS returns 2000 results exactly (as mine did), and MAX ROWS is 500, phRets will mistakenly try to do five batches instead of four (with the fifth starting with an offset of 2001, beyond the available 2000 listings), and resulting in an error. phRets then returns no results to dRealty, and dRealty will then delete all of your listings in that listing class because expire_listings() gets an empty listing array passed to it, so all listings in your DB look like they no longer exist in the MLS and are therefore deleted as they would normally be.

So, a bit of an edge case, but fairly serious because it could result in your site suddenly losing a large number of listings, and you won't get them back until your MLS contains a number of listings no longer evenly divisible by your batch size. In my case my client adds some PDFs and other things to their listings, and losing the listings would mean they could lose a lot of uploaded data.

So, two things need to happen. #1, phRets needs to get fixed, and #2, dRealty needs to not delete all listings in expire_listings when it gets an error from phRets. This would probably require some more robust error handling than the current drush_log in fetch_listings_offset_supported_default, perhaps return false and then skip further processing in the case of an error?

Comments

camidoo’s picture

Assigned: Unassigned » camidoo
Priority: Normal » Critical

good catch, i'll look into this and see what i can come up with, going to mark this as critical

stockliasteroid’s picture

Here's my pull request on phRETS to fix the issue I mentioned: https://github.com/troydavisson/PHRETS/issues/4. You can also grab the fixed version of phRETS from my repo: https://github.com/bit-forge/PHRETS

camidoo’s picture

Status: Active » Needs review

Looks like troy accepted your pull request and committed it, does this take care of that issue?

camidoo’s picture

Status: Needs review » Needs work

just was thinking, i'm pretty sure all the documentation states to download phrets from troy's website and not github, should probably change the documentation to point to the github rep instead.

stockliasteroid’s picture

He actually hasn't committed it, the pull request is still open: https://github.com/troydavisson/PHRETS/pull/5

So for now, not sure what to do... Maybe a mention in the readme and a link to the pull request or something. It's very much an edge case, but could save someone some hair pulling if it was mentioned somewhere.

kevinquillen’s picture

This might be what I just saw.

http://drupal.org/node/1803422

kevinquillen’s picture

I'd advise grabbing that patched version of pHRETS and looking at the changes here. It might stop this.

http://drupal.org/node/1803422

kevinquillen’s picture

Status: Needs work » Needs review
shauntyndall’s picture

Status: Needs review » Closed (outdated)

This issue seems to be dependent upon PHRETS (https://github.com/troydavisson/PHRETS/issues/4) and is possibly related #1803422. I'm going to close this as there have been no additional updates in 4 years. If additional information becomes available based on a more up-to-date version of drealty please reopen.