admin/amazon_import/manage doesn't do anything

I've created a new default download. It has a number of bug fixes.

There's an update to run, lengthening the title field.

Now. You've had serious issues. I'd like to know the OS, database server and PHP versions. Save a phpinfo() output page. You can email it to me if you don't want to attach it.

> You've had serious issues. I'd like to know the OS, database server and PHP versions.
> Save a phpinfo() output page. You can email it to me if you don't want to attach it.

I recently switched to Drupal 5.0, so this applies to ver. 5.x-1.0-beta from 2006-Dec-17; the behaviour of Amazon associate tools is basically the same like for Drupal 4.7; however, now the page admin/content/amazon_import/manage is completely empty.

Basic System information:

OS: Debian GNU/Linux Sarge ("Stable")

Apache: 2.0.54
PHP: 4.3.10-18
MySQL: 4.0.24_Debian-10sarge2-log

I'll send you the complete output of phpinfo() by mail.

Regards, -asb

I just made the first non-beta release. Let me know if you have any problem with it.

Hi,

I installed the release 5.x-1.2 from 2007-Jan-29; now, the "Manag" tab indeed exists on /admin/content/amazon_import/manage, and it works at least partially. Currently, I ran into four main problem areas:

(1) "Pass Count" counts wrong
(2) Updating of stale records works only partially
(3) "Search Category" doesn't work properly
(4) Displayed prices are still wrong.

Ad 1) "Pass Count" counts wrong

Total registered items: 7433

Selecting "Pass Count" of 5 starts like this ("-" = one click on "Refresh prices", "..." = some clicks left out):

5894 - 5864 - 5834 - 5774 ... 4334 - 4314 - 4294 - 4274 - 4254 ... 3928 - 3923 - 2918 - 3913 ... 3842

It says, "Ten records are refreshed per pass", selecting a pass count of 5 should result in 50 records, right? However, it starts with steps of 30 records and then slows down to steps of 5 records; somewhere half the way, it stops refreshing anything at all. So approximately half of all my registered items still have amazon links without the proper affiliate code embedded (resulting in a bug from mid-2006).

Ad 2) Updating of stale records works only partially

As said above, refreshing stale records stops half the way; the Cron tasks seem to have no effect, however, the watchdog registers no errors. Over the hours, the amount of total stale pices rises insted of getting smaller. As far as I can see, Cron works properly on my server.

(3) "Search Category" doesn't work properly

The "Search Category" are a nice enhancement; however there are special characters like German Umlaute embedded which should be properly encoded as entities; currently, it displays wrong characters and thus searches in categories that don't exist, resulting in nothing being searchable. "Search Category" works as usual in categories without wrong characters.

Also, an entry to not* limit the search to a sub-category is missing, which results in many items not being found.

(4) Displayed prices are still wrong.

As mentioned several times before, the Amazontools mostly display wrong prices, meaning: prices not matching those currently valid on the Amazon sites (usually much higher than at Amazon's website).

Regards, -asb

amazontools-5.x-1.2:
Content > Amazon import: The import search looks ok. When I click books or DVD or music, I get:

The search boxes are there and empty.

While looking through your code, I changed line 585 from:
                     'title' => ($product_type),
to:
                     'title' => t($product_type),

In amazon_import_page() I replaced:
  $_search_form_values['SearchIndex'] = arg(4) ? arg(4) : $search_indexes[0];
with:
  if(arg(4))
    {
      foreach($search_indexes as $si_key => $si_value)
        {
        if(strtolower(arg(4)) == strtolower($si_value))
          {
          $_search_form_values['SearchIndex'] = $si_value;
          }
        }
      }
  else
    {
    $_search_form_values['SearchIndex'] = $search_indexes[0];
    }

petermoulding.com/web_architect

peterx:

You should have opened a new issue. However, wait 24 hours from now and try the latest download.

asb:

Do you have non-books in your tables?
Did you load them from a store other than Amazon.de?

This I have noticed: each Amazon store uses different IDs to identify their products...that they all use ASINS for books kind of hides that fact. If you have a bunch of, say, DVDs from, say, the UK store in the database, and you subsequently switch to, say, the French store, those UK DVDs may never get updated again. That would look like your first and second problems.

I don't understand how your search problem comes about at all. The search doesn't send the text description of the search categories, it sends the category's ID number, which is the index of the array entries in amazon.DE.inc. You can leave out the umlaute(sp?), misspell the term or pass it a black string. If the ID number is correct, it's all the same.

I did have to add an "All" option to search all categories, though.

Tomorrow (Sorry, Superbowl today) I'm going to run a few more tests, then release a beta that will have a database upgrade. I'll have most of the data Amazon sends serialized and saved in a "data" field so folks can present whatever they want via theme functions.

> Do you have non-books in your tables?

Yes.

> Did you load them from a store other than Amazon.de?

No.

Just installed 5.x-1.3 from 2007-Feb-04

Ad 1) "Pass Count" counts wrong

Can't tell if this is fixed; the Refresh-Script still stops at 4658 records:

Total registered items: 7473
Total stale prices: 4658

Having reached this number of stale prices by clicking on "Refresh prices", no further updates are made. After leaving the "Manage" tab alone, the count of stale prices slowly rises again.

Ad 2) Updating of stale records works only partially

As above.

Ad 3) "Search Category" doesn't work properly

Seems to be working now.

Ad 4) Displayed prices are still wrong.

No change, still wrong prices displayed.

Greetings -asb

asb:

Well, I finally got a grip on the incorrect price problem. The MerchantID I'm using is "Featured," which unfortunately is a US-only deal. If I don't specify a MerchantID, I get multiple offers returned and no real way of picking just one. Using it gets what we got.

This will require a little thought.

On the refresh problem, do you have an idea of what types of products are being updated vs skipped? If all the skipped products (or all the updated ones) are books or non-books, that could be a helpful clue.

> On the refresh problem, do you have an idea of what types of products are being updated vs skipped?
> If all the skipped products (or all the updated ones) are books or non-books, that could be a helpful clue.

No, sorry, I don't have any clue and no idea how I could find out. The table "amazonitem" contains 7473 records according to PhpMyAdmin; since PhpMyAdmin doesn't numer the record sets and the table doesn't contain a continuous counter, I don't know what "record 4658" (or currently: 4946) is. I don't even know in which sequence the update script works through the records.

If I select "Show 10 records, beginning with 4658" in PhpMyAdmin, the first diesplayed entry is a book; doing the same with "4946" I also get (another) book. "PriceDate" for all these items is "2001-01-01 00:00:00", but this applies to all records on the PhpMyAdmin pages before and after these ten records.

If I additionally select "Sort by key" and "PRIMARY (increasing)" which seems to be the field "asin", record # 4658 appears to be a another book, also with "PriceDate" "2001-01-01 00:00:00"; record # 4946 is abook, also, but with "PriceDate" "2007-01-31 17:22:18".

Can you give me instructions how to identify the record(s) where the update script stops?

Greetings, -asb

This SQL

ASINs for books, being ISBN numbers, are strings of digits. For other products they're alphanumeric.

That said, I may have found the problem. If an item isn't priced for any reason (say it's discontinued) it isn't deleted or any such because it's attached to a node somewhere. But that means it sits there waiting to be refreshed during the next loop.

Would it be possible for you to dump the amazonitem table and ship it to me by email?

> Would it be possible for you to dump the amazonitem table and ship it to me by email?

Sure, when I get an e-mail-address ;) Or shall I uuencode and paste it into http://drupal.org/user/10137/contact? The compressed SQL dump of the "amazonitems" table is approx. 1.5 MB large, I don't know if drupal.org will handle this...

> [...] I may have found the problem. If an item isn't priced for any reason (say it's discontinued)
> it isn't deleted or any such because it's attached to a node somewhere. But that means it sits
> there waiting to be refreshed during the next loop.

Sounds very plausible, however, the items i got when jumping to "record" # 4658 or 4946 as described above are still in Amazon.de's native catalogue (not marketplace or some other 3rd party retailer). But probably these are not the records where the script stops.

This seems to cause a major design issue; at least I wouldn't be too happy if anything would delete my attached nodes when Amazon decides to cancel a product ;) Additionally, a product might be discontinued by Amazon itself, but could appear under the same ID (ASIN = ISBN in category books) again if someone is selling it second hand - at least on Amazon.de huge parts of the catalogue consist of such 3rd party stuff.

How should such items be handled? E.g., should the images be cached locally, or deleted if Amazon deletes them? Does Amazon even delete images of older products? At least some 2nd-hand-books do have cover shots, and I doubt that one-time-2nd-hand-sellers can upload images to the Amazon database. If images and product information would be deleted, what would happen to the attached nodes? Most propably they would look pretty broken. Does this really make sense?

That said, from a functional point of view I'm not too worried about prices to be "up to date"; as mentioned before, most displayed prices are wrong anyway. The originating problem was a bug with wrongly embedded affiliate codes ("...redirect?tag=AMAZON_SEARCH_AMAZON_ASSOCIATE_TAG..." instead of affiliate ID) from mid-2006 which still are creeping around in my database; if I remember correctly, these broken URLs were the reason to introduce the update script in Amazontools a few months ago, the manage tab was supposed to take care of those broken URLs. Maybe there is a simpler way of fixing at least this old issue since the refreshing of prices respectively records seems to cause more and more trouble...?

When querying the database for

SELECT *  FROM `amazonitem` WHERE `detailpageurl` LIKE '%http://www.amazon.de/exec/obidos/redirect?tag=AMAZON_SEARCH_AMAZON_ASSOCIATE_TAG%'

I get 777 records; wouldn't it make sense to repace the wrong strings directly in the database?

Regards, -asb

When querying the database for

SELECT *  FROM `amazonitem` WHERE `detailpageurl` LIKE '%http://www.amazon.de/exec/obidos/redirect?tag=AMAZON_SEARCH_AMAZON_ASSOCIATE_TAG%'

I get 777 records; wouldn't it make sense to repace the wrong strings directly in the database?

That would handle it, yes.

Sounds very plausible, however, the items i got when jumping to "record" # 4658 or 4946 as described above are still in Amazon.de's native catalogue (not marketplace or some other 3rd party retailer). But probably these are not the records where the script stops.

You can't count on row numbers being the same between running an identical query twice...at least in theory. You'd use the primary key to identify a row uniquely.

This seems to cause a major design issue;

Well, a design issue anyway. It would make the amazon-node entries would come up blank and amazon entries would just lack the product details...the review itself wouldn't go away. It would be a mess, but survivable.

On the other hand, you've got a genuine bug here that I wouldn't have found without the volume you've entered over time, and it needs new functionality to fix. I need to track the last time pricing was attempted, as well as when it succeeds, I need to present a list ASINSs with failed attempts and identify the associated nodes

Hi,

issue still exists in release 5.x-1.6.

Greetings, -asb