F'up to #622076: invalid_asin filter should be a boolean filter or at least numeric

Watchdog gives me a warning like: "Error retrieving Amazon item AWS.InvalidParameterValue, message: 3531231472 is not a valid value for ItemId. Please change this value and retry your request..". The location reported is, for example, ./taxonomy_vtn/term/875 - that's not a node, but an index page of the taxonomy_vtn module which has no Amazon items attached. I'm not sure how and when this warning is constructed.

When looking up this ASIN at ./admin/settings/amazon/test, I'm getting: "Test failed for this ASIN. Please check the error log for messages." Watchdog has logged this and reports: "Error retrieving Amazon item AWS.InvalidParameterValue, message: 3531231472 is not a valid value for ItemId...", this time with location = referrer ./admin/settings/amazon/test.

So I'm back to the beginning where I tried to build a view which displays me where these ASINs come from ;) Since - at least this particular ASIN - was reported by the test facility with "Test failed for this ASIN", might we assume that this is an invalid ASIN? If so, I'm still getting:

mysql> select * from amazon_item where invalid_asin=1;
Empty set (0.04 sec)

Accessing http://www.amazon.de/exec/obidos/ASIN/3531231472 outputs an error.

However, other ASINs referenced in watchdog with the message "Error retrieving Amazon item AWS.InvalidParameterValue" seem to exist somehow, e.g. http://www.amazon.de/exec/obidos/ASIN/B001MEW9MU. That one gives me an redirect to ASIN B001MEISV6 (URL http://www.amazon.de/gp/product/B001MEISV6?ie=UTF8&redirect=true).

Testing 3531231472 from within Drupal fails, testing B001MEISV6 succeeds. I think, if Amazon API can't pull those redirects from Amazon, this would qualify as "invalid", also.

Another example: ASIN B001E3IKS6, also reported with "AWS.InvalidParameterValue", on Amazon.de also redirected (-> http://www.amazon.de/gp/product/B001DXCS4E?ie=UTF8&redirect=true, ASIN is now B001DXCS4E).

Greetings, -asb

CommentFileSizeAuthor
#2 watchdog-log.jpg295.87 KBasb

Comments

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

An item is marked as invalid only if it's already in the database (amazon_item). You can get the "error retrieving" when entering the item, for example, and it would not get into the db at that time. So an item that you *test* is not going to get put into the db. And it skips the db anyway.

I think it's only at cron time that an existing item would be marked as invalid, and that's by design. So we just have to figure out where your messages are coming from - what even triggers them.

asb’s picture

asb commented
StatusFileSize
new watchdog-log.jpg295.87 KB

> So we just have to figure out where your messages are coming from - what even triggers them.

Where these messages are coming from I can't tell before I get a back reference from the erroneous ASIN to the node where field_asin or field_legacy_asin stores it. No wait, I could try a fulltext database search... (Edit: no, I can not; Search & Replace Scanner does not support ASIN fields, arrrr).

But why do you think that it is not Cron that triggers the errors? I would think that 'Amazon' periodically (= a batch of "n" items per cron run) tries to refresh its data, as the Amazon contract requires. On these Cron runs some items can not be found at Amazon anymore, so it outputs these "Error retrieving Amazon item" messages (and others like "SimpleXMLElement::__construct() [simplexmlelement.--construct]: ^ in /var/www/drupal/sites/all/modules/amazon/amazon.module in Zeile 256." ;). According to the timestamps from watchdog, the warnings correlate not exactly with cron runs, but occur more or less in "groups" (screenshot attached; watchdog filtered to "type: amazon" and severity: warning").

Greetings, -asb

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

@asb: I need the detail of the watchdog entry, not just the log.

BTW: I fixed an error just the other day that had the simplexml traceback #677508: HTTP Error 500 results in exception being thrown. If you haven't refreshed your dev version since that was committed (January 6, 2010 - 18:07), please do.

asb’s picture

asb commented

> I need the detail of the watchdog entry, not just the log.

Sorry, I thought the details wouldn't give enough detail, either ;) Some records, filtered for 'amazon' and 'warning':

Error retrieving Amazon item AWS.InvalidParameterValue, message: B002E3KCF4 is not a valid value for ItemId. Please change this value and retry your request..
Error retrieving Amazon item AWS.InvalidParameterValue, message: B002EE80QG is not a valid value for ItemId. Please change this value and retry your request..
Error retrieving Amazon item AWS.InvalidParameterValue, message: B002EECZBM is not a valid value for ItemId. Please change this value and retry your request..
Error retrieving Amazon item AWS.InvalidParameterValue, message: B002EDZDP8 is not a valid value for ItemId. Please change this value and retry your request..
Error retrieving Amazon item AWS.InvalidParameterValue, message: B001E3IKS6 is not a valid value for ItemId. Please change this value and retry your request..
Error retrieving Amazon item AWS.InvalidParameterValue, message: B000ONOGS0 is not a valid value for ItemId. Please change this value and retry your request..
...

Usually no referrer is given, the host addresses differ.

Filtering for 'amazon' and 'message' indicates, that "Amazon items were updated" more or less once per hour (18:06, 17:07, 16:06, 15:06, etc.); that matches my crontab entry for this site (set to '06'); the timestamps when the warnings from above are issued into watchdog do not match exactly the times cron is running.

Greetings, -asb

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

I tried creating just one amazon node (cck) with the first ASIN you provided (B002E3KCF4). It failed, as it should. (I was using amazon.de, assuming you were doing the same).

So I went to the database and created an entry in the amazon_item table with asin B002E3KCF4. Then ran cron. It correctly marked the item with invalid_asin=1.

So there's something I don't know about your database. Why don't you email me the db or a link to it.

Thanks,
-Randy

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

@asb, when you get a chance, please let me have a database that demonstrates this, and I should be able to get it solved. It does sound like there's a bug loose in the wild here :-)

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented
Title: Error retrieving Amazon item » "Error retrieving Amazon item" after update from D5
Status: Active » Fixed

Here is what I think is happening. You have 59 items in your CCK content_field_asin table that are not in the amazon_item table (because they were invalid so they never got there):

SELECT * FROM content_field_asin c left join amazon_item a
on a.asin = field_asin_asin
where a.asin is null

The failures to access them (I think) don't happen during amazon's hook_cron, but they may happen during some other module's cron.

You can remove the 59 items listed in that query and I suspect you'll be rid of your problem.

Thanks,
-Randy

asb’s picture

asb commented

Thanks Randy,

sorry, limited SQL skills. Your SELECT statement from #7 works, but simply replacing SELECT with DELETE doesn't work:

mysql> DELETE * FROM content_field_asin c left join amazon_item a on a.asin = field_asin_asin where a.asin is null;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the rig
ht syntax to use near '* FROM content_field_asin c left join amazon_item a on a.asin = field_asin_asin ' at line 1

MySQL manual gives this syntax:

DELETE [LOW_PRIORITY] [QUICK] [IGNORE]
    tbl_name[.*] [, tbl_name[.*]] ...
    FROM table_references
    [WHERE where_condition]

Also I haven't been able yet to build a report that displays records where "Invalid ASIN" becomes "TRUE" for field_legacy_asin or field_asin. If I expose the filter and set it to "NOT TRUE", I'm getting a lot of records, but the view always comes up empty if set to "TRUE". So either the condition "Invalid ASIN = TRUE" becomes never TRUE, or I'm doing something wrong. I have a required relation with field_legacy_asin respectively field_asin, and the other Amazon data is showing up properly.

What's the catch here?

Greetings, -asb

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

@asb: I doubt that you want to delete the entire node. I was suggesting a few minutes of manually deleting ASIN fields from your nodes using the edit-node web ui. The query gives the nid and the asin.

I'm sure that a simple SQL delete is possible, but I'm not sure if it might have side-effects. If you're willing to consider doing it manually, it might work for you.

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

But this would probably work for you:

DELETE c.* FROM content_field_asin c left join amazon_item a on a.asin = field_asin_asin where a.asin is null
asb’s picture

asb commented

Sorry, I thought the SQL statement would delete the items in the CCK content_field_asin table that are not in the amazon_item table.

OK, now there's a different problem: The nodes I'm getting when running 

SELECT nid FROM content_field_asin c left join amazon_item a
on a.asin = field_asin_asin
where a.asin is null

might have an "ASIN" and a "Legacy ASIN" field (e.g. nid 27177) with no ASIN values entered, or they might not even contain ASIN type fields (e.g. nid 27329). Im not sure I understand what I could delete here; removing the ASIN fields at ./admin/content/types would affect all nodes, and simply editing and saving the nodes does not remove these "NULL" values.

OK, views won't work for these nodes because there is no record in amazon_item table, if I understand you correctly; so I'll review these nodes manually and most probably delete them.

That would leave the question why I'm not getting any "Invalid ASINs", not even in the database (select * from amazon_item where invalid_asin=1; is still empty). Is there anything I'm missing, or is it even remotely possible that I'm having not a single invalid ASIN on this site?

Greetings, -asb

rfay’s picture

rfay
he,him
Primary language English
Location Palisade, CO, USA
commented

@asb: You're not getting the "Invalid ASINs" because the items never made it into the ASIN table, because they couldn't be looked up. But they were ported into the node anyway. The invalid_asin marker is on the amazon_item table, but the items you have are in CCK's world.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.