Hi all,

Thanks for the great module. Use it quite extensively for calculating prices on my websites. Sadly, Yahoo seems to be not as reliable as usual the last few days returning things like:

currency 18/07/2011 - 11:15 currency: 1 CNY = 1 EUR
currency 18/07/2011 - 11:15 currency: 1 USD = 505570816 EUR

Checked the code for caching a little while, but it seems there is no mechanism to prevent bad data to be used. Could we make something like; "only update to new rate if difference with old rate is < 20%"?

Would anybody be interested in that?

Thanks
Mark

Comments

norbert.banfi’s picture

I have the same problem too.
This prevent mechanism would be so cool. (Unless the old rate is 1 or 505570816)

panoskop.net’s picture

I am having the same issue

mollyow’s picture

Same issue here.

karengrey’s picture

same here too. On friday the Yen was screwed up. Today its Euros.
Need a fix ASAP!

alan d.’s picture

Version: 6.x-1.3 » 7.x-1.x-dev
Category: feature » bug

Well this is not a bug in the module, but there should be safe guards for this. There would be issues with say the Zimbabwe Dollar as inflation has hit 10 Million + %, so the threshold would be constantly hit. Luckily for this currency, external currencies are mainly used now that appear to be keeping the depreciated dollar in check...

info@handmadeIT’s picture

Version: 7.x-1.x-dev » 6.x-1.x-dev

Hmm, i recently refiled this bug as critical (which had been removed) because using the module on a productive e-commerce-site definitly screws up the system and freaks out the customers.

Do you know what exactly the problem is? Does Yahoo delivers wrong data?

My idea was to define a threshold where the admin or another user with sufficient rights has to approve the new currency exchange rate.

alan d.’s picture

Version: 6.x-1.x-dev » 7.x-1.x-dev
Priority: Normal » Major

I'm totally refactoring the module for a new 7.x-2.x branch. If this gets accepted, then the safeguard built into this version can be easily backported to the older 7.x-1.x / 6.x-1.x branches too. I really dislike the results from Yahoo, it was the least accurate and produced strange results on pegged currencies - returning 0.0 instead of 1.0 for example.

So the issue is that Yahoo produces

a) randomly bad results
b) bad results for pegged currencies

This is the solution from the 7.x-2.x version that I'm working on is:

/**
 * Menu callback; module settings form.
 */
function currency_api_admin_settings($form, $form_state) {
....
  $form['currency_api_new_rates_margin_threshold'] = array(
    '#type' => 'select',
    '#title' => t('Currency new rate threshold'),
    '#default_value' => variable_get('currency_api_new_rates_margin_threshold', 20),
    '#options' => array(
      0 => t('Disabled'),
      5 => t('!percentage%', array('!percentage' => 5)),
      10 => t('!percentage%', array('!percentage' => 10)),
      15 => t('!percentage%', array('!percentage' => 15)),
      20 => t('!percentage%', array('!percentage' => 20)),
      25 => t('!percentage%', array('!percentage' => 25)),
      30 => t('!percentage%', array('!percentage' => 30)),
      40 => t('!percentage%', array('!percentage' => 40)),
      50 => t('!percentage%', array('!percentage' => 50)),
      75 => t('!percentage%', array('!percentage' => 75)),
      100 => t('!percentage%', array('!percentage' => 100)),
      150 => t('!percentage%', array('!percentage' => 150)),
      200 => t('!percentage%', array('!percentage' => 200)),
      250 => t('!percentage%', array('!percentage' => 250)),
      500 => t('!percentage%', array('!percentage' => 500)),
    ),
    '#description' => t('Provides a safeguard check on the rates returned by the web service. If these exceed this margin, the results are considered invalid and the response is ignored and an error is logged. Note that cached results are used for this comparison when available, otherwise the base currency table rates are used.'),
  );
...
}

# and in the exchange rates lookup, the results per web service are checked using.
# if multiple web services are specified, each is tried until a valid rate is returned
# if no results are found, the older cached rates are used, or if non-exist, there are
# base conversion rates in the DB to fall back on.

/**
 * Validates the given rate against possible zero values and that they are
 * within any specified boundary thresholds.
 */
function currency_api_is_valid_rate($key, $from, $to, $rate, $old_rate = NULL) {
  // We parse values less than 0.1 trillionth to mean 0 as float == float may
  // give unexpected results. Negative rates are considered invalid.
  if (!empty($rate) && is_numeric($rate) && $rate > 0.0000000001) {
    // An optional safe guard for invalid lookup results.
    // We use the last known rate, falling back to the base rates.
    if ($margin = (int)variable_get('currency_api_new_rates_margin_threshold', 20)) {
      $validate_rate = empty($old_rate)
          ? ((float)$from->base_rate) / ((float)$to->base_rate) : $old_rate;
      $margin = ($margin * $validate_rate / 100);
      if ($rate > ($validate_rate + $margin) ||
          $rate < ($validate_rate - $margin)) {
        // Log as an error as this suggest a failure in this service provider.
        currency_log($key, t('Returned a rate outside of defined safe margin while doing the conversion lookup between %from to %to. The existing rate is %rate, the new rate was %new_rate. The margin was %margin%',
            array(
              '%from' => $from->iso3,
              '%to' => $to->iso3,
              '%rate' => $validate_rate,
              '%new_rate' => $rate,
              '%margin' => $margin,
            )), WATCHDOG_ERROR);
        return FALSE;
      }
    }
    return TRUE;
  }
  return FALSE;
}

We store base conversions to the USD of every currency in the DB, so we always have a base rate to compare to. This is editable via the UI and updated via cron.

And there are two other built in currency exchange rate providers; Google calculator and the ECB currency feed. The EU bank gives daily rates for 30 or so main currencies and the Google calculator covers most of the currencies with better precision than Yahoo. However, the Google lookup is based on an un-documented API that should not be used excessively and should not be used on commercial sites.

I wrote a xe.com rates scraper for myself in 9 lines of code which is the default on my test site and this is working great. But, I can not add this as it is against the TNT of xe.com. If I can get access to a feed / api key, then I'm happy to write a plugin for any commercial exchange rates web service.

Issues with back-porting this is that currently there is no way to validate against an old rate if it does not already exist in the currencies cache table. My timeline is to try and finish this today, and then it is up to the maintainers to accept, tidy up, or reject this new code base and test. I can not give a timeline on this process, and then potentially back-port to a 6.x-2.x branch.

alan d.’s picture

Hi Guys,

Testers and reviews needed for #1219512: New branch & API. Both for Drupal 6 and Drupal 7 versions.

1) This has 3 exchange rate providers, and it is easy to write a scraper for other services.
2) It has base rate conversions with the USD stored in the db, so ABC to XYZ should always return a result
3) It has a safe guard margin, from 5% to 200% for new lookups.
4) More caching options

And heaps more.

This represents about 4 days programming and it is a complete rewrite of the currency_api module INCLUDING an updated API.

So backup / test on test sites only.

The API has changed, so read the README for details or ping me for help. Report issues to the other thread.

Cheers

amateescu’s picture

Status: Active » Closed (duplicate)

As @Alan D. said in #8, work on alternative currency exchange providers is happening in #1219512: New branch & API so I'm closing this issue as a duplicate in order to concentrate the focus over there.