Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
I'm working on this feature. I'll submit a patch when ready.
I need it for OSM Nominatim (https://operations.osmfoundation.org/policies/nominatim/) but it should be available for any geocoding service provider with two configurable values:
- period in seconds
- number of requests allowed for the period
Comment | File | Size | Author |
---|---|---|---|
#9 | geocoder-throttle-3081749-9-8.interdiff.diff | 1009 bytes | GaëlG |
#9 | geocoder-throttle-3081749-9.patch | 13.27 KB | GaëlG |
#8 | geocoder-throttle-3081749-4-8-interdiff.diff | 2.59 KB | GaëlG |
Comments
Comment #2
GaëlGHere's a first patch, it seems to work well for my needs.
Comment #3
itamair CreditAttribution: itamair as a volunteer commentedthanks @GaëlG ... giving a look into this, will write my feedbacks to you asap ;-)
Comment #4
itamair CreditAttribution: itamair as a volunteer commentednice feature @GaëlG ... and your patch implementation makes sense to me.
Considering that the Nominatim geocoding policy states clear limits on the Geocoding requests frequency, it also makes sense that the Nominatim Geocoder server provider setup/annotation definition already bundles throttle presets. (isn't it?).
The new attached patch corrects a couple coding standards related to correct class method and constructor documentation.
Please let us know if everything makes sense and that you think this new version might be committed into dev.
Comment #5
itamair CreditAttribution: itamair as a volunteer commentedsorry @GaëlG ... another question. I am not that expert on the Nominatim geocoding service and provider.
I couldn't never try it out but I notched that, in the Geocoder module the Nominatim provider configuration require the following "required" parameters:
- Root URL
- User agent
- Referer
I couldn't clearly realize what those parameters are for, but what I ask you is:
- should all the above be required? does it makes sense to you? ... or some of them might be set as optional to let the provider work correctly anyway.
Comment #6
GaëlGThank you very much itamair for the review. I'll answer your questions, but I just figured out that the patch I submitted does not contain the last changes I made. I think I messed up at some time!
Comment #7
GaëlGHere's the correct patch. It contains two bug fixes I made that were mysteriously not in patch #2. It also contains your coding standards improvements from #4.
To answer your questions:
- Yes, it makes sense to me that the Nominatim Geocoder server provider setup/annotation definition already bundles throttle presets. Their doc says "an absolute maximum of 1 request per second" and "Make sure you stay well below the API usage limits.". So I put 1 request per 2 seconds. But I just figured out that I put it in the generic nominatim provider instead of the OSM one. I'll submit a new patch for this.
- About the parameters, for the OSM free service, Root URL is obviously required, but it should not change. OSM Nominatim asks for User agent or Referer, so it's required to fill at least one of the two. Personally I filled both and was not banned for that.
Comment #8
GaëlGWhoops, here's the patch.
Comment #9
GaëlGHere's the fix for the Generic Nominatim vs OSM Nominatim problem.
Comment #10
itamair CreditAttribution: itamair as a volunteer commentedthanks again @GaëlG ... great work.
I will test and review it again. Do you think that this is ready for commit (if I don't find major issues?) into dev and into new 8.x-3.x release?
I have got another question and doubt, though.
The 8.x-3.0-beta1 might be still updated without worrying too much about possible back compatibilities,
but still updating the 8.x-3.0-beta1 version users (that already installed the module) would face a big error in the module (the GeocoderThrottle won't work at all), as long as they don't run the following composer additional command:
composer require davedevelopment/stiphle
Isn't it? do you agree?
For sure, I will highlight and underline that need (to require the new davedevelopment/stiphle library) in the new version release specification.
But do you have any clue to make this update easier, more aware and more automatic for already installed 8.x-3.x releases?
Due to this, this upgrade would be even harder if (wanted to be) replicated into the 8.x-2.x branch ...
Comment #11
GaëlGHi @itamair, I think it's ready for commit if the review is OK.
About the composer problem: I guess it won't work if people update without using composer. A hook_requirements() implementation may help to warn people? But if they use
composer update drupal/geocoder
, I guess the dependency will be downloaded automatically by Composer, no? I'm not a composer expert, I'm not able to confirm this without being able to test it.Comment #12
itamair CreditAttribution: itamair as a volunteer commentedNo ... unfortunately the
composer update drupal/geocoder
won't download the new library (it seems to me).May you reproduce the real user scenario (applying your patch to an existing 3.x release ... and verify that it would break the site),
and then may you better/rather warning a hook_requirements() or hook_update() warning that would instruct the user on this?.
Comment #13
itamair CreditAttribution: itamair as a volunteer commentedComment #14
GaëlGAs I could not commit to drupal/geocoder to test, I cloned the official repo as "insite/geocoder" in our own composer repo, and required it in a project. Then I committed the patch to this clone and did:
So, if people use composer to update geocoder to a newer version/commit, they will get the dependency automagically*.
There is a problem if people do not use composer (until #2622420: Site-builders should be able to use projects with composer dependencies is done). So I started writing a hook_requirements() implementation. But I remembered that composer is already a requirement according to the module documentation, and there is for example no check to see if Geocoder-PHP is installed, whereas it's a composer dependency. We might add that kind of check for people trying to use the module without Composer, but that's another issue, no?
*This is not the case if they simply apply the patch using cweagans/composer-patches, as the patch is applied AFTER composer has computed the dependencies.
Comment #16
itamair CreditAttribution: itamair as a volunteer commentedRight you are. Thanks @GaëlG ... great job on this.
Committed into dev, this new feature (and #9 patch) will be part of the next release.