Integrate Invisible reCAPTCHA option from Google

looking for your patch...

Hi, i tested out the patch provided by VacuumWizaard. But I think a part about the actual form submission is missing. I therefore wrote my own patch based on his. Would love some feedback ...

I'm trying to patch with first the second, then the first file, the module on a working D7 site and I cannot achieve that. Perhaps I should apply it over d8 or directly on the repo? Anyway if it isn't too difficult would be interesting to officially add support for this. Thanks in advance by your help.

I made this patch based on @bobdekinder code, with some little improvements.
This is for version 7.x-2.0, once the issue for it was closed earlier (https://www.drupal.org/node/2859577).

Please review.

Thanks!

D8 first, than backports. We always write patches against DEV.

Hiding bad patches.

@chgasparoto #8 I don't achieve to get this running. I've applied the patch but the options don't appear as modified in recaptcha.admin.inc. I've already reinstalled the module. Ideas? Thanks. Thanks @hass #9.

@hass alright, thanks!

@inovegil did you look up on size option? I got this running on a simplytest.me instance without any problems.

Yes I did. The options are normal and compact. I'll try completely erasing and reinstalling. Thanks.

I tried that in simplytest.me it works fine, It's strange because uninstalling and erasing the module (captcha and recaptcha) and then reinstalling them the option is not appearing. One posible reason is that my site is in spanish but I added english, changed the language of the page and the box size was the same options. It's really strange. I tried applying the patch both with patch and git apply without errors. Thanks.

@inovegil That's weird. Later I'll try with another language and I'll let you know.

I wrote a patch for the dev version.

Please review.

Thanks!

Re-uploading the patch.

I made a mistake forgetting right blank spaces on Badge options array.

@inovegil I tested with a Portuguese site and worked fine too. Did you get any errors?

integrated your patch to Drupal 7

Why is testbot not running on the patches? :-(

D7 uninstall code is missing.

Hello. It seems that there isn't any error. It's very strange, because on all my tests outside this site it worked properly. Regards.

Hide the chgasparoto patch with errors (8.x). He said that they reuploaded the patch dued to mistakes.

I tried #17 patch, but it doesn't work for me. It shows always:
The answer you entered for the CAPTCHA was not correct.

I'm not sure why it happens. Firstly I think invisible reCaptcha works only with https, I tested it on local instance without https:
https://developers.google.com/recaptcha/docs/invisible

The script must be loaded using the HTTPS protocol and can be included from any point on the page without restriction.

Secondary I'm not how it should work. Follow docs save/submit button must have attributes:
class="g-recaptcha" data-sitekey="your_site_key" data-callback='onSubmit'

But now I see another button has the same attributes, not save/submit button.

Will the Invisible reCAPTCHA option be incorporated into a D7 version of this module?

Is there a time frame for that?

Thanks Mike

Reattach for testing

Failing patches cannot committed.

We should add a note that people should uncheck Add a description to the CAPTCHA in CAPTCHA settings or an empty fieldset with just a bit confusing text is shown.

#17 works fine for a simple form.

But it does not work when two forms use invisible reCAPTCHA on the same page.

To be able to make it work, I had to do a few things:

1. Apply the following patch to enable reCAPTCHA to be set on two forms in the same page: https://www.drupal.org/node/1833822#comment-11430341

2. Modify the code of the patch like I suggested here: https://www.drupal.org/node/1833822#comment-12034233

3. Modify line 54 of recaptcha.invisible.js :
grecaptcha.execute();
by:
grecaptcha.execute(Drupal.behaviors.recaptcha.widgets[$('.g-recaptcha[data-size="invisible"]', $form).attr('id')]);

That way, you'll make sure reCATPCHA executes the right widget and not the first to have been defined when loading the page (without the widget ID as argument, grecaptcha.execute() automatically fallback to the first defined CAPTCHA).

Sorry, I did not do a patch, I'm not sure how that works and I'm lacking time. I still wanted to throw the solution out there though, it may help a few folks.

Cheers

I tried the patch in #17 as well as the changes suggested in #31
The captcha badge shows up on the form, but on submit I get "The answer you entered for the CAPTCHA was not correct"
I'm getting this in the logs:
"post blocked by CAPTCHA module: challenge reCAPTCHA (by module recaptcha), user answered "Google no captcha", but the solution was "1""

My fault, I had applied the patch incorrectly.
#17 Appears to be working in my case.

Made a little fix for 8.x-2.x version.

Patch and interdiff attached.

Hm... seems that previous patches will not work on forms with multiple submits, for example with multiple field and "Add more" button.

Tried to fix this issue with new patch. Interdiff attached.

#31 I put everything in one file, this work, Thanks!
But, I have problem with ajax webforms, maybe who can help me

I had integrated the new Google Invisible reCAPTCHA options for Drupal 7.And it supports multiple forms in one page.In the recaptcha-customize.js file, Users can rewrite two functions before or after Google reCAPTCHA widget rendered.Here's my patch:

I had integrated the new Google Invisible reCAPTCHA options for Drupal 7.And it supports multiple forms in one page.In the recaptcha-customize.js file, Users can rewrite two functions before or after Google reCAPTCHA widget rendered.Here's my patch:

I had integrated the new Google Invisible reCAPTCHA options for Drupal 7.And it supports multiple forms in one page.In the recaptcha-customize.js file, Users can rewrite two functions before or after Google reCAPTCHA widget rendered.Here's my patch:

I had integrated the new Google Invisible reCAPTCHA options for Drupal 7.And it supports multiple forms in one page.In the recaptcha-customize.js file, Users can rewrite two functions before or after Google reCAPTCHA widget rendered.(ps.Please ignore my last three comments,I was really confused with the drupal org issue rules.)Here's my patch:

@hass
I had updated the new Google Invisible reCAPTCHA options into the 7.x version.Can you merge the patch into the development release?

1. Failing patches cannot committed.
2. D7 can only committed after D8 has been committed.

I am new to Drupal.

I just wanted to launch my own blog and installed Drupal 7.54.

Everything was fine until I installed the Google reCaptcha module from here: https://www.drupal.org/project/google_recaptcha

I have configured reCaptcha correctly, entered my web site secret and public keys.

However now I lost my admin access to Drupal because there is probably a bug in this reCaptcha module from here: https://www.drupal.org/project/google_recaptcha

After I pass the Captcha chalenge and enter correctly my Admin name and password I always get this error:

Notice: Undefined index: timeout-or-duplicate in function g_ask_google() (line 118 in file /home/superbank/www/www/blog/sites/all/modules/google_recaptcha/google_recaptcha.under_hood.inc).
Google reCAPTCHA does not accept this submission. Try again please, or contact to Site support services.

If you cannot see the image please click HERE

As I'm not able to log in as Administrator now I cannot disable the Drupal reCaptcha module.

As a result my Drupal web site is dead now.

I need an urgent advice on how to fix this annoying issue.

I have refined a patch of Matroskeen (Thanks for your work =) ) for D8, that was added here #36. Tested it with Form API Examples, seems it works both on ajax and non-ajax form. Please, test it again.

There is a typo in recapthcaOnInvisibleSubmit. I guess that the noscript code no longer works and may need to be removed. Not tested.

Hass, I have updated patch according to you comment. Seems, that reCaptcha doesn't work at all with disabled javascript option. So I removed completely that feature from module. Actually, I think that we need some features if javascript is disabled, but not sure if it's related to this issue. Moreover, I have updated recaptcha.invisible.js code, due to some issues in it. Please test it.

#39 didn't apply cleanly, re-rolling to fix that problem. Re-roll was verified with a visual diff tool: no code changes, only diff metadata and file paths were updated.

Switching to D7 to run tests.

Trying to fix the tests by taking code from https://www.drupal.org/node/1833822#comment-12136661

Tests are passing now; let's now fix coding standards compliance.

I don't know how to use 'use' ;)

Coding standards not yet fixed..

Just 1 warning about coding standards, will leave it like that. Manual testing is still to be done.

Switching to 8.x while I perform manual testing and perform some other adjustments (we may need to call .reset() as done in #2493183: Ajax support / Use behaviors).

Hiding interdiffs.

Have you ever seen the test with select box next to file upload button?

this in not work with ajax, I try patch https://www.drupal.org/files/issues/2493183-71-ajax-support.patch with #61

@hass: hey! I've seen it :), it only offers a single test setup, but I need more comprehensive testing than that. Any other way you know to get multiple test envs without (temporarily) switching the issue to 7.x?

Some results from manual testing for #61 (D7, using Examples module):
reCaptcha V2:

• Works fine for normal forms (e.g. login form in block).
• Works fine for AJAX-enabled forms.
• Two forms with reCaptcha get a separate reCaptcha widget.
• A valid reCaptcha is "cached" when submitting the login form; so if there are validation errors, no reCaptcha widget is shown again (I'm guessing it gets stored in the form cache?).

Invisible reCaptcha:

• Click handler to trigger background fetching of reCaptcha is not triggering for AJAX-enabled submit (try examples module, /examples/ajax_example/wizard path). This is related to Drupal's AJAX using mousedown to trigger the AJAX submission and disabling the click to prevent other problems. See https://drupal.stackexchange.com/questions/11638/how-to-programmatically...
• Multiple forms with invisible reCaptcha generate multiple badges (icon at bottom left/right); subtle, but ugly.

In summary:

• reCaptcha V2: multiple widgets working, AJAX-enabled forms working.
• invisible reCaptcha: works for normal forms, doesn't yet work for AJAX-enabled submit buttons (it may be possible to get it working by overriding Drupal.ajax.prototype.beforeSubmit()).

Given how difficult it seems to be to get the invisible reCaptcha working with AJAX-enabled forms, I won't even attempt it.

@mozh92: #61 and https://www.drupal.org/files/issues/2493183-71-ajax-support.patch can not be applied together. I think #61 does everything that https://www.drupal.org/files/issues/2493183-71-ajax-support.patch does (only exception may be calling .reset() when the behavior is triggered again in a form that was already processed, but I don't yet know why is that necessary).

Using Drupal.ajax.prototype.beforeSubmit() for this may require https://www.drupal.org/node/1907932; this is in order to halt the AJAX submission while the reCaptcha request is performed asynchronously; later, the callback can be used to re-trigger the AJAX submission, but this time with a flag that should let it run in its entirety.

Given how difficult it seems to be to get the invisible reCaptcha working with AJAX-enabled forms, I won't even attempt it.

I try this do, but I no have high experience

I'm using 7.x-2.2 and would like to implement the invisible reCaptcha. I've used #17 patch but I'm getting "The answer you entered for the CAPTCHA was not correct." How to make this work? Thanks.

The invisible captcha patches and seems work fine on the 8.2.x-dev branch. Are there specific concerns other than being able to add an invisible captcha and seeing that it triggers validation when it should (and doesn't otherwise)?

If this is working acceptably now, at least for conventional single-form applications, I'd love to see it back-ported to D7 soon. I have a lot of D7 clients that I need to migrate from Mollom before it shuts down, and I expect that I'm hardly the only one in that situation.

@jerry: you can still use the module. No need to wait for invisible recaptcha.

I remember some people reported issues with above patch that are not fixed yet.

No need to wait for invisible recaptcha.

Understood (and thanks), but I hope to wait for the invisible recaptcha if possible as it provides a user experience closer to Mollom's (no visible captcha in most cases).

the patches worked for me!

I have almost finished applying that patch for webforms form. I will try to complete it and update here in near feature.

There have been bugs / issues reported with this patch.

Is this any different than https://drupal.org/project/antibot ?

Is this any different than https://drupal.org/project/antibot ?

@mstef antibot uses another method to indicate a bot. For example, it waits for a mouse to move to indicate that the visitor is human. This module provides a Google Recaptcha functionality.

The patch is causing us to not be able to contact reCAPTCHA. With 8.x-2.x-dev ((HEAD detached at 572d857)) ...

When I change the widget size back to Normal, reCAPTCHA is contacted again.

esod,

When you register for a new site with Google, you have to choose between reCAPTCHA V2 or Invisible reCAPTCHA. So maybe you have to get a new API key for Invisible to work?

Which also brings up a question: Can an API request tell what type it is? So maybe we could even detect the type and display it accordingly? (I'm not sure this is how it works... I'm just noticing how it looks from the above registration screen--that it seems you sign up for an API key for the classic reCAPTCHA or the Invisible reCAPTCHA.)

@dandaman, redoing my keys with Invisible reCAPTCHA does get my site contacting reCAPTCHA again using the widget size invisible. Thanks.

However, the reCAPTCHA is never invisible and after ten page reloads it never launches the reCAPTCHA widget itself.

Please ignore the screen shot recaptcha-no-contact.png. It's there by mistake.

I am seeing g-recaptcha on the page here:

<input data-drupal-selector="form-nfpouo2wpk2xnrsiiykat1hajjv8hll0fffzzcucbag" type="hidden" name="form_build_id" value="form-NfpoUO2Wpk2XnrsIiyKat1hAJjV8Hll0ffFzZCucBAg" />
<input data-drupal-selector="edit-user-login-form" type="hidden" name="form_id" value="user_login_form" />
   <div class="captcha js-form-wrapper form-wrapper details" open="open"><h4 role="button" aria-expanded="true" aria-pressed="true" class="summary">CAPTCHA</h4><div class="details-wrapper"><div class="details-description">This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.</div><input data-drupal-selector="edit-captcha-sid" type="hidden" name="captcha_sid" value="329245" />
<input data-drupal-selector="edit-captcha-token" type="hidden" name="captcha_token" value="c67585fc88267b5b4235ef3e8b454120" />
<input data-drupal-selector="edit-captcha-response" type="hidden" name="captcha_response" value="Google no captcha" />
<div class="g-recaptcha" data-sitekey="6LegHkYUAAAAAL9wIc5seYwv65n1FZ7zrar2z2ma" data-theme="light" data-type="image" data-size="invisible" data-badge="bottomright" data-callback="recaptchaOnInvisibleSubmit"></div></div>
</div>

As well as the protected by reCAPTCHA sticker.

Now if I can just get the reCAPTCHA widget itself to fire...

I tested patch #56 and it works without problems.

I applied patch #56 and it works without problems. Thanks!

We're also going to use this patch, but not on a modal or in an ajax form.

Note that the reCAPTCHA only fires when Google decides you're a bot or a mischief maker.

It's recommended (not sure by whom; maybe Google, maybe a post I read that seemed knowledgeable) to use CSS to hide the recaptcha badge. Which makes sense. A snippet like this is working for us:

/* Hides the recaptcha badge with css*/
.g-recaptcha {
  display: none;
}

You can also uncheck "Add a description to the CAPTCHA" at /admin/config/people/captcha and the result is truly invisible reCAPTCHA.

Update #56 via latest 8.x-2.x-dev

Works great even with latest 2.3 release. Thanks!

#80

Spent a bit of time with this issues trying to track down the remaining open issues...

I addressed #27 in a new patch attached here.

#31 Multiple invisible reCAPTCHAs on a single page is still an issues, but maybe should be blocked by #1833822. This issue can be reproduced by using Drupal's standard install profile, and enabling the User login block in the Sidebar first region on the Website feedback form, and enabling the invisible reCAPTCHA on both. User login will fail.

Ajax form submission on a form provided by Webform module seemed to work fine.

If we can round up any other open issues, I can take a pass at resolving them.

Actually patches this time.

Needs review for the testbot.

Ajax form submission on a form provided by Webform module seemed to work fine.

@bonus, I think that is related on what is reported on #2493183.

The patch looks good.

Starting from #61, I implemented AJAX support for normal and compact reCAPTCHA (invisible is too hard, AFAIK). Patch attached, also attaching interdiff vs. #61.

Tested on a form loaded inside an AJAX modal (ctools modal, leveraging modal_forms module).

Apologies if this isn't the best issue to post this, I considered #2493183, but given I started from a patch from here it seemed more natural to post it here for proper context.

Switching to 7.x-2.x to check if the existing tests still pass. No tests yet for the AJAX functionality, sorry.

7.x tests queued, I guess switching to 8.x-2.x-dev won't interrupt those.

Oh boy, I haven't learned how to talk to the test bot :(. Trying again...

Ugh, one more try... (same patch and interdiff as 101).

Trying to fix pre-existing tests.

I just applied the patch in #97 to the latest release version (which is dated after the current -dev version).

It appears to work (how to trigger an captcha challenge when using invisible captcha?), but I when I enabled a description ("prove you're not a robot") and customised it, the Catcha box appeared with that message in it, but no captcha. Not very invisible!

Disabling the description made the form disappear.

To trigger reCAPTCHA, use the user agent Googlebot/2.1. I think the easiest way to change your user agent without installing a chrome plugin is by creating a new device in Chrome DevTools.

Re. the patch in #108, I had a bit of a problem integrating this into one of my client's sites - it eventually boiled down to me taking a closer look at this line in recaptcha.invisible.js:

$form.find('input[type="submit"]').click(function (e) {

All the auto-generated forms on this particular client's website actually have <button type="submit"> as their means of submitting the form, however there potentially are other ways to submit a form too:

Button elements with no type attribute
An input with a type of "image"
A button outside the form, linked to the form through the use of the form attribute, e.g.:

<form id="myform">
...
</form>
<button type="submit" form="myform">Submit</button>

I could not apply #97 because I had already applied the patch from #2493183: Ajax support / Use behaviors. Here's a patch that will apply only if you already applied this one before: https://www.drupal.org/project/recaptcha/issues/2493183#comment-12537531

@hass, or anyone else, what else do we need to accomplish in the scope of this issues given #97?

Fixing the issues that different people have reported. Code seems not stable and reliable.

Thanks, @hass. All reported issues are either part of separate reported issues or addressed in my latest patch, I believe. I am very willing to help move this issue forward. Would you want me to start combining issues? Or, do you or anyone else have any other feedback on outstanding bugs in this issue -- I went through all the back comments and not seeing anything specifically called out?

Reroll of the patch in #113 to apply against the latest patch (#110) of #2493183: Ajax support / Use behaviors.

Oops, didn't want to run that patch past the testbot - it's obviously gonna fail.

I found the following issues with patch 117 / 97:

1) Drupal.Ajax.prototype.beforeSubmit() is totally replaced, so no other module can do similar and 'hook in' to this. So I've ensured the previous version of the function is still called too. (Other replacements for the function ought to always do similar, so it shouldn't matter which order the overrides are made. Even if they don't, let's set a good example.)
2) For me, my ajax behavior binds to the mousedown event before the new one in recaptcha.invisible.js is bound. (Presumably just because ajax.js comes before our JS file in the source order). This means that the beforeSubmit() gets called before the clickedSubmit variable is correctly set, and so the AJAX is never aborted, and thus fails validation (because recaptcha hasn't yet had chance to execute before sending the POST data). So instead of checking clickedSubmit at that point, check for a custom data property, which gets set in the initial each loop that binds our own mousedown/click handlers.
3) There is a use of .ajaxSend() that will run for every time behaviors are attached. So I've avoided that (by setting a data property on the document and only binding when it hasn't previously been set).
4) Some browsers won't support checkValidity() for HTML5 form validation. So only call that if it is available (otherwise, the form is always considered valid enough to submit to the server anyway, i.e. the pre-HTML5 behavior).
5) For those that do support checkValidity(), as the default browser behavior is aborted, the browser may not actually show the HTML5 validation errors. So I've added a call to reportValidity(), which should do this.

And here's a version that compatible with #2493183-111: Ajax support / Use behaviors.

#121 reroll via latest recaptcha-8.x-2.x-dev, apply after ajax patch #114

Here's a D7 re-roll #108 against latest dev. But I get following browser console error: `reCAPTCHA couldn't find user-provided function: drupalRecaptchaOnLoad - recaptcha__da.js:426 `.

Edit: Patch missed recaptcha.js and recaptcha.invisible.js

Proper re-roll of #108.

Manually tested #124 and it seems to be working as expected.

#122 reroll via latest recaptcha-8.x-2.x-dev, apply after ajax patch #115

Although valuable, all of these combination patch comments are really distracting from moving this issue forward. Does anyone have additional feedback on #97 and what additional outstanding issues need to be addressed?

If not, does someone want to RTBC #97?

@bonus - not #97, no, because of the points I listed in comment #120. Given much of the recent work in this ticket has built upon #2493183: Ajax support / Use behaviors, I suggest getting that one sorted first?

Nope.

#126 reroll via latest 8.x-2.x-dev (please don't ask for interdiff because for upstream changes).

1. recaptcha-ajax-2493183-127+nocaptcha-2852269-136.patch: apply after ajax patch #129, can't test with CI
2. recaptcha-nocaptcha-2852269-136.patch: complete patch via 8.x-2.x including ajax patch #129, can test with CI

Test passed locally, need CI check.

Thanks @hswong3i, looks fine :-)

Fixup: as long as noscript support removed, the new $recaptcha_src_fallback no longer required, too ;-)

Hi @hswong3i the recaptcha-nocaptcha-2852269-139.patch works perfectly on both a fresh install added to the login form and on a site with webforms. Thank you!

Syntax cleanup for $captcha['form']['#attached']['library'][] = 'recaptcha/recaptcha';, better sync with #2493183: Ajax support / Use behaviors style.

Remove all <noscript> related test cases as no longer valid.

Hi All,

I had an issue with invisible reCAPTCHA preventing client side validation. I was using jQuery validate and when I would click the submit button no validation would occur. In order to fix this I slightly modified the preventFormSubmit function of recaptcha.invisible.js by checking to see if novalidate was set, as it was with my form.

Old code:

if (form && typeof form.checkValidity === 'function') {

New code:

var novalidate = form.getAttribute('novalidate');
if (form && typeof form.checkValidity === 'function' && !novalidate) {

Now the form successfully validates upon completion of the challenge if it is triggered. Otherwise it validates normally.

I am essentially forcing the captcha to validate before the client side validation runs. Can anyone chime in on this, maybe with a better solution to this problem?

@harveyk good spot, yes, a check needs adding for that attribute.

I've lost track of how the patches are being made in combination with #2493183: Ajax support / Use behaviors. We really need that one progressed towards commit. So due to that, I'm only attaching a single patch & interdiff, sorry!

Please add documentation to all conditions and not so obvious settings in the js files. Something that explains the code in detail and its intention. On many lines I have codewise no idea why the condition exists and what it covers/does. This mostly affects recaptcha.invisible.js. Better write more than less. We need to be able to directly understand the code - without reading every comment in this case why a change was done.

Can we also either document somewhere for everyone to remember OR find a better selector for the code referencing to submit buttons assuming they have markup of <INPUT> element?

Several sites I have worked on do have theme alterations which replace <INPUT> with <BUTTON> markup on form submission buttons, and then the whole Recaptcha detection functionality fails, always saying "The answer you entered for the CAPTCHA was not correct." because nothing is in fact being verified with Google on submit.

I keep forgetting about this but and keep banging my head around it for hours until I come back to the same discovery :))

I believe it relates to both patches: D7 and D8

Patch reroll via latest 8.x-2.x-dev, with #144 changes + minor bugfix (see interdiff)

Patch #148 reroll via 8.x-2.x-dev...

No interdiff (just diff) could be given due to following error message:

1 out of 4 hunks FAILED -- saving rejects to file /tmp/interdiff-1.dMemyi.rej
interdiff: Error applying patch1 to reconstructed file

How annoying for a child issue for another RTBC issue...

Just a note that the combo patch from #14 no longer applies.

Hi guys

I've found a bug with multiple recaptcha on one page. If you connect recaptcha to more them one form, recaptcha validation token always will be added to the first form. And after submit second form you get error 'The answer you entered for the CAPTCHA was not correct.'

I added fix

Patch recaptcha-nocaptcha-2852269-151.patch on #151 works fine.

But as last test shows, recaptcha-ajax-2493183-156+nocaptcha-2852269-151.patch fails to apply the patch. I manually merged the changes between nocaptcha-2852269-151.patch and recaptcha-ajax-2493183-156.patch, but nocaptcha isn´t working on ajax forms.