Problem/Motivation

I tried this on a completely fresh install of the latest Drupal 7.17 and Addressfield 7.x-1.0-beta3.

  1. Enable addressfield.
  2. Add an address field instance to the user entity. Require its inclusion on the user registration form and require the user supply their address.
  3. Enable drupal page cache and set the minimum cache lifetime to something like an hour.
  4. Clear your page cache
  5. Visit the user registration page at /user/register. This should be a cache miss.
  6. Address field should default to the US. Select any US state (like "Washington").
  7. Change the country to Brazil.
  8. Navigate to the user registration page again. This should be a Drupal cache hit.
  9. Address field should default to the US. Select any US state again (like "Washington").
  10. Change the country to Italy.

You should now see something like the following image where an "illegal choice has been detected" message is shown and the form shown represents the Brazilian address field variation (with Brazilian states), even though Italy is selected as the country.

Addressfield country switch page cache validation error

The net result is that when page caching is enabled, users are able to negatively affect the user experience for independent and completely unrelated users who subsequently attempt to submit the form.

This is especially aggravated if you're automatically selecting state/country values on the frontend.

Seems like addressfield shouldn't cache form and form state values when a user selects a different country.

Comments

iamEAP’s picture

Version: 7.x-1.0-beta3 » 7.x-1.x-dev
Status: Active » Needs review
StatusFileSize
new564 bytes

After extensive investigation, it would appear as though there may be something unexpected with how core suppresses validation errors when #limit_validation_errors is set. I don't know the full scope of the issue.

I've confirmed that Address Field is using #limit_validation_errors correctly, and form.inc ultimately calls form_set_error(NULL, '', $form_state['triggering_element']['#limit_validation_errors']); as expected, but even still, the validation error isn't adequately suppressed.

We can hack around it in address field with this patch. This may be a terrible idea, but it works around the problem.

For a slew of related, but ultimately unhelpful reading:

iamEAP’s picture

Title: Selecting a country performs validation against cached value, producing illegal choice error » Country selection can cause validation errors
Priority: Normal » Major
StatusFileSize
new0 bytes
new0 bytes

To make this even more obvious and even simpler to reproduce, I've provided a test. In addition to more basic AJAX tests, it tests the exact scenario described in the issue summary above.

I've attached two patches: a test-only patch (which should fail when run), and a patch with the fix from #1.

I've also renamed the issue to be more clear. I'm also going to bump this to major as per this page describing major as an "error which is only triggered under rare circumstances or which affects only a small percentage of all users"

iamEAP’s picture

jasen’s picture

Priority: Major » Critical

I can confirm this bug, and it's more critical if you try to populate the form with country and state/province information that is different than the default set.

I.E. if the default is the U.S. and I try to populate the country with Canada and the province of British Columbia, the form will break.

rszrama’s picture

Priority: Critical » Major

Why is this form getting cached? That seems odd to me... how can a user with a different session hit the form as rendered by another session to begin with?

iamEAP’s picture

This is related to the Drupal form API's misuse of cache backends as semi-persistent storage for form structure.

The key used to access the form data (stored in cache_form) is based on the validation token associated with the form, which was generated when the form was initially rendered. As such, the validation token for a given form on a given page will be identical for all anonymous users when the Drupal page cache is enabled.

Initially, this doesn't matter because they see the form rendered as it was originally rendered (with all defaults / no data filled out).

When you change the country on an Address field, for what I believe are reasons related to form validation (don't quote me there), it actually changes (and saves in cache_form) an alternate structure for the form relevant to the country. The difference most relevant here is on administrative area where it can change from a textfield to a select list (or from one select list to another). Once the form is actually submitted (as opposed to AJAX-submitted), it attempts to validate the user's input against a now-invalid form structure.

Again, because cache_form's keys are based on the validation token of the originally rendered page, and because all anonymous users have access to that form/token/page, all any anonymous users have the ability to mess with the structure against which actual form POSTs will be validated.

iamEAP’s picture

Note that even with the patch above, we were still seeing this error come through on occasion. We ended up going even further:

  // If country change triggered the submission, kill all errors and messages.
  if (_form_element_triggered_scripted_submission($element, $form_state)) {
    drupal_get_messages();
    drupal_static_reset('form_set_error');

    // If the country was changed via AJAX, and the user is anonymous, the
    // existing page cache entry will be invalid, so we have to wipe it. Note
    // we're using the referer here because we're in an AJAX context.
    if (isset($_SERVER['HTTP_REFERER']) && $GLOBALS['user']->uid == 0) {
      cache_clear_all($_SERVER['HTTP_REFERER'], 'cache_page');
    }
  }

It's a big hammer to swing, and even still we see one or two slip through each day (presumably in situations where users are filling out a given form simultaneously).

rszrama’s picture

Title: Country selection can cause validation errors » Prevent Drupal from caching pages with forms that include an address field widget
Category: Bug report » Feature request
Priority: Major » Normal
Issue summary: View changes
Status: Needs review » Active

I've reproduced this locally, but the solution is still unclear to me.

Just to restate the issue, the kernel of the problem is that the page cache is retaining form HTML with a build ID that matches a cached form state primed by some other user. Honestly, I don't consider this a problem of Address Field any more than it would be the problem of any other module using the #ajax framework in a similar way. It's just a weakness in Drupal itself that isn't easily solved at the code level in the "offending" module itself.

This is exactly the use case http://drupal.org/project/cacheexclude was created to solve: dynamic pages being served to anonymous users. The real solution isn't to nuke the page cache every time the form is validated (note: even if you're using that solution, you should just be removing that page's row in the cache - no need to clear the whole thing) - what you really want is to prevent the page with the form from being cached in the first place.

And that's why it's hard to solve - Drupal doesn't know when it receives a page request whether or not any given page is cacheable. In your case, your user registration form became uncacheable. It's arguable that the registration form should never be cached - I wouldn't have a problem using Cache Exclude to prevent it on any site I build - but at least in your case you could be using Cache Exclude since you know it's an issue.

So we have two ways to solve this:

  1. We can just document that if you're exposing address fields to anonymous users, you should be using Cache Exclude to prevent those pages from being cached. I like this solution, because it's explicit for the site administrator. No guessing why some random page on my site isn't being cached for anonymous users - I've told it not to.
  2. Or we can decide to use drupal_page_is_cacheable(FALSE) any time an address field is rendered. I'm fine with this solution if we do it in a #pre_render callback, as I don't like the idea of a form taking an action every time its build callback is invoked. (Still, I think my problems with that are theoretical, not practical in this case.)

I'm going to make this a feature request to add cache exclusion in a #pre_render callback and see if I can test it as a working fix locally. Note: we may as well only prevent the cache if the Address Field element is actually #access -ible. No sense preventing the cache if the anonymous user wouldn't see the widget anyways.

rszrama’s picture

Oh, that said, thanks for the tests! The same test should apply to my proposed solution above.

iamEAP’s picture

Thanks for the attention, Ryan.

Unfortunately, #2 would not be a workable solution in my particular case because I'm using Form Block to place user registration forms across whole content types, the effect of which would render a vast majority of the site uncacheable.

rszrama’s picture

Hmm, ok. Well, in that case it sounds like it would be best for you if the module remained as is and your custom solution stayed in place. However, if you're clearing the page cache on anonymous page requests to a vast majority of the site, why bother with the cache at all? : P

I don't know your exact situation here, but it sounds like you should consider an alternate strategy: perhaps hiding this form until an anonymous user actually has a need for it and then giving that user a session cookie so they are no longer served pages from the cache. In other words - if you need "cache exclude" type functionality on a wide variety of pages on the site, I'd instead cache everything but change the architecture of the site such that the bits that break when cached are never cached in the first place and users who actually need the functionality can get it when they need it.

iamEAP’s picture

Agreed that it'd be better to leave as-is than to introduce the drupal_page_is_cacheable() call.

My use-case includes (but isn't limited to) paid search landing pages where you want the page to be as fast as possible (cached), but also have the form ready immediately (so no ideal way to instantiate a session on the fly). In the code snippet above, only the page cache entry of the associated page is cleared (including query params), and only when a country is changed. It could probably be optimized a little more by only clearing the page cache if the country selected happened to include a select list of administrative divisions (because otherwise, the country change is of little consequence).

And related to that, just to back up a little bit... Something that I often forget when I'm analyzing this issue...

Just to restate the issue, the kernel of the problem is that the page cache is retaining form HTML with a build ID that matches a cached form state primed by some other user. Honestly, I don't consider this a problem of Address Field any more than it would be the problem of any other module using the #ajax framework in a similar way.

You're correct in identifying the underlying oddity of Form API's AJAX component with regards to anonymous users. However, the specific problem that arises that's rarely seen with other AJAX-enabled modules is that Address Field can modify the #type of certain fields under certain circumstances (namely, administrative division).

In situations where it doesn't modify #type, it's of little consequence (e.g. if the default country is the UK and someone changes it to Vietnam, there are no issues because the state field #type was never changed, and thus, Core never gets a chance to run its FAPI validation for select fields). Problems can even be avoided in narrow circumstances where two countries coincidentally have administrative areas with the same shortcode (e.g. starting with Washington, US and switching to Western Australia, Australia).

Perhaps instead of mucking around with cache clearing, we might be better off focusing on better suppressing the core select field validation handler. This area of code, fore reference: https://github.com/asmallwebfirm/scale_addressfield/blob/7.x-1.x/scale_a...

It feels very wrong/dirty, but it might be the only way to support dynamic field #type swapping via AJAX

If you're not comfortable with committing something that takes that approach, I think I'd be alright with option #1 you mentioned in comment #9.

rszrama’s picture

I'll have to think about it more, maybe bounce ideas off Damien Tournoud. It's still unfortunate that there's no way to generate a new build ID for a form even when served from cache. I'm not sure how exactly Drupal could do that, but it at least smells like a core bug here for it to allow a user to associate a cached form state matching a cached form build ID. #ajax requests refresh any element on the page, so why not issue the form a new build ID if we detect it was served from the cache, for example?

rszrama’s picture

Status: Active » Closed (duplicate)

Sorry I haven't updated this issue in so long, but I've been involved in solving the core security issue this uncovered for me that was fixed as part of today's security release. Updating to Drupal 7.27 will resolve this issue. : )

See https://drupal.org/SA-CORE-2014-002 for more information.

iamEAP’s picture

Thanks again, Ryan. Just "verifying" that Drupal 7.27. resolves this issue.