Hi, when using Views exposed filters with multiple values per filter the $_GET variables are submitted as an array, like ?var[]=value1&var[]=value2.
This results in $GET['var'] to be an array.
The stored static cache item contains "Array" in the filename, and is never served.

Real example:
?maat[]=219&maat[]=220
results in:
_maat=Array.html

CommentFileSizeAuthor
#10 boost-637212.patch4.03 KBmikeytown2

Comments

mikeytown2’s picture

Quick fix is to turn off "Only allow ASCII characters in path". I'll be looking for a proper fix soon.

mikeytown2’s picture

above comment is false. Root of the issue has to do with multi dimensional arrays in the $GET php variable. Here is the problem code

  foreach ($_GET as $key => $val) {
    if ($key != 'q' && $key != 'destination') {
      if (!empty($val)) {
        $GLOBALS['_boost_query'] .= (($GLOBALS['_boost_query'] == BOOST_CHAR) ? '' : '&') . $key . '=' . $val;
      }

$GET structure for specials?page=1&field_type_value[0]=3 & specials?page=1&field_type_value[]=3

	Array

      [q] => specials
      [page] => 1
      [field_type_value] => Array

            [0] => 3

I'll be searching for code that reverses this; reproduce the URL given the $GET variable. I would rather not use $_SERVER['REQUEST_URI']; but I might have to. Looks like I need to have 2 sets of URLs as well, [0] & []

mikeytown2’s picture

http://php.net/http-build-query PHP 5 Function with it's PHP 4 backwards compatibility function

// php.net/http-build-query#90438
if (!function_exists('http_build_query')) {
  function http_build_query($data, $prefix='', $sep='', $key='') {
    $ret = array();
    foreach ((array)$data as $k => $v) {
      if (is_int($k) && $prefix != null) {
        $k = urlencode($prefix . $k);
      }
      if ((!empty($key)) || ($key === 0)) {
        $k = $key.'['.urlencode($k).']';
      }
      if (is_array($v) || is_object($v)) {
        array_push($ret, http_build_query($v, '', $sep, $k));
      }
      else {
        array_push($ret, $k.'='.urlencode($v));
      }
    }
    if (empty($sep)) {
      $sep = ini_get('arg_separator.output');
    }
    return implode($sep, $ret);
  }
}

From this I need to do a http://php.net/urldecode on the string; and then deal with [] and [0]; [] [] and [0] [1]

mikeytown2’s picture

Opened up an issue in views for the duplicate URL issue
#637900: More consistent looking URL's when submitting exposed filter

keesje’s picture

Thanks for diving in to this so fast.
This is not gonna be fixed in Views, these URL types are common practice.

Proposed solution:
Is there a way to replace these [] chars with e.g. _?
This can be done for the static cache filename easy, but can this be done in .htaccess? (not my cup of tea). I think it is, because ? chars seem to be replaced this way already.

Example:
?maat[]=219&maat[]=220
results in:
_maat__=219&maat__=220

(don't know about & char for sure)

mikeytown2’s picture

[] are valid character names so why would I replace them with _? That would create a huge headache in htaccess. Long story short, I need to cache these 2 different files, one with [] the other with [0]; which brings the total number of files created with one action up to a potential 8 files... kinda nuts but necessary. This will take some thinking on how to do this correctly. I've started to implement the framework for this since boost can create up to 4 files for each cache action already, using this function

/**
 * Returns all possible filenames given the input and current settings
 *
 * @param $filename
 *   Name of file
 * @param $base_dir
 *   Value from base_dir column in database
 * @return
 *   returns a 2 dimensional array
 *    1st dimension key is either gzip or normal
 *    2nd dimension contains all the filenames
 */
function boost_get_all_filenames($filename, $base_dir = NULL) {
  $names = array();
  $filenames = array();
  $names[] = $filename;
  $paths = explode('/', $filename);
  $end = array_pop($paths);
  $base_dir = is_null($base_dir) ? BOOST_FILE_PATH : $base_dir;

  // Generate urlencoded filename, if name contains decoded characters
  $chars = '*:<>|?#,';
  if (preg_match("/[" . $chars . "]/", $end)) {
    $end = urlencode($end);
    // '=' sign should not be encoded
    $end = str_replace('%3D', '=', $end);
    // '&' should not be encoded
    $end = str_replace('%26', '&', $end);
    // '#' should not be encoded
    $end = str_replace('%23', '#', $end);
    // '?' is not an issue because the first ? is converted to a _ with the default settings.
    // If not using the default then this could be an issue. Will deal with it when that happens.
    $paths[] = $end;
    $names[] = implode('/', $paths);
  }

  // Generate gzip filenames
  foreach ($names as $name) {
    $filenames['normal'][] = $name;
    if (BOOST_GZIP) {
      // Replace the correct dir with the gzip version for the given base dir.
      $gzip_base_path = implode('/', array_filter(explode('/', str_replace(BOOST_ROOT_CACHE_DIR . '/' . BOOST_NORMAL_DIR, BOOST_ROOT_CACHE_DIR . '/' . BOOST_GZIP_DIR . '/', $base_dir))));
      $filenames['gzip'][] = str_replace($base_dir, $gzip_base_path, $name) . BOOST_GZIP_EXTENSION;
    }
  }
  return $filenames;
}
mikeytown2’s picture

I should always have the number in the brackets [0] and then use the boost_get_all_filenames() function to strip all numbers out of []. This means I can not use $_SERVER['REQUEST_URI'] and must use something like http_build_query()

keesje’s picture

You are right, replacing doesn't solve a thing, strange thinking.
Isn't it possible to just explode the array (if any) for creating the filename to replicate the URL .htaccess looks for? I understand you cant ever figure out if the array keys where in the URL or not once you read the $_GET variable. Can't this information be taken straight from the URL? Like if(stristr($url, '[]')){$containskeys=FALSE;}else{$containskeys=TRUE;}
Maybe this is too simple.

mikeytown2’s picture

Plan:
Copy $GET to $query
remove q & destination from $query
run $query through http_build_query()
run that through urldecode()
this is the filename I save in the database
boost_get_all_filenames() will remove all [n] and replace with [] as a variant of the filename - will probably have to use regex to do this.

http_build_query() explodes the array for me, using built in php functions rather then my own.

mikeytown2’s picture

Status: Active » Needs review
StatusFileSize
new4.03 KB
mikeytown2’s picture

Status: Needs review » Fixed

committed

keesje’s picture

Fix confirmed as working.
Thanks a lot for this amazing fast action!

keesje’s picture

Just want to add a note:
When using exposed views this way, combined with pagination,this might result in duplicate caching for the first pagination page. This is because the resulting URLs from filter form submission does not contain numeric keys, the pagination URLs do.
This might be browser related too, although I did not encounter, there might be (exotic) browsers that insert these keys when building this type of URL.

mikeytown2’s picture

@kees
I already took care of that, each page that contains a [0] will also get a page that has [], at the same time. One action will generate 2 files. As merlinofchaos stated, it's a browser issue.

keesje’s picture

I saw that, great.
Thanks again.

keesje’s picture

Maybe for another FR issue, I have a case where the duplicate filenames would be unnecessary, because the filter is rendered as HTML links, never containing the browser specific keyless [] arrays.
Is it an option to prevent Boost from saving this file twice?

mikeytown2’s picture

Is there a reason why saving the link twice would cause any harm?

keesje’s picture

The only reason would be performance. Since filters can produce a lot of URLs (filter argument combinations), this can result in a lot of files. So, if this can be reduced with 50% this might be a slight performance enhancement when generating static cache.
(I have no clue how much effort it takes to write 2 files or 1 to the filesystem, so this is based on gutfeeling, not knowledge.)

mikeytown2’s picture

if "Asynchronous Operation: output HTML, close connection, then store static file" is enabled then the performance impact should be minimal. My guess is that the worse case, is an extra 20ms added due to this extra file being written; something I'm not too worried about since its a one time occurrence; a cache miss is generally more expensive then writing an extra file.

keesje’s picture

Thanks for explaining, its not worth the effort then.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.