I've got content that has it's created date set to 1st July 2011, but when faceting search results it appears under the 2011 > June. See screenshot

Comments

acbramley’s picture

Modules used:
Facetapi 7.x-1.0-rc4
Apachesolr Integration 7.x-1.0-rc2

cpliakas’s picture

Status: Active » Postponed (maintainer needs more info)

What implementing module are you using? Facet API works with the data that is passed to it, so often times this happens out of Facet API itself. We have had issues in the past where Facet API's "gap" calculation caused issues, so we will need some help debugging and tracking this down to resolve it in the appropriate place.

Thanks,
Chris

acbramley’s picture

Apachesolr Search is what I'm using for implementing search (if that's what you mean?), I can do some debugging if you let me know some useful places to do so

Cheers

cpliakas’s picture

Status: Postponed (maintainer needs more info) » Active

acbramley,

Yes, that is what I mean, and thanks for pointing it out. This happened in the past and may be related to #1302844: Inconsistent behavior of data facets producing unlimited filtering, inconsistent counts, or gap mismatches. What would be really helpful is getting the output of var_export($raw_data, TRUE); from line 148 of Apache Solr Search Integration's date query type plugin. This is the raw data returned by the Apache Solr server, which will allow us to plug in locally for testing. Then we can start to narrow down where the issue is. The date stuff tends to get pretty tricky, so there might have to be a few back-and-forths here before the underlying issue is discovered :-/.

Thanks,
Chris

acbramley’s picture

Yeah I remember this issue, as you can see I was commenting on it as well but this problem seemed to get lost in other things.

Here's the output that you requested:

array(4) { ["2011-06-30T00:00:00Z"]=> int(2) ["gap"]=> string(5) "+1DAY" ["start"]=> string(20) "2011-06-01T00:00:00Z" ["end"]=> string(20) "2011-07-02T00:00:00Z" }
acbramley’s picture

Has there been any progress on this?

cpliakas’s picture

Nope, not yet. In between switching jobs and hanging out with my 7 month old, it is going to take some time :-) There are also some other items that are more critical to get to since a lot of people who are running Facet API aren't seeing this issue (please prove me wring by posting!!). If this falls squarely on me then there is no timeframe on getting to it, although it is definitely on my radar. If the community wants to get involved, I will give as much guidance and support as necessary to give people the tools to help out.

acbramley’s picture

@cpliakas understandable, if you give me a bit of guidance as to what to test/debug to figure this out then I will be more than happy to do so!

Cheers,
Adam

cpliakas’s picture

Status: Active » Postponed (maintainer needs more info)

I can't for the life of me replicate this bug. I am using Apache Solr Search Integration 7.x-1.1 w/ Solr 3.5 and Facet API 7.x-1.2. I explicitly created a node and modified the "Authored on" date to be 2012-12-01 00:00:00 -0400, and it correctly shows under October.

I am think that this has something to do with timezones, but it is just a guess. I cannot proceed on this issue until I have detailed steps to reproduce using the latest version of the various modules.

acbramley’s picture

Status: Postponed (maintainer needs more info) » Active

After you mentioning it, and speaking with some other devs, it's almost certainly a timezone thing. I've done some more testing on a new site and I managed to see that the filter was 30th October 2012 9:31 PM where the created date was actually 31st October 2012 10:31am, that's exactly 13 hours which is the timezone difference with day light savings. Would you mind setting your timezone to Pacific/Auckland +1300 and testing?

I'm running Apachesolr Integration 7.x-1.0+2-dev, and FacetAPI 1.2, Core 7.16, and I believe Solr 3.6.

cpliakas’s picture

acbramley,

Thanks for posting back! That makes some more sense as Apache Solr Search Integration has a known bug. Let's play around with posting content in different timezones and see if that effects the problem. I'll do some testing today and report back.

Chris

cpliakas’s picture

StatusFileSize
new40.09 KB
new39.5 KB
new40.1 KB

Replicated!

So, confirming that is a timezone thing. I am also pretty sure that this is not an issue with Facet API itself and more with the backend, however I have no problem continuing to debug here. A couple of observation and possible new bugs are illustrated below. All content is posted at 12AM on April 1st, but under different timezones.

  • User's timezone is UTC, content authored on 2012-04-01 00:00:00 +0000
  • Everything works as expected.

1667866-12-UTC.png

  • User's timezone is America/New York, content authored on 2012-04-01 00:00:00 -0400
  • There is some weirdness here. The month is correct, however there is clearly a bug in that multiple times are posted. Another interesting thing is that the times are displayed in UTC, i.e. 4AM as opposed to 12AM.

1667866-12-NY.png

  • User's timezone is Europe/Moscow, content authored on 2012-04-01 00:00:00 +0400
  • Bug is replicated! Content shows up in the previous month, because as noted above, it is treated as UTC. So in the user's timezone it is 12AM on April 1st, but as far as Solr is concerned the date is treated as 8PM on March 31st.

1667866-12-Moscow.png

cpliakas’s picture

cpliakas’s picture

Regarding the UTC stuff, I think it requires a (excuse the cheese) multi-faceted approach.

First, Apache Solr Search Integration should probably detect the user's timezone and pad the dates accordingly. For example, increment the bounding times 4 hours if the user is on east coast time. There could be some issues with this, though. One is that in order to make use of the filter cache, generally you have to pad the dates down to the nearest gap. For a detailed explanation, please see http://searchhub.org/dev/2012/02/23/date-math-now-and-filter-queries. There are also daylight savings issues which I don't even begin to comprehend. To be honest, all this stuff makes my head hurt.

Second, Facet API's facetapi_map_date() function isn't timezone friendly at all. It is pluggable so that you can modify the display using the technique illustrated at #1392676: Make date format more easily customizable. Sorry for the suck, but it can be done.

Finally, Solr itself it trying to fix this issue. See the JIRA issue posted against Solr at Date Math should allow clients to override timezone used for rounding (faceting & queries).

So, I think the problem has been identified, but I would like to loop in the Solr experts to see if a solution is possible. Would love to hear ideas and solutions from people in the community who have solved (or attempted to solve) this issue.

Thanks,
Chris

acbramley’s picture

Thanks for the replies guys, I'm glad other people can replicate this finally!

smithmilner’s picture

Hi, I'm currently experiencing this same bug but at the day level. I think that we're also experiencing a gap calculation issue as well. I'll try to explain this as best as I can.

We're only using timezones at the site level, there is no user specific timezone. Our node type has a date field that with a time value that defaults at 17:00. Our site timezone is -0700 so it's actually saved UTC at midnight of the next day. This means if I create 2 nodes one is on the 4th at 5PM and the other on the 5th at 5PM. It's actually saved for the 5th at 00:00:00 and the 6th at 00:00:00. So my available date facets actually show for the 5th and 6th (I've corrected this issue using your workaround for the facetapi_map_date() callback but we'll ignore that for clarity). If I select the 5th as my current facet though both items show in the results.

I noticed that the facet in the url is displayed like "f[0]=my_date_field:[2012-12-05T00:00:00Z TO 2012-12-06T00:00:00Z]". I would expect it to show "f[0]=my_date_field:[2012-12-05T00:00:00Z TO 2012-12-05T11:59:59Z]" but I'm not show about the inner workings here. Is this the reason I'm getting items from the 5th and the 6th in my results? Another note, If I change the time value of the later node to ANY time before 5PM it still shows in the wrong results. This makes me think that the timezone issue is not at fault for that.

I'm using the search_api backend with Solr. Thanks!

cpliakas’s picture

Hi smithmilner.

Have you tried applying the technique in #1649158-10: Date Facets (without a time) can show in Multiple Months to Search API? I would first check to see if adding that param fixes the issue.

Thanks,
Chris

smithmilner’s picture

Thanks for the quick reply Chris.

I upgraded my solr to 3.6.1 and tried adding that param in with hook_search_api_solr_query_alter() but I didn't see any change. I'm not sure using that hook is the right way to do this though.

Thanks

cpliakas’s picture

Version: 7.x-2.x-dev » 7.x-1.x-dev
Priority: Normal » Major

Bumping back to the 7.x-1.x version so this stays on my radar.

acbramley’s picture

Can confirm the bug still exists on Facetapi 1.3 and apachesolr 1.3

cpliakas’s picture

Status: Active » Closed (duplicate)

Confirmed. At this point we are waiting on Apache Solr Search Integration. Closing as a dup of #1649158: Date Facets (without a time) can show in Multiple Months.