Upgrade to Date 1.7 from 1.6 caused Views to Malfunction

Haven't yet reproduced why this happened, but I *think* upgrading Date and/or Calendar caused this which is probably the same issue.

I never got the fatal error, but had the same "related problem" mentioned above. I think it's also causing folks to wonder what's going on here and here.

After rolling back to 1.6 the problem went away. All other modules on my site are up-to-date including Views, CCK, etc...

I suspect the Views cache is getting cleared at an inopportune time...

This explain why I was unable to reproduce this behaviour on a clean install. I agree that the problem is probably in the upgrade path itself.

It may be that simple, but I didn't try it since I had a very recent database backup.

One thing that could be an issue, but which I don't have time to test, is that I've got a Custom date format for one of my CCK fields. Since not too many people seem to be complaining, perhaps this could be something to look into.

Shai - do you have any custom date formats set?

Looks like your problem may be the same as mine.

http://drupal.org/node/193324

I'll have to do a bit of testing to try and confirm.

BTW I also get a problem on a VIEW(not all just one in a block with an ARG) that existed prior to the upgrade

On the existing VIEW I get the following error

[Sun Nov 18 00:27:56 2007] [error] [client 99.99.99.99] PHP Fatal error: Call to undefined function: () in /removed for security.../httpdocs/sites/default/modules/views/views.module on line 1581

In my case the problem was also the 1.7 version of DATE. Disabling or going back to 1.6 made the problem go away. I now have my fields back on the VIEWs argument option.

I originally went from 1.5 straight to 1.7 and then tried 1.6 . I tried looking at the install files to see what type of SQL changes were made but while I could understand the test in one version the other has some nested logic that thru me. The two versions used very diff methods to do their SQL checking. I got some error msgs and some other upgrade related msgs(pathauto + more I think) that should have come out earlier but didn't till I dropped back to 1.6 . I'm thinking that part of my msgs were caused because of going from 1.5 to 1.7 (But I haven't checked enough to confirm that)

But having said all that, everything seems to working so far. But I haven't done any heavy testing yet on date related content.

The main SQL changes I remember seeing in the notes was allowing null fields, but I'd have to check the install fills again to be sure. But that might have been related to coming from 1.5 and not 1.6

Its possible that part of the problem is in tyhe upgrade process itself. But I don't know enough to be sure.

I can also confirm this.

I recently upgraded both date and calendar and had to rollback.

I can not, however, notice which one has the bug.

I cannot replicate this problem at all. What version of Views are you using? There were some significant changes to Views so you may be using a version of the Date module that requires a more current version of Views.

Also confirm what version of CCK you are using. I have tried every combination of CCK, Views, and Date versions and cannot replicate the problem yet.

Also reported in http://drupal.org/node/193324, http://drupal.org/node/189955, http://drupal.org/node/190643.

Also reported at http://drupal.org/node/193286.

I've been playing around with this and am still unable to replicate this problem, but I did notice that modules were not always made available in the order I would have expected. So one possibility is that some of the functions are not available at the right time on systems reporting problems, even though they are on my system. I changed the way all these files are included in the very latest commits to the Date module to see if that would fix the problem for people who have seen it.

You'll have to get these changes from the cvs repository today, or wait until tomorrow to get them in the dev tarball.

Please let me know if that fixes things (or even changes anything).

I am running MySQL-4.1.15 PHP-4.4.2 CCK 5.x-1.6-1 VIEWS 5.x-1.6 all on Linux and apache2 In my case I had found the problem while bringing my system up-to-date, so everything drupal was current(except for DATE). as of Nov18.

I am the author of issue http://drupal.org/node/193324 which was under VIEWS since that was where I first saw the problem. But it did go away when when I dropped DATE to 1.6 instead of 1.7 I had been at release 1.5 prior to upgrading to 1.7 When I did the drop back(and ran update as I did after each module upgrade) I also got some msgs that were related to the upgrade of previously upgraded modules. Kind of like they had been stuck (?) I'd hjave to check to see if I still have them

I'm in the middle of some stuff so it will be at least a few days before I'll be able to recreate my testbed and try the updated module.

Although I *still* cannot replicate the problem, I've been tracking through all the code that creates and updates fields and arguments to find any place where data might get lost and did another re-work of the way files are included and functions are called to hopefully catch anything that might be a problem. Just committed a fix to Date and am going to do the same with Calendar shortly.

I ran the 5.x-dev version with the latest changes and I still have the problem. Rolling back to 1.6 fixes it. When I upgraded date (after upgrading to the latest version of CCK and Views) about half of my arguments disappeared from views.

Do you need it tested with you recent change of date_views_arguments() to _date_views_arguments()?

Yes, please test that. That is another idea of what might be causing the problem. Since I still can't replicate the problem, I can't test the fix.

I tested it yesterday on a 5.5 site with the latest released Views and CCK. It's still doing the same thing. I'm in the process of writing up some steps to attempt to reproduce the problem -- here's what I've got so far (although I haven't done this because I'm working on an existing site):

1. Install Drupal 5.5
2. Install the latest CCK and Views
3. Install Date 1.6 (DO NOT INSTALL 1.7 YET)
4. Create a CCK node type called Job
5. Create a CCK node type called Invoice with a date field, an amount field, and a nodereference (that can only reference Job nodes)
6. Create a View that lists all Invoices that nodereference a Job that's passed as an argument
7. Create some Jobs and Invoices
8. Test to make sure the View created in step 6 works
9. Upgrade Date to 1.7 (or the latest Dev snapshot)
10. Repeat step 8
11. Edit the View created in step 6 and see if your arguments are all valid

Step 6 needs more clarification, but someone who's well-versed in Views & Arguments should be able to take it from here.

I reproduced this on a fresh 5.5 install with some contrib modules. You can download the whole thing here and save a lot of time configuring things. Uncompress and untar the thing and you'll have a working 5.5 site with Date 1.6 and Calendar 1.6 installed.

In the databasedump directory there's a datebug.sql file that will populate your database with everything all configured. There are 3 Views: Clients, Invoices and Related_Jobs. There are 4 CCK node types defined: Client, Invoice, Job and Note.

User id and password are both datebug.

Log in and take a look at related/jobs -- you'll see 3 jobs listed.
Look at related/jobs/1 -- you'll see 2 jobs listed (jobs that nodereference Client 1).
Look at related/jobs/2 -- you'll see 1 job listed (jobs that nodereference Client 2).
Look at related/jobs/edit and notice the Arguments.

Now upgrade Date (and Calendar) to 1.7.

Repeat the "Look at" steps above. The view's argument is messed up and the views are broken -- the argument is not respected.

Thanks for providing this. I've downloaded your file and I'll play around with it. Unfortunately with all the holiday goings-on I won't be able to try it out for a few days, but I'll work on it as soon as I can. Since I *still* can't reproduce the bug on my own setup, maybe this will help :)

I've see this issue is reported in a lot of places. My issue haven't an exception in watchdog, just made the CKK fields as arguments to disappear, same screens as reported from McRonin in http://drupal.org/node/193324#comment-633619. This caused block views to ignore reference field arguments and showing all the recordset.
Also using date 1.6 but problem disappeared only after disabling audio module

@marcp, I downloaded your files and set up a new folder and database on my system for them. I checked that all the views worked correctly in the original download, and they did, I then updated Date and Calendar to the 1.7 tarballs, ran their updates, and checked their views and everything looks exactly the same after the updates as it did before.

After the update I can pull up the view and all the same arguments are shown in the drop-down list of arguments. The argument originally in the view is still there and everything else about the edit screen of the view looks exactly the same as it did before. When I view the view, it displays exactly the same nodes it did before the update.

I tested without emptying the Views cache and it worked fine, then I tried emptying the cache in case that made a difference, and it still worked fine. I tested the View before trying to edit it and it worked fine, I then pulled up and saved it in case editing it changed something, but it still worked fine.

The only thing I changed in your download was the database username and password and I disabled your clean urls, which weren't working right on my setup for some reason -- I couldn't even get to the admin screen until I disabled them. I use clean urls all the time, so it's not that my system can't handle them.

I'll do some investigation into why clean urls were a problem in case that has something to do with it. The only other possibilities are if you didn't run the update, which I assume you did but I need to confirm that, or some other environment difference.

I'm testing this in:

MYSQL 5.045
PHP 4.4.7
Apache 2.0.61
Windows XP

Hmmm.

We're running:

mysql 4.1.9, for pc-linux-gnu (i686)
apache 1.3.27
php 5.1.2
linux 2.4.9

@Shai - are you able to reproduce this?

I just reproduced this on a completely different Linux box running Fedora Core 4, PHP 4.3.11, MySQL 4.1.11. I didn't have to change anything regarding the clean urls -- it just seemed to work out of the box.

Attached is a zip with pictures of the Arguments section of related/jobs/edit before and after the upgrade. Also in the zip is an rtf with the results of running update.php in case that gives you some more info.

If you can't repro this on a *nix box, let me know and we'll figure out how to get you access to ours, if you want.

Thanks for sticking with this issue!

Marc

Aha! I kept playing around with different configurations and your download and *finally* found a setup where this error appears. I just got this far, so don't have an answer yet, but at least I am finally to a place where I can see what is going on.

Found it and just committed a fix to the dev version of the D5 branch. This was very elusive and odd, it turned out to be caused by the date module's use of a foreach on content_fields() inside a cck use of foreach on content_fields() that worked fine in some situations but not in others. In situations where it didn't work, it caused the CCK foreach to quit working as soon as the Date module finished adding its fields, which left out all fields that had not yet been processed.

Thanks for the database dump, I'm not sure how long it would have taken to find this without it :)

BTW, you'll need to pick up the latest dev version with the fix and also clear all your caches for this to work, since bad data would be stored in those caches.

KarenS- Thanks for all your work on this ...

@Shai - While I did put up some screen grabs and notes it was marcp who put together the downloadable test environ that helped KarenS reproduce the problem

While I was fixing this I saw that the argument handling was more complicated than necessary (making this harder to track down). That complicated handling is a throwback to the way CCK used to work and was never adapted after CCK changed. So I've reworked and simplified the argument handling in a way that should also keep this bug from re-appearing.

Just to be sure, though, can one or two of the people who had problems before please test the latest dev version? If it's working right, I'm going to create a new official release with these fixes since this was a critical bug to anyone who was affected by it.

Marking it needs review to get feedback.

I just got the latest 5.x Date code out of CVS and installed it along with Calendar 1.7 on top of that distro that I put up there for you. I ran update.php and got this the first time I went to related/jobs:

warning: Missing argument 1 for _date_views_arguments() in /home/drupal/html/cointoss/demo/htdocs/datebug/sites/all/modules/date/date_views.inc on line 226.

Didn't get any warnings upon going to related/jobs/1 or related/jobs/2, and got the correct results in all three places. Then I tried related/jobs again and it worked fine without any warnings.

Looked at admin/build/views/Related_Jobs/edit and the argument looks great. Maybe just missing a views_cache clearing on update?

That error was actually one place I missed making my change in the argument handling code. You only saw it once because it caches its result, but it is an error that needs to be fixed.

I just committed a fix for that.

Looking good so far. I'll wait a day or two to see if any errors get reported and if not, I'll create a new official version with these changes.

Thanks to everyone for the great group effort in getting this fixed!

I pulled the code from CVS after you updated it, installed it in 2 places, and everything seems to be fine.

Yup, just tested it again and it looks good!

One thing I'm a little concerned about is the initial thing that fixed the problem was pulling the call to content_fields() out of the foreach statement and assigning it to a local variable before looping over it. content_fields() winds up returning some static CCK data in _content_type_info(). Is this not just another accident waiting to happen? Are nested content_fields() calls inherently broken in certain situations (ie. if the underlying info is being modified)? Was it only broken in certain PHP versions?

The nested content_fields() calls worked fine in some situations and not in others. I never really pinned down for sure what was the combination that broke. I was using PHP4 when I finally got it to break, but I was also using PHP4 when it worked fine and at least one of the reports of places where it was broken was using PHP5. There's no logical reason why they wouldn't work, but apparently in some situations the original value ends up being affected by the value in the nested one. Very odd.

I think the lesson is that we should just not nest them, and there really is no reason to do so. The fact that I was nesting them here was just because my code was created when the content module handled arguments completely differently and I never went back and changed it.

I don't think there's anything else that we can do or change, either here or in the content module, we just have to watch out for nested foreach loops.

Also, the underlying data was *not* being modified, both loops were just reading in the field data, neither was altering it. That's what makes it so odd.

I just issued a new official release, Date 5.x-1.8, with these fixes.

The fix in comment #28 was originally suggested almost an entire month before (Nov 27) in http://drupal.org/node/193286#comment-641389.

@superkc9 so what is your point?

This issue was reported in dozens of places and each issue had lots of ideas, none of which got me to a point where I could reproduce the problem so I could tell if the fix was right. Your suggestion made no sense until I could see why it was not working right and no one else reported that your fix worked for them, so I couldn't just blindly commit it.

I took the time to carefully search for all related issues and posted on each of those issues to point to this one as the place where we would work on this problem. Here I found people who helped me get to the place where I could reproduce the problem and see what was happening and finally fix it. You could have come here and reported your fix to see if it worked for others or tried to help me get to the place where I could reproduce the problem myself. If you had done that, maybe it wouldn't have taken a month to get this fixed.

I spent many many hours trying to get this fixed, I don't think it was necessary for you to come here and post your criticism of the way I handled this issue.

KarenS,

I apologize for posting this here. I did not mean to discredit or criticize the hard work you did put in. I was just frustrated by the lack of feedback and made a mistake in posting my last comment. Please forgive me.

Thank you for all of the great contributions you have made to Drupal.

I guess we were both frustrated, no offense taken. At least it is now fixed :)

i think a similar issue is happenning in token module queue - http://drupal.org/node/149222