Ron, all:

Is there a way to fix the following with some alter table statements? I didn't do a backup before upgrading the bibliography module. If I roll-back to my backup, I'll have to redo an upgrade from drupal 6.4 to 6.9 *and* all my module upgrades.

Thanks for any quick help as my site has to be offline till this is fixed...

Drupal database update

* user warning: Unknown column 'bct.biblio_type' in 'on clause' query: UPDATE biblio b INNER JOIN biblio_contributor bc on b.vid = bc.vid LEFT JOIN biblio_contributor_type bct on b.biblio_type = bct.biblio_type and bct.auth_type = bc.auth_type SET bc.auth_category=bct.auth_catagory in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1503.
* user warning: Unknown column 'auth_catagory' in 'biblio_contributor_type' query: ALTER TABLE biblio_contributor_type CHANGE COLUMN auth_catagory auth_category INTEGER UNSIGNED NOT NULL DEFAULT 0, DROP PRIMARY KEY, ADD PRIMARY KEY USING BTREE(auth_category, biblio_type, auth_type) in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1506.
* user warning: Unknown column 'auth_type' in 'field list' query: INSERT INTO biblio_field_type_data SELECT auth_type AS ftdid, CONCAT(TRIM(ctd.title),'s') AS title, NULL AS hint FROM biblio_contributor_type_data ctd WHERE auth_type>1 AND auth_type<6 in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1537.
* user warning: Unknown column 'auth_type' in 'field list' query: INSERT INTO biblio_contributor_type SELECT auth_type,0,auth_type FROM biblio_contributor_type_data ctd WHERE auth_type<10 in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1550.

Updates were attempted. If you see no failures below, you may proceed happily to the administration pages. Otherwise, you may need to update your database manually. All errors have been logged.

* Main page
* Administration pages

The following queries were executed
biblio module
Update #6011

* CREATE TABLE {biblio_import_cache} ( `id` INT unsigned NOT NULL auto_increment, `session_id` VARCHAR(45) NOT NULL, `data` LONGBLOB DEFAULT NULL, PRIMARY KEY (id) ) /*!40100 DEFAULT CHARACTER SET UTF8 */

Update #6013

* ALTER TABLE {biblio_contributor} ADD `auth_category` INT unsigned NOT NULL DEFAULT 1
* Failed: UPDATE {biblio} b INNER JOIN {biblio_contributor} bc on b.vid = bc.vid LEFT JOIN {biblio_contributor_type} bct on b.biblio_type = bct.biblio_type and bct.auth_type = bc.auth_type SET bc.auth_category=bct.auth_catagory
* Failed: ALTER TABLE {biblio_contributor_type} CHANGE COLUMN auth_catagory auth_category INTEGER UNSIGNED NOT NULL DEFAULT 0, DROP PRIMARY KEY, ADD PRIMARY KEY USING BTREE(auth_category, biblio_type, auth_type)
* UPDATE {biblio_fields} SET maxsize=20 WHERE name='biblio_year'

Update #6014

* UPDATE {biblio_fields} f SET fid=fid + 4 WHERE fid>1 ORDER BY fid DESC
* INSERT INTO {biblio_fields} VALUES ('2', 'biblio_secondary_authors', 'contrib_widget', '60', '255'), ('3', 'biblio_tertiary_authors', 'contrib_widget', '60', '255'), ('4', 'biblio_subsidiary_authors', 'contrib_widget', '60', '255'), ('5', 'biblio_corp_authors', 'contrib_widget', '60', '255')
* UPDATE {biblio_field_type_data} f SET ftdid=ftdid + 4 WHERE ftdid>1 AND ftdid<100 ORDER BY ftdid DESC
* Failed: INSERT INTO {biblio_field_type_data} SELECT auth_type AS ftdid, CONCAT(TRIM(ctd.title),'s') AS title, NULL AS hint FROM biblio_contributor_type_data ctd WHERE auth_type>1 AND auth_type<6
* UPDATE {biblio_field_type} ft SET fid=fid+4 WHERE fid>1 ORDER BY fid DESC
* UPDATE {biblio_field_type} ft SET ftdid=ftdid+4 WHERE ftdid>1 AND ftdid<100
* UPDATE {biblio_field_type} ft SET cust_tdid=cust_tdid+4 WHERE cust_tdid>1 AND ftdid<100
* INSERT INTO {biblio_field_type} SELECT tid,ftd.ftdid,ftd.ftdid,ftd.ftdid,1,1,1,2 AS weight,1 FROM {biblio_field_type} ft, {biblio_field_type_data} ftd WHERE ft.fid=1 AND ftd.ftdid > 1 AND ftd.ftdid < 6
* Failed: INSERT INTO {biblio_contributor_type} SELECT auth_type,0,auth_type FROM {biblio_contributor_type_data} ctd WHERE auth_type<10

Update #6015

* ALTER TABLE {biblio_keyword_data} MODIFY COLUMN kid INTEGER UNSIGNED NOT NULL AUTO_INCREMENT
* Copied 481 keywords to the new database stucture from taxonomy
* Moved 393 keywords to the new database stucture
* ALTER TABLE {biblio} DROP biblio_keywords

CommentFileSizeAuthor
#1 biblio.install.beta8-rc2.txt64.77 KBrjerome
#22 biblio.install.beta8-rc2.txt64.77 KBcatdevrandom

Comments

rjerome’s picture

rjerome commented
Title: Fixed- I Think (Was Many Errors with upgrade from beta8 to RC4) » HELP! Many Errors with upgrade from beta8 to RC4
Status: Needs review » Active
StatusFileSize
new biblio.install.beta8-rc2.txt64.77 KB

Hi,

First, let me apologize, I thought I had put a warning in the RC release notes, however I see it's only in the -dev version. I hate to tell you this, but the simplest way out of this it probably to roll back your db and start again. There are other ways, but it would probably take longer. If you don't want to do that, contact me via my contact page and we can work something out, but if your willing to roll back your db, the following will work...

1) Restore your database to the version prior to biblio update
2) Install biblo version 6.x-1.0-rc2
3) Overwrite the biblio.install file from rc2 with the file attached to comment #22 (the file attached to this comment has an error)
(renaming it to biblio.install)
4) Run update.php
5) Install biblio version 6.x-1.0-rc4
6) Run update.php

Ron.

unarsu’s picture

unarsu commented

Ron,

First of all, no need to apologize; I really should have backed up. To do step one, namely, restoring my database to the version prior to biblio update, does that mean that I have to uninstall drupal 6.9, and reinstall drupal 6.4, as well as all of the old modules? (I'm not even sure what the old versions were, but I can probably determine that with some digging).

Or can I just follow your plan without worrying about 6.4 and the old modules?

Thanks for your previous prompt response.

-u

rjerome’s picture

rjerome commented

No, you don't have to touch any of the PHP files EXCEPT for the biblio module.

All you have to do is restore the database, delete any existing biblio files and directories, install this one... RC2, replace the install file in RC2 with the one attached to this issue and then run update.php. Then install RC4 and repeat the update.php process.

unarsu’s picture

unarsu commented

Oh, that's good. One question: you say "delete any existing biblio files and directories." It's been a while since I have used the bibliography module. To what files do you refer? And where (what directory) would I look to see if I have any of them?

Thanks, again.

rjerome’s picture

rjerome commented

I mean the module files, usually stored under the drupal directory in "sites\all\modules" you should find a directory called "biblio", however it could be just about anywhere under the drupal directory tree. As long as you find and replace your existing biblio directory with the RC2 version you should be good to go.

Ron.

unarsu’s picture

unarsu commented

Ahh. I had assumed they would be replaced when doing the rc2 and rc4 updates that's why your comment confused me. All set. Thanks for responding promptly. I'll close this ticket as soon as I get the procedure finished.

unarsu’s picture

unarsu commented

Well, the procedure did not work. I get the following error:

An error occurred. .../update.php?id=18&op=do
Fatal error: Call to undefined function _system_update_utf8() in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1220

I'm going to do a diff of the rc2 biblio.install and the one you sent. Maybe I can spot what's wrong...

unarsu’s picture

unarsu commented

It appears that the rc4 biblio.install file has removed updates 1-20 with this comment:

/*
* Removed updates 1 - 20 since they dated back to ver. 5-1.2
*/

The errant utf8 call was used in Drupal 5.x. I am going to remove the 1-20 updates in the file you sent and see what happens (after I diff the files).

unarsu’s picture

unarsu commented
Title: HELP! Many Errors with upgrade from beta8 to RC4 » Still Fails! Many Errors with upgrade from beta8 to RC4

OK. I was able to get rc2 to go without any errors. But upgrading from rc2 to rc4 yielded errors.

Any ideas?

Drupal database update

* user warning: Unknown column 'auth_type' in 'field list' query: INSERT INTO biblio_field_type_data SELECT auth_type AS ftdid, CONCAT(TRIM(ctd.title),'s') AS title, NULL AS hint FROM biblio_contributor_type_data ctd WHERE auth_type>1 AND auth_type<6 in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1537.
* user warning: Unknown column 'auth_type' in 'field list' query: INSERT INTO biblio_contributor_type SELECT auth_type,0,auth_type FROM biblio_contributor_type_data ctd WHERE auth_type<10 in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1550.
* user warning: Unknown column 'biblio_keywords' in 'field list' query: SELECT nid,vid,biblio_keywords FROM biblio where biblio_keywords != "" in /home/umbccast/public_html/sites/all/modules/biblio/biblio.install on line 1577.
* user warning: Can't DROP 'biblio_keywords'; check that column/key exists query: ALTER TABLE biblio DROP biblio_keywords in /home/umbccast/public_html/includes/database.mysql-common.inc on line 322.

Updates were attempted. If you see no failures below, you may proceed happily to the administration pages. Otherwise, you may need to update your database manually. All errors have been logged.

* Main page
* Administration pages

The following queries were executed
biblio module
Update #6013

* UPDATE {biblio_fields} SET maxsize=20 WHERE name='biblio_year'

Update #6014

* UPDATE {biblio_fields} f SET fid=fid + 4 WHERE fid>1 ORDER BY fid DESC
* INSERT INTO {biblio_fields} VALUES ('2', 'biblio_secondary_authors', 'contrib_widget', '60', '255'), ('3', 'biblio_tertiary_authors', 'contrib_widget', '60', '255'), ('4', 'biblio_subsidiary_authors', 'contrib_widget', '60', '255'), ('5', 'biblio_corp_authors', 'contrib_widget', '60', '255')
* UPDATE {biblio_field_type_data} f SET ftdid=ftdid + 4 WHERE ftdid>1 AND ftdid<100 ORDER BY ftdid DESC
* Failed: INSERT INTO {biblio_field_type_data} SELECT auth_type AS ftdid, CONCAT(TRIM(ctd.title),'s') AS title, NULL AS hint FROM biblio_contributor_type_data ctd WHERE auth_type>1 AND auth_type<6
* UPDATE {biblio_field_type} ft SET fid=fid+4 WHERE fid>1 ORDER BY fid DESC
* UPDATE {biblio_field_type} ft SET ftdid=ftdid+4 WHERE ftdid>1 AND ftdid<100
* UPDATE {biblio_field_type} ft SET cust_tdid=cust_tdid+4 WHERE cust_tdid>1 AND ftdid<100
* INSERT INTO {biblio_field_type} SELECT tid,ftd.ftdid,ftd.ftdid,ftd.ftdid,1,1,1,2 AS weight,1 FROM {biblio_field_type} ft, {biblio_field_type_data} ftd WHERE ft.fid=1 AND ftd.ftdid > 1 AND ftd.ftdid < 6
* Failed: INSERT INTO {biblio_contributor_type} SELECT auth_type,0,auth_type FROM {biblio_contributor_type_data} ctd WHERE auth_type<10

Update #6015

* ALTER TABLE {biblio_keyword_data} MODIFY COLUMN kid INTEGER UNSIGNED NOT NULL AUTO_INCREMENT
* Copied 481 keywords to the new database stucture from taxonomy
* Moved 0 keywords to the new database stucture
* Failed: ALTER TABLE {biblio} DROP biblio_keywords

catdevrandom’s picture

catdevrandom commented

Hello!

I don't know if this would be feasible for you, but what you might try is to export your bibliography entries to, say, Bibtex, and then completely uninstall the Biblio Module and install a fresh version of it, and then import the entries. If you decide to do so, it might be a good idea to test beforehand if the Bibtex file is exported correctly, and, of course, make a backup of the database.

I found that using the importing method and just doing a clean install of the updates was the easiest way to go for me, although so far I have only been messing with a testing version of my website, and the production website is still running beta7...

Good luck!
Maira

rjerome’s picture

rjerome commented

The warning you are seeing may not be harmful, the question now is does your installation work?

unarsu’s picture

unarsu commented

Maira: Thanks for the idea. I may end up trying that. I'm loathe to go down that path because getting the data into biblio required a lot of massaging of the input data, with scripts and whatnot. I'd like to hope that exporting and importing will be painless, but at the very least it will require a database restore, and a restore of beta8.

Ron: It seemed okay when it first failed, going from beta8 directly to rc4. Is there any particular test cases that stress the changes made between beta8 and rc4? That might make determining whether I have an error or not easier.

Thanks again.

rjerome’s picture

rjerome commented

Assuming your publications appear correct in either citation of tabular format, and you can edit them, you should be OK. The main things that may be lost are any customizations you may have made to the fields on the input form (admin/settings/biblio/fields).

unarsu’s picture

unarsu commented

No; I think the entries aren't in good shape. I'm getting things like:

warning: preg_match() expects parameter 2 to be string, array given in /home/umbccast/public_html/includes/bootstrap.inc on line 761.

and:

user warning: Unknown column 'b.biblio_keywords' in 'order clause' query: SELECT DISTINCT n.*, b.*, bt.name as biblio_type_name FROM node n left join biblio b on n.vid=b.vid inner join biblio_types bt on b.biblio_type=bt.tid WHERE n.type='biblio' ORDER BY SUBSTR(LTRIM(b.biblio_keywords),1,4) asc LIMIT 0, 25 in /home/umbccast/public_html/sites/all/modules/biblio/biblio.pages.inc on line 85.

I suppose I will have to give Maria's method a try.

unarsu’s picture

unarsu commented

Maira,

Is there a way of bulk exporting all of the entries to Bibtex?

Thanks.
-u

rjerome’s picture

rjerome commented

Your preg_match issue is unrelated and I just fixed it see... #371867: preg_match Error

The second one is also unrelated, I'm guessing you were trying to sort by keywords, which is also a known bug.

Ron.

unarsu’s picture

unarsu commented
Title: Still Fails! Many Errors with upgrade from beta8 to RC4 » Fixed- I Think (Was Many Errors with upgrade from beta8 to RC4)
Status: Active » Fixed

OK, thanks. I guess I will put the site back online and see what happens. I have updated the status of this entry to fixed.

BTW, Ron (and please, please do not take this as a criticism: I am grateful for all the hard work you've put into the bibliography module), I'm curious if your development follows the test-driven development style (see: http://www.lullabot.com/articles/drupal-module-developer-guide-simpletest and http://groups.drupal.org/node/9412.

I ask, because now I don't really know if my database table entries are correct. But if there was a test case I could run that would tell me if my tables and entries are ok, then that would be really useful during the upgrades.

-u

rjerome’s picture

rjerome commented

I don't take it as a criticism, and I am very familiar with the automated testing being implemented in Drupal these days, (in fact I was just looking into again last night).

Do I believe biblio could benefit from it: YES

Have I done anything yet: NO :-( well not quite, I wrote one test. So I only have about 500 more to go :-()

The main issue is that, aside from some occasional help, biblio is pretty much a one man show and writing all those tests takes time. I estimate that it would take about the same amount of time to write the tests as it did to write the module itself. Granted it would probably save time in the end, but the initial investment must be made. If I can get this version stabilized, maybe I can focus on this, but there seems to be a never ending stream of support and feature requests that consume my time fairly well.

I'm not trying to make excuses, but you have to realize that the 6.x version is a MAJOR/COMPLETE rewrite of the 5.x version which has been dragging on for quite a long time, so there are bound to be more than the usual number of issues. With all the various permutations and combinations of settings/import/export formats, other modules etc, it does prove rather challenging to get it all working (a strong argument for automated testing, I know).

And finally, you must also be cognizant of the fact that you were using a BETA version, and it is called that for a reason (it's not finished yet), and you will also notice that I'm not calling the current release "1.0" yet. The RC means that it's a release candidate, which requires testing. Like it or not, there's no better way to test code than let a bunch of people use it, since they will almost always try to do something I may not have thought of.

Cheers,

Ron.

unarsu’s picture

unarsu commented

I understand completely!

BTW, please let me know if there's a way we users can help write some tests. Or failing that, help out with a paypal (or other contribution).

Thanks,
-u

rjerome’s picture

rjerome commented

If you would like to volunteer to write test cases that would fantastic and I would more than welcome the added help and support. Obviously you would need a development environment and a sound working knowledge of PHP, MySQL and Drupal internals. If you feel your up to the task, contact me via my contact page and we can get the ball rolling.

Cheers,

Ron.

catdevrandom’s picture

catdevrandom commented

@unarsu:

I think this is no longer relevant, but yes, there is a bulk export option inside the Biblio Module. If I am not mistaken, you need to add the permissions for that for the user, and then you can export the results via the links on top of the list of biblio entries. To import them back, you can use the Import option in the Biblio settings (/admin/settings/biblio/import).

:)

catdevrandom’s picture

catdevrandom commented
StatusFileSize
new biblio.install.beta8-rc2.txt64.77 KB

I tried to make the process of updating from beta8 to rc2 with the modified biblio.install file that you provided, Ron, and I think that there is a small mistake in the name of one of the columns. In the modified install file, in update 6010, there is a mention to column "ctid", but I believe that the correct name is ct_id. I have changed the file manually and managed to perform the update without problems.

Attached is the fixed version of the install file.

Cheers,
Maira

catdevrandom’s picture

catdevrandom commented
Status: Fixed » Needs review
rjerome’s picture

rjerome commented

Indeed that works better (I would have sworn I tested that without errors, but I've got so many versions floating around the possibility of error is quite high :-)

unarsu’s picture

unarsu commented
Title: HELP! Many Errors with upgrade from beta8 to RC4 » Fixed- I Think (Was Many Errors with upgrade from beta8 to RC4)
Status: Active » Needs review

Hi Maira!

Nice catch. I wish I had seen this last month.

Would you (or Ron) know if it's possible to execute this script "over" the previous one? Or should I just edit the attached script and run update 6010?

-u

rjerome’s picture

rjerome commented

What version are you currently using? All that's really going to do is change the name of one of the columns in the biblio_contributor table from ct_id to auth_type, so you might want to check that table first and see what the column names currently are.

srwestma’s picture

srwestma commented

I am just now upgrading to Drupal 6 and was not able to find an rc code. I therefore just used the biblio.install in posting #22 and it worked fine except for getting the error:

user warning: Unknown column 'ct_id' in 'biblio_contributor' query: ALTER TABLE biblio_contributor CHANGE COLUMN `ct_id` `auth_type` INTEGER UNSIGNED NOT NULL DEFAULT 1, DROP PRIMARY KEY, ADD PRIMARY KEY (`vid`, `cid`, `auth_type`, `rank`) in /export/scweb/lib/sites/default/modules/biblio/biblio.install on line 1756.

I looked at the biblio_contributor table after I ran update.php (it wasn't in the database before) and there is an auth_type column that is an unsigned integer, not null, and has a default of 1. There is also a primary key with the characteristics described above. So I am assuming that I can ignore that warning.

rjerome’s picture

rjerome commented

What version were you upgrading from?

bekasu’s picture

bekasu commented
Status: Needs review » Closed (fixed)

Marking this topic closed, its been 2 months since the last discussion point.

Please do not re-open this issue. It would be better to start a new one rather than hijack this issue again.

thanks,
bekasu