Posted by Dave Reid on April 30, 2009 at 8:05pm
| Project: | XML sitemap |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | task |
| Priority: | normal |
| Assigned: | Dave Reid |
| Status: | closed (fixed) |
Issue Summary
Creating a task so everyone can know when the new 6.x-2.x branch has been imported from my sandbox. Yay!
Comments
#1
Yay, very exciting. Thanks Dave!
#2
xmlsitemap.module, xmlsitemap_node.module, and xmlsitemap_menu.module have been added to CVS. Once the 6.x-2.x-dev release is repackaged, I'll show it on the project page. Working on getting the xmlsitemap_engines.module converted...
#3
The build has been officially packaged!
http://drupal.org/node/449710
#4
AWESOME! But no one is watching this thread! What's wrong with everyone!? A working version, a real, working version! I'm so excited. Great job Dave. We'll be working on deploying this next week, so get you patch applying hat on. ;-)
#5
@quicksketch: Thanks. :) I'll be looking forward to any followups.
A big note to anyone else reading this: Upgrading from any other version to 6.x-2.x-dev is NOT currently supported. To use this version you MUST disable and uninstall any current XML Sitemap modules.
#6
woot subscribe!
#7
Quick update: xmlsitemap_engines is almost ready and the test suite is making good progress as well.
#8
A quick update on the timeline for 6.x-2.x:
5/10-5/16: Add _engines. Polish and test _node and _menu.
5/17-5/23: Add _taxonomy and _user. Polish and test _engines.
5/24-5/30: Polish and write remaining tests. Add documentation (adv_help).
5/31: Beta release?
Also please read http://blog.davereid.net/content/state-of-drupal-xml-sitemap-2009 for ways to help support the rewrite.
#9
The _engines module has been added to CVS.
#10
Got a little behind schedule for _taxonomy and _user because of #472598: Architectual problem: Mass updating links. In the meantime as a bonus, I managed to get multilingual support added in #470602: Support for multilingual sites.
#11
#12
#13
This isn't a support request, it's a task. I'm the one that originally created the issue.
#14
Dave, thanks for your great work. 2.x is fast!
I hope it won't be impolite if I ask about taxonmy...?
... :/ :) :*
Szy.
#15
count me in for using 6.x-2.x-dev.. gime 5 minutes. just found this thread,
I've been running one from 6 months ago..lol
EDIT ...This works really nice...no fuss, no patching, no back dating cus it don't work..
Your a hero !!
thanks Dave
#16
Major update committed today. xmlsitemap_taxonomy has finally been included. Rejoice!
#17
Side note: for maximum xmlsitemap settings page enjoyment, download and install the Vertical tabs module.
#18
Great! subscribe.
#19
Subscribing. Nice work. Need this very soon for www.kpbj.com.
#20
Subscribing!
Thumbs up, thnx Dave!
#21
For those of you waiting to install this.. lol. here goes
Do it NOW !
It works very nice.... even has a style sheet option, neat
see one at http://buckeyelake.org/sitemap.xml
#22
Yes, but will it handle 6300 nodes?
#23
lol.. give me a little time and i will run a test on that.. should we up that number a few notches? -maybe so, we don't want any limits. 'scalable'. how many links do you want on each page. 1000 or less. be back later w/answer
#24
I've tested it with over 50,000 nodes, and 25,000 taxonomy terms and users. It takes a bit to help 'import' all the old content into the sitemap on cron like it's designed to do (just like the search.module). You can use admin/settings/xmlsitemap/rebuild with a large site to get all your content indexed right away, but it will take a while.
The 6.x-2.x version was written specifically to be stable and scale to large sites.
#25
yep, i did 9000 nodes, with comments and taxonomy.. here is what it spit out, one index page and 2 sitemap list pages.. i had it do 5000 at a cron
#26
Totally awesome. Works FAST and WELL. :-) Thank You!
#27
Yup. I tested this on a local machine that mirrors the server. More than 50,000 nodes and it works great with a small footprint.
I guess that I can finally replace the November 8 version that is currently installed.
#28
I could really use some input and feedback on the one last remaining architectural problem in the 6.x-2.x version: #512216: Last architectural problem: override a link's status in the sitemap (inclusion/exclusion).
#29
@Dave - Thank you for the excellent rewrite. As per our (obviously) brief twitter conversation, we're going to start include 2.x dev in our shop's base Drupal install (the basis for all production sites). I'll report back if there are any issues. Should I assume that once #512216: Last architectural problem: override a link's status in the sitemap (inclusion/exclusion) is closed a beta release will be rolled? Running "dev" code in production is always scary, no matter how stable and improved.
#30
I've started tagging the issues that I consider 6.x-2.0 alpha blockers:
http://drupal.org/project/issues/search/xmlsitemap?issue_tags=6.x-2.0-al...
#31
A kind of pre-alpha release has been made: 6.x-2.0-unstable1. For all intensive purposes, it's basically an alpha release, but needs some testing to make sure all the latest big changes work just fine. We are back on track to get an actual alpha out soon if we get lots of testers, and onwards toward a 6.x-2.0!
#32
I'm intending that someone on our team will get to it within a few days, then hopefully we'll have plenty of time to rock 'n roll :)
#33
I assume the newest dev version is ok for test run too.. it's in there.lol
#34
@dbeall: Yeah, it's basically the same code now that the nightly snapshot has been regenerated.
#35
Subscribe. Very excited about this - it comes just about the time when I need it.
#36
6.x-2.0-unstable2 is now available which fixes a few annoying, but minor bugs from unstable1.
http://drupal.org/node/554996
#37
I like a daily upgrade, shows rapid development progress..lol . I think Dave Reid deserves a vacation after this.
#38
I must admit... loading a version labeled "unstable" is a lot more scary than even an "alpha" version. Shouldn't this be an alpha? :-)
#39
I rather like this warning from a year ago.(can't remember where in Drupal it was.)
Totally a super alpha release that just barely works. It'll probably break your site, crash your server and burn down your house so
MAKE A BACK UP OF YOUR FILES DIRECTORY AND YOUR DATABASE!
edit: or maybe this one..lol
This is experimental software meant for advanced users; assume nothing works, and you may be pleasantly surprised. And when it breaks, throw the broken bits at the issue queue.
#40
@gregarios: I'm doing unstable releases right now since:
1. Following the same pattern as Drupal 7 since they're also creating unstable releases before we go into alpha where it's just bug fixes and working on translations only.
2. Not finished writing planned features for 6.x-2.0. But there is not much more left. Alpha is close.
I'd prefer to start getting people off using the 'dev' releases now that we have unstable releases because I can have control when the unstable releases are made. The 'dev' builds are just automatically packaged every 12 hours from CVS. I may be half-way through a feature and commit my progress to CVS, and it gets packaged while it doesn't work and incomplete. Plus this gives people using the update status core module an easy way to tell when there is an update. The only way to do it with the 6.x-2.x-dev version is if you use the cvs_deploy module.
Because of the way that 6.x-2.x is being written, its very stable already. Lots of people have been using 6.x-2.x-dev and liking it. Use whatever you're comfortable with.
#41
*good info on the auto packing of dev, didn't know that.
Every 6.x-2.x release I have installed to date has worked very well. If there was any issues, I didn't find them.
#42
Just tested unstable2. Seems to be working quite well so far. It's so great to finally have a working XML Sitemap module for Drupal!
I'll let know if I run into any issues.
#43
6.x-2.0-unstable3 will be available shortly and I'm proud to announce it has full-multilingual support and integrates very well with i18n. Voilà! Your French XML sitemap could only contain French content, etc.
#44
Only two remaining issues that are blocking the alpha:
http://drupal.org/project/issues/search/xmlsitemap?issue_tags=6.x-2.0-al...
#45
I wish there was a better way to subscribe...subscribing.
#46
subscribe
#47
subscribing. sorry ;)
#48
Allright, what's the best module to use for a new website right now?
- 1.1 ?
- 2.0 unstable 5 ?
- 2.x dev ?
There's no upgrade support from 1.1 to 2 right?
So I'd better start using 2 for a new site straight away?
And then of course: which 2 version?
#49
I'm using Unstable5 right now on a site with 8000+ nodes and it is working just fine. :)
#50
Hi Greg,
Thnx for the quick reply,
good to know about your experience with unstable 5.
I've checked the issue list for this one and it looks pretty ok:
- http://drupal.org/project/issues/xmlsitemap?text=&status=All&priorities=...
Just wondering: why is it called unstable?
Anyway, I'll give it a try.
Cheers mate,
Marc
#51
It's called unstable because:
1. It's not yet alpha because the APIs haven't been completely frozen yet.
2. We're limited to a few types of 'pre-release' names: unstable, alpha, beta, and rc.
3. The 6.x-2.x-dev build is re-packaged every 12 hours. So I could be in the middle of committing a huge change, gets repackaged, and when I'm ready to commit the next part, all of a sudden there is a broken release. Situation is not likely, but using an actual 'unstable' release gives me the opportunity to control *when* that release happens.
4. Plus using an actual release and not 6.x-2.x-dev makes it easier to keep track of versions with Update status module.
#52
Thnx, this makes things clear ;)
Keep up the good work Dave, cheers!
#53
I'm using 6.x-2.0-unstable5 on http://www.ferienwohnung.de/ with ~140.000 nodes and ~1200 taxonomy terms. Seems to work very great. THANK YOU VERY MUCH.
#54
Also see: #666382: XML sitemap Drupal 7 port. Am working to make sure I meet the D7CX pledge, but don't worry, my biggest priority for the module is still getting a 6.x-2.0 official version. Developing for both Drupal 7 and 6 will help each version out.
#55
I'm using 6.x-2.0-unstable5 on Healing Thresholds Autism Therapy, which has 45,000 nodes and 1,100 taxonomy terms. It's working great, while 6.x-1.x brought the server down. Thank you for this valuable module.
#56
Subscribing... I've been waiting for this to upgrade from D5 to D6! Great to hear that it seems to be quite close now.
#57
@altsal, waiting... hmmm. your not waiting on xmlsitemap are you. It's fine, I've been running 6.x-2.0-unstable5 for what seems like many months. I have it on 15 web sites so far.. yep, I'm a small time website guy, but wish I had a hundred. But then if I had a hundred to update I would want 15..lol
Go for it, unstable5 is just fine!
#58
yes sir, 6.x-2.0-unstable6 upgrades like a dream, lots of neat new stuff, and it's real nice looking now too, just for us Humans.. long list of fixes and improvements, you are a busy person..
The whole Drupal world thanks you...
Nice work Dave Reid
#59
Deja vu...subscribing.
#60
subscribing
#61
Also confirming that 2.x works like a charm for a while already. Looks + feels like 2.0 is in order. :)
#62
As much as I'd like to agree, there a few APIs that are about to have major changes and I need to get a lot of tests written. I am debating making an alpha release however.
#63
Do eeeet!
#64
I actually really, really want to get a simple 1.x -> 2.x upgrade path in place quick before alpha1. I'm hoping I can actually get that done tonight.
#65
One big thing slowing development has been fixed. We're now at 100% pass for tests on the D6 branch and 100% for enabled tests on D7.
#66
Nice job Dave. :-)
#67
It would be very useful for me for the one remaining task before 6.x-2.0-alpha1 is to get some copies of the data from XML sitemap 1.x installs. Basically, if people feel safe sending all SQL from any 'xmlsitemap%' tables as well as any records from {variables} that have
name LIKE 'xmlsitemap%'.#68
Dave,
What are the exact MySQL commands you wish to have the output from?
#69
I think this will get it all?
mysqldump --add-drop-table -u user -p databasename xmlsitemap > xmlsitemap1-data.sqlmysqldump --add-drop-table -u user -p databasename xmlsitemap_node >> xmlsitemap1-data.sql
mysqldump --add-drop-table -u user -p databasename xmlsitemap_term >> xmlsitemap1-data.sql
mysqldump --add-drop-table -u user -p databasename xmlsitemap_taxonomy >> xmlsitemap1-data.sql
mysqldump --add-drop-table -u user -p databasename xmlsitemap_user >> xmlsitemap1-data.sql
mysqldump --add-drop-table -u user -p databasename xmlsitemap_user_role >> xmlsitemap1-data.sql
mysqldump -u user -p databasename --where="name LIKE 'xmlsitemap%'" variable >> xmlsitemap1-data.sql
Ignore if you get table not found for some, it depends on which sub-modules are enabled.
Can mail results to dave AT davereid DOT net
#70
@davereid: please let me know if you got the file I emailed.
#71
@DamienMcKenna: Yep I got yours. Thank you very much!
#72
It is on the way...
#73
@gregarios: Got yours as well. Thank you!
#74
@gregarios: Your data seems to be from a 2.x install? Do you happen to have 1.x data?
#75
Uh... haven't used 1.x since it crashed my server 10 months ago. Isn't this post named "XML sitemap 6.x-2.x-dev progress"? I thought that was what you needed. Sorry... I didn't read carefully enough. :)
#76
@gregarios: Yeah, I'm working on an upgrade path from 1.x to 2.x so I can get an official alpha.
#77
Is it a matter of custom URL's in 1.x transferring to 2.x? The rest gets rebuilt, right?
#78
@gregarios: Its mostly a matter of saving links that have been adjusted and have custom settings, like a node which has been set to manually excluded, etc. If links are using their defaults, we don't have to migrate them since they can be re-imported on cron
Mostly its a fact of there is just a *lot* of different database structures and variables. I want to make sure I get this right and everyone can easily upgrade to 2.x.
#79
Updated plan: today release alpha1 without upgrade path from 1.x so we can get all the 2.x users on the correct schema versions. Alpha 2 will contain upgrade path for 1.x users. Any unstable users should upgrade to alpha1 before going to alpha2 since it may cause 1.x -> 2.x upgrades to run.
#80
Yay a hero!
#81
7.x-2.0-alpha1 and 6.x-2.0-alpha1 are out. Reminder there is not yet a 1.x->2.x upgrade path in place yet. This will happen on alpha2. Please update and test!
#82
admin/settings/xmlsitemap/custom/add
Fatal error: Call to undefined function xmlsitemap_get_priority_options() in /home/users/rad/blablabla.cz/web/sites/all/modules/xmlsitemap/xmlsitemap_custom/xmlsitemap_custom.admin.inc on line 98
#83
Please do not put bug reports, etc in this issue queue. Please file a new issue.
#84
admin/reports/status/run-cron
Fatal error: Call to undefined function xmlsitemap_var() in /home/users/rad/znojemskezeste.cz/web/sites/all/modules/xmlsitemap/xmlsitemap_menu/xmlsitemap_menu.module on line 43
#85
Sorry
#86
#87
Upgraded to 6.x-2.0-alpha1 with 9000+ nodes... seems to work great! Nice Job. :-)
#88
Anyone following along, is it reasonable for the upgrade path to only worry about nodes or taxonomy terms that have been manually set different from the defaults for the content type / vocabulary? Once the module is upgraded to 2.x, it can index the rest of content on cron. Or should we do all current data in the upgrade path?
#89
I don't know much, but if all content is updated at cron, then that should be fine.. I would think most cron runs are quite often on mission critical sites. I suppose a message could be shown to let the admin decide if cron should be manually run just after upgrade to complete the process 'if needed', or to manually rebuild. I am thinking if this makes the module upgrade more bulletproof, that's a good thing. Simple is usually better.no?
+Just upgraded 5 sites, all went fine..
#90
Officially committed a supported upgrade path from 1.x to 2.x. The only thing it doesn't handle is links that have default inclusion/priority (will be handled whenever cron runs) and the xmlsitemap_user module. I'll probably have to get a basic version of the user sub-module working so that upgrade path can run as well.
NOTE: The 6.x-2.x-dev download hasn't been repackaged yet with the latest code and the new upgrade path, so don't download or test quite yet. I'll let you know when it's ready!
I'd LOVE to have some gutsy 6.x-1.x users test out the upgrade path. Please remember to backup your database before hand in case anything goes wrong. I did test the upgrade about 20 different times until I got it 100% working for me, but because of the, well, unfortunate nature of the 6.x-1.x intra-updates there might be some differences or things a little wonky still. I tried to account for as much as I knew about.
Once we get enough testers on the upgrade, I'll release alpha 2!
#91
Just added a *basic* version of the xmlsitemap_user sub-module to both D6 and D7 branches. The D6 version should have a working upgrade path.
T-minus about 5.5 hours until a testable, upgradeable version is ready to download!
#92
The latest 6.x-2.x-dev build is now ready and includes the 6.x-1.x -> 6.x-2.x upgrade path! Feel free to start testing and make sure to report back here with your input/results/errors.
Download: http://drupal.org/node/449710
#93
Anyone? :)
#94
Dave, I have been following your work closely and want to upgrade, but I'd like to first make sure that my current configuration is "upgradeable" or if I need to remove everything and start from scratch with your latest build. Would you be so kind as to have a look at the attached image and let me know please if the upgrade path will work on my site? Many thanks, David
#95
Upgrading from the v6.x-1.2 release (with the data I sent you last week) to the current v6.x-2.x-dev release resulted in the following:
The following queries were executed
xmlsitemap_menu module
Update #6200
No queries
xmlsitemap module
Update #6196
No queries
Update #6198
DROP TABLE {xmlsitemap}
CREATE TABLE {xmlsitemap} ( `id` INT unsigned NOT NULL DEFAULT 0, `type` VARCHAR(32) NOT NULL DEFAULT '', `subtype` VARCHAR(128) NOT NULL DEFAULT '', `loc` VARCHAR(255) NOT NULL DEFAULT '', `language` VARCHAR(12) NOT NULL DEFAULT '', `access` TINYINT NOT NULL DEFAULT 1, `status` TINYINT NOT NULL DEFAULT 1, `status_override` TINYINT NOT NULL DEFAULT 0, `lastmod` INT unsigned NOT NULL DEFAULT 0, `priority` FLOAT DEFAULT NULL, `priority_override` TINYINT NOT NULL DEFAULT 0, `changefreq` INT unsigned NOT NULL DEFAULT 0, `changecount` INT unsigned NOT NULL DEFAULT 0, PRIMARY KEY (id, type), INDEX loc (loc), INDEX access_status_loc (access, status, loc), INDEX type_subtype (type, subtype), INDEX language (language) ) /*!40100 DEFAULT CHARACTER SET UTF8 */
Update #6200
No queries
Update #6201
DROP TABLE {xmlsitemap_modules}
UPDATE {system} SET weight = 1 WHERE type = 'module' AND name = 'xmlsitemap'
xmlsitemap_node module
Update #6196
No queries
Update #6198
No queries
Update #6200
No queries
Update #6201
DROP TABLE {xmlsitemap_node}
UPDATE {system} SET weight = 0 WHERE type = 'module' AND name = 'xmlsitemap_node'
xmlsitemap_taxonomy module
Update #6196
No queries
Update #6198
No queries
Update #6200
No queries
Update #6201
DROP TABLE {xmlsitemap_taxonomy}
DROP TABLE {xmlsitemap_term}
UPDATE {system} SET weight = 0 WHERE type = 'module' AND name = 'xmlsitemap_taxonomy'
xmlsitemap_engines module
Update #6196
No queries
Update #6198
No queries
Update #6200
No queries
Update #6201
UPDATE {system} SET weight = 2 WHERE type = 'module' AND name = 'xmlsitemap_engines'
#96
@bwv You'll want to disable and uninstall all the modules in the upper fieldset since they were all technically removed a while ago from the 6.x-1.x version.
@DamienMcKenna: Well if you only got that one error, hey I think that's pretty good. Not sure where that came from since it's got the old {xmlsitemap} table definition being used. :/
#97
@davereid: It might be that the old schema was still cached? I'll try again.
#98
Thank you, I will let you know how it goes.
#99
This module requires PHP version 5.2.* and is incompatible with PHP version 5.1.6.
Good thing I tried the latest release on a test site. ;-)
#100
@bwv: I just made some changes that allowed me to bump the PHP requirement down to PHP 5.1, but I don't think I will be able to go any lower than that. Once the 6.x-2.x build gets repackaged (every 12 hours), you could try that out.
#101
@DamienMcKenna: I added a manual refresh of table schemas in the update so once the dev build gets repackaged, would you be able to try the upgrade again?
#102
@davereid: I'll see if I can get time to do so.
#103
@davereid: thank you... I'll give it another go and get back to you. Many thanks again.
#104
@davereid: I've noticed that the Drush commands in the latest dev snapshot won't work with drush-2.1, it requires drush-3.0-alpha1; when you execute the commands with drush-2.1 it does not respond, does not provide any output, does not record anything in the logs, it just exits quietly.
#105
@DamienMcKenna I removed a little too much from xmlsitemap.drush.inc. :) I added some callback definitions back and tested with both Drush 2.1 and HEAD.
#106
@davereid: Excellent, thanks! Will do lots of testing tomorrow.
#107
Just a note to say that the Drush commands are working well, I hope to re-test the upgrade path in the next day or two.
#108
FYI to everyone I had a hard drive failure on my development laptop so its going to take me a day or two to recover and be able to get back to development.
#109
Major step of revamping the concept of 'contexts' for better integration with Domain access and i18n has been finished.
#110
The 2.x version has finally reached beta1 for both Drupal 6 and 7. Now fully integrates and provies an API to work with an unlimited amount of 'contexts' whether its domains (via domain.module + domain_xmlsitemap.module) or languages (via i18n.module + xmlsitemap_i18n.module).
I think I'm actually going to mark this issue as fixed as we're really nearing the official 2.0 release. Happy moments. :)
#111
Dave_Reid++. This is great.
#112
Automatically closed -- issue fixed for 2 weeks with no activity.
#113
Hi, just wanted to follow up with a report on how the upgrade went. I upgraded from 6.x-1.x-dev to 6.x-2.0-beta1 (I know that this is the queue for 6.x-2.x-dev but I was not sure if I needed to create a new issue to report this) in the following manner:
1. Backed up database
2. Deactivated modules associated with the 6.x-1.x-dev release.
3. I did not uninstall the modules; I was somewhat confused as to whether doing so would erase all the data in the database (my assumption was that uninstallation in fact would do so) so I skipped the uninstall process. (Question: Should I have uninstalled the old module?)
4. I deleted the old 6.x-1.x-dev Sitemap_xml directory from sites/all/modules.
5. I installed the 6.x-2.0-beta1 version in sites/all/modules.
6. Ran update.php; no database errors were recorded. However I did get an error (which I failed to copy down, unfortunately) that had to do with location, if I recall correctly, of the update.php script.
My site (with about 60,000 nodes) appears to be running faster now.
Could someone please advise me whether I should now upgrade to the 6.x-2.x-dev release, or is there a more current version that is out for testing?
Thank you and regards, david
Thank you