This past week, a group of serious Drupal developers assembled in Cambridge, MA to work on upgrading Drupal.org and infrastructure. This team includes Damien Tournoud, Derek Wright, Neil Drumm, David Strauss, Chad Phillips, Jeremy Andrews, Narayan Newton, Austin Smith, Moshe Weitzman, Dries Buytaert, Susan MacPhee and Gábor Hojtsy, and others helping through virtual channels. The team included performance experts, project module maintainers, an infrastructure manager and others. It was a great fit for our goals.
We worked hard, putting in 12-14 hour days, fleshing out solutions over breakfast, lunch and dinner, leaving a short time to sleep. We took on a big job: updating Drupal.org to Drupal 6, accomplishing as much as possible while in the same room. We managed to run test upgrades multiple times a day, integrating everyone’s work, testing interactions of new items as they became ready.
We ran the upgrade on the d6.drupal.org staging site so you can see and test as well. While we tried to make it work as it does in Drupal 5, you will notice some changes. The four most important changes are:
- We migrated to ApacheSolr for search. This will play a big role in the redesign implementation. We hope you enjoy the new faceted search features and help us find any kinks remaining here. The project browsing pages are now based on faceted search.
- Project issue queues are now based on Views. This changes some URLs, but gives us cool new exposed filters. Issue advanced search is now on the same page as the results, allowing you to drill down to the issues you care about.
- We replaced Simplenews with Mailman subscriptions, which we use for discussion lists. We developed a subscription interface under the user settings.
- We built a subversion repository for managing core, modules, themes, and deployments of all Drupal.org sites.
To properly test the upgrade, we set up a staging CVS repository, mail server, mailman lists and other components. Our upgraded code allows you to subscribe and unsubscribe on lists on our staging mailing lists. We welcome you to help test the site features, browse pages, search for projects, filter issue lists, submit and follow up on issues. All existing subscriptions were removed, so you will not receive email from the test site without testing it. Committing to staging CVS tests the related functionality; there is a snapshot of the cvs.drupal.org repository at cvs-new.drupal.org. To test this scratch CVS repository (committing, tagging, creating release nodes), you will need to set your CVSROOT to:
:pserver:cvs_username@cvs-new.drupal.org:/cvs/drupal-contrib
There are probably issues to be worked out, so please look at and test what you usually use. If you find a problem, check the list of known issues related to the drupal.org upgrade and see if it has already been reported. If not, you should report a new issue, and please tag it as "drupal.org upgrade" (expand the "Tags" fieldset when creating the issue and enter the tag name).
We still need more fundraising to continue working on the redesign, any contributions are appreciated. In addition to the many individual contributors who have donated money already, various companies have stepped up to donate human resources. Kudos go to One Laptop Per Child (OLPC), AF83, Four Kitchens, DrupalTherapy, OpenBand, Looforyoo, NowPublic, Tag1 Consulting and Acquia.
Comments
ApacheSolr reindexing for project browsing
One additional note about the ApacheSolr-based project browsing on the test site: to provide an additional facet for filtering modules and themes by Drupal core version, we needed to reindex all the project nodes in solr. This is going to take a little while (given the large number of project nodes). The facet for filtering projects by core version will appear once the projects are reindexed, so please be patient and that will be coming ASAP.
Thanks,
-Derek
___________________
3281d Consulting
Google Groups ?
Why not use Google Groups for mailing lists?
-------------------------------
Read jQuery howto's and tips at new jQuery HowTo Resouce
Because we believe in Open Source
We don't want to outsource key elements of our infrastructure to closed-source commercial services (even if they are "free" as in "you only have to pay for it by looking at their ads"). Drupal is Free and Open Source Software (FOSS) and the community in general (and the infrastructure team in particular) believes very strongly that we should only rely on other open source tools.
___________________
3281d Consulting
Just for clarity: note that
Just for clarity: note that this view is not the official view of the Drupal Association -- who is ultimately responsible for the infrastructure. We've never voted on such a policy (and I don't think we should). The Drupal Association probably relates to this view, but might decide to make exceptions when it makes sense.
It may not be the view of
It may not be the view of the association, but it is the view of the current infrastructure manager of the association. :)
Strongly supported
And a strongly supported view by other members of the community!
hear hear besides .. do you
hear hear
besides .. do you really want to give up your privacy to a corporation?
__________
http://namima.in-egypt.net اجدد اخبار ممثلين
__________
http://namima.in-egypt.net اجدد اخبار ممثلين
http://www.echognumix.com
I agree
I agree
It's really a very hard
It's really a very hard task, but why replacing Simplenews?
unified services
Presently, on the live site, we basically have two mailing list handlers: simplenews and mailman. Simplenews is used for three newsletters while mailman is used for many more discussion lists. They have different signup procedures, different archives, different issues on their own, and they are two sets of software we need to take care of, troubleshoot and upgrade when needed. Mailman mailing lists are pretty well undertood in the infrastructure team, and they are here to stay.
Drupal is not the best for every task. Just as we use an "external' (as in outside of Drupal) searh engine for better results and performance, we use an "external" mailing list manager software. Drupal search is not bad, and simplenews has some great advantages, but we beleive we made the good choices here in terms of user experience and maintenance.
In short, we are just consolidating our discussion and newsletter type mailing lists to under one roof.
Actually I am thinking that
Actually I am thinking that once we've merged everything under mailman we should set up a working group that tries to develop a more Drupal-like solution. It is simply quite hard to integrate with mailman from another web solution. This solution would then also take care of issue status mails.
Looking forward to discuss simplenews v mailman as new topic
That is a topic of interest to me for a big new site later this year. I would like to see a discussion on improving simplenews versus a module interface to mailman. What could be achieved during the remaining life of Drupal 6, given that Drupal 7 will be next year?
I also need to look at a multisite or Domain Access configuration where some mailing lists go across all sites and some are specific to one site, plus the related permissions. Expanding simplenews to or the joining together of several modules to create could let me use more Drupal features than a connection to an external application.
petermoulding.com/web_architect
petermoulding.com/web_architect
Issue status?
I am not sure what you mean by issue status. Issue followup signups are highly user specific. You either sign up to all issues of a project (even new issues coming up), or you sign up to one specific issue (node), currently by commenting on it. Making these mailing lists does not seem like a reasonable tool for the job, so I assume you mean something else, but I don't know what.
No, I exactly mean these
No, I exactly mean these issue follow-ups. The registration mechanism would of course remain different, but the sending mechanism (queuing, bounce handling, ...) could be the same.
Searching by project name
Will this fix the issues with doing a site search for an exact project name? For example:
http://drupal.org/search/node/signup
http://drupal.org/search/node/views
http://drupal.org/search/node/cck
I imagine that the poor results for searching for modules or common Drupal terms is a pretty big turn-off for outsiders :(
--Andrew
We can configure search relevancy by all sorts of things...
See #366738-1: Add integration with ApacheSolr for facet-based project browsing for a proposal on how to solve this problem. Should be very trivial. DamZ and I wrote project_solr.module from scratch on Friday in a flurry of activity, but there are a lot of other things it could be doing. ;) Thanks for the reminder about this.
___________________
3281d Consulting
Amazing
I think the work you guys have done is absolutely amazing, Thank you.
I second that!
Thank you for your hard work! The improvements are great!
I love the module browsing with faceted search, it should be really cool once there are more terms to sort by like, core version, dependencies, associated modules etc. faceted search is sweet ^_^
Overall this is a fantastic advancement!
First than everything:
First than everything: Thanks!
Not only for improving user´s experience and stuff, but I think this will get more non-upgraded modules to get to D6 sooner :)
The new version is ok, everything seems to be perfectly normal, search is faster, but getting to some common pages is slower (ie. documentation).
Anyway, I can´t wait for the shiny new drupal.org redesign!
Rosamunda
Buenos Aires | Argentina
www.ligadelconsorcista.org
Yeah, slow
Note that http://d6.drupal.org doesn't run on the production infrastructure, but on a separate (and much slower) virtual machine. That explain why the navigation can seem much slower on some pages.
Are all the project* views available in CVS?
There's some fascinating advanced stuff you're doing in the issue tracker that'd be neat to analyze. I see http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/project_iss.... Is that exactly what the new Drupal.org is running, or are we overriding a few views, either in Views UI module or in drupal.org customizations module?
Those are exactly the Views
Those are exactly the Views we use on the test site.
For the theming part to work, you will need to apply a small improvement to Views manually (#364637: Let table rows emit custom classes).
Yes, everything is "vanilla" project_issue default views
Thanks for the kind words. ;) Yes, as Damien mentioned, it's all the default views from project_issue. If you (or anyone else) has specific questions about how/why certain things work as they do, I'd be happy to explain. Probably support requests in the Project issue tracking issue queue would be the best place to start, though I'll be writing up more about my D6 project* + Views2 fun some where more visible (time permitting).
Cheers,
-Derek
___________________
3281d Consulting
Thanks!
I can't wait to dig in and see the results of your work. Thanks much, there are lots of folks who appreciate all of you. -NP
how do I subscribe to forum posts here in drupal.org
Hello, how do I subscribe to forum posts here in drupal.org ? Or how do I keep a "list" of posts I want without receiving anything in email ?
Are we using the new search now ? Example - Searching for simple thing like openid documentation or open id documentation did not give me any result or needed result. Or gave "The Apache Solr search engine is not available. Please contact your site administrator."
If this is improved drupal.org how do I simply get the new posts I made || the comments I made &/or the comments I got || my unanswered posts. Digging through bloblike tracker one has to dig and sort everything everytime.
By the way on a side note, what search is recommended to be installed on our Drupal sites - the core serach module or Apache solr ? Is the core search is not working satisfactorily or modifiable ?
upgraded, somewhat improved
Our goal was to upgrade the site as-is without adding functionality. We did add some functionality after all, in the spirit of doing some (a small set) of the redesign items "ahead of the plan". On what way the redesigned drupal.org is planned to solve you tracking forums, collecting lists of stuff you follow and so on, compared to the current and identically upgraded tracker, see the redesign posts at http://groups.drupal.org/drupalorg-redesign-plan-drupal-association
There are multiple search solutions you can choose from, the Drupal core search and ApacheSolr are two of these. You need to know your own requirements and possibilities. ApacheSolr requires a Java application server and a complete seaprate running search indexing/lookup service, compared to Drupal core search which is built into Drupal and is written in pure PHP. While ApacheSolr has more resource requirements and more expertise need to set up, it has a broader feature set and people across multiple communities use and improve it, unlike the Drupal core search.
Sort by date or author not working on d6.do
Congratulations to all for their hard work in upgrading do to D6.
What's the best way to report bugs or incomplete features? For example, the sort by date or author is not working on d6.do.
/*_*/
http://www.xmacinfo.com
Read the post.
Read the post.
me too. look
me too. look look
---------------------------------------------------------------------------
Drupaluser.org---Sea water droplets together
In other words, There are
In other words,
Report Problems Here
But please search first before posting... ;)
We're trying to fix issues quickly, but there have already been a lot of duplicates posted. As always, please search before you submit a new issue. In this case, there's an easy way to view all issues related to the drupal.org upgrade, regardless of which project they're posted against, so if you find something broken or have a request, please check that list first to make sure we're not already aware of it. Thanks!
___________________
3281d Consulting
I thought drupal.org was
I thought drupal.org was based on Joomla?
lunk_rat is Acquia Certified Grand Master
haha
that's just made my day!
Filter issues by version numbers
I often find myself looking through issue queues to see if my issue is already reported, and for some projects there is an annoyingly large number of bugs for older versions mixed in with the ones for the current version. It would be great if I could filter the issue queue for just the version I am going to report a bug for.
Paul K Egell-Johnsen
Paul K Egell-Johnsen
submit a feature request
Please submit your feature request against the project issue module.
You can do this already
The version is project-specific, not site wide. So, you can only filter by version on a per-project issue queue (which are easy to get to, either by clicking on the values in the "Project" column on the site-wide issue queue pages, or via the direct links from the project pages). So, for example, core's issue queue lives here:
http://d6.drupal.org/project/issues/drupal
There's a single-select filter for version.
If you want multi-select, you need the more powerful (and ugly) advanced search page:
http://d6.drupal.org/project/issues/search/drupal
Enjoy,
-Derek
___________________
3281d Consulting
Perfect
Exactly what I was looking for.
Paul K Egell-Johnsen
Paul K Egell-Johnsen
Being able to quickly filter
Being able to quickly filter by version is a great improvement. Thanks for putting that in.
Not new
Being able to filter by version has been there as long as I can remember... I can't swear it's been there all 4 years I've been here but I sure dont' remember ever not being able to do it. The fact that people are surprised by this makes me think that maybe it could be made more obvious somehow?
It's hard to go back to not knowing it's there for me... For those of you who just found it, is there somethign that would have made it easier to find?
Michelle
--------------------------------------
See my Drupal articles and tutorials or come check out life in the Coulee Region.
Thank You
I am not a developer, but I maintain a simple site and just want to say thanks for SO much hard work you all put in on such a complicated task(seriously I can't even imagine). Thanks for making D.O awesome! When I get my next pay check I'm chipping in for this sprint too
I think it's a really really
I think it's a really really hard work! But i believe you guys can do it. I ChipedIn, this is all i can :)
Future of Drupal search and simplenews?
With Drupal search and Simplenews being dumped from the d.o infrastructure, does this mean curtains for these modules in terms of future development?
----------------
Dominic Ryan
www.iis-aid.com
Nope.
Drupal.org wasn't the only site to use SimpleNews - in fact it existed before Drupal.org used it. I'm confident it will continue to be maintained. As for search, it is still a great part of core, and essential to Drupal's future. The success of ApacheSolr has actually focused more healthy attention on core search because people start to see what they've been missing =) Here's the list of stuff that needs to be done: http://drupal.org/node/366930
- Robert Douglass
-----
my Drupal book | Twitter
Agree
I agree.
Nice work,now it looks is
Nice work,now it looks is ok.
i have a website build with drupal5,how can i up to drupal6? where is the teaching as "setup-by-setup" for me?
Hope to see.
For upgrade instructions,
source: http://drupal.org/drupal-6.0
Bug/Gotcha in Drupal.org post-Drupal 6 upgrade
Ladies and gentlemen,
I think the work you have done here is truly phenomenal, and the upgrade seems to have gone off without many problems.
I did want to let you know about an issue I saw that may need to be addressed. I logged into my Drupal.org account and searched for an "advanced help" module in the "Search modules" widget. While I found what I was looking for, each search result was repeated twice. Without knowing anything about the code or the structure of the data, this strikes me as a query which has not been sufficiently constrained, or maybe needs a GROUP BY.
Again, thanks for all the hard work; looking forward to greater participation in the community!
Problem with the Search Doohickie?
I tried searching a couple things at random, and both gave me this error:
The things I tried searching:
ICK. Wamp = baddness :P Use
ICK. Wamp = baddness :P Use Lamp to see the light! lol
Actually...
My post wasn't about what I was searching for but for the online searching of this forum board. I was just copying and pasting the description from another user's question in another thread.
But it does appear that since that time, the error has been resolved. So apparently someone else figured it out from a different post, or something just started working - I just tried another search to verify. :0)
WAMP goodness
W: Works everywhere
W: Works on the machine your customers supply
W: Works on the cheap slow Dell machine your customer put on your desk because all the other staff demanded the good machines.
W: Works on the machine you supply to replace the slow Dell machine supplied by your customer because you really want to get the Web page finished and, well, to be frank, even with modern medicine, our maximum life expectancy is only 100 years, so you really do need something faster than a Dell.
W: Works on Vista. Nothing else works on Vista so putting WAMP on Vista gives you something to show people when you try to sell your Vista machine.
petermoulding.com/web_architect
Winblowz And Microcrap ==
Winblowz And Microcrap == Pathetic heheheh
Linux/Apache/MySQL == Powerful. Beat those lol (without reversing them ;) )
If you can't be civil...
If you can't be civil, I'd prefer you not post at all.
Themes by first letter of the name
I like the changes so far. Finding a theme is still painful. Is that part of the change or is there a separate thread for the theme list?
The everything in one page was slow because of the more than 200 themes (after selecting D6). The current 5 themes to a page means we have to browse 40 pages. I like the themes listed by first letter of the alphabet used at http://d-theme.com/. Some pages are large but not unmanageable. You get related themes on one page. For example, all the Plutardo themes are on the P page. It is really easy to find a theme when you remember the name but not the exact way it was written. Again using an example, you like that blue theme, blue? The B page has blueprint, bluetrip, Blue Bars, Blue Zinfandel, ...
petermoulding.com/web_architect
Thanks for your contributions keeping Drupal ...
...up alive and getting even greater.
Developing , Drupal 5 - 6 upgrades, Drupal 7 and the Drupal.org site and the Drupal modules
Thanks for using your dear time for the Drupal community