[meta] Improve search/navigation for d.o community docs

sub

new mockup link

sub

new issue summary since we haven't decided after all

I would draw attention to there being (at least) two related needs to be served:

• Allowing users to step from one page to conceptually nearby pages up, down, and across the hierarchy.
• Presenting to users (and authors and maintainers) a way to browse and understand a conceptual landscape consisting of thousands of pages (as they might get oriented to a paper book by looking at the table-of-contents).

One tool that advances both of these objectives is a collapsible tree view, which is often used for site navigation. MSDN Library (as mentioned in #117 in the summary above) has a particularly extensive example: http://msdn.microsoft.com/en-us/library/default.aspx (and choose "Classic" view, if that's not the default.)

To demonstrate the utility, I created a demo showing drupal.org's entire doc hierarchy as a collapsible tree: http://grahamwideman.wikispaces.com/Drupal+documentation+trees (look at the "Key to symbols" section for what is clickable: icons in addition to the [++] and [--].)

Now, individual pages could not afford the load and crunching time to use the entire tree for their nav. One could imagine a version of Book implementing a speedier Ajax-fied version of this.

Meanwhile, depending on whether others find the complete tree view as powerful as I do, it is something that might be implemented as a separate "complete docs map" page. That has the prospect of making more powerful docs exploration available, while taking pressure off the Book apparatus as the only solution.

That said, I don't feel I've thoroughly looked yet to see who's been down this path before before inventing wheels.

jhodgdon: By way of a quick response: I'm actively consolidating some thinking about this, and will be back with more discussion in a couple of days. If the issue summary hasn't been tackled by then, I'll have a go.

I reaaaaaaaaaaaaaaaaaaally like the example in #9 on the sandbox site! It achieves 2 things - keeping the sidebar somewhat condensed on first page load (though of course not as much as a dropdown), and also allowing people to easily browse through the book menu *without* having to actually navigate to all the pages. I think this would be a significant improvement to what we have now, and a less drastic (and hence less controversial) change.

@arianek: Thanks for the positive feedback. I can't tell if you noted that I'm not advocating this be used on every book page, for reasons of performance, page-real-estate and likely implementation-effort. A slimmed down version possibly.

But I am much more interested in the full-featured version, implemented as a stand-alone page, which I suspect is more tractable to implement, technically and within d.o.'s situation.

I discuss what I think are the requirements in play here, with my suggestion and rationales:
http://grahamwideman.wikispaces.com/Drupal.org+--+Thoughts+on+TOC+and+na...

@jhodgdon: I didn't get to actually revising the Issue Summary on this page. As just mentioned, I've written up my thoughts on the matter on my site. I hope this contributes to the discussion, but I don't want to just substitute my views for the existing Issue Summary as my version encompasses some different territory than the Issue Summary envisions.

That said, if no one in the next day or two at least tackles getting some screenshots for the Issues Summary, I will do that.

jhodgdon: On looking at editing the issue, I noticed that adding an image calls for a URL, so I'm guessing I need to upload images as a separate step... I don't see a way to upload, or add attachments, during the issue editing process, so perhaps I need additional powers? However, I do see a way to add an attachment to a comment (like this one)... is that what one is supposed to do?

-- Graham

Uploading images for revised Issue Summary

moved list of options from parent issue to this one.

gwideman++ for a great summary.

My five cents on this is that MSDN-classic seems like the best option (as the summary table describes), as long as we don't find any new innovative way of browsing the docs. Gonna think some more on it, and try to be innovative.

Or may be we can have mixture of search + menu like this:
putting search like this on top of menu [ http://harvesthq.github.com/chosen/ search type entitled ' Support'] & having menu bar like this which looks more decent with all '+-' things than just msdn classic menu [http://www.pixelclever.com/custom-drupal-themes-drupal-5-and-drupal-6 see in rightsidebar]
this can have an impact on performance but don't think it will be big.

Just a thought!

My vote is for the MSDN classic too, and also with some kind of linked search textbox above it which allows searching the tree somehow, if that's feasible.

Here's my attempt at documenting the use cases of docs navigation and different pain points for the docs navigation. I'm sure I've left some out.

Use Cases

"End User" and General Use Cases

1) Searching for instructions for a specific task but not sure of module (e.g. "how do I <something> with Drupal")

2) Searching for instructions on how to use a specific feature/module/functionality (e.g. "export view to CSV file")

3) Searching for introductory documentation/tutorials (e.g. "Adding new users")

4) Contributing to community docs by expanding an existing section or adding a new section

"Site Builder / Developer" Use Cases

5) Searching for a configuration recipe on how to build something (e.g. "Create a mailing list")

6) Looking for additional module documentation or examples (e.g. "display suite hide field label")

"Docs Maintainer" Use Cases

Could someone suggest more?

7) Managing / Reviewing page structure

8) Merging duplicate or related content

Pain Points

A) Locating content when you're not sure of the right keywords. Visual inspection of the tree is too slow, with all the page reloads, and inability to see how the tree is organized. (1,3,7,8).

B) Docs search results are not obvious enough on google. Try a google search for "drupal add new user". It would be nice if the best answer "Tutorial 14: Create a new user" was more prominent among the other results (somehow) (meta tags or something to help google out?) (1,3,4,5)

C) It's confusing when results are from older Drupal versions only. (e.g. from documentation page, click on "administration guide", then "managing users", but there is no Drupal 7 section, only Drupal 6 sections. The Drupal 6 section is almost totally accurate though, so is "the best" we have to offer. But an end user won't know that. Solving this one is probably better done by enforcing a style guide (don't make things version specific so they can be updated) rather than navigation. (1,2,3,5,6)

D) The "Search" box in the top right is not limited to documentation so is probably useless for someone searching for docs. (e.g. Visit drupal.org, click "documentation", read list of titles, don't know where to go, type "add new users" into search box). (1,2,3)

E) The Site Building Guide is full of multiple branches of content that are hard to explore. Information is organized by module, under HowTo's, under the individual tutorial trees, and under Site Recipies. It's currently very hard to navigate to the best spot, since each change of tree branch reloads the page and your old location (and its siblings) are lost. (1,2,3,4,5,6)

F) Contributing: Finding related content / convincing yourself you're doing it in the right spot. It's hard to know whether to create a new page or edit an existing one - especially when you find similar content in two different branches of the tree. (e.g. say I want to now add/update the "add a user" instructions for Drupal 7, having just figured it out. Do I add a new "User management in Drupal 7" page, or update the "User Management in Drupal 6" page? Any kind of novice user will likely just end up doing nothing rather than do the wrong thing. (4)

Pain Points Addressed by the current proposal

The current plan with the MSDN classic type tree on the right solves or helps out A, E, and F. If we add some kind of keyword search to the tree, then we have improved D as well as doing a nice job of A and E.

B and C are not addressed. B is out of scope. C we could maybe implement with some kind of faceted search capability on top of our search box, but I recommend we just get out of the habit of making things too version specific (like the "adding new users in Drupal 6" example), and it would work itself out.

damien: I like your initiative to enumerate some use cases. On the end-user list, I note that 1,2,3 are all "how to do something" topics, and 4 is end-user-as-contributor.

I would add that another end-user need is simply to understand the "what" of drupal: Gaining an understanding of what is drupal's structure: what are its components, what are their functional relationships, and how do components and functions map onto the artifacts that are easily inspected, such as the file layout and the database structure.

I raise this, because this type of info is often complementary to, but not subsidiary to, the task-oriented docs ("orthogonal" on might say). A user may hunt for it as a quest in itself, or as reinforcing info while conducting a task.

Here is the data on number of pages at different levels of the hierarchy, by book (similar data to that posted by lin. Mine is from 2012-03-12 or so). This is useful for assessing the typical size and depth of the tree widget as users wrangle it to navigate around the docs. So, for example, for the Site Building Guide they'll spend a lot of time exploring at the 4th to 7th levels.

The plots are sorted in order of total pages in the book (biggest first). The plots are separately scaled so their peaks are at "4 units", with that value marked on the Y axis. In general, the more pages a book has, the more the peak is shifted "deeper". This is because individual pages tend to "fan out" to only a limited number of child pages, so a larger book tends to get deeper, rather than wider at each tier.

More stats at:
http://grahamwideman.wikispaces.com/Drupal+docs+--+Statistics

-- Graham

I think breadcrumbs on docs pages is a great idea if we're going to have a tree that you can alter by pressing + and -. That way no matter where you explore to at the right, or how you transform that menu tree, the Breadcrumbs will tell you where your current page is.

I think they are a very intuitive navigation tool in their own right, that is not as overwhelming as the entire tree. They are especially handy when you land at a page via a search result. It's very easy to quickly broaden your focus a bit by moving up a few levels in the book hierarchy, and you can really do that with just a glance as the page loads.

I *would* support breadcrumbs, but I think they're going to bring disaster in the future. As soon as we get into using multiple book outlines, breadcrumbs will become a massively complicated maintenance burden - I think we're better off sticking with just the book nav block at the bottom of the page.

#31: That *is* a good point – well done to find it. (Otherwise: yes, breadcrumbs totally makes sense.)

I checked out the notebook navigation in OpenAtrium, and I quite like it, but have one major reservation. The like: It's basically a simplified version of a tree control that doesn't let you have two branches open at once, if I get it correctly? Playing around in it, I was able to figure out opening and closing the branch of the tree I was on, and therefore how to back up and find other branches.

The reservation: What concerns me is losing the ability to visualize portions of the tree at once. One of my major pain points on community docs pages anyway is figuring out what sections are available once locked into one branch of the tree. That goes for contributing and just exploring to read about stuff.

I think that would be harder with only one path able to be open at a time, as you're back to trying to remember. For example, choosing between sub-trees of "Contributed Modules - Contributed Module Documentaiton" vs "Howtos" vs "Site Recipes" vs other places still in the Site Builder docs would be easy with a tree that can be open to all three places to visually compare, but not with the admin approach. Because only one path is open at a time it would be just like it is now - you have to remember it / write it on a scrap of paper / have multiple tabs or windows.

Just curious if anything has spawned from here - I found that DHTML module will perform these functions but the module is buggy and not really being supported from what I can tell. On core 7.15 it will crash CKeditor if bullets are used.

Add new section for proposed resolution, rearrange some of the requirements slightly, put links in TOC