Heavily Classified Content ;) [#726060]

Hi Folks,

I have an extremely high impact use case in which most of my site content will be placed into numerous categories and where most of the content will be listed and accessed from within those categorized pages.

So, I am thinking about simple stuff like Views and paging and how most efficiently to structure this for a site what might end up with anywhere from 100k to 1million daily users. So, I am thinking that in the relationship table, I need to have some sort of "content category entry number" which would be sequentially incremented and which could be used for time sequenced displaying of content within a category, as well as for a paging mechanism that would never require reading more content than what is required to fill a single page.

As a side note, a piece of content that is published in one of the categories, may not be published in all of the other categories.

Are there any modules out there that would already support most or even some of this?
What node relationship and/or other modules should I be looking at for this?
Is anything happening in Views3 that might help with this?

Thanks folks.

Comments

Comment #1

dawehner

German

commented 26 February 2010 at 06:48

I need to have some sort of "content category entry number" which would be sequentially incremented and which could be used for time sequenced displaying of content within a category

I don't think there is currently a module for this, so you would have to write your own.

What node relationship and/or other modules should I be looking at for this?

Is this really a general drupal support issue queue? I don't understand the relationship to views :)

I'm not sure how related it is but there is https://launchpad.net/mv which builds tables for every view, which makes querying insane fast.

Comment #2

tesliana commented 26 February 2010 at 21:48

Mr. Wehner,

Most of the Drupal world revolves around Views.

Storing hundreds of thousands of nodes in hundreds of categories and being able to pull them out efficiently, without added counting and without added sorting is the goal. So, direct Views related question is, given a sequentially numbered and indexed field/column, could I use it in Views to more or less "slide" up and down that index by calculating number of pages based on the last/highest entry and reading one page set of nodes at a time?

No matter what, I am extremely appreciative of *your* work and especially the praise that you are getting for your contributions on Views 3 - which is taking Drupal to that next level of developability - yes, developability - that crucial piece without which there is no "usability" ;)

Thanks again.

Comment #3

dawehner

German

commented 1 March 2010 at 23:02

Views is pluggable nearly 100 percent. So as long as mysql supports it you can make it by custom coding. I hope you see the reason, that views core has to be small and "simple".

I still don't get what you mean with:

could I use it in Views to more or less "slide" up and down that index by calculating number of pages based on the last/highest entry and reading one page set of nodes at a time?

Comment #4

tesliana commented 2 March 2010 at 06:29

First, thanks a million for responding.

Suppose that there are 5 million nodes of a certain node type and that each node has a sequentially assigned sequence field that is indexed. Then, all of the paging could be calculated and in addition to sliding through records (via prev and next pages) we could jump to any page because we could calculate the range of the sequence values that are displayed on that page.

The reason why I am preoccupied with this is that I have a node type that could have millions of nodes and using the devel module to see how long queries and page construction are taking, there seems not to be significant difference between viewing a table of nodes on first or any other page. I do not know what is happening under the covers, but there seems to be lots of resources being chewed up just to figure out what records should be shown on a specific page. When I get rid of paging and change a filter to give me only a page worth of records, it's done in fraction of time.

Thanks again.