Hi all:

Asking around in IRC as well, but here's the deal:

I started a role on a large, big-budget, high profile social networking project here in NYC. I'm making the case for Drupal, they want to decide today. Some code is already built from the ground up, but the case for Drupal is still valid as much of that code can be brought over, plus Drupal will handle so much else that would otherwise need to be built from the ground up.

This project is more complex than a lot of Drupal sites. Some big questions:

One issue is node relationships. This project has a lot of nesting, i.e. a parent content type has a bunch of children to be nested in that node (page), while each of those children is its own node as well. We need to be able to pull all the children (associations) at many different points. Different assets (pieces of content) are associated with different users and parent items, and we need to pull these associations at different points. Put simply, there might be a "main project", with multiple users applying to be in the project in some way, and they'd bring certain "assets", or pieces of content (say, the promotional poster) -- so we need to be able to pull the users on the project and the assets they've submitted to the project and more.

Anyone here used Node Family or considering it for big projects? Other relationship modules to consider?

Anyone here know of some real complex Drupal sites that utilize relationships or have a more complicated structure/hierarchy? I don't believe that mtv.co.uk (Lullabot) did this kind of stuff, but maybe someone here knows.

Voting. We want certain assets to be tied to the parent based on their votes. Certain relationships to exist based on the vote.

Any help and insight is greatly appreciated.

And on that note, if you are familiar with this area and may be available for contracted work, please let me know as there may be opportunity to bring in external resources on this project and it'd be good to know now.

This could be a very high profile site, and will bring contributions to the Drupal community as well. Making the case for them to go the Drupal route, but there are surely pros and cons (your thoughts?), the above is a big issue, and they need to decide quick.

Thanks!

--Dave

Comments

nevets’s picture

While there are ways to handle that with contributed modules I have tended to do it with custom modules. Probably in part because I want more than just the relation, for example context sensitve blocks and links. I did the work for www.dcaudubon.org which uses this approach for building up a field trip. A field trip has field site and optional carpool site. It can have one or more field reports and can include a bird count.

Sid_M’s picture

Hi Dave,

This sounds like a very cool project. My company is currently completing a project which sounds like it has some structural similarities to yours. For example, we have a number of places where nodes of one type are used to categorize other types of nodes. We are getting a lot of mileage out of the combination of the following modules: CCK (which will be part of core drupal in 5.0), Views and Category.

In addition, we have written our own drupal extension which lets us run code between CCK and output to the browser. This is essentially theming code, but with significantly more flexibility and power than the default system allows. We're very open to sharing that extension code with the Drupal community (we just haven't had time to do so yet).

We haven't worked with voting. Your idea of categorizing by vote sounds interesting. I suspect it would take some custom coding, but it sounds relatively straightforward to me.

We'd like the opportunity to chat with you, and to show you some of what we've done if that would be of interest to you. We'd write up a more extensive reply to your post, but this seems too time-sensitive to do so.

Feel free to contact me via my profile here at Drupal. You can read about my company at www.CraftySpace.com

effulgentsia’s picture

Hi Dave,

I work with Sid, and I wanted to take a moment to give you some more info based on my experience with the project we're working on together.

The Views module is extremely powerful. Get to know it well. It's perfect for querying nodes based on various properties, and then displaying the matching nodes. In addition to having a view as your page or in a block, you can have code that puts a view pretty much anywhere. You can also customize how you theme the view. And with a little more in-depth coding, you can add plugins to the view module if you want to query based on criteria that there isn't a plugin for already.

Get to know CCK well. When you make a node using CCK, all of its fields are automically available as selection criteria for a view (assuming you're using field types that are properly implemented). CCK also includes a node reference field type, which can be used for setting up node to node relationships, although there's pros and cons to that approach (pro is that it's easy, con is that the reference can be orphaned when the target node is deleted).

Get to know the Category module. It lets you use nodes instead of taxonomy terms for categorizing other nodes. A lot of relationships can be represented as a "A categorizes B" relationship, and this module is perfect for those. It can also be used for creating "A is a parent of B" relationships, since category trees can be heirarchical.

Node Relativity is a great module. It lets you create generic "A to B" relationships between nodes. In theory, this can be used for "A is a parent of B" relationships where you don't want to make A or B into categories, although I prefer to use it for associative relationships rather than structural relationships. For example, "related articles" or "related documents". I like the UI this module uses for setting up the relationships.

There's also the Relationship module. I haven't worked with it yet, but I like what it says in its description -- that it has an extensible mechanism for specifying what kind of relationship you're creating when you associate A and B.

Node Family is good if the relationship is based on node type rather than individual node. Although, if that's what you want, it's pretty easy to do with views instead. It also lets you restrict how many nodes of a particular type can be created by any one user, and that's pretty cool.

There's a great discussion here (http://groups.drupal.org/node/1323) about the issue of node to node relationships.

Hope this helps,
Alex.

DaveNotik’s picture

Imagine a hierarchy like so:

  • Users
    • Assets
    • Proposals
      • Roles (engineer, sound guy, others)
        • Asset associations
          • Comments

There'll be pages like so:

Proposal detail page. Show proposal detail here, show top roles here, show this associated asset there, show comments on this proposal (including on sub roles and assets).

Role detail page. Show particular role's detail, comments specific to this role (a subset of comments from proposal).

Asset detail. All the assets associated with a particular proposal, ability to comment on each.

Needs comments on list of roles, comments on each individual role, each individual proposal, asset, etc...

There are a lot of nested relationships, a lot of associated information that needs to be displayed, heavy theming.

Now I know Drupal is extremely powerful and in fact there are too few sites that really take advantage of its power, though some are starting to emerge. I myself have never gone this deep in practice, though I've explored the possibilities some.

Again, they will decide imminently whether to go w/ Drupal or continue their from-the-ground-up approach. I think going the Drupal route will serve them best, provided these issues are easily addressed. Having such a project go the Drupal route will be extremely beneficial to the community as well, so help me make the case.

Any and all insight is appreciated. Please offer your thoughts.

--
http://dave.notik.com

Sid_M’s picture

Hi Dave,

Alex and I talked this over. Here's what we came up with.

Our sense is that Drupal provides almost all of the functionality that you describe. Yes, it will definitely take some amount of custom coding to make it solve your specific needs, but most of it is already there. So, unless the already built pieces available to you also do most of what you need, we'd bet that you'll be able to build a successful site more quickly, and thus less expensively, using drupal as opposed to rolling your own. (We return to these questions in our final paragraph.)

This sounds like a pretty complicated project. Much as we'd like to, we can't really do a complete analysis and design on the forums. To do an adequate job, we'd need to understand quite a bit more about what we are trying to model.

Roughly speaking we would see a process working like this:

1) Analysis Phase: Understand what the objects in the system are, determining what node types and nodes we need, and the relationships that need to exist among them. I appreciate that you've taken a stab at outlining this, but we'd need to understand a good deal more before we could really answer these questions.

During this phase we would ask questions such as the following. Are the relationships many to many or many to 1? Can the same role be associated with multiple proposals? If so, when an asset is associated with a role, is it always specific to the proposal, or can it be associated to the role in general, regardless of proposal? Depending on the answers to these types of questions, it will become clearer which specific modules to use for building the relationships.

2) Design: determine what content types to build with CCK, and what fields those types have. Determine what views are needed. Determine what modules to use build the relationships we need (see Alex's previous post on the various modules that are useful for relationships).

Once a complete A&D was done, it would be rather straightforward to actually build the system.

The key concept I think is the theming work that goes on between the CCK and sending a page to the browser. That theming work can involve running any number of views to gather the needed information, and then to format its output as necessary.

Just to take one of your examples: "Proposal detail page. Show proposal detail here, show top roles here, show this associated asset there, show comments on this proposal (including on sub roles and assets)." Quite likely there would be a proposal node type with a node for each proposal. When a user went to that node, the theming function would do something like the following:

1) Run a view to gather all roles associated with the proposal that met the criteria for being considered a top role.

2) Run a view to gather the relevant information about associated assets.

3) If comments are Drupal built-in comments, run a database query to collect the needed ones. If they are custom nodes, then run one or more views to gather the needed ones.

We would already have the information for the proposal node, which would likely include the details for the proposal. If it didn't, we'd use another view to gather them.

Once we had all this information, it would be very straightforward to output it as html.

Finally, CSS would be used to fine tune the look and feel as needed.

Needless to say, there'd also have to be a whole set of interfaces for creating entities (such as users, proposals, assets, roles and comments) and relationships among them. The nice thing about CCK is it already does most of the basic work involved in creating admin interfaces.

Everything you've outlined is possible. But depending on the specifics, it could be more or less work. Our sense is that it will be much easier to start with Drupal and build the custom stuff than to build the whole thing from scratch. But we say that as folks who have a lot of Drupal experience. For someone without enough Drupal experience, it might be easier to not have to figure out the "drupal way" of doing things.

DaveNotik’s picture

Hi Sid and Alex:

Thanks very much for your clear responses! Your analysis of the different options is very helpful.

I am in fact very familiar with Views and CCK -- I've followed their development since their respective inceptions, and have used them on a number of projects. I fully expected Views and CCK to be central to a project like this. I was considering the node reference field in CCK too and I'm glad to have learned that it recently reached some level of maturity -- though I'm still gathering more information about how well it works.

I know there is a lot of parallel working going on in the world of node relationships, so it was good to get an updated sense of what's out there.

So it seems I can have views that pull detail of nodes related to the current node (e.g. assets related to the current role in the current proposal). Pretty straightforward as far as these things go, yet was cautious as this is pushing the boundaries. It seems I can use some of CCK to enforce limits to the amount of node type x that can be associated with node type y as well.

Been talking directly with a few people about the possibilities. Integration of CCK in 5.0 core makes for some great options.

Keep up the conversation. I hope that the client decides to go with Drupal so I can really tackle this stuff.

--D

--
http://dave.notik.com

moshe weitzman’s picture

i think you will get a lot of mileage from cck on this. specifically the node reference field. my suggestion is to do rapid prototyping in cck. this can validate than the data relationships that you need are possible. after that, it is just themeing.

you might also use viewfield field type in cck to show all children when viewing a node.

DaveNotik’s picture

Thanks, Moshe, and for the help in #drupal.

You giving it your nod means a lot.

Bottom line is that this is possible -- using views + cck + node_reference + panels as a base.

I empathize w/ the client's concerns, yet I think they'll be quite thrilled w/ what doing all this w/ Drupal makes possible. I'll know soon.

--D

--
http://dave.notik.com

effulgentsia’s picture

Remember to also consider the category module, even if you don't look into the other relationship modules. Given how fundamental the taxonomy system in drupal is, it's really useful to have taxonomy terms be nodes. For example, the project Sid and I are working on right now is for a bicycle manufacturer. A lot of our nodes are associated with a bike model. Specific bicycles are categorized by model. Dealers are categorized by the models they sell. Customer reviews are categorized by the model they're about. Even users can be associated with the model(s) they ride. However, a model is more than just a name, which is all it could be if it were setup as a taxonomy term. Instead, it's a full-fledged CCK node type, with its own complex theming similar to what you're trying to do with your proposal node type. Of course, you might decide that on your site, it doesn't make sense to think of proposals as categories. But maybe it does. Given everything you can do with the taxonomy system, it's at least worth thinking about.

The node reference field type is awesome, but at least the version I'm working with (from a couple months ago) doesn't remove the reference when the target node is deleted. That's not a problem for prototyping, but you may want to consider whether it's ok for the finished project. Of course, maybe this is something that's already been patched or can be patched.

I haven't used viewfield yet, but it sounds great. Could be especially useful if you want different nodes of the same type to use different views. If you want to constrain all nodes of the same type to use the same view, putting that into the theming code of the node type might be better.

Anyway, good luck convincing your client to use Drupal. Would be great to have you spend some paid time making Drupal better!

robertDouglass’s picture

Yes, of course you can build this site with Drupal. You and I know that, as do all of the people who responded here.

What your client needs to hear is that the answer to the following equation is positive:

(time and effort needed to build from scratch) - (time and effort to learn how to use Drupal + time and effort to customize it) = x

One thing is clear; you can't build any site this complex using only stock Drupal modules. Customization is needed. So your best chance of selling the idea of using Drupal comes from showing the things that Drupal can do for the project, out-of-the-box, without any customization. This includes:

  • User management, including roles and node level access, password recovery, user tracking, and session management.
  • Revisions systemy
  • File uploads
  • Publish/unpublish framework with lots of contrib modules to customize
  • other features go here... what does Drupal do that your project needs?

The custom object model that you describe above, with nested relations, has to be programmed. This is a custom need and Drupal likely isn't equipped to do it, 100%, with existing tools. But what's the alternative that they're considering? Custom programming it. So there is no loss there. Instead, you need to focus at that point on the nice APIs available.

  • First and foremost, the hook system. Show that you can customize behavior at nearly every point in a node or user's lifecycle.
  • Close second, the FormsAPI. Show how the FAPI (especially in conjunction with hook_form_alter) can, in just a few tiny lines of code, change any existing form (including how it is validated and submitted)
  • CCK + Views. Prepare in advance, since making both is time consuming, but show them how the custom displays of their special object model can be achieved using these two tools. You'll have to do what Moshe said and rapid prototype with CCK's node reference field. Especially in 5.0, this is very powerful.
  • Other contrib modules. Pick the ones that fit best to the project of course, but emphasize how much code is probably already written.

Don't try to convince anyone that Drupal will solve all the problems faced by a complex site, because it isn't true. Instead, argue that by letting Drupal do the parts that it does well, and custom programming only the rest, you can focus better on fixing the bugs in your own code because compared to steaming-fresh, newly crafted code, Drupal is pretty bug-free.

- Robert Douglass

-----
Lullabot | My Drupal book | My Digg stories

sime’s picture

In my experience with complex relationships between entities, the client *must* be able to define the rules before you agree to proceed.

DaveNotik’s picture

Thanks to all. It really came down to how Robert portrayed it.

Alas, it seems the decision is to *not* go with Drupal.

It really came down to that there are a lot of pages with a lot of different pieces pulled from different objects, themed in very unique ways. I know Drupal is extremely powerful in those regards, but there is a serious lack of credible examples for sites of this scale. Many Drupal sites have a few content types and don't get as complex with regards to the relationships. It's daunting, even for me, knowing that we'll have this level of complexity and that there may be unique challenges going the Drupal route whereas it's easier (or just plain feels less daunting, even if it may not be when all is said and done) to go the custom PHP, object model route. I did firmly believe that they'd be thankful in the long-run, but I respect their caution and agree with how they went about their analysis.

All in all, I think that this company needs to first go with Drupal for a smaller site so as to understand its power -- the node API, forms API, database abstraction layer, hook system. Those are all things that are seriously underutilized in so many sites -- it's like there are great developers who built a great product, but nobody (including them) really has used it for a really large, complex social network-like project like this. When thinking of big examples, I thought mtv.co.uk, but even they didn't use much... I'm still very interested if others have better examples.

Let's keep this crucial discussion going. The future of Drupal and its usage depends on answers to these questions.

--D

--
http://dave.notik.com

MrTaco’s picture

ah, too bad you didn't go with Drupal

but anyway, there is now a really easy way for parent nodes to have related child nodes, and inline child node view/create/delete/update: with the cck subform module

http://subform.googlecode.com/svn/trunk/subform_trunk.zip

mparikh’s picture

Would this be available in d5.x, when?

- M

seanmclucas’s picture

Still looking for a solution here. I need a sibling-sibling relationship not really a top-down relationship.

I have a patient node type that might have multiple research study nodes associated with it. The patient also might have multiple status nodes associated with it. I need to be able to draw a relationship between one of the patient's studies and one of the patient's statuses. Just in case this isn't clear I'll give an example. Patient A is enrolled in a Diabetes Study and his status for that study is "qualified", while his status for an asthma study that he participated in last year was "disqualified".

I could create another intermediary node to manage the relationships but that just seems messy. Is there any way to easily build in these relationships with CCK or another module like Node Family?

Ideas?

WorldFallz’s picture

Why not just use a nodereference field for the patient(s) field?

===
"Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime." - Lao Tzu
"God helps those who help themselves." - Ben Franklin
"Search is your best friend." - Worldfallz