I'm needing the ability to draw many-to-many relationships between different node types and am wondering if anyone is working on this functionality.

Example: node type (a): company. node type (b): product. A company could have many products and each product can be sold by many companies. Of course, this could be tracked on a custom basis (e.g., by creating a linking table and hard-coding references to that table in both the company and product modules), but a more general solution would be desireable. Ideally, such a solution would allow multiple parameters to a relationship (e.g., for each company, the current selling price of the product).

Adding this functionality would significantly enhance Drupal, making it capable of handling complex relational data sets.

The flexinode module provides much of the needed functionality: defining new "tables" with fields of various data types.

Has anyone taken a stab at coding a "relationship" module that would extend a flexinode-like approach to generic node-to-node relationships? Anyone interested in collaborating on this?

Comments

robertdouglass’s picture

Hi Nedjo,

This is, in my opinion, the absolute most important feature that we could possibly add to Drupal at this point. I already use Drupal as a rapid-prototyping system for custom data models, but I always write custom modules to handle relationships. As you know, while easier than with some web frameworks, it is still a time consuming task.

Here are some thoughts that I've had about what this functionality might look like:

  • The relations must be defined in metadata. It is not enough to be able to start at a node of your choice and 'link' it to one or more other nodes of your choice. Rather, one must be able to say in advance that Companies have Employees and that Employees have Addresses, for example. Then, Company nodes would automatically offer the ability to link to Employees.
  • It must be possible to distinguish between 1.n, n.m and, ideally, inheritance relationships.
  • The relationships should be bi-directional; a Company is aware of its Employees and an Employee knows for which Company s/he works.
  • It should be possible to make changes to the data-model of a live system with minimal and foreseeable affects. For example, adding a data type PensionPlan and linking it to Employee should automatically add the possibility of choosing a pension plan to Employee screens (the default ones, at least). Deleting the PensionPlan data type should clean the tables relating to Employee of all PensionPlan data.

I feel that Flexinode is a great module, and that its interface for defining node types should be used for development in this direction. I do not feel, however, that the data should be stored in serialized form. I think, rather, that database tables should be automatically created and maintained. So, when I create types Company and Employee, these tables would be created in the db. We would have a table Types which would track the different object types (Type_id, name, [plural], inherits_from).

If I then said that a Company can have Employees, an entry in the Links table would be made; (link_id, type_id_1, tpye_id_2, single_or_multple) or (1, 'Company', 'Employee', SINGLE). A foreign key column would be added to the Employee table as well. The Links table is necessary for all sorts of things. For example, making sure that Employee screens offer the possibility to link to a Company. It is also necessary for the administration interface so it can show which relations have been created.

Then, if I change the relationship between Company and Employee to be n.m (as a person sometimes works for more than one company), we would update the Links table (1, Type_id_1, Type_id_2, MULTIPLE), create a Company_Employee table (Company_id, Employee_id), look in the Employee table for an existing foreign key, and finding it, move the links to the Link table. In other words, the system should be flexible enough to manipulate existing data intelligently.

There are other considerations as well. We will need a Fields table to track every individual Field in these automatically generated tables. Why? Because, depending on data type, we will want an intelligent default user interface for data entry. We will need to have a table like this; (Field_id, Type_id, Data_type, [options...]).

Taking the above mentioned approach would have many benefits. A large amount of SQL would be moved out of the insert, delete and update hooks of individual modules and pushed down to a lower level. It would be unnecessary to make contrib modules just to represent a data object. Making tabular representations of the data would be easier than with Flexinode where the data is serialized. Drupal would shine as a web application framework and we could throw away the term CMS.

I have used a framework that does exactly what I've described above and the potential is astonishing. An auto manufacturer was holding a beauty-contest to award a major contract for the development of a web application. The three companies in the room were IBM, T-Mobile (I think) and the company I was working for at the time (I wasn't there). During the day-long presentation from the auto manufacturer, my colleagues used the data modelling system (like described above) to translate what was being demonstrated into a data model. All three companies had a chance at the end of the day to make a short presentation about what they had to offer and what experience they had in the specific field. The first two contestants showed a lot of credentials and some very impressive PowerPoint slide shows, and my colleagues showed a functioning web application which covered 100% of the data collection needs and ca. 60% of the workflow - already implemented. The other two companies never submitted final proposals - the contract was awarded on the spot.

Anyway, I diverge. The point is, with Drupal, I'm already able to make applications in a week or two that others make with things like Struts in a month or two. The ability to manipulate a data model and show relationships without having to write the SQL every time would be a great boon.

- Robert Douglass

-----
visit me at www.robshouse.net

nedjo’s picture

I like the idea in this case of generating new link tables, rather than the flexinode approach of using a generic table.

I agree that relational support would be a major step forward. MySQL and Postgresql are after all relational datase management systems, but like those of most other CMS softwares (and aside from the limited linking possible through taxonomies) Drupal's content types are basically flat.

Getting this right would be more work than I have time for at present, but I'd be interested in contributing to discussion and potentially taking on a portion of coding.

What we'd need is a lead.

Candidates? Volunteers?

javanaut’s picture

I find it quite interesting that after writing a module that handles node relativity, I come across this post and find that I've implemented quite a bit of the features that you mention. I haven't really touched flexinode directly, but have node associations implemented. You've faced similar challenges to what I'm currently faced with in my current project - complex data structures, needing to rapidly alter core structures and still have things work, etc..

I guess that's why software design patterns are so popular nowadays..

alexmc’s picture

I would very much like to be kept informed of your progress -even if it remains closed source. I am a C++/java/perl web person and I am finding this sort of thing missing from Drupal. I thought that Flexinode did this already but I guess I was wrong.

moshe weitzman’s picture

i often have this need as well. one approach worrth exploring is to create a new field type for flexinode (i.e. an include file) which shows a dropdown of options to the user. Instead of having the admin define each option though, we let the admin define a SQL query returns the option names and values. The admin should decide if one or more can be selected, and maybe some more bits.

Then flexinode will present the fied to the user and save the data. Sounds simple enough ...

robertdouglass’s picture

I'm all for extending flexinode to have some or all of the functionality described above, which I've suggested here before, and I've even simulated what you just described by manually creating the select items when the set of possibilites isn't that large. This worked marvelously for the projects where I used it, but isn't suitable for anything where the user can create objects that increases the set of link targets. Your suggested approach would cover this, and that would already be a very good solution.

Still, I feel that serialized data is not the way to go in the long run. People will immediately start building huge data models with the functionality you just described, and it won't be long before the limitations of the flexinode approach show, I predict. Would Drupal.org be able to function if all 13,000 nodes were in serialized form? Furthermore, reporting views of the data would be exceptionally difficult; I often have several sortable/searchable/paginated tables in my applications... we can forget that. Therefore, I think it is probably time to start discussing a more long term solution. Ber Kessels has already been thinking about this issue, from what I could gather from the dev. mailing list. Is that right, Ber?

- Robert Douglass

-----
visit me at www.robshouse.net

moshe weitzman’s picture

flexinode is getting an overhaul in 4.6 and might actually support what you need. certainly the data module will be cleaner. JonBon mentions this in his Battle Plan and also see his content construction kit specification. alas, this page is currently broken. he will surely fix it soon.

alexmc’s picture

> Still, I feel that serialized data is not the way to go in the long run.

seconded

robertdouglass’s picture

I recently read the flexinode.module code very carefully for the first time and came to the conclusion that moving the responsibility for generating sql from the flexinode.module itself into the field includes would allow us to completely get rid of serialized data. This would enable any further efforts made on the node to node relationship front an effective compliment to flexinode.

- Robert Douglass

-----
www.robshouse.net
www.webs4.com

moshe weitzman’s picture

i think this will have to happen at some point. look for some progress during DrupalCon this month.

matteo’s picture

In the past I worked with a product named 'Rochade' (http://www.rochade.com), that was a kind of network database.
It let you create relations between items and specify the versus of the relation, so it was possible to define a start and an end and traverse all the tree displaying all the affected items.

The syntax was proprietary, but, in a node you simply had to specify the number of the items connected, and the versus (with a simple > or <)

Adding such a feature to Drupal would be FANTASTIC !!
You could easily creates sites like Linkedin (www.linkedin.com)
where you can manage relations...

I can help with the development (if time permits)

Matteo

robertdouglass’s picture

The system I described above also had a query language that was somewhat similar to XPath but let you declare a node result set from the starting point of the current node. For example, if the current node were a Company, you could write:

[Employees{Address/State == 'Michigan'}]

The syntax is probably off, and it is also proprietary, but that would have returned a list of Employees who live in Michigan. This would theoretically also be possible in Drupal, though I'd want to consider this aspect very carefully before coming to any design decision.

- Robert Douglass

-----
visit me at www.robshouse.net

nedjo’s picture

Node relationships are now possible through javanaut's relativity module. Great work!

matteo’s picture

Really impressive !! I'll take a closer look....
Matteo

javanaut’s picture

Be sure to check out what's in CVS this afternoon. I just uploaded a revised version with a connected node query tool. All I need now is a network visualization feature so you can see the relationships that are being searched.

bomarmonk’s picture

I am using flexinode for an "organization contact" node-type. For each organization, I would like to add children nodes called "individual contacts," and be able to display these results in the corresponding table view for flexinode.

I've seen a flexinode-relations .inc file, http://drupal.org/node/8468, but I think this merely adds a link to the child node, and does not support a relational table display of the two node types. It also causes problems with drupal 4.6.

I would try the node relativity module, but it doesn't work with 4.6.

Any suggestions? Thanks in advance!

Bèr Kessels’s picture

http://drupal.org/project/clipper does this, and more

---
Next time, please consider filing a support request.

[Bèr Kessels | Drupal services www.webschuur.com]

bomarmonk’s picture

Thanks for the suggestion, but I tried clipper with 4.6 and had difficulties. Has it been updated or should it work with 4.6? I would be glad to try it again.

As for node relativity: is there really an issue with it under 4.6? It seems to be working! I've just installed it, deciding that I did not want to assume incompatibility based some rather ambiguous forum posts. I'm glad I've tried it, because I have zero problems with it so far.

I'm still wondering how I might get these relationships to display in the flexinode-table view (or similar view)? Any ideas?

Thanks again for your input.

ec’s picture

Just try clipper (ok , it's not very to figure out how it works) but it works fine on 4.6. Thanks Ber for this nice piece of code. regards. eric

bomarmonk’s picture

I just tried reinstalling the clipper module and it still doesn't work for my site (maybe its a conflict with another module?)

However, check out my bug report for the clipper project (http://drupal.org/node/20463). I'm still having this problem and Ber seemed to agree that clipper does not work with 4.6. Is there an update out there somewhere?

Please let me know how you got it working in 4.6, and I'll give it a try.

ec’s picture

Well I can confirm that it works for me on 460.
- assign a flexinode type to be a clipper
- the choose a vocabulary where clipper term are to be store
- then create a new clipper enable node, it will store its title as a term in vocabulary choosen above
- then create a new node with that vocabulary enable and choose the new term just created
- you're on
- go to the previously created clipper enable node an you will see in the body "children link" and below the node created as a child !

It's terrific !

One idea, do you have flexinode on your site ?

regards, eric

bomarmonk’s picture

I do have flexinode. The errors occur when I first enable the module in admin>>modules.

How about the table view for flexinodes? Can clipper be integrated with the table view of flexinode so that children are listed below parents in the table? Just curious...

ec’s picture

Hey Bomar, my previous mail was for 460. I make an install on a brand new 461 with only flexinode an event and found this warning when creating a node warning: in_array(): Wrong datatype for second argument in /var/www/drupal-4.6.1/modules/clipper/clipper.module on line 72. I must check a bit deeper. eric