Grouping nodes in a hierarchy
I have a database of music that I converted to song nodes (the song content type) and I need to create parent nodes by pulling data (artist, album) from the imported child node and implement an appropriate hierarchy.
Each song node has the following fields
Artist
Album
Song Title
I need to pull fields from the song node and automatically create parent nodes (Artist and Album parent nodes).
An example of the needed hierarchy:
Artist
+ Album
++ Song
++ Song
++ Song
+ Album
++ Song
++ Song
In other words, there will be many artist nodes (parent). Each artist will have his/her albums (child node). Within each album will be song nodes (grand children node). Within some song nodes will be song samples.
I understand that the hierarchy module creates a hierarchy from nodes, but how can I create parent nodes by grouping of the child nodes?
Thanks!
Heirarchy of nodes
Hi,
I'm doing something similar at the moment, building a little personal site with a list of all my vinyl records. The approach I took was this: 3 content types -
- Artists,
- - Discs ('Albums' in your case),
- - - Tracks (or 'Songs', if you prefer),
(actually more, as I included some non-essentials like record label for my trainspotter friends!)
Anyway, the Disc content-type has a CCK node-reference field for 'Artist'. The Track content-type has a node-reference field for 'Disc', so each track belongs to a disc, which belongs to an artist. I also added a 'Artist' node-ref field to Track, as it makes it easier to get the Artist for a Track without having to load the Disc node first.
Then I started building Views, and discovered the wonders of Views 2 relationships, which enable you to retrieve all kinds of related data thanks to your node-reference field. So by adding a relationship on the 'Artist' node-ref field to a Disc view, you can display extended info about the Artist. As I mentioned earlier, I added a second node-ref field to the Track content-type to store the Artist, as it makes life easier, but you could even use relationships to retrieve Artist info by building a Track-Disc-Artist relationship. But it seemed easier to add an extra field (which can be pre-populated on the node-form anyway) and the query times were shorter.
Anyway, that's the way I did it. There are, as you say, other modules out there to help you create relationships between nodes, but the CCK node-ref field combined with Views is an very flexible way to do it. There are also some taxonomy fields in there for things like format (7", 12", etc). Not saying it's necessarily the best way, but that's the route I've gone, so I hope it is!
You might want to look at this case study - http://drupal.org/node/241344 - for some more ideas. It also has some details about Panels setups and Views arguments if you want to build something quite complex.
One thing I haven't decided yet is how to deal with track ordering for an album. I did wonder about creating a node-queue for each album, but I might just settle for 'weight' and keep things simple.
Hope that helps.
Sorry, just realised that I've assumed you know about CCK node-reference fields! Basically it adds a field that stores the nid of another node. You can limit it to certain content-types if you want, and it has a neat auto-complete widget so that you don't need to remember the nid of the artist you want, but can just start typing the name and Drupal will suggest any matching nodes. If you want to create a new Artist while creating a new Album, I found a module called 'noderefcreate' which will create a new node, if auto-complete doesn't find an existing match (it doesn't have a Drupal project page though and I can't remember off-hand where it came from!). There's also 'Popups: Add & Reference', which works well but has a few to many dependencies for my taste. Anyway, I can't think of a project where I haven't used CCK node-refs - Give 'em a go...
P.S. Found it: http://www.darrenmothersele.com/drupal-blog/node-reference-create-drupal...
Wow.. thanks for the reply
Wow.. thanks for the reply Matkeane! I was starting to get discouraged a little. I'll give this a try tonite. I'm a novice so I'll probably ask more questions. I know very little about the relationship aspect of views but from what you're telling me it can be really usable.
Views help
There's a nice screencast about Views and relationships that would have saved me about an hour of swearing the other day, if I'd watched it first, rather than just diving in and fiddling around in Views and trying to guess how it worked!
http://drewish.com/node/127
So every time you want to
So every time you want to display an artist and all his/her albums or an album with a tracklisting, you have to use views right? Does having to use views every time you query that data slow down performance? I just wanna make sure I'm on the same page in understanding all this.
Thanks for the link too!
Views listings
Well, yes, using Views to display a list of nodes is never going to be as fast as displaying a single node, but Views will cache its displays, so things get quicker after the first viewing.
For a tracks-by-artist View (with 1 relationship), I'm getting these times:
Query build time 4.23 ms
Query execute time 20.09 ms
View render time 108.72 ms
It might be possible to speed things up by, for example, passing in the artist argument as a nid rather than a title, but that makes for a non-friendly page URL (eg tracks/17 as opposed to tracks/depeche-mode), although that wouldn't matter if you're only showing the list as a block. Hmmm, let's try that...
With no relationship and a numerical nid argument, I get:
Query build time 3.53 ms
Query execute time 1.7 ms
View render time 109.59 ms
Apart from the query execute time, that's not a huge gain in time.
So anyway, the fastest way (for display) would be to add a HTML list of tracks to the body of an album node and just show that, but you'd lose a lot of flexibility. It all depends what you want to do with your data, I guess. There's not a lot of point adding dozens of content-types and normalizing things to the nth degree if the site isn't ever going to be cross referencing things in complex ways.
It looks like I have to
It looks like I have to perform some trickery here.
I restarted everything and set up my 3 nodes. song, album, and artist.
The fields from the Artist node are:
Artist (new node reference Autocomplete with create)
Album (node reference from album node)
The fields from the Album node are:
Album (new node reference Autocomplete with create)
Artist (node reference from artist node)
The fields from the song node are:
song (text field)
artist (node reference from artist node)
album (node reference from album node)
After setting up all 3 nodes, I used the nodeimport module to import my flat .csv file of song data (song name, album, artist) into my song node. It errored on the artist & album fields because they were node references. Am I doing something wrong with the nodeimport module? It looks like I'll have to import all my data by changing all my song node fields to text, then somehow convert to node references... but i'm not sure how. Am I even setting up my child node types correctly to begin with?
I thought the noderefcreate module creates the parent nodes (album, artist) as soon as the child nodes are created. If this is so, and the only way I can create child nodes is to take out all the node references during import, how is this possible?
Imports and 2-way relationships
Hi,
OK, I hadn't realised you wanted to import a lot of existing data and create, or retain, the relationships at the same time - that can be a bit tricky. As you've seen, the node-reference field is just the nid of the related node but, since Drupal assigns the nid at creation (or import) you won't be able to reference nodes before they exist.
The noderefcreate module intervenes when a form is submitted with a node-ref field that doesn't match an existing node, but I don't think it gets triggered during import so you'll have to find another way round that. It's designed with the node-creation form in mind, with users being able to create new nodes as they fill out the form manually.
Firstly though, I'm not sure I see the need for node-references in both directions, if I've correctly understood your field list. If your Albums have a node-reference to the Artist, you don't need to reference the albums from the artist as well, since you can just make a View of all albums related to an artist.
So, if you want to import all your existing data, and tie it together with node references at the same time, you will need to perform some trickery.
Now, I don't in any way claim that this is best practice, but I had to do something like this recently when I imported about 1000 nodes from a Typo3 site to a Drupal project, and maintain the parent-child relationships and translation groups. It requires a fair amount of mucking around in MySQL and PHPmyAdmin to import data directly into the Drupal database, without using the UI - and so it's quite easy to mess things up! - but you can set the nid and node-ref fields directly.
Depending on how much data you have to import, it might be easier to do things in stages. If you import all your artists first, you'll then know their nids, which you could then add to your album data before importing them, which means you'd then have nids for the albums to add to songs. It's still a bit messy though...
[Edit ] Ah, OK - just re-read your original message. So you already have the album and artist data stored in your song content-type, and now you want to create the album and artist nodes and add the node-refs automatically. Off the top of my head, I can't think of a contrib module that will do that for you out of the box. I think you might need to write a custom module to scan your song nodes and pull the data out of the artist & album fields and save them as new nodes, then grab the new nid and update the song with that as a node-ref. Obviously it's easier to say that, than to write the module to do it!
I see... Wow, I was afraid
I see... Wow, I was afraid you would say that. If the project is going to be this involved, I might as well do it the way I REALLY want it done. The nodeimport/noderef create modules was simply a manual workaround to get what i really want to accomplish. Is there a way to automate all this... by way of a chron event running every minute or so?
The big picture:
I have a live database of music that's constantly being changed (outside of drupal). I'm trying to merge the song data from this database into drupal on a regular basis... (check for changes on song db, and update changes to drupal every minute, every hour, or every day). At first the node import module and node ref create module seemed to be a perfect manual solution of manually importing the changes every day and synching up the nodes. Seeing as how a custom module is required either way...might as well get what i'm 'really' trying to do.
Is this even possible? Or is this time to bring in hired help?
maybe an RSS feed?
Could your live database output an RSS feed with the updates? There are several modules you could use to import RSS data, and the process is designed for regular automatic updating.
I'd have to write code
I'd have to write code against the database to pull the changed data. If I do use RSS feeds or any other method, to update I still have to write custom code to create/update nodes automatically based on the changed data.
Automatic imports
OK, that's a whole new can of worms just got opened!
If you have access to the database and can control the way it exports data you could, as you say, get it to send any new & updated nodes to Drupal on a cron call. Otherwise, as suggested above, perhaps the aggregator module (http://drupal.org/node/289) could poll an RSS feed. Aggregator will create nodes, but you'd still have to add some custom code to deal with the node references (or maybe somebody has a better way to deal with that?). It's definitely possible though...
with a library of over 6000
with a library of over 6000 songs would it be too large an xml file/rss feed for the aggregator module to handle?
probably not
You might have to break the dataset to load them in the first place, but after that the XML feed should only include the records which have been updated or added.
That being said, you'd need to set the aggregator to update existing records which adds more load... sorry, but this is all guesswork on my part. Might come down to a question of update frequency vs update volumes.
Aggregator is good for standard RSS feeds, but might not be the best tool here - maybe look at FeedAPI and Feed Element Mapper, which allow you to customize the input processing.
I see. So I have to manually
I see. So I have to manually load all 6000 song records into the song node, Then load all 500+ artist records into the artist node... and the same for album node?
As far as the Feed Element Mapper/Feed API, does it work well with node ref create? As in automatically create nodes upon feeding?
Okay - some points:
1. I didn't advise loading the song records manually - just divide the data into smaller RSS feeds to avoid choking the system. Whether you could divide the data automatically or manually, I've no idea.
2. One record = one node. Therefore your model would wind up with 6000 song nodes, 500+ artist nodes, etc (not that there's any intrinsic problem with this).
3. I've no idea about node ref create in this situation - haven't used it.
4. You might want to think about an alternative to matkeane's method. I'd suggest:
- create individual song nodes as discussed
- create a taxonomy of artists
- create a taxonomy of albums
If the feed from your database treats a song as an RSS item and artist/album/other information as tags, then:
- songs can be created or updated as necessary (I'd guess the latter wouldn't happen very often?)
- songs can be classified by using the tags for taxonomies (new tags would be added automatically)
5. Or you might want to check out the Collection module (http://drupal.org/project/collection) - haven't used it myself, but it looks as if it might automate a few of the nodereference functions you're looking for.
As you're discovering, there are generally several different ways to perform a function in Drupal and you'll get a lot of different opinions, often from people like me who have no direct experience of what you're trying to do :)
Taxonomy vs Node-refs
As you can see, there are always at least half-a-dozen ways to skin a cat with Drupal! And usually one new module to do so that you hadn't heard of before (Collection, in my case).
Taxonomy is a great way of classifying things, with all kinds of built-in tools in Drupal for retrieval and display. That said, the reason I didn't go that route was that I wanted the Artists and Albums to be full nodes with extra fields (album art, release date, artist biog, etc). With the Taxonomy module, extra info about terms is pretty much limited to a description. So for this project, I'm using Taxonomy for all the basic categorization stuff - disc format, genre, etc, but content-types for the, well, content.
Anyway, I suggested that method because I think it's the best way to skin the particular cat I'm dealing with, but that doesn't mean it's necessarily the best way to deal with your own taxidermy needs!
For what I'm trying to do,
For what I'm trying to do, do you think a behind-the-scenes approach would be better in bypassing drupal to directly write to/update the database?
Maybe it's the frustration with Drupal sinking in...
I'm still trying to figure out how to use Feed API to try that method (I already have my test data in xml form) but don't know how to change my xml to rss for the API feed module to understand. Are there any API feed tutorials out there geared for beginners (the one on drupaltherapy.com is a little too advanced)?
Drupal API
> do you think a behind-the-scenes approach would be better in bypassing drupal to directly write to/update the database?
No! I've used that approach a few times when I needed to initially import a load of data to set up a site, but I think it would be a mistake to maintain a site that way, so I hope I didn't give that impression in what I wrote earlier. Besides Drupal gives you a hell of a lot of tools to manipulate nodes without having to touch the database itself.
I'm not sure how much influence you have over the output from source database, from which you want to pull the data, or how much you want to code yourself, but I can see a couple of options. Either way though, I think you will need some custom code to tie things together and to build the node references.
- use Feed API to import the RSS feed from the database and create new 'song' nodes. Then, either using a trigger when a new song node is created, or on a cron run, you could create a custom module to check the artist and album fields - linking existing ones, or creating new ones.
- use XMLRPC to build a custom import system that would poll the music database, pull in and parse the song nodes and deal with the node-references all in one hit. Not the easiest option, but the most flexible, and would require access to the source database system.
It seems sensible to build on the Feed API module as much as possible - that will get your song nodes into the Drupal database. And, thinking about it, messing around with triggers may be unnecessary, as your custom module should be able to look at and modify the node object of a new song just before it gets saved to the database with the 'presave' op in the node_api (http://api.drupal.org/api/function/hook_nodeapi/). While that might seem a bit complex at first, the code from noderefcreate module should be a very good guide as to how to go about it.
OK I feel a little more
OK I feel a little more optimistic. Thanks.
I have complete control of the file extracting data from the database. It can be a txt file:
song: Tin Man
artist: America
album: America's Greatest Hits
OR an xml file:
Tin Man
America
America's Greatest Hits
I can configure it to recreate every time a song is played... I can configure another file to be created everytime a new song is added to the database. So in other words, I have complete control of the file(s) being created that extract database information and can create any type of file with this information and how often it(they) get created. Have you heard of SAM broadcasting software? It's done real easily with a little variable plug & play. Anyhoo... I hope that was the hardest part because I got that covered.
Taxonomy term fields
can be added with http://drupal.org/project/term_fields
But I think I've confused things enough, so will step out of the discussion now :)
Extending taxonomy
You know, as soon as I said that Taxonomy was limited to only a term and description, I had a feeling that somebody would point out a module that allowed it to do more than that! But somehow, I still have a lingering feeling that extending taxonomy in that way is a bit of a hack - in that it's becoming a content type, but without the API support of a node, or without being properly integrated with CCK. Still, it's useful to know that module exists for cases when you want to associate a bit more info with terms.
_
Just an fyi-- but with fields in core for d7 this will be the standard rather then an exception. Everything will be 'fieldable'-- users, profiles, nodes, comments, taxonomy, etc.
_
Don't be a Help Vampire - read and abide the forum guidelines.
If you find my assistance useful, please pay it forward to your fellow drupalers.
Anybody interested?
http://drupal.org/node/423840