I'm working on a social networking website based on Drupal and am thinking of the best way manage the massive amounts of data efficiently. I will post the obstacles I'm facing and my solutions to those obstacles.I'd like some input from you guys regarding my solutions. Hopefully this will make some good discussion!
1) Issue: There are literally dozens (probably eventually over 100 or more) data types for a social networking website: statuses, messages, posts, profile information, etc... I wanted to use nodes because I love the API but there's two much 'extra' stuff with nodes (because they're designed for articles) that do not apply to many of these data types. I also don't want 100 different entities to deal with.
Solution: I created a new entity called BSE (basic storage entity). I wrote an API for dealing with this entity that gives me everything I wanted from nodes without the things I don't want (like being able to access each node at node/nid). It allows me to easily create bundles, field them, etc (CRUD). Additionally, any module can define its own type/bundle with hook_bse_info() (my version of hook_node_info()). Because this entity only has a few fields defined by default in the schema (bid, type, uid, created, status), it allows me to use this flexible new entity to manipulate almost any type of data. Hundreds of content types, one API.
2) Issue: I'm trying to figure out the best way to quickly query my BSE bundles. I'm not sure how querying cached entities works in terms of performance so my solution might be erroneous - I need you guys!
Solution: I'd like to set up a server used only for caching BSEs. The server would be running a non-relation database like Mongo for speed. Every time a BSE is saved, a copy of it will be saved to the caching server. Basically, every time an entity is loaded it will come from the caching server. The same is true for search queries, etc. Write to Drupal DB, read from caching server.
There will be more to come, but we'll start here for now!
Comments
This is a very interesting
This is a very interesting concept. Following.
It sound to me all you are
It sound to me all you are doing is creating an alternative to nodes.
That was kind of my idea.
That was kind of my idea. Looking for an entity more flexible, howerver, to accomodate non-article content types easily.
Personally I do not see any
Personally I do not see any gain to your approach. I use nodes for non-articles all the time.
do you just leave them
do you just leave them unpublished so they are not accessible?
It depends, if generally
It depends, if generally inaccessible I would leave them unpublished. If inaccessible to a sub-set of user I would probably use the content access module. Some thing I learned more about this week end is taxonomy access which can address more complicated cases.