Media Gallery uses 12 different fields to work:
- Description: Long Text
- Gallery media: Multimedia asset
- Gallery format: List (Text)
- Lightbox title and description: Boolean
- Column number
- Rows number
- Media information: List (Text)
- Allow downloading of the original image: Boolean
- Create a block of most recently added media: Boolean
- Column number
- Rows number
- Gallery collection: Term Reference
This is 12 Join, in 12 different tables, which is very bad for performances. Also, what happen if a user deletes a field with the field API ? Let's say Rows Number, is the gallery will continue working ?
We should put all the requiered field into one table, creating an entity type 'media_gallery' instead of using the base table 'node'. We can then store the required fields in this entity, and use field for Term Reference and Multimedia asset.
What do you guys think ?
Comments
Comment #1
sylvain lecoy commentedIt also introduce a lot of overhead web server side, for instance the Column number field in case of a localized website will be:
$node->column_number['und'][0] . It makes no-sense to localize a Number, and even worse a boolean.
Comment #2
JacobSingh commentedUnfortunately, this is a larger problem with the fields api. Using fields has several advantages though and allows for greater flexibility and maintainability. For instance, if someone wanted to add another option to the "Media information" field?
Fortunately, much of the performance pain is mitigated through caching layers. While I share your general concerns about the fields api, I don't think this is going to change in this module.
Comment #3
sylvain lecoy commentedI admit that for the media information it is great, but for rows number and column number this is not, same for block creating and allow downloading.
Keeping field for Media information, Term reference is a must do. But having a new entity "media_gallery" with its own table (rows number, column number, allow downloading, ...) would be awesome. I understand the cache layer but caching should be used sparingly, its not a good way to make software scales.
Maybe I can try to provides a patch ? Make some benchmark comparisons, submit results to the community, and see if we can improve this ?
Comment #4
David_Rothstein commentedI'd be surprised if replacing a few fields with a custom database table helped performance - if anything, seems to me like it would be the opposite (we'd be adding additional queries that are uncached by the field system, so unless we added our own caching layer this would tend to result in worse performance). Happy to be proved wrong, though :)
Also, I would think that things like row and column numbers make at least some sense as fields (if someone wanted to configure e.g. the allowed options for number of columns, it's better to let them do that rather than leave it hardcoded, right?).
Comment #5
David_Rothstein commentedAlso, I'm not sure why you're assuming that 12 fields in 12 different database tables means that there are 12 joins.
That's not really how the field system works - it doesn't use joins at all as far as I know; for example, see field_sql_storage_field_storage_load() for what happens in the default storage engine when an entity's fields are being loaded.
And if there is any custom code out there that is using 12 joins in this case, I would think it's very likely that it's because the code isn't using the entity/field API correctly.
You might want to look at http://groups.drupal.org/node/17564 where this issue has been discussed in great detail.
Comment #6
sylvain lecoy commentedThank you for answering me and providing link to further information about this topic.
Following your sentence:
I tried some experiments. We can't delete a field so this is a good thing (protected by code). Unfortunately, we can't modify the field, drupal returns:
The field Number of columns is locked and cannot be edited.We are loosing the flexibility of fields. For instance the block module allows max 4 columns, I have a design where I need 5, but I can't modify it without hacking the field definition (I can't overrides by UI).
I'm not speaking about the formatter, nor the widget, just the way they are stored. We create a Single table for this field, and here is the structure:
There is a big loss of space, first we don't care about entity_type (this is a node), neither bundle (this is related to a gallery), deleted is just useless (we can't really display a gallery without this), entity_id is where the JOIN is processed*, we don't really care about versionning our gallery structure -> revision_id, language is not usefull as well, delta is not used (no multiple values). Only value is interesting. So this is 7 attributes which are not useful in the 8-tuple but still loaded by the API**.
About the
field_sql_storage_field_storage_load(), I effectively see that there is no JOIN, but, even worth, a loop with SELECT executed in series. And we can effectively see with the query log that it chain load cached fields. I think database Normalization is the good way to implements a Media Gallery, like users, comments or nodes with their own tables, Media Gallery would become an entity (then you can still attach fields like "Media information" and stuff. But remember that localize a boolean or a row number is over-heading).If this job is done for, let's say description, lightbox, column number, rows, allow downloading, create block, column block numb, rows block numb. 8 fields which can be retrieved in a single SELECT, instead of having 8 chained SELECT for each fields. Moreover, the 8 * 7 attributes are not loaded (nor stored) which is a huge boost performance to me. Also, all the adapter logic from the field API is skipped (putting it in an array, with the language as key, and deltas values, etc.).
Let me work on my way, I'll provides you some results, I'm very confident about this and I am pretty sure that it will benefit. Do the caching work as well for identified users and administrators ? This is the counter part of relying too much on caching, versus real time systems.
* Not a stricto sensu "SQL JOIN", on the field_sql_storage_field_storage_load, its actually a clause added as a condition on the select statement:
->condition($load_current ? 'entity_id' : 'revision_id', $ids, 'IN')** According to this article:
http://www.morningtime.com/Drupal-6x-Performance-Guide/II-Advanced-Drupa...
Reference: 56 Drupal Performance Tips (http://www.morningtime.com/Drupal-6x-Performance-Guide/513)
Comment #7
sylvain lecoy commentedJust want to highlight that Row and Column fields are not editable. Drupal says:
Comment #8
Taxoman commentedComment #9
sylvain lecoy commentedWe should redesign the media_gallery, instead of using fields, creates an entity.
I can work on this.
Comment #10
lsolesen commentedA patch is welcome. We cannot guarantee that we will change how media_gallery works though, as maintaining our own entity is also extra work.
Comment #11
ivnish