Reviewed & tested by the community
Project:
Commerce Core
Version:
7.x-1.x-dev
Component:
Product reference
Priority:
Normal
Category:
Feature request
Assigned:
Unassigned
Issue tags:
Reporter:
Created:
24 Oct 2011 at 15:52 UTC
Updated:
8 Jan 2022 at 15:08 UTC
Jump to comment: Most recent, Most recent file
Comments
Comment #1
rszrama commentedYou can already do this. There's an order_id column on the line item table that joins to the order table via the Commerce Line Item: Order ID relationship.
Comment #2
valante commentedThanks for the quick reply!
The field might be there, but I don't see the option under Relationships. I created a view based on Products, then added a relationship to Line Items. At this point, here are my options for further relationships:
Commerce Line item: Referenced product
Commerce Product: Creator
Commerce Product: Node referencing products from field_title_products
Content: Referenced product
No Order that I can see. What am I missing?
Comment #3
rszrama commentedAhh, the difference is the relationship appeared for me when the base table was Commerce Line Item, not Commerce Product. I didn't even know the reverse relationship worked from Product -> Line Item. I'm not sure why the second level relationship wouldn't be present, but perhaps it's just a limitation of Views? I wouldn't think so, but ya never know. You might try pinging bojanz about it in IRC, #drupal-commerce.
On a side note, any reason you can't make Commerce Line Item the base table? That's going to get you all purchased products. Or are you trying to build a report that includes products that have never been purchased?
(I'm not sure there's really a need in Commerce here, unless it turns out this is something we screwed up with our data. The relationships exist, but if it's not accessible, I'm not sure why.)
EDIT: ahh, the reason I didn't know the reverse relationship existed is that it doesn't. I don't know why the Commerce Line item: Referenced product relationship is showing, but it doesn't work (hence the errors). That's a relationship for going from a line item to a product, not the other way around.
Comment #4
valante commentedStranger and stranger. I now notice that the reversed relationship works only partially--it multiplies the rows as if the Line Item table was inner joined correctly, but there are no Line Item fields available for display.
You guessed right that I need this for a report of all products, even those that never sold. Commerce is sorely missing a strong reports package (AFAIK). Trying to build one is proving challenging, and this direction of relationships is one thing I'm constantly missing.
Comment #5
joachim commentedI was looking at a similar problem over at #1318358: provide a views argument to filter products in or out of a given order. Some of my earlier patches over there may be relevant here. (The final patch over there won't be, as that has to be quite specific about the way it joins.)
The first step to fixing this sort of problem is figuring out the query you want Views to produce.
Do you need to limit by order ID within the join, or are you trying to show *all* line items (in every order, in every cart) for that product? The latter will produce multiple rows per product unless you're very specific with filters.
Comment #6
EmanueleQuinto commentedWell, as pointed out, without the relationship between Product and Commerce Line is not possible to create reports starting from product itself.
I can see some scenario where this relationship it's necessary:
In summary, with the available relationship we can use the Commerce Order or Commerce Line Item but not Commerce Product to build a view like this:
So what we need a is a join to show *all* line items (in every order, in every cart) for any product; filters will allow to limit the rows.
Comment #7
joachim commented> I want a list wiht ALL the products related to the entity with (if any) orders:
For that you need a relationship based on the product reference field.
If you then go from product to orders, you will get multiple rows per product.
Without the code in front of me, to get the relationship, you need:
- define field_data_commerce_product, the field table for the product reference -- it's possibly already there thanks to Field SQL module
- define its join back to the product base table (again, might be there already)
- define on it a relationship to the line item base table.
Where it gets complicated is if the relationship join to the line item needs limiting in any way. Should carts be excluded (ie, only orders that are complete)? This may be doable with filtering on the order once you add that relationship to your view; or it may be a custom relationship handler is needed.
Comment #8
valante commentedI see no reason to assume that completed orders are the only valid use case. For example, I may want a report of all products (including the ones that were never ordered), and how many times each product was abandoned, namely, has an order that's not completed or pending.
For that I would need the following:
Product -> Line Item -> Order -> Filter on Order status <> completed.
I think creating the basic relationship via a handler and leaving the rest to filters is the way to go.
Comment #9
EmanueleQuinto commentedI'm not completely sure but the attached patch add the needed relationship to product based views.
The first part of the patch expose product -> order_line relationship in commerce_line_item_field_views_data hook. Indeed the same code is present in the commerce_product_reference_field_views_data hook (line 11 in ./product_reference/includes/views/commerce_product_reference.views.inc).
The second part remove a line in commerce_product_reference_views_data_alter hook that, for reasons that are not clear to me, excludes relationship with 'commerce_line_item' entities.
I'm using plain vanilla Commerce 7.x-1.0, Views 7.x-3.0-rc1.
Comment #10
valante commentedPlease review the patch from #9 and submit if it's good. I find myself yet again in dire need of this relationship.
Comment #11
rszrama commentedThe patch in #9 assumes too much; keep in mind the Line Item module has no dependency on any module pertaining to products. Also, I'm not sure how necessary this all is now with a much improved http://drupal.org/project/commerce_reports and changes to our Views handlers over the last year. Closing for now, but we can always reopen if this remains impossible or a better approach isn't devisable.
Comment #12
geynen commentedThe patch # 9 does not work, rather it worse (the relationship did not allow order - line item - product), what I need is to be able to make the relationship of product - line item - order.
please anyone who can help, sorry for the English
Comment #13
rszrama commentedSorry, I'm going to reclose this per my comment in #11. Please look at the Reports module to see how it's doing it. This particular issue is well over a year old now and isn't something we'll be adding as a new feature to Commerce 1.x.
Comment #14
geynen commentedInstall as you indicate in #11, is an excellent module, I like a lot, but still does not give me a report of the type that I need (I'm sure that others in need).
pose such a case: Report sales (including those that were not sold)
I think it might raise other cases, but just this example, and I think if you need to create a view that allows relating the product commerce - (line item) - order.
would greatly appreciate it please review it and possibly make a patch
PD: investigate all threads of commerce and found no answer I work
Comment #15
alan_hugo commentedThe case arising # 14 is required for many situations.
If we create a view to try starting from a product relating it up to an order (to avoid the exclusion of those products that were not ordered). just the situation we find that the relationship dies in line items.
If we look at raising # 11 all views that use started from commerce order - line items - product, but there is no the case (product - line items - order)
So what happened? with backward relations (product - line items - order). Commerce 1.x. not see this as a novelty (# 12), or did not imagine the countless cases where this relationship would be useful, and to provide flexibility to the module.
Comment #16
geynen commentedI found a solution, maybe not the best way, but I work myself and share it for other newbies like me, have things a little easier or maybe improve it.
analyzing the table commerce_line_item in the database we see that there is no product_id but looking line_item_label field which contains the SKU for each product (in my case I use AutoSKU, which assigns a unique identifier for each product and in my case product_id equals the SKU).
this requires editing the hook_field_views_data () in the file commerce/modules/line_item/commerce_line_item.module and add the code snippet for adding the link.
being as follows:
recommend checking if the hook is running (I had problems because other modules inabilitaban the hook) and cleaning caches to save many headaches. (active line to verify drupal_set_message ('ok');)
Comment #17
geynen commentedComment #18
nigelw commentedThe solution proposed in #16 is a hack to the commerce module and not a solution. I also disagree that this issue is moot because the commerce reports module is now available as suggested in #11. Commerce reports is great but it doesn't provide this relationship and is overkill for small stores.
If we can get the following relationship to work: commerce order - line items - product, then we should be able to get: product - line items - order
Comment #19
dydave commentedHi guys,
Thanks to all very much for your feedbacks and follow ups on this issue.
I'm not sure whether I missed something here or perhaps didn't get your points, but it seems people have been complaining in this post that there was no reverse relationship from products to line items.
Honestly, I really didn't spend much time looking into that, but in a very short amount of time, I managed testing the following code on a local test environment and it seems to implement correctly a reverse relationship:
From products join with line items on SKU = line_item_label (If I'm not mistaken from what I understood in previous comments).
Since I didn't want to disrupt the existing code and change commerce's files, here is the version based on hook_views_data_alter that allows adding the relationship without any change in the commerce module (custom module enabled).
Simply place this code in any of your custom modules and replace [MODULE_NAME] with your own custom module name.
Otherwise, please find attached to this comment a patch against commerce-7.x-1.4+29-dev (2013-01-26, at 365eed9) which extends the existing SKU views definition to add/implement the relationship, see code in commerce_product.views.inc, line 43.
File attached as: commerce-relationship-product-line-item-1319810-19.patch.
I tested both approaches and they all seem to work as expected, in particular, I checked the query generated (SQL code, especially the JOIN clauses) in Views preview while changing the relations in my test product view.
I'm not sure at all whether I was accurate in this reply and didn't completely miss the point here, so I have changed the status of the ticket to needs review, since I would greatly appreciate a review on this and getting me back on the right track if I've been off topic.
Feel free to let me know if you would have any questions, comments or issues on any aspects of this comment, suggested code or patch, I would certainly be delighted to provide more information.
Any feedback, testing, comments, further issues, ideas, suggestions, recommendations or questions would be surely highly appreciated.
Thanks to all in advance.
Comment #20
summit commentedHi, and how to go from Product-> Line item -> Order please?
greetings, Martijn
Comment #21
rszrama commentedMy point in #11 wasn't that people should use Commerce Reports but look at it to determine how it configured the Views relationships.
Comment #22
nigelw commentedSorry, rzarama. You are right. I misread your post. The views in commerce reports does use the relationship commerce order - line items - product
So the question is why it cant go the other way around?
e.g. for core you can go content - author - profile, as well as profile - author - content
Comment #23
rszrama commentedDo you need it to go the other way around, though? If you have the relationships established, can you not generate whatever output you need? (Not saying we can't add a relationship, but wondering if you might not have an intermediate fix until we can get to it.)
Comment #24
nigelw commentedOh for sure. As an interim solution I can go the other way (commerce order - line items - product). However I found myself creating a view starting with entity product and built a very complex view with a ton of exposed filters, charts et. Then I went to go add the order relationship to get some more fields and found I couldn't. I now have to go back and rebuild the complex view starting with order entity first. A pain, but not impossible. :)
I figure many others may find themselves going down this road in the future only to find a dead end and then have to start over. Would be great to support the other way when possible.
Comment #25
nigelw commentedUpon further review going the other way (commerce order - line items - product), will only show products that have been purchased. Where as I need to show all products regardless if its been purchased. Essentially I am trying to emulate the product admin page at /admin/commerce/products that also includes total purchased and revenue.
I have tested the patch in #19 and this doe not give a new "line item" relationship.
Comment #26
dydave commentedHi nigelwaters,
Thanks very much for your feedback and testing for the patch for the approach suggested in #19, it is certainly highly appreciated.
I'm not sure exactly what you meant by:
Could you perhaps elaborate a bit more on this?
Just a few notes concerning the patch and the posted code for the implementation of hook_views_data_alter, at #19:
This is the most direct solution I could find within a short amount of time, so at first, the idea was to try to add as quickly as possible a JOIN in the SQL query between the label and sku of the commerce_line_item and commerce_product table, respectively.
a. The patch/code posted at #19 adds a relationship in the Product group on the SKU field, since the base table is commerce_product and the Line Items don't have any explicit relationship declared with products, they don't appear in the possible choices for relationship.
So after applying the patch or adding the code to a custom module, in any products view, the relationship with Line Items would be provided by the SKU field.
Practically speaking, after applying the patch you should see a new choice in the add relationship form dialog in views, for the SKU field and this is the one that should be selected to add the JOIN on commerce_line_item and would correspond to this particular request/requirement.
b. To show all products even the ones that haven't been purchased:
Require this relationship should not be selected in the SKU field relationship settings form dialog in the view. I have tried sorting on Line Item ID and it seems to work fine.
c. Cache should be flushed after applying the patch or adding the implementation of hook_views_data_alter to a custom module.
d. Implementation of hook_views_data_alter in a custom module, as suggested in #19 would require hook_views_api to be implemented as well in the module file.
I have tested both suggested methods (patch and hook_views_data_alter) and they seem to work as described, adding a relationship on product SKU (to commerce_line_item label).
Now, as I mentioned above, this was a rough initial idea, mostly taken from a direct query angle, but with a little bit more time I was able to take a much closer look at the Commerce data structures and associations between products and line items.
It seems for the entity type
commerce_line_item, the bundle typeproductis added by the commerce_product_reference module which is required by the commerce_cart and commerce_checkout modules (among others).So I believe, in this particular case, it would probably be wiser to investigate in the direction of the Product Reference module, since we would be essentially looking at line items referencing products.
See implementation of hook_commerce_line_item_type_info,
commerce_product_reference_commerce_line_item_type_infoin commerce_product_reference.module, line 1099.On top of that, the Product Reference module also seems to add the commerce_product field (of type commerce_product_reference) on the commerce_line_item Product if you look at:
commerce_product_line_item_configuration, in commerce_product_reference.module, line 1124, and more particularly the doc comment block:corresponding to a callback (configuration) defined by the Line Item module, for more information, see:
commerce_line_item_type_callback, in commerce_line_item.module, line 528Which brought me to the integration of Product Reference with Views in commerce_product_reference.views.inc where it seems the relationships for Product Reference field types would be handled.
In fact, there is already an existing reverse reference relationship from Entity to Product defined for each Product Reference field instance (that we have all probably used with nodes):
See inline comments in:
commerce_product_reference.views.inc, line 84
Almost to the point, I would like to particularly draw your attention on the next few lines in:
commerce_product_reference.views.inc, lines 92 to 94
pretty much skipping the commerce_line_item Entities from being provided a reverse relationship.
To test this quickly, I tried commenting these 3 lines, flushed the cache which displayed the expected relationship that I added:
Commerce Product: Commerce Line item referencing products from commerce_product(see
'!entity_type referencing products from !field_name'in commerce_product_reference.views.inc, line 104)I have tested a little bit and tried to add fields, sorts, filters, other relationships, etc... and it seemed to work fine.
The query generated by views seemed to work as expected as well, adding two JOIN relationships with commerce_product field and commerce_line_item tables.
At this point, I would be greatly interested in finding out more about these particular lines of code and the reasons why they would be needed. I haven't investigated much further than that, but I wouldn't necessarily see a problem to providing a reverse Product Reference field relationship to Line Item Entities. Would they be excluded for any specific reason?
I would greatly appreciate if anyone could let me know if I overlooked or missed anything related with implementation choices for Line Item Entities or the Product Reference Field.
In any case, please find attached to this comment the corresponding patch against the commerce version at 365eed9 which sums up this second approach, simply removing lines 92 to 94 from commerce_product_reference.views.inc:
commerce-relationship-product-line-item-1319810-26.patch.
In short, after applying this patch and flushing the cache, a new Product Reference reverse relationship should be displayed in the Commerce Product group , most likely called:
Adding this relationship seems to provide a solution for this particular request.
I would greatly appreciate to hear any of your questions, comments, feedbacks, suggestions, recommendations or concerns on the attached patch or any of aspects discussed in this comment, I would surely be glad to provide more information.
I would certainly be highly interested if maintainers could shed some more light on this particular approach and the code/choices of implementations discussed in this thread.
Any feedback, testing, reporting, comments, is highly appreciated.
Thanks in advance.
Cheers!
Comment #27
nigelw commentedThanks DYdave for your effort on this. I have applied the patch and have followed your instructions. Unfortunately a lot of this is over my head and I don't even know what to report back other than I cannot achieve the view I require. I do not see the new relationship, nor do I see the Commerce Order: total. The patch applied without issues, caches flushed etc...
Essentially I am trying to modify the default Product listing view that comes with commerce core (found at /admin/commerce/products) add just add a "Total Number Sold" and "Total Revenue" for each product in the list.
If you apply your patch, can you achieve the two new additions to that view? If so, can you paste an export of your view here so that I can see how you achieved it?
Comment #28
nigelw commentedI have been investigating this a bit more. It appears that this is a much larger issue than just one missing relationship. There are many missing relationships between all of the entities that commerce provides
Entities in core commerce:
Product entity
Customer profile entity
Line item entity
Order entity
Payment transaction entity
Another example or where I cannot get the relationship is in the default Customer Profile view. In Customer Profiles try adding a relationship to Orders to get the total amount that a customer has spent on your site. Doesn't exist.
Abstracting each of the components of Commerce into their own entities is a great architectural improvement. We just need the views support to create the relationships (joins) to get out the info we require.
Unless I am missing something here, we should be able to start with any base table and branch our way out to any of the commerce entities. rzarama, do you have any insight into this or am I barking up the wrong tree?
Adding better issue title.
Comment #29
rszrama commentedReverse relationships aren't a given in Drupal in general; we do them in some cases, and there's no particular reason not to do them in others, but oftentimes these challenges can be overcome. I keep pointing to the Reports module as an example, and it does have such a customer report where you can find your most valuable customers. Honestly, it doesn't make sense to do it from the customer profiles page, because any given customer is going to have 1 to n profiles on that list - to have an effective report, you'd only want one row per customer, and to accomplish that you'd make a View on the commerce_order table aggregated on the uid column.
I'd think our time is better spent just addressing the actual needs that come up instead of falling into overengineering by making every potential relationship work in reverse just to say they do. The reason you can only go from orders to customer profiles is because the relationship isn't from entity table to entity table but from entity table to field table to entity table - in other words, the relationship is a result of the customer profile reference field, not the fact that orders and customer profiles exist as related entity types.
Comment #30
dydave commentedHi nigelwaters and rszrama,
Thanks very much for your follow up replies and further clarifications on this ticket, it is surely highly appreciated.
I'm glad I could spend a little bit of time looking further into this particular issue and documenting a bit more on the Product Reference module.
Following up with #28,
See more at: #1917156: Views extra module to add a few more Commerce views relationships
Point 1 would correspond to the use case described at #26 and Point 2 to the one at #28:
To fulfill this particular feature request, and since I assumed
I went ahead and posted a feature request in Commerce Extra to add a new sub-module that would provide these relationships and I certainly invite you to try out the patch at: #1917156-4: Views extra module to add a few more Commerce views relationships.
All the testing instructions for this method could be found there.
Additionally, although this doesn't seem to be recommended (see comment from rszrama at #29), but if you would have any particular reason to go down this road, please find attached to this comment a patch against commerce-7.x-1.x version at ed9d68b that does exactly the same thing and adds two reverse relationships as described above or at #1917156: Views extra module to add a few more Commerce views relationships.
[File named: commerce-relationship-product-line-item-1319810-30.patch]
After applying the patch, it is important to remember to flush the cache (otherwise declared relationships wouldn't be displayed initially).
Patch has been tested and seems to work as expected.
If you really needed to go the "other way around" from what is done by Commerce Reporting, as suggested multiple times by rszrama in this ticket (no further code needed), whichever way you would decide to follow:
should both work fine and add the corresponding reverse relationships for the Product Reference and Customer Profile Reference fields.
Lastly, to follow-up with #27 and for testing purposes, feel free to test the patch or the module with an export of the commerce_products view to demonstrate the use of the reverse product reference relationship on Line Items, as suggested at #26.
For more information on the testing instructions see #1917156-3: Views extra module to add a few more Commerce views relationships.
As a side note, for this example of views export, field aggregation is enabled with a SUM on Line Items Quantity, a SUM on Line Item Total, along with a convenient latest order date (MAX).
Sorts are enabled on all columns, and I allowed myself to add a few filters just for testing the results on order status (through a relationship Product > Line Item > Order). Everything seem to work pretty well when I filter on status, for example, pending or completed orders (SUM results change accordingly) and it also appears to be possible to display the products that have never been part of an order (not referenced by any Line Item, as specified in #25).
I think this is pretty much what you wanted to achieve initially and hope I was able to understand your questions, comments and points more accurately this time.
Please let me know if you would have any feedback, questions, objections, comments, suggestions or concerns on this patch, the feature request or any other aspects discussed in this comment, I would be glad to explain in more details, provide more information or re-roll the patch if necessary.
I would greatly appreciate some help from module maintainers and if any of you could take a bit of time to look into the patch for Commerce Extra or attached patch to give me your feedback/opinion on this ticket.
Any questions, feedback, testing, changes, recommendations would be highly appreciated.
Thanks to all in advance.
Comment #31
nigelw commentedHooray that did it! Well done DYdave. Thank you so much for your work on this. Let's see what others (including ryan) think about it and let's get this RTBC.
Comment #32
pixelsweatshop commentedCan confirm patch in #30 works nicely. :)
Comment #33
nigelw commentedTrying to wrap my head around the entities and relationships in commerce and I put together a rudimentary diagram to aid. Hopefully this may help others. Please correct me if I am wrong too.
So for example, if I wanted to show all of a user's billing profiles on their user account page, I can't get the relationship directly from the user entity- I would need to add the relationship to the order entity first.
Comment #34
vadym.kononenko commented#30 works perfectly for me.
Comment #35
mariancalinro commentedI have #30 in production on 3 sites now, and it's working well.
Given the other reviews, I will RTBC this.
Comment #37
adamgerthel commentedWorks well for me too. My use case is that I want to make a list of shelfwarmers, i.e products that have never sold. Using a reverse reference is the only way I can think of to achieve that.
Comment #38
caesius commentedPatch #30 seems to have been malformed with non-patch info at the top. I am attaching a proper patch from the most recent head.
Comment #40
caesius commentedNot sure why it says it's an invalid patch format. First guess is I didn't strictly follow the patch naming conventions by omitting the comment number. Second guess is that the patch has Windows line breaks. Either way SimpleTest seems overly pedantic.
This time I created the file on the server and FTP'd it to my desktop, so hopefully it'll work this time.
Comment #41
caesius commentedOkay, I see what I did wrong now... shouldn't have tried to take a shortcut by copying output directly from PuTTY.
Comment #44
damienmckenna@caesius: As a reminder, you're not supposed to mark your own patch as RTBC, but I've tested it and it's fine.
Comment #45
rszrama commentedTagging for Contrib Kanban.
Comment #47
rszrama commentedJust gave this a review - the new relationships are defined in the wrong module's Views include file (Product Reference instead of Customer). Additionally, labels should follow the same format as other reverse relationships. See for example this code from commerce_customer.views.inc:
Speaking of which ... this looks surprisingly similar to what the most recent patch appears to be adding in. Any reason for that?
Comment #48
dydave commentedHi Ryan!
Thanks a lot for your feedback on this patch, it's greatly appreciated and I apologize for disappearing from the discussion like that, but I knew this would take time to be tested or further discussed....
Anyhow, I have reviewed and tested the patch from #41 and indeed, it needed a bit of re-work.
Indeed,
commerce_product_reference.views.inconly needs the patch from #26: removing 3 lines of code, which adds a new relationship, corresponding to point 1 from #30:The rest of the code concerns point 2 (from #30):
There are 2 modifications to the file
commerce_customer.views.incnecessary to achieve point 2:If no join is added there is no point for the rest of the code since the reverse relationship wouldn't find the column and fail with an error:
(I have tested this)
The code quoted at #47 allows relating an entity (For example a Commerce Order) to a Customer profile entity.
This patch and code should add the reverse relationship allowing relations from a Customer profile entity to a referencing entity.
Agreed. I tried my best to edit and modify the labels and code format/variables so they would be closest aligned with
commerce_product_reference.views.inc, line 77 (see the end of the file).Please find attached to this comment an updated patch against commerce-7.x-1.x at b6674ec which implements the changes described in this comment.
File attached as: commerce-relationship-product-line-item-1319810-48.patch.
Once again, just to sum things up:
I hope I didn't misunderstand or overlooked any point in the review #47, and would greatly appreciate your feedback on this updated patch.
Feel free to let me know if you would have any additional comments, requested edits, feedback, questions, issues, objections, suggestions or concerns on this updated patch or the feature request, I would be glad to provide more information or explain in more details.
Thanks again very much to everyone for your testing, reviews, feedback and comments on this issue.
Cheers!
Comment #49
damienmckennaFWIW this has been working for us on a (small) production site for several years.