Product Creation & Management

On this end I'm hoping I've overlooked something.

yes the product creation seems very clean + modular. But management seems so mindbogglingly tedious I'm sure there is somthing Ryan and team has designed that I have misundertood.

Example: I create product type with 2 attributes/fields. Each field has multiple options - inthis case 6 each. Go to bulk product create,create my product SKUs. All is fine.

Creating a product display now means I have to manually select 36 product references?

Whoops realized i don't need that product type any more, go to delete product type, leads to "error, you have 36 products which use this product type."

So now (without even the use of selecting all products at once via check boxes or similar), I must delete 36 individual entities?

As tyromind mentioned, what if i have 4 attributes each with 50 options. Not unreasonable if you are selling garments with a sophisticated range of measurements.

Now i have to manually delete 6 250 000 products????

I find myself resorting to direct queries to the DB which is exactly the opposite of the 'Drupal' way,for what essentially should be one delete operation.

Is the project really that unwieldy? Or are me missing something fundamental here?

I'm not sure we really need a thread with speculation. We do have projects already in the works and/or receiving design consideration to solve these things, but I still disagree that every possible use case must be accommodated by the core UI. My personal pet project is to have a product / product variation creation form inline on the node page through what I call an "Inline Product Form" widget for the Product Reference field. You can follow along / comment on the mock-ups produced just two weeks ago at the Colorado sprint on this project: http://drupal.org/sandbox/rszrama/1181848

The other project out there from last Summer to assist in bulk product creation is http://drupal.org/project/commerce_bpc. This handles the creation problem you described, it just doesn't group things yet. However, even that may be resolved now that Views has support for reverse relationships whereby we might be able to group products in the admin interface according to the nodes that reference them. In other words... this may have just been solved at the dependency level in just this last week.

And for more complex cases... maybe administering everything through a web app simply isn't called for. One of our partners rolled out a site with a few products each containing about 10,000 variations. These were managed externally and imported / updated into Drupal regularly. I don't think that's a bad solution, but I also don't think we've seen the end of product UI contribs. Given how recent the work I linked to above is, I hardly think development on this stuff has stagnated. : P

Hi Ryan
Thanks for the reply and vigilantly following user comments as always.

I'm not sure we really need a thread with speculation

I think that's kind of unfair. I'm reporting on my personal experience trying to tackle management of Drupal Commerce and the particular issues I've come across so far, i'm not just taking stabs at your work for the sake of being mean spirited. Some of the features of Drupal Comemrce are just outstanding, and I think its a vote of confidence that I'm trying my best to make it work.

When trying to express the frustration of not, for example, being able to delete product in a rational way: you give an almost typical developer response- which is: "because it doesn't. maybe that doesn't belong in a web app"

If the commerce platform, deleting product instances is a short command to the DB: delete all product rows. And yes, I'm not that lazy, and yes I can write my own module to fulfill this action.

I guess I'm just completely unable to figure out why something so simple is not included. What is going on in your mind when you decide not to include a feature like that which would have taken someone like you very little time to implement?
That's a serious question! Drupal was quite a steep learning curve but once you 'get' how to put things together it start to make sense why people build modules in certain ways. Having trouble understanding your design here.

but I still disagree that every possible use case must be accommodated by the core UI
I agree - so maybe tell us what your ideal use case would be?

How do you recommend we mass delete, was modify, mass add fields etc?
If these question have all been answered, a point in the right direction would be appreciated (I have been looking!)

Would Views Bulk Operations not work for you? I believe it can mass delete, though there may be some lingering problems with it not using the proper entity deletion functions. (That's only hearsay, though, as that came up in relation to the Feeds integration where Feeds apparently wasn't properly deleting entities.)

And perhaps I should've been more specific: I can't respond to "what if" scenarios about huge product sets, but if someone has a real use case and cannot achieve it with Commerce, I'd like to hear proposed solutions - especially in a "feature request." The original post was more or less a complaint about the user interface and a false impression about our expectations from the community and what is actually already happening. See for example,

"If this was Ford, would you make a car with no steering wheel, and hope a 3rd party came up with something good?"

Your follow-up didn't offer much more and seems to me to misunderstand the way things work elsewhere in Drupal. I don't know of any mass delete buttons for nodes of a particular type or for all users on a site. Views Bulk Operations has been in place to accommodate things like this for some time, though. If we don't prevent type deletion, we end up with a bunch of orphaned products that cannot just be magically switched to a remaining product type (thanks to field data). Right now if you delete a node type that has nodes created for it, your site breaks (and the confirmation form even tells you it will). I don't think that's more usable than just not allowing the deletion of a product type until the products have been dealt with.

The projects I did link to exist to address your frustrations, so I don't believe I responded with your typical developer response. I think you misunderstand the 'Drupal way' and will be helped further by VBO. I want to be helpful, but the queue isn't about responding to complaints - it's about acting on suggestions (i.e. actual feature requests, not just "fix X please") or reproducible bug reports.

Hi Ryan,
thanks once again for your personal response. And I apologize for not being more helpful in my own comments/feedback.

I have used VBO in the past, and was not aware this was a suggested method for store management. I did not come across a reference to that for example in your docs on drupalcommerce.org

I will take care to try and be more constructive in future feedback.

I know you are trying to address exactly what is mentioned here, but can I just ask my question in a more straightforward way? Something that you suggest may help me understand Drupal and Drupal commerce better?

Moving into mysql queries, lets say I have bulk created many products of a certain product type with several fields with various options.

so yes,first i have to delete the product displays (node which reference those product type).

delete from commerce_product where product = [PRODUCT_TYPE]

truncate commerce_line_item
truncate commerce_order

Seems to successfully clear the way for me to now delete that product type.

Obviously, this only works if the there are no real orders / line items and we can begin again.
What am I missing though? Why not include extra table in the DB that allows me to only remove specific line items by product type, or orders which include a particular product type?

If worried about orphaned products, why is there the ability to delete product instances one by one?

Sure, I can answer that. : )

The bigger issue here is that an entity has data stored all over the place, not just in its base table. So, even if you truncate commerce_order, you would still have revision data in commerce_order_revision and a whole lot of field data in field_data_commerce_customer_billing, field_data_commerce_order_total, field_data_commerce_line_items, and their associated revision tables. For each of the fields on the order you'll also have data in the field cache.

With products, the issue is that the product type is the "bundle" of the entity, nothing more. In the database it's represented by the data in commerce_product_type. Every row in commerce_product has a type column that references one of the product types either defined in that table or in code (it all comes through hook_commerce_product_type_info()). Just deleting a type has nothing to do with deleting product data, much like just deleting a node type has nothing to do with deleting node data. All it really does is get rid of the "bundle" definition and then leave all these entities referencing deleted types. That's what causes the errors.

If you want to, though, the data is there in the database for you to build queries like you're requesting. You just have to join through the appropriate reference fields. A better solution would be to use Drupal 7's built in EntityFieldQuery object. This is a query builder that lets you load entities based on data in their fields (i.e. load the ID of every line item referencing a particular product). You can read up on it in includes/entity.inc. The class is heavily commented.

There's nothing in the IPF module just yet as I had to shift approaches based on Bojhan's feedback. The ability to modify additional field data and such through Commerce BPC would make a great feature request, and I was surprised to see they may have removed product display creation from the wizard (I think it should go back in).

With your scenario, I'd say that is not unreasonable. While it may be displayed via one product display, each of those things is a separate product. You wouldn't manage stock for each of those things with one stock level - you'd have to track your stock for each individual combination. I'd like to investigate how to better handle product groups for 2.x, but even in the meantime we can recommend solutions... but those solutions must come with developer hours.

One thought we had last week in Paris was that a product could contain a product reference field to function as a product group. Said field might get a custom widget that simplifies administration of the various combinations, perhaps in some sort of BPC manner. You could then filter your Product View to only show these product groups instead of every individual product. The missing piece would be making sure the Add to Cart form gets loaded with all the referenced products instead of just with the product group in the form. We just need someone with the time to start moving in this direction, and I can ensure that anything blocking this behavior in core would be taken care of as quickly as possible.

But the fact of the matter based on Ubercart usage and our experience at Commerce Guys is that most stores don't have such hectic scenarios. For a good year and a half we didn't even build any sites involving shipping. Drupal based e-commerce is used to power such a wide range of stores that the selling of physical products with a broad set of variations is a minority use case. So, as a Drupal e-commerce solution, the core modules shouldn't be delivering features specific to this use case. Our development philosophy from the beginning has been to see contribs come in on top of this base architecture to tailor the UX to various use cases, and this is definitely one of the ones in our sights. We just can't put the cart before the horse...

heh, well, I don't want to be seen as shutting down your request. I definitely know we're behind the ball on this. ; )

To help generalize the need, here's my situation, and a reference to a product that handles it well.

My current burning need is to build the catalog efficiently. It's a company that sells signs, hardware to mount signs, and a bunch of other stuff. There are ultimately many thousands of products, in a variety of categories that are quite distinct (different product type, etc).

But within a category (and sometimes across multiple categories/subcategories), products are of the same type (same fields).

The key is that many of the products are almost the same. For instance, there could be 200 products that are the same dimensions, materials, colors, etc, but have a different sign legend. Probably 80% or more of the work of adding such products to the catalog could be bypassed if:

-- an existing product can be readily copied, then edited.

-- and/or the prior field values can be carried forward as defaults when creating the next product.

Aside from a few text values (in custom fields) that differ, the most common change after copying a product would be the image. So it gets a little bit complex.

OpenX, the open source ad serving software, allows this. Maybe it can provide ideas on how Drupal Commerce might approach the situation. In OpenX, an existing advertisement instance, or campaign or advertiser can be copied with one click, then fully edited to become a different ad/campaign/advertiser. I think this can be readily seen in the demo at openx.org. They make it smooth and pleasant to create and manage many, many entities, which strikes me as similar to a need in Drupal Commerce. (Consider that advertising management and serving is not a trivial endeavor.)

I apologize if this is discussed (and/or dismissed) elsewhere, but I didn't find it.

Also, I should say that the company makes many custom signs, so handling that is mandatory. But it could possibly be handled via a Comment or Custom Text or similar field(s) that a human must interpret for internal processing of the order. Trying to handle all the variations via dozens of options lists and check boxes seems unlikely.

Do you think this would work for you? #1033950: Add "clone" link to admin/commerce/products to make it easy to clone products (like node_clone)

Yes, the suggested "clone" link would do it.

BUT, that suggestion seems to have been kicked over the wall into the arena of Drupal-wide enhancements, where it will likely be low priority for some future version of Drupal. Maybe in general it's not of much value.

But for Commerce, cloning Products is a huge need. Think of almost any business with hundreds or thousands of products, and chances are half of them are almost the same.

Cloning similar Products is not the same as a product having attributes, which is limited to certain types of businesses and products, like the oft-cited "shirts". Even then, my experience is that most companies consider each size and color to be a separate product, so they can show an appropriate image, so they can track inventory precisely, etc. (In some cases, what is offered as "same but different color or size" is actually a different product from a different supplier, especially the case with garments.)

One-click Product cloning could save hundreds of typing hours. In my case, 80% of the cloned data would be the same, just 20% revised, vs. always having to input 100% of the data for each product.

Considering the time- and labor-saving benefit of Product cloning, I think this capability can be the difference between Commerce being adopted or not -- since most other e-commerce systems I've evaluated offer this already. (And other online products, like OpenX ad-serving system, as I mentioned.)

So, I hope this will be considered as high priority for today's Commerce, not left dangling in hopes a general Drupal method is devised "someday".

i'd just like to add from some further experience.
We ended up doing all product management through feeds and feeds importing.
We spent some time building a master CSV file that we would upload periodically to make adjustments to product instance definition.
This way, data entry and product management can be offloaded from the developer.
This method can be used to create prod references to prod disp0lays at the same time.
before we had legitimate orders in our store, we 'deleted' individual products by clearing all products from the DB, and re-importing a modified CSV.

edit: also if anyone is following along on this thread, yes VBO for delete purposes worked fine as well. At the time of first commenting, we had had trouble getting VBO working on our install without bugging.

Interesting approach...

How many total products (including variations) are there -- how many spreadsheet/CSV rows? Our store will end up with maybe 10,000 products, seems like quite a lot to manage and repeatedly import as a batch.

Also, every product has at least one image, sometimes more. How are these handled using a feed?

@hawkdrupal
yes its a bit unwieldy isn't it? By this method, we would have 10,000 rows. do your products have attributes? Remember to multiply the number of products * attributes to get the actual row count :)

The situation I was describing was really only a workaround to deleting products. (i.e. clearing out all and doing a complete re-import) For editing, Feeds is pretty smart, and will just detect changes. 10,000 wouldn't take that long to update.

For product images, we are attaching product images to the product display via a custom field. (i.e. we've disassociated picking the SKU from the image gallery) But, yes, default practice of Commerce is that one of your columns in the CSV I described would be an image file.

edit: also yes just to address your other point, let's say you've already defined your 10,000 products. If you wanted to update/add only a handful of products at some later point, you could easily upload a new CSV with only the pertinent rows to save time. ( i.e. didn't mean to imply you must continually import 10,000 products for minor changes).

Commerce itself is agnostic about image storage; it isn't required to attach images to products, but I do it in my example sites to demonstrate field swapping by the Add to Cart form. Seeing the image change is pretty dramatic and helps people "get it". It would be fine to attach images to your display nodes instead if the product set would all share the same images, and I presume you could ignore the images on subsequent updates.

As another method to contemplate...

For years I ran a Yahoo Store (a subset of it is now a Ubercart store). I'm not recommending Yahoo Store, lots of funky things about it. But consider their approach to product management.

The product catalog can be viewed and updated via HTML tables, each row a product, columns for SKU, description, price, stock, etc, including path to an image. (One field controls whether the particular product is active in the store, very helpful.) The large catalog is broken into pages of such tables, maybe 20 products per page. Bit ultimately it was painfully slow to work with so many pages spanning the entire product catalog.

We switched to managing the product catalog as spreadsheet rows, which Yahoo Store makes easy. Just one click downloads the CSV, one click loads it into a spreadsheet, one click resaves as CSV, one click uploads back to Yahoo. On upload we could overwrite the entire catalog, or just the rows we were uploading (thereby updating just those products), or append to add products.

Yahoo Store handles all the messy stuff of generating the CSV from the database and vice versa, so user's don't have to deal with data types or have any notion of how the flat file spreadsheet is mapped to multiple database tables.

On upload, Yahoo's script moves all the data into temp tables with the same structure as the live product database. This processing helps discover and alert the user to apparent errors (inadvertent letter in a numeric field, missing value, etc). The result is then provided for visual review -- the user can walk through the temp product catalog, item by item, or jump to particular items to check or edit. Upon approval, the temp tables are merged into the active product catalog.

Of course, a large catalog can result in a huge spreadsheet, but the catalog can be grouped into categories (such as product types) that can be downloaded/updated/uploaded individually. This makes the process reasonable and logical, because product updates are usually by category anyway.

It might seem crude to manage products externally, in a flat spreadsheet, but users really liked working this way -- fast and straightforward. Every spreadsheet row is a product/every product is one row, so its simple for a user to grasp. Multi-value options are a single column with all the values delimited.

For the case of many similar products, users would simply copy spreadsheet rows then change individual values as needed. In our case, values such as SKUs, images, options, were logically constructed so it was very fast and simple to copy and modify values. In some cases we implemented spreadsheet macros to make this even easier.

Obviously, working in a flat spreadsheet that allows direct value editing can lead to data discrepancies -- is that shirt size option "small or Small or SMALL" -- but thanks to copy and paste (and careful users) this is a small matter given the other benefits.

From what I've seen the ability to manage the product catalog via offline download/upload is found in MANY e-commerce products. It seems to be a fundamental feature, in addition to live online editing.

Obviously, generating then accepting flat CSV files can be complex if the database is really relational, especially if there are custom fields for some product types/categories. But Yahoo Store's download/upload scripts handle it.

The same capability in Drupal Commerce would be a huge help. (And apologies if it's already there -- where?)

(Disclaimer, we stopped using Yahoo Store 3 years ago, don't know what might now be different.)

This module may be a solution to this problem:
http://drupal.org/project/commerce_feeds

First off I bow to rszrama and rfay for their great work on Drupal Commerce. I stumbled upon this issue while trying to solve my own problems. Using Views Bulk Operations and exposed filters actually goes a long way to solve the product management issue. I dumped my modified version of the product admin view (the one one finds at at mysite.com/admin/commerce/products/list) here http://www.leanderlindahl.se/content/managing-products-drupal-commerce in case it helps anyone along. It would have saved me some time and worrying if I had found this info right away...

Excellent! Thanks for the kind words and for sharing the View. : )

Well, I too have run into the frustration of the whole "attribute" issue and Yes, I understand how the current setup of DC expects every variation of a product, to be it's own product. It has it's perks and it's downsides, I'm still sitting on the fence on what I think about it...

I really have no idea where to post my question since there are a bunch of threads out there dealing with this topic, so here goes. The one thing I haven't quite figured out, is there any way to reference another product, as an option? If so, how and if not, are there any thoughts to open discussion on it?

I honestly can't wrap my head around any viable solution at this point that will work in the current design of DC, but I do believe that "Attributes / Options" with automatic price adjustments all tied into core, is a must. I'm not saying to cave to the initial design of "each option is a product", I'm saying I think it might be time to step back and see how it can be built to accomplish such a common model practice of e-commerce. While some stores use attributes to fine tune the product before checkout, the other side to the coin is marketing and up-selling a product by offering upgrades.

---

I guess the way I see it, I should be able to setup two similar products:
widget AA, widget AB
Then create a third product that will be sold as either stand alone, or upgrades:
widget S

With the current DC design, I have to create 5 products to tie them all together:
widget AA, widget AB, widget S, widget AA+S, widget AB+S

I would rather be able to show widget AA or AB, but simply call on S when I need it and have widget S automatically tied to the line item of whatever called it. (obviously, if it was added as a standalone, it would have it's own separate line)

Here's the fun kicker: Allow a different price for widget S when referenced as an upgrade vs standalone. (This makes it a marking ploy: if you buy it as an upgrade, you save money vs buying it as a standalone product)

---

I just installed the Inline Entity Form mentioned above to see if it might open some doors, but after 10 min of playing, I can't figure out where to even begin with it. Update: After digging, finally figured out it was the widget for the "product display". I guess I should have watched the screencast?

Sorry for the whole post, it was a collection of thoughts and ideas gathered over a 2 hr span of trying to figure out how to get across my point...

Hey Guys,
so finally I came across the same issue.
Product management could be easier for the user.
It was hard for my customer to understand what a product variation is, and since they had a different system with a different approach, it was tough to hear my system was not good :-(

So here is the thing:
There are times when the user wants to have a product, with a unique image and price but supporting different colors and sizes. So conceptually, we have many product variations (for sizes and colors) but only one product display. Now, the user expects to create only one product and just specify what sizes and colors it uses.
So everything would possibly be done in only one form, in one screen with no further steps.
I love the power of Products - Variations - Displays, but I completely understood why my customer was upset.

Requirement
-----------
In my opinion, the core UI should take care of this with a new variation type.
Currently variations are used to have different prices or images.
But we may have a "soft variation" entity, which is only used to add common attributes to the product.
Why add it as a variation and not just as fields in the product? Because the user wants to have those attibutes present on the "Add to cart" form.

I hope the complete idea is understood and we find a way to take care of this.
Thank you all !
Ecommerce is great!

Here's how I went about solving the issue of product creation and management. I hope it helps others. Some of this may have required some custom module work by myself, or helper contrib modules. I'll try to update later with more details.

1) Identify basic features/attributes that are shared across all objects, and normalize them into a "Master Product", or if necessary, a hierarchy of abstracted products. These could be nodes or entities, but I have found in D7 using Nodes gives you a lot more flexibility and access to different tools.

2) Make a required reference field in your product tying it to the Master Product.

3) Use views to build out views blocks that pull desired attributes from your Product, and separately from your Master Product (since in my case, one is a node and the other a Commerce entity). Use custom views theming templates as necessary to programmatically tie data together, or to reveal data "hidden" in the results, build links, etc.

4) Use Page Manager to create your displays, incorporating views blocks from your Products and Master Products as needed.

You can see an example of this at work in my last project, JANUS et Cie. This ended up being a pretty great way to go, and allowed me huge flexibility and efficiency in my site design and product management. When you need to update a shared attribute that's stored in a Master Product, for one, you only need to update it in one place.

What would improve this? Better mass editing and auditing tools. The preferred solution for such for most small-to-mid shops has pretty much always been a spreadsheet. There is a pretty cool jquery-based, Drupal view spreadsheet module out there somewhere, but it's really not ready for primetime. What would be best? Maybe some kind of tie-in to Google Docs?

What do people think about this?