Port Webform to Drupal 8

Hi @CarlHinton, thanks for opening this issue. Right now I'm not inclined to begin the Drupal 8 version of Webform because there is no code overlap between Drupal 7 and Drupal 8. This means that the D8 version will be difficult to maintain at the same time as the D7 version. Since we're still regularly making changes to the D7 version in the 4.x branch, it'd take a lot of effort to keep the two branches synchronized.

At the same time, Webform tends to keep making changes even after the final release of a version, so it's unlikely there would ever be a time when it's completely frozen. So even though now is probably a more difficult period than after the 4.0 release, even afterwards we'll still have the same problem.

So in short, if you (or anyone) is interested in porting to D8, we can begin at any time, but I'm not going to be investing time into managing the two branches simultaneously at this point. Considering we're still updating the 3.x branch (and D6) for bugs, this would make a total of 4 branches needing to be maintained all at the same time.

I should also point out that I don't have much enthusiasm for Drupal 8 as a whole, so anyone interested in porting and maintaining the D8 branch please chime in.

I'd also be up for helping out on the webform D8 branch. @CarlHinton - will you be at DrupalCon Prague next week by any chance?

Hi guys, I've added both of you as maintainers. I think @EclipseGC is also interested in maintaining/porting the module.

With D8 the entire Form API has changed

It hasn't changed much. The API itself is mostly identical, but the way forms are defined is a little different from the view where you put your code. The actual form callbacks (build/validate/submit are more or less the same.

we have all the Symfony forms stuff to deal with

The form API does not use Symfony. Perhaps you mean that we have a Symfony route controller that can automatically render a form?

I'm also putting my hand up to help out with the D8 port. Unfortunately I've not been involved in webform for the last year however we will be using it on D8 and I'd love to help out.

As 7.x-4.x has recently reached beta and API changes should hopefully be minimal from here on in it seems like a good time to start the port. In terms of workflow I see anything @quicksketch commits to the 7.x-4.x branch will need to be marked as "patch (to be ported)" at which point it falls to those of us maintaining the 8.x branch to do exactly that.

Quick question to @quicksketch, should we go with 8.x-4.x or reset the version to 8.x-1.x?

I've created the 8.x-4.x branch and done the initial port of the webform.info file, humble start, much work to do!

I'll be at the Pacific Northwest Drupal Summit this weekend and hope to get some work done on the port. If anyone is attending or can help out online it'd be great to make a decent start on the port.

Again in terms of workflow if @quicksketch or anyone working on < 7.x branches could please mark any committed issues as patch (to be ported) and those of us working on the 8.x branch will do the port. The goal here is to get the 8.x branch in working condition and mirror functionality in the 7.x-4.x branch.

I have one request that is very important for DX: can the D8 version please get rid of all the large array structures? "Big piles of naked arrays do not an API make" (and there are more sources that say this). I asked @quicksketch about this in Prague for 7.x-4.x, and I was surprised to hear that this still hadn't been documented, let alone that the arrays were actually replaced by something developer-friendly.

Webform is a nice tool, but its DX is terrible and that will need to change. I hope, with all the useful changes made in Drupal 8 and the use of OOP in general, this won't be too hard to do during the port.

@Xano yep, I totally agree.

However given just porting webform-7.x-4.x to Drupal 8 is a non trivial task I'm just focusing on feature parity at the moment. After we get a working port we can look at API changes (moving away from naked arrays, etc) probably in a 5.x branch.

BTW any help with the 8.x port is much appreciated ;)

Updating issue summary.

It's not just feature parity. If Webform for D8 will keep using info hooks, nested arrays, and pseudo hooks for its plugin definitions, for instance, it will be worse DX than in D7, because D8 simply does not work that way. Sticking to what Webform did in D7 will also make unit testing close to impossible, whereas converting code to plugins with dependency injection straight away allows for perfect unit testing, which in turn improves the quality of the port and prevents bug hunting.

@Xano, again I agree. Moving to plugins in place of info hooks is the goal for 8.x-4.x. However, at the risk of sounding like a broken record, the first goal is just to get the module working in D8.

@Xano, just to clarify a little more. quicksketch is actively developing features and fixing bugs on the 7.x-4.x branch. I've only just started porting that to 8.x. My first goal is to get 8.x-4.x actually working, at the same time as keeping it in step (as much as possible) with 7.x-4.x. I'm really looking forward to refactoring things like webform components into plugins however right now a working port is the priority. Once again I'd love to get some community help on the port, patches most welcome!

I'm ready to help out too with specific patches. Will try to keep related issues on my radar.

With all the changes in Drupal 8, shouldn't the Drupal 8 version of Webform be 8.x-5.x? 8.x-4.x and 7.x-4.x will not be compatible in the least bit and pretending they are is a mistake.

Or it could be 8.x-1.x. Version counters can be reset when the core version is advanced.

I originally planned for 8.x-1.x (as you said @Liam) but eventually went with 8.x-4.x.

I think @Xano's argument in #16 isn't valid, look at webform 6.x-3.x which is the same (feature compatible) as 7.x-3.x (obviously with the exception of working on top of Drupal core 7.x api's). The first goal is to get 8.x-4.x feature complete to match 7.x-4.x hence I think the 4.x version makes sense.

On a general progress note on the 8.x port I'm going to have time mid Feb to make a more concerted push on this, until then patches are most welcome!

I'm sorry, but that's not right. The major version indicates code compatibility. In practice, the same module for different Drupal versions are always incompatible, so the major version number MUST be increased if we want it to have any value at all. Otherwise it no longer functions as a version number, but as a plain identifier.

Take a look at the Semantic Versioning rules for an explanation of how the rest of the world does this. This is especially relevant, as Drupal 8 may very well adopt this standard.

@Xano indeed, however that is still undergoing much debate as per #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) with no resolution (as of yet).

I come back to the fact that the initial port of webform 8.x-4.x is aiming for feature parity with 7.x-4.x. Until semvar lands for contrib the shared 4.x branch naming indicates feature parity. Happy to switch branches once a policy is adopted.

#2293945: Convert components to plugins is related.

I think the D8 port would go best if someone wanted to step up and take a lead role. Ideally this would be someone who is really familiar with D8 as there may be architectural issues. We have been muddling along porting our D7 patches to D8, but really no work has been done for a while on the D8 port.

One key decision is how much to embrace D8's new OOP architecture. If we seek to do the minimum to get webform working on D8 with as few API changes as possible (e.g. 8.x-4.x, absent semantic versioning), then what does that mean for the massive number of webform-related modules? Is it good to let them port to their 8.x-4.x, or is it better to make them immediately port to a new API (e.g. 8.x-5.x)?

Last, I am fearful of the amount of work involved in this effort, particularly if in one go we convert to a new OOP paradigm. It might be possible to raise some funds, but reading about the rules funding efforts, it took a huge amount of work to raise a fairly small amount of money (1000 hours of work to raise < $20K US).

Webform is the #2 non-utility module, after Views. With Views in D8 core, it is presumably the #1 non-utility module.

as few API changes as possible

Anyone who ports modules to Drupal 8 with as little API changes as possible, pushes that module to the brink of extinction. Seeing as Drupal 8 is so different because it has finally caught up with the rest of the PHP world, modules can't just be ported. They'll need to be rewritten almost entirely.

This is not a bad thing, however. With proper OOP, it's easier than ever to unit test and refactor code, even after the first stable version has already come out. I've found that this approach makes it easy to 'compartimentalize' the work, so to speak. Because we can now unit test, code can be ported and put into classes and tested, without any other code using it yet. This way, the code base can be converted step by step. If you want to start out with a solid code base for Drupal eight, I'd completely skip a port version and go straight to 8.x-5.x instead. Webform has gone too long without any API improvements, and we all know nobody is going to make two Drupal 8 versions of their Webform plugins anytime soon.

Taking this huge leap will inevitably cause other developers to drop out, just like they have done with Drupal entirely. It will also attract new developers, or spark renewed interest in developers who still use Drupal, but not Webform (like me, because it's just too much of a pain to develop anything for it). I don't think this is anything that needs worrying about, as Drupal in general is catching up with popular best practices, and once the conversion to OOP and full unit test coverage has been made, maintenance and continued development is a breeze.

If you would like more info about OOP and major new patterns in Drupal 8, such as plugins, drop by on IRC. I'm there quite often and there are many others who can answer your questions.

@xano - I personally do not plan to lead the D8 version, and I'm pretty sure quicksketch isn't either. Would you be interested in taking the lead? You seem to have a vision for how webform should be rewritten for the D8 OO architecture and enthusiasm for D8. :)

I can't take on more coding projects. I have enough contrib work as it is :) I can, however, help with understanding Drupal 8, which is partly why I'm in this issue. The other reason is that if Webform isn't rewritten, I won't be porting any of my Webform code to Drupal 8, as it's just too much work and the lack of a(n) (consistent) API makes testing hard to impossible.

:( Hence the D8 conundrum. We are standing on a nice, well-developed island build with narrow roads, old terracotta sewer pipes, and aging lead-soldered water pipes. We are looking a a new undeveloped island with great modern water, sewer, and roads. No one has the time, energy or money to rebuild the city on the new island. And some don't even want to swim to the island.

Porting/rewriting a small contrib module is one thing. Webform is huge. My efforts in the queue have been focused on getting 7.x-4.x stable.

Maybe someone else will step forward. Or maybe a business will sponsor the effort.

Are you on IRC?

Very interesting comments @Xano and @DanChadwick.

I'm going to go out on a limb here and put my hand up to lead the D8 rewrite.

Over the last 6 months looking at and recently starting to work with D8 I've come to the same conclusion as @Xano: webform-8.x must leverage all modern D8 API's. However @DanChadwick's superb "well-developed island" analogy is frighteningly close to the mark, so in practical terms this basically means a rewrite. The risks and benefits @Xano outlines in #23 are spot on. Everything points to reworking (rewriting if you like) webform to fully leverage D8 best practices.

When I started the 8.x port 9 odd months ago I was against a rewrite. Given I've made no real progress (12 months of holiday and part time work from the road can do that) we're still basically at square one. However in 2 weeks time I can commit more consistent time as I'll be back at work full time. I'm not promising the world here, but I will give it a solid shot.

@Xano I will be taking you up on your offer of guidance and pointers to better leverage D8. Watch this space.

@fenstrat -- you are used to island life, since you live on a big one. :p

Big round of applause for fenstrat, quick, before he comes to his senses. (Whistle blows, gangplank retracts, ship departs "port".) ;)

Xano and I had a brief IRC chat about the various considerations in upgrading webform. There are big areas of webform that I've never even peeked at, much less am familiar with. Currently webform supports:

• A huge number of different, unrelated webforms (>10,000?)
• A huge number of results per webform (> 100,000)
• A huge number of components per webform (>100)
• Webform as node and as block
• Intra- and inter-page conditional, multi-page, draft, and anonymous webforms (which complicate each other)
• Analysis (which needs to access all the results
• Views integration (currently no filtering/sorting/grouping on component data)
• Reporting and download
• Data migration

Some deep thought should be put into how component data is stored, and the consequences of changing what we have. For example, currently the component ids are meaningless, which complicates views. If they were mnemonic machine names, you could filter and sort, assuming the data is stored appropriately. Would the db load of using alphanumeric component ids versus integers be significant?

I don't think that the field_api can be used to store component data and maintain these spectrum of use, but is sure would be nice to re-use all the work that has gone into field types. I'm not sure if some sort of adapter pattern could be employed, at least for the widgets.

This might best be another issue but I wanted to capture Xano and my chat.

@DanChadwick ... leave the gangplank out a bit if you would, I may come to my senses. Or not. Onwards!

Your list of what webform supports is a nice summary. One thing we cannot do is remove the ability to support huge number of webforms nor huge number of components on webforms because for small forms entityform already has webform trumped. Therefore any redesign we make *must* bear this in mind.

I'd be most interested in your and @Xano's IRC chat log.

I'd be most interested in your and @Xano's IRC chat log.

@fenstrat - Nothing much more really. Mostly me whining about how much different D8 is relative to D7. Xano estimated 500-1000 hours for the job, including learning D8. I estimated 1000 hours, plus learning D8.

Sobering estimates, either of them. In 2 weeks I'll make a start, and plug along. Appreciate any assistance. Lets see where we get.

One thing to note is that Drupal core's contact module includes a field-able form.
There are now bundles (ala Contact categories).
There is no storage/administration listing/views.
I have been planning to write 'contact_storage' and 'contact_admin' modules to complement the core functionality.
I think this would meet the most common use-cases of webform/entityform. It won't go near replacing all of the functionality in webform/entityform, but it would meet the 80% use case.
Given the choice between adding those components and porting all of webform to the new architecture or porting entityform to D8, I'd go for adding those components.
My 2c.
One thing blocking this approach is #2289063: Change contact message entity to behave more like a normal entity

Come to #drupal-contribute on IRC you might find more than one person who is willing to help with D8. I also try to make sure questions under http://drupal.stackexchange.com/questions/tagged/8 are answered.

Erm... is there any point to suggesting joining forces with the EntityForm folks?

@chx I will certainly do that, cheers.

@DamienMcKenna as @DanChadwick said in #29 webform allows huge forms, AFAICT entityform does not. There's other reasons but from my point of view that single one is a show stopper. Same argument applies to @larowlan idea of core's contact module.

From what I can see, it looks like EntityForm hits a limit on fields because of Tokens and how much memory it takes to generate the form (https://www.drupal.org/node/2091819) . Surely there is a work around? Caching, disabling tokens, etc.

Otherwise, the concept of using core drupal form api and storing submissions as entities are very powerful. Even if we can't use EntityForm itself, perhaps those ideas can be used? Maybe have the webform UI simply create/store the forms using drupal form api, instead of actually creating components.

Alternatively, is there any other PHP form builder that might be workable? Something related to Symfony Forms maybe?

Here's a comparison of various webform-like modules.

@rlnorthcutt - At least for D7, fields would be too heavy for a webform with 100 fields. That would be a 100 table join. :( Is D8 different?

In my use case, a category of user which is less than an administrator, but more trained than most authenticated users can create or modify webforms. I don't know that entity forms can do that unless you put a different UI on it. Certainly you don't want non-admins creating fields, right? Undoing an errant webform is as easy as deleting its node.

I would love to see a bridge / adapter that would allow field widgets to be used as the UI for webform components. Not sure that's possible.

> @rlnorthcutt - At least for D7, fields would be too heavy for a webform with 100 fields. That would be a 100 table join. :( Is D8 different?

The data structure is the same but neither D7 nor D8 joins the field tables to the entity tables. It's why we have multiple load instead: it's one query per field table per load (multiple or single load doesn't matter).

The data structure is the same but neither D7 nor D8 joins the field tables to the entity tables. It's why we have multiple load instead: it's one query per field table per load (multiple or single load doesn't matter).

Load a submission with 100 components:
Webform: 1 indexed query on 1 table returns 100 results.
Fields: 100 indexed queries each on 1 table each returning 1 result.

Not sure that's better or not. We can know whether this is an issue, however. Someone just needs to make an Entity Form with 100 fields and see how it performs. Then ponder how analysis and other features would be. Interesting.

re: earlier comments about re-using Field API widgets without necessarily using field storage :
That would be an awesome combination, and would probably be easier with the D8 shape of the API.

Widgets do require the following to work, though :
- FieldDefinitionInterface objects for field definitions,
- FieldItemListInterface objects for values, meaning you'd still need to build an $entity

So in practice, webform submissions would just be a ContentEntity, that possibly chooses to use a custom Storage controller to store field values the way it sees fit, if the default "table-per-field" storage is not suited.

Disclosure: I know nothing of the D8 field and entity API, so consider this pondering rather than informed opinion.

I'm wondering about a site like webform.com or surveymonkey.com, in which there are 10^5 to 10^9 order unrelated fields. I'm thinking that maybe webform D8 could still have its own proprietary component (which knows how to analyze itself, export itself to csv, manage it's storage), but uses a stock field API widget for the form and view front end. This might imply constructing a faux field (not present in the schema) to use with them.

This would drastically reduce the effort to use a non-core-webform field (say Address Field) with webform. Make a wrapper or adapter component, fill in the missing webform methods (analysis, export, storage definition), done. Not as elegant as using native Fields, but still leveraging them.

Here's the crux of my concern. Field API was created from the use case of making a modest number of fields per site and attaching a modest number of fields to any one entity, where fields have persistent meaning. Webform's use case is much broader. Example: Deleting fields is rare. Delete a webform? Hundreds of fields go away (for a big form).

Another example. A busy webform might create 10,000 results in one day. Creating submissions is light. Would entities perform well under this sort of load? No idea; wondering.

While I usually subscribe to the "make it work, then make it fast" principle, it would be horrific to expend a person-year building webform on a design that doesn't scale for its use case. OTOH, if Field API could be made to work, then it would be very reasonable to see if we could merge the Entity Forms and Webforms efforts.

I'll post to the Entity Forms queue to see if they care to join in.

Hi, I created and maintain Entityform. Just got alerted to this conversation wanted to my 2c's in. My involvement in D8 in general has been pretty limited so forgive me I don't understand some the architectural changes.

From yched above

So in practice, webform submissions would just be a ContentEntity, that possibly chooses to use a custom Storage controller to store field values the way it sees fit, if the default "table-per-field" storage is not suited.

Would there be any problem to this method in the fact that this would make every Webform type be it's own bundle. Presumably this would be a new entity type Webform. So for a site like webform.com or surveymonkey.com where there are thousands of bundles in this entity type. Probably there are not other entity types in core or future contrib that would require thousands of bundles. How would performance be affected?

These would all be configuration entities, correct? It seems like often webforms are treated as content. They enable the client to make ongoing forms in production like other content. It seems like this could complicate configuration management movement.

It seems like Webform provides to 2 big solutions that other modules in Drupal 7 or 8 don't solve.

1. The ability to have thousands of forms.

It seems like this would be hard to solve by just making Webform an entity that used Field API even if it used special fields that had a different storage mechanism. This is because of the thousands of bundles it would create. I can think of a few situations in the admin interface(presuming it is similar to D7) where this would cause major headaches the View Wizard, , View filters on bundle type, and Rules reaction bundle selection.

2. The ability to have hundreds of fields(components) on a single form.

This is a different problem from #1 and some sites might have 1 and not the other. This seems like it could be solved my making different field types that use different storage(or can you swap for existing field type?) but still use the Field API. But for this problem why solve it on the level of Webform. It seems like it would be better to solve this by creating a ne new module like "Field Lite" or "Webform Components" that other modules could use to make this type of field. Then Entityform, Node, Contact Forms or any entity type could use these types of field.

I am not sure if this would be crazy hard but one possibility for Webform for D8 would to remain being nodes instead configuration entities. But instead of having it's own Components make it be able to use and be dependent on "Webform Components" compatible fields. In the case when these "Webform Components" fields are used in a webform their configuration is stored with the node as content. In the case when these "Webform Components" are used in other configuration entities such as Nodes or Entityform types they would be stored as configuration.

there are thousands of bundles in this entity type

This is a main key point for webform.

Currently in D8 result of \Drupal\Core\Entity\EntityManager::getFieldDefinitions() is cached and allowing to hook here in runtime probably will kill performance.

So having some facade for that in webform would allow use entity-based approach for components as fields.

We can know whether this is an issue, however. Someone just needs to make an Entity Form with 100 fields and see how it performs.

I made this sandbox module Field Max that simply lets you add hundreds(any integer) of textareas to any bundle for testing.

I can't remember how many I got to. I think 100 but the page loaded very slowly if I remember.

I think the token browser becomes an issue. You can use this to help it: https://www.drupal.org/project/token_tweaks

Another critical performance issue of entity are the insertions: Compare the performance of creating (saving) an entity with 20 fields vs. the performance of inserting a webform_submission with 20 fields. In our measurement there is a difference of about factor 100. There is use-cases where that matters a lot. That means: entity (as good as it is) is simply no alternative.

Drupal 8 is now in beta. Those who are interested in working on the port might take this an a poke. It seems that the need for a D8 version of webform is high, even assuming a well-developed D8 version of entityform. To the extent that tedbow can be involved, it would be great to find common ground. The extreme version of this would be a unified module, in which there are two back ends, one using fields and the other using components. Not sure how possible that would be.

A unified module sounds like a good idea to me. It is just a matter of where the module stores its data. If this was configured on a per-form basis it might simplify migration.

Agreed this is now critical. Will pick up where I left off a long while ago tomorrow. There is a lot of work ahead. Any assistance will be greatly appreciated.

I also agree that a unified webform/entityform makes a lot of sense. Then allow a choice of backends of either fields or components on a form by form basis as @Liam Morland says. Sounds ideal, but also like a hell of a lot of work!

There may also be a 3rd way as @yched outlined in #41, i.e. re-using Field API widgets without actually using field storage. However as @tedbow points out that would lead to each webform type being a new bundle. Has the bundle system ever been pushed into the thousands?

Unification seems like it requires a fairly in-depth understanding of the D8 field and entity APIs and implementations. The installed base of webform is about half of views, making it near-core in terms of prevalence.

<blunt> Will some D8 core contributors who created the need for this new version contribute to its design and implementation?</blunt>

It's not exactly like core contributors created the need for webform to pursue field based (or other) storage. That possibility exists in D7, hence entityform. I dare say most core contributors have more than enough on their plate. A more feasible option is to have a crack at what @yched suggested and then ask for assistance.

The way I see it right now the priority is getting webform as it exists, using components, ported to D8. I'm going to focus on that. If at some point unification comes to pass the component functionality will likely still be needed as the "scalable" backend option.

I think a parallel effort is needed to to explore different backends. I wish I had the bandwidth to do that, but right now I don't.

Sorry. Should have been clearer. D8 core created the need for webform D8 (rather than the need for unification or use of fields). Webform is not a trivial module. It sure could use some help from core contributors -- unified or not.

I support your plan to try to get to feature parity. I also support adopting D8 design patterns and APIs where possible.

Thanks for clearing that up Dan. Agreed help from core devs would be nice, but I still don't like our chances. I think reaching out when walls are hit or general advice is needed is the best way.

Thanks also for your solid work on the 7.x-4.x branch, and for keeping 8.x-4.x in sync. It'd be great to keep that up going forward so we can try and keep feature parity while adopting more and more D8 design patterns.

I really sure that core's contact module and entity display components is a way for webform.
Also please tale a look at https://www.drupal.org/project/contact_storage

@andypost Yeah I've certainly been looking at the great work you guys are doing with core contact module.

I think what it keeps coming back to is webform's ability to scale.

I personally don't think that combining Webform and Entityform would be a good idea. They are built on 2 different systems and will have to be built on 2 different systems even in Drupal 8 because all of the reasons given in this thread that moving Webforms to a Field API based system would really just defeat the purpose of Webforms and what it is really good at.

I also agree that a unified webform/entityform makes a lot of sense. Then allow a choice of backends of either fields or components on a form by form

I think this would add incredible complications to both the Webform and Entityform. Most of this lies in the fact that everytime you create an Entityform you are creating a new bundle(part of why it won't scale) and everytime you create an Webform you are just creating a Node entity(though could be different one for d8).

Will some D8 core contributors who created the need for this new version contribute to its design and implementation?

I would have read through this thread again but the fact is there is not a NEED to completely change the architecture of Webform for Drupal 8. Webform doesn't need to use the field api anymore than it did in Drupal 7.

The Field API is great for a lot of things but it is not great for the huge scaling of number of forms(thousands?) and number of components(hundreds per form?) that Webform. It is also great for handing of the ability to make forms on an ongoing basis as "content" that webform is good for.

@DanChadwick I think I misread your comment earlier. Yes I think Webform will need a major re-write because it should be class based.

Thanks for chiming in @tedbow. The more I've been thinking about what combining webform and entityform would actually mean the more my head starts to hurt! If anything combining entityform with the great work that's going into the core contact module makes a lot of sense.

The need for a scalable webform module in D8 is real. Which is why I'm working on getting webform-7.x-4.x ported to D8 adopting D8 design principles and APIs. There is a lot of work ahead.

I just discussed performance implications of lots of bundles with yched and berdir. It's actually not so bad, as in most situations we only load field definitions by field and bundle. However, we've got a field map which covers the use case of "get me all fields of my field type" which loads all entity bundles and fields in memory - it's EM::getAllBundleInfo(). It's a known problem and currently hardly used, there are some uses in admin UI *and* a usage by comment module. The usage by comment module probably should be fixed, as it's applicable to node/entity views - but that should be doable.
The other problem, is entity storage which is going to load field storage definitions for the entity type. That could be circumvented by doing an improved storage controller for webform that is more careful with when to fetch which storage definitions.

@fago, thanks for your input.

I just discussed performance implications of lots of bundles with yched and berdir. It's actually not so bad,

As "lots of bundles" I think there might be some sites with thousands of Webform. Is that the scale you were refering to by "lots".

For thousands of bundles would there implications if some modules implemented hook_entity_type_alter and looped through entity types bundle, like display suite does

That could be circumvented by doing an improved storage controller for webform that is more careful with when to fetch which storage definitions.

Would this still allow webform to use standard Drupal fields? Are fields tied to a entity type storage class in any way?

I think there are sites with 10's of thousands of webform nodes. And there are sites with 100,000's of submissions for a single webform. A field type can be (and usually is) used many times per bundle.

I was hoping there would be a way to plug in either fields or components and be able to reuse features like multi-page, drafts, e-mail triggering, conditional fields, analysis, downloading, and reporting, leaving the storage and (probably) widgets to be implemented either as fields or components. It sounds like this may be too ambitious.

In addition to the numbers @DanChadwick outlined there are also webform nodes with hundreds of fields (400+).

Tagging. And in the issue summary, please mention https://www.drupal.org/project/contact_storage as another option.

@heddn -- This issue relates to the porting of webform to D8, rather than a comparison of alternatives to webform.

There is a documentation page that compares webform to entityform. Maybe make reference to contact_storage there??

re: #64
I'm looking at it from a feature replacement path. Right now the issue summary is fairly broad and doesn't seem to consider any of the design concepts that are introduced in D8. Shouldn't the summary incorporate or at least rebuttal the long string of comments/questions? To that end, I'd think that listing alternate modules to solve the problem space is a good thing...

User interface changes
None, we're aiming for feature parity with webform-7.x-4.x.

API changes
None, we're aiming for feature parity with webform-7.x-4.x.

@heddn Fair point. I've made some adjustments to the issue summary.

I quickly skimmed through this and I think an initial port of Webform more or less as-is, aside from the adoption of core APIs that are easy to leverage is a sane plan. If anyone is still willing to try more ambitious approaches, I'd like to point out that the new Entity Storage API supports also storing fields in shared tables, besides the regular dedicated field tables. In core, only base fields (which can be defined also by contrib modules) are stored in the base table(s), but it should be possible to write specialized storage (controller) + storage schema classes that extend core ones and do fancy things like storing all fields belonging to each bundle in a single data table. This should alleviate load/write performance issues, although it does not address the bundle/field scalability issues.

@plach Thanks for that. The new Entity Storage API sounds very promising, though as you say it wouldn't address the bundle scalability (i.e. thousands of forms). Would be great to see someone step up and tackle this, even just as a raw experiment. I've hardly got bandwidth enough to focus on the raw port to D8 so while I'd love to I'm not putting my hand up for that one!

@plach -- Thank you for the expertise. It is appreciated. If I understand you, D8 fields can be stored as additional columns in a table that represents the particular entity, such as they way "status" is stored for a node (at least in D7). While this would help other use cases, I don't see it being ideal for webform, unless you have a very small number of webform entities (each of which is rather like both an entity and an entity type).

Webform -- for better or worse -- stores its data in one table. Alas, the table is not keyed by useful machine names (such as webform machine name and component-path keyname. I would like to see this addressed in D8, as it would make views much more powerful.

The single table does have the advantage of not continually growing tables or columns as additional webform nodes/entites are created. In my use case, end users, not the administrator, create the webforms.

It seems like one could *almost* unplug the entity storage back end. It seems that the details of the data definition of an individual webform aren't sufficiently abstracted, if I understand correctly.

@DanChadwick:

Yes, core storage classes store base fields (that is fields shared among all bundles) in the entity base table. Actually, other tables may be created if the entity supports revision or translation (for instance, see the node schema on a plain D8 install), but that's the key concept.

But what I was trying to outline is that storage is fully pluggable now, thus if you defined a custom entity for submissions, you could have a separate table for each bundle (like CCK's per-bundle storage) storing just field data. Or you could even write a storage class relying on the current schema approach, where all components are stored in a single table, and still exploit the Entity API for all the rest. The current Entity Storage API should flexible enough to support that, and if it's not it's a bug :)

One more thing: the Entity Storage API includes an Entity Schema API that allows to notify the system new bundles/fields were created and the system will take care of automatically generate/update the schema (when data migrations are not involved). Also this part is swappable and it might be not so hard to adapt it to the use-cases we were mentioning here.

If a huge number of fields/bundles is not actually an issue (which is a big if :), as @fago seemed to suggest above, this might be actually a viable approach.

I agree with some of the previous posters that the most practical option is to port the D7 schema to D8. Making the greatest possible use of D8 API's (e.g. widgets) is a good idea, as well.

In principle I like the idea of using the Field API both for maintainability and ease of integration with other widely used modules like Views and Rules, but making it scale to hundreds of fields is a big unknown at this point. D6 CCK put all single-value fields into the same table, and Per-bundle Storage module took this a step further by adding finite multi-value fields in the same table, as well. But this poses a problem for shared fields (which D8 still allows amongst bundles of the same entity type), and unlimited cardinality fields would still need to be spun into separate tables.

I think something like Entityform or the new Contact module will meet a large majority of use cases. It's those sites that require hundreds of fields per form (and/or thousands of bundles if bundle scalability ends up being a real issue) that would be left out in the cold without an alternative like Weborm. To solve read/write performance issues they could turn to a denormalized backend like MongoDB, but that a) brings with it some negative consequences (e.g. no joins), b) obviously not every site is able/can afford to implement it, and c) this may not solve all issues if there are other problems with Field API or Entity API type scalability.

I came across an interesting and relevant discussion of the Field API history and improvement in the backdrop issue queue:

https://github.com/backdrop/backdrop-issues/issues/56

@mesch: Thanks for the backdrop thread link. Interesting reading.

At this point, I don't see how we can use entities for webform 8 and support having hundreds of different webform nodes. Unless the fields within the entity definitions themselves can be moved into a webform-specific storange engine, we would end up with 10^3 to 10^5 content types. I'm sure core won't like this.

Here's a related schema issue:
#2453511: Define data storage schema for webform 8.x.

For people interested in learning more about the Entity Storage API, I just wrote this:

http://drupalwatchdog.com/blog/2015/3/entity-storage-drupal-8-way

@plach -- I *just* read this. Thanks for the link. And the article, of course. Very helpful.

You are welcome :)

As a way of getting more familiar with the innards of Drupal 8, I've started porting Entityform to D8.

This and other discussions have made me curious about the scalability of bundles and fields in D8. In the drush branch I've written some commands to help explore performance characteristics/issues.

To create entityform bundles, use:

drush ebct [number_of_bundles_to_add]

To delete all bundles created by the previous command, use:

drush ebdt

To add integer fields to a bundle, use:

drush ebaf [number_of_fields_to_add] [bundle_machine_name]

I started benchmarking performance by creating a bundle "test", and then adding 300 fields to it via:

drush ebaf 300 test

Then I used xhprof to profile the following pages:

1. Manage fields: /admin/structure/eform/test/fields
2. Manage form display: /admin/structure/eform/test/form-display
3. Manage display: /admin/structure/eform/test/display
4. User-facing submission form: /eform/add/test

System details:

• 8.0.0-beta9
• Ubuntu 14.04
• Mysql 5.5
• PHP 5.5.9 w/ opcache enabled (opcache.memory_consumption=256, opcache.interned_strings_buffer=16, opcache.max_accelerated_files=20000)
• MongoDB 3.01 (for config and cache backends only)
• VirtualBox w/ 2 cores at 100% w/ 2GB RAM and spinning disk

Manage Fields

Overall Summary	
Total Incl. Wall Time (microsec):	4,498,783 microsecs
Total Incl. CPU (microsecs):	4,385,966 microsecs
Total Incl. MemUse (bytes):	24,693,000 bytes
Total Incl. PeakMemUse (bytes):	30,538,208 bytes
Number of Function Calls:	833,234

Drupal\Core\EventSubscriber\MainContentViewSubscriber::onViewRenderArray -> 3s
Drupal\field_ui\FieldConfigListBuilder::render -> 1.4s (including 0.4s to load field configs)

Manage Form Display

Overall Summary	
Total Incl. Wall Time (microsec):	9,204,364 microsecs
Total Incl. CPU (microsecs):	8,926,458 microsecs
Total Incl. MemUse (bytes):	35,949,640 bytes
Total Incl. PeakMemUse (bytes):	58,478,456 bytes
Number of Function Calls:	1,350,454

Drupal\Core\EventSubscriber\MainContentViewSubscriber::onViewRenderArray -> 7.3s
Drupal\Core\Controller\FormController::getContentResult -> 1.8s

Manage Display

Overall Summary	
Total Incl. Wall Time (microsec):	10,424,286 microsecs
Total Incl. CPU (microsecs):	10,102,690 microsecs
Total Incl. MemUse (bytes):	39,796,648 bytes
Total Incl. PeakMemUse (bytes):	65,067,256 bytes
Number of Function Calls:	1,601,731

Drupal\Core\EventSubscriber\MainContentViewSubscriber::onViewRenderArray -> 8.2s
Drupal\Core\Controller\FormController::getContentResult -> 2.1s

Submit form

Overall Summary	
Total Incl. Wall Time (microsec):	3,594,923 microsecs
Total Incl. CPU (microsecs):	3,510,328 microsecs
Total Incl. MemUse (bytes):	27,928,032 bytes
Total Incl. PeakMemUse (bytes):	34,421,968 bytes
Number of Function Calls:	563,867

Drupal\Core\EventSubscriber\MainContentViewSubscriber::onViewRenderArray -> 2.2s
Drupal\entityform\Controller\EntityformController::add -> 1.3s

Observations and Thoughts

• There are performance issues on the front-end as well because the browser has to process a lot of javascript and css to render a page with this many fields.
• Most of the time appears to be spent rendering the page, rather than fetching content and config and building render arrays. This is especially true for the manage form display and manage display pages.
• Perhaps we could look into pagination to reduce the amount of rendering that takes place per page. Certainly on the front-end I can't imagine anyone wanting to expose a form with 300 fields on a single page to an end-user; you'd probably want a multi-step form.
• Are there ways to improve render performance? These renders times seem extremely high.
• CPU load is high in all tests, whereas memory usage is comparatively low.

Lowering the priority not because this isn't a vitally important question, but to allow room for critical porting tasks to stand out.

Re #78: Not as bad as I expected, although that's a pretty spiffy test environment. Would would also be interesting is how the site performs when there are, say 100 or 1000 entityform nodes, each with 300 unique fields, each with a thousand submissions. What might or might not scale?

Is there a way to "borrow" widgets from fields and use them as components (keeping the webform schema)? It appeared not, but this idea is very interesting.

If my results are representative and I'm interpreting them correctly, it seems like actually turning the render arrays into HTML is taking most of the time. So even if we did use fields with a custom storage handler that implements the webform schema - which looks to be possible - the render process would still be a very time consuming process.

That said, it's possible something in my setup is causing these large render times. It's also possible as the API solidifies significant performance improvements will be made.

I'm wondering how these results compare to Webform performance in D7? I don't have any experience with Webforms this large.

As for where it's possible to use field widgets without using fields, I don't know.

@mesch Thanks for that profiling, interesting results. If the rendering is the bottleneck that is unexpected.

We do have experiences with webforms this big, sometimes 600+ components. However they're often, but not always, multi page forms. @DanChadwick makes a good point in that performance would get interesting with hundreds/thousands of entityform nodes each with hundreds of unique fields (the submissions should be a moot point as the entity system is fine with large volume of entity submissions).

As @plach and others have made clear, the D8 Field API is flexible enough to change storage/schema implementations, basically to minimise required tables. If, as per your profiling, rendering is the bottleneck then that is an important factor to consider. All possible, just needs to some to further explore it. I'm focusing my limited time on the D8 port.

Re: #79, I've experimented with scaling with more types and fields and the results aren't pretty.

// Add 1000 content types.
// This executes very quickly.
drush ebct node 1000

// Add 10 fields to each content type.
// This starts off fast, but gets progressively slower.
drush ebaf node all 10

After doing this, when I tried to delete a single field via the UI the request timed out. I ended up deleting the database and re-installing (no big deal as this was for experimental purposes only).

Digging into core, these are the possible reasons I've identified for why fields aren't scaling:

(Background: 'field_storage_config' is a config entity that stores how a field is stored in the database and is per entity, whereas 'field_config' is a config entity that stores details of how the field is to be used with a bundle of that entity. So there is just one instance of 'field_storage_config' per field per entity, whereas there can be multiple instances of 'field_config' for the same field)

1. A serialized list of all field storage definitions is stored for each entity type in {cache_discovery} (e.g. {cache_discovery.cid} = 'entity_field_storage_definitions:node:en' for nodes)
2. A similar serialized list of all field storage definitions is stored for each entity type in {key_value} (e.g. {key_value.name} = 'node.field_storage_definitions' for nodes)
3. At various points of the code all storage definitions are being loaded. For example, see \Drupal\fields\Entity\FieldConfig->getFieldStorageDefinition().

So it looks like storing and querying the field storage config entities by entity is a problem. A lot of other stuff is inserted into the {config}, {cache_*}, and {key_value} tables that looks like it would scale with an appropriate stack setup.

I have considered the possibility of overriding the \Drupal\fields\Entity\FieldConfig and \Drupal\fields\Entity\FieldStorageConfig entities just for webform/entityform so that we can control how all of this configuration is stored and cached, but so far I don't see a way.

It looks like changes to core would be required to make fields more scalable. If there is an appetite for this I can open an issue.

Another possibility I'm thinking about is using fields but bypassing field and field_ui modules entirely. The field module is there to basically handle field configuration/storage, while the field_ui module is there to handle field CRUD operations for those configuration entities. Base field classes and interfaces reside in core (see \Drupal\Core\Field), not field module. Could we possibly implement our own custom field configuration and UI? We could perhaps even ignore the new config system entirely and use custom tables as per webform in D7.

This approach would give us access to field types/formatters/widgets via the field plugin manager, and allow us to use the content entity infrastructure (e.g. bundles, translations, revisions, etc.).

The content entity code lives in \Drupal\Core\Entity and doesn't depend on field or field_ui modules.

I have looked through fields defined in modules (e.g. link, options, telephone), and although they show a dependency on fields, I don't believe they need to. The one module that gives me pause over this approach is entity_reference which has a hard dependency on fields module.

On the topic of custom storage of field info, according to \Drupal\Core\Field\FieldDefinitionInterface->getSchema(), possible schema keys for fields are:

   *  - columns: An array of Schema API column specifications, keyed by column
   *     name. This specifies what comprises a single value for a given field.
   *     No assumptions should be made on how storage backends internally use
   *     the original column name to structure their storage.
   *   - indexes: An array of Schema API index definitions. Some storage
   *     backends might not support indexes.
   *   - foreign keys: An array of Schema API foreign key definitions. Note,
   *     however, that depending on the storage backend specified for the field,
   *     the field data is not necessarily stored in SQL.

So if we were to implement the D7 webform storage in D8 using fields, it looks like it would be a matter of fetching the value from the columns key, and serializing if more than one column.

Performance and scaling is one of the main D8 focus these days, if there is a way to make the field system more scalable, we should definitely open a core issue to do that. Depending on the perf gain it can even be marked as critical and thus block the D8 release.

I created a core issue:

Issue #2473983 - Evaluate Entity Field API Scalability

Per @plach's ideas in #68 and #71, I've written a ginormous patch against 8.x-4.x that:

1. Creates webform entity scaffolding.
2. Creates custom storage and storage_schema handlers for the webform entity type that implements a webform-like EAV db schema. All config fields (that don't control their own storage) of all bundles have their values stored in one table {webform_temp_submissions}.

To keep things as simple as possible for this proof of concept, revisions and translations aren't supported. And obviously no test coverage.

So far I've manually tested against link (multi-column field schema), entity reference, integer, integer multivalue, and text with summary field types.

You may need to do a drush cr all after install depending on what version of D8 you're using. I developed this against 8.0.0-beta9 and also seems to work against HEAD.

I'll do some benchmarking on load/insert later.

As you see there's a lot of boilerplate code to define entities, that's why there's contact_storage so you can re-use contact entities just swapping a storage

Here's some nitpics, while scimed

1. +++ b/src/Form/WebformForm.php
   @@ -0,0 +1,83 @@
   +  public function save(array $form, \Drupal\Core\Form\FormStateInterface $form_state) {
   +    $webform = $this->entity;
   +    $insert = $webform->isNew();
   +    $webform->save();
   ...
   +    if ($webform->id()) {
   +      // Tell the submitter what happened, and log the action.
   +      if ($insert) {
   

   The proper way to check result of save()

2. +++ b/src/Form/WebformTypeForm.php
   @@ -0,0 +1,98 @@
   +    $status = $type->save();
   +    if ($status == SAVED_UPDATED) {
   

   like that

3. +++ b/src/Form/WebformTypeForm.php
   @@ -0,0 +1,98 @@
   +    elseif ($status == SAVED_NEW) {
   

   should be just else

4. +++ b/src/Storage/WebformStorageSchema.php
   @@ -0,0 +1,128 @@
   +    // @todo Add purging to all fields: https://www.drupal.org/node/2282119.
   ...
   +      // @todo Revisit this once we are able to instantiate the table mapping
   +      //   properly. See https://www.drupal.org/node/2274017.
   ...
   +    // @todo Remove when finalizePurge() is invoked from the outside for all
   +    //   fields: https://www.drupal.org/node/2282119.
   

   nice

5. +++ b/webform.links.action.yml
   @@ -0,0 +1,11 @@
   +    ¶
   
   +++ b/webform.links.menu.yml
   @@ -3,3 +3,16 @@ webform.settings:
   +  ¶
   ...
   +  ¶
   
   +++ b/webform.links.task.yml
   @@ -2,3 +2,24 @@ webform.content_overview:
   +# Webform type  ¶
   ...
   \ No newline at end of file
   
   +++ b/webform.module
   @@ -697,11 +697,14 @@ function webform_theme() {
   +     * ¶
   
   +++ b/webform.routing.yml
   @@ -12,3 +12,82 @@ webform.settings:
   +    ¶
   +##### Webform Types #####     ¶
   ...
   +      ¶
   ...
   +    _entity_form: 'webform_type.edit' ¶
   ...
   +        ¶
   ...
   +    _entity_view: 'webform'  ¶
   ...
   +    ¶
   ...
   +    ¶
   ...
   +    _entity_form: 'webform.edit' ¶
   ...
   +    ¶
   ...
   +  ¶
   

   trailing whitespace

6. +++ b/webform.module
   @@ -697,11 +697,14 @@ function webform_theme() {
   +    /**
        'webform_form' => array(
          'render element' => 'form',
   ...
   +     */
   

   entities needs base template, there was an issue about that, you should get notice in watchdog

Re: #86 save benchmarking:

I used a fresh 8.0.0-beta9 install for each storage handler. The times don't measure just the actual SQL query time but everything that happens with the save() method (e.g. mapping fields to the schema).

@andypost, thanks for taking a look. I learned a lot about D8 core through this proof of concept exercise. It looks like I need to fix my project's/IDE's whitespace trimming settings for YAML files.

I agree, if - and it continues to be a big if - we can solve the entity field api scalability issues, there's probably not much point in having a separate webform module what with the work already put into contact and contact_storage.

@mesch - Can you elaborate?

I agree, if - and it continues to be a big if - we can solve the entity field api scalability issues, there's probably not much point in having a separate webform module what with the work already put into contact and contact_storage.

I'd love nothing more than to abandon webform D8 to a core replacement, but I was unaware that the D8 contact modules was even vaguely capable enough to replace webform, except for very simple contact form needs. I'm wondering about things like (off the top of my head):

1. Multi-page
2. Conditional logic (including intra-page jQuery)
3. Drafts and draft-resuming
4. Analysis
5. Batch downloading
6. Extremely flexible e-mailing
7. Self-service submission management (list of your own submissions)
8. Anonymous submission tracking
9. Submission limits (per-user/global, time-limited)
10. Survey-oriented components -- e.g. grid
11. Fieldsets and other layout tools (e.g. webform_layout module)

@DanChadwick, what I meant to say was it probably wouldn't make that much sense to have a *separate* module that provides the base entity/field scaffolding since contact + contact_storage does that already. In that case existing contrib modules could address many if not most of what you list here; there would probably still be a need for add-on module(s) to bridge the gaps in functionality and/or enhance the user experience. And again, this is only if the entity field api scalability issues can be addressed; my read of the consensus today is they can't.

At least that's my take on things as of now.

@mesch Very interesting proof of concept.

If the field API is scalable (happening in #2473983: [meta] Evaluate Entity Field API Scalability right?) in terms of thousands of bundles (i.e. different webforms) and hundreds of fields on each then it'd seem like contact and contact_storage would basically take over entityform functionality. The many additional feature of webform as @DanChadwick mentions in #90 might then live in contact_extra or the like? There's a lot of "ifs" there, all pending the scalability of the field API in terms of bundles/fields.

@fenstrat re: #91 yes I think that sounds about right. And yes, lots of ifs.

I should be clear that even if fields/bundles can scale (the big "if" - maybe better to look at for D9), we'd need to make a number of changes to various contrib modules to accommodate large numbers of fields (some of which have already been covered above). A few examples:

- The add field form allows you to select an existing field; need to turn that into a drill-down form, or perhaps autocomplete so that not all (thousands... hundreds of thousands) of fields are exposed.
- Views; similar need for drill-down/autocomplete.
- Same with rules, other contrib modules.

So really it would be a considerable amount of work. The beauty though would be that it would benefit all fieldable entities in core/contrib, not just one use case.

The add field form allows you to select an existing field; need to turn that into a drill-down form, or perhaps autocomplete so that not all (thousands... hundreds of thousands) of fields are exposed.

Is it possible to configure a field that is entity-type + bundle specific? I thought I read that fields in D8 can't be shared between entity types, but can between bundles of the same entity type.

This bring in some thoughts from other issues about portability (e.g. using features for webforms and views of webforms) and identity (uuid).

I have also been wanting a way to categorize or group webforms that are similar or (nearly identical). In my use case, I have a great many webforms, which are clones of one of several master webform templates. In specific, each relates to evaluations at a particular event.

So what I would like is a "webform group machine name" which could be used to classify a webform. With that, I can more easily create views (or other reports) on a specific webform (e.g. event) or a group of webforms (e.g. all the evaluations for events from a particular time period, hosted by a particular organization, etc).

In addition, I would like fields to be shareable between webforms in the same "group", but not within other groups. So a component might be identified not just by cid or by form_key, but by group:component_machine_name. This is how views written for a particular webform group can operate on the right component, and do so without having to load the webform.

With this, I can write a view on all webforms of in a particular group, expose additional filter, and include components by their group:machine name, regardless of their cid or form_key. All without loading the webform node(s) -- directly from the database.

I raise these issue here for thoughts about how this might work in D8.

Re views for webforms and features:
#2412497: Make views integration more portable

Where are we maintaining the Drupal 8 version of Webform module?

I too am planning to lend a helping hand in porting the module to D8 and releasing the beta version of Webform Module..

@Nitesh Sethia work on Webform for D8 is happening right here in the issue queue. Patches most welcome.

Where can I find d8 release of Webform? Or is it too early to try webform d8?

It's too early to use. It's not too early to code. :)

You are considering moving webform components over to be entity fields?? As has been stated, you're going to limit the scalability of webform by doing this (for refernce see the entire history of Drupal and software design). Can someone clearly explain to me the benefits of switching over?

I'd recommend that you leave webform to manage its own component/submission tables/data to start with -- of course there will be forms API stuff to update, but otherwise it should be fine as is no? No schema or model changes for example. Upgrade the rest of Webform to work with Drupal 8, let the D8 version stabilise, and then consider switch to entity fields in an 8.x-5 version?

Re #100 -- No, after discussion, the D8 team is no longer considering using entity fields for components due largely to scalability issues.

Regarding #100, there's zero point in Webform using normal entities, at that rate sites might as well use EntityForm/eForm.

The mockups/tests discussed above and in related threads suggests that using fields/entities is a no go for D8 despite advances like a pluggable storage backend end allowing us to use the same EAV DB schema. The obstacle remains the scalability of the config system and likely UI issues with having so many bundles.

In my opinion it remains a worthwhile goal to ultimately use fields instead of components given their similarity and the additional development and maintenance time involved in creating a separate set of plugins - really a whole different ecosystem - for which we can all thank DanChadwick, fenstrat, and others.

I vaguely remember someone, somewhere, suggesting that perhaps with D9 we could look at optionally decoupling fields from the config system, which would be a major step forward in this regard.

Would be great if the progress of an actual D8 port was tracked / highlighted in the summary of this issue (maybe with mentioning of sub-issues)?

@danielnolde, see #2574683: [webform] Webform

you should move to entityform (alias eform in drupal 8) that allow to create form using the entity api (and not a completly rewritten field system)

I don't see any future for webform itself, the work should be concentrated for an upgrate path from webform 7.x to eform 8.x.

@Alexandre
Webform fits a lot of usecases better than entityform.
I doubt there will ever be an upgrade path between those modules because they're so different.

Architectural suggestion:

Take a design page from multifield in D7 (and CCK in D6). For all fields with cardinality of 1, store those values with a special storage handler that puts them as properties on the base db table. For all fields with a cardinality of greater than 1, store them using the normal D8 storage handler.

The idea here is that the primary use case in D8 for Webform is for really *long* forms. Most simple use cases could use something like https://www.drupal.org/project/contact_storage & core contact, which solves +/- 80% of form needs. But there are going to be massive forms that having lots of fields just isn't going to scale. But even in the case of those *long* forms, most of the fields are cardinality of 1. So, storing them as properties get's around most of that problem. In the more rare case that a form field needs to have a cardinality greater than 1, then the author of the form can just change the cardinality and the field will get storage as per normal D8 methods.

There is no export or analytics functionality in Contact form like there is in webform.

There is also no page break option in the contact form. Really needed for longer forums. Question. I am testing the 8.x-4.x branch is there a reason it isn't listed as a developmental branch on the project page?

Thanks
Robert

I'm going to have to agree with @Alexandre360. Webform was designed around Drupal 5 (4?) for the specific scenario of providing a contact form that was above and beyond what was available at the time. In my mind, the 4x branch should have been written around the entity / field structure of Drupal 7, but it was not. Now that we are sitting at a major fork in the road, I think the efforts would be far more efficient to move people to the new D8 entity form field system as opposed to re-writing Webform from scratch to live outside of both what is already available and the standards of what has been designed.

Me personally, I would rather just see a really good migration platform to get the data from Webform, into Drupal 8. No offense to any of the D8 maintainers, but if quicksketch isn't going to be around to lead this project, there's really not much reason to keep Webform alive in its current form for D8. The contact form for D8 is leaps and bounds ahead of anything before it, why not leverage and write around it?

There's also the thought of having to again port it to D9 etc. Cut the head off and migrate the data to a format that will be much easier to migrate in the future with far less overhead.

If Webform will require an entire rewrite, why not spend that time making the Drupal forms work more like the older Webform? From a logistics standpoint, it doesn't make a whole lot of sense to port this when there's already a fully functional contact form available and I am very confident the only reason the majority of folks want this ported, is to save the existing data on live sites that are using Webform. Seriously, what IS the driving motivation to port this?

Webform is a great module in it's current form, but I feel that if the lead maintainer is looking to move on, it's time for the community to move on as well. Migrate, not port, Webform through the course of Drupal 8, that way it can officially die with Drupal 9 and continue to live peacefully in Backdrop.

@philsward, the main difference between webform and contact form imo is the target audience.
Contact forms are config (= developers)
Webforms are content (= editors)

You don't want editors to create 100+ fields on a website.
So I'd state that, unless you can leverage contact forms as content, there is still a valid use case and added value for webform.

@mpp if we could enable parts of config not to be managed by CMI that problem could be tackled imo.

@philsward about migrating webforms to D8 entitfy field stuff, that could be doable. We have the migration destinations in core. So an migration from a webform to an entity, from components to fields and form displays should be doable.
I once started something (but never finished) as a POC. Pushed the code real fast to https://www.drupal.org/sandbox/mallezie/2752175

As @mpp said in #113 - primary difference that field system is not scallable enough like webform does and if you need hundreds of different Forms (with different fields) you'll need other data model - that's exactly webform case

I also would like to remark what @mpp said in #113. Our team has been involved recently on a migration to D8 from a D6 site with nearly 200 webforms, in some cases with dozens of fields, managed by tens of users. In that case, contact form + contact storage is not a solution, because apart of having to deal with the migration or missing funcionalities (not all functionalities of webform are available yet on CF+CS), we think that give permissions to editors to manage entity fields could be a little dangerous and complex for them. As a workaround we've migrated webform content types to an individual D7 site, to get some time while we see what occurs with webform in D8.

Hi, I think we are coming to the conclusion that there may not be a port of the Webform module to D8. It is simply too daunting of a task.

As stated in #33, the Contact Storage module should be able to address 80% of Webform's use cases. I think everyone is concerned about the other 20%. Personally, I really need this other 20%, which for me includes performance, scalability, customizability, and a simple workflow/UX...build a form, collect submissions, and download the results. So I have been building out the YAML form module for the past 6 months.

The YAML form module was originally intended to be a somewhat temporary developer centric FAPI based form builder for D8, but it has slowly been reaching feature parity with the Webform module. Last week, I added a UI and this module is now becoming more of a site builder centric form building solution. The YAML form module can definitely handle 100's of fields with 100,000's of submissions but I am not ready to say that it can handle 10,000's forms.

Please note this thread is about porting the Webform module to D8, so any ongoing discussion about the YAML form module should be moved over to the YAML form module's issue queue.

Related to the Webform module is the Form Builder module. Has anyone considered working on it, or something like it, to improve the core field UX?

@jrockowitz please update issue summary with yamlform, it looks great and real competitor to webform! Also please check #2582955: Contact module roadmap: 80% usecase of webforms in core

Adding YAML form to issue summary.

There is a place for a form builder module that doesn't use entity/fields - as someone who cares about what goes into the database, I don't want site editors creating hundreds of tables for their random surveys. This is not a question of whether webform (or an alternative) still has a place in D8, it's a question of whether anyone wants it enough to do the port.

Given some of the discussion, my question now is:
"Would it be possible to leverage Drupals contact form but use a different backend storage mechanism?" Instead of the fields creating new tables, have a single table that houses the same fields. It probably needs to be approached as a separate generic API where a mass number of "fields" are needed by XYZ module, then create a specific module to tie the contact forms to the alternative storage mechanism.

To me, it makes more sense to create a separate storage system that will handle (all) use cases not covered by core, instead of porting a single module because "cores storage" won't cover the 20% which then locks that custom storage mechanism into a single module.

If the cost for repairs for your car, exceed the value of your car, you're probably better off getting a different car. That's what we have here. The cost of porting webform is exceeding the value of what it's worth. Either way, it sounds like it has to be rewritten from scratch... what's the logical route?

@philsward, see #88 in this thread, and the posts above for discussion of using a different storage back end.

When I experimented with this it was possible, and the early results were promising, but it still doesn't get around other scalability issues associated with the field system that I believe are show stoppers to this approach in D8. I think a significant goal of D9 should be to enable developers to decouple fields from the default config system (to get around scalability issues), or to deal with those scalability issues head on.

Just dropping again, that scalability is more an edge case IMHO versus allowing contact forms to exclude from config.
I think most cases won't hit scalability issues with contact forms, and the current entity / field implementations.
#2473983: [meta] Evaluate Entity Field API Scalability

Just in case people missed it:
https://www.drupal.org/project/eform

Just to add to this, YAML Form now has a complete UI and really covers all that webform for Drupal 7 covered as far as I can see:

• Contact messages not stored in fields for scale
• Multistep forms
• Extensive number of fields (likert etc)
• Multiple emails on submission
• Table views for results
• Download to CSV with numerous options
• Honeypot integration
• Per form access controls

Contact messages not stored in fields for scale

Even performance for large forms seems to be addressed with the YAML Form module. :-)

YAML Form: I guess the security implications would be a show-stopper for many webform use-cases:

This module allows developers to have full access to Drupal's Render API, this includes the ability to set callbacks, which are PHP functions that are executed during the rendering process. This means anyone who can administer and build a YAML form can call any PHP code on your website.

Only the most trusted users should be granted permission to administer and build YAML forms.

But it's very close! With a more limited interface for editing this forms this could well be a good replacement for webform in D8.

Hi torotil,

Actually we've resolved that in this issue. You can remove (or not grant) access to edit the yaml for untrusted roles.

Is that what you mean?

As long as there is no way to configure things as #preprocess in the interface, then yes this would solve that too.

Hi, I am the maintainer of the YAML form and I can confirm this security issue has been resolved. I just removed that warning from the project page.

Since the YAML form module is still in beta it has not been reviewed by the Drupal security team. As I get closer to an RC release, I will reach out them. Right now, I really need people to start using and testing the module.

Finally, I just posted #2765691: [meta] Roadmap - YAML Form 8.x-1.0-rc - Release Candidate

As I understood in YAML form some things need to be written in code instead of through a UI, compared to Webform. For instance conditional fields. That might be a show-stopper for many who offer Webform to site-editors.

Also YAML forms (whilst looking very promising for some use cases) uses the Form API #states system (eg. javascript) for conditional branching, so is much less powerful than the webform approach.

The UI does allow you to set conditionals now; however in yaml format. It's on the road map to make a non-yaml UI for it.

In terms of conditional functionality via states vs webform, I'm not so familiar with what is used under the hood in webform: is there a particular feature that is missing? I'd imagine since its so close to a full replacement for webform that adding what is missing as a feature request would be the best course of action perhaps.

One of the main goals of the YAML Form module is to expose (and occasionally extend) Drupal's native APIs, which includes forms and #states. I completely agree that the current #states system is less powerful than the webform approach. The #states system in core needs to be improved and this might be something that gets worked on by me or someone else while people begin using the YAML Form module.

Yes, @scott_euser and I are trying to add a UI to all the common form element properties (including #states and #options). This is a bit of work and is going to take some time.

At the same time, the YAML Form module is currently the only online form builder (with a UI) that developers can also 'view, understand, and edit the source code' behind the form.

I think everyone should remember that YAML Form is a new module built completely from scratch, not a port of an existing module, and hasn't reached v1.0 yet. If there's something that it doesn't have which you wish it did, please get involved in its issue queue with feature requests and patches, that'll get a working Webform-alike solution than putting effort into a direct port of Webform.

I have posted a sandbox project that performs a D6 Webform to D8 YAML form migration at YAML Form Migrate.

Nate Haug – ‏@quicksketch

YAML Form may become the official Webform 5.x for Drupal 8. All maintainers coming to agreement at https://www.drupal.org/node/2805097

https://mobile.twitter.com/quicksketch/status/793668919206617088

Now that #2827845: [roadmap] YAML Form 8.x-1.x to Webform 8.x-5.x is done, and there is a 8.x-5.0-beta5 release, should this issue not be closed?

I am closing this issue and anyone interested in tracking the progress of Webform 8.x-5.x should follow #2574683: [webform] Webform.

Yeehaw! :D