DIscussion Summary (9/13)

- Pretty much everyone is on board conceptually with Create.js being a good path forward.
- Many people though are concerned about Create.js meeting the D8 feature freeze timeline, especially with all of the various dependencies of doing it the "right" way (JSON-LD, RDFa improvements, a de-coupled Entity Validation API)
- Even to the point where people are seriously advocating not doing inline editing functionality *at all* for D8. :(
- There are other unknowns as well, like ultimate file size / JS payload impact for mobile, how stable the underlying dependencies are, etc.

So. Here's the proposal: Treat Create.js-ification of Edit module as a *post-feature freeze* "enhancement" to Edit module / inline editing in core. (Assuming we can get it in by Dec 1) Reference: http://buytaert.net/updated-drupal-8-release-schedule

This would solve a few problems:

- We'd now have until April 1 to do this, rather than Dec 1, which helps alleviate timeline pressures causing people to question the do-ability of the entire inline editing functionality in core.

- This delay would allow TYPO3 / Symfony Full Stack Framework / other CMSes that Henri's working with to "alpha test" Create.js integration in advance of us using it in order to further flesh out and fix bugs/missing features (such as the ability to work on backend forms), rather than us having to take that on ourselves (we're already taking on a lot upstream collaboration work w/ the Aloha stuff as it is).

- Create.js's development will largely be done in December, so some of the unknowns around file sizes, dependencies, shifting library code, etc. would be nailed down at that point.

- The feature set of Drupal 8 will also be largely done in December, so we'll know the state of underlying Drupal 8 dependencies like RDFa, JSON-LD, entity validation API, etc.

Unless we hear dire cries about this plan, this is what we (the Spark team) plan to move forward with. I'll update the issue summary with this plan in a moment, and aim to close out this issue by next Friday, if not sooner so we can move forward on completing our feature enhancements for Edit module and get it on the road to D8.

Note that this should not stop anyone from working on necessary/"very" nice to have CreateJS Drupal plumbing in the meantime! A quick hit-list of at least some of those issues would be:

- #1606794: Implement new routing system
- #1778226: [META] Fix RDF module
- #1696648: [META] Untie content entity validation from form validation
- #1533366: Simplify and optimize Drupal.ajax() instantiation and implementation
- #1149866: Add Backbone.js and Underscore.js to core
- The ability for CreateJS to work on back-end forms (see comment #36)

Thanks all, for the really productive discussion here!

Summary

Create.js would allow for much cleaner code and less code to maintain. There are Drupal-specific aspects that limit the extent to which we can leverage it though.

Overall, it looks like it's a good idea to use Create.js. But can we get all of it done before code freeze (December 1, 2012)?

Introduction

Much like when we were selecting a WYSIWYG editor for Spark (#1580210: Figure out what WYSIWYG editor to use), we'd again like input on whether we should use Create.js or not. To be honest, I think the answer will be "Yes, we should use it." — many people including Larry Garfield, Stéphane Corlosquet, Nate Haug and others are in favor, varying from "at least in favor" to "very strongly in favor".

However, for the Edit module, the following in-place editing functionality still needs to be designed/implemented: validation, editing of teasers, ability to make multiple edits and only then publish the changes, accessibility testing.
Then there is also the minimal plumbing to make in-place editing in core possible, which means we need to help move some Drupal 8 core patches forward.
That will most likely mean that migrating everything to Create.js *and* getting the additional plumbing necessary for Create.js support in D8 all done before feature freeze is effectively impossible

So: if we add Create.js support, is it acceptable to do it during code freeze, i.e. after December 1, 2012?

Create.js?

In the current implementation of the Edit module, we're using HTML5 data- attributes to provide enough contextual information for Edit's JavaScript to know what it should do to provide in-place editing for each Field.

Create.js relies on RDFa-annotated content to know what is editable and uses JSON-LD to communicate with the server. You can configure on a "per semantic type" basis which editing widget should be used. So, roughly, Create.js == Edit's plumbing plus a default UI, minus the ability to use server-generated widgets/forms to edit fields.

Because Drupal already has solid RDFa support, it makes for a seemingly natural match.

To properly leverage Create.js, Drupal needs to become more RESTful and ideally would provide JSON-LD support out-of-the-box. In Drupal 8, that will be possible, thanks to WSCCI. If Drupal 8 doesn't have JSON-LD, we'd have to reimplement Backbone.sync to use a different transport layer.

Create.js depends on VIE.js, Backbone.js and underscore.js. When a page is loaded with Create.js enabled, the page will be searched for RDFa annotations, populate that into Backbone.js Models and Collections and then BackBone Views will be created for the original RDFa-annotated parts of the DOM. This way, the DOM automatically stays in sync when the data changes, whether that happens because of the user editing, or through server communications (e.g. collaborative editing over WebSockets).

Create.js consists of "widgets", which each have a different responsibility and most of which are optional. The following widgets are relevant to Edit:

  • Editable: the actual editing functionality/abstraction
  • Storage: localStorage save/restore capability, keeps track of what should be saved to the back-end
  • Workflows: retrieves workflows from the back-end and handles running them: can be simple publish/unpublish, or more complex
  • Notifications: when something has been saved, if you have local modifications (i.e. stored in localStorage but not saved to the server)

Typo3 has already adopted Create.js. As has Symfony CMF.

How Create.js and Edit would interact

See ethanw's excellent write-up of case studies for use of JSON-LD, case 2.

Advantages

  • We have to maintain less JavaScript!
  • Much cleaner code.
  • Ability to switch from one WYSIWYG editor to another without having to make server-side adjustments.
  • We automatically get functionality such as localStorage and notifications. Although the notifications would then not be routed through Drupal's theme system, unless we override the default implementation.

Disadvantages

  • A boatload of external libraries that are still (rapidly) evolving in Drupal core: Create.js, VIE.js, Backbone.js, underscore.js.
  • More JavaScript code needs to be loaded and executed, probably 30 to 50 KB minified + gzipped.
  • A "soft dependency" on the RDFa module. The site owner may not want RDFa output on every page, so then the RDFa module would be disabled. But it would need to be enabled for users who have the ability to do in-place editing, because Create.js needs the semantic annotation to be there! How that would work, I'm not sure.
    To further complicate things in this area, not every field has a default semantic annotation (and for good reason: a "text" field can be used in many semantic ways); this implies we'd need to generate local schemas (which is really, really simple and a non-issue), but that would imply potential dual annotations. (needs confirmation)
  • Create.js is only very useful on the front-end, on the back-end it is far less so. There, it would only be useful to provide abstraction for the WYSIWYG editor, and localStorage (confirmed by Henri Bergius). Create.js' Workflows is likely less useful, because we probably want that to be hook_form_alter()able etc., like it is today.
  • Create.js' funding (its development is funded by the EU) ends at the end of 2012. Henri Bergius (and his company) have commercial interests to continue its development, but there is an inherent risk there that we end up having to maintain something we didn't write.

Caveats

  • Create.js does not explicitly support server-generated forms, which we need to use Drupal's rich ecosystem of custom widgets for fields. Otherwise, you wouldn't be able to use the same widget for e.g. selecting taxonomy terms on the back-end and on the front-end. It is possible to do this though, through a custom Create.js Editing widget for Drupal. (For the record: this was not possible back in June, it's only possible since a few weeks.)
  • The ability to "just switch" WYSIWYG editors is a euphemistic simplification of reality: this assumes alternative WYSIWYG editors are equally deeply integrated with Drupal, e.g. imagine an "image caption" filter being tightly integrated with WYSIWYG editor foo and not being integrated at all with WYSIWYG editor bar, making it impossible to interact with a certain piece of content, likely displaying that content incorrectly and potentially even corrupting/deleting it, because WYSIWYG editor bar doesn't know how to deal with it.
  • In Drupal 7, the most feasible way to implement Backbone.sync, is on top of Form-based saving and AJAX-commands-based loading. This is not elegant. However, that's also how the Edit module currently does it — but it's a relatively natural fit, once Backbone.sync becomes a layer of indirection, it's less of a natural fit.

Adoption concerns

Blockers

Soft blockers (if these things don't happen in D8, we won't be able to optimally/cleanly implement Create.js)

  • No JSON-LD in Drupal 8 would imply that we have to reimplement Backbone.sync to use whatever format *is* available. Or continue using a hacky D7 "piggy backing on forms" implementation.

Existing code

At DrupalCon Munich, I sat down with Henri Bergius and we worked on migrating Edit module to use Create.js, the result is on GitHub (diff, excluding newly added files). (He did 90% of the work, FWIW — makes sense because he wrote VIE.js and Create.js, but still, impressive! :))

The code works only partially though — I was working on implementing Backbone.sync on top of the existing Form-based saving infrastructure (see the related caveat above), but was running into issues, potentially because I was exhausted at the end of DrupalCon (but certainly also because Drupal.ajax sucks — see #1533366: Simplify and optimize Drupal.ajax() instantiation and implementation to fix that).


I'm certain that I've forgotten to mention/explain some things, so I will update the above as people start pointing them out :)

CommentFileSizeAuthor
#5 create-per-type-add.png12.82 KBhenribergius
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jcisio’s picture

There should be more time, at least two more months ;-). Code is freezed (no API change) on February 1, 2013 (on Dec. 1st, 2012 it's feature freeze).

Wim Leers’s picture

I pinged Henri Bergius as well as Crell, Sun, scor, quicksketch, ethanw and Dave Reid on Twitter, to hopefully kickstart the discussion in this issue.

quicksketch’s picture

Wow Wim, nice writeup! I didn't realize that Create.js had so many shifting foundations currently (though I do understand that Create.js itself is still pretty rapidly changing). Considering we already have a huge chunk of what Create.js would give us (though in considerable less "elegant" code perhaps), I feel like we're already missing out on what the library is supposed to provide (that is, a quick bootstrap time). Of course we're also going for compatibility and low cost of maintenance, but compatibility isn't free anyway (as you mentioned) and at this point maintenance isn't going to be cheap either. Many Drupal people probably remember when Drupal 5 adopted jQuery (a good choice), but then jQuery released *at least* 2 API-breaking releases within the first year and left poor D5 with a hopelessly outdated version of jQuery that nobody supported (but that we couldn't upgrade because the APIs broke). Take that problem and multiply it by 4 new libraries (5 if you count Aloha itself) and we could be in a lot of trouble with getting all our external libraries both stable and widely supported.

In any case, neither Create.js nor Drupal (through WSCCI) are ready for integration with each other anyway. The overall idea seems like a good one, but given the overall situation this sounds like something we should keep on the backburner.

lslinnet’s picture

What alternatives is there?

Is there talk about including other framework to do similar tasks other places in D8?

henribergius’s picture

FileSize
12.82 KB

Thanks for the great write-up, Wim! I posted some thoughts on Spark and Create.js to my blog after DrupalCon.

While adopting VIE and Create.js do provide some new dependencies, I believe these are beneficial to the Drupal community in reducing the code base that you need to maintain alone, and providing useful new capabilities to front-end developers. For example, Backbone and VIE would essentially give Drupal a standard JavaScript API which could be used in mobile and SPI applications.

Another area where Create can help is collection management, which is something the Edit module doesn't yet do. I'm currently in Copenhagen helping the TYPO3 team with their Create.js integration, and collections has been a major focus area here. Here is a simple example:

Choosing entity type when adding to a collection

There are many other features in Create and VIE that are not currently aligned with Spark. But in the future they may be, and there again having a shared, maintained library will help in getting things done quickly. Maybe you'll want automatic content-based linking at some point, for instance?

henribergius’s picture

Take that problem and multiply it by 4 new libraries (5 if you count Aloha itself) and we could be in a lot of trouble with getting all our external libraries both stable and widely supported.

Certainly selecting any particular library to depend on is risky. However, the situation with Create.js is somewhat different from for example jQuery in that it has been written from ground-up with the content management systems and their concerns in mind. There are several mainstream CMSs that are shipping their Create-powered editing interfaces still this year, and this certainly encourages us to not break stuff :-)

In general, feature development in VIE and Create.js is very much driven by what CMSs need. The collection handling improvements I mentioned in my other comment, for example, came to be because I had requests for that from several CMS projects. And yet the old collection handling is still the at the base of that feature and older CMS implementations of Create also work.

Anyway, the go-or-no-go decision on Create.js is the Drupal community's to make. What I can do is to pledge my support for the effort to keep Spark Edit going with Create if that is what you decide.

MustangGB’s picture

Yay for localStorage, a way to solve the "No data lose" that I suggested at #1702250-15: [META] As a content editor, I want to make my edits quickly and then hit save

Wim Leers’s picture

#7: localStorage is only good for "not losing anything". It's inferior to incrementally saving to the server (i.e. the Drupal DB) if you want to be able to continue editing a draft on a different device (or hop from your tablet to your laptop, for example).
localStorage is certainly very useful, but it's not a silver bullet.

MustangGB’s picture

Angie expressed some performance concerns with the save to server only approach, so maybe some combination would work best.

moshe weitzman’s picture

Nice Summary. I agree with all of it - we want Create.js, but D8 might not be feasible. Personally, I'd like for Drupal to move to annual releases and thus D9 is not that much farther away. Another year (or even 2) for semantic awesomeness is a bummer, but ultimately I recommend waiting for D9 for Create.js. We'll work on those prerequisites for D8.

I don't feel that strongly about it though; if anyone wants to take this on you are very encouraged!

henribergius’s picture

we want Create.js, but D8 might not be feasible

Given that we were able to do the bulk of work needed for porting Spark Edit to run on Create.js in just two days during DrupalCon, I don't see the schedule as a particular issue. The bigger question is whether to have inline editing in D8 at all :-)

Crell’s picture

As Wim said, I'm highly +1 in concept. One of the reasons we invited Henri to speak at DrupalCon Munich in the first place was precisely because I wanted to get the VIE/Create model in front of people, because I think it is the right way to go for Drupal. At the time I figured it was Drupal 9 material, since it would be too hard to retool Drupal for it so quickly. I didn't expect Spark to be happening and for there to be a potential for anything in Drupal 8, but I'm very glad to see that it is.

That said, my biggest concern is timing, especially around Entity API and JSON-LD support. At the moment, I am somewhat resigned to core not shipping with JSON-LD out of the box and focused instead on building the plumbing to make a standard contrib implementation of it really easy. That's just what our resourcing looks like. Also, Lin Clark has identified some issues with JSON-LD that are going to make it a tougher fit for our data model, especially since the more advanced parts of the format are still being futzed with.

The RDFa dependency doesn't worry me. If someone is going to turn off RDFa to get rid of "all those ugly extra attributes", then they'll also want to turn off whatever Edit module does to generate "all those ugly extra data- attributes". It's a dependency, we deal. Non-issue.

What I would suggest we do is push as much plumbing for D8 as we can to enable a Create-based Spark, but target Spark as a Drupal 8 contrib module. Let core itself ship without an uber-fancy new-wave editor. Really, no one uses core directly and Dries' emphasis on "Drupal core as a product" is woefully misguided. (There, I said it, and you can quote me on that.) For Drupal 8, I think we're better off trying to position Spark/Edit/Create/VIE/Aloha as a go-to-contrib (maybe even available when D8 officially ships) that anyone who's anyone installs as a matter of course, like Views has been. (Or maybe Panels is the better analogy; either way...)

That buys us several extra months, as long as the core plumbing is in place, and I'd be fine with a freeze exception for "Spark-enabling features". We can revise and track with any changes upstream in contrib way more easily than in core. Then we can revisit full integration in core for Drupal 9.

What we should *not* do, and what I fear us doing, is falling into the "not core so it doesn't exist, OMG get something in!" trap. E.g., deciding we don't have time to do Create, so we rush in a non-Create Spark for core, which then makes it harder to do a full Create-based alternate version in contrib and means we have to do even more work to get a Create-based version in core for D9.

Core is not a product. Core is a platform. Drupal distributions are products.

Wim, if we shifted the target to a Create.js-based Spark as a contrib module, what becomes the for-core task list to make that easy?

webchick’s picture

I'd really love to keep dogma out of this post (what's a product vs. what's a platform) and keep it focused instead on the issue at hand.

One of the main questions before us is, given Create.js's capabilities, whether it makes sense to put it in front of us as a hard dependency in order to propose the inline editing feature as a whole into Drupal 8 core. What I'm reading here is "no, as long as the inline editing feature we push for in core still leaves open the possibility for contrib to enable full semantic editing capabilities." I'm also reading that if for some reason the dependency chain (WSCCI routing system, JSON-LD, Entity API validation de-coupled from Form API, etc.) is in place prior to the cut-off for functionality like this (it's unclear to me atm whether this would actually count as refactoring to an existing feature (subject to code freeze) or a brand new feature (subject to feature freeze)), we could even pursue semantic editing w/ Create.js in Drupal core, though the general stability concerns of all of Create.js's dependencies would need to be addressed.

henribergius’s picture

One of the main questions before us is, given Create.js's capabilities, whether it makes sense to put it in front of us as a hard dependency in order to propose the inline editing feature as a whole into Drupal 8 core.

Create.js can be an enabler for inline editing even without extensive semantic features, as we arguably already proved by making it run as a replacement engine for Spark Edit. Having the base Create.js in place on this level of course makes it easy to add more semantic features once Drupal is ready for them (through WSCCI and the other efforts). Progressive enhancement, of sorts.

It is possible to do inline editing without Create.js, as the original Spark Edit version already showed (great work from Wim!). But then the burden of maintaining the whole codebase falls on the Drupal community, while by basing it on Create this can be shared between multiple CMS projects.

Parallels can be drawn between sharing common components like Symfony or Aloha Editor, and our libraries providing inline editing features.

webchick’s picture

It is possible to do inline editing without Create.js, as the original Spark Edit version already showed (great work from Wim!). But then the burden of maintaining the whole codebase falls on the Drupal community, while by basing it on Create this can be shared between multiple CMS projects.

Parallels can be drawn between sharing common components like Symfony or Aloha Editor, and our libraries providing inline editing features.

This is definitely a valid point. Though another consideration is the JS payload size, especially on mobile. If we are trading 2K of custom JS for 80k of external JS, of which we are only using 10K, the mobile initiative guys are gonna kick our asses. :D

That's why it seems to make more sense conceptually to me to pull in Create.js at a time when we can use all of its functionality, in order to justify the increase in page size. But obviously interested in counter-arguments to this view as well.

webchick’s picture

Even more interested to have actual data instaed of those numbers i made-up too :)

henribergius’s picture

Even more interested to have actual data instaed of those numbers i made-up too :)

Sure :-)

For Aloha Editor (1.1 MB) and Spark itself you already had jQuery and jQuery UI (full jQuery UI build is 208 KB alone, but we're not loading that all).

Underscore and Backbone add 36 KB. But those are useful also outside of VIE/Create.js. For example, the Spark Edit codebase became a lot cleaner by using them.

Full VIE build is 72 KB, but we could easily provide a build without the services you are not likely to use at this point, like Zemanta and OpenCalais (services are more than half of VIE code size).

Create.js is 40 KB, but again we could drop the widgets Spark is not using, which could easily drop this to half.

So, to summarize:

  • jQuery: 92 KB
  • jQuery UI: 208 KB (can go down to ~30 KB with custom build)
  • Aloha Editor: 1.1 MB
  • Underscore: 16 KB
  • Backbone: 20 KB
  • Spark Edit: 100 KB (unminified)
  • VIE: 72 KB (can go down to ~45 KB with custom build)
  • Create: 40 KB (can go down to ~20 KB with custom build)

This means the different combination sizes would be:

  • Standalone Spark: 1.32 MB
  • Spark with Backbone: 1.36 MB
  • Spark with Create: 1.42 MB (custom build)
  • Spark with Create: 1.47 MB (full build)
  • Spark with Create and a less full-featured editor: 0.46 MB

As conclusion, porting Spark to Create either changes JS size to 107% of current, or (theoretically) drops it to 35%. It should be noted that we can still also remove some code from Spark itself as result of the Create.js refactoring.

These numbers are for JavaScript alone. If Spark needs only its own CSS (and not for instance CSS from Aloha), that doesn't add much. The current Spark Edit CSS is 32 KB, while Aloha Editor's stock CSS is 68 KB.

henribergius’s picture

Another related note: even though Create.js adds that about 7% to the download size, in practice Spark Edit can be made a lot faster with it. This is because we use in-memory Backbone models and views, which enables us to remove a lot of DOM parsing that Spark used to do for the same things. DOM is generally quite fast, but the amount of operations adds up.

Currently the Create.js version of Spark is actually a bit bigger in code size than original Spark, but this is mainly because we didn't yet remove most of the code made reduntant by the port. Once this is done, the code size of Spark itself should drop some 20-30%, and the rest should be easier to maintain thanks to the MVC model Backbone gives us.

Also, thanks to the editor abstraction in Create.js, Spark could potentially serve different editors in situations where for example more accessibility is needed or the download size is critical.

Crell’s picture

OK, in hindsight I went a bit overboard in my earlier comment. Sorry about that.

To rephrase, I'm concerned that the dependency chain to really take full advantage of Create.js won't be in place by code freeze, or will end up in contrib for a version. Given that, I think we should be open to the entire Aloha/Edit/Create.js chain living in contrib and then not putting anything new user-facing in core, and not view that as a failure at all but as a strategic decision to make sure we get the best possible system out of it, and can refine it according to contrib rules, not core rules. The fact that it's not "in core" doesn't diminish how seriously awesome it would be to have a fully Create/VIE/Edit-ified Drupal 8.

Technically yes that means another version of "core doesn't have a WYSIWYG out of the box", but I'd much rather have a smoking awesome easy-to-install contrib solution that everyone agrees on than a not-quite-there-due-to-scheduling core implementation just so we can check that line item off.

Of course, if we can pull off the smoking-awesome version in core within the time available, I will certainly be among those cheering. :-)

webchick’s picture

Wow, thanks a lot Henri! That level of information is super valuable. I'll try pinging some of the JS/mobile folks and see what they have to say.

Crell: Let's be careful about disenfranchising the do-ocracy by letting the perfect be the enemy of the good. If we can get an inline editor together that passes code review, architecture review, performance checks, and all of the other things required to make it through the core gauntlet by feature freeze, then I don't see a reason it wouldn't be allowed into core. But I agree that if we don't build it on top of Create.js (though Henri's laid out a road for being able to do that in an incremental way without moving all the various WSCCI dependencies ahead of it, which is nice), we need to at least make it possible for contrib to pick up where core left off.

webchick’s picture

Also, just to point out in response to Crell's fears that the plan for Spark is that it's always targeting one major core version ahead, but using the current stable release as the place to try out ideas, so they can be tested out in the real world. In other words, if Edit module got into core without the tricked-out engine, musical horn, and the shiny rims, Edit module would still exist in contrib as a 8.x-2.x branch, and would continue to have features such as these added to it that were then targeted at Drupal 9. Distros could opt to use the tricked-out version, everyone else would get something that worked. Hopefully this approach will help assuage some peoples' fears about core being "locked in" to "only" an inline editor with a V8 engine and flames on the side. ;) But once again, this is taking us very far off-topic, and this conversation has been beaten to death already in the Drupal core queue, where such "meta" discussion really belongs.

sun’s picture

I obviously should leave a larger comment on this, but right now, my most pressing question is:

@henribergius: Is there any chance to get the "backend" behavior natively into CreateJS? I.e., given a <textarea> HTML element as target/trigger, take its value (+ fire an event), hide it (+ fire an event), create a DIV at the same position (+ fire), and inject the value into the DOM (+ fire). (or alternatively require editor libraries to perform all of this when triggering a certain event?)

Train of thought: The difference between "frontend" in-place editing vs. "backend" (form-based) editing seems to be the primary gap between the possible directions to take. We will focus on the "backend" first, in order to solve the "simple" case first and make sure to ship with something. Unless I blatantly overlooked something and misunderstood the situation, then CreateJS doesn't provide anything for the latter. Which in turn would mean that we'd have to implement something custom "anyway" in order to pull off the Minimum Viable Thingie.


[And just for the record, I (obviously) agree with @Crell on the core vs. product topic and try to push it hard. However, given the remaining timeline for D8, and the incredibly large impact of the Aloha/Wysiwyg topic, I resorted to making reasonable and realistic compromises, and thus I want to see D8 core to ship with Aloha, at minimum in the backend. We can still adjust the overall core vs. product situation later. And also, to combat possible misunderstandings early; no, I'm not related in any way to the Spark efforts. ;)]

scor’s picture

great summary Wim!

To further complicate things in this area, not every field has a default semantic annotation (and for good reason: a "text" field can be used in many semantic ways); this implies we'd need to generate local schemas (which is really, really simple and a non-issue), but that would imply potential dual annotations. (needs confirmation)

correct, this would not be a problem in the context of a basic title, date, body set of fields (where a default annotation can be found), but as soon as you start adding custom fields (even as simple as text fields), you need a way to uniquely identify each field among the rest of the other fields. A local schema isn't hard to generate, and that's something we'll need for JSON-LD as well. This means dual annotations on some of the fields - but maybe these extra local annotations could only be exposed when inline editing is allowed (much like the contextual links appear when you have access to their tasks).

I don't think the core or contrib question needs an answer now, though at a minimum we should make sure core offers the right APIs to allow create.js to work, regardless of where create.js or spark live.

Wim Leers’s picture

#11: It's more than just a question of "can it be done?". Indeed, it's still yet to be seen whether we can get in-place editing in D8 at all, because not all of the absolutely essential foundations are ready yet. But using Create.js requires *more* of those foundations to be completed (most notably the validation one in the issue summary). On top of that, Create.js itself *and* its dependencies will need to undergo the scrutiny of the review process (which is a major — if not the main — reason for starting this issue in the first place). Writing *some* code that gets the job done does not equate to writing code that meets the quality requirements for Drupal core.


#12:

The RDFa dependency doesn't worry me. If someone is going to turn off RDFa to get rid of "all those ugly extra attributes", then they'll also want to turn off whatever Edit module does to generate "all those ugly extra data- attributes". It's a dependency, we deal. Non-issue.

How about this POV: "if I can't have semantically correct RDFa annotations because I don't know how to properly define the mappings, I don't want it at all"? I think that's a valid reason.
Of course, you could just say "well, in that case, you don't get in-place editing". Plus, it could be seen as a motivator for site owners to take the time to properly define the mappings.
Alternatively, the RDFa module could be split into an API module and a UI module. Enabling the API module wouldn't result in any RDFa on the page, only when using the UI module, that would be unavoidable. That way, the Edit module could decide to only describe the content using its local schema.

Wim, if we shifted the target to a Create.js-based Spark as a contrib module, what becomes the for-core task list to make that easy?

The task list for "core inclusion" vs. "contrib enabling" is identical, I would say. You could argue that JSON-LD is a necessity to implement Create.js cleanly in core, but that if Create.js would live in contrib, that it'd be totally fine to use the contrib JSON-LD module.


#17: thanks, very helpful. But Spark Edit is actually 50 KB unminified, not 100. Our Aloha Editor build is not 1.1 MB, but 800 KB, only partially minified, and including require.js. (It includes everything we want, minus the Table plug-in.)
That affects the big picture somewhat.

Overall, though, the difference is negligible once you take jQuery UI and Aloha Editor into account. The two of them will trump anything else.


#18: Devil's advocate: we could just use Backbone.js ourselves, too.
Besides that, we don't do continuous DOM operations, only upon init and when the user explicitly is performing actions such as starting/stopping to edit. The networking cost far outweighs the DOM cost.
Finally, it's absolutely correct that a lot of code can be removed from Edit's JS after finishing up the migration to Create.js.


#22: Very good point. In any case, Create.js would only be useful for WYSIWYG editing on the back-end, it wouldn't be useful for editing various other fields, since the widgets/forms to do that already exist there anyway.
FYI, over at the Aloha module, where we're working on getting "just" Aloha Editor in core by first doing just that as cleanly as possible in D7 (see #1760386: Migrate Aloha Editor integration from the Edit module and make it work on the back-end), Aloha Editor is already doing the work of: "given a textarea, create a sibling div and apply the WYSIWYG editor to that".
However, please also keep in mind that far less "JS infrastructure" is needed for "just" WYSIWYG editing on the back-end. We don't need to re-render the text without certain filters, because we always get the raw, unfiltered/unprocessed text anyway. Saving = submitting the form, not saving via AJAX + re-render the text field.


#23: Yes, I would say it's a requirement that those local schemas are only applied when the current user has permission to edit things in-place.


Overall, I think it's useful to know the full scope of issues that need to be solved to enable in-place editing:

If we want to use Create.js as a foundation for in-place editing, we need these on top of the above:

nod_’s picture

I'm very much agreeing with quicksketch on this one.

A very big concern is the size overhead, second would be the complexity added. Core is not using backbone, and not planning on using it. So that wouldn't really help. underscore is very much possible, only in case we don't need jQuery anymore (of jQuery2 is awesome and taht's not an issue anymore).

I'm all for getting the APIs in core to make it possible but since it's still "new" I'd rather not have that in core at this point.

corbacho’s picture

Excellent thread and summary.

I'm totally in

Create.js + VIE.js + Backbone.js + Underscore.js It looks like lots of libraries, but this is the way front-end technologies are now: well decoupled , *minimalistic* and following good design patterns and ready to be extended with plugins, with possibility of custom-build, etc in many different levels. It's a plus, not a minus.
All together could be around 80-100Kb. Compare that with 800Kb of Aloha

If we don't adapt Drupal Backend to this paradigm, we are doing it wrong. I'm afraid we will end up with a monolithic half-baked Editing inline that fulfills some basic use cases and feel trapped when needs to be extended (see Drupal Ajax library for an example)

The only negative aspect is the technical debt that we *could* introduced in Drupal core. But when we talk about JavaScript.. and the fragmented landscape that is becoming the mobile devices, it's better to rely on something that is not only developed debugged by us. If Drupal community adopts Create.js will increase a lot the chances that other CMSs jump in too. win win

Finally, Henri's help for the integration is a big plus

henribergius’s picture

Sorry for not answering to the various points yesterday, I'll look at them in detail soon. I was in Copenhagen helping the TYPO3 team with their Create.js integration, and the trip home was longer than anticipated.

This sort of "code sprint" model has been very beneficial when major projects like TYPO3 and OpenCms have done their integrations. I would like to offer to do the same with Drupal in order to get Create.js in Spark Edit into a good state before the various D8 deadlines, if that is the course we want to follow. We could do this either here in Berlin, or whereever Wim (and others involved) find convenient. Two or three days would be a good was to finalize the integration and polish the Edit module in general.

frega’s picture

I am 100% with @corbacho on this. But we can always do it in contrib if core get "RESTier" (untie form/entity validation, new routing) - what should be avoided imho is custom "inline-edit-js" stuff in core w/o createjs.

Speaking as one of the Drupal Usergroup Berlin co-organizers, we'd be happy to help organizing such a code sprint and take care of the nitty-gritty. I'd be great if we could get as much JS+drupal expertise in one room here (wim, _nod, seutje, corbacho?).

There are lot of nice web-related events that could encourage people travelling to berlin this fall, anyway :) jsconf.eu and lots of other js-related events (even drones: nodecopter!) basically take over berlin from 4.10. to 8.10.2012. @bergie is speaking at t3con (on createjs :) on 6.10.2012 but things before that would probably work.

Obviously, we don't have to tie it to any other event (symfony live is in berlin end of november, too, btw), just let me know if this sounds good/worthwhile ...

alanburke’s picture

what should be avoided imho is custom "inline-edit-js" stuff in core w/o createjs.

That's the key point.
Whether this makes Core for D8 is not the issue - the key point is that we use components are that gaining traction throughout the web.
If thats is core - great. If its's contrib - that's not ideal, but it's better than a custom solution we have to maintain ourselves in core.

Wim Leers’s picture

I feel like @corbacho and @frega are overly optimistic/simplifying things.

The currently "custom" (as @frega put it) implementation of Edit's in-place editing is actually very similar to the one of Create.js in concept. Where Create.js uses semantic annotation, Edit uses data- attributes. That's not a sufficiently good reason to migrate to Create.js, IMHO. Where Create.js becomes worthwhile, is in:

  1. providing a layer of indirection for different editing widgets (which we'd only use for switching WYSIWYG editors, but that does not imply you can plug in any WYSIWYG editor and have an equally full-featured experience: there's still Drupal-specific integration needed for each WYSIWYG editor)
  2. providing a set of basic functionality, such as localStorage and notifications, plus a framework that Edit would "just" have to fill in the implementation gaps for

It has been quite clearly indicated by @Crell that it's very unlikely we'll have JSON-LD in D8 core. Further, #1696648: [META] Untie content entity validation from form validation is absolutely required but hasn't even begun yet.
So, as @webchick pointed out in #20, we have to be careful to let perfect be the enemy of good. We have only a few months remaining and it'll be hard enough already to get in-place editing without Create.js in. It may be even more difficult *with* Create.js. There is of course the aspect that Create.js might allow us to have cleaner code. But we could achieve equally clean code by leveraging e.g. Backbone.js ourselves directly.
However, Backbone.js implies saving of models (equating to Drupal's entities here, which is not really supported yet in Drupal, #1696648: [META] Untie content entity validation from form validation is necessary for that), whereas currently, saving on a per-field basis seems like a more natural fit. I'm certain that Edit's JavaScript could be significantly cleaned up without Backbone.js as well. Please keep in mind that Edit's JS was developed at a very, very high pace (over the course of what amounts to about 1–1.5 month), by a sole developer (I) who doesn't claim to be a JS guru, to be able to show what is possible and to have a certain minimum level of code quality. I didn't have the luxury of planning everything ahead, I haven't even had the luxury of doing refactoring. The scope was continuously changing. Now that the scope and direction are clear, I'd certainly be able to deliver a smaller code base, with more elegant code, that achieves the same, without any external dependencies. While that smells like NIH, that just might be the better way to do it, for now, in Drupal 7 and 8.
There is no way around the fact that it makes a lot more sense to build this kind of "clean REST-based in-place editing" once you have a solid, properly documented REST foundation to build on. That will be the case for Drupal 9. So then we could at least have good, yet imperfect in-place editing in Drupal 8.

That being said, @nod_ might have time this week-end to finish off what @henribergius and I started at DrupalCon Munich: http://github.com/wimleers/edit-createjs. If he succeeds, that should give us a better view on the potential of Create.js. But that doesn't yet address the concerns about continuously changing dependencies (Create.js/VIE.js/Backbone.js/underscore.js) and how acceptable that is for core inclusion, nor does it address the fact that there's even more "foundational" Drupal 8 core patches necessary before it can happen.

Overall, it seems to me that Create.js was ready too late (it didn't do everything we needed it to do in June), Spark started too late and Create.js' dependencies are still evolving too rapidly (still maturing) to say with sufficient confidence that "Edit on top of Create.js" is ready for D8 core. I'd be more comfortable with getting a cleaned up, minimal, "custom" (data-attributes + Form API-based) in-place editor in D8 core, which may be more limited, but from which we can at least learn thanks to widespread adoption/usage. At the same time, we could work in D8 contrib on a more complete, more flexible, "more founded in web standards" approach based on Create.js, which we could then propose for D9 core.

This is my current POV, and it is highly dependent on: the evolution of the required "foundational" D8 core issues I pointed out in #25, proof of little to no API changes and roadmaps of Create.js' dependencies, written code that shows Edit's JS will indeed become significantly cleaner/smaller and that that is impossible without Create.js/Backbone.js.
As such, the above is a snapshot. If the listed "POV dependencies" change, my POV will change as well.

If it wasn't clear enough yet: if the D8 core issues in #25 don't get solved, it's also very unlikely to get any in-place editing in D8 core.

henribergius’s picture

Some clarifications on the roles of the various libraries that are on table here:

  • Backbone is a very popular JavaScript library, used in production by many quite big web applications. What it brings on the table is a better way to structure any JavaScript applications through the use of Models, Views, Routes, and the events connecting these with each other
  • Underscore is mainly included as a dependency of Backbone, but it is a small and lightweight library that simply provides JavaScript with some handy functional programming utilities that take away a lot of common problems. each, map, filter, etc.
  • VIE is the library underneath Create.js that provides connection to various semantic standards and services like RDFa, JSON-LD, Apache Stanbol, and Zemanta, and maps these to the relevant Backbone views and models
  • Create.js is a set of jQuery UI widgets that utilize VIE's services to provide inline editing and other common CMS tasks

The actual work of connecting Spark Edit to Create.js was quite simple, we just switched from keeping entity and property information in custom data attributes to VIE entities, and started using the editable widget for loading and interacting with Aloha and other editors.

A bigger part of what we did during DrupalCon was actually to port the Spark Edit UI to Backbone Views. This process is still partly incomplete, but would make the codebase a lot more manageable regardless of whether Create.js and VIE are used there or not.

In nutshell:

  • Backbone helps keeping the Spark Edit codebase manageable, which will be useful in the future. This is something independent of Create.js
  • Create.js adds editor abstraction, notifications support, and localStorage to Spark Edit. It also has many other features that are not in scope of current Spark but could become useful in the future
  • Create.js works best if you have RDFa annotations and a RESTful back-end (JSON-LD or not), but these are not absolute requirements

On that last point, the way Create.js works in default configuration is that it loads RDFa annotations, makes them editable, and then provides a way to save stuff to server via RESTful JSON-LD operations. However, RDFa parsing is an implementation detail that comes from VIE's RdfaServer, not from Create.js itself.

We already implemented a parser that instead uses Spark's data attributes to read entitity information to VIE. To make this completely transparent to Create.js, this could even be made to register itself as a VIE service. Then Create.js wouldn't care where the entities come from, and how DOM handling happens. And later when Drupal's RDFa features are improved this custom service could be dropped in favor of RdfaService.

Same thing with saving (and loading) to back-end. The default RESTful method is implemented by Backbone's sync method, which can be overridden by something else (I've even written an implementation doing all communications via WebSockets). This way again, in current Drupal we could ship a Backbone.sync that does its interactions via forms, and drop that if JSON-LD services are available.

This showcases the main benefit of going with Create.js: since the codebase is very modular, handling the transition from the current state of Drupal (data attributes, saving via forms) to the future Drupal (correct RDFa, RESTful JSON-LD) would be just a question of which services to load.

Of course there are other benefits as well. Editor abstraction simplifies the code, as Aloha, plain contentEditables, and form widgets can be treated the same UI-wise. And it allows using another editor when needed (for reasons of accessibility or mobile performance, for instance).

The other big benefit is that by using VIE and Create, Drupal would be part of the bigger momentum of CMSs using these libraries. Widgets and other code could be shared between Drupal, TYPO3, Symfony CMF, OpenCms, Midgard, and other projects (I've heard of five new CMSs going Create.js just on my August conference tour), and the shared codebase would be maintained together.

And, as I've stated before, I would be there to help. I don't want to overstate the utility of this, but let it just be said that I've been working on inline editing features for CMSs since Midgard's circa 2005 "AJAX editor", and on decoupled content management for far longer...

Midgard's AJAX editor is actually an interesting case here, as the way it worked was based on loading form widgets from the back-end, quite like Spark Edit.

To conclude, there seems to be quite some support in the Drupal community for both Create.js and Backbone (see for example @frega's comment above), and so I believe having Spark Edit with them would enable us to create a very nice collaborative effort, both inside Drupal and with the CMS ecosystem at large.

henribergius’s picture

@nod_ might have time this week-end to finish off what @henribergius and I started at DrupalCon Munich: http://github.com/wimleers/edit-createjs.

I should be able to help with this as well. It would be great if you could join the #iks channel of FreeNode so we can discuss how to proceed and resolve possible issues more quickly.

henribergius’s picture

I'm certain that Edit's JavaScript could be significantly cleaned up without Backbone.js as well. Please keep in mind that Edit's JS was developed at a very, very high pace (over the course of what amounts to about 1–1.5 month), by a sole developer (I) who doesn't claim to be a JS guru, to be able to show what is possible and to have a certain minimum level of code quality. I didn't have the luxury of planning everything ahead, I haven't even had the luxury of doing refactoring. The scope was continuously changing. Now that the scope and direction are clear, I'd certainly be able to deliver a smaller code base, with more elegant code, that achieves the same, without any external dependencies.

I'm quite confident that Edit can be made clean and elegant without any of the stuff we're discussing here. But what I'm saying is that doing this with Backbone and Create.js will be a lot easier, and will leave you with a smaller code base to maintain.

And if Edit has so far been done by a single developer, having more people involved probably wouldn't hurt. This is something we can at least offer. Not just my work, either, but the united efforts of JavaScript developers from Symfony, TYPO3, OpenCms, and other communities.

Example of this collaboration in practice: the editor abstraction in Create.js was developed by OpenCms. The Aloha team then contributed support for Aloha Editor. And of course I've since done many changes as part of maintaining the library.

corbacho’s picture

#30
I agree with you Wim. Probably I oversimplified before. And by highlighting positive aspects of using other popular-proven-tested libraries, I didn't mean to put down your awesome-pace work. It's incredible what you have achieved in so short time. I clearly can see the immediate benefits of developing much faster a jQuery-based-custom-not-bloated solution adapted to Drupal, without having to learn any new framework and dropping also the RDFa dependency.

But we are losing the benefit of all those things that others have done around their libraries: JS Unit tests, travis-continous-integration, debugging edge cases, cross-browser issues, etc.

You are the one who has a better "overview" of all the moving parts to make work a in-place editing solution in the next 5 months.

I agree on @Crell and others about.. what's the rush ? We can have a Create.js/VIE/Backbone as an awesome contrib module that doesn't depend on the D8 core schedule. So we *could* switch to it for D9.

The problem of leaving Create.js "for later" is about wasting efforts in parallel solutions.
As @henri said (sorry for the misspelled Henry before) we could do it by parts.

But again, I see the benefit of both sides.

About backbone.js: Still could be used as Wim said without Create.js Backbone.js is not a "idealistic-pure-only-works-with-Restful", it's like a boilerplate framework, and at our company we are overriding many methods of it to make it work as we want, aka sometimes not RESTful way, for some minor operations like you have said "updating only one field". But the core philosophy of it, the model-view separation is what we could benefit from, at least for the sake of having a structured code that other JavaScript-MVC developers can understand easily if they want to jump in
(Including myself)

henribergius’s picture

what's the rush ? We can have a Create.js/VIE/Backbone as an awesome contrib module that doesn't depend on the D8 core schedule.

I think the main reason why some decisions should be made ASAP is that we already put some effort with Wim for porting the current Spark Edit to Create.js back in DrupalCon. If this is the way that the Drupal community wants to go, this patch would just need some more polish and would be ready for merging.

If, on the other hand, Spark Edit development continues indepently of this work, then the delta gets bigger, and merging the Create.js port back in becomes more difficult. Essentially the port would have to be redone at some later point.

henribergius’s picture

Is there any chance to get the "backend" behavior natively into CreateJS? I.e., given a HTML element as target/trigger, take its value (+ fire an event), hide it (+ fire an event), create a DIV at the same position (+ fire), and inject the value into the DOM (+ fire).

Making a version of the Editable widget that does this shouldn't be a very big task, and then you would indeed have the same event model as on the front-end, and the same flexibility in configuring those editor instances.

Anonymous’s picture

I hate to be a killjoy, but our RDFa output simply isn't reliable enough to depend upon for something like this.

I have created #1778226: [META] Fix RDF module to collect the problems and have started adding detailed issues. We will need to fix these before we can consider having something like this, in core and relying on RDFa.

I explained this to Henri at DrupalCon and he later indicated that microdata support in Create.js might be an option. Unfortunately, Microdata module isn't in a state where we could depend upon it for this functionality either. We would need much more support in contrib field formatters before we could move forward with this.

Crell’s picture

Wim, let me ask this: If we say that a Create/VIE-backed Edit module was a D8 contrib and D9 core task, not a D8 core task, then what path would result in us writing the least amount of code that we have to throw out and/or redo later?

Eg, there's no sense in making an all-custom Edit module really really smoking optimized if we know that we're going to replace it before long with a Create-backed version; instead we should be getting it "good enough" (for some definition of good enough) and focusing energy on a smoking optimized Create-backed version for contrib.

Does that make sense?

effulgentsia’s picture

This showcases the main benefit of going with Create.js: since the codebase is very modular, handling the transition from the current state of Drupal (data attributes, saving via forms) to the future Drupal (correct RDFa, RESTful JSON-LD) would be just a question of which services to load.

I'm intrigued by this idea. Writing something to the Create.js framework, but still using data attributes and forms under the hood until such time that we can switch those out. There still remain development effort and code library sizes / dependencies issues to sort out (some good discussion in this thread about that), but if those do get sorted out, then I really like this approach of removing WSCCI plumbing from the critical path of using Create.js.

nod_’s picture

Like it's been said before, beside all the plumbing that needs to happen, core needs to decide on a strategy about library updates. If we can't/won't update during D8 stable release, that will not work.

(edit) The priorities of D8 are also to consider. Mobile first, awesome editor experience. It's hard to justify this amount of JS when a "low-tech" Drupal-way of doing this result in a much smaller footprint. I think it's just too late for D8. If the people you're talking about are indeed available, they don't really know Drupal (or else what are you doing, we need help in the issue queue! :) and well, getting to know it takes time (especially JS since it's not well documented, good luck with the ajax framework). I mean all this takes time, and we don't have that anymore at this point.

Totally fit contrib for the D8 lifecycle though.

nod_’s picture

To clarify a bit, I'm totally for the edit module to get in D8. It might need some clean-up to make it go through JSHint and all that but essentially, that's an awesome thing to have (finally!).

On the other hand, the createjs integration is what I was talking about in the previous post. To me, it won't work for D8. As for backbone/underscore. Totally agree on using that if we're able to not use jQuery. With the mobile objective we need to rely on 1 library as small as possible to make it work.

pcambra’s picture

henribergius’s picture

On the other hand, the createjs integration is what I was talking about in the previous post. To me, it won't work for D8. As for backbone/underscore. Totally agree on using that if we're able to not use jQuery. With the mobile objective we need to rely on 1 library as small as possible to make it work.

jQuery is a DOM abstraction library and Backbone is client-side MVC, so they have quite little functional overlap. jQuery helps working with different browsers and making the DOM API more ergonomic. Backbone helps in structing larger applications in a clean way, and providing a nice data and event model.

As I mentioned in comment 17, Create.js adds some bulk to the existing Spark Edit, but at the same time provides three things that will be highly useful:

  • Editor abstraction allows serving a smaller editor to mobile clients, making the whole package a lot smaller. It can also help with things like accessibility
  • Create can support different parsing services, allowing us to now work with Spark's existing data attributes, and then to switch to RDFa or Microdata when that is properly available
  • Backbone.sync (which Create supports) allows us to now send content via forms, and to switch to proper RESTful JSON-LD when that is available

...so, with these you can have a much smaller package (down to say 40% of the current Spark Edit) for those clients that need it, and a inline editing interface that can adapt to the changing Drupal platform.

Besides, all of this works already, as has been proven in the edit-createjs fork. So we're not just talking about theoretical possibilities and work that still would need to be done.

henribergius’s picture

I hate to be a killjoy, but our RDFa output simply isn't reliable enough to depend upon for something like this.

...which is why currently we use custom data attributes also in the Create.js fork of Spark Edit. Create.js doesn't actually care about RDFa, but instead needs a VIE service that provides a entity reading and writing with DOM. By default we use RDFa, as that is the universally available option, but this doesn't mean every CMS has to do so.

There is a custom parser that provides an interface compatible with VIE's RDFa service (I'd still like to expose this as a proper VIE service, actually): https://github.com/wimleers/edit-createjs/blob/7.x-1.x/js/util.js

To make Spark Edit use RDFa instead (when that works properly), you'd just need to remove these three function overrides: https://github.com/wimleers/edit-createjs/blob/7.x-1.x/js/editable.js (which of course can be made configurable)

henribergius’s picture

Wim, on Twitter, tells that gzipped and minified Aloha Editor is only around 200KB. The other libraries will be reduced by somewhat similar counts, so in this case our sizes would be about (estimates, based by the 1/4 size reduction reported for Aloha, which is similar to what libraries like VIE and Create get when gzipped):

  • Standalone Spark: 0.33 MB
  • Spark with Backbone: 0.34 MB
  • Spark with Create: 0.35 MB (custom build)
  • Spark with Create: 0.36 MB (full build)
  • Spark with Create and a less full-featured editor: 0.11 MB

It should be noted that maybe one third of Spark code is related to custom data attribute parsing (instead of RDFa) and form handling (instead of JSON-LD). Much of this would be unnecessary if Drupal had those two features. So, with Create.js, I'd handle those in separate files that we can simply drop when Drupal is ready:

  • Backbone.sync implementation that uses form submissions instead of REST
  • VIE service that implements loadable and savable using Spark's data attributes instead of RDFa

This would keep the solution quite adaptable. It also means that if Drupal were to have a mature Microdata implementation before an RDFa one, we could just swap the VIE service to use that. That way the end result of Edit with Create would not be much bigger than a fully custom one.

nod_’s picture

Could you post the KB size instead of MB?

That'd be a more appropriate scale, as I personally care about a couple of kb that are not showing up in those numbers. Thanks for digging all this up :)

henribergius’s picture

Could you post the KB size instead of MB?

I could, but I think we shouldn't treat the numbers as quite that absolute yet, as there are some variables that will affect the end result on that level:

  • What widgets from Create will be included?
  • What services from VIE will be included?
  • How much can we shave away from Edit's own JS by finishing the refactoring?

On a quick look, the effect on Edit's JS size can be anything between 10-30KB. Could be less if we can refactor Edit enough, could be more if we decide to use more of VIE and Create (collections, annotation services like Stanbol, etc). And of course we still have the possibility of offering a smaller editor for mobile clients, making the total download size for them significantly smaller than it is now (from 0.33MB to 0.11MB).

In any case, I'd treat these numbers as ballpark estimates for now.

The decision is obviously something you in the Drupal community need to make, but in my view this size increase ought to be worth it given greater maintainability, help from outside the traditional Drupal contributors, and many features that you can start using when you need them.

bfr’s picture

I'm all for create.js, and we at Druid are willing to contribute, but honestly i think it's too late for D8 Core. There's no reason however why it could not live in a contrib module until D9(or D29 ;) ). Ofcourse we support the decision if you want to get it to D8 also.

nod_’s picture

Ok so let's not talk about size (:

Fact is, D8 today is not ready to use CreateJS as-is. We will need custom stuff to make it work in D8, custom stuff we'll have to maintain. For a long time. Custom stuff we (I, at least) are not familiar with. To benefit from CreateJS we need some more work done on core, the timeline is not in favor.

But let's say that is not a problem (still is, but.) what does CreateJS gives us? a common way of making WYSIWYG work on the front-end. Now, what's the point? We went through a long process of trying out iframe-less WYSIWYG editors and Aloha came out way above all the others (and I guess we agree with TYPO3 here). So, we want an abstraction layer underneath Aloha, to make using a less-good editor possible. D8 is supposed to have the best editor experience possible. I'm pretty sure sun will agree that the current WYSIWYG module that kinda abstract that doesn't really put all the editors at the same level and some just works better than others.

When you think about time constraints, the lacking features of D8, the overhead of CreateJS and the marginal gain as far as features goes (since we will need custom code to integrate Drupal with CreateJS, see the second item on the list) I just don't see it as relevant for D8 core. Now contrib and D9 core is an other story of course.

Title of the issue is "Create.js: yay or nay" to which I reply for D8 core: nay. Regardless of the good ideas and very relevant solution to a problem we just don't have.

henribergius’s picture

Fact is, D8 today is not ready to use CreateJS as-is. We will need custom stuff to make it work in D8, custom stuff we'll have to maintain. For a long time

The custom stuff was already written by Wim (as Edit needs to parse the editable fields from DOM in any case), I just refactored it a bit: https://github.com/wimleers/edit-createjs/blob/7.x-1.x/js/util.js

Regardless of the good ideas and very relevant solution to a problem we just don't have.

I wrote about some of this in my DrupalCon blog post. To summarize, here are some of the things Create.js would give you right now:

  • A more maintainable structure for Edit codebase via VIE entities, Backbone views, etc.
  • Possibility to collaborate on the Edit implementation with other CMS projects, and to reuse code and experiences between them
  • Editor abstraction (including between Aloha, form widgets, and plain contentEditable, all of which you already use)
  • localStorage, aka. "never lose content"
  • Workflows support (publish, unpublish, etc)
  • Annotations and automatic content tagging
  • Notifications and client-side scriptable tutorials
  • Collection management (add, remove, etc)
  • VIE and Backbone as a client-side API to Drupal (same client-side API as in many other CMSs!)
  • Very easy transition to JSON-LD and RDFa when they become available in Drupal

But anyway, if the decision is to not collaborate, then Edit can certainly work standalone, as it already has. It just means you'll have to do it by yourselves. With Create-based Spark Edit I would help ;-)

The whole Create.js thing of course can be revisited at any later stage. That obviously won't benefit of any of the work we did at DrupalCon, as I'm sure Edit will diverge from that codebase quite a bit as the work goes on. But maybe https://github.com/wimleers/edit-createjs can be used as reference to whoever undertakes this effort at some later time.

henribergius’s picture

Also, to remind: we're not talking about something theoretical here. There are currently two separate codebases implementing Spark Edit:

Apart from some rendering glitches I introduced while refactoring some of our views, these two are functionally exactly the same. They use the same data attributes for loading editable content, and provide exactly the same editing widgets, decorations, save functionality, etc. The only difference is that in Create port some parts of the logic have been offloaded to be handled by Create and Backbone.

So the question here is essentially, which codebase should the continued development of Edit happen in?

Here is a screenshot of Spark Edit running on Create. You can see the rendering glitch I mentioned by the edit toolbar being a few pixels off:

Only local images are allowed.

Crell’s picture

Henri, with the way create.js/VIE is put together, if we moved forward with Spark/Create/custom in D8 core, and then in D8 contrib were able to squeeze in the JSON-LD and RDFa we need (itself an open but separate question), how easy would it be for a module to simply slip in the alternate connectors for Spark/Create/Semantic-yummy? And in so doing would the amount of code that gets dumped to the browser go up or down?

I'm just trying to get a sense for how "cleanly" we could turn a custom-backed Spark/Create into a semantic-backed Spark/Create via contrib. I think that, to me, is the biggest deciding factor in how to proceed. My biggest concern is that we end up with a core solution that makes doing the more complete, better contrib solution more difficult than if we had just saved it all for contrib.

ethanw’s picture

I am definitely a "Yay" for CreateJS in terms of enthusiasm, and I agree with Corbacho, Frega, and, I think, Wim, that CreateJS (or something very like it) is the "correct" way to do inline editing in Drupal.

That said, I also very much respect Wim and Quicksketch's reservations about it, as they have put so much in to the project already, Webchick and Nod's focus on mobile usability, and Crell and LinClark's warning about what sort of JSON support D8 core is realistically going to ship wth and the limitations of current RDFa/JSON-LD/microformats.

With all that in mind, I see the following main themes of concern:

1. CreateJS is too heavy, especially for mobile.
2. D8 is too soon, and won't meet the requirements of a CreateJS-based implementation.
3. CreateJS doesn't might not give Drupal that much, actually, and we may be able to roll something smaller ourselves that fits the platform better and is smaller.
4. We should offer a non-CreateJS implementation as default, with support for CreateJS in contrib. (Not really a concern, but a recurrent theme).

Re: 1, I think this is likely a solvable problem. Whatever solution is adopted, it must gracefully degrade/progressively enhance. Using require.js/yepnope/etc., in place editing can be lazy loaded for supported platforms, and device type could be one criteria. Yes, this would require some sort of AMD loaded support.

Re: 2, I can't speak to whether JSON-LD will be done in time for D8, and though I hope to be able to put significant time into development later in the cycle, I cannot commit now so I don't feel I can say a whole lot.

What I can say, though, is that some of the major open tickets, such as moving away from Form validation, will need to be closed if we're going to get D8 restful in any meaningful sense. Entity-based validation is required for a wide, wide range of usecases, not just in place editing. Much of the complexity in Spark is dealing with its lack. If D8 ships with a form-based validation paradigm for entities, it will hinder both core and contrib inline editing options greatly, as well as content staging/deployment, REST app development, etc.

Re: 3, I think there are actually two strands in this point.

Firstly there is the feeling that the way Drupal needs to be integrated with is nonstandard and so we'll need to roll a lot ourselves (IE form-based editing, field-based saving, etc.). I think that these concerns mainly grow out of the lack of entity validation, and we risk perpetuating the problem by investing a lot of energy and developing a complex system built around the existing substandard validation paradigm.

Secondly there is the feeling that much of the work has already done for Spark, and so much would need to be redone.

Here I'd respectfully submit that while Wim, quicksketch, et. al. have done an amazing job with the initial proof of concept for Spark and in place editing, my sense from trying to work with the code a bit is that many of the difficult challenges are just around the corner: how to handle complex widgets, handling the validation errors, issues with field-bases saving and entity revisions, etc. I don't know if we can compare things like the size of the current Spark libs with Create, with a good chunk of functionality pending. I also worry that some may be optimistic about how much effort remains to take Spark to functional completion with it current architecture.

At what point is it worth it to address some of the deeper architectural issues, such as validation, if it might save a good deal of implementation time and complexity for in-place editing?

I don't doubt that the in-place editing team can make it work, regardless, but this also plays into the ability to get other contributors, to extend the system, etc.

re: 4, I'm not sure what this would look like. I feel like the non-Create/REST system would look radically different from the Create one, with very little re-usable and most of the energy in the former case going toward edge cases and Drupalisms, and much in the latter case going toward re-implementing elements not completed in core.

Finally, I am curious what a non-CreateJS Backbone/REST editing system might look like. My intuition is that it would likely largely duplicate the work of Create, but perhaps outlining a general architecture of form-based in place editing w/ Backbone would shed some light on that.

I hope this feedback is useful. I don't claim to be able to speak to either the planning or inner workings of all of this, but from the peripherary (and w/ a heavy pro-REST/Backbone bias), these are my impressions.

henribergius’s picture

if we moved forward with Spark/Create/custom in D8 core, and then in D8 contrib were able to squeeze in the JSON-LD and RDFa we need (itself an open but separate question), how easy would it be for a module to simply slip in the alternate connectors for Spark/Create/Semantic-yummy?

There would be two connectors, each with two or multiple variants:

  • Backbone.sync: Form submissions vs. RESTful JSON-LD
  • VIE DOM parser: data attributes vs. RDFa (or for example Microdata)

In both of these we can have the current versions separated to their own files that we just drop when Drupal is ready for the standards-based solution (which is already built-in in VIE). I'd imagine in the end these would amount to about 1/3 of Edit's current codebase.

To be really clean, these would still need some work:

  • Move the DOM parsers from Edit's util.js to a separate file that implements a VIE-compatible loadable method
  • Move form submission handling to behind a Backbone.sync override

These are not big things to do, I'd estimate couple of hours from me for the former. The latter would be best handled with somebody more familiar with Drupal's form handling. And then really the variant being used would just depend on which extra files we load.

nod_’s picture

I'm giving my opinion since I was asked to, I'm not dead set on not having CreateJS in D8. I think it's too soon but whatever, I'm not the only one around.

That said I'll give the same kind of review any other JS patch gets from me and will again raise the size concern if it turns out the estimates ended up to optimistic in regards to the features we get from it.

(edit) and you said you'll help, that great. I'd just like to know for how long.

Anonymous’s picture

The JSON-LD might take some additional time. The way we plan to handle JSON-LD for Drupal 8 uses some new features (as well as some currently-under-development features) of JSON-LD. I assume that the library you use hasn't been updated with those features. However, that is something that could and probably will happen independently of this work, so it might not affect the time estimate.

henribergius’s picture

Re: 1, I think this is likely a solvable problem. Whatever solution is adopted, it must gracefully degrade/progressively enhance. Using require.js/yepnope/etc., in place editing can be lazy loaded for supported platforms, and device type could be one criteria. Yes, this would require some sort of AMD loaded support.

TYPO3 is using Create and VIE via require.js. I don't have much experience in this area personally, but I'm sure they'd be happy to give some pointers for how to go about it.

Re: 2, I can't speak to whether JSON-LD will be done in time for D8, and though I hope to be able to put significant time into development later in the cycle, I cannot commit now so I don't feel I can say a whole lot.

Without JSON-LD Create and Spark Edit have to resort to saving via forms, but this is something that is already implemented. When JSON-LD goes in, we can then drop most of this code (we still need to be able to load form widgets for some editables, but saving could be Backbone.sync with those as well).

Secondly there is the feeling that much of the work has already done for Spark, and so much would need to be redone. Here I'd respectfully submit that while Wim, quicksketch, et. al. have done an amazing job with the initial proof of concept for Spark and in place editing, my sense from trying to work with the code a bit is that many of the difficult challenges are just around the corner: how to handle complex widgets, handling the validation errors, issues with field-bases saving and entity revisions, etc.

This is why the Create port was done by refactoring the existing implementation, not writing it from scratch. Wim has done a great work with Edit, but obviously moving it to Backbone Views, etc, can help to make it more flexible for the future. Migrating it to those was actually much more work than hooking Create in.

But again, this work is largely already done. Some bugs that I didn't have time to handle in those two days we had, but if there had been a consensus that this would be the way forward, they'd be fixed already :-)

re: 4, I'm not sure what this would look like. I feel like the non-Create/REST system would look radically different from the Create one, with very little re-usable and most of the energy in the former case going toward edge cases and Drupalisms, and much in the latter case going toward re-implementing elements not completed in core.

The original Edit codebase resembled a lot the first versions of Create that we had done internally, before we started using Backbone and RDFa. So at least the initial direction is quite similar.

But of course the further Edit development goes standalone, the more the two concepts are likely to diverge, and the harder it will be to merge them later. Right now we're in the stage where getting Edit to run on Create is just a question of deciding to do so, as the porting work is already mostly done (see comment 51). Later, it would be a lot more work.

henribergius’s picture

and you said you'll help, that great. I'd just like to know for how long.

I've been doing open source CMS work since 98, so I'm likely to stick around for quite a while yet :-)

This year I still have time available via IKS that I'm splitting between the CMSs that need my help with Create and VIE. For example, last week I spent three days in Copenhagen working with the TYPO3 team, and today I've been helping the Symfony CMF developers. I'm sure with the time I have we can move Edit quite a bit forward during the fall if that is what we want to do.

After this year I'm still going to be the maintainer of both Create and VIE, and will certainly be happy to work with cool projects around them. I like the general direction Drupal is headed with 8, and so I don't see why I wouldn't help you also in the future.

For getting things going, the code sprint model we discussed earlier could be a good idea.

lightsurge’s picture

The too soon rather than too big argument seems to have more weight to me. Maybe it would be possible to chop down Aloha by whatever gets added by CreateJS.

Henri seems really enthusiastic for this and it gives the community the opportunity to influence CreateJS in its favour. I realise Aloha isn't as great a risk because even though it has similar 'will this last' concerns, it might spider its way into Drupal innards more irrevocably, but I guess that's going to happen (again) eventually and repeatedly as Drupal moves on and as someone mentioned earlier it doesn't seem any greater risk than jquery.

Henri seems to be making a reasonable case for why the risk is worth it for the innovation, and seems to be countering the arguments so far, and if that carries on as such, it doesn't seem fair to avoid it because it's young, but rather to take it on while it's young and be part of evolving it into something that everyone wants to adopt and it survives like jquery.

bfr’s picture

"Maybe it would be possible to chop down Aloha by whatever gets added by CreateJS."

Like stated before, Create already supports Aloha, but the native/example/shipped/whatever editor, that is called Hallo, is actually pretty good and really light weight.

When i said that it's too late for D8 core, i actually dont even mean it's bad idea for Create, i think the whole concept of spark for core is questionable. The freeze(s) are really close and some modules were removed from core recently because of the burden the core maintainers get from the bloat.

This is however another conversation, and i totally respect Wim, Webchick and all the others for all the work they have done, and would really LOVE to see something like this in core if it's actually possible. Moving Spark on Create would outsource much of the work away from the core maintainers, which would of course be great, and also, for what i know Henri and his team, they would be really valuable addition to the community. Who knows, maybe we even succeed in converting Henri to real drupalist in the long run ;)

webchick’s picture

Assigned: Unassigned » Wim Leers
Priority: Normal » Critical
Issue tags: +Spark Sprint 5

Slotting in for this sprint. A lack of conclusion here blocks further efforts, so this is critical.

lightsurge’s picture

Like stated before, Create already supports Aloha, but the native/example/shipped/whatever editor, that is called Hallo, is actually pretty good and really light weight.

Hallo was mentioned a lot in the http://drupal.org/node/1580210 editor choice thread but with some doubt as to its maturity and feature-set. I suppose the beauty of using create.js is that it actually means it might be easier to interchange between full-featured (Aloha) and light (Hallo). Its been mentioned elsewhere that it would just be confusing to have more than one editor available on installation, because it wouldn't mean anything to the average joe, - but actually, if rather than naming them we have a choice of 'full-featured' and 'lite' - that's not actually too bad? I mean, it opens up options like that, I'm not suggesting that be done for December, and also opens up options for example in terms of editing on mobile falling back to Hallo to keep the size down.

Create.js was dismissed in the inline editing thread for similar concerns as here in terms of avoiding bloating the js stack, but I don't think it got much time in the spotlight so happy its been brought back up here http://drupal.org/node/1494628#comment-6035712

Edit: Henri mentioned the editor abstraction stuff already:

Of course there are other benefits as well. Editor abstraction simplifies the code, as Aloha, plain contentEditables, and form widgets can be treated the same UI-wise. And it allows using another editor when needed (for reasons of accessibility or mobile performance, for instance).

henribergius’s picture

I suppose the beauty of using create.js is that it actually means it might be easier to interchange between full-featured (Aloha) and light (Hallo). Its been mentioned elsewhere that it would just be confusing to have more than one editor available on installation, because it wouldn't mean anything to the average joe, - but actually, if rather than naming them we have a choice of 'full-featured' and 'lite' - that's not actually too bad?

Yep, and you can even run different editors side-by-side. For example TYPO3 uses Hallo for simple content fields like, say, an article abstract, and Aloha for those where a full-featured editor is needed. This is very similar to how Spark Edit uses Aloha for article body, and a plain contentEditable for the title.

Create provides a quite flexible way of configuring which editor to use and where, see for example https://github.com/bergie/create/blob/master/examples/example.html#L33

frega’s picture

@wimleers - would a code sprint with @bergie (et al.?) in Berlin make sense at all? If so, let me know. The user group/I would be more than happy to help with organising that, but we'd need a little heads-up and a date to sort it out.

webchick’s picture

Status: Active » Needs review

So now that this discussion has died down a bit, here's my read of it.

- Pretty much everyone is on board conceptually with Create.js being a good path forward.
- Many people though are concerned about Create.js meeting the D8 feature freeze timeline, especially with all of the various dependencies of doing it the "right" way (JSON-LD, RDFa improvements, a de-coupled Entity Validation API)
- Even to the point where people are seriously advocating not doing inline editing functionality *at all* for D8. :(
- There are other unknowns as well, like ultimate file size / JS payload impact for mobile, how stable the underlying dependencies are, etc.

So. Here's the proposal: Treat Create.js-ification of Edit module as a *post-feature freeze* "enhancement" to Edit module / inline editing in core. (Assuming we can get it in by Dec 1) Reference: http://buytaert.net/updated-drupal-8-release-schedule

This would solve a few problems:

- We'd now have until April 1 to do this, rather than Dec 1, which helps alleviate timeline pressures causing people to question the do-ability of the entire inline editing functionality in core.

- This delay would allow TYPO3 / Symfony Full Stack Framework / other CMSes that Henri's working with to "alpha test" Create.js integration in advance of us using it in order to further flesh out and fix bugs/missing features (such as the ability to work on backend forms), rather than us having to take that on ourselves (we're already taking on a lot upstream collaboration work w/ the Aloha stuff as it is).

- Create.js's development will largely be done in December, so some of the unknowns around file sizes, dependencies, shifting library code, etc. would be nailed down at that point.

- The feature set of Drupal 8 will also be largely done in December, so we'll know the state of underlying Drupal 8 dependencies like RDFa, JSON-LD, entity validation API, etc.

So unless we hear dire cries about this plan, this is what we plan to move forward with. I'll update the issue summary with this plan in a moment, and aim to close out this issue by next Friday, if not sooner so we can move forward on completing our feature enhancements for Edit module and get it on the road to D8.

Note that this should not stop anyone from working on necessary/"very" nice to have CreateJS Drupal plumbing in the meantime! A quick hit-list of at least some of those issues would be:

- #1606794: Implement new routing system
- #1778226: [META] Fix RDF module
- #1696648: [META] Untie content entity validation from form validation
- #1533366: Simplify and optimize Drupal.ajax() instantiation and implementation
- The ability for CreateJS to work on back-end forms (see comment #36)
- Backbone.js/Underscore.js in core

Thanks all, for the really productive discussion here!

Crell’s picture

My concern with #65 is what happens if the necessary CreateJS-ifiication plumbing doesn't make it by 1 December? Does it "inherit" the "can happen later" attribute or does that mean that Edit-in-D8-core is stuck being non-Create-ified? The RDFa/JSON-LD stuff is my biggest concern there, cf #1784216: [META] JSON-LD support

webchick’s picture

Btw, as frega pointed out in IRC just now, going this route does indeed mean we'll have to re-do the work done in the Wim/Henri's Github sandbox later on. However, given that work that's there now was completed in only a couple of days, I'm not too concerned about that, personally. It seems like CreateJS is a pretty robust solution that's easy to integrate, and the work going on right now w/ TYPO3 and Symfony FSF can only help with that.

I'm more concerned about some of the hard requirements (like Backbone/Underscore in core, the ability for CreateJS to work on backend forms...) blocking us from actually finishing Edit module, which is both still not yet feature-complete (it's missing image integration, still has open questions about how to work with revisions, etc.) and also has some bugs and other janky interaction patterns that really need to be cleaned up before we can even think about proposing it for D8 core.

So in my ideal world, the Spark team could focus hardcore on the UX/polish of inline editing functionality and get that really solid, using the code that's working there right now as the base. "Team CreateJS/Drupal" (hopefully formed by several of the people in this issue who've expressed interest) could in parallel work on the underlying dependencies/nice-to-haves for CreateJS in Drupal 8 core, collaborate with Henri on upstream improvements necessary to make CreateJS work better with Drupal, etc. It sounds like frega is into the idea of hosting a sprint for interested parties to help make this happen (awesome)!

Then, at feature freeze, each teams shifts gears from trying to get as much into core as possible before feature freeze, to instead integrating with each others' work, both of which is now way more awesome than we could possibly achieve by working on these two efforts in "waterfall" method.

Make sense?

Btw, if anyone does want to form "Team CreateJS/Drupal," and wants to take that code from the Github repo and move it to a branch of Edit module or an edit_createjs module that's kept in sync with upstream changes, we would definitely be supportive of that. We just don't think we can take on both that and the necessary front-end work to make Edit module shine.

webchick’s picture

Re #66: My interpretation of Henri's comment in #31, is that none of those things are hard dependencies for CreateJS inclusion. AFAIK the only things that would be hard dependencies are the Backbone.js/Underscore.js libraries (for createJS) and the ability for CreateJS to work on the backend (for us). RDFa/JSON-LD support would be "services" that could be loaded by a contrib module that offers those features.

Of course, would love to hear confirmation/denial about that from Henri. :)

lightsurge’s picture

I can understand going with a custom solution because you have the resources to meet deadlines, it has that short term advantage. But in that timespan without Drupal on its shared shoulders create.js may run off in a direction that Drupal doesn't want to go, maybe createjs would be a winner anyway or maybe it would be a loser, but Drupal wouldn't benefit or lose out, but that might mean maintenance burden.

On balance I would probably do the same, these are pretty severe deadlines, and if create.js isn't useful at the point at which it's possible to consider it, maybe there'll be something else.

But I don't think 'Pretty much everyone is on board conceptually with Create.js being a good path forward' sums the gist of the positives of create.js in this thread up for me, my take from the argument in this issue was more that something like create.js could help define a good path forward, a path it would be difficult to synchronise if you're not walking together.

sun’s picture

Here's my stance:

I'm not a fan of exceptions, neither in PHP nor in release cycle schedules. There's a non-trivial amount of work to do to just get the backend editing capability (properly) into D8 core. This goes beyond Aloha module + AE — it means and requires to adjust Drupal core in various other spots to take a client-side editor into account, which isn't the case yet, and was deliberately ignored in the past. This has to be rock-solid — half-baked solutions won't help anyone; before we do something half-baked, we better keep it in contrib. Also, I seriously don't want to offend anyone who made this awesomeness possible (especially not @Wim), but I will say that, from a Drupal core perspective, the overall current state of code and art needs some serious love; it still looks very "prototype-ish". However, if we're able to focus on that, then there's hope that at least the backend Wysiwyg editing will be a feasible goal for D8.

Second, in terms of product features that users actually care about, all of aforementioned core plumbing would have to be amended with a bullet-proof implementation of that bare minimum AE image caption thingie, but actually much rather an inline macro system for editables. This matters. To put it bluntly, a client-side editor without this functionality does not allow for true client-side editing and thus fails to solve the actual problems of users. There's working code for these things, but there's still a lot of plumbing and collaboration necessary to make it happen. That's a hard requirement in my book. I object to putting an editor into core that doesn't address the actual needs of content authors.

With that out of the way, there's the entire frontend/in-place editing, RDFa/microdata, RESTful router, JSON-LD, Backbone, Underscore, and CreateJS drama. — Frankly, given the aforementioned two major roadblocks, I even doubt it will be possible to pull the current non-CreateJS implementation into core for D8. That is, because the architectural requirements to (properly) support "edit capabilities while in view" involve to rethink a full stack of subsystems from the ground up. Again, the train of thought is that contrib exists to do the things that are tacked onto the system; trying to tack something within core onto an architecture that isn't prepared and ready to support the concept natively will end up in a major #fail. So, even when putting all the other nice-to-have things and dependencies like RDFa/JSON-LD/CreateJS/etc.pp. aside, and just looking at what the current Edit module tries to do and how we'd have to adjust various aspects of the architecture to properly support that, even the limited goal of current Edit module appears unrealistic. We can certainly try. (but above mentioned topics should have priority) And thus, to finally come to the actual topic and question, the list of dependencies appears way too long, problematic, complex, and uncertain for me, and even more so, we're discussing only theoretical problems at this point and didn't come to practical problems yet, so, honestly, I'd leave CreateJS & Co for contrib (and possibly D9).

That said, I definitely plan to get my feet wet and hands dirty on CreateJS as soon as possible, since I seriously love the idea. :)

Lastly, another shameless plug and reminder for everyone to whom it may concern: The more we ignore D7, the more is possible for D8. ;)

I'll try very hard to spare time to work on and help with this stuff (though no idea how yet), so we can get at least the primary objectives done for D8.

webchick’s picture

That's all good feedback. I just want to call out one thing.

"I'm not a fan of exceptions, neither in PHP nor in release cycle schedules."

I'm not a fan of exceptions, either, and so I want to be crystal clear that we are not asking for an exception here. This proposal fully abides by the guidelines that Dries laid out at http://buytaert.net/updated-drupal-8-release-schedule which includes "Better integrating features added before feature freeze." So as long as both Edit module and CreateJS made it into core before Dec 1, refactoring to better integrate the two is completely on-topic for the post-feature freeze time without any exceptions and without bending any rules.

The rest of your points/concerns about whether we could pull those things off though, and which things to focus on if we want to have a prayer of doing so, are completely valid.

webchick’s picture

Oh, also to respond to this:

"The more we ignore D7, the more is possible for D8. ;)"

We are 300% on that same page. :) Spark's D7 work is a means to an end, and that end is getting "real world" testing of this stuff before it's proposed for D8 core inclusion. So people can touch it, and play with it, on a real site with real users and real content.

Since DrupalCon, we have been trying to focus as much on D8 as possible, and we've split off certain things in Spark to be done done in a "D8 first" manner with later D7 backport (e.g. the mobile toolbar), as well as spending time on various plumbing bits. However, there are various UX stuff, particularly around inline editing, that still need to get figured out. Putting solutions to those problems into a format where they can be tested by people who aren't D8 developers really helps to ensure the directions are solid so that we don't code ourselves into a corner and not understand it until 12-18 months later.

We hope to shift all guns 100% to D8 as soon as possible, but it still feels like we're not quite "there" yet with the inline editing functionality (comments here and elsewhere back this up), and so some continued D7 development still feels like a necessary evil for at least the next couple of weeks.

ethanw’s picture

FWIW, I'll cast my vote for parallel, but coordinated, CreateJS/REST and non versions of inline editing for D8, and maybe 7.

My main concern would be that the workarounds/compromises/decisions made to get editing working on D7/8 w/o REST will be very difficult to undo/adapt when we move to a more standards-based implementation. I think some of that could be addressed by developing both strategies somewhat in parallel.

As much as I've been focusing on D7 lately, and spend the vast majority of my working days there, I do take Sun's point to hear about the importance of focusing on D8 at this point in the development cycle.

henribergius’s picture

Sorry for not responding in the past couple of days, I was on a conference trip.

Anyway, I'm not sure I like the proposed approach of either:

  • Ditching the current Create.js porting effort, keeping on working on "plain Edit", and porting over later, or
  • Maintaining two separate Edit implementations ("plain" and "Create.js") in parallel

Both of these will mean at least some duplication of effort. They also mean that the Edit team may have to implement a lot of functionality that Create.js would've already provided on their own, negating some of the benefits of Create. And given the number of JavaScript developers available for working on Edit, having to reinvent wheels and maintain two versions will cause problems in getting the thing done.

There are some incremental paths that could be taken to minimize this issue, for example utilizing the Backbone Views work already done in the Create port also in "plain Edit". That way Create could be brought in piece-by-piece, for example first implementing Backbone.sync, then hooking that to VIE, and finally starting to use Create widgets. This would mean Backbone dependency also for the "plain Edit" module, but that should be something useful even without Create.

As for schedules and stabilization, Create.js is already working and integrated in TYPO3. They are somewhat similar to the Drupal approach here in that they also use a custom UI on top of it. Symfony CMF integration is only pending some issues related to Hallo Editor which they use as the default, so not directly Create-related.

In any case, it would be great to have a resolution. There are several things I'd like to do next in edit-createjs, but I've been holding off as there has been no go-or-no decision yet.

Related to this, there are two other things that should probably be addressed:

  • The Code Sprint proposed by frega. This would be a good way to solidify the plan, and also to get more people involved with developing Edit
  • Test automation. I recently migrated Create's unit tests to use PhantomJS. The same could be adapted for Edit
henribergius’s picture

AFAIK the only things that would be hard dependencies are the Backbone.js/Underscore.js libraries (for createJS) and the ability for CreateJS to work on the backend (for us). RDFa/JSON-LD support would be "services" that could be loaded by a contrib module that offers those features.

Correct. The dependencies for Create.js are:

  • jQuery UI widget factory (which Aloha Editor also depends on)
  • VIE core, entity, collection, views, and a DOM parser service (for example RDFa or a Drupal-specific data attribute parser)
  • Backbone and Underscore (that VIE depends on)
  • A Backbone.sync implementation (JSON-LD or form submission, depending on what Drupal back-end can support)
  • A WYSIWYG editor implementation (in this case Aloha)

So, with Create we do have some flexibility. Choosing what DOM parser to use to understand the content structure of the page, how to communicate changes back to the server, and what widget to actually use for rich text editing are all essentially just configuration questions.

Wim Leers’s picture

They also mean that the Edit team may have to implement a lot of functionality that Create.js would've already provided on their own, negating some of the benefits of Create.

This is wrong — let me clarify. "Bare" Edit (i.e. without Create.js) would focus on the shortest/simplest path to core inclusion, which would exclude functionality such as localStorage. The ability to plug in other WYSIWYG editors, however basic it may be, is already supported. In general, we would specifically avoid implementing non-essential functionality.

The major things still need to be implemented for Edit: validation, how to deal with Drupal's teasers, and an improved UX for saving (which comes with tight integration with Drupal's revisioning system). None of those are wholly or mostly provided by Create.js.

frega’s picture

i’ll be happy to work on keeping the edit-createjs repository up-to-date with upstream changes and continue working on some of the remaining kinks. As @ethanw writes this should be possible without too much effort and duplication if we keep clear track of workarounds and compromises.

a final thing regarding backbone.js/MV* though: introducing a significant new js codebase without a MV* foundation might make sustaining the effort throughout the d8/d7 lifecycle more difficult imho. choosing a library is a tough architectural decision and may not be possible at this point but the idea of a “js app” in core w/o it still makes me just a little uneasy :)

henribergius’s picture

let me clarify. "Bare" Edit (i.e. without Create.js) would focus on the shortest/simplest path to core inclusion, which would exclude functionality such as localStorage

Ok, I think I misunderstood webchick's comments above. This sounds like a reasonable approach, provided that the current functionality is more or less enough, meaning that no additional functional overlap with Create.js that would need to be implemented from scratch.

What would be the approximate time to start merging things under this proposal?

The major things still need to be implemented for Edit: validation, how to deal with Drupal's teasers, and an improved UX for saving (which comes with tight integration with Drupal's revisioning system). None of those are wholly or mostly provided by Create.js.

True, though for validation Backbone has the necessary hooks in place. Our intention is to provide a more declarative layer on top of this and VIE's type system so that validation rules are easier to load from CMSs without having to generate JS (see Flack's proposal on this). I could prioritize this.

What are the requirements for revision control when saving? Is this just a "commit message" entry, or something else?

As for teasers... If I understood this right, they would be more or less just a custom WYSIWYG editor plugin that would be able to insert the right HTML comments.

henribergius’s picture

True, though for validation Backbone has the necessary hooks in place. Our intention is to provide a more declarative layer on top of this and VIE's type system so that validation rules are easier to load from CMSs without having to generate JS (see Flack's proposal on this). I could prioritize this.

Even better, I could actually work on the implementation :-) We now have support for validating required fields and limited collection fields: VIE Entity.js (tests)

I'll add custom validation support tomorrow, and hook this into Create's Storage and Editable widgets so that they can show notices.

effulgentsia’s picture

Issue summary: View changes

Adding discussion summary to issue summary.

effulgentsia’s picture

"Team CreateJS/Drupal" (hopefully formed by several of the people in this issue who've expressed interest) could in parallel work on the underlying dependencies/nice-to-haves for CreateJS in Drupal 8 core

I think #1149866: Add Backbone.js and Underscore.js to core would be an appropriate issue for doing this, unless someone thinks otherwise and wants to open an alternate issue. I added it to the summary.

Wim Leers’s picture

What would be the approximate time to start merging things under this proposal?

Post-feature freeze. Feature freeze is Dec 1. So potentially, as soon as Dec 2.

True, though for validation Backbone has the necessary hooks in place. Our intention is to provide a more declarative layer on top of this and VIE's type system so that validation rules are easier to load from CMSs without having to generate JS (see Flack's proposal on this). I could prioritize this.

It's not quite that simple. Two different "number" fields might have different validation rules. Plus, it's all embedded in Drupal's Form API, which other modules can use to add *more* validation rules that are not known to the system (i.e. it's not configuration, it's not declarative, it's "we insert an extra validation callback here that is written in PHP and can thus have arbitrary logic").
So, when I said "validation", I really meant: "we do have validation already, but if there's a validation error, the Edit module's JS currently just fails, what we need to do is simply *show* the validation errors in a sensible location".
(I'm sorry, there's a lot of implied Drupalisms here.)
In other words: Backbone.js may have the necessary hooks in place, but Drupal does not, and that will most likely not change in Drupal 8. #1696648: [META] Untie content entity validation from form validation is the relevant issue.

Believe me when I say: "I wish we could use that", but we can't, yet, unfortunately.

What are the requirements for revision control when saving? Is this just a "commit message" entry, or something else?

It is that, and more, and the design/UX/workflow of it hasn't been fully defined yet, because we need to be able to play with it to figure out the very best way to deal with it.

As for teasers... If I understood this right, they would be more or less just a custom WYSIWYG editor plugin that would be able to insert the right HTML comments.

That's only one of the three ways Drupal supports teasers. (Yes, it's a mess.) Because of these different supported ways of doing teasers, there are conceptual issues. In short, the three ways are: 1) the <!--break--> tag which you pointed to, 2) automatic capping at X chars, 3) completely custom piece of text (i.e. not a subset of the whole). As you can see, that's a UX morass.

Even better, I could actually work on the implementation :-) We now have support for validating required fields and limited collection fields: VIE Entity.js (tests)

I'll add custom validation support tomorrow, and hook this into Create's Storage and Editable widgets so that they can show notices.

That's great! :) But please also see the above; Drupal is unfortunately simply not yet ready for it. Though if we could get the foundation of #1696648: [META] Untie content entity validation from form validation in by feature freeze, we could complete the conversion after feature freeze, so then this could stand a chance.

henribergius’s picture

Post-feature freeze. Feature freeze is Dec 1. So potentially, as soon as Dec 2.

Ok, then in that case it makes sense to focus on the enabling, non-UI features before that.

  • Implementing the data attribute parser as a VIE service (so it can be swapped to RDFa/Microdata later)
  • Implementing form submission -based Backbone.sync
  • Ensuring Create and VIE have the features Drupal will need in the future
  • ...and finally, trying to keep edit-createjs at least somewhat compatible with "plain Edit"

We'll try to meet with Frega next week and discuss how to proceed. Still, the proposed on-site code sprint would probably make sense, maybe around November?

It's not quite that simple. Two different "number" fields might have different validation rules. Plus, it's all embedded in Drupal's Form API, which other modules can use to add *more* validation rules that are not known to the system

That is not a problem. There could be different datatypes "subclassing" number and providing their own validation rules.

But anyway, that is just client-side validation. We still need server-side validation as well, and the main question here is being able to communicate those validation errors back to Create's Storage widget on Backbone save so that Storage can then display them accordingly.

We had some chat about this with Flack yesterday, and I'm pretty close to having a plan/initial implementation for it.

It is that, and more, and the design/UX/workflow of it hasn't been fully defined yet, because we need to be able to play with it to figure out the very best way to deal with it.

The original Create had a popover in the menu for seeing older revisions of a document, and reverting to those. This was done in quite Midgard-specific manner, and so I ripped it out when we made Create support other systems.

I think it would be good to do some UI sketches here to start with. I can post some pictures how Create used to do it a bit later.

That's only one of the three ways Drupal supports teasers. (Yes, it's a mess.)

Ok, sounds like this is something that mostly needs to happen on the WYSIWYG editor level. But maybe there are some parts that would make sense to make generic enough that they could be ported across systems.

effulgentsia’s picture

@henribergius: As mentioned in #3, one of our concerns in adding any vendor library to core is that we have long (1-3 years) cycles between major Drupal releases, and are very conservative about breaking backwards compatibility of any API (including vendor library APIs) within an already released major version. In the case of jQuery, this has meant having to freeze Drupal 5, 6, and 7 on versions of jQuery that quickly became outdated.

In #1451056: [policy] How to handle unforeseen diversion of Symfony code in stable/API-locked Drupal core?, the Symfony team helped us arrive at a better solution due to their backwards compatibility policy for the components we rely on.

Please share any info you can on the version management and BC policy of Create.js and the libraries it relies on in #1787222: [meta] Strategy for updating vendor JS libraries within a major stable version. Thanks!

henribergius’s picture

Please share any info you can on the version management and BC policy of Create.js and the libraries it relies on

Briefly, backwards compatibility is where both Create.js and VIE benefit from the fact that they consist of many components with relatively low coupling.

In VIE, many new features can be brought into the system by implementing them as new, pluggable services. We can for instance support both RDFa 1.0 and 1.1 by having two separate RdfaServices, and bring something like Microdata, or Drupal's custom data attributes in the same way.

VIE's public APIs have a very good test coverage, which also helps us to ensure we don't break things. We also used the service model to supply the VIE 1.0 API as a configuration option when we shipped 2.0. Even though I'm sure nobody uses VIE 1.0 APIs at this point, we still keep the "Classic" service around and tested just in case. The soon-to-be-released VIE 2.1 will be fully backwards-compatible with 2.0, and the intention is to keep things that way in the 2.x series. If 3.x will require BC breaks, we'll provide a compatibility layer like we did with the 1.x API.

With Create.js the same applies in that Create consists of different widgets. When we need to support a new way of handling workflows, a new notification system, or for instance a different major version of a WYSIWYG editor, we can bring that in as a new widget.

Create's test coverage is still much lower than VIE's, mainly because the Zombie headless browser we use for VIE's tests couldn't deal with the advanced browser features Create needs to test against, like contentEditable and localStorage. I recently swapped that to PhantomJS and now we can work to improve the test coverage.

henribergius’s picture

I had missed it earlier, but here is a pretty good post on how the Create integration was done in TYPO3.

henribergius’s picture

Also useful for Spark integration: Create.js now supports swapping the DOM parsing VIE service from RDFa to another in a clean way: https://github.com/bergie/create/issues/116

henribergius’s picture

We've been working on edit-createjs with @frega today. The DOM parsing has now been moved to a custom VIE service, meaning that we can run the Create Editable widget more-or-less as it is.

There is still refactoring to do, but this shows that we can abstract Edit's current DOM handling quite nicely behind a service that can later be replaced by VIE's standard services like RDFa.

henribergius’s picture

Two updates relevant to this bug:

effulgentsia’s picture

Also, yays or nays on #1149866: Add Backbone.js and Underscore.js to core would be most helpful.

RobLoach’s picture

They really need a package.json to define their package information. Then we could get it working with under AMD/RequireJS quite easily.

frega’s picture

Re: #90/@RobLoach - it's a little confusing, but there are two "create" javascript projects (createjs.com and createjs.org); you'll find the inline-editing "createjs" here: http://createjs.org or https://github.com/bergie/create (and it already has a package.json but for build purposes rather than client-side dependency declaration)

RobLoach’s picture

Ah, thanks so much for the clarification. This will help us build pre-release versions for the 8.x betas. I'll check out the sources! Know if bergie has plans to add it to his list of projects on npm?

webchick’s picture

Status: Needs review » Postponed

Just as an update, Jesse Beach is currently working with frega on a Backbone-ification of Edit module that will make Create/VIE integration easier down the road. That of course assumes that Create/VIE manages to survive the core gauntlet before feature freeze (it took us 100+ comments for Backbone/underscore alone, and we're not there yet :\ #1149866: Add Backbone.js and Underscore.js to core). frega also has some work in a sandbox (sorry, I don't know where - link? EDIT: found it! https://github.com/frega/edit-createjs) to get Edit module started down the Create/VIE path.

I'm going to formally postpone this particular issue until after Dec. 2, since that's the point at which we'd be looking back at this again. It does mean that there will be some "chasing HEAD" with the Edit module changes we make prior to proposing it as a core patch for the createJS fork. Let us know how we can as helpful as possible in enabling you to track those.

webchick’s picture

Status: Postponed » Fixed

Well, what a difference a week or so makes. ;)

So Jesse sat down to do just what I outlined in #93. But it turns out that in doing so, she discovered that the path to getting the JS cleaned up for core inclusion would actually be much easier by moving to Create.js directly, rather than trying to do an interim step to just Backbone. So atm she and frega are working together in the node/1808076-createjs branch of Edit module to do just that.

Given that most people in this issue were tentatively in favour of Create.js, given that Henri is actively working on getting the file size of VIE/Create.js down to acceptable levels, and given that there's pretty strong movement in various sub-issues that this depended on (for example, widgets and formatters are now plugins, there's some funding behind the JSON-LD movement, Backbone's in core, etc.) and given that frega is helping out here as well, it seems like this is the direction we're headed.

So. Answer? Yay. When? Now-ish. :)

Crell’s picture

w00t!

bfr’s picture

\o/ The community in action!

henribergius’s picture

Great! I'll be happy to help wherever needed. Hopefully we'll get to push things forward in the Gent code sprint...

jessebeach’s picture

Working with @henribergius, we augmented the build scripts to produce an additional build of both Create.js and VIE that include only the core scripts. These builds are significantly smaller than the previous, more feature-rich builds.

VIE build change pull request.

439K Oct 13 12:11 vie-ALL-latest.debug.js
195K Oct 13 12:11 vie-ALL-latest.js
241K Oct 13 12:11 vie-latest.debug.js
 71K Oct 13 12:11 vie-latest.js
131K Oct 13 12:11 vie-core-latest.debug.js
 32K Oct 13 12:11 vie-core-latest.js

The new minified core build (32K) of VIE is 16% of the minified ALL build (195K).

Create.js build change pull request.

 19K Oct 22 11:29 create-editonly-min.js
 41K Oct 22 11:29 create-editonly.js
 58K Oct 22 11:29 create-min.js
113K Oct 22 11:29 create.js

The minified editonly build (19K) of Create.js is 33% of the feature-full minified build (58K)

nod_’s picture

That is some awesome results :)

jessebeach’s picture

Wim Leers just brought up with me in chat that sizes in #98 are pre-gzip as well. Gzipped, we get

vie-core-latest.js.gz: 195K -> 32K -> 9.6K, yielding a build that's 4% of the minified ALL build (195K).

create-editonly-min.js.gz: 58K -> 19K -> 5K, yielding a build that's 8% of the feature-full minified build (58K).

henribergius’s picture

Not bad! :-)

vie-core-latest.js.gz: 195K -> 32K -> 9.6K, yielding a build that's 4% of the minified ALL build (195K).

VIE could probably go a bit smaller by doing some refactoring in the type system, like switching types, attributes, and namespaces to Backbone collections instead of the custom code they have now. But this would be a bit of work.

The good thing is, this build contains the full VIE core, and so Drupal modules can add services like Stanbol or RDFa as they need them. This sort of modularity on the JS side should fit the Drupal concepts quite well.

create-editonly-min.js.gz: 58K -> 19K -> 5K, yielding a build that's 8% of the feature-full minified build (58K).

Here we could also make the build somewhat smaller by removing widgets that Drupal is not using right now, including:

  • Collection handling widgets
  • Editor widgets other than Aloha and plain contentEditable (the build includes integration also for Hallo and Redactor)

But I guess this might not be necessary as the sizes are pretty good even now. And especially the Collection features may be something you'll want to have in Spark later on.

Wim Leers’s picture

@henribergius: It would be necessary eventually (before release), but for now it'd indeed be fine like this :)

henribergius’s picture

Minor update: we will be in in Gent next week hacking on this with Wim and Frega .

Anonymous’s picture

Great! I'll be sprinting on JSON-LD in San Fran at the same time. If your team could figure out specifically which properties you will need in the JSON-LD and give a few examples, it will help to define the requirements.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

effulgentsia’s picture

Thanks to the Gent sprint mentioned in #103, and some follow up from it, we now have #1849526: Add VIE and Create.js libraries to core (upstream fixes) ready (in my opinion) to commit to core despite some known upstream build management work still needed, but that can happen after feature freeze.

And the Edit module, now using these libraries, is ready for review at #1824500: In-place editing for Fields. Great job and thank you, Wim, henribergius, and Frega!

effulgentsia’s picture

Issue summary: View changes

Updated issue summary.