Create a new project to develop Drush entity support

FYI: here's an implementation for node-delete: http://elnoti.xsto.info/~jonhattan/node.drush.inc

and also I leveraged node overview filters (what is at admin/content/node in d6) at http://drupal.org/node/766930#comment-3451732 .. it wasn't shipped in that issue's commit.

I'd like for us to polish this and commit to Drush5.

Adjusting status...

One really minor thing:

+++ commands/node/node.drush.inc
@@ -0,0 +1,212 @@
+  );  ¶

You've got trailing whitespace here.

Powered by Dreditor.

+++ commands/node/node.drush.inc
@@ -0,0 +1,212 @@
+    foreach ($nids as $nid) {
+      $node = node_load($nid, NULL, TRUE);
+      db_query('DELETE FROM {node} WHERE nid = %d', $nid);
+      db_query('DELETE FROM {node_revisions} WHERE nid = %d', $nid);
+      node_invoke($node, 'delete');
+      node_invoke_nodeapi($node, 'delete');
+      if (function_exists('search_wipe')) {
+        search_wipe($node->nid, 'node');
+      }
+    }
+    cache_clear_all();
+  }

Couldn't you use just:

node_delete($nid);

Instead of copying it into Drush?

Powered by Dreditor.

node-delete is implemented this way to prevent calling cache_clear_all() for each node.

@jonhattan: makes sense.

Not sure it is needed, but probably wise to assure that temp file gets escapeshellarg(). Something like drush_shell_exec_interactive('$EDITOR %s', $tmpfile);

Mark's demo of these commands at Drupalcon was a big hit. See last 20 mins of our presentation at http://chicago2011.drupal.org/sessions/advanced-drush

Not sure it is needed, but probably wise to assure that temp file gets escapeshellarg(). Something like drush_shell_exec_interactive('$EDITOR %s', $tmpfile);

Mark's demo of these commands at Drupalcon was a big hit. See last 20 mins of our presentation at http://chicago2011.drupal.org/sessions/advanced-drush

Took another look at this. Items to fix up. For testing, the built-in article page type is pretty good. It has body and tags Fields on it.

1. Convert fully to entity from node
2. Somehow add line breaks to files that open in the editor. Is there a character that acts like linebreak that we can use and strip out before json_decode()?
3. entity-create from scratch is needs some help. We probably should add each Field to the array with empty value. For more points, add with default value. I fear though that this goes down the road of re-implementing devel's generate-content.

+++ commands/entity/entity.drush.inc
@@ -0,0 +1,259 @@
+ * @file
+ *   Drush support for nodes.

This should work for nodes only?

+++ commands/entity/entity.drush.inc
@@ -0,0 +1,259 @@
+    'aliases' => array('ns'),

A weird alias: entity-show versus ns ... I would suggest ens

+++ commands/entity/entity.drush.inc
@@ -0,0 +1,259 @@
+    foreach ($nids as $nid) {
+      $node = node_load($nid, NULL, TRUE);

Add comment about not using node_delete($nid) for D5/6

subscribe

Anyone available to push this along?

I've coded the comment from clemens.tolboom into a branch of my drush sandbox on top of the patch from #10
To better work together I've granted vcs push privileges to clemens.tolboom.

My sandbox branch: http://drupalcode.org/sandbox/helmo/1277350.git/shortlog/refs/heads/mast...

The commands are not consistent enough:

- entity-show(nids)

- entity-create()

- entity-edit(type, nids)

- entity-delete( nid)

as for D7 we require a type so helmo and clemens suggest to make type required.

For D6 the type is required but only implemented for nodes. At least for now.

So the commands become

- entity-show node 12
- entity-create comment
- entity-edit taxonomy 123
- entity-delete user 12

At least that's the idea for now :)

My (D7) setup

drush --yes site-install
drush --yes dl devel
drush --yes en devel_generate
drush genc 10 10
drush vocabs 2
drush gent tags 50

The following commands then function (all D7)

drush entity-list
drush entity-show node 1
drush entity-show taxonomy_vocabulary 1
drush entity-show taxonomy_term 1

I'm puzzled with the save flow. This works sometimes :(

drush entity-edit node 1

[Note: this is output from drush_helmo sandbox]

I'm not sure how to pipe the output to the next drush command.

This example: print entity ids from all available entities drush entity-list | drush enity-list should do the job. Instead I need to run drush el `drush el`.

# removed drush 'trivial' log
drush --verbose @drupal.d7 el `drush @drupal.d7 el`
Available ids for comment                                     [notice]
  1 2 3 4 5 6 7 8 9 10 13 15 16 19 22 23 24 25 26 28 29 11 12 14 17 18 20 21 27
Available ids for node                                        [notice]
  1 2 3 4 5 6 7 8 9 10 12 13 14
Available ids for file                                        [notice]
  1 2 3 4 5 6 7 8
Available ids for taxonomy_term                               [notice]
Undefined index: taxonomy_term entity.drush.inc:115           [notice]
array_keys() expects parameter 1 to be array, null given entity.drush.inc:115   [warning]
join(): Invalid arguments passed entity.drush.inc:115      [warning]
  
Available ids for taxonomy_vocabulary                         [notice]
  1
Available ids for user                                        [notice]
  0 2 3 4 1

After a lot of work and refactoring of clemens.tolboom and myself in the sandbox branch we feel that this is ready for review.

It includes a testcase with 10 assertions that tests the support for working with node entities in D7.

entity-delete only works on node.

The sandbox http://drupalcode.org/sandbox/helmo/1277350.git/shortlog/refs/heads/mast... is updated.

Update to latest implementation.

If fixed the entity delete. It works now for D7.

There is a warning when deleting entities ... not sure what/why?

array_flip(): Can only flip STRING and INTEGER values! entity.inc:178

This is what I did.

drush site-install --yes

# What is created during install
drush el `drush el`

drush --yes dl devel
drush --yes en devel_generate

drush gent tags 20
drush genc 20 5
drush genu 20 

drush el `drush el`

# Delete some comments
drush el comment
drush ed comment 26 20

# Try delete some files ... no force delete
drush el 'drush el`
drush el file
drush ed file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
drush el file

drush el comment
drush ed comment 1 2 3 7 9 10 12 13 14 15 16 18 19 23 24 25 27 28 29 4 5 6 8 11 17

drush el `drush el`
drush ed taxonomy_term 9 8 17 4 6 12 3 7 18 10 13 1 16 11 15 5 19 20 14 2

drush el `drush el`
drush ed user 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

drush el `drush el`
drush ed node 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

drush el `drush el`
drush ed taxonomy_vocabulary tags

Fixed the array_flip() warning in git, phpunit now runs OK without warnings.

Added sandbox link and rephrased what works.

Added h4 testcase section.

Added some todos

Added list of current testcases

Added link to sandbox issues.

We could use some feedback on our sandbox.

Here is a patchfile of the sandbox progress.
All tests are passing!

Update proposed solution to current state

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+ */
+function _drush_entity_type_read_list() {
+

Empty function

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+/**
+ *
+ * @param type $entity_type
+ */
+function _drush_entity_read_list($entity_type) {

Missing documentation

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+
+//  if (!drush_get_option('json')) {
+//    drush_die("You must specify --json");
+//  }
+

Remove this code.

Or should we require the --json?

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+// TODO this should be similar as drush_entity_save
+function _drush_entity_load($entity_type, $ids) {

No function documentation.

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+/**
+ * Permanently save an entity - Mostly stolen from entity api.
+ *
+ * In case of failures, an exception is thrown.

This documentation is not current. Code is moved into _drush_entity_op

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+    // D7 operator rename
+    $op_alias = 'deletion';
+    // D6 user_delete requires two arguments

These fixes need more documentation ... especially when adding D8

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+function _drush_entity_delete($entity_type, $ids = array()) {

No function docs

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+function _drush_entity_delete_node($nids) {

No docs.

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+/**
+ * Wrapper for entity_info()
+ *
+ * We wrap entity_info to make D5/6 compatible

Please document the D6 array structure.

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+      // What keys to delete on create
+      $entities_info['node']['drush']['new'] = array('nid', 'vid');
+      $entities_info['user']['drush']['new'] = array('uid');
+      $entities_info['file']['drush']['new'] = array('fid');

Are these settings not derivable from entity_info?

And please check devel_generate.

+++ b/commands/entity/entity.drush.inc
@@ -0,0 +1,767 @@
+      $entities_info['node']['drush']['defaults']['type'] = '';
+      $entities_info['node']['drush']['defaults']['title'] = '';
+      $entities_info['node']['drush']['defaults']['language'] = 'und';

This construct misses for all other (and new) entity types.

Can't we derive default values from somewhere ... field_info_* and friends?

See drush field* commands?

Update list of tests and options

Patch has some of the comments above.

I'm unsure whether out entity-create workflow is satisfying enough ... we still haven't dive into #13 (devel generate)

I like --fields=**/label which extract all label keys.

While creating a video I noticed (a forgotten) bug multiple value fields are indexed ... [0] seems to get ignored.

Fixed original user.

Added link to video.

Updated issue summary. Added link to gitweb diff

Reordered some links and added links to video transcript of commands

The latest patch (it's getting bigger)

Some remarks upfront:
- yaml conversion should be place into a drush-lib
- filter_fields should be made available into a drush-lib too.
Both are awesome to analyse data spit out by any drush command ie views

There are still unanswered questioned and tests reported in the issue summary. Feel free to answer these without reviewing the patch. We _need_ some answers :)

Perhaps msonnabaum can review this, as the original author of this feature. For me, the proposed implementation is a bit too feature rich for core drush.

@moshe weitzman please specify this 'too feature rich' a little for us as you spend some time reviewing this :_)

I wouldn't say yaml is not necessary. Instead it might be contributed in a separate patch that takes php objects and turns them into yaml — and we could use it in most of drush commands.

@barraponto thanks for the feedback :-)

I still have to dive into the --format

As I hope was clear from my video the yaml output is great for learning the internals so this should be a separate patch #1366098: Automatically download Symfony YAML component.

The variable-set and variable-get commands parse and print different data formats. That code should be factored out and re-used here, so yaml and other enhancements deemed necessary will be usable in a consistent fashion across the different drush commands where they are relevant.

@greg.1.anderson thanks for the hints.

What this means for this issue is I have to build out the yaml output (and probably more output stuff) as #34 first mentioned the --format switch.

I'm puzzled about the --fields ... it is so darn handy too for querying the entity system. My guess now is it should be in a separate issue. Not sure right now ... need some chewing on it :(

I have a small extension I wrote recently (couldn't find it mentioned in this issue) for 'entity-list'. Does what you expect really; provides a table of entities with a count, bundles, base table, etc... This has been useful for me. Is it wanted here? and worth providing a patch for?

@damiankloip sounds like a nice extension but not sure how much overlap there is.

We do not have counts so that sound like a nice addition. The output (and workflow) of

drush entity-read node | wc -w
   16

is ugly. I'm curious about your output :)

We do have the entity-type-read command which provides for the info (bundle, base table, etc) but presentation has it's value too. I'm not sure whether the example below works right now
drush --format=table entity-type-read user node file --fields="*/bundle,*/base table"
but this should be possible when format related issues like #1366098: Automatically download Symfony YAML component and/or #1396178: Add properties output as a --format get fixed together with the table format.

Ie drush entity-type-read node file user --fields="base table,bundles" --format=properties gives

file.base table = file_managed
file.bundles.audio.label = Audio
file.bundles.audio.admin.path = admin/config/media/file-types/manage/%file_type
file.bundles.audio.admin.real path = admin/config/media/file-types/manage/audio
file.bundles.audio.admin.bundle argument = 5

When patching make sure to use the git sandbox http://drupalcode.org/sandbox/helmo/1277350.git/shortlog/refs/heads/mast...

Helmo and I discussed whether move this patch into a separate project so people can install it easiers. What do you think?

I planned to work on this issue again when building a web spider on top of it. That is trying to clone a website though drush-entity :)

@clemens.tolboom I was thinking something simpler (although I like aspects of the above implementation) like views-list does. So an entity-list command that shows alot of this info, including counts, as a table. I didn't realise about the other thread to add a --format option for the output. I was thinking it would be nice to have commands that are easier to use/type too.

Ha, clone website with drush entity :)

@damiankloip please provide an example output (or patch :p) ... I did not know views-list so tnx :)

@clemens.tolboom No problem. Currently it is in it's own module. I will post this for you to see. I think this is the easiesy way for you to see what it does. If we need to we can then make a patch for the appropriate file?

Here is a small module showing a similar implementation to views-list functionality. I think one of the main benefits is the ease of the command; Just "drush el" or "drush el user,node" for example rather than the current more verbose proposal. I think maybe both have their place?

Let me know what you think!

@damiankloip tnx for the code.

drush entity-list node,file
Entity count: 2

 Entiteit  Label    Bundles                        Base table    Revision table  Aantal  Fieldable  Entity class  Controller class              
 node      Node     Article (article), Basic page  node          node_revision   2       TRUE       Standaard     NodeController                
                    (page), Hammer (hammer)                                                                                                     
 file      Bestand  Bestand (file)                 file_managed  NULL            2       FALSE      Standaard     DrupalDefaultEntityController

@clemens.tolboom: Dodgy formatting but yes that is the idea :)

Pushed #27 into issue summary
Linked to #1330504: Test the File entity

We moved our code to http://drupal.org/project/drush_entity to have better issue management, easy download for interested people and more developers.

Current developers now @helmo @damiankloip @clemens.tolboom ... feel free to join.

I mark this issue postponed (hope it will not hide on the issue overview).

I put a pointer to drush_entity on the Resources page of drush.org. If we bring drush_entity back to core, we can do it via another issue.

@greg.1.anderson tnx

Added link to project page.

Updated issue summary.