Proposal : simpletests for CCK field storage [#209391]

Thanks to Karen, CCK now exposes a real field CRUD API (field management was previously not abstracted from the UI forms), and Jeff Eaton started some work to build the long-awaited feature of having modules define content types their cck fields through a hook (think : image_gallery module based on imagefields...)

But getting CRUD operations out of the controlled and limited form-based workflow to an API extends the possible edge cases. The complexity of the code, the variety of possible combinations, make it critical that we have a testsuite that ensures the thing is rock-solid and that subsequent commits don't break it to pieces.

CCK db storage of fields data, while sometimes seen as convoluted, and undoubtedly translating into consequent code in content.module, really boils down to 3 simple rules :

- Fields that accept multiple values *and* fields that are shared between several node types are stored in dedicated 'per field' tables (content_field_FIELDNAME).
- All other fields (single-valued, non-shared fields) are stored together in a 'per node type' table (content_type_TYPENAME).
- Field storage can require several db columns, depending on the field type and the user-defined settings for the field.

(see the excellent article
Robert Douglass for detailed insight on whys and hows, but the above rules says it all)

The testsuite would consist in a series of field manipulations. On each step, 3 tests have to be done :
- check drupal_get_schema is the expected schema
- check the actual db structure matches drupal_get_schema (the testsuite would require Barry Jaspan's schema.module for its db inspection features)
- check a node_load returns the expected field data

While the actual list of tests that would ultimately constitute an exhaustive suite can be discussed and refined later, I think a great scope for a GHOP task would be to lay down the overall structure and reusable code, and write only the first few tests.

In order to fix some ideas, here's what the list of tests to be perfomed would look like. I repeat that the GHOP task itself would be considered finished when, say, the first two '**' items would be actually implemented.

content types : type1, type2, type3
nodes : node1 (type1), node2 (type2), node3 (type3)
Each '**' item below starts with a fresh set of types and nodes.

**** Basic tests : creation, multiplicity, sharing, deletion

** Check single->multiple + sharing multiple
field1 : text field, plain text, single
- add field1 to type1, fill in a value for node1
- change field1 to multiple
- fill in value*s* for node1
- add field1 to type2, fill in values for node2
- add field1 to type3, fill in values for node3
- remove field1 from type3
- remove field1 from type2
- remove field1 from type1

** Check multiple->single + sharing single
field1 : text field, plain text, multiple
- add field1 to type1, fill in a value for node1
- change field1 to single
- fill in values for node1
- add field1 to type2, fill in values for node2
- remove field1 from type2
- remove field1 from type1

** Repeat the above using 'formatted' text fields (uses 2 storage columns)


**** Check altering both multiplicity and columns set

**
field1 : text field, plain text, single
- add field1 to type1, fill in a value for node1
- change field1 to formatted, multiple
- fill in values for node1

** 
field1 : text field, plain text, multiple
- add field1 to type1, fill in values for node1
- change field1 to formatted, single
- fill in value for node1

(...)


**** Check altering both shared status and columns set

(...)


**** Check altering both shared status and multiplicity

(...)

Comments

Comment #1

yched CreditAttribution: yched commented 13 January 2008 at 15:56

As a sidenote : this task does not require any particular knowledge of CCK's internals. One of the (probably rare...) areas that can bring CCK to a new level without requiring a CCK guru :-)
The tests will mainly involve :
- playing with $field and $schema definition arrays,
- using a set of api functions (node_load, node_save, content_field_instance_create, content_field_instance_update, content_field_instance_delete, drupal_get_schema, schema_compare_table...),
- being logical when writing the expected values to check against.

I'll also add that I think this is both a challenging task wrt tests structuring and reusability, and a highly rewarding one, since it is a really critical step for the future of CCK. We're actively working on getting a beta D6 release, but I consider these tests a condition to a 'stable' official release, and an enabler for highly-awaited stuff like 'hook-defined fields' and utimately 'fields in core'. Really.

Comment #2

cwgordon7 CreditAttribution: cwgordon7 commented 13 January 2008 at 18:48

Brief comments:

Mentor?

Resources?

This needs to be written up according to the "How to Write a Good Task" guidelines.

Comment #3

cwgordon7 CreditAttribution: cwgordon7 commented 13 January 2008 at 18:48

Status:

Active

» Needs work

Setting to cnw

Comment #4

yched CreditAttribution: yched commented 14 January 2008 at 00:35

Status:

Needs work

» Active

My initial goal here was to gather some feedback before writing the 'official task description' :-), but you're right, here it is refactored :

-----------------------------

CCK is one of the most widely used contributed modules for Drupal. It allows site administrators to define the fields (text, date, image...) that compose their content types, and handles fields data storage. It has become a basic block in building drupal sites

CCK2.0 for Drupal 6 finally exposes a real field CRUD API (as opposed to hard-tied to admin-UI form submission in D5).
This was a necessary step for highly-awaited features such as hook-defined fields (allowing modules to define their own content types and populating them with a predefined set of fields - think image gallery module based on imagefields)

Getting CRUD operations out of the controlled and limited form-based workflow to an API extends the possible edge cases. The complexity of the code, the variety of possible combinations, make it critical that we have a testsuite that ensures the thing is rock-solid and that subsequent commits don't break it to pieces. It is also an important step for the anticipated inclusion of the field storage engine into drupal core.

CCK's dynamic db storage of fields data, while sometimes seen as convoluted, and undoubtedly translating into consequent code in content.module, really boils down to 3 simple rules :
- Fields that accept multiple values *and* fields that are shared between several node types are stored in dedicated 'per field' tables (content_field_FIELDNAME).
- All other fields (single-valued, non-shared fields) are stored together in a 'per node type' table (content_type_TYPENAME).
- Field storage can require several db columns, depending on the field type and the user-defined settings for the field.
More detailed insight on whys and hows can be found in the RESOURCES mentioned below, but the above basically say it all.

The testsuite would consist in a series of field crud manipulations. On each step, 3 tests have to be done :
a) Check drupal_get_schema is the expected database schema according to the rules above.
Checking for tables and columns is probably enough, without going as far as actual column definitions.
b) Check the actual db structure matches drupal_get_schema.
We can use Schema.module's db inspection features : see schema_compare_table() function.
c) Check that a node_load returns the expected field data.

In order to lay some ideas about the needs at hand, I posted an initial checklist in http://groups.drupal.org/node/8148.
The scope of the GHOP task is to lay down the overall structure and reusable code, and actually code a limited initial set of tests, for instance the first two in the list. The actual list of tests that would ultimately constitute an exhaustive suite can be discussed and implemented later.

Mentors
yched
probably KarenS, but I'll leave it to her to confirm :-)
Rok Zlender has agreed to provide support for simpletest aspects

Deliverables
The task is considered completed when a patch has been posted to the CCK issue queue and marked 'ready to be committed' by one of CCK's maintainers.

Ressources
CCK :
- CCK handbook
more specifically : http://drupal.org/node/82661, http://drupal.org/node/133705
- Robert Douglas's detailed walkthrough
- PHPDocs for content_field_instance_* CRUD functions in CCK's content_crud.inc

Simpletests :
- Simpletest handbook
- Angie Byron's and Robert Douglas's excellent introductions

Required modules
CCK HEAD (for D6)
Simpletest module
Schema module

Comment #5

KarenS CreditAttribution: KarenS commented 14 January 2008 at 01:09

Of course, I'll help mentor this! This project would be a great boon for CCK and for anyone who depends on it to work reliably. I see lots of Drupal Karma ahead for anyone who picks this up :)

Comment #6

aclight CreditAttribution: aclight commented 14 January 2008 at 02:28

Status:

Active

» Needs work

I've never used simpletest before and am not all that familiar with it, but my feeling about this is that you have a very thorough background but the task itself is a bit vague still. I could be wrong, however.

The deliverable should also specify what version of CCK the patch should be rolled against.

Comment #7

KarenS CreditAttribution: KarenS commented 14 January 2008 at 11:29

How about the following? I made a few small changes to yched's proposal to try to clarify things:
--------------------------------------------------------------------------------------------------------------------------

Task
Create simpletests for the D6 version of CCK to test the new CRUD (create, read, update, delete) API. The tests should create a user who makes various changes to the field settings, using examples at http://groups.drupal.org/node/8148 as a starting point, then test that they produce the right results.

On each step, 3 tests have to be done :

a) Check drupal_get_schema is the expected database schema according to the rules above. Checking for tables and columns is probably enough, without going as far as actual column definitions.

b) Check the actual db structure matches drupal_get_schema. We can use Schema.module's db inspection features : see schema_compare_table() function.

c) Check that a node_load returns the expected field data.

The scope of the GHOP task is to lay down the overall structure and reusable code, and actually code a limited initial set of tests, for instance the first two in the list. The actual list of tests that would ultimately constitute an exhaustive suite can be discussed and implemented later.

Background

CCK's dynamic db storage of fields data, while sometimes seen as convoluted, and undoubtedly translating into consequent code in content.module, really boils down to 3 simple rules :

More detailed insight on whys and hows can be found in the RESOURCES mentioned below, but the above basically say it all.

Mentors
yched
KarenS
Rok Zlender has agreed to provide support for simpletest aspects

Deliverables
The task is considered completed when a patch has been posted to the CCK issue queue and marked 'ready to be committed' by one of CCK's maintainers.

Simpletests :
- Simpletest handbook
- Angie Byron's and Robert Douglas's excellent introductions

Required modules
CCK HEAD (for D6)
Simpletest module
Schema module

Comment #8

aclight CreditAttribution: aclight commented 14 January 2008 at 14:32

Honestly, I still don't quite get this task, but I'm willing to admit that it may be that I am dense :)

So, with regards to the structure of the task idea, there are a few changes that should be made:

A.) Deliverables section should specify which branch of CCK the patch should be rolled against.

B.)

The scope of the GHOP task is to lay down the overall structure and reusable code, and actually code a limited initial set of tests, for instance the first two in the list. The actual list of tests that would ultimately constitute an exhaustive suite can be discussed and implemented later.

For the sake of Google, you need to actually include the list of tests in the task itself, instead of an external site.

You may want to take a look at another simpletest task that has been completed already:
http://code.google.com/p/google-highly-open-participation-drupal/issues/...

chx used this template for many of his simpletest tasks. The last 3 paragraphs are key, I think.

I'm willing to let this get created as an official task even if I don't completely understand it, but the things I pointed out in A and B should be fixed so the student doesn't get confused and so that it's clear from just the task description what is expected of the student.

Comment #9

KarenS CreditAttribution: KarenS commented 14 January 2008 at 16:47

Another try, with no formatting, since it looks like there is no formatting on
the Google site. Yched, please double check my table name results to be
sure I didn't miss anything.
===========================================

Create simpletests for the D6 version of CCK to test the new CRUD (create,
read, update, delete) API. This task is to create an overall structure for unit
tests that will ensure this functionality works, and implement a few tests
to use as a starting point for further test development.

The Simpletest module provides a framework for running automated unit
tests in Drupal. Unit testing involves taking a small portion of code, a
unit, and subjecting it to programmatic tests to prove its correctness.
This can be extremely useful to help insure that new code does not
introduce bugs into existing functionality. Thus, having a full suite of
unit tests for Drupal 6.x core functionality using the Simpletest module
would be a major step forward for the project and would help insure that
only high-quality code is committed. In addition to helping developers
test their code, new tests for core Drupal functionality will be used for
automated testing by http://testing.drupal.org.

For this task, you need to first install and familiarize yourself with the
simpletest module (use the cvs HEAD version with Drupal HEAD/6.x RC):
http://drupal.org/project/simpletest
http://drupal.org/simpletest
http://www.lullabot.com/articles/introduction-unit-testing

A very very crude start for the tests can be created from the browser with
http://drupal.org/project/simpletest_automator -- generated tests need
serious cleanup and editing.

The tests should create a user who makes various changes to the field
settings, then test that they produce the right results.

What to test in each step below:

* Check drupal_get_schema is the expected database schema according
to the rules above. Checking for tables and columns is probably enough,
without going as far as actual column definitions.

* Check the actual db structure matches drupal_get_schema. We can
use Schema.module's db inspection features : see
schema_compare_table() function.

* Check that a node_load returns the expected field data.

Create two tests:

============================================
Test 1
============================================
Check single->multiple + sharing multiple field1

* Create three new content types : type1, type2, type3.

* Create three new nodes, one each in each content type,
node1 (type1), node2 (type2), node3 (type3)

* Create a field, field1, a single value text field that uses formatted text.

* add field1 to type1, fill in a value for node1
A table called 'content_type_type1 should be created with the following
columns: nid, vid, value, format.
Check that the node has the right values.

* change field1 to multiple, fill in values for node1
The 'value' and 'format' columns should drop out of the table 'content_type_type1'
and a new table should be created called 'content_field_field1' which should
have columns nid, vid, delta, value, and format.
Check that the node has the right values

* add field1 to type2, fill in values for node2
A new content_type_type2 table should be created with
columns nid and vid.
Check that the new node has the right values.

* add field1 to type3, fill in values for node3
A new content_type_type3 table should be created with
columns nid and vid.
Check that the new node has the right values.

* remove field1 from type3
The content_type_type3 table should disappear.
Node3 should now have no field.

* remove field1 from type2
The content_field_field1 table should disappear. The value and format
columns should move to the content_type_type1 table.
Node 1's field should have lost all values except the first value.
Node 2 should have no field.

* remove field1 from type1
The content_type_type1 table should disappear.
All nodes should lose their field values.

============================================
Test 2
============================================
Check multiple->single + sharing single field1

* Create three new content types : type1, type2, type3.

* Create three new nodes, one each in each content type,
node1 (type1), node2 (type2), node3 (type3)

* Create a field, field1, a multiple value text field that uses formatted text.

* add field1 to type1, fill in values for node1
A new table should be created, 'content_field_field1' with
columns nid, vid, delta, value, and format.

* change field1 to single, fill in values for node1
The content_field_field1 table should disappear. The value and
format columns should be added to content_type_type1.
Check that node1 retains the first value in the field.

* add field1 to type2, fill in a value for node2
A new content_type_type2 table should be created with
columns nid and vid.
The content_field_field1 table should lose the value and format columns,
and a new content_field_field1 table should be created with
columns nid, vid, value, and format.
Check that node1 did not lose its value and that node 2 has the right
value.

* add field1 to type3, fill in a value for node3
A new content_type_type3 table should be created with
columns nid and vid.
Check that all nodes have the right values.

* remove field1 from type3
The table content_type_type3 should disappear.
Node 3 should lose the field value.

* remove field1 from type2
The table content_type_type2 should disappear.
The table content_field_field1 should disappear.
The table content_type_type1 should now have
columns nid, vid, value, format
Node 2 should lose the field value.
Node 1 should still have the right value.

* remove field1 from type1
The table content_type_type1 should disappear.
None of the nodes should have a field value.

This suite should be written as a single .test file and should achieve
RTBC status at http://drupal.org/node/?????

Resources
CCK :
- http://drupal.org/node/101723 (CCK handbook), especially http://drupal.org/node/82661, http://drupal.org/node/133705
- Robert Douglas's intro to CCK at http://www.lullabot.com/articles/an_introduction_to_the_content_construc...
- PHPDocs for content_field_instance_* CRUD functions in CCK's content_crud.inc

Simpletests :
- http://drupal.org/simpletest (Simpletest handbook)
- Angie Byron's and Robert Douglas's unit test explanation at http://www.lullabot.com/articles/drupal-module-developer-guide-simpletest

Required modules
http://drupal.org/node/96064 (CCK HEAD (for D6))
http://drupal.org/project/simpletest (Simpletest module)
http://drupal.org/project/schema (Schema module)

Comment #10

aclight CreditAttribution: aclight commented 14 January 2008 at 16:53

Status:

Needs work

» Needs review

This looks much better to me.

@yched: If this looks good to you, feel free to RTBC this and create an official task according to the instructions at http://groups.drupal.org/node/7360#tasks (you can start with the 3rd or 4th bullet, your choice).

Comment #11

KarenS CreditAttribution: KarenS commented 17 January 2008 at 16:47

Status:

Needs review

» Reviewed & tested by the community

@aclight, yched asked me to go ahead and finish this up, which I've done. Here is the updated task, but I don't see any way I can actually add it to the Google GHOP page.
====================================

The Simpletest module provides a framework for running automated unit
tests in Drupal. Unit testing involves taking a small portion of code, a
unit, and subjecting it to programmatic tests to prove its correctness.
This can be extremely useful to help insure that new code does not
introduce bugs into existing functionality. Thus, having a full suite of
unit tests for Drupal 6.x CCK functionality using the Simpletest module
would be a major step forward for the project and would help insure that
only high-quality code is committed.

A very very crude start for the tests can be created from the browser with
http://drupal.org/project/simpletest_automator -- generated tests need
serious cleanup and editing.

The tests should create a user who makes various changes to the field
settings, then test that they produce the right results.

What to test in each step below:

* When checking the database structure, check two things:

** Check drupal_get_schema is the expected database schema according
to the rules above. Checking for tables and columns is probably enough,
without going as far as actual column definitions.

** Check the actual db structure matches drupal_get_schema. We can
use Schema.module's db inspection features : see
schema_compare_table() function.

* When checking a node's values, test that a node_load returns the
expected field data.

Create simpletest code for CCK that will easily build tests, i.e. using setUp and
tearDown functions to provide self-clearing test context , finding a
way to have "now re-run tests 1-10 using a different node types/fields
context".

Create two initial tests:

============================================
Test 1
Check single->multiple + sharing multiple field1
============================================

* Create three new content types : type1, type2, type3.

* Create three new nodes, one each in each content type,
node1 (type1), node2 (type2), node3 (type3)

* On content type type1, create a field, field1, a single value text field that
uses filtered text (this will create a 2 column field).
A table called 'content_type_type1 should be created with the following
columns: nid, vid, field_field1_value, field_field1_format.

* Fill in a value for node1, set the format to Full HTML.
Check that the node has the right values.

* change field1 to unlimited
The field_field1_value and field_field1_format columns should drop out of the
table 'content_type_type1' and a new table should be created called
'content_field_field1' which should have columns nid, vid, delta,
field_field1_value, and field_field1_format, with the values that
were in the previous table.

* fill in values for node1
Check that the node has the previous values in the first available slot
and that both the value and the format are correct.

* add existing field1 to type2
A new content_type_type2 table should be created with
columns nid and vid.

* fill in values for node2, and set the format to Full HTML.
Check that the new node has the right values.

* add existing field1 to type3
A new content_type_type3 table should be created with
columns nid and vid.

* fill in values for node3
Check that the new node has the right values.

* remove field1 from type3
The content_type_type3 table should disappear.
Node3 should now have no field1.

* remove field1 from type2
The content_field_field1 table should disappear.
Node 2 should have no field.

* remove field1 from type1
The content_type_type1 table should disappear.
All nodes should lose their field values.

============================================
Test 2
Check multiple->single + sharing single field1
============================================

* Create three new content types : type1, type2, type3.

* Create three new nodes, one each in each content type,
node1 (type1), node2 (type2), node3 (type3)

* Create a field, field1, a multiple value text field that uses
filtered text (to create a 2 column field) and add field1 to type1.

A table called 'content_type_type1 should be created with the following
columns: nid, vid.

A new table should be created, 'content_field_field1' with
columns nid, vid, delta, field_field1_value, and field_field1_format.

* fill in several values for node1, set the formats to Full HTML
Check that the node has the right values.

* change field1 to single
The content_field_field1 table should disappear. The
field_field1_value and field_field1_format columns should
be added to content_type_type1.

* fill in values for node1
Check that node1 retains the first value in the field.

* add existing field1 to type2
A new content_type_type2 table should be created with
columns nid and vid.

The content_type_type1 table should lose the field_field1_value
and field_field1_format columns, and a new content_field_field1
table should be created with columns nid, vid, field_field1_value,
and field_field1_format.

* fill in a value for node2
Check that node1 did not lose its value and that node 2 has the right
value.

* add field1 to type3
A new content_type_type3 table should be created with
columns nid and vid.

* fill in a value for node3
Check that all nodes have the right values.

* remove field1 from type3
The table content_type_type3 should disappear.
Node 3 should lose the field value.

* remove field1 from type2
The table content_type_type2 should disappear.
The table content_field_field1 should disappear.
The table content_type_type1 should now have
columns nid, vid, value, format

Node 2 should lose the field value.
Node 1 should still have the right value.

* remove field1 from type1
The table content_type_type1 should disappear.
None of the nodes should have a field value.

This suite should be written as a single .test file and should achieve
RTBC status at http://drupal.org/node/?????

Required modules
http://drupal.org/node/96064 (CCK HEAD (for D6))
http://drupal.org/project/simpletest (Simpletest module)
http://drupal.org/project/schema (Schema module)

Comment #12

KarenS CreditAttribution: KarenS commented 17 January 2008 at 16:48

Note that I still need to update it with the right Drupal node for the task, but I didn't want to create that until this is committed.

Comment #13

aclight CreditAttribution: aclight commented 17 January 2008 at 17:03

@KarenS: Go to http://code.google.com/p/google-highly-open-participation-drupal/issues/... to create an issue. You'll need to log in using a google/gmail account. You should be able to create an issue, though you won't be able to set the status to open or change tags. For that, you'll need to be a member of the group. You can either just submit the issue sans tags and I'll fix those later, or you can contact me via contact tab on d.o or aclight on IRC and send me your gmail/google account, and I'll add you.

Make sure to also create an issue in the appropriate queue on d.o to follow the progress internally. I'd create that first, then create the google task with a mention of that issue URL, and then once you've submitted the ghop task (and know what the task # is), you can change the title of the d.o issue to reflect that, eg. "GHOP #160: Simpletests ..."