Not really sure how to explain this one but here's what i've found so far.

When creating the cck fields $field['field_name'] is set to pirets_{systemName} system name being the System Name from the RETS Server.

line 483 in pirets.admin.inc:

content_field_instance_create($cck_field);

which eventually calls:
_content_field_write($field, $op); the 'create' is the one that inserts the field name into content_node_field. Problem is that this field has a length of 32:

  $schema['content_node_field'] = array(
    'fields' => array(
      'field_name'      => array('type' => 'varchar', 'length' => 32, 'not null' => TRUE, 'default' => ''),
      'type'            => array('type' => 'varchar', 'length' => 127, 'not null' => TRUE, 'default' => ''),
      'global_settings' => array('type' => 'text', 'size' => 'medium', 'not null' => TRUE, 'serialize' => TRUE),
      'required'        => array('type' => 'int', 'size' => 'tiny', 'not null' => TRUE, 'default' => 0),
      'multiple'        => array('type' => 'int', 'size' => 'tiny', 'not null' => TRUE, 'default' => 0),
      'db_storage'      => array('type' => 'int', 'size' => 'tiny', 'not null' => TRUE, 'default' => 1),
      'module'          => array('type' => 'varchar', 'length' => 127, 'not null' => TRUE, 'default' => ''),
      'db_columns'      => array('type' => 'text', 'size' => 'medium', 'not null' => TRUE, 'serialize' => TRUE),
      'active'          => array('type' => 'int', 'size' => 'tiny', 'not null' => TRUE, 'default' => 0),
      'locked'          => array('type' => 'int', 'size' => 'tiny', 'not null' => TRUE, 'default' => 0),
    ),
    'primary key' => array('field_name'),
  );

Herein lies the problem, it's not uncommon for "pirets_{systemName}" to be longer than 32 characters. Not sure exactly what the correct fix is for this, don't think it's really appropriate to modify the schema['content_node_field'] to increase the 'field_name' to longer than 32.

Long story short, what ends up happening is that when cck goes to build the query, it does so with the truncated column name, which it subsequently can't find in content_type_pirets_prop. Causing that insert to fail. Which just chains down to a big mess, as some of the field instances are created, the record is inserted in node and node_revision, but not in content_type_pirets_prop.

Need to come up with a way to handle field_names longer than 32 characters, or 'pre-truncate' them before creating the cck field.

CommentFileSizeAuthor
#5 pirets-726152-2.patch6.96 KBcamidoo
#4 pirets-726152-1.patch6.2 KBcamidoo

Comments

camidoo’s picture

Title: Inserts faile due to cck fieldnames being > 32 chars » Inserts fail due to cck fieldnames being > 32 chars
Garrett Albright’s picture

Hmm, interesting. Outright truncating field names could be problematic, however, because if multiple field names have the same first (or last) 32 - strlen('pirets_') = 25 characters, there will be ambiguity. Unlikely, but so is a RETS field name longer than 25 characters… (Out of curiosity, what is the name of the field(s) causing this problem?)

I'll mull over this a bit.

camidoo’s picture

specifically:

  • L_listings_associated_doc_count
  • LFD_SHOWINGINSTRUCTIONS_32
  • LFD_LANDLORDRESPONSIBILITY_38
  • LFD_EXTERIORCONSTRUCTION_73
  • LFD_TENANTRESPONSIBILITY_88
  • LFD_UTILITIESONPROPERTY_48
  • LFD_MISCELLANEOUSFEATURES_64
  • LFD_SHOWINGINSTRUCTIONS_85

not that the field names are > 32, it's that the strenlen('systemName') + strlen('pirets_') > 32.

I'm about 3/4 the way through a fix, which knocks out 2 birds with one stone.

1. Add a column to pirets_fields called something like `cck_field_name` varchar(32)
2. Add a textfield to pirets_sets_fields_form(), make it's default value:

#default_value = $field['cck_field_name']?$field['cck_field_name']:$substr(field['system_name'], 0, 25);

3. Modify pirets_sets_fields_form_submit() to set the $cck_field['field_name'] = the new cck_field_name from #2.
4. Then go through and ensure the loops defining cck fieldnames now use the cck_field_name field from pirets_fields instead of system_name where you would basically have some mapping / correlation like:

$cck_field[$pirets_field_instance['cck_field_name']] = $from_rets[$pirets_field_instance['system_name']];

the $pirets_field_instance being a row from pirets_fields `pirets_fields`.cck_field_name && `pirets_fields`.system_name

I think it'd be a pretty elegant solution, gracefully pre-truncates the field names to 'pirets_' + 25 characters. Allows the user to define the field names if they like for less cryptic use in theme files. (ie: $pirets_re_bathrooms_upper vs. $pirets_LM_Char_10_27.

As far as ambiguity, can always pull the _form_validate straight from cck's _content_field_overview_form_validate_add_new()

    
    ....

      // Invalid field name.
      if (!preg_match('!^field_[a-z0-9_]+$!', $field_name)) {
        form_set_error('_add_new_field][field_name', t('Add new field: the field name %field_name is invalid. The name must include only lowercase unaccentuated letters, numbers, and underscores.', array('%field_name' => $field_name)));
      }
      if (strlen($field_name) > 32) {
        form_set_error('_add_new_field][field_name', t('Add new field: the field name %field_name is too long. The name is limited to 32 characters, including the \'field_\' prefix.', array('%field_name' => $field_name)));
      }
      // A field named 'field_instance' would cause a tablename clash with {content_field_instance}
      if ($field_name == 'field_instance') {
        form_set_error('_add_new_field][field_name', t("Add new field: the name 'field_instance' is a reserved name."));
      }

      // Field name already exists.
      // We need to check inactive fields as well, so we can't use content_fields().
      module_load_include('inc', 'content', 'includes/content.crud');
      $fields = content_field_instance_read(array(), TRUE);
      $used = FALSE;
      foreach ($fields as $existing_field) {
        $used |= ($existing_field['field_name'] == $field_name);
      }
      if ($used) {
        form_set_error('_add_new_field][field_name', t('Add new field: the field name %field_name already exists.', array('%field_name' => $field_name)));
      }

    ....

Further, i don't think it's all that uncommon to find systemNames that are fairly long, most these MLS implementations that i work with vary WIDELY, and are practically left up to the whim of the person setting them up, or adding Fields for the associations as to whether they follow any naming convention, standard, or a rhyme or reason. Or the Associations themselves adding fields to the system.

Anyhow, my .02 =)

Thoughts / Comments?

camidoo’s picture

StatusFileSize
new6.2 KB

Implementation of #3 minus the validation..

camidoo’s picture

StatusFileSize
new6.96 KB

small fix to the patch from #4

camidoo’s picture

Status: Active » Needs review

Setting to needs review

Garrett Albright’s picture

Adding another column and doing the CCK field mapping like this feels like a rather "heavy" solution to this problem. And I know it also solves the theming-funky-field-names problem, but I'm not really convinced that's that big of a deal in the first place either. I'm starting to lean back towards simple truncation - brutishly simple, but it'll be quick, require no new DB columns, and better than nothing.

camidoo’s picture

Status: Needs review » Closed (fixed)
Garrett Albright’s picture

Status: Closed (fixed) » Needs review

No, keep it open. There needs to be a solution to this. I'm just not sure what form it will take yet.

Garrett Albright’s picture

Okay, I've implemented a workaround. It stores the CCK name in {pirets_fields}, but instead of adding a new table column, it redefines a current one (the "cck" field, which was previously just a 1/0 field indicating whether the field was used by CCK or not). It also will truncate field names which are too long, truncating from the end instead of the beginning. So it kind of meets in the middle between your approach and mine. Please give it a try when the latest dev release becomes packaged and available (or go fetch a tarball from the Bitbucket page, if you're up for it).