I am batch loading some legacy data (several hundred articles) into a special node type consisting of standard node fields and several CCK fields. With the exception of this truncation problem, the new nodes and all the fields and cck fields are being populated correctly.
The problem is that while I am passing to $node->body a string of about 15,000 - 20,000 characters representing a html tagged article what is written to the DB is randomly truncated. The behavior is strange!!!
In my program, I confirm that $node->body is populated and the strlen returns the full count of characters. All looks correct before the write. After a write of the new node via node_save($node) I examine var_dump($node); and it still contains the full string in the body. My understanding is that var_dump is dumping the resulting object that was modified by the node_save.
However, if I then immediately go to SQL and look at the body field in node_revisions its been severely truncated. While each legacy article I am loading has a body of over 15k characters, the DB ends up only with some random truncated value for body: Sometimes its 19 char, sometimes 551, 870, 23, etc. I can't make any sense of it and it does not seem to line up with any of the html tags in the original string either.
Drupal 6.16
I don't think its the html tags, at least there are no special tags just the normal paragraph and image tags but none of the tags seem to line up with how the truncation occurs. I notice that the truncation seems to be on word boundaries.
I need to get the full 15k+ body text written to DB.... Any ideas on what this problem is? I could not find a reference when looking through forum...
Comments
node_save and truncation of $node->body...
Okay - I found the problem. The source files that I am using to load the content ($node->body) had used left/right curly quotes, along with various other special characters within the text. I used str_replace() on the data to replace with standard single or double quotes and like wise with the other special characters.
All works perfectly now. So this is a heads up for anyone that is loading data programmatically - make sure to address the special characters that may be in the text data, otherwise SQL will truncate upon the first encounter.
I could not find a Drupal based text data filter, although I am sure one should exist. I would have thought that node_save() would have done that automatically.... (that is scan the body for both html tag filters and special - problem causing characters) If there is a way to do this with standard Drupal, please post details.
Subscribing. This is the same
Subscribing. This is the same issue as Inserting node with UNICODE characters.