Can't import date values into Date module fields

Subscribe.

Subscribing. date just went to 6.x-2.0 so its looking reasonably stable now...

Could you try date.inc from CVS? Copy the file into sites/all/modules/node_import/supported/date/date.inc (creating a new directory).

If you could give me feedback on how it works or when it doesn't I'll be able to fix those and include CCK Date into -rc5.

Thanks.

I tried to import the attached file. In Set import options (step 5 of 8), I chose "Custom Format" and "m/d/Y". On the preview screen, I got "An illegal choice has been detected. Please contact the site administrator." for each previewed record, as well as a var_dump of the values array. When I remove the date from the fields to be imported, the import proceeds correctly.

In the var_dump(), I noticed that field_date is set to be

Thanks for your help!

Here's some code for date.inc that seems to work in the case of date fields of type 'date' and of widget type 'date_select':

When I added this code to date.inc, the imported worked without complaining. Note that the date_convert() function takes a timezone argument that I left as default for the time being. I'll be working on other widget types and other date types.

Attached is a new version of date.inc that I verified to work with all CCK Date types, widget types, and also with 'To date' enabled. Whoever is interested please test it and send your feedback here. Thanks!

Nice work. After some reviewing I'll include it in -rc5.

A question:

I tried to set output_format directly to DATE_ARRAY and avoid the extra conversion but got the following behaviour:
* warning that mb_strlen (inside drupal_strlen) is passed an array on line 602 of node_import.inc. I commented that line just to let the processing continue.
* the DATE_ARRAY values end up in values[cck:field_date:value] but *not* in values[field_date] which is what is desired. I did not further debug the issue.

Also, I made the following change to case 'value2': in date_node_import_fields():

I've been testing out the ability to import Date fields with Node Import and been successful using the date.inc that was posted in #6 above.

However, if the date field for my content type has the repeat options enabled, I get a white screen with "Fatal error: Cannot unset string offsets". This error points to /sites/all/modules/date/date_repeat/date_repeat_form.inc on line 212.

When I am setting up the import, on step for for my field, called "Event date", there is an entry for "Unsupported: Event date -rrule". I assume that this means that importing repeat options is currently unsupported in the Node Import module.

If I set up my date field without repeat option, I am able to successfully import a CSV of dated nodes into my installation.

This has been reported over on the Date modules issue queue as http://drupal.org/node/379102 by someone else, but I thought I would mention this over here, too.

I hadn't tested the repeat option :-) I'll try it and report my progress here. Thanks for testing!

I'm getting the following error

This date field and it's corresponding To date is required. If I make them both optional the message about 'Some value myst be entered in the To date.' goes away but the message about To needing to be greater than From and the foreach warning continue. Any ideas?

I have tried both the 2.0 and the dev versions of date with no difference.

@presleyd: Did you specify multiple values for your date field? When I did so in mine it gave me those errors.

Started over with reloading the csv file and recreated the Date field and that seemed to do it. I don't think that field ever had multiple values defined though so not sure what was causing the problem. I did switch from the JS popup tool to just a simple select widget though, maybe related?

@presleyd: I had tested against the JS popup date widget so it's less likely to be the cause. I'm still working on the multiple values issue.

@jmbeach: the date repeat feature uses a multiple values date field, which currently doesn't work. So I'll fix that first.

@kratib: It would be nice if it could just drop a NULL into the rrule field for now. My date field has the repeat options enabled for events that I hand-enter, but the datasets I'm trying to import are strictly single occurrence dates.

Tested this (no multiple or repeating dates though), and it worked great for me.

@jmbeach: Even if your dates are single-valued when you import them, the fact that they have the repeat option turned on makes them automatically multi-valued internally, and hence subject to the problem of multi-valued dates in general. I did try to set the rrule field to NULL but got the same erroneous behaviour.

I've tried to debug through the date module to find the source of this error but I was unable to find it so far. I won't be able to put more time into this for now.

The state so far is that the latest date.inc file contained in this thread seems to work for single-valued date fields, but not for multiple values nor for repeat dates. Sorry everyone!

#6 has been committed to CVS (together with small fix in #8).

Patch needs more work.

I'm just trying out the latest date.inc

It looks like it's working well for me but i notice it's not automatically selecting the field in my CSV on step 4

my CSV field is called "DATE OF ISSUE", and my date field is called "Date of Issue/date_of_issue" but it's not automatically selected on step 4 of the wizard like title and other fields i have (simple text).

Great module though :)

@kratib

I have to confirm my problems sure seem related to the date popup widget, I get this error whenever that is enabled and never with select list ceteris paribus.

I can confirm this causal relation too.

Just for the import I changed the widget to 'select list', then switched back after import.

Thanks everyone for this report. I was able to reproduce this behaviour. I'll look at it during this coming week.

Great Work!

Comment #6 patch worked without any issues for me!

I am using the latest dev version of Node Import and the latest dev version of CCK Date and I am not able to get the date import working. Keep getting an error that "The dates are invalid".

@undoIT -

I had the same issue last night - took me a while to get to work - but my problem was the "select list" issue reported in #21 and #22. The import only worked for me when I had the date widget in my CCK content type set up as a "select list". It steadfastly refuses to work when you have it set to be a text field.

The other thing is to make sure that your import format matches the format of the data in your CSV file. You might need to select a "custom format" in one of the final steps of the import process, and then set that to match your CSV data.

Thanks, changing the date widget to Select List did the trick. Also, had to change taxonomy vocab to Tags to get term import working. No more errors :)

ive tried this as listed above but still a few days of head-scratching , is the recommend type "date" or "datestamp" ?
what have others that got this to work doing for timezone ? i cant tell if that is my problem. ive tried many combinations

i keep getting this error "warning: date_timezone_set() expects parameter 1 to be DateTime, null given in [...]/drupal_6_10/sites/all/modules/date/date/date_elements.inc on line 229."

ive tried CSV with these options,

"2005-11-16 12:00:00";
"2005-11-16";

maybe i need to try 12/12/2009 like others list here

to #6, kratib
Thank you for your help.

but there is a waring like this.
warning: date_offset_get() expects parameter 1 to be DateTime, null given in C:\xampp\htdocs\sites\all\modules\date\date\date_elements.inc on line 500.

so I have to make a small modify in the file(/sites/all/modules/date/date/date_elements.inc).

line 467
// add by zw111aaa start
if (isset($element[$from_field]['#value']) && !is_array($element[$from_field]['#value'])) {
$element[$from_field]['#value'] = date($element[$from_field]['#date_format'],strtotime($element[$from_field]['#value']));
$element[$from_field]['#value'] = array('date' => $element[$from_field]['#value']);
}
// add by zw111aaa end

I think it is not a good way,it is just no warning.
If you have a good way to modify this warning,please tell me.

Thank you very much, kratib.

I too have the problem of date repeat making the import not work. When I change the datetime field type to a select list, the dates don't get entered at all and the forms do not work so doesn't seem to be an option.

Comment #6 patch worked without any issues for me too :)

Kudos for this module, the ability to import Dates is a real plus !

Same here as #28

Hav' tested a lot, but no clue, also with the latest snaps of node_import and date. Sadly, i am not able to get in the drupal code yet.

Help really appreciated...

Same problem for me... using dev version of today of node_import.

I've attached the screenshots...

The date field is a text field... What to do to get it working ?

Thanks !

@pol:

try to switch to "select list" in date field properties like #27 and do it again

I'm working with the most up-to-date dev releases of both Date and Node_Import and I can't get date values to import at all. Here is my problem description:

1. I create a CSV file where each row looks like this:

2. My Date field is a date (not datestamp) "select list" with a prefab format that looks like "2009-04-23 18:22". The to-date is optional.

3. I'm not using repeating dates or popup widgets or anything. It's just a straight up, normal field containing a Date_module value.

4. I start the node import process. I upload the file and map the values into the Title field and the Date field. After mapping I am presented with the date format choices in step 5 of the import process and I choose the date format that matches the both the values I suggested above ( which is, again, "2009-04-23 18:00"). I set my timezone for New York.

5. When I get to step 7, I get an error, "An illegal choice has been detected. Please contact the site administrator." I can't figure out why I'm getting this. The scheme dump beneath the error messages says:

6. When I do not include the date field value in the import then everything goes smoothly. When I attempt to fill in the to-date value with something, I get the "Illegal Choice" error twice.

7. I take a chance and try to import the date values as a custom format in step 5, by selecting "custom format" and entering "Y-m-d h:i" and I get the same results ("Illegal Choice").

Hopefully this is enough of a write up that someone can figure out what am I doing wrong? Is it me or is there a bug out there that I can't see? Thanks for the help.

I'm unfortunately failing heinously with this, considering just entering the 200+ records by hand.

Seems like this should be simple.

I'm using the latest DEV. I replaced my /sites/all/modules/node_import/supported/date/date.inc with the one up on #6 (I assume that's the right place).

Got my CSV file, my dates are formatted as mm/dd/yyyy (but not necessarily leading 0s), so I selected a 'custom' form in step 5 and used 'n/j/Y' as my custom format, and by the time I get to step 7, I get:

Thoughts? Ideas? Suggestions?

Edit: Nevermind, didn't read everything above, I'm seeing something that might be my issue.

Update 2: Yeah, that was it, the popup date widget. Hopefully this gets logged as an issue to still resolve. I would like to be able to enable this so some power users can upload data, but if the only solution for dates is to let them go change the date picker from the popup on the content type before/after the import, that's not a very good solution. I could leave the date input as the select, but in all the years I've been doing this, 'everyone' wants the cute date picker and throws a fit when they see a date field without it, and I don't blame them.

Refocusing this issue with updated reports on the bug at hand.

Importing records into nodes that have dates tend to fail because of several reasons, including both the date field widget setting and the formatting of the date value itself.

The solution might be to talk to KarenS about how to make her Date module more accessible to Node_import.

Better name to suit this issue, too.

subscribe

I don't have time to do much on this, but wanted to post a suggestion. I'm not sure how node_import works, but I assume from the problems you're having you are using drupal_execute() somewhere. I always recommend using node_save() instead of drupal_execute() because drupal_execute() is horribly fragile and breakable (and there are known bugs). Even Jeff Eaton (one of its creators) suggests avoiding it. I did a whole long thread on all the problems it creates when trying to fix brokenness in Content Copy (which was using drupal_execute()).

If you use drupal_execute() you have to format the data in ways that are very specific to the widget the field is using, and even then you'll have all kinds of problems because drupal_execute() validation fails if you try to do it more than once in the same session (it's a known core bug that has a looonnnggg thread) and anything that should happen during validation will fail. And the date module does work to re-factor the widget values into the right format for the field during validation, so things will break if that doesn't get triggered.

If you use node_save() you don't need to know anything about the widgets, you just import data in the format the field expects -- YYYY-MM-DD HH:MM:SS for datetime, YYYY-MM-DDTHH:MM:SS for ISO and 9999999999 for Unix. Super super simple.

The only reason for using drupal_execute is for validation, and the validation doesn't work anyway.

Using the patch in #6 with the latest -dev for both the Node Import and Date modules and this got me most of the way there but two problems still occur:

I get the following error in step #7: "The date is invalid" (triggered by the check in line ~480 in date_elements.inc in the date module). My data is formatted thusly: 20090510 for which I set a custom format of "Ymd". The field is set to not have a "To Date" and allow blanks.

If I comment out that error I can thru the import but no date is actually saved into the new node and no further errors are generated. The previews all look correct and nothing obvious is wrong at that point but nothing is stored.

subscribe

subscribe

I was running into a problem importing dates with times like 14:00. The date module (6.x-2.1) was barfing when asked to use minutes with a value of zero: the hours minutes and seconds are not properly zero-padded. The attached patch takes care of this.

I've been struggling with this a lot today, trying to import around 15,000 (!) relational records from different tables exported to CSV files. The first issue I had was listed above, as something along the lines of "date_offset_get() expects parameter 1 to be DateTime, null given in /var/www/sites/default/modules/custom/date/date/date_elements.inc on line 500.". Setting the CCK field to a select list, as has been mentioned elsewhere in this bug report, fixed that issue.

The next issue was I kept getting the "An illegal choice has been detected. Please contact the site administrator." and a var_export of the node objects on the preview for the dates, on some, but not all of my dates. I finally figured out what was causing that - some of my records go back as far as 2001, and the CCK date field defaults to "-3/+3" as the years back and forward to select from. So setting the min/max years for the records to -10/+10 on the CCK date field fixed that issue.

I'm still in the process of importing several hundred event records from 2001 through now (where I got stuck on the dates). After that, I have about 13,000 image nodes to import that will combine into galleries for each event that will be glued together with Taxonomy that will have to be hand built using a SQL query and the event ID imported to a temporary CCK field. But anyhow this particular bug report has been my saving grace trying to get everything imported to the system thus far. Thanks,

-Rich

Scratch that. It was 33,410 image nodes that needed importing. And since node import doesn't support file or image attachments to nodes *but* it does support CCK files and images, I came up with a work around.

Attached is my import, 14 hours in. I wrote up in detail how to import image attachments for nodes (and presumably file attachments for nodes should work as well) using a small bit of SQL and CCK fields as an intermediary here: http://drupal.org/node/483248

Just want to confirm that the 2009-04-22 dev version of node-import still doesn't handle date fields with "Text field with custom input format". You MUST use "Select List" as your date widget. Otherwise, you will get a "The dates are invalid" on the first date it finds.

Note also that the "The dates are invalid" message only shows once (on the first date it tries to import), even though you may have multiple problematic date fields using the text widget.

It would be nice if this problem was at least noted in the support docs....

(Canceling this comment. Missed the fact that you have to use the latest -dev release, not RC4.)

(Restoring priority to how it was before I messed with this issue.)

Look at #444884: Integration with nodereference fields only works if nodereference field set to "select" widget. The solution is the same: use a select field. Could this problem be unrelated to the Date module?

Subscribing.

subscribing

subscribe...

Can you guys try replacing node_import/supported/date/date.inc with the attached file and importing with your date pop-up fields? I added the different cases for the different kinds of widgets -- the Date module actually has at least 3 different ways of arranging the field value depending on the type of widget and chosen format, but I think I have a nice generic solution for all three.

Attached is an even more improved version which corrects an issue with AM/PM being in the wrong case, and the default values not working when the value being imported was empty (used to default to January 1, 2009).

I hope it helps!

This file worked well, thanks very much. I had to add a section handling timezones, attaching the file with this change.

[EDIT] one thing that this file doesn't do very well is adapt to the chosen input format. It essentially depends on dates being entered as "[date] - [time]", as it looks for space-dash-space to split the time and date portion.

subscribing

@robomalo could you try the attached file, and report if it worked for you? If not, then add what your date field configuration and widget was. It would help the maintainer to know if the file works. Thanks.

The file doesn't seem to be working properly for me.

The following widgets had the following errors:

select: "An illegal choice has been detected. Please contact the site administrator."
text field: the right month and days, but the wrong year
text field + popup: the right month and days, but the wrong year

I tried Y-m-d H:i and a custom format Y-m-d. Both fail. I made sure my CCK field input format matched the import settings in each attempt.

I also tried it with the "-" between the date and time...

I did just discovered it breaks my dates before 1900. strtotime() doesn't support this and is used by this .inc. Correct me if I am wrong, but I _think_ Date 2 supports older dates. Therefore, I think this file will have to be altered to support Date 2's functionality.

In the supplied date.inc I changed line 113 from

$values[$fieldname][$i][$colname]['date']  = date($widgetFormat, $time);

to

$values[$fieldname][$i][$colname]['date']  = $value[$colname];

and it worked for me. I know this won't work for everyone and only applies to a small group of people. This just shows
that the line

$time        = strtotime($value[$colname]);

is the source of my errors. I had to make sure my CSV date Y-m-d matched the input format of my content type Y-m-d.

I am in a similar situation to robomalo in that I have historical dates I'm dealing with. I used the code in #56 with the hotfix in #61, but at step 7 I am getting errors of "A valid date is required for Dates of existence 2 From date. Dates of existence 2 To date is invalid." I tried to import with date fields of type Datetime and Date.

An excerpt of the output of the record preview:

I am trying to figure out what part of node_import_check_date() is failing, but am unsure how to get debug messages injected in the output. The csv import only had date info (Y-m-d). Any suggestions what might be happening?

I ran into this a couple of days ago - what you need to do is configure the CCK date field you are trying to import the date to, and change the granularity to Year, Month, and Day.

The field on the CSV is something like "2007-01-29" (Y-d-m) and the granularity of the cck field is Year, month and day but it still fails.
I still have the "illegal choice" error message.

any suggestion?

I did a double-check and the granularity was already set to year, month, and day.

My field has an optional to-date.

hotspoons, did this really work for you? Not for me. Using the latest date.module, node-import module + advanced help.
I've reset my dates to granularity to year month day and apparently you can't select a timezone. So no timezone. Tried to import but that still failed.
If you have further info that I'm not using rite now, please let us know.
Thanx.

ok i think i finally figured out what's going on, but am unsure about where the code needs to be altered.

after a lot of dsm()... it looks like what was going on was my data in the csv was formatted as "Y-m-d" but the field itself has a required format of "m/d/Y".

i was able to determine this by looking at date_text_input_value() in date_api_elements.inc. there's a line that says:

$value = date_limit_value(date_convert_from_custom($input, $element['#date_format']), $granularity);

the #date_format attribute is set to the node's format, not the csv file's specified format.

i am going to re-format those fields in my csv and try re-importing again and see if that fixes the issue.

#55 worked for me: date set to jquery popup calendar, 8/1/2009, m/d/Y custom format on import, no timezone conversion.

i cant get any situation to work...my csv has format of 08/2009..ive tried all scenarios and patches...illegal choice!..added timezone/removed to no avail.

can some kind soul help here..ive been up for 15 hrs on this!

Could you not use a normal dd/mm/yyyy format, and after importing, if you don't need the day of month, remove it from granularity and/or display formats?

thing is, ive already done that!

Importing csv formatted as yyyy/mm/dd into a datestamp field that uses minical popup widget worked for me using the patch in #56. Granularity down to the minute, and TZ handling set to 'Date's timezone'. Used the latest DateAPI and latest -dev version of node_import.

is a datestamo field different from a regular date field?..just making sure

It's stored in the database differently, but to the user there's no difference IIRC. Please check date api docs, though.

did exact same thing, tz on field set to date, tx on import set to newyork...used datestamp as well

i get this:
There are errors in Article Pub Date:

* The dates are invalid.
thanks for helping with this..i hope i resolve it !

r

not sure if this will help, but the out put is like this in the error:

[field_pubdate] => Array
(
[0] => Array
(
[value] => 2009-06-01T00:00:00
[timezone] =>
[offset] =>
)

)

could this be part of the issue?

[value] => 2009-06-01T00:00:00

ok this is horrible....ive just spent 15 hrs triyng to import a date field that should work out of the box with a bare bones drupal install...latest versions of node import and date....and ive developed over 20 drupal sites. can anyone help? i need to launch another site tonight...see my prev. posts...ive literally tried everything...even formatting my csv like this: 2009-06-01T00:00:00

all i get is The dates are invalid.

ok i have a temp fix for anyone with this issue: use date 6.x-2.0

it worked no problem with patch #56

r

I hate to sound like another "me too" but...

I'm running Acquia Drupal on a Windows local dev box (hmmmm could this be it?)
I attached the "a Node import debug report"
I have the date.inc file from #56 put into l\sites\all\modules\node_import\supported\date
The node cck field is: Date, select list, custom imput format: n/j/Y, to date: never, granularity: year month day

I get these errors on step 7
# warning: mb_strlen() expects parameter 1 to be string, array given in C:\Documents and Settings\guru\Sites\acquia-drupal-site\acquia-drupal\includes\unicode.inc on line 404.
# An illegal choice has been detected. Please contact the site administrator.

And of course no nodes import 8^(

Any thoughts?

ADDITIONAL INFO:

When I remove the date from the import, everything imports fine, but the date field in question gets populated with 1/1/2009 (must be a default value of sorts)

ADDITIONAL ADDITIONAL INFO:

I've reworked my csv import so the values are in yyyy-mm-dd format, and the step 7 import says

"There are errors in Purchase Date:

* The dates are invalid."

and the step 7 values contains the following :

[cck:field_purchase_date:value] => Array
(
[0] => 2008-03-18
)

I'm just not getting it...my csv has yyyy-mm-dd format, my CCK date field has Y-m-d format, my Set import options (step 5 of 8) has Y-m-d format (and custom format selected)

-James

Hotspoons, I just wanted to say thanks for the -/+ tip in 45 - that was the one piece I needed to get date import to work for me.

Thanks!

The only way I could get Dates to import using 6.x-1.x-Dev (22 Apr 09) was to set in CCK the Date format via 'Select List' (#47). All others failed.
Not correct. Select List (from Basic Information) is only the first step...
2. Had to ensure that Repeat Rules was disable in CCK content type
3. Then took out default values from CCK field
4. Then Customized Date Parts: For each element of the date/time, I selected the Text Field radio button(as opposed to Select List)

An after 2 days' effort, it looks like 500+ calendar entries are actually importing. Yay!

Now have to remember to put the CCK date fields in my content type back to something user friendly.

PS. Apologies to any email subscribers for re-editing this twice.

I've tried with patch 56 it's work for the other widget, very thanks.

But you can't have the "to date", the problem is here. not in date field but in date field with to date.

I managed to import by doing the following:
1) Change widget to select list
2) Change date range to +/- 10 years
3) Convert my dates to unix timestamps

Only 4 hours later...

The patch in #56 works for me. I'm using date pop-ups as input and required To date.
I hope this module will make it to the next version because it really looks promising. Keep up the good work.

See http://drupal.org/node/477392#comment-2111134, it may be related to some of the problems discussed here.

So close I can taste it:
using Date 6.x-2.4, Node Import RC4 with date.inc from #55.
Date field using jquery popup; Default value is "now"; Default value for To date is "blank";
Input format: MM/DD/YYYY; Granularity: Year Month Day

Happy to report everything works great if both From and To dates are supplied. But if the to date is blank, it fills in 12/31/1969. Now the To date is before the From date, and it throws the simple error "The To date must be greater than the From date."

It's important in this context to be able to import a blank To date - not this odd default 1969 date. Can anyone point me in the right direction?

Thanks!

Figured it out... sweet. This helped me:
http://us.php.net/manual/en/function.strtotime.php#60579

The default values that are entered if there is no value get skipped if there is a From date but no To date - plus you still want it to evaluate the From date, so I re-checked for a FALSE result to $time. Weird that date(FALSE) returns "12/31/1969", right? How is that useful?

Thanks GuyPaddok and AlexK for the work on this. Is this planned for inclusion in RC5?

I'm adding my piece to this puzzle. My issue was that the code in date.inc wasn't taking into account custom date definitions. The attached file does.

#89 works for me with node_import rc4 and dates in format 18/10/2009 - 00:00
Thanks!

I'm having big problems importing dates as well. Not sure what I'm doing wrong.

I added the patch from #89 to the Oct 10 dev version. Made sure none of the date fields in the content type are required. Set them all to the Select List widget, +10:-10, hour and minute granularity. It's still not working. I keep getting "The dates are invalid." on step 7 of the import.

One of the fields is part of a content multigroup from CCK3, but I still get the error even when I don't import that field. The other 4 date fields in that content type are single-value fields ("from" date, but no "to" date). In the .csv I'm importing, dates look like this: "08/16/2006 - 00:00", and I'm using the matching date format in step 5 (the default format).

Dates all display like this on the output from step 7:

[cck:field_date_sent:value] => Array
(
[0] => 2006-08-16T00:00:00
)

I've been wrestling with this thing for a couple days now, and at this point, I'm completely lost. Can anyone suggest something else that I can try? Any ideas would be welcome!

The date.txt from #89 does not fold in the fix regarding historical dates. strtotime() returns FALSE for me.

I need to fold back in the fix by replacing this line (108 in the attachment in #89):

to

Subscribe

Subscribing. Edited prior issue out.

subscribing

Removing improper assignment from #24.