Problem/Motivation

All core schema columns using integers for storing UNIX timestamp use 32bit for the value and so are affected by the Year 2038 bug.

Proposed resolution

Move schema columns using type = 'int' as Unix timestamp storage to size = "big" in order to use 64bit and so support dates until 15:30:08 UTC on Sunday, 4 December 292,277,026,596.

Remaining tasks

User interface changes

No.

API changes

No.

Data model changes

Size changes from normal to big on some affected schema columns.

Release notes snippet

TBC.

---- original issue ---
I tried entering the date "2100-08-02 17:35 -0500" into the authored on field, and Drupal said it was invalid. Obviously it's a valid date. It's only a hundred years into the future. So if Drupal wants to last for the next century, I suggest this gets fixed.

https://dev.mysql.com/doc/refman/5.7/en/integer-types.html

BIGINT 8

https://www.postgresql.org/docs/10/datatype-numeric.html#DATATYPE-INT

bigint 8 bytes large-range integer -9223372036854775808 to +9223372036854775807

https://www.sqlite.org/datatype3.html

INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.

Comments

stevenpatz’s picture

I attempted to use that date in one of my submissions and Drupal resets it to a date in 2038. See:

2038

killes@www.drop.org’s picture

Status: Active » Postponed

Come back about 2036 and I guess we'll have resolved it by then.

lilou’s picture

Status: Active » Postponed

Maybe the date validation message should be "You have to specify a date prior than January 19, 2038." instead of "You have to specify a valid date."

lilou’s picture

Title: Dates to far into the future are invalid » Y2K38 : Unix Millennium bug
Version: 4.6.7 » 7.x-dev
Status: Postponed » Active
alexanderpas’s picture

Status: Postponed » Active

may I point everyone towards:

ISO8601: Numeric representation of Dates and Time

Wikipedia: http://en.wikipedia.org/wiki/ISO_8601
Summary from ISO: http://www.iso.org/iso/support/faqs/faqs_widely_used_standards/widely_us...
Full document (zip-pdf, 228KB): http://isotc.iso.org/livelink/livelink/4021199/ISO_8601_2004_E.zip?func=...

how about a varchar field that simply stores the ISO date and/or time

webchick’s picture

varchars are horribly slow to sort, find times in between, and so on. There's a great discussion on the pros/cons of UNIX timestamps, datetime fields, and ISO8601 on the Drupal Events group @ http://groups.drupal.org/node/731.

casey’s picture

Version: 7.x-dev » 8.x-dev
LarsKramer’s picture

I suggest either marking this as a duplicate of the issue mentioned by lilou in #4: #154477: Expand date handling capabilities in core (Item 3 in its summary covers expanded date ranges). Or we might narrow down the scope of this issue to provide a more explicative error message (see comment #3).

BTW, this problem was fixed in Date contributed module as of version 2.x. Could a similiar approach be applied to Drupal core's node creation date?

On my shared hosting environment, with Drupal 7.7 I got this nasty red error message when trying to set the node creation year to 2039:

PDOException: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'created' at row 1: UPDATE {node} SET vid=:db_update_placeholder_0, type=:db_update_placeholder_1, language=:db_update_placeholder_2, title=:db_update_placeholder_3, uid=:db_update_placeholder_4, status=:db_update_placeholder_5, created=:db_update_placeholder_6, changed=:db_update_placeholder_7, comment=:db_update_placeholder_8, promote=:db_update_placeholder_9, sticky=:db_update_placeholder_10, tnid=:db_update_placeholder_11, translate=:db_update_placeholder_12 WHERE (nid = :db_condition_placeholder_0) ; Array ( [:db_update_placeholder_0] => 287 [:db_update_placeholder_1] => article [:db_update_placeholder_2] => und [:db_update_placeholder_3] => Causa Iaceo Nimis [:db_update_placeholder_4] => 22 [:db_update_placeholder_5] => 1 [:db_update_placeholder_6] => 2575319906 [:db_update_placeholder_7] => 1313725250 [:db_update_placeholder_8] => 1 [:db_update_placeholder_9] => 1 [:db_update_placeholder_10] => 0 [:db_update_placeholder_11] => 0 [:db_update_placeholder_12] => 0 [:db_condition_placeholder_0] => 287 ) in drupal_write_record() (line 6861 of /drupal_root/includes/common.inc).

However, I was not able to reproduce on my local setup where I just got the standard message: "You have to specify a valid date."

swentel’s picture

valthebald’s picture

Version: 8.x-dev » 9.x-dev
Status: Active » Fixed

Rough calculation shows this issue should belong to Drupal 15.x or so, bumping to 9.x meanwhile

cweagans’s picture

Status: Fixed » Postponed

This might be the more proper status.

valthebald’s picture

#11: postponed was my intent, actually. Missed the option

joachim’s picture

Version: 9.x-dev » 8.x-dev
Status: Postponed » Active

I think we should actually fix this now -- show a validation error rather than crash.

It seems like a silly thing to have to fix, but when people test sites they input silly dates. These people include clients, who then see Drupal crash and are unimpressed.

acrosman’s picture

Issue summary: View changes

Just retested this issue with the current 8.0.x-dev, and it provides a validation error on submit if using a date after 2050-12-31, but any date between 2037-12-31 though 2049-12-31 still crashes. It fails to handle a database error.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

barami’s picture

Actually, This problem on php is not a problem.
But drupal 7 & 8 depends on integer type database column for this operations.

Many created properties use datatype of 'normal int' on db schema.
And it indicates 'int' type database (It depends on data type definition of database handlers)

As you know, MySQL's normal integer limit has 32bit limit.
https://dev.mysql.com/doc/refman/5.7/en/integer-types.html

There are two solutions.
1. Convert all properties about datetime from int to datetime type (eg. DATETIME on MySQL)
2. Convert all properties about datetime from int to big int.

Actually, PHP's timestamp is 64bit value on 64bit systems. It solves Y2K38 problem.
Almost drupal systems already running on 64bit systems. But Drupal don't use 64bit integer value for timestamp on database until now.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

markhalliwell’s picture

Title: Y2K38 : Unix Millennium bug » Y2K38: Unix Millennium bug
Version: 8.6.x-dev » 8.8.x-dev
Priority: Normal » Major
Related issues: +#1003692: PDOException: SQLSTATE[22003] after entering large value in integer and decimal field

Running into this issue with the expire field on key_value_expire, which system_schema() still defines as:

      'expire' => [
        'description' => 'The time since Unix epoch in seconds when this item expires. Defaults to the maximum possible time.',
        'type' => 'int',
        'not null' => TRUE,
        'default' => 2147483647,
      ],
andypost’s picture

Issue summary: View changes

This issue needs split for validator error, also added child issue #2885413: Timestamp field items are affected by 2038 bug

gambry’s picture

Better to start organising the fix for this potential issue (which is not potential any more in some cases, see #22 for example, or #2885413: Timestamp field items are affected by 2038 bug).

I scanned core for all instances of 'type' => 'int', and considered only when the schema fields are used as unix timestamps. See the output list below:

File (schema.)field name signed/unsigned line number notes
core/lib/Drupal/Core/Batch/BatchStorage.php timestamp signed (L226)
core/lib/Drupal/Core/Cache/DatabaseBackend.php expire signed (L488)
core/lib/Drupal/Core/Field/Plugin/Field/FieldType/TimestampItem.php value signed (L50)
core/lib/Drupal/Core/Flood/DatabaseBackend.php timestamp signed (L216)
core/lib/Drupal/Core/Flood/DatabaseBackend.php expiration signed (L220)
core/lib/Drupal/Core/Queue/DatabaseQueue.php expire signed (L307)
core/lib/Drupal/Core/Queue/DatabaseQueue.php created signed (L313)
core/modules/comment/comment.install comment_entity_statistics.last_comment_timestamp signed (L98)
core/modules/dblog/dblog.install watchdog.timestamp signed (L76)
core/modules/forum/forum.install forum_index.created unsigned (L136)
core/modules/forum/forum.install forum_index.last_comment_timestamp signed (L143)
core/modules/history/history.install history.timestamp signed (L30)
core/modules/locale/locale.install locale_file.timestamp signed (L224)
core/modules/migrate/src/Plugin/migrate/id_map/Sql.php last_imported unsigned (L362)
core/modules/migrate/tests/src/Unit/MigrateSqlIdMapEnsureTablesTest.php last_imported unsigned (L62)
core/modules/statistics/statistics.install node_counter.timestamp unsigned (L47)
core/modules/system/system.install key_value_expire.expire signed (L1115) funny this defaults to the max time 2147483647 (2038-01-19)
core/modules/system/system.install sessions.timestamp signed (L1164)
core/modules/taxonomy/src/TermStorageSchema.php taxonomy_index.created signed (L57)
core/modules/tracker/tracker.install tracker_node.changed signed (L49)
core/modules/tracker/tracker.install tracker_user.changed signed (L93)
core/modules/views/src/Tests/ViewTestData.php views_test_data.created unsigned (L94) dates are manually set, but worth moving this to bigint too in case we need to set new date > 2106 (as unsigned)?
core/modules/views/tests/src/Kernel/Handler/FieldDateTest.php views_test_data.destroyed unsigned (L27) this is a funny one. Set 2050 as date year, but works becuase field is unsigned. Same assumption as above though?
core/modules/migrate/tests/src/Kernel/HighWaterTest.php changed unsigned L38 name can mislead BUT THIS IS NOT USED as timestamp. Test set only int values < 10.
  • I've included when the field is unsigned. Although unsigned will last until 7 Feb 2106, we all hope Drupal will live even longer and so these need to be updated too
  • I've excluded all the fixtures of course.

The question is: do we do all in one go, or we split each component with its own issue?

gambry’s picture

The question is: do we do all in one go, or we split each component with its own issue?

After sharing this on Slack, the consensus seems to be with a mixed of both: groupping similar issues (i.e. same module, same sub-component like Cache backends), but generally having separated issues to tackle the upgrades to big int.

I'll update the IS with the long list of upgrade.

However just came up in mind we still do support PHP 32bit, as we just raise a warning. As part of this parent issue we should also validate if dealing with big_int in a 32bit PHP environment is possible.
I'll add this task when updating the IS.

pedroraf’s picture

This is an issue that should have been fixed as soon as it was brought up.
A field called expiration date seems common place for multiple areas, and in fact the date can be much higher than 2038.
Can someone tell me if a fix for D7 is considered? We are still not moving to D8... unfortunately.

stevenpatz’s picture

Reminder that patches are welcome, and that people are volunteering their time.

pasqualle’s picture

13 years old ticket with 20 comments, should just be closed as outdated. The issue reporter does not even use Drupal any more.

I hope my comment will help move things forward now.

gambry’s picture

Title: Y2K38: Unix Millennium bug » [meta] Y2K38: Unix Millennium bug
Issue summary: View changes

Updating IS reflecting the plan to tackle this issue (first draft)

gambry’s picture

Issue summary: View changes

IS updates. Adding first issue.

gambry’s picture

I hope my comment will help move things forward now.

LOL! I'm not sure how exactly can help? :D

The list of components requiring updates is up in the issue summary. Either pick and help on an existing issue or create a new one against one or more files, if they can be grouped.

Also this issue is against Drupal 8. If anyone want to work - or support/fund the work - against Drupal 7 feel free to create a sister issue targeting Drupal 7 and add each other as related issues so we can synchronise work and share solutions.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

catch’s picture

Component: node system » base system
Priority: Major » Critical
Issue tags: +Needs issue summary update

Bumping this to critical. This would likely end up being an actual release blocker for Drupal 16 or something, but the earlier we fix it the better - although we need to bear in mind #3108658: Handling update path divergence between 11.x and 10.x (which might mean scheduling this for when 9.2.x is opened).

andypost’s picture

Just faced today that PHP (64bits) itself has issues handling time64 which added in latest linux kernel. Basically it using time_t in wrong way few places.
Will provide more details when it become more clear how to fix https://bts.adelielinux.org/show_bug.cgi?id=266#c2

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Issue summary: View changes

Removed duplicate issues from the IS, the full list is available in #24.

What is needed for the IS update requested in #34?

Added an issue for tracker.

quietone’s picture

Issue summary: View changes
quietone’s picture

Issue summary: View changes
quietone’s picture

StatusFileSize
new5.51 KB

I created a patch for Batch, Cache and Queue and uploading it here because it is late and I am not up to making a new issue.

The only table left to do is Flood.

andypost’s picture

@quietone probably instead of using direct query in update hook it should use connection->schema

quietone’s picture

Issue summary: View changes

Per this suggestion by alexpott this is being done in two child issues.

quietone’s picture

Issue summary: View changes

Formatting change.

quietone’s picture

Issue tags: +Bug Smash Initiative
quietone’s picture

Category: Bug report » Plan
Issue summary: View changes
Issue tags: -Needs issue summary update

This is really a plan, changing category. The child issues are bug reports.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.