Problem/Motivation
All core schema columns using integers for storing UNIX timestamp use 32bit for the value and so are affected by the Year 2038 bug.
Proposed resolution
Move schema columns using type = 'int' as Unix timestamp storage to size = "big" in order to use 64bit and so support dates until 15:30:08 UTC on Sunday, 4 December 292,277,026,596.
Remaining tasks
- Validate if we can still support PHP 32bit, or if we need to drop it. We are currently only warning users.
- Complete the child issues.
User interface changes
No.
API changes
No.
Data model changes
Size changes from normal to big on some affected schema columns.
Release notes snippet
TBC.
---- original issue ---
I tried entering the date "2100-08-02 17:35 -0500" into the authored on field, and Drupal said it was invalid. Obviously it's a valid date. It's only a hundred years into the future. So if Drupal wants to last for the next century, I suggest this gets fixed.
https://dev.mysql.com/doc/refman/5.7/en/integer-types.html
BIGINT 8
https://www.postgresql.org/docs/10/datatype-numeric.html#DATATYPE-INT
bigint 8 bytes large-range integer -9223372036854775808 to +9223372036854775807
https://www.sqlite.org/datatype3.html
INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.
| Comment | File | Size | Author |
|---|---|---|---|
| #41 | y2038-batch-cache-queue-do-not-test.patch | 5.51 KB | quietone |
Comments
Comment #1
stevenpatzI attempted to use that date in one of my submissions and Drupal resets it to a date in 2038. See:
2038
Comment #2
killes@www.drop.org commentedCome back about 2036 and I guess we'll have resolved it by then.
Comment #3
lilou commentedMaybe the date validation message should be "You have to specify a date prior than January 19, 2038." instead of "You have to specify a valid date."
Comment #4
lilou commentedSee also : #154477: Expand date handling capabilities in core
Comment #5
alexanderpas commentedmay I point everyone towards:
ISO8601: Numeric representation of Dates and Time
Wikipedia: http://en.wikipedia.org/wiki/ISO_8601
Summary from ISO: http://www.iso.org/iso/support/faqs/faqs_widely_used_standards/widely_us...
Full document (zip-pdf, 228KB): http://isotc.iso.org/livelink/livelink/4021199/ISO_8601_2004_E.zip?func=...
how about a varchar field that simply stores the ISO date and/or time
Comment #6
webchickvarchars are horribly slow to sort, find times in between, and so on. There's a great discussion on the pros/cons of UNIX timestamps, datetime fields, and ISO8601 on the Drupal Events group @ http://groups.drupal.org/node/731.
Comment #7
casey commentedComment #8
LarsKramer commentedI suggest either marking this as a duplicate of the issue mentioned by lilou in #4: #154477: Expand date handling capabilities in core (Item 3 in its summary covers expanded date ranges). Or we might narrow down the scope of this issue to provide a more explicative error message (see comment #3).
BTW, this problem was fixed in Date contributed module as of version 2.x. Could a similiar approach be applied to Drupal core's node creation date?
On my shared hosting environment, with Drupal 7.7 I got this nasty red error message when trying to set the node creation year to 2039:
However, I was not able to reproduce on my local setup where I just got the standard message: "You have to specify a valid date."
Comment #9
swentel commentedThere's also #800628: Y2K38 problem
Comment #10
valthebaldRough calculation shows this issue should belong to Drupal 15.x or so, bumping to 9.x meanwhile
Comment #11
cweagansThis might be the more proper status.
Comment #12
valthebald#11: postponed was my intent, actually. Missed the option
Comment #13
joachim commentedI think we should actually fix this now -- show a validation error rather than crash.
It seems like a silly thing to have to fix, but when people test sites they input silly dates. These people include clients, who then see Drupal crash and are unimpressed.
Comment #14
acrosmanJust retested this issue with the current 8.0.x-dev, and it provides a validation error on submit if using a date after 2050-12-31, but any date between 2037-12-31 though 2049-12-31 still crashes. It fails to handle a database error.
Comment #20
barami commentedActually, This problem on php is not a problem.
But drupal 7 & 8 depends on integer type database column for this operations.
Many created properties use datatype of 'normal int' on db schema.
And it indicates 'int' type database (It depends on data type definition of database handlers)
As you know, MySQL's normal integer limit has 32bit limit.
https://dev.mysql.com/doc/refman/5.7/en/integer-types.html
There are two solutions.
1. Convert all properties about datetime from int to datetime type (eg. DATETIME on MySQL)
2. Convert all properties about datetime from int to big int.
Actually, PHP's timestamp is 64bit value on 64bit systems. It solves Y2K38 problem.
Almost drupal systems already running on 64bit systems. But Drupal don't use 64bit integer value for timestamp on database until now.
Comment #22
markhalliwellRunning into this issue with the
expirefield onkey_value_expire, which system_schema() still defines as:Comment #23
andypostThis issue needs split for validator error, also added child issue #2885413: Timestamp field items are affected by 2038 bug
Comment #24
gambryBetter to start organising the fix for this potential issue (which is not potential any more in some cases, see #22 for example, or #2885413: Timestamp field items are affected by 2038 bug).
I scanned core for all instances of
'type' => 'int', and considered only when the schema fields are used as unix timestamps. See the output list below:core/modules/migrate/tests/src/Kernel/HighWaterTest.phpchangedunsignedL38The question is: do we do all in one go, or we split each component with its own issue?
Comment #25
gambryAfter sharing this on Slack, the consensus seems to be with a mixed of both: groupping similar issues (i.e. same module, same sub-component like Cache backends), but generally having separated issues to tackle the upgrades to big int.
I'll update the IS with the long list of upgrade.
However just came up in mind we still do support PHP 32bit, as we just raise a warning. As part of this parent issue we should also validate if dealing with big_int in a 32bit PHP environment is possible.
I'll add this task when updating the IS.
Comment #26
pedroraf commentedThis is an issue that should have been fixed as soon as it was brought up.
A field called expiration date seems common place for multiple areas, and in fact the date can be much higher than 2038.
Can someone tell me if a fix for D7 is considered? We are still not moving to D8... unfortunately.
Comment #27
stevenpatzReminder that patches are welcome, and that people are volunteering their time.
Comment #28
pasqualle13 years old ticket with 20 comments, should just be closed as outdated. The issue reporter does not even use Drupal any more.
I hope my comment will help move things forward now.
Comment #29
gambryUpdating IS reflecting the plan to tackle this issue (first draft)
Comment #30
gambryIS updates. Adding first issue.
Comment #31
gambryLOL! I'm not sure how exactly can help? :D
The list of components requiring updates is up in the issue summary. Either pick and help on an existing issue or create a new one against one or more files, if they can be grouped.
Also this issue is against Drupal 8. If anyone want to work - or support/fund the work - against Drupal 7 feel free to create a sister issue targeting Drupal 7 and add each other as related issues so we can synchronise work and share solutions.
Comment #34
catchBumping this to critical. This would likely end up being an actual release blocker for Drupal 16 or something, but the earlier we fix it the better - although we need to bear in mind #3108658: Handling update path divergence between 11.x and 10.x (which might mean scheduling this for when 9.2.x is opened).
Comment #35
andypostJust faced today that PHP (64bits) itself has issues handling
time64which added in latest linux kernel. Basically it using time_t in wrong way few places.Will provide more details when it become more clear how to fix https://bts.adelielinux.org/show_bug.cgi?id=266#c2
Comment #38
quietone commentedRemoved duplicate issues from the IS, the full list is available in #24.
What is needed for the IS update requested in #34?
Added an issue for tracker.
Comment #39
quietone commentedComment #40
quietone commentedComment #41
quietone commentedI created a patch for Batch, Cache and Queue and uploading it here because it is late and I am not up to making a new issue.
The only table left to do is Flood.
Comment #42
andypost@quietone probably instead of using direct query in update hook it should use
connection->schemaComment #43
quietone commentedPer this suggestion by alexpott this is being done in two child issues.
Comment #44
quietone commentedFormatting change.
Comment #45
quietone commentedComment #46
quietone commentedThis is really a plan, changing category. The child issues are bug reports.