Goal

  • Properly migrate date + date-formats/types + regional + locale system settings into configuration.

The data that needs to be managed includes:

  • A variable for the site timezone
  • A variable for the user timezone
  • A variable for the first day of the week
  • Variables for the date format preferences for each type of date format (short, medium, long, and custom)
  • A table of date formats that needs to be removed and replaced with configuration
  • A table of date format types that needs to be removed and replaced with configuration
  • A table of localized date format types that needs to be removed and replaced with configuration

The variables are straight-forward. But there are open questions:

  1. How to name the new configuration objects?
  2. Which config object holds which variables?
  3. How to handle custom date formats? (vs. pre-defined system date formats that have to exist)

The tables are going to be more complex:

- The current system provides a way for modules to add values, and a UI for users to add values in the UI. Both need to be retained.

- The date format locale tables will have some of the complexities of the conversion of multilingual values to CMI.

Other thoughts:

For the naming of this and the rest of the configuration that needs to be updated, how about this:

date format variables - system.date:format.value
date format table - system.date:format.options
date format type table - system.date:format.types
date format locale table - system.date:format.locale

Then change the two date-related settings in the regional config:

date_timezone - system.date:timezone
date_first_day - system.date:first_day

This would give us a way to get a config object using the 'system.date' prefix for all the date-related configuration, and 'system.date.format' for all the date format configuration.

Concrete conversion issues

Related issues

Some issues that can help provide guidance on how to attack these problems include:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Gábor Hojtsy’s picture

Conversion of date format setup sooner than later is an important step for Drupal 8 Multilingual Initiative so we can ensure all the multilingual underpinnings are well taken care of (some of which might require pre-feature freeze changes).

dagmar’s picture

Issue tags: +Configuration system

Tagging

dagmar’s picture

Issue summary: View changes

Make the list a list.

sun’s picture

Title: META Date and Time Configuration » [meta] Date and Time configuration

I still wonder whether we shouldn't simply have a system.locale config object that combines all the regional/date configuration.

Meanwhile, I think that would make most sense.

Also, unless there's a misunderstanding, the issue summary suggested to put all the individual config keys into separate config objects/files; essentially leading to a range of config objects that only contain a single key. However, we're combining configuration by topics into single config objects/files. I've therefore adjusted the issue summary accordingly.

# core/modules/system/config/system.locale.yml:
first_day: '0'
country:
  default: ''
timezone:
  default: ''
  user:
    configurable: '1'
    default: ''
    warn: '0'
format:
  long: 'l, F j, Y - H:i'
  medium: 'D, m/d/Y - H:i'
  short: 'm/d/Y - H:i'
  html_datetime: 'Y-m-d\TH:i:sO'
  html_date: 'Y-m-d'
  html_time: 'H:i:s'
  html_yearless_date: 'm-d'
  html_week: 'Y-\WW'
  html_month: 'Y-m'
  html_year: 'Y'

The system module would have to ensure that the predefined formats cannot be deleted from the config (and only custom formats can be added).

Alternatively, and if I get the summary right, then System module might have to know about the owner/origin of a date format. This could be accomplished by turning the format sub-keys into arrays, each:

...
format:
  long:
    format: 'l, F j, Y - H:i'
    module: system
  medium:
    format: 'D, m/d/Y - H:i'
    module: system
  short:
    format: 'm/d/Y - H:i'
    module: system
  html_datetime:
    format: 'Y-m-d\TH:i:sO'
    module: system
  html_date:
    format: 'Y-m-d'
    module: system
  html_time:
    format: 'H:i:s'
    module: system
  html_yearless_date:
    format: 'm-d'
    module: system
  html_week:
    format: 'Y-\WW'
    module: system
  html_month:
    format: 'Y-m'
    module: system
  html_year:
    format: 'Y'
    module: system

However, I'm also not sure whether that functionality is strictly necessary to keep -- do we actually know whether there are any contrib modules that define their own date formats? I'd imagine that those modules could as well just add a new date format to the system.locale configuration upon installation, and, be done with it.

Lars Toomre’s picture

@sun I am confused. Why are you referencing *.locale.yml? This issue is about date and time configuration, which presumably can be included in a *.locale.yml file.

Locale information/configuration includes additional information to date and time, like language, currency, location, etc. This issue is strictly about date and time which is complicated enough in itself.

sun’s picture

Sorry for the potential confusion. I'm not referencing Locale module.

I'm precisely suggesting system.locale.yml as the config object to hold date + timezone + regional + country configuration.

The underlying idea for combining them is that there's hardly a case in which you need date but not timezone configuration. The same potentially applies to the default country config, too (although it's hard to validate that, since it's not used in core).

If we go with system.date now, then that would cause an almost empty system.regional config file, which only contains the default country config.

Aside from that, I feel uncomfortable with putting each date format in its own config object. I think we want to combine them into one config object, since it is very likely that the system needs other date formats in the very same request.

Gábor Hojtsy’s picture

As for confusion with Locale module, we hope to rename it sooner or later to Interface translation module or something similar, now that it does NOT handle anything related to locale (such as languages or currency). It is not 100% that would happen, but AFAIS it can happen up to Feb, it is not a new feature :)

KarenS’s picture

I'm not suggesting that each selected format have its own file, I'm making a suggestion for the way the file is named. Your example is more or less the file I was imagining (a single file with all the values), except that you're still calling these 'system.format' and there are lots of other 'format' values that we haven't addressed yet. These particular variables are the assignments of a selected format for each format type. Still to be converted are the list of all possible formats, the list of all formats grouped by localization, and the list of all format types. What are we going to call all those things if we already used the name 'system.format'? I was trying to give the names more clarity so when we get done they will all make sense by discussing all these names at once.

The odd man out here is country, everything else is about dates. Throwing all these date values into a file that does not even contain the word 'date' just so 'country' isn't alone seems odd to me. I think the names should be intuitive, and arbitrarily calling date values 'locale' is not intuitive, IMO. The only reason we're using this name at all is really because that's where it got placed in the admin menu a long time ago, not a good reason for locking those names in.

sun’s picture

except that you're still calling these 'system.format' and there are lots of other 'format' values that we haven't addressed yet

Still some confusion there ;) #3 calls them system.locale : format, so system.locale is the config object, and format is a key within.

We don't have an official standard for how to reference keys within config objects in communication and documentation yet, but so far we went with a colon as delimiter/separator; i.e., config.object.name:key.subkey

That said, you're right in that the key should be date.format, which would lead to keys:

date:
  format:
    long: 'l, F j, Y - H:i'
    medium: 'D, m/d/Y - H:i'
    short: 'm/d/Y - H:i'
    ...

Alternatively, we can go with a system.date config object, which holds all date + timezone + format settings. Leaving the country + potential future currency settings to system.regional or system.locale or similar.

KarenS’s picture

So my example for the ultimate design would be something like the following, where 'value' contains the things now stored in the date variables, 'options' contains the values now in the date_formats table, 'types' contains the values now stored in the date_format_type table, and 'locale' contains the values now stored in the date_format_locale table. I'm not sure of how the table values need to be constructed or if all those values would be in the same yml file or different ones, just trying to create a vision for how it will be named.

# core/modules/system/config/system.date.yml:
first_day: '0'
timezone:
  default: ''
  user:
    configurable: '1'
    default: ''
    warn: '0'
format:
  value: 
    long: 'l, F j, Y - H:i'
    medium: 'D, m/d/Y - H:i'
    short: 'm/d/Y - H:i'
    html_datetime: 'Y-m-d\TH:i:sO'
    html_date: 'Y-m-d'
    html_time: 'H:i:s'
    html_yearless_date: 'm-d'
    html_week: 'Y-\WW'
    html_month: 'Y-m'
    html_year: 'Y'
  options:
    'l, F j, Y - H:i'
    'D, m/d/Y - H:i'
    'm/d/Y - H:i'
    'Y-m-d\TH:i:sO'
    'Y-m-d'
    'H:i:s'
    'm-d'
    'Y-\WW'
    'Y-m'
    ...
  types:
    'long'
    'medium'
    'short'
    ...
  locale:
    'fr':
      'long':
        'l, F j, Y - H:i'
      'medium':
        'm/d/Y - H:i'
    'en':
      'long':
        'l, F j, Y - H:i'
      'medium':
        'm/d/Y - H:i'
    ....
sun’s picture

ok, let's simply assume we go with system.date for now. :)

There are a couple of further details involved though:

  1. I think the 'locale' keys/values will most likely vanish entirely. That is, because localization of configuration data will be handled through context plugins for the config system that are capable of overriding config values. This will approx. look like the following:

    system.date.yml
    language=de/system.date.yml
    

    I.e.: we'll have the original file + a Language module override for a specific language, which is merged into the original when the particular language context is active.

    That's essentially the multilingual (not translatable) aspect of configuration that @Jose is working on. The ground for that has been laid with #1646580: Implement Config Events and Listeners, and storage realms for localized configuration but isn't actually used yet - which explains why @Gábor Hojtsy is eager to have some config in core that needs it. :)

  2. I'm not really able to make sense of the 'options' and 'types' sub-keys... What are they for?

    In any case, I think this needs to be simplified. :) (That said, even though I signed off the original date formats/types patch some years ago, I never really understood why we have and need as much as 3 database tables for handling date formats... this system really deserves some huge simplification ;))

KarenS’s picture

Options are all the possible date formats your system knows about. You can add as many as you like because we can't possibly know all the formats someone might want to use. They are currently stored in a table that needs to be converted to CMI. Before they were in a table they were hard-coded in core and you couldn't alter them.

Types are all the types of dates your system knows about. Core gives you 'long', 'medium', and 'short'. You can add others, either in the UI or modules can provide them in code. That lets you create a 'time' type and a 'date' type, for instance, as well as the core types that have both time and date. They are also stored in a table that needs to be converted to CMI. Before they were stored in a table they were hard-coded in core and you couldn't alter them.

The original variables let you decide which format you want to use as your 'short' format, 'medium', and 'long' format. Those are variables that we've had for a long time, as long as I've used Drupal.

The locale table makes is possible to say that 'short' looks different in France than it does in America, and ditto for all your other types. And that mapping is currently stored in a table. I have no idea how that was handled before we added this table. [Edit - I think there was no way to do this before we added this table.]

Yes it is complex, but I'm not sure there is any easier way to allow people to set default date formats for their system and have locale-specific variations of them. Lots of people have looked at this and no one who understands what all the problems are has come up with any better ideas, this was the best solution we could find, and it was a joint effort of a lot of people who do a lot of work on multilingual sites.

sun’s picture

Issue tags: +D8MI

Let me try to make sense of #11 and "translate" it into simplicity:

I'll use pseudo-code to keep things small.

  1. "Options are all the possible date formats your system knows about."

    That's:

    $options = array_values(config('system.date')->get('format'));
    
  2. "Types are all the types of dates your system knows about."

    That's:

    $types = array_keys(config('system.date')->get('format'));
    
  3. "The original variables let you decide which format you want to use as your 'short' format, 'medium', and 'long' format."

    If we want to retain a facility for "mapping" type identifiers to formats (not sure whether we want), then that would be:

    $format_id = config('system.date)->get('type.' . $type);
    $format = config('system.date)->get('format.' . $format_id);
    
  4. "The locale table makes is possible to say that 'short' looks different in France than it does in America"

    Yeah. As mentioned, that's localization of configuration, not to be baked into the original config objects.

KarenS’s picture

Not quite. You're assuming the only format options we need are the ones that are already assigned to variables. That format options table provides the list of all the possible formats to use, some of which are not currently being used. There has to be a place where this option list is stored, and that list can't be limited to the formats that have been assigned to format types. The list also can't be limited to the short list of formats that core is interested in. Users have to be able to extend the list. They now can see a list of all the formats that core has created, plus all the formats they created that they think they might need in the future plus all the formats that contrib modules have provided.

Similarly, the format type table may include types that haven't been used yet. For instance, in the UI someone can create a new format type and perhaps not immediately assign a format to it. I guess if they do that we could say we won't save the new format type and you can use the keys of the variables to retrieve a list only of types in use. But there still would be a problem because the format type is what you use in Views and formatters, so someone could create a new format type, set up a number of fields to use that type, and just not get around to assigning the type a format. If we save the configuration of their system at that point and don't save the unmapped formats, their fields will be using format types that won't get saved so things will be out of sync. Do we want to create this possibility?

Do you really want to re-invent this whole system right now? Can't we just get what we have into CMI and think about that later?

sun’s picture

To respond to the last question first:

The problem I see with the current, complex system is that types and formats would have to be converted into two independent Configurables (ConfigEntities), because they can be configured independently, in addition to the static list of formats being used in production; including proper handling of created/deleted/updated types and formats when staging/importing configuration.

However, the actual, resulting effect of all this immense amount of configuration is just a single list of key/value pairs being used at runtime. This list contains a couple of predefined formats that ought to always exist, and can be extended with custom formats added by the site admin.

Converting this list into configuration and implementing the required staging/import logic is relatively straight-forward.

Contrary to that, implementing the types/formats/mapping with the inherently required individual Configurables directly means to have to handle dependencies between the individual configuration items, as well as figuring out what to do in case a dependency cannot be resolved for any reason.

Thus, when comparing these two possible directions, it indeed makes sense to evaluate whether the complexity absolutely has to be retained, or whether a configuration-based system for date formats cannot look much simpler.

KarenS’s picture

I don't understand how to create whatever it is that you're trying to describe, so there's nothing I can do with this and I'm blocked. Maybe someone else understands how to do what you're describing without breaking the functionality we have now.

cosmicdreams’s picture

@sun, @KarenS and date-in-D8 team met today to discuss this issue.

I'm going to try to implement a patch that implements this tomorrow night so that we have something concrete to look through / discuss. Should I add that patch here or the #1571632: Convert regional settings to configuration system issue?

cosmicdreams’s picture

@sun my understand of your idea is illustrated with the following picture:

see below

1. use CMI's localization to relate Date Formats and Date Types
2. Denormalize Date Formats and Date Types so that Types are the keys and Formats are the values of an associative array
3. Convert "Options" into state.

Do I have that right?

cosmicdreams’s picture

FileSize
30.68 KB

second try at the picture
IMG_20120913_154059.jpg

cosmicdreams’s picture

I'm working on this tonight. I'm going to start with the latest patch available from #1571632: Convert regional settings to configuration system and go from there.

cosmicdreams’s picture

FileSize
31.09 KB

OK, this is obviously unfinished. But I started down the path of converting the obvious variable_gets and sets for date based config and quickly found functions such as a system_date_* that have been tasked with retrieving information about date formats and types from the system and from third party modules.

If we aim for the objective of storing all information about date_formats in configuration files we'll likely need to change these functions as well since they current pull information about date formats and types from the db.

Things I tried:

  • Implemented sun's config file for system.date.yml
  • changed system_get_date_types() to use CMI (possible performance regression, since I'm not using drupal_static). If the functions reduced complexity
  • Modified system_date_format_locale() to use CMI. For that function that means pulling the lanaguage specific config file. We might want to rename this function to system_get_date_formats_by_locale()

Questions:

  • How important is it that 3rd party modules be allowed to define types but not formats, or formats but not types?
  • Would it be a huge problem if 3rd party modules were instructed to do something like
    config('system.date')->set('format.new_type', $format)->save();
    

    if they wanted to add type / format pairs?

I hope this patch serves as a continuation of our conversation here. Let's keep this going.

sun’s picture

Yes, I think that's basically the structure and functionality I meant in #12.

- How important is it that 3rd party modules be allowed to define types but not formats, or formats but not types?

- Would it be a huge problem if 3rd party modules were instructed to do something like config('system.date')->set('format.new_type', $format)->save(); if they wanted to add type / format pairs?

Yeah, we did the same with other config defined "in-code" elsewhere already; e.g., modules were able to define default image styles/effects through special module hooks. Those are entirely gone. All configuration has to be saved explicitly instead.

Explicit saving and thus declaration of configuration is required to properly resolve dependencies in configuration as well as for staging configuration between servers.

Therefore, the removal of these date type/format hooks is expected, and actually required.

--
On the patch: As long as we're using this issue/patch to hash out the goal and vision only, that's fine. However, the actual changes should really happen separately, back in the respective implementation issues.

cosmicdreams’s picture

In order to pull data formats and types from other modules we'll need to access those module's config files. do we need to establish a new naming convention for that purpose?

cosmicdreams’s picture

The structure of a date_type and date format would need to change. But I don't think this simple pattern fits.

format:
  long: 'l, F j, Y - H:i'
  medium: 'D, m/d/Y - H:i'
  .
  .
  .

date_types and date_formats CAN be merged, but maintaining the human readable and machine_readable names are important (I think). I also think that the properties:

  • locked
  • is_new
  • locale

would be obsolete now that CMI would provide the localization of this config. Therefore the new formats part of the config would look like this

formats:
    system_long: 
      name: 'long'
      format: 'l, F j, Y - H:i'
    system_medium: 
      name: 'medium'
      format: 'D, m/d/Y - H:i'
  .
  .
  .

also:

  • system_get_localized_date_format is a tricky function. I am wanting to convert this to a function that provides the set of date formats for a specific language instead of looping through all date formats for all languages. Also, I need to read up on how we're handling locale-specific config
  • cafuego has some ideas on how to use PHP 5.3's IntlDateFormatter to further reduce the complexity of our date formatting. If we ok comformtable with the date formats that function provides for IntlDateFormatter::FULL, IntlDateFormatter::LONG, IntlDateFormatter::MEDIUM, and IntlDateFormatter::SHORT we could potentially handle those date formats with that function. see: http://www.php.net/manual/en/class.intldateformatter.php and #1733316: Switch to IntlDateFormatter in format_date()
  • As a result we could have the system.date.yml file only contain user-defined date formats. I'm kind of hoping that a side effect of that would mean we don't need to assign user-defined
  • The multi-linqual and translation aspects of this work is related to #1616594: META: Implement multilingual CMI
  • Current work in progress posted here: #1571632: Convert regional settings to configuration system
KarenS’s picture

#1571632: Convert regional settings to configuration system now has a patch with a rework of the date format system to simplify it along these lines. That doesn't yet address how to localize the format, but looks like it may work all right for non-localized formats.

KarenS’s picture

Status: Active » Fixed

#1571632: Convert regional settings to configuration system has landed and I think it addresses the main concern that triggered this issue. There's no point in keeping this open as a meta issue since it's totally out of date now. I'm not sure where localization of date formats stands or if there's still a problem, but if so this issue is not going to be much help. Most of the discussion in this issue got incorporated into the regional settings patch. So I'm going to mark it fixed.

KarenS’s picture

Status: Fixed » Closed (duplicate)

Actually, this ended up being a duplicate of the regional settings patch, so I'm marking it as that.

KarenS’s picture

Issue summary: View changes

Updated issue summary.