This is a follow up issue for: #1622974: Aggregator fails to parse author entries longer than 255 characters. Please read this issue for a full background.

Summary

Currently Aggregator truncates for example the author field to 255 characters. In cases where there are more authors, and authors need to be listed this can lead to output problems.
Users should be able to set the length of the amount of characters in an aggregator field. This to increase user friendliness. Another option is to raise the character limit from the start to something like 1500 characters for example (This is 30 names at 50 characters a piece).

The usability aspect of this issue: The user should be informed what the limit of the amount of characters is for an aggregator field. This so that it is easier to figure out if e.g author names get truncated why it is happening.

For some publications it is not uncommon to have a list of 10 names or more. It would be nice if this could work without worrying that it breaks, looks bad, and be able to intervene at some point.

Comments

Anonymous’s picture

Issue summary: View changes

Fix spelling mistake

Anonymous’s picture

Issue summary: View changes

Writing correction

Anonymous’s picture

This is a writeup for further exploring the issue.
.
One option to solve this could be to increase the value in the database of varchar.

According to the Drupal F.A.Q.:

Why does Drupal need a database? What databases are supported?
Drupal stores its information - the individual pages, the registered users, and so on - in the database. The database forms the back-end for your Drupal site. At this time, Drupal supports MySQL (or an equivalent such as MariaDB), PostgreSQL, and in Drupal 7, SQLite.

Varchar limits of databases:

This list claims in no way to be complete, but as a starting point for a feasibility study.

MySQl version lower than 5.0.3: 255
MySQl version MySQL 5.0.3 and higher: 65,535
PostgresSQL: 10485760(?)
MariaDB: unknown at this time
SQLite: Does not enforce the length of varchar

Other databases Drupal is run on in the wild (there may be more):
Oracle: 4000
MSSQL: 8000

Adding the limit to a form field in Aggregator

Instead of manually setting the limit in the database, what might be a solution is allowing the user with the proper rights to set the limit of the varchar field. This based on the limit of the database.

The Aggegrator field for the author could then say something like this in the case of MySQL:
Author Field Length [ ..Please enter value...] (Max Length = 65,535)

or in the case of Oracle:
Author Field Length [ ..Please enter value...] (Max Length = 4000)

The standard limit value could either be entered manually, or even better would be if it could be asked from the database. So let's say for the sake of illustration that the user enters a value of 1500. This would then set:

Pseudocode

$varchar_limit = 'varchar limit entered in form (1500 in this case)'. 

$item['author'] = truncate_utf8($item['author'], $varchar_limit, TRUE, TRUE);

Temporary solution?

Another question is can the length of the varchar field be manually edited at this point? So for example changing the value of truncate_utf8 through a module as well as changing the varchar field in the database to a larger number (when the database allows for it)? This would allow for a solution where core code would not have to be changed at the moment. Leading to the following point.

How many people will have need of this solution.

The length of the field can be a semantic discussion. Some users may never run into this issue, while others will run into this. From what I can see there are two options at present:

1. That the Aggregator module has a set limit for certain fields and if users need a bigger length they should user another module. Stating this in the documentation leads to clarity, and the user knows the limits of the module. Which in its own right is a good thing.
2. The Aggregator module should account for a larger length, whether through a manual coding adjustment or through an integral solution.

Remaining Questions

1. Should fields in the database be a varchar, text, or something else?
2. Should there be a limit on the field? There are varying viewpoints on this.
3. Which leads to the best performance (out of the box)?
4. How does truncate_utf8 in the Aggregator module work when a feed has another encoding?
5. Would you want to be able to change the limit of the field of truncate_utf8 or the database field from within the application itself, or keep this in a seperate file for example or manually editing settings.php and manually configuring the database?

Other things to consider

Not entirely sure if these need seperate issues in the cue.

1. How many database fields in Drupal use varchar? Are there other issues where setting the value of varchar could be a solution? Should this then be in core or a module?
2. There should be an alert message given/ logged in the backend if the value of a field in the feed exceeds the limit. (At the time of writing I could not find any information on whether Aggregator already does this. Please see the following support request).

Anonymous’s picture

Issue summary: View changes

Updated issue summary.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev
Spokje’s picture

Project: Drupal core » Aggregator
Version: 9.3.x-dev » 1.x-dev
Component: aggregator.module » Code
Issue summary: View changes

The aggregator module has been removed from Core in 10.0.x-dev and now lives on as a contrib module.
Issues in the Core queue about the aggregator module, like this one, have been moved to the contrib module queue.

larowlan’s picture

Status: Active » Closed (works as designed)

Now these are entities, you can do this via e.g. hook_entity_base_field_alter