As demonstrated by #357970: Undefined function token_get_all, Drupal suffers a fatal error when the php_tokenizer extension is not enabled (it's required by the code registry to parse PHP sources).

[#444718] therefore calls for this extension to be checked for during the install process. The obvious place for this central requirement is the system module, so this patch adds it to system_requirements.

CommentFileSizeAuthor
#12 Screenshot from 2015-11-11 19-18-19.png120.77 KBfizk
#7 drupal.tokenizer-revert.patch1.16 KBsun
#3 system-requirements-tokenizer-445062-3.patch1.16 KBcburschka
system-requirements-tokenizer.patch1.16 KBcburschka

Comments

cburschka’s picture

cburschka
they
commented

It appears the filter will really only take project issue nodes.

#444718: 2009-04-25 & 26 Core patch review sprint

Status: Needs review » Needs work

The last submitted patch failed testing.

cburschka’s picture

cburschka
they
commented
Status: Needs work » Needs review
StatusFileSize
new system-requirements-tokenizer-445062-3.patch1.16 KB

Wow, I must have been tired. That, or having a testbot around made me lazy. :P

Here's a fix for the syntax error.

dropcube’s picture

dropcube commented
Status: Needs review » Reviewed & tested by the community

This looks good to me. Just to point out, this requirement check at this point is OK, but we should include this requirement check at other points where hook_requirements is not reachable yet, the registry is not able to run if this extension is not enabled.

dries’s picture

dries commented
Status: Reviewed & tested by the community » Fixed

Made some minor changes and committed this to CVS HEAD. Thanks!

sun’s picture

sun
Primary language German
Location Karlsruhe
commented
Status: Fixed » Needs work

This requirement should be limited to the phases 'install' and 'update'. There's no point in showing it later on at runtime, because you won't see the status report page without Tokenizer.

sun’s picture

sun
Primary language German
Location Karlsruhe
commented
Status: Needs work » Needs review
Issue tags: +API clean-up
StatusFileSize
new drupal.tokenizer-revert.patch1.16 KB

The registry is gone, and chx confirmed in IRC that class names are now identified with a regular expression.

So let's revert this requirement.

dawehner’s picture

dawehner
Primary language German
commented
Status: Needs review » Reviewed & tested by the community

This makes sense. token_get_all is not used anymore in core.

Perhaps coder_review module would need this requirement, because it uses this function.

dries’s picture

dries commented
Status: Reviewed & tested by the community » Fixed

Committed to CVS HEAD. Thanks.

Status: Fixed » Closed (fixed)
Issue tags: -API clean-up

Automatically closed -- issue fixed for 2 weeks with no activity.

Issue tags: +API clean-up

Restoring issue tags, see #2125755: System messages removed all issue tags during D7 upgrade.

fizk’s picture

fizk commented
Version: 7.x-dev » 8.0.x-dev
Priority: Normal » Critical
Issue summary: View changes
Status: Closed (fixed) » Active
StatusFileSize
new Screenshot from 2015-11-11 19-18-19.png120.77 KB

I'm still seeing this error in Drupal 8 RC3:

Screenshot

Fatal error: Call to undefined function Doctrine\Common\Annotations\token_get_all() in /var/www/drupal-sites/8/vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/TokenParser.php on line 56

fizk’s picture

fizk commented
Priority: Critical » Major

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

  • Dries committed 5c629b7 on 8.3.x 
    - Patch #445062 by Arancaytar: check for PHP tokenizer extension to be...
  • Dries committed 105bd63 on 8.3.x
    - Patch #445062 by sun: removed system_requirements check for tokenizer...

  • Dries committed 5c629b7 on 8.3.x 
    - Patch #445062 by Arancaytar: check for PHP tokenizer extension to be...
  • Dries committed 105bd63 on 8.3.x
    - Patch #445062 by sun: removed system_requirements check for tokenizer...

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

  • Dries committed 5c629b7 on 8.4.x 
    - Patch #445062 by Arancaytar: check for PHP tokenizer extension to be...
  • Dries committed 105bd63 on 8.4.x
    - Patch #445062 by sun: removed system_requirements check for tokenizer...

  • Dries committed 5c629b7 on 8.4.x 
    - Patch #445062 by Arancaytar: check for PHP tokenizer extension to be...
  • Dries committed 105bd63 on 8.4.x
    - Patch #445062 by sun: removed system_requirements check for tokenizer...

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

  • Dries committed 5c629b7 on 9.1.x 
    - Patch #445062 by Arancaytar: check for PHP tokenizer extension to be...
  • Dries committed 105bd63 on 9.1.x
    - Patch #445062 by sun: removed system_requirements check for tokenizer...

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

catch’s picture

catch
he/him
Primary language English
commented
Version: 8.9.x-dev » 9.2.x-dev
Issue tags: -API clean-up +Bug Smash Initiative

This is still an issue, and token_get_all() is used in the installer.

catch’s picture

catch
he/him
Primary language English
commented
Issue tags: +Novice
lendude’s picture

lendude
Primary language Dutch
Location Amsterdam
commented
Status: Active » Fixed

Opened #3222618: system_requirements should check for tokenizer extension so this one can go back to fixed since it was already committed twice.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.