Fatal error: Call to undefined function token_get_all()

Well, you need at least PHP 4.2.0 to use potx, but Drupal itself has a higher requirement, so I don't know how do you experience this error. What's your PHP version?

I get this error, too. At first I thought it was because token was not installed. After installing token I learned that it does not define a function token_get_all. Both potx and coder call token_get_all, but it's not defined anywhere.

Attached patch adds a dependency on token, but does not solve this problem.

D'oh! I'm catching on now. token_get_all is defined by PHP. Confusing.

But you have to compile PHP with token support: http://us.php.net/manual/en/ref.tokenizer.php

So this is probably a configuration issue. If I can get potx to display a friendlier error message, I'll attach another patch...

This is not perfect, because the error only appears after a submit (it would be better to warn sooner). But it's better than the current behavior.

Dave, token_get_all() has nothing to do with the Drupal token module. It is PHP's built in function: http://php.net/token_get_all So your patch would not help anything. In fact, since potx does not use token module, it would be plain wrong.

The PHP documentation does not mention any cases when the tokenizer in not compiled into PHP, but you might need to look for this extension in your distribution (I'd expect you are on some Linux server, since Windows and Mac builds certainly include this). I found some evidence that some distros need the php-tokenizer package installed separately for PHP. See: http://rpmfind.net/linux/rpm2html/search.php?query=php-tokenizer for example

Let us know if this solved the problem for you, so I can add some docs, and possibly some check to the module which warns about this requirement.

Gabor,

I think d.o caching is keeping us out of sync. I figured out that token_get_all comes from PHP, but you didn't see my most recent comments on this thread.

Anyway, the patch #4 above takes 'token' out of the dependencies, and produces a better error message when PHP's tokenizer is not found.

I recompiled PHP (using the tokenizer use flag on gentoo) and potx now works correctly. Thanks!

I see this as well. I have configured my system to build my PHP with Tokenizer now, but want to test this before I do that. However, just by looking at the patch, I see some flaws:

1. It modifies potx.info – removing $Id$ (which should be in all files) and adds two empty lines?? Needs to go.
2. It is linked directly to us.php.net. Php.net has their own router which will direct the visitor to the best mirror, so just linking to php.net would be better both for the end-user going there and for the servers serving php.net. The link also assumes the one getting it prefers English. And it's very long. All in all, I'd recommend using http://php.net/tokenizer instead. That'll direct the user to the proper page, while still letting php.net do all the magic it wants to do to make sure they serve the right page (language, PHP version, mirror, and so on and so forth)!
3. There's an unneededly double-quoted string. (Yes, minor, but when we're at it anyway. :))

I'll go make a re-roll and test that.

I'm still getting it on running cron, during which l10n_server calls potx and thus makes potx call token_get_all(). So I'm thinking the check needs to go into another part of the code, than potx_select_form_validate().

But for now, the revised patch from before has been attached. (I'm also not so fond of the wording. Perhaps POT Extractor needs the PHP Tokenizer functions/extension to work. or at least something a bit more verbose/descriptive than the current text. Gábor?)

- We don't needs to break the form_set_error() to multiple lines!
- I'd say The <a href="%tokenizer">PHP tokenizer functions</a> are required.

Here's a hand-edited re-roll based on Gábor's comments.

@Gábor: Did you really mean %tokenizer and not @tokenizer?

I also still believe this check should go somewhere else in the code, to catch other modules' taking advantage of potx, thus triggering the error.

hass suggested on similar issues to just use hook_requirements(). I've declined his suggestion quite a few times, since it does not enforce that you would not be able to install the module in Drupal 5 (it does enforce it in Drupal 6). Since nagging the users on the requirements screen is generally the better practice, I've implemented this as a hook_requirements() check in Drupal 6.x-2.x and will backport it to 5.x when I get around to it.

Committed this completely new potx.install file to CVS.

Suggestion in case you face the same initial error:
check that you don't have multiple versions of PHP installed. The one available at the command line may not be the one used for the website....
Or the php.ini file may not be the one that should apply to your php version (Windows for example may have a few old ones in different places...)

PHP tokenizer is a requirement for Drupal 7.x now that the registry uses it. See #315901: Add server capabilities check to installation/upgrade tasks http://drupal.org/node/315901#comment-1518374

@dropcube: indeed, the .install file was even removed in the D7 module to accomodate for this.

Since 5.x is not maintained, moving back to fixed.

Automatically closed -- issue fixed for 2 weeks with no activity.