Hey there,

Is there a chance we could avoid the MySQL-based UUID generation, either by default or via some admin setting? The reason I'm asking for this is that MySQL-generated UUIDs are extremely similar to one another, and I would want them to be as random as possible, i.e. as would be the case with the fallback method inside uuid_uuid.

Thanks for your comments.

Cheers,
Martin

Comments

arski’s picture

If you want, we could make this into a setting in the admin interface.. I could even provide a patch if you have no time for that and agree with the proposition.

recidive’s picture

Yes, I support that.

IMO, it should be a option checkbox: "always use random based UUID generation" with a description saying the default is MySQL generated when using MySQL.

I also think a new argument should be added to the uuid_uuid() function, to by-pass this setting when needed, like uuid_uuid($random = FALSE) with a check if ($random || variable_get('uuid_random', FALSE)).

I'm looking forward to commit your patch.

Thanks!

arski’s picture

Hey,

Thanks for the answer, sounds good.

I'm just wondering though - is there any use at all for the MySQL method? I personally don't see any advantages for it, maybe it would be simplest just to remove it? I mean I don't mind writing a patch, but won't it make things just more confusing for the user, what would be the reasons for him NOT to pick the random method and go for the MySQL method instead?

Thanks,
Martin

recidive’s picture

Hi arski. While it looks more consistent having only one way to generate the UUIDs, the ones generated by MySQL are supposed to be better, version 2 UUIDs.

UUIDs are good because they are generated for a single cpu clock for a given domain (machine), it means it will not clash in any way.

In the other hand, random UUIDs CAN clash, and for me if something CAN happen, it WILL happen.

It's hard for PHP to generate version 2 UUIDs since this don't have access to the low level information, e.g. C has. The algorithms for generating version 2 UUIDs I've found out there don't support multiple platforms.

In the long run, I think it would be better having UUID module generate UUIDs of all versions and having the users choose which one they want to use.

That said, for now, it's better having the random part as a simple fallback.

What do you think? Would you mind writing a patch for that?

Thanks

recidive’s picture

I've found this implementation:

http://www.shapeshifter.se/2008/09/29/uuid-generator-for-php/

Also, there's a PECL module for UUID generation we should use when avaiable:

http://pecl.php.net/package/uuid

arski’s picture

Hey recidive,

Thanks for the reply, I see your point. It's a pity that MySQL doesn't make the UUIDs more random though.

Anyway, I will work on a patch early next week and will get back to you with it.

BTW. Just making sure, the module has a way of checking for conflicts, right? I think I saw something like that in the code.

Cheers

arski’s picture

Title: Avoid MySQL SELECT UUID() when generating a new UUID » Allow admins to select the desired UUID generation method
Status: Active » Needs review
StatusFileSize
new4.62 KB

All right, here we go.. added a new fieldset to the admin screen which enables you to select the method of your choice. The default on first installation (or until the variable is set) is whichever is first available from the old priority list (MySQL>uuid_create>random), so for those people who are not interested in changing any of this, nothing will change.

Let me know if there is anything wrong with it.

Cheers,
Martin

apotek’s picture

I think we should avoid the MySQL function completely. I'm working on a patch, with an included class that will create 3 different uuid versions, selectable in the admin interface. This method does not require the PECL library, or the uuid_create functions to be enabled in PHP. I'll share it here once it's ready.

arski’s picture

sounds good, provided they are guaranteed to be unique in your implementation :) otherwise there's not much use for additional methods it seems. (maybe only in terms of different uuid-version implementations).

And also it feels like maybe you could move yours to a different issue - this issue was only about creating a new admin interface feature that would allow the selection of different methods. The implementation of these methods itself doesn't really fall into the scope of this. (Just saying this so that things don't get messy and so that this patch gets accepted quicker, I'm sure you don't mind)

Thanks

recidive’s picture

@arski thank you for the patch. The patch looks good. I'm just wondering, though, if we shouldn't hide a little of implementation from users. I.e. what if we just have two options for now in the selection fields: "Use the best method available" and "Always use random UUIDs". That way we can choose by ourselves in the code what method is better in each scenery. I think the better, if available, is the PECL implementation, so we should use that instead of MySQL when available. What do you think? Maybe later when we have all UUID versions available regardless of the platform, we can present uses with more choices.

@klktrk great news. A platform independent way of generating UUIDs was what I wanted when I was writing this module. But I think we should still give users the opportunity to use the PECL version, and maybe even the MySQL one, if they want to.

arski’s picture

Hey,

hmm, imo "Use the best method available" is misleading as there are different criteria for a "best method" (what started this whole process). So in general MySQL is possibly best because it guarantees uniqueness... while the random version guarantees randomly looking UUIDs.. I don't know if the PECL method achieves both of those.

I don't care if we hide the actual names of the methods, but then the descriptors should be clear enough, like
"Globally unique, optically similar" vs. "Locally unique, optically distinct" vs. something else maybe. What you think?

recidive’s picture

Status: Needs review » Needs work

"Globally unique, optically similar" vs. "Locally unique, optically distinct", sounds ok to me.

apotek’s picture

We need to distinguish between uuid generating _methods_ and uuid _types_.

In the patch, the form radio buttons
* mysql
* pecl
* rfc4122_v4

mix those two concepts together. mysql and pecl are methods of generating UUIDs. rfc4122_v4 is a type of UUID.

People should be able to choose preferred type, and on the basis of the type chosen (v1, ... v5), they should be presented only with available methods that can create those types of uuids.

The patch also should also modify uuid.install hook_uninstall() to delete the new variables being set

There is a php class here http://drupal.org/node/918888 that gives some additional types of uuids, as well as possible suggestions regarding uuid generation and admin.

arski’s picture

Status: Needs work » Needs review
StatusFileSize
new4.9 KB

Hey again, sorry this dropped out of my scope somehow.

Anyway, here's an updated patch with 2 options as we discussed and a small addition to hook_uninstall.

@klktrk: It's admirable that you want to add more methods to UUID generation etc., but it would be nice if you could open a separate issue for that. The goal of this issue was solely to make different options selectable to site admins, just that functionality and nothing else. The number and types of different methods is something for a different day and once we have more options, we can easily extend the selection provided by the patch in here - np. Thanks.

apotek’s picture

I'm sorry, I didn't mean to change the scope willy nilly on this. And you're right to limit the issue.

the patch looks good within the scope addressed by the ticket.

However, I still think there is an issue with blurring the difference between generation-method of a uuid and a uuid *type* (or format).

+  $gen_method_options = array();
+  if ($GLOBALS['db_type'] != 'pgsql' || function_exists('uuid_create')) {
+    $gen_method_options['universal'] = t('Globally unique, optically similar');
+  }
+  $gen_method_options['optical'] = t('Locally unique, optically distinct');

The 'universal' option basically equates to "use MYSQL/PECL if available" (if you still want to talk about Method.

Since your interface exposes to the user a "type/format" of uuid rather than a method (The method is chosen by the app, not by the user, based on the type the user selects), shouldn't the variables etc reflect that? e.g. uuid_type_options[], or, better, uuid_format_options[]

Also, if the user is on PGSQL and does not have the Pecl extension, why not just hide this from the user completely? Why show just one radio button? I guess this last bit is just a matter of taste. Perhaps its useful for a user to see what format uuid they are using, even if they have no option to change it.

recidive’s picture

Status: Needs review » Needs work

Hi patch looks great. Though I think there's still room for improvements:

1. Please rename $gen_method_options to $method_options
2. The 'universal' and 'optical' seems meaningless in code context, maybe it should be the UUID versions, or 'mysql' and 'random
3. In uuid_uuid(), I think we could split the switch blocks into separate functions in preparation for abstracting the various ways of generate UUIDs into separate modules, this way the uuid_uuid() function would look like this:

function uuid_uuid() {
  $method = variable_get('uuid_generation_method', uuid_default_generation_method());
  $function = 'uuid_uuid_' . $method;
  if (function_exists($function)) {
    return $function();
  }
}

Thanks for working on this!

recidive’s picture

I agree with @klktrk on not showing only on radio button, but I don't care much about this.

Also, it's important to keep in mind that the PECL extension is capable of generating all UUID versions, and the default one used can change between php installs. So we can't ensure it'll always be 'universal'.

arski’s picture

Hey,

from your comments it seems that it's best to split the options in the code into "global" and "local", where "global" corresponds to the MySQL method only, and "local" to PECL, if available, and otherwise the random method. (While the random method does not even guarantee local uniqueness per se, we really have to make sure that the UUIDs are locally unique anyway, hopefully the patch in #915118: Make sure new UUIDs are unique will be accepted soon :)).

So as far as I can see, for a normal user, there is no further difference between the PECL and the random option. The only thing I don't know is how are the PECL UUIDs optically? will they look very similar like the MySQL ones? In that case we could sort them into "Locally unique, optically similar" (making them always inferior to the MySQL method when that is available and thus possibly not even worth displaying?)

Umm.. comments welcome :)

PS. is the "if (function_exists($function)) {" clause really necessary if we make sure that those functions do actually exist? :)

arski’s picture

bump, any thoughts?

ilo’s picture

apologies arski, I can't undestand the 'optically similar' meaning. MD5 or SHA1 are very similar optically and nobody has a doubt in their validation. Does really that kind of separation have a sense for the admin user when configuring the module? What are the use cases for the user to chosse one method prior to another?

And no, I'm not following you in the issue queue ;)

arski’s picture

hmm, just wondering - are you a new maintainer of the module? you seem to be the third person over the last couple of months who suddenly starts answering on all issues and sadly all the re-explaining has kind of prevented any progress :o

Anyway, the optically similar option is the MySQL one - the problem with that is that MySQL generated UUID can be very similar for subsequent nodes.. basically they may differ in 1-2 letters only, which creates a feeling of anti-randomness.. for some sites it is nice to have the UUID look really random for every piece.. see the very first post that started this all :)

ilo’s picture

Arski, I would not go that way. No, I'm not a module maintainer; no, I'm not answering on all the issues, and no, I'm not my intention is not to get all issues stucked, I'm trying to find the rationale behind the issues I don't understand. If this is bothering or you think a single comment in an issue would hang all the work already done until now, then the problem is not me comming to this module queue.

I'm, however, the one that is providing the token integration patch, I'm the one writting the testcase for this module and I'm one of the module users, position legitim enough to participate in the discussions but not to take decissions, that is not my job anyway, so I'm going to leave that task for others.

After re-reading the first post (and the last), I'm still thinking the same, so thanks for your reply. I'll wait a couple of days to see if someone points additional arguments.

arski’s picture

hehe, no offense, I was just wondering :) I have to admit I'm a little frustrated with the slow nature of progress in here, but I'm fully aware that it's not your fault, or any other commenter's.

By "I still think the same" you mean that there is no need for the options? Maybe the point wasn't quite clear, I didn't care about validation or the format of the UUIDs when I submitted this, the main concern was that MySQL generates UUIDs that (this is just a shortened example) look very similar, like the string 'abc1def' and 'abc3def'.. a random generator would generate UUIDs that would look really distinct, like 'lkjhgf' and 'qwerety', which was the initial demand :)

anyway

ilo’s picture

For sure the options are needed, in fact, I'd say that it is required in this case, because depending on your uuid usage you may need one provider or another. What I don't understand is why have you considered this separation: "localy unique" or "optically similar", I'm pretty sure nobody will be able to remember more than 4 different uuids in the first attempt, so it is just an 'sthetic' issue, but I'm not opposite to this approach, which I've found better than the complex technical approach "mysql V-x" or "pecl V-y".. in fact, if of the uuid providers name is "Random", does that means the rest of methods are not random?

Thanks for all your work in this queue arski, and count with me to push your issues untill they get fixed. Apologies if my style has been rude, I'm just trying to understand the 'why' behind the issues, so we can argue enough to convince recidive to commit or not, I never meant to block, hang or drop the issues or the ideas at all.

arski’s picture

Hey, no worries, constructive criticism is good anyway :)

OK, well it's not true that I separated the generation methods into "locally unique" and "optically similar". To be more precise - these two options are not at odds, they are rather 2 properties that are both considered. So in my patch the text for the user is:
- Globally unique, optically similar
- Locally unique, optically distinct

so the user can see the advantages and trade-offs of the different methods and choose appropriately to their needs. I agree that noone will be able to remember several UUIDs, regardless of how similar they are, but having very similar ones (eg. distinct in 1-2 letters) opens the door for easy lookup for example if the UUIDs are used as access tokens..

As for the "random" method - it is indeed simply implemented by PHP rand calls without any particular seed (that's why it doesn't guarantee uniqueness).. for example the MySQL UUID() call returns a string based on a particular date-based seed taken from some db-clock (thats why it DOES guarantee uniqueness), and other PHP UUID generation methods also make user of either the server or client clock (from what I gather).

pol’s picture

Hello,

I made a fork on GitHub of the project, I've included some patches found on this issue list and especially this one and some improvements.

You can find it and improve it if you want, here: https://github.com/Polzme/uuid

arski’s picture

umm.. the only thing is.. why?? you don't seriously expect people to use some private versions of modules all over the place, or am I missing something?

pol’s picture

It's just that it's easier to maintain... as I said, I don't want to get people frustrated by this fork.

I'm also a module developer and I never get the chance to have a cvs account, so I made my own repository on Github.

I think Github is also easier to use than CVS but that's not the main point.

There are also people who can't use CVS at work like me, so using Github on the web is perfect.

arski’s picture

hey, well to be honest it just makes things confusing.. if you want it for personal use then nobody cares obviously, but if you want to maintain the module I'm sure you can just ask the owners, supply a couple of patches and they'll be fine with it.. especially considering the module is in "seeking co-maintainers" state..

I'm not a fan of CVS myself, but that's completely irrelevant, since this is the place where all the issues are, where new official releases are posted and where accepted patches are committed.. *shrug*

pol’s picture

You're totally right of course... I'll see what we can do about it !

Thanks !

recidive’s picture

@arski, a late response to your question in #18: the PECL can be set to generate all versions of UUIDs, it's not well documented. I've found a forum topic on this. Despite it doesn't seem relevant, I think supporting this is good since this is already packaged for some linux distros.

@ilo, It's not about convincing me, you have to convince everyone here ;)

@Pol, if you have an improved patch, please post it here so there's a chance that everyone will benefit from that rather than only the ones tracking your github fork.

Let's work on an improved patch, adding what was discussed from #18 to now, and I promise I'll review and commit it when it's ready.

Thanks!

skwashd’s picture

Issue tags: +drupal 7 port

I'm considering this for the D7 port - subscribing

skwashd’s picture

This has been implements in the D7 port, I don't think it would take too much effort to back port it for D6. The code is available at https://github.com/skwashd/uuid/blob/DRUPAL-7--1/uuid.inc At this stage I don't have much time (or interest) in back porting to D6 as most of my new work is D7 based.

arski’s picture

Umm, cool, but why is this in a private repository? I'm sure if you ask for it, the owners of the module can give you access to develop the D7 branch.

skwashd’s picture

I have been granted comaintainer status, but I started this months ago before d.o switched to git and haven't have time to migrate back. I am waiting to see there is some consensus develops around the current implementation before i move the code back.

arski’s picture

hehe, I see you got everything covered. That's cool, sorry for bugging you then :)

luksak’s picture

Version: 6.x-1.x-dev » 7.x-1.x-dev
Issue tags: -drupal 7 port

What happened to this? In uuid.api.php there is documentation about hook_uuid_info() to define new generators but there is no example on how to switch it:

<?php
/**
 * Defines one or more UUID generators exposed by a module.
 *
 * @return
 *   An associative array with the key being the machine name for the
 *   implementation and the values being an array with the following keys:
 *     - title: The human readable name for the generator.
 *     - callback: The function to be called for generating the UUID.
 *
 * @see uuid_get_info()
 */
function hook_uuid_info() {
  $generators = array();
  $generators['my_module'] = array(
    'title' => t('My module UUID generator'),
    'callback' => 'my_module_generate_uuid',
  );
  return $generators;
}
?>

Looking at this code makes me think it is not possible:

<?php
/**
 * Generates an universally unique identifier.
 *
 * This function first checks for the PECL alternative for generating
 * universally unique identifiers. If that doesn't exist, then it falls back on
 * PHP for generating that.
 *
 * @return
 *   An UUID, made up of 32 hex digits and 4 hyphens.
 */
function uuid_generate() {
  $callback = drupal_static(__FUNCTION__);

  if (empty($callback)) {
    if (function_exists('uuid_create') && !function_exists('uuid_make')) {
      $callback = '_uuid_generate_pecl';
    }
    elseif (function_exists('com_create_guid')) {
      $callback = '_uuid_generate_com';
    }
    else {
      $callback = '_uuid_generate_php';
    }
  }
  return $callback();
}
?>
luksak’s picture

Issue summary: View changes
StatusFileSize
new676 bytes

I created a patch. It currently just uses a variable to store the function name of the generator. Now we could introduce a settings form where this function can be configured.

luksak’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 7: uuid_method_selection.patch, failed testing.

markaspot’s picture

Status: Needs work » Needs review

38: 902176-generator-method-38.patch queued for re-testing. *edit* Sorry for that!

Status: Needs review » Needs work

The last submitted patch, 38: 902176-generator-method-38.patch, failed testing.

skwashd’s picture

Version: 7.x-1.x-dev » 6.x-1.x-dev

I am putting this back to 6.x. @recidive can decide if he wants to implement this functionality there.

We won't be implementing this in the D7 version of the module. We use the most efficient function available for generating the UUID.

balintbrews’s picture

Is there any chance, that you would consider having this in the 7.x version? I think it would be a very nice addition to this module.

I'm encountering the second project in a row in which the client needs identifiers to certain entities, which are not the primary entity identifiers. I ruled out having the UUID made up of 32 hex digits and 4 hyphens, as it produces not so nice URLs. Having the possibility to write our own UUID generator would gracefully solve this. We could use something like hashids or whatever we come up with in our own application. I know that producing nice identifiers that look great in URLs is not really what this module is about, but everything is already implemented to make this happen, it's just a matter of invoking a different callback. So I would hate to write something new for this use case.

The only thing that is remaining here is an administrative UI, which I'm willing to work on. I would focus on not imposing the configuration on those who don't care.

perusio’s picture

@balintk I second that idea. This module can be much more useful if it can be repurposed for URL aliasing/shortening and similar things. hashids look very nice for that.

skwashd’s picture

UUIDs are an IETF standard. If you want to use a non compliant unique identifier system, such as hashlib, I recommend you start your own module.

perusio’s picture

@skawshd it's your module so you get to decide whatever you do with it. The fact that it's an IETF standard is irrelevant for this discussion because there are multiple ways to do unique IDs. For distributed systems causality plays a very important role and there this module as it is will be useless. If OTOH you provided the option to plug in other algos then we could implement a client of the ID generation service. I'm happy to code a module for that myself. I don't need it for tomorrow either, in a couple of months I will.

skwashd’s picture

Status: Needs work » Closed (won't fix)

@perusio this module, like D8 core, is only interested in supporting RFC 4122 compliant UUIDs.