I have started working on porting UUID to Drupal 7. recidive asked me to discuss my work via the issue queue rather than via private email.
I have already started working on the code using git so I could merge everything and check stuff in as I went. The repo is at https://github.com/skwashd/uuid/tree/DRUPAL-7--1 It is still early stages of the port, but it gives you some idea of what I'm doing. I'll post follow up comments here as things progress.
As part of the port there are a few things I'd like to see implemented. If there are issues in the queue that I am considering for the D7 port I will tag it with "drupal 7 port".
As part of the port I am splitting things up into separate modules, one each for comment, node, taxonomy and user. The existing uuid module will contain a set of helper functions. I also plan to create hook_uuid_sync so other modules can implement uuid support.
| Comment | File | Size | Author |
|---|---|---|---|
| #62 | 1005138-50.patch | 1008 bytes | jherencia |
| #63 | 1005138_61.patch | 445 bytes | jherencia |
| #52 | 1005138-4315584.patch | 553 bytes | jherencia |
| #50 | 1005138-50.patch | 1008 bytes | jherencia |
Comments
Comment #1
recidive commentedLike I said in the email, for Drupal 7 I think the route to be taken is to enable all fieldable entities to have UUID created for. That would make several core entities uuid enabled as well as enable it for other fieldable entities added by contributed modules automatically.
I'm not sure, but there might have API functions which return what entities are fieldable as well as metadata about this entity. So no need for submodules at all, since you'll have everything you need for displaying checkboxes for uses to chose what fieldable entity they want UUID created for. Nodes, taxonomy terms,
users, comments, etc are all fieldable entities in D7.
That said, I think the port to Drupal 7 will be pretty much a rewrite of the module.
Comment #2
skwashd commentedMy first attempt at porting the module is now up on github.
New features:
* Supports adding UUIDs to any Fieldable Entity
* Adds dependency on hidden_field module - http://github.com/skwashd/hidden_field
* Adds admin support for choosing UUID generator
Known issues:
* Can't remove UUID field from a bundle
* Tests and translations no longer work
* Sync needs to be rewritten
Feedback appreciated.
btw I can now see why it didn't make sense to break up the module in D7.
Comment #3
skwashd commentedMy proposed code was reviewed by chx (, aspilicious and beejeebus) on IRC this evening (AEDT/UTC+11). After a review of what was in my git repo, (I think) we agreed on the following battle plan:
default_value_functioncallback will be dropped and UUID setting will be handled inhook_field_attach_presave, with revisions supportuuid_find($uuid)which returns some meta data about the record matching the UUID anduuid_find_revision($uuid)which returns revision specific meta data.I have some time to make this happen this week.
Comment #4
chx commentedPlease do not do two functions. I would provide a uuid_find($uuid, $age = FIELD_LOAD_CURRENT):
Comment #5
recidive commented@skwashd Awesome! Thanks for getting the ball rolling!
I had a quick look at the module code and it's looking promising.
And thanks @chx for mentoring :)
Comment #6
chx commentedWe , as in the Drupal community need uuid module in core in Drupal 8. It's awesome someone is working on it. I would love to see a hook_init implementation that looks at the URL and if it looks like a UUID then it quickly and quietly changes the request to point to the relevant entity URI so that example.com/$uuid can work. That'd be bliss.
Comment #7
skwashd commentedI have pushed some changes to github. Here is a quick summary of where things are at:
hook_field_attach_presave()Known Issues
field_uuid"I have a better idea of how I'll handle
uuid_find()after discussing it with @chx on IRC.The good news is that the basics now work. Feedback / reviews are appreciated.
Comment #8
q0rban commentedsubscribe. I'll be downloading your code now to give it a test run. :)
Comment #9
q0rban commentedOne other BIG missing piece is the upgrade path. The schema is completely rewritten, so DO NOT use this module if you are upgrading a D6 site to D7. It's not ready for that yet. :)
Comment #10
skwashd commentedComment #11
q0rban commentedOne issue that has been discussed in IRC with skwashd and hejrocker is that switching to this new model means that a new UUID would be generated for every entity revision. This can be preferable in some scenarios I presume, however, it may also be preferable to have a canonical UUID for a particular entity, no matter it's revision. This should possibly be a option you choose on the field itself, specifying whether the UUID changes when a new revision is created. Another option would be to create a separate field type for revision UUIDs.
Comment #12
quazardous commentedhi,
you should commit your module on cvs.drupal.org :p
subscribing with #11
Comment #13
recidive commented@skwashd: I gave you commit access. Please check in your code to HEAD and make a dev release when you feel it's appropriate.
Thanks!
Comment #14
skwashd commented@recidive Thanks for that. Once I have a first cut I'm happy with I'll check it in
@quazardous I prefer using git, but once I have something in a usable state I'll start keeping the repos in sync.
Comment #15
BenK commentedSubscribing
Comment #16
danielb commentedsubscribe
Waiting on this issue for D7 Node export integration #1030678: Components missing from Drupal 7 version
skwashd - hopefully I'll spot you at DDU this weekend if you want to have a chat about this
Comment #17
skwashd commentedThe code on github is starting get bedded down. After a discussion with chx on IRC I have a plan for implementing sync in a sustainable way. I still need to work on hidden_field. Once this is all done, I will commit my code to CVS and create a dev release.
@danielb uuid_find() is likely to solve a lot of your problems. When I'm not presenting, my favourite place at conferences is the hallway track. I'm happy to talk about #1030678: Components missing from Drupal 7 version in general terms, not just this issue.
Comment #18
recidive commented@skwashd: Awesome! Thanks for working on this!
Comment #19
q0rban commentedMore conversations with skwashd led to a realization that using the D7 Field API for UUIDs doesn't make sense. It is equivalent of turning the node id into a field in Drupal 6. There should be a central UUID table that stores all UUIDs for all entities/bundles/vids. The only reason for implementing the D7 Field API would be for UUID references, not for storing the generated UUIDs. This is a critical task IMO, but may actually not take that much rewriting. The current field API implementation would just need to be rewritten so that those fields are actually references, and then a new schema for the UUIDs themselves written, and implementations for the entities begun.
Comment #20
eaton commentedJust wanted to post a BIG -1 and a warning about this. FieldAPI is a TERRIBLE place to put something that needs to act as a reliable, unique identifier for a given entity. FieldAPI imposes a huge amount of storage and structural overhead and its advantages only appear when we deal with data that needs to be edited and presented in a variety of ways on different sites.
As q0rban says, UUID references in fields make sense, just as node references in fields make sense. But moving the UUID itself into a field would be equivalent to moving nid, or uid, or cid, into a field. FieldAPI is a fine hammer, but it would be a mistake to pound this particular screw with it.
Comment #21
karens commentedI'd also vote against having UUID be a field, it's an identifier.
Comment #22
skwashd commentedI have pushed some more code to github. I'm not confident it is the best solution for the problem. I look forward to spending more time coming up with the best solution. The more I think about it, the more I think the Field API isn't the answer. As I've discussed with q0rban, I think a global ID table might be the solution.
Comment #23
skwashd commentedI chatted with Dries at Drupal Downunder about this issue. Although it is slightly off topic, I will summarise my vision for UUID in D7 and on to D8.
I think a global UUID table is the way to go. The schema would be pretty simple uuid, entity, id, revision. SQL stored entities could use joins for accessing the data. When UUID is used with noSQL a db hit would be required before retrieving the entity(?). A clean API would need to be defined, but I think the key piece would be uuid_find().
The D7 implementation will probably involve a few compromises which can be dealt with if/when the D8 port is completed. For example all references could use UUIDs in D8.
Comment #24
q0rban commentedMore communication with skwashd in IRC. We need a uuid column and a revision_uuid column, the same way that the node table has the nid and vid columns. The uuid value will never change throughout the life of an entity. The revision_uuid will be unique for each newly created revision. This will allow you to easily query the table on either the entity uuid or the revision uuid and get exactly what you want without any surprises.
Comment #25
andypostI think this issue should have mirror issue in D8 core!
IMO UUID should be tightly involved in RPC, RDF as superglobal to identify Entities and probably Fields
At first we need UUID implementation as field then module could _alter core entities to inject uuid field into core's entities
EDIT: @chx been involved in db_sequience implementation as I remember so core could get great alternative of "sequiental" identifiers as for now!!!
Comment #26
recidive commentedI also think making this a field doesn't sound like the way to go. But I don't have any sound arguments about it and am not sure on the implications it should have in practice.
For making this a single table, there are some advantages, like being able to check for duplicates and having a central place for looking and resolving to a single object. One question would be what would happen with entities that have two or more primary keys. I'm not sure fieldable entities support this, so don't know if it would be a problem.
I think we need to point the pros and cons of the approaches we can take. This would help making a design decision with less chance to regret.
Comment #27
danielb commentedThat's a good point about the primary keys, they could require different schema fields for some entities.
Comment #28
gddIn Deploy I used separate tables for each of the mapping (node has node_uuid, users has users_uuid, etc). This way you can deal with compound keys. I am a fan of this approach and less of a fan of the approach in #23 because it implies that everything in Drupal that needs a UUID is going to be an entity which is not going to be the case.
Comment #29
andyposthaving uuid as field does not make it dependent on entity primary keys.
Also having ONE huge table is a performance bottleneck - mostly about that we need more JOINs and we loose ability to identify relation of record to entity (sub entity)
Comment #30
moshe weitzman commentedwe may need multiple tables for different primary key types. but for now i would focus on nailing entities and thats it. all core entities have a single int ID so lets stick to one table at first. My .02
Comment #31
skwashd commentedI haven't forgotten this. I have been overwhelmed with work. I plan to spend some time on this over the weekend.
Comment #32
interx commentedInteresting discussion...subscribing
Comment #33
dave reidFor other modules to rely on this port in a non-field usage, it does not make sense to have people have to install hidden_field. :/
Comment #34
apotek commentedSubscribing.
Comment #35
skwashd commented@Dave I will be dropping the dependency on hidden_field.
Expect action before Chicago
Comment #36
dixon_subscribing
Comment #37
clashar commented+1
Comment #38
rorymadden commentedsubscribe
Comment #39
quazardous commentedhi,
just to let know I ve started a new project around data importing and Id stuff...
http://drupal.org/project/datasources
Comment #40
Remon commented+1
Comment #41
citlacom commentedsuscribing
Comment #42
qbnflaco commented+1
Comment #43
skwashd commentedI'm back working on this. Expect a new code drop this week. The HDD in my laptop died while I was in Chicago, which slowed things down a lot :(
Comment #44
skwashd commentedA new version of the code is available on github at https://github.com/skwashd/uuid/tree/DRUPAL-7--1
There are some limitations, namely only nodes and comments are supported and there is no upgrade path or tests currently available. I plan to continue working on this to get it finished ASAP.
Feedback welcome.
I plan to move development to git.drupal.org once things settle down a little.
Comment #45
skwashd commentedThe direction of this module in d7 (and 8) was discussed on IRC tonight (AEST).
The consensus seemed to be that the module should be implemented using a collection of hook_schema_alter()s against the core entity tables (node, users, comment, taxonomy vocabs and terms, files etc), the various hook__() hooks will then be used to manage the data in the tables.
Where revisions are supported a canonical (or consistent) entity UUID will be recorded against each revision of the entity. This will add some processing overhead, but was felt to the be the best way to implement the functionality.
There is still no clear direction on how this functionality will be implemented in d8, but this will provide one possible example.
Comment #46
catchSubscribing.
Comment #47
jherencia commentedSubscribing...
Comment #48
skwashd commentedMore updates in github. Only node currently works. Feedback appreciated.
Comment #49
skwashd commentedBraindump before I forget. There is a some elegance to this approach, but a severe limitation is that it doesn't track deleted items properly.
Comment #50
jherencia commentedYou've forgotten to delete dlm funcition, so install fails and no indexes are created and vuuid field is not added to node_revision table.
I've also changed a minor error intializing schema in hook_install.
Now node_uuid works.
Comment #51
skwashd commented@jherencia thanks for the patch - committed. I always grep for dpm() before a commit, but forgot about the dlm() call.
Comment #52
jherencia commentedIn order to let node creation from other sources (i.e. a Mobile phone app), i think it's necessary to check if uuid is already generated. Right now, in that situation, hook_node_presave overwrites it so I've add that comprobation.
On a different topic, @skwashd could you explain the need to track deleted items?
Comment #53
jherencia commentedIn addition, I've been checking uuid.inc and I think it would be better to rename $callback static variable into a more specific name, maybe $uuid_callback, to prevent possible conflicts with other modules.
Other thing I would like to know is where $generators[$generator] is declared and assigned, because I can't find it, and I don't know if could be an error.
Comment #54
skwashd commented@jherencia
#52 I will think about this one. As the uuid_node module has responsibility for handling UUIDs for nodes and in this case it is only acting on new nodes, I think that the uuid_node module has a right to set (or override an existing) UUID for the new node. I will see if I see things differently in the morning.
#53 As $callback only exists in the scope of uuid_uuid() there is no risk on conflicts with other modules. I think the variable name is clear enough and so doesn't need to be changed.
Thanks for catching the $generators bug. It is fixed in my local git repo and will be included in the next push.
Comment #55
dave reidJust a note that I'm experimenting with an entity-only UUID module for D7 originally based off skwashd's github code. I put it into a sandbox at http://drupal.org/sandbox/davereid/1119768
Comment #56
jherencia commented@skwashd and I have had an IRC talk today where I explain him the necessity of letting node uuid to be generated before node_save call. In cases where you have to sync different sources of data, ie a Drupal Web and a Mobile app in which nodes/files/etc could be generated in both contexts, the best way is to generate the UUID in the source and then export it.
As it is implemented right now, it will be overwritten.
Comment #57
skwashd commentedRecently there have been some discussions on IRC about how to go about implementing UUID for D7 contrib and D8 core. The other night there was a discussion between q0rban, heyrocker, catch, myself and others. The summary of that conversation is available on drupalbin. An earlier discussion about handling revisions using the one big table approach is referenced from the paste.
Comment #58
gddI agree with jherencia that if a node is saved with an already existing UUID, then it should retain that UUID rather than having it be replaced. The whole idea of being able to synch content between instances of Drupal relies on this functionality.
Comment #59
skwashd commentedI have pushed another batch of code to github. Unless there is another push for a rewrite, I think I will switch to git.drupal.org this weekend and cut an alpha release. I will then start working on the upgrade path.
I have dropped the sync functionality for now, but I plan to revisit the issue of retrofitting UUIDs to existing records later.
Thanks everyone who has been involved in getting the module to this point, I've really appreciated the feedback and comments. I hope we can get this version over the line and then start looking how we incorporate this stuff into D8 core.
Comment #60
jherencia commentedIt looks good, I'll test it later and give you feedback here.
Great work!
Comment #61
jherencia commentedThere is a bug with uuid_file, this patch fixes it. I've tested uuid_node, uuid_user, uuid_file + patch and they all work as expected.
I change the status till the patch is commited.
Comment #62
jherencia commentedWrong attachment.
Comment #63
jherencia commentedOk, I hope this is the last comment... lol
Comment #64
skwashd commented@#63 Fixed in github. I'm starting on the upgrade path. I plan to focus on that this week, given the absence of any outcry over the current implementation.
Comment #65
skwashd commentedForgot to update the status
Comment #66
jherencia commented@skwashd function uuid_file_file_presave still comprobates if the fid is set, not the uuid.
I think it is time to move the repo to Drupal and create here the branch, that way more people could test and create possible new issues.
Comment #67
skwashd commentedI have switched to git.d.o and have created a 7.x-1.x branch.
The outstanding tasks are:
* Implement upgrade path
* Implement sync - I have notes for this already
* Implement tests - Patches welcome
* Implement tokens
Nightly builds are now available. I will probably cut a 7.x-1.0alpha1 release later this week.
Comment #68
skwashd commentedSync is now working in git - or the next -dev build.
I have a couple of potential problems to sort out with the upgrade path, but I will tackle that this week.
Comment #69
mrsinguyen commented+1
Comment #70
skwashd commentedToken support is now available in git and should shortly be available in the next nightly build.
The only oustanding issues are tests and the upgrade path. I plan to deal with the upgrade path over the easter break.
Comment #71
skwashd commentedI've just checked in code which handles the upgrade. I tested it with devel generated content. I would really appreciate it if someone could perform some real world testing on it.
Comment #72
googletorp commentedSubscribing
Comment #73
skwashd commentedViews support is now available. If there are no critical bug reports over the coming days I'll release 7.x-1.0alpha1 and close this issue.
Comment #74
googletorp commentedI noticed some minor bugs still exist, so while we have still issue active, I wont create a new issue each time I find a bug.
I created a sandbox project for my commits for UUID which is located: http://drupal.org/sandbox/googletorp/1137642
And I have a dedicated branch for all the port fixes: http://drupalcode.org/sandbox/googletorp/1137642.git/shortlog/refs/heads... which currently only have a permission bugfix/improvement.
Will try to post here, if I'm able to find and fix more stuff.
Comment #75
skwashd commentedThe configure option in the ini file was on my mental todo list :)
Thanks for catching the hook_perm() change.
Please create separate issues with patches attached so problems can be tracked and discussed as needed rather than having to cherry pick patches.
Comment #77
skwashd commentedClosing. Any other work on this (including add test support) will done via separate issues. The main part of this work is complete.
Comment #78
recidive commentedCool! Nice work @skwashd!
Comment #80
doublejosh commentedSorry, what ticket is the work for this continuing in?
Comment #81
skwashd commentedUse separate issues for problems found.