Split modules into their own directories.

Additionally, the other module-specific files (i.e. forum icons, menu icons, watchdog icons, etc.) can move the their respective module's directories. It will be really nice to clean out misc...or eliminate it altogether!

we're not going to want to just directly *move* the files in the repository, since then if someone tries to checkout an older copy of core (e.g. the 4.6 or 4.7 releases), the files will be in the wrong places. on the other hand, "cvs rm; cvs add; cvs commit" means that folks have to look in the old locations for the older history, which sucks. luckily, there's a solution:

what we'll want to do for maximum goodness is:
a) copy the ,v files from the repository into the new location (e.g. "cd drupal/modules; mkdir book; cp book.module,v book")
b) cvs remove the files on the trunk from the current location
c) remove all the existing tags (esp the DRUPAL-* branches) from new copies

i've got a script that does this automatically, but sadly, upon testing it locally, it needs a little work before it's ready for this task. i'll follow-up again when i have the chance to fix-up the script. just wanted to post now so i can easily track progress of this issue and so folks know this is how you should do the actual cvs renaming.

ok, i finally made some time to work on this. 2 follow-ups:

this one is a "cvs_rename.pl" script. mostly authored by Marv Solomon, a very clueful co-worker of mine (he's a faculty member at University of Wisconsin, Madison). i've modified it to handle branch tags correctly. the perl doc comments are pretty clear. let me know if you have any questions. note, this must be run *directly* on the local file system with the cvs repository lives (since it's directly copying the repository's ",v" files around). therefore, this is only really going to be able to be used by dries for the final rename. however, if someone wants to play with this in a local mirror of the drupal core cvs repository, this should work fine.

next follow-up will be a new version of the "dir.php" script from the original post that uses this cvs_rename script. i'll put more comments there on how to test both of these if there are any interested parties.

new version of the dir.php script that calls cvs_rename.pl from my previous comment with the right args.

to test/use:

1. make a copy of the core drupal cvs repository. if you're directly on the drupal.org filesystem, just do a "cp -r" or something. ;) otherwise, you can simulate a local copy (with a few branches/tags, etc) fairly easily. i'll attach a script do do this in another follow-up
2. download both of these scripts to your local filesystem, rename them to *.pl and *.php respectively, and chmod 755 them.
3. edit drupal_directory_renames.php
   • the $cvs_rename variable should be set to the full path to where you downloaded the "cvs_rename.pl" script
   • the $cvs_repo variable should be set to the full path to the "drupal" directory from the drupal core cvs repository. for example, if the core repository really lives in the filesystem as "/cvs/drupal", and there's a "drupal" subdirectory in there for the actual "drupal" directory that you check out, then you'd want:
     $cvs_repo="/cvs/drupal/drupal" in this script
4. check out a copy of the trunk of the drupal core into a workspace:
   

   % cd /tmp
   % cvs -d/cvs/drupal co drupal
   % cd drupal
   

5. run the rename script: % /full/path/to/drupal_directory_renames.php

that's basically it. it does all the magic itself. basically:

cd modules
foreach module as foo {
  mkdir $foo
  cvs add $foo
  cvs_rename.pl $foo.module $foo/$foo.module
}

the cvs_rename.pl basically does this:

cp $cvs_repo/modules/foo.module,v $cvs_repo/modules/foo/foo.module,v
rm [workspace_copy]/modules/foo.module
cvs rm [workspace_copy]/modules/foo.module
cvs update [workspace_copy]/modules/foo/foo.module
foreach tag in [workspace_copy]/modules/foo/foo.module {
  if (is_branch($tag)) {
    cvs tag -dB $tag [workspace_copy]/modules/foo/foo.module
  }
  else { 
    cvs tag -d $tag [workspace_copy]/modules/foo/foo.module
  }
}
cvs commit [workspace_copy]/modules/foo.module [workspace_copy]/modules/foo/foo.module

so, the end result is that the trunk copy of modules/foo.module is now dead. if you try to check out from the trunk, it's not there. however, all the old branch tags are still in the old location, so if you check out from "-r DRUPAL-4-6", you still get the "modules/foo.module" copy, as always. the copy in modules/foo/foo.module has no branches or tags yet (which is what we want), but as soon as we add a "DRUPAL-4-8-0-RC1" tag on the trunk, that'll hit the modules/foo/foo.module version, not modules/foo.module (which is dead on the trunk). furthermore, the copy in modules/foo/foo.module still has a complete cvs history, so if you do "cvs log" in there, you see everything, and "cvs annotate" still works along the trunk.

the only slightly weird thing with this system is that after the rename, if you want to see the cvs annotate output for a given branch (e.g. DRUPAL-4-7), you have to run the command in the old location, like so:

cvs annotate -r DRUPAL-4-7 modules/foo.module

with a lot more effort, i might be able to come up with a solution that would allow this to work, even if you ran it on modules/foo/foo.module (basically, instead of removing old branch tags from the new location, to systematically remove the new location of the file from each branch). not sure if anyone cares enough about this to make it worth the trouble.

i don't have a viewcvs installation on my laptop to play with, so i'm not sure how gracefully that handles this kind of rename. i suspect it'd all just work, but it'd be nice if someone could test that.

as always, just let me know if anyone has any questions or problems with this.

thanks,
-derek

p.s. i'll quickly investigate how well the cvs.module handles such a rename. stay tuned.

as i suspected, the cvs.module does basically what you'd expect by default: the commit for the rename shows up for both /modules/foo.module and /modules/foo/foo.module. if you click on the "/modules/foo/foo.module" link, you only see entries for the new location. if you want to see log messages for the old location, you have to use the old link (e.g., click on "/modules/foo.module" somewhere on a cvs message page).

if we wanted the cvs.module commit logs to mirror what happens in the actual cvs commit message history (you can view the same history from either file location), i could maybe dream up a solution to this, too. for example, we could add duplicate records to the cvs_files table for all commits using the old filename that pointed to the new one or something. alternatively, we could add weird special-case logic to cvs.module so that if you do a "file=/modules/foo/foo.module" arg to the URL, it has special-case logic to also display entries from /modules/foo.module. however, i think any such solution will just be far more confusing than helpful. at this point, i think only seeing cvs.module commit messages on whatever location the commit happened to is fine.

for the interested reader, here's a hack of a script to "mirror" the core drupal cvs repository to your local disk, so you can try out the scripts i posted above. just change $local_repo to the full path to a directory where you want the local cvs repository on your filesystem. this will export the drupal core from HEAD, import it into your local repo, and make a bunch of dummy branch and release tags.

then, once you run the scripts above, you should be able to do:

cvs co -d/full/path/to/your/local/repo -r DRUPAL-4-6 drupal/modules

and see stuff like "drupal/modules/book.module", vs:

cvs co -d/full/path/to/your/local/repo drupal/modules

where you'll see stuff like "drupal/modules/book/book.module".

not rocket science, but i thought i'd share...

unable to leave well enough alone, took a stab at cvs annotate on old branches in the new location. i even got it working. but as i was testing it and thinking about it, i decided that we don't want this behavior. if people check out an old branch, they're going to cvs annotate along that branch from the old location, which will continue to work just fine. if they're on the trunk and they want to annotate an older branch, they should be forced to annotate in the old location. otherwise, all they'll be seeing is the snapshot of what the old branch in the old location looked like at the time of the rename (when the ,v file was copied into the new location), not the most recent annotation of the old branch (which is what people would want). so, the way annotate is handled in the aftermath of running the previous version of cvs_rename.pl is a feature, not a bug. ;)

that said, i noticed a minor annoyance with the older version. when commiting the "cvs remove" of the old location, the old script used to attempt to commit to both the old and new locations. this is unnecessary, and depending on minor details, can lead to spurious warning messages or needless commits that only end up changing the $Id$ but nothing else. so, the version attached here is (a little) better. but, now i can sleep soundly, knowing this is the best solution cvs can offer to this problem, that i'm not leaving a sub-optimal solution for drupal core modules simply because i didn't spend a little extra time...

Looks awesome. I'll do some testing but I encourage others to do the same.

I tried the script but couldn't get it to work. I'll try again when I can get hold of dww for real-time support.

i just helped dries with this, and he's now got a test repository that seems to be doing just fine. he'll probably do the renames on the live core repository sometime in the next few days. setting to "RTBC", which seems like the most appropriate status. ;)

in related news, i'm working on a handbook page about the cvs and other implications of the move. i'll post a link here once it's done.

This all looks good to me. I have not tested it myself on a local copy of the repo, though. But I have done stuff like this before with other CVS repositories.

Just be sure to make a snapshot copy of the repo before running it, so the worst case would be to copy back the original while the problem is solved. :-)

unpublished handbook page about cvs implications of the move: http://drupal.org/node/69668

Committed to CVS HEAD! :)

Cannot access a site at all after upgrading to latest HEAD. Wasn't able to get the site working again, until I manually changed the 'system' table entry for the following modules, to reflect the new path:

- block
- filter
- node
- system
- user
- watchdog

(basically, all the 'required' modules).

We need a new update routine, to change the path _manually_ in the 'system' table for the 6 modules listed above, to clear the cache, and to then call system_modules(), so that the other modules on a site also get their paths updated. I think all of that should do the job.

Otherwise, nice improvement!

UPDATE system SET filename = 'modules/block/block.module' WHERE filename = 'modules/block.module';
UPDATE system SET filename = 'modules/filter/filter.module' WHERE filename = 'modules/filter.module';
UPDATE system SET filename = 'modules/node/node.module' WHERE filename = 'modules/node.module';
UPDATE system SET filename = 'modules/system/system.module' WHERE filename = 'modules/system.module';
UPDATE system SET filename = 'modules/user/user.module' WHERE filename = 'modules/user.module';
UPDATE system SET filename = 'modules/watchdog/watchdog.module' WHERE filename = 'modules/watchdog.module';

OK, hunmonk has created a separate issue for fixing the upgrade path needed as a result of this patch:

http://drupal.org/node/73456