Drupal ports collection - automate package installation and dependency checking

see "drush pm", the DRUpal SHell Package Manager. It does this. For example:
#192651: Add CVS support to drush package manager

sorry, not duplicate, but certainly closed... ;)

@moshe: sorry for messing with your issue queue
@dww: i would not call this request solved with drush pm because:

drush pm only fetches projects which are collections of modules.

my patch #220945: patch: drush mm enable/disable/dot/uninstall is only able to work with modules.

The question remains what project provides (apt term?) what modules?

We cannot fix this for drupal 5 but I think I can solve this as an extension to drush mm later on as orchestration together with drush pm.

Ie drush mm enable uc_cart requires (in the end) module token provided by project token so this must be fetch through drush install token

My guess is the best solution would be by adding a .project file to each project autogenerated by the packaging system stating the provides list. All .project files are then zipped into drupal.dep file as a dependencies lookup.

Hope this helps a little.

Regards,
Clemens

Requires loads of help from core. Not doable in drush now.

Here are some notes on what I'm planning

http://groups.drupal.org/node/21295

Sorry you guys missed my drush_mm. Or is it the other way around :)

Removing the link to drush_mm from the drush home page was not nice though :(

I guess the best way now is to join the plans Adrian has and try to add some thought over there.

here's my code to build the package manifests :

you need spyc.php from http://spyc.sourceforge.net/
and you need to modify the paths yourself.

fyi: http://drupal.org/project/project_api

adrian's code looks nice and clean. do we have any sense of what an API for this ports dataset would look like? i would not mind offerring a ports service on drush.ws

As near as I can tell, the minimum we need is a mapping from extension to project (content => cck). If we had that, then pm-enable could offer to download missed dependencies, and then run enable again... which might find more missed dependencies.

To make that more efficient, we could cache another data set that mapped from extension to dependencies, built from the .info file. Then pm-enable could download all missed dependencies in one go.

Required API:

function getProjectForExtension($extensionName); @returns string containing project name

Optional API:

function getDependenciesForExtension($extensionName); @returns array containing project names

Even the minimum would be extremely useful.

Sounds like we need to download all of git.dupal.org and record the extensions for each project. thats quite different from the project/release code that adrian posted.

Yes; I'm not sure that a simple mirror of project / releases is going to help us here.

I have an algorithm floating in my brain... extensions are stored in a table, which also has a field for dependencies and a field for deep dependencies (the later is precalculated and cached; this might not be necessary). If the web interface is called, then the table is calculated via lazy evaluation, which gives us the opportunity to decide to flush the cache and repopulate. In typical use, the table will be prepopulated once. If drush asks for a dependency list and the result fails to resolve dependencies, then drush will ask again, this time with a flag that forces the cache to be cleared and rebuilt.

Wouldn't be too hard to write. I just need to figure out how to do function addMoreHoursToMyDay(); first. ;)

Edit: "Lazy evaluation" was a poor choice of words, as clearly the list must be prepopulated once in order for the algorithm to work. The incremental update can be done similarly to a lazy evaluation; however, if someone downloads a new module "Y" that depends on a new module "Z", and no one had downloaded "Z" via drush before, then you cannot help but occasionally do the occasional full sweep to pick those up (save for the optimization that you can always check, and see if extension "Z" can be found in project "Z").

I was thinking of an easier approach:

$ drush dl imagecache

1) downloads module
2) reads module.info and checks dependencies variable
3) if dependency is missing, goto 1

What do you think about that? I'm ready to code this.

modules are dependant on modules not on projects. Thats always been the crux of the problem. We have no module => project lookup service.

#15 is actually implemented, except it is done in en, not dl, and it is missing #16.

Yes, I get it.

But for an interim solution, until Drupal ports collection is completed we could add automatic download of dependency modules, just like in en.

Something easy for use like:

$ drush dl --download-dependencies imagecache

Which would download imageapi as well, if not already present.

We have no module -> project lookup, but simple recursive scan for .info files would reveal all possible dependencies.

I see benefits from this, but please let me know what do you think, before I start coding?

From your suggestion in #18 you could map from project -> module, but you could not map from a module that has never been downloaded to the project that defines it. Take a look at the existing code; there is a very-poorly coded cache that, at dl time, will save the project -> module mapping. This map will be used at en time, so you will get modules that drush has seen before. It also tries the obvious guess of project == module, so it can get some simple dependencies right (e.g. views -> ctools).

Maybe we should split this discussion to a different issue, even it is kind of similar.

What I meant was to ease downloading modules.

So when I call drush dl as:

$ drush dl --download-dependencies imagecache

It would download imagecache, then (in PHP) do something like following:

$ find sites/all/modules/imagecache -name \*.info -exec grep dependencies '{}' \; | awk '{print $3}' | uniq

Take those modules, and download them, if they're not already downloaded.

See what I mean?

@kotnik: Yes, I see, and this is in essence what drush en imagecache does right now. I think it is better to download the dependencies in pm-enable; some projects contain a lot of modules, and each module has different dependencies. If you wait until pm-enable time, then you only need to resolve the dependencies for the modules that you actually need and enable.

@greg.1.anderson: If you think it's better to handle this on en time, than that's okay with me.

This issue was marked closed (won't fix) because Drush has moved to Github.

If this feature is still desired, you may copy it to our Github project. For best results, create a Pull Request that has been updated for the master branch. Post a link here to the PR, and please also change the status of this issue to closed (duplicate).

Please ask support questions on Drupal Answers.