Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
I'm wondering if we use drush_make to create a drupal site, and then we update the make file, say change a module to a new version, can we then update/re-make changes by rerunning 'drush make'?
Comment | File | Size | Author |
---|---|---|---|
#77 | drush_make_remake_04.patch | 22.04 KB | sir_squall |
#23 | drush_make_remake_03.patch | 21.23 KB | dman |
#22 | drush_make_remake_02.patch | 21.22 KB | guddomeNt |
#18 | drush_make_remake.patch | 6.78 KB | guddomeNt |
#1 | drush_make-remake-00.diff | 7.05 KB | smokris |
Comments
Comment #1
smokrisCurrently this is not possible --- drush_make only builds fresh sites; it can't modify existing sites.
I've attached a first attempt at implementing this. The attached patch adds a new command, "drush remake [makefile] [basepath]", which does the following:
(And, of course, the eventual workflow would involve switching the site into maintenance mode prior to doing "drush remake", and running database updates afterward.)
Comment #2
mradcliffeAn initial look: I think it might be better to parse the make file, and invoke the same functionality as the "upc" command in drush (or the update command, but I like to run db updates separately).
That's the approach I was going to take.
Comment #3
smokrisThanks for your feedback, mradcliffe. Could you elaborate on what's better about doing it that way?
(I understand that my method is more resource-intensive, since all the packages are downloaded again, but this can be easily mitigated with a proxy server if bandwidth is a problem.. And I see some advantage in producing a fully fresh rebuild when updating, to fight bitrot and those kitten-murdering core hackers.)
I considered parsing the makefile and performing incremental changes to the existing site, rather than rebuilding it, but ran into a few snags:
Comment #4
dmitrig01 CreditAttribution: dmitrig01 commenteddrush updatecode.
Comment #5
smokrisOh, cool. I hadn't realized #434944: Allow minor upgrades of drupal core was already committed.
Comment #6
smokrisSo... @dmitrig01 and other maintainers:
1. Are you open to including a "remake" feature in drush_make?
2. If so, what are your thoughts regarding how to implement it?
Comment #7
dmitrig01 CreditAttribution: dmitrig01 commentedoh I've thought of 2c... its coming. So yes, I'm open to this, but the current solution isn't robust enough. Let's postpone this.
Comment #8
smokrisOK. Could you elaborate on what would make a robust solution?
Comment #9
drewish CreditAttribution: drewish commentedsubscribing. this would make drush_make much more useful for some projects i'm working on.
Comment #10
dmitrig01 CreditAttribution: dmitrig01 commentedbasically, we need to keep track of what's already been downloaded. but this is going to have to wait until after the 2.0 release. at least.
Comment #11
rjmackay CreditAttribution: rjmackay commentedsubscribing
Comment #12
Steven Jones CreditAttribution: Steven Jones commentedSubscribe
Comment #13
bibo CreditAttribution: bibo commentedSubscribe
Comment #14
TheDoctor CreditAttribution: TheDoctor commentedSubscribed.
Comment #15
franzsub
Comment #16
djroshi CreditAttribution: djroshi commentedSubscribed
Comment #17
aidanlis CreditAttribution: aidanlis commentedsub
Comment #18
guddomeNt CreditAttribution: guddomeNt commentedHi.
I really think this feature should be prioritised up, as
drush make
is otherwise only suitable for initial install - Which is a big shame.I couldn't use the supplied patch, as it will override any checked out working copies. Instead, I've created a new implementation of the same thing, where the make is built in-place - repositories updated, rather than re-exported, if checked out as working copies. Instead of a new "remake" command, I simply added a
--remake
option to the existing make command. Combine it with--working-copy
for to keep working copies in sync.There's clearly still some overlap between
drush make
anddrush upc
, but until this gets sorted out, I think a remake-option is an acceptable workaround.Patch attached. Here's a brief summary of the changes within:
drush_make_tmp_build_path
and updated all hardcoded references to__build__
to use this.drush_drush_make_make
to set globaldrush_make_tmp_build_path
if --remake option is set.DrushMakeProject#findDownloadLocation
andDrushMakeProject#applyPatches
so they will pass with --remake optiondrush_make_download_git
to update working copy, if --remake is set and it's a working copy.drush_make_download_svn
to update working copy, if --remake is set and it's a working copy.drush_make_download_cvs
to update working copy, if --remake is set and it's a working copy.DrushMakeProject#applyPatches
to read and parse PATCHES.txt and skip previously applied patchesI also fixed a few places where
drush_make_error
was called with a single argument.Comment #19
phayes CreditAttribution: phayes commentedsubscribe
Comment #20
dman CreditAttribution: dman commentedI attempted a re-roll of this from #18, as it sounds great
I tried to get that applied to 6--2 dev. Failed to apply cleanly, with a few broken hunks.
Tried to patch by hand but the patch didn't have context info, so it's a bit hard. Also, the tabs/spacing is off, I can't get that in with confidence that the right thing is happening.
Can we get a re-roll? or at least a patch with
cvs diff -up
?Comment #21
achtonSubscribing.
Comment #22
guddomeNt CreditAttribution: guddomeNt commentedHere's a new patch without any whitespace mumbo-jumbo, rolled against latest cvs.
Comment #23
dman CreditAttribution: dman commentedHey, thanks for that!!
2 syntax error typos, (fix attached)
but after that it seems to do the job pretty well!
I note it works more as a version update than a 'revert' (which I sorta expected) But it is still a great tool for the workflow I think we can use.
I'll play with it a bit harder...
Comment #24
guddomeNt CreditAttribution: guddomeNt commentedYeah sorry - I didn't test it after patching. Good to hear you got it working.
What do you mean by that?
Comment #25
dman CreditAttribution: dman commentedI discovered the logic was nice and careful to not re-download stuff that was already downloaded.
I was trying to test it by making some local modifications and seeing if they would be reverted ('rebuild' means that to me - rebuild from scratch).
What I saw was an 'update'. Which is good and all.
I need to work a bit more with it and see how it cooperates with our version control checkouts.
Comment #26
guddomeNt CreditAttribution: guddomeNt commentedI use git, so I haven't really tested it with svn and cvs, but I did make changes and I believe they should work. With a git repo, it will re-use the checkout and simply update you to the revision listed in your makefile (or HEAD). It'll stash any pending changes first, so you won't lose anything.
Yes, perhaps "update" would be a better name. But on the other hand, I don't really think there's any reason to distinguish between the initial make and subsequent updates. One command to do either would be simpler. Currently I have a shell script that wraps
drush make
, to hide the difference.Comment #27
franzI agree with troelskn, 'drush make' itself on an existing installation previously made with drush make should be all you need to trigger an 'update'.
Comment #28
inductor CreditAttribution: inductor commentedSubscribing. This would be a killer feature for drush!
Comment #29
dmitrig01 CreditAttribution: dmitrig01 commentedso basicaly, this is like drush updatecode, with rcs-specific code too?
I'd like to see ab it more than this. I think that the code should be modified so you can make 2 files on top of each other, and when you maek the second file, it will overrwrite the versions in the first.
Comment #30
inductor CreditAttribution: inductor commentedExacty. We`re developing a bunch of modules in our svn repository, and I`d like to be able to update them as simply as with plain 'drush updatecode'.
Comment #31
dman CreditAttribution: dman commentedYeah, I've been developing our workflow. ..
Using svn exports from our own repositories, pointed at by the makefiles. That is so far being very rewarding.
I was planning to try to use remake to 'freshen' those distributions inline. That's not how it's working right now for me.
I'll be going with periodic full re-downloads of everything it seems. Plus some work with svn checkouts in the middle.
I'll keep pushing the corners and see what I really need.
Comment #32
guddomeNt CreditAttribution: guddomeNt commentedUnless I misunderstand you, that is what this patch will do.
Comment #33
dmitrig01 CreditAttribution: dmitrig01 commentedhm, i may have misread -- i haven't actually tried it out, sorry for the misunderstanding. I will look into it soon. for now, i'm quite busy w/ schoolwork
Comment #34
inductor CreditAttribution: inductor commentedAny progress so far?
Comment #35
dmitrig01 CreditAttribution: dmitrig01 commentedOk. I looked a little bit more about the patch. What I don't really like about it is the fact that it's modifying the download library. The download library is there to download, and do (nearly) nothing more -- and it's quite good at what it does. If possible, it should use the download library to download the newer versions of projects, but I don't think it should actually be modifying the download library,
Comment #36
smls CreditAttribution: smls commentedsubscribe
Comment #37
Wim LeersSubscribing.
Comment #38
willieseabrook CreditAttribution: willieseabrook commentedsubscribing
Comment #39
drnikki CreditAttribution: drnikki commentedsubscribe
Comment #40
crantok CreditAttribution: crantok commentedsubscribe
Comment #41
perandre CreditAttribution: perandre commentedsubscribing!
Comment #42
alozie CreditAttribution: alozie commentedsubscribe
Comment #43
areynolds CreditAttribution: areynolds commentedsubscribe
Comment #44
mxmilkiib CreditAttribution: mxmilkiib commentednt - sub
Comment #45
johngriffin CreditAttribution: johngriffin commentedsubscribe
Comment #46
ablevine1 CreditAttribution: ablevine1 commentedsubscribe. This seems like a very key feature, especially if you have staging and production environments whose module versions you would like to keep in sync
Comment #47
izmeez CreditAttribution: izmeez commentedsubscribing
Comment #48
Leeteq CreditAttribution: Leeteq commentedSubscribing.
Comment #49
Zach Harkey CreditAttribution: Zach Harkey commentedSubscribing (You guys are my heros)
Comment #50
Anonymous (not verified) CreditAttribution: Anonymous commentedsubscribe
Comment #51
Grayside CreditAttribution: Grayside commentedGoing by #820954-35: Update core and modules with Drush make, a solution which adds logic to the download library is not acceptable.
Comment #52
mlncn CreditAttribution: mlncn commentedI'm subscribing/bumping, but really i'm stalking Grayside.
So it's better for the patch to do exactly what it's doing, but duplicate some code from the download library, rather than modify it? Am i understanding that correctly?
Comment #54
greg.1.anderson CreditAttribution: greg.1.anderson commentedI am ambivalent about drush remake. On the one hand, I might use it in drush patch #1078108: Drush issue queue commands; on the other hand, I also wonder if it is not better to just do the remake workflow in two-steps:
The reason I am interested in this is not to turn drush make into drush deploy, but rather, I am interested in simplifying what I commit to my repository for my Drupal sites. It just seems wrong to mirror the releases already archived on d.o. when I could do better and make an easier-to-review history by generating and committing a drush make file, and leave the actual module contents out of my repository.
For modules that are patched, dog looks like it is really the way to go. I am trying to figure out how to make a nice workflow involving the hacked module to automatically detect modified modules, and then generate a makefile that does the right thing vis-a-vis the patches. If the patches come from some issue queue (which you could detect if the module was installed via drush make or patched via drush patch, then just putting a "patch" directive in the makefile would be simplest (and would preserve the source of the modification). For modifications that do not come from a patch, then we could get on the dog sled and make a local repo to store the changes to, and set up our makefile to pull from there. I'm experimenting with a modified variant of drush generate-makefile to do this sort of thing, but have not come to resolution on exactly how it should work or where it should live; ergo, I go slightly OT on this existing thread. Sorry about that. To tie it back in, my point is to explain the workflow where I find drush remake to be interesting & bring up alternate and related projects in that same workflow.
Comment #55
marji CreditAttribution: marji commentedsubscribe
Comment #56
Robin Millette CreditAttribution: Robin Millette commentedsubbing
Comment #57
Cyberwolf CreditAttribution: Cyberwolf commentedSubscribing.
Comment #58
Bußmeyer CreditAttribution: Bußmeyer commentedSubscribe
Comment #59
likewhoa CreditAttribution: likewhoa commentedSubscribed
Comment #60
helmo CreditAttribution: helmo commented[ Powered by #1115636: Issue Macros and Templates - Drush Make]
Drush make is being merged into drush core (discussed in issue:#1310130: Put drush make in drush core)
This means that the issue queue is also moving. The Drush project has a component called 'Make' for this purpose.
We would like to take this opportunity to leave behind old/obsolete issues, allowing us to focus on a stable make command in core. E.g. one of the major tasks ahead is making more use of the Drush core code for handling downloads and decompression.
If you feel that this issue is still relevant, feel free to re-open and move it to the Drush queue.
More information will be posted on the drush_make and drush project pages.
Comment #61
smokrisThis issue is still relevant, and many users are interested in seeing it happen.
Still waiting on a plan from @dmitrig01.
Comment #62
michaek CreditAttribution: michaek commentedI agree that this issue is still relevant, and it seems like a pretty important feature of drush make. It seems to follow philosophically from "bundle install" in the (Bundler-using) Ruby world. http://gembundler.com/bundle_install.html
Comment #63
moshe weitzman CreditAttribution: moshe weitzman commentedFWIW, I am not too interested in this. We are working on integrating the download classes of make so they reuse the caching features of pm-download and pm-updatecode. That means that re-make is going to be a very fast operation. From there, you can diff or rsync or do whatever operations you want. You an even use shell aliases to give short command names to those combo operations. I'm not yet seeing a clear use case for this, despite this being the 64th comment. I'm not a big Make user so I might be missing how folks plan to use this.
Comment #64
greg.1.anderson CreditAttribution: greg.1.anderson commentedI agree with #63, and propose that we set this to "Won't fix".
Comment #65
michaek CreditAttribution: michaek commentedI can see that it might not be an important use case, but it seems to me like there's value in being able to re-run your makefile in order to add new configuration. I know it's possible to always run make with a new directory, keeping your existing code by rsync-ing it in, but that doesn't feel right to me.
Looking to Bundler as a pattern seems worthwhile - there's a single place where a developer configures dependencies and their versions, and running `bundle install` get the new goodies added to your Gemfile. I think there's some confusion in this issue on rolling in updates automatically (which the above patches seem to be about) and being about to re-run `drush make` to reflect changes in the makefile (which is what the initial issue seems to be about).
I may be missing something, but it also seems like allowing make to re-run is as simple as disabling the check for an existing directory in 319-322 of make.drush.inc, or running `drush make` on '.'. I probably don't know enough of the reason Drush is making that check - is there a reason not to run make on an existing directory?
I, also, am not a big user of Drush make, but I'm trying to work out how to use it sensibly. Thanks!
Comment #66
michaek CreditAttribution: michaek commentedIf `drush re-make` is meant to do what I'm describing, then whatever I've said here is pretty worthless. Searching for "drush re-make" pretty much just brings me to this issue, though, so I don't have much perspective on it other than the above!
I'm pretty excited about the caching. Definitely looking forward to it.
Comment #67
smokris@michaek (comment #65) :: running make after disabling the check-for-existing-directory doesn't handle the following situations:
Comment #68
greg.1.anderson CreditAttribution: greg.1.anderson commented#67.1 is kind of hard to solve through drush re-make. In its normal mode, drush make will merge in with existing projects on the site; this is kind of necessary if recursive make is to be used.
#67.1 is easy to solve per #63; re-make the whole site from scratch in a clean directory, then you can rsync it with --delete if that's what you want to have happen. (Beware of your 'files' directory.)
We could also implement
drush re-make
as a shell alias that did those two operations. Trying to have a re-make command that "fixed up" an existing directory structure seems like a really bad idea to me. If the operation failed in the middle, your site would be trashed. Hopefully folks will only re-make scratch sites, but you never know.Comment #69
mradcliffe#67.1 is dirty as hell. You shouldn't have to rebuild your entire site if you're updating a module. That doesn't fit a rapid change model if you broke something.
Here are some valid use cases re #63 and #64:
A better, more ideal use case:
Comment #70
marianov CreditAttribution: marianov commentedsub
Comment #71
helmo CreditAttribution: helmo commented[ Powered by #1115636: Issue Macros and Templates - _default]
@marianov: Please do not subscribe anymore. Just click the Follow button next to the issue summary.
Comment #72
michaek CreditAttribution: michaek commentedI think the use cases in @mradcliffe's comment (#69) are well-described. Out of curiosity, are folks here familiar with using Bundler in Ruby development? I really think it's a pattern that's worth looking to for inspiration.
Regarding comments about what "re-make" does and doesn't do - is there someplace we can see either discussion about the scope/functionality or the current source of re-make? As I mentioned in #66, searching for it just leads me back to this thread.
Comment #73
mradcliffe@michaek: Maybe a new topic on the Drush working group on groups.drupal.org. Now that this is in the Drush issue queue.
Comment #74
joelcollinsdc CreditAttribution: joelcollinsdc commentedcross posting this here, i think there is a use case for a feature like this; or else i'm missing something. faster build times are awesome, dont get me wrong...but it doesn't seem to help a common use case.
http://groups.drupal.org/node/207678
Here is my current workflow:
1) download an install profile (i'm trying out openpublic) from git.
2) modify the distro.make (its called something else in OP) so you can build from your local repo.
3) drush make to a folder called build with the --working-copy flag
4) install drupal, do work, make changes to code as well as openpublic.make (for module changes and such)
Now, here comes the crappy part...
5) rebuild with drush make to take advantages of the new modules you added to the openpublic.make file
6) copy in your sites folder to the newly build folder
7) continue working
Has to be a better way, right? This is even more frustrating because you have to keep a repo inside the build as well as globally.
For reference, this is my folder structure
Comment #75
jsagotsky CreditAttribution: jsagotsky commented+1 for #69's ideal use case. That describes exactly what we're trying to do. I'm going to give the make-then-rsync alias a try, but would prefer something a little less hackish.
Comment #76
Grayside CreditAttribution: Grayside commentedI would use the first use case in #69 as part of a deployment script. post-receive hooks are awesome though :) "remake" might be an alias, but not the actual command name.
The alternative to this feature request is probably to run an expensive, complete drush make process on a server somewhere and use that as a deploy source. Avoiding impact on D.O. might mean looking into creating a local drush dl/git clone cache.
Comment #77
sir_squall CreditAttribution: sir_squall commentedHi,
I have tried the #23 patch with the last version of drush_make, and i have some conflict during the patching.
Hi have fixed this conflict and redo the patch, is exactly the same witouht the conflict.
Comment #78
greg.1.anderson CreditAttribution: greg.1.anderson commentedFor the record, it is still my opinion that this issue should be closed 'will not fix'. Drush re-make implies #67; if you don't remove items from the site that are no longer in the makefile, then you are not really doing a re-make, which could lead to confusion an use error. If you do remove items, you risk bigger problems.
I think that the comments in #69 are on the right track vis-a-vis what sort of use cases are desirable; however, I think it might be more fruitful to think about modifying pm-updatecode to take a makefile. pm-updatecode could then use the specific version number supplied for each project in the makefile, or default to its usual behavior for projects that do not have a specified version number, or that do not appear in the makefile at all.
Comment #79
DamienMcKennaAn additional use case that pm-updatecode does not handle: automatically applying patches. I haven't read through the code to know for certain, but I have a feeling that it might be easier to handle as an argument for "drush make" rather than running it as a separate command, bicbw.
Comment #80
DamienMcKennaOh. BTW you can currently do this with Drush 5:
That updates everything in-place, kinda the way that everyone wants it to work.
Comment #81
DamienMcKennaOne thing to watch for when using the 'make' command indicated in #80 is that it will, by its very nature, remove many files, e.g. custom modules, features, themes, etc that are not loaded via lines in the make script but which only exist in the repository. Do you think it would be difficult to get it to not remove these files?
Comment #82
dman CreditAttribution: dman commentedThat super-dangerous quirk has now been fixed in dev (though still exists in todays stable)
#1539076: drush make overrides modules folder with downloaded modules
Comment #83
DamienMcKenna@dman: Thanks for the update, yay contributors!
Comment #84
DamienMcKennaPer #82 this can be marked as done.