[META] D7 port of media_mover module

Porting to D7 won't happen until the first 6.2.x release happens- unless some people are interested in helping getting the 6.2.x code stable now so that process can begin.

D7 begs some larger architecture questions which likely means that there would be a 7.3.x release after the current code base is ported.

The D7 version is dependent on the 6.2.x version getting finished, so if you're interested in speeding things along if you could start testing the 6.2.x version that would be great. If you're not so familiar with coding, giving good bug reports (screen shots, urls, configuration info) would be very welcome. I prefer to get actual code fixes, but at this point anything can help. Realize that the 6.2.x is still under heavy development. I could also use somebody closely watching the issue queue for 6.2.x looking for duplicate issues etc. And if you can help recruit people to work on it, that'd be great too!

IMHO first get an 6.x 1.0 RC out of the door then a release.

Since people still can file a defect against an RC or final version anyhow. The 6.x 1.0Beta7 is stable enough to get released. And since Drupal 7 will be out soon D6 will get maintenance fixes only so nothing can break a D6 release anymore e.g. due to D6 changes.

@Andy B

Don't have any know-how on Media Mover, but I think I can answer these without:

1. What modules would need to be installed to give a useful test? If you're checking from scratch, start off with the most basic you can (in dev), and play around until something bad happens. Check issue queue to see whether anyone else has mentioned it. Flesh out details if it exists (maybe "I'm running the bare essentials of X, Y, and Z modules", and I'm still getting this on shared hosting with Siteground" or whatever.) The more detail the better. You could also start with posted issues, and try to duplicate them with minimally enabled modules, then flesh out details as above. :)

   Oh, and I didn't do the best job of it here, but formatting is your friend! (Headers, lists, bolding -- the whole she-bang)

2. How do you watch the queue for duplicates and mark them? If a root problem seems to be identical in two issues, close one (usually, the one with less info) by marking as duplicate, while making reference to the new "official" issue. Refer back to the closed duplicate in the official thread. Feel free to summarize in the official if there's anything particularly useful in the duplicate. If you're unsure whether sometime is a duplicate, just bring it up and ask others before changing the "Status" (in the "Post new comment" section whenver you submit a post).
3. If I end up being somewhat familiar with D6/7 code, do you want patches against D6.x-2-dev? Yes :)
   I'd say let the folks who are most familiar with the 1.0 architecture work on that, and you'll be able to do more good working and learning on 2.0. Correct me if I'm wrong, maintainer-folk.

Hopefully this helps! Cheers

Yes- 2.0 is all my efforts are . 2.0 will be ported to d7 once it is stable and then there will be a 3.0 version which utilizes some of the new d7 file handling features. This is my plan any way. But I'm really really really looking for help with 2.0. Right now it's extremely close to the first alpha. I need to start getting some real world testing going on so that I can get bug fixes rolling.

subscribing

I think the real question is how do we port it without api change.
If it's inevitable that there will be api changes then we might as well make the significant changes that need to be made to take advantage of d7 api(and work properly in general). Don't break things with api changes over and over.

The significant changes for 2.x media mover to port to d7 are mostly sql changes. 2.x uses db_write_record in most places so that ports pretty well. The only major work will be in the modules that are doing file selection from db records- those can be fairly complicated queries.

The 3.x version that I mention above will be lots of work. It will also mean some pretty damn awesome functionality, but 2.x has to get out the door first. And then the d7 port. And then....

...we pop open the champagne ;) you deserve it. Thanks already for the effort, super!

arthurf: it would be interesting if something like maestro could be used for the 7.x-3.x branch to create a visual workflow of the process.

@jackinloadup - yes, could be pretty interesting. I'm not sure how extensible their setup is but it seems like ultimately it could be a front end for the API itself- note that all of the UI for media mover lives in media_mover_ui, so there could be alternative front ends.

That being said, I'm going to let someone else be in charge of that one for now :)

subscribe

subscribe

Subscribe.

subscribe

subscribe

subscribe

Seems like there should be a media_mover codesprint sometime. or however many are needed to crank out both versions 2 and 3 :)

subscribe - would be great to determine a game plan now :)

subscribe

sub

subscribing

+1

Subscribe

Maybe the best move at this point would be to get some various experimental sandboxes for D7-mediamover set up. That way the basics could get stubbed out to begin with :)

Subscribing.

Subscribed!

subscribe

Subscrizzle.

subscribe

subscribing

subscribe

subscribe

subs

There is a d7 port started but it's going to take a significant amount of time to get to an alpha. It's based on the architecture of the 6.2x branch but with some significant improvements to some of the configuration handling. The file handling remains a major question

That's great @arthurf. How about making that known on the project page, as well as providing a D7 -dev snapshot. I suspect this will help raise awareness and encourage testing and feedback.

@mrfelton well, we're not even talking alpha code yet. I'm working through the configuration add/edit process right now- I haven't even touched the actual file handling yet. There are very large scary questions about stream wrappers that I'm happily ignoring right now.

subscribe

There is a 7.x dev branch on the project page. At this time only mm_dir is working as that is the simplest module that I could start testing with. If you start testing be aware that you should not expect it to work. Any bugs being filed should have patches, code, or cookies and milk. In all seriousness while I'm happy to see the interest here to get the major piece working (file field integration) is going to take time. Help is greatly appreciated. My next step is getting ffmpeg_wrapper rolling- note that that already has a 7.x release.

Arthur,
Could you please tell me what the plans are with regard to the D7 port? What is the likelihood there will be an alpha version sometime within the next six months, say?
Thanks very much for a great module.

There is actually quite a bit of code in the d7 branch already. I simply haven't had time to work on things due to other obligations for the last lots of months. I'm also wondering about more seriously trying to join forces with media derivatives. I welcome any help folks have to offer.

Arthur, when you say

"I'm also wondering about more seriously trying to join forces with media derivatives"

I am not sure what you mean by "media derivatives". Are you saying (for example) you would like to have the media_youtube module handle more of the youtube-related processing?

When you say

I welcome any help folks have to offer.

I am wondering whether it might be a good idea to publish a list of the "chunks" of work that need to be done, and let people sign up to handle some of the chunks.

By media derivatives I mean this module: http://drupal.org/project/media_derivatives It has some concepts that are similar to Media Mover and leverages CTools to do some of the heavy lifting of configuration management. There are some shortcomings that I think it has but it might be a good place to share some code.

As per asking for help- as of right now the configuration build is mostly functional. The configuration runner is also mostly done. It really comes down to converting some of the conversion and storage tools, many of which require some significant overhaul (ftp, s3 in particular). Testing and converting modules are really the main things I think- maybe writing some tests to ensure that the API works as spec'd. Actually that one would be huge.

Ultimately the file handling in d7 (which is vastly improved) has an annoying shortcoming that prevents me from leveraging Drupal files as the containers for Media Mover files (unique key on the URI- perhaps this has been removed, I should check). Because of this I feel somewhat in architectural limbo- while Media Mover does it's own thing, I don't like that it has segmented off its own concept of files- which also means it does not play well with things like media_derivatives.

So, I'm not sure if that helps clarify. I'm interested in continuing work on it but I just haven't had the time to really pay the attention that it needs.

Just as an update- I have the end to end api working. There are simpletests for the major components of it. There are probably still some hidden bugs in it but it's functional. The mm_dir, mm_antiword, mm_ffmpeg, and mm_compress are working.

That is really good news. Can't wait mm_youtube !

I can confirm that the module is indeed working (I even have been able to add Dailymotion Cloud storage.

I'll try to remove some UI bugs (API changes, things like that) and propose some patches.

Awesome!

I ran coder upgrade on the whole module directory, then sifted through a bunch of @todo's (db updates, coder upgrade notes), updated a bunch of theme_*() calls, hand-scanned other api changes, then tried loading the modules and running and chasing WSODs, errors and warnings. I have to stop at this stage and would be a pity if this work goes unused. I will split into core and separate sub-modules and post patches to this issue. Would be great to see those committed to git.

Here's the first one - core (media_mover_api, media_mover_ui, tests).

Next patch is for submodules mentioned in #48 (mm_dir, mm_antiword, mm_ffmpeg, mm_compress)
It is mostly coding conventions.

Third patch is for mm_node and mm_utilities. It has a lot of API updates and bugfixes, but does not make them work yet.

Last patch of the bunch - submodules (mm_export, mm_ftp, mm_getid3, mm_imageache, mm_mailhandler, mm_s3, mm_token, mm_views)
I did not try to run these modules so no bugfixes. Most value here is in DB API updates and items highlighted by coder upgrade.

@iva2k - wow- this is quite a bit of work. For what it's worth mm_node doesn't really make sense as a module anymore- it needs to be replaced with field api calls and entity handling. The mm_utilities needs to be refactored in some significant ways and ideally I'd like to move some of that functionality into the api module and just expose the configuration stuff in a separate module.

Would you mind creating a new issue for the core code cleanup- it seems like that 'd be the best place to start implementing your work.

No problem.
I've created #1920280: Media Mover core D7 code fixes, coder upgrade, drupal API and coding conventions
I copied all 4 patches there as submodules should patched to keep them in sync with the core changes (mostly due to the use of new MMA_CONFIG_PATH constant).

To close this issue I propose to track D7 submodules development here:

• #1920280: Media Mover core D7 code fixes, coder upgrade, drupal API and coding conventions
• #1926464: mm_fields for Drupal 7
• #1926420: Merge mm_dir, mm_compress, mm_imagecache, and functions from mm_utilities
• #1938558: Update mm_export for D7

It would also be great to have a "select from a view" file selector. A nice to have but a really nice way to build flexible file selection.

I'm done with heavy patching of this module and I've submitted all my patches to the issue queue. Since there are some outstanding "needs review" issues for D7 that create a mess of patch dependencies, here's my full code snapshot for reference. I also noticed that few changes and bugfixes submitted in patches were not committed in full, so this snapshot can be used to dig them out.

One more code snapshot - this one handles uid through the media mover process and supports mm_fields to set file ownership selected from target uid, source uid or source file uid (based on step settings). Also uses field widget settings for file_directory (tokenized) to save file to where the field is configured to save its files. I won't roll it as a patch as it won't apply anyway and depends on 4-5 other patches in the queue.

Seeing as there's a (party) working version of the module, I suggest closing this issue and focus on identifying what would be left to release 7.x-1.0: #1939252: META: Plan for Media Mover v7.x-1.0 release

#62; agreed. Closing this issue, continue over at the META issue and/or in new feature requests.