Move creation of db tables to mapi module

I agree with that idea. I'll do that as soon as I can.

Ok, I'll be updating soon the mapi, with both the mapi_admin section, and install tables.

I've kept them out of the mapi.install, since some developers may only want the media display functionality. It seems that if people require more out the MAPI module itself, they can install the tables.

I'm up in the air as to whether to force them to be installed regardless, mainly due to the integration that exists within MAPI, and the media module.

I'm tossing up whether to completely remove all traces of the media modules capabilities from the mapi, in which case the database tables would go back to the media module. If you could see a clear way through this issue, I'd be more than happy to hear from you.

casey that sounds like a good framework. this would allow other modules like biblio (reference manager) and galleries and such to be set up to handle media together much easier with standard hooks but not be tied to any one thing. I like it.

Funny thing is, last night I came up with the same structure, because it was looking a little funky as it stood.
The only few things I didn't think of are the hooks section, for defining ext, types, programs, transforms, and presenters.

I've attached a somewhat current version of how the mapi module looks, I'm not going to update the CVS until I've got both the mapi, and the media module functioning together.

It's the one off my machine, that I'm working with, so it contains a few "extras" that will not make it into the CVS.

I'd look forward to it, however, I'm almost completed the whole thing myself anyway, including some bug fixes that I discovered while going through the code. This includes a solution for the non-saving of program data. Also, I've created several hooks, which seem to work well. I'd be interested to see what you think of these.

 hook_media_extensions($op, $name, &$data, $read)
  - list
    This provides a list of the extensions this module can handle, and what their mime-type
    is.
  
  - metadata : $filename, &$metadata, $read_write
    Modifies the metadata within the filename, if write, or adds the metadata
    to an the array if read. Returns FALSE if unable to do either.
    
  - dimensions : $filename, &$dimensions, $read_write
    Works the same as the metadata, except for details about the file.
    whether including possible height/width.
    
  - mime : $ext
    Returns the mime type for the extension.
    
  hook_media_types($op)
  - list
    List of the types defined by the module
  
  - info
    Info for a node to be created for the type      
    
  - configure : $type, $edit
    Configures the details for the type....
    
  - validate : $type, $edit
    Validation for the configure
  
  - save : $type, $edit
    
  
  - extensions 
    This returns a list of extensions for a particular "type".
    These are keys for the mime-type they represent.

  hook_media_programs($op, $program, $edit)
  - list
    This provides which programs are handled by this module.
    
  - path
    Attempts to locate the program path.
    
  - configure $program, $edit
    This allows configuration of the program
  
  - validate : $program, $edit
    
  - save : $program, $edit
    This saves the configuration for the program, allows modification of details before they

    
  hook_media_transforms($op, $transform, $edit)
  - list
    This provides a list of the transforms handled by the module, with a list of the extensions that
    the transformation can handle.

  - info
    This provides specific "info" on the transformation, such as a description.
    
  - configure : $transform, $edit
    This allows configuration of the transformation, basically providing the form for which the
    transform is going to be applied to.
  
  - validate : $transform, $edit
    This provides for allowing validation of the form details for the transform.
    
  - save : $transform, $edit
    This allows intervening before the details of the transform are stored in the database.
    
  - transform : $transform, $edit
    This performs the transform. The $edit contains a structured array which should always have the following 
    information located within: 'source', 'dest', 'temp', 'state'.
    'source' must contain a valid filename, located on server, to which the transform is going to be applied.
    'dest' should be updated with the filename that has been produced. If blank, or non-existant, file is
    assumed to have failed the transform.
    'temp' provides for a temporary working location.
    'state' is the state it is in, if there a multi-state transformation that occurs. (State is empty '' when
    on first iteration).
    
    Needs to return 'processed' === TRUE, 'failed' === FAILED, 'state' === 'string', 
    states longer than 32 characters are truncated.

  hook_media_presenters()
  - list
    This provides a list of the presenters handled by the module, which also includes a list of extensions
    that the presenter can handle.

  - configure : $presenter, $edit
    Allows configuration of the presenter
    
  - validate : $presenter, $edit
    Validates the configure
    
  - save : $presenter, $edit
  
  - display : $presenter, $edit
    This generates the HTML for a presenter
    
  - playlist : $presenter, $playlist
    This generates the playlist for a particular presenter.

This is something like what I worked out would be necessary to solve these problems. I'll post the updated version of the MAPI I have in a couple of hours (hopefully).

Try this for size, it's still not completed in the mapi_derivatives section, and I've kept the filter in mapi, since it's not really related to any specific media, it allows embedding of media (i.e. youtube and company).

I've revamped the admin area, to really deal only with those things which have be specified as MAPI related. In effect, I've tried to implement as much as possible as previously refered to.

This is thoroughly untested. But I wanted to give you guys a glimpse of the directionality, since you may have something more to add.

Ok, so I made somewhat the changes you mentioned, I'll go through and debug tomorrow all these changes.

I've dealt with quite a few of the things you posted in my own debugging, but there were some really important ones I'd missed.

I think the idea of the workspace is quite effective, but I'm not sure how, short of getting modules that "need" to use a seperate workspace (not including things such as mapi_display), to define their own areas.

What I propose, is that there be a general use workspace (i.e. for those that don't use the set_workspace function), which will allow functions to not have to worry about it, unless they require a separate workspace.

Soe the proposal is something like:

mapi_set_workspace($module, $paths)
* which defines where the module can utilize paths.
- $paths needs to be an array, in which named keys provide the indicator for which path is available.
  This allow for specific temporary directories, or deliberately established directories to used by a function
  Simply by referencing a key. These keys do not need to be strings. (i.e. a user directory system based
  on UID).

mapi_get_workspace($module = null)
* returns the module name for the current workspace, if null, and the entire pathset if module specified.
  returns false if non-existant.

mapi_use_workspace($module = null)
* allows for stored workspaces to be used by modules, with a value of null returning to the 
   general workspace use that there is available.

mapi_get_workpath($key)
* allows the specific request for a path indicated by key, relative to current workspace defined.
  This should always returns an actual path, and failing finding the path, will return the 
  file_directory_temp() result. 

The general workspace paths are:
* file_directory_path()
* file_directory_temp()

With this style of coding, I think we can handle almost all the situations where it is necessary to shift. Unfortunately it does mean that if a module is sloppy, and forgets to change out of it's workspace, then there are problems with potential clashes coming in.
I think this approach will allow for specialized work spaces, as well as functions not having to worry about where it is they are working.

Also, I don't see it as a problem that functions are required to call two more functions.

mapi_use_workspace('module_name');
mapi_use_workspace();

If they require a specialized workspace, then they need to be prepared to clean up after themselves before leaving the function.

The only other method is to use complicated processes, and excess parameters for functions, hooks, etc. This method I consider the cleanest, and most likely to be beneficial to those module designers that require such functionality.

Yes, these are for the first stable release, I'm rewriting the media module now to take advantage of what I consider to be very good, and important changes.

I know that will inconvenience people, but it is after all a "development" module.

If you find other bugs with the one I posted, let me know. I'm currently trying to tie down as much as possible, so that I can get the stable releases out there. It means a lot of testing on my part.

Also is there some way to force MySQL to use the InnoDB? I think all the media module tables should be using those.

Ok, so here is how it stands currently, including the implementation of the workspace idea.

As you can see, the basics should function for dealing with the mapi_directory_locations is to use the mapi_get_workspace() concept.

Right now, I'm focussed on getting the rewrite of the media module done, so I can really get testing going on my machine, to solve some of the problems currently coming up.

I'd also look forward to seeing what you have for the CCK integration.

Forgot the file.

You'll get errors regarding the InnoDB engine is not installed, so the table won't be created as far as I know.

In general, like with the rest of drupal optimization (node table etc), I would say as part of the install you state that for performance, xyz tables should be converted to InnoDB format and leave it up to them with a link to the MySQL site on converting a table to InnoDB http://dev.mysql.com/doc/refman/5.0/en/converting-tables-to-innodb.html

@konfuzed:

It might be possible to have the ALTER TABLE ... ENGINE=INNODB... as a database query that occurs after the others, thereby
switching it over if it has the ability already. This could be done on install. It would give errors for those tables that do not support it, but it should be possible to check if there is InnoDB installed for a given installation.

@casey:

Thanks for being rigorous. I was sure I changed the mapi_admin functions, they seemed alright on my machine. However, I was missing the others. Regarding the hooks for item and folder, these still exist and I don't see any reason to change the way they function.
I did however create a seperate hook for doing the menu system, that is the same as the way that mapi_admin does it.

function media_invoke_menu($op = null, $item = null);

called in media_menu()

/**
 *  Implementation of hook_menu()
 */
function media_menu($may_cache) {
  if ($may_cache) {
    $items = media_invoke_menu('cache');
  }
  else {
    // are we dealing with the foldering system?
    if (strpos($_GET['q'], MEDIA_PATH .'/folder') === 0) {
      if ($folder = media_folder_load(arg(MEDIA_ARG + 1))) {
        $items = media_invoke_menu('folder', $folder);
      }
    }
    // are we dealing with the media system?
    elseif (strpos($_GET['q'], MEDIA_PATH) === 0 && ($media = media_item_load(arg(MEDIA_ARG)))) {
      $items = media_invoke_item('media', $media);
    }
  }
  return $items;
}

Here are the changes from last weekend. It contains both the MAPI and the MEDIA module.
I haven't really converted everything over to the new mapi system yet, since I wanted to get the media module into a functional setting.
I've changed quite a bit as to HOW the media module functions, but not the API within it.
You'll be able to use all the functions per normal, with some of them have specifically override capabilities for the workspace that it uses.
It's missing a few of the functions, but they'll come soon.
I've seperated out the media section, and the foldering section, so that the foldering section can be done through the menu system.
This works quite well, and gives the impression of actually browsing a foldering tree.
If you've got suggestions on how to improve the user experience here, I'd be very happy for the comments.
Right now, it's just the media module, I'll rearrange the others on the basis of the work on this module.

Ok, here is a somewhat more debugged version of what I just provided.
It's got almost the full media handling for the main module.
I've updated the mapi to be at the same state as the current one on drupal.
I need to do some testing on the playlist process.

Ok, this contains almost a fully debugged media module, and the beginnings of a media_attach module.
I've added to the media module, an autocomplete process, which makes things very easy to find, especially with the current method of add attach media. (I'm thinking that it should be possible to extend this somehow, so that it could include potentially a gallery, but currently I'm limiting it to the fields presented).

Also, I had to do some serious debugging of the MAPI module, since there where problems moving, renaming, and deleting files. This was creating random disappearances, but I think I've now got it covered. I'd be very appreciative of any debugging comments so that I can get this out, hopefully by the end of the weekend coming up.

Ok, so here are the modules as they currently stand, I did some work to attempt to get the mapi_derive functioning, and somehow integrated into the process as a whole.

I modified mapi_display, since it is possible to use the drupal 6 method of passing options in an array, which allowed for definition of the 'context' => 'teaser' idea, allowing a simple call to the mapi_display with a filename, and the 'context' => 'somewhere', that returns a different filename given the context (this was to simplify the usage of derivatives automatically within the system as a whole).

I'm still testing, and I haven't quite experienced some of the problems that you mentioned, however, the bugs you've mentioned, I believe that I solved.

Other than that, I've had a busy week, so not as much done as I'd have liked.

Thanks for picking those up.

The question I have about the workspaces creating the directories, is, how is that regulated?
Should we just allow any module to specify a directory and it gets created? Or should the module itself check that before it returns the path in the hook_mapi_workspaces?

I'm a little vague as to how workspaces function, but I've tried to implement them as best as possible.

I look forward to all other suggestions you have.

Ok, so I'm implementing the suggested media API functions, as well as fixed the _media_folder_cache bug.

I haven't done much testing, due to other web related incidents. I'll get back to testing and updating hopefully in tomorrow.

Ok, so here are the modules as they stand right now.
This includes changes to the database tables, to incorporate the standard metadata from #230079: Default information fields and metadata, or at least a good enough version of it.
I also fixed up "hopefully" the ffmpeg, which I have not tested, as well as quite a few little bug fixes.
Please comment ASAP, as this will be going up as HEAD in the next couple of days.

I've added another MAPI module, called mapi_layouts, so that standard HTML layouting options are available, as themable functions. This will be somewhat of a necessity for the display of attached files, etc.
I've redone the filtering system, to allow for media_attach to define a whole series of filtering tags, which will include 'image', 'video', 'audio', 'slideshow', 'audio-playlist', 'video-show'... with all the appropriate options to match. I really have to seriously test this.

Uploading newer versions, since the ones uploaded had some fairly bad bugs in them... tiredness won out.

Uploaded into drupal CVS, should be appearing sometime soon. I'll get working on upcoming issues as they are presented.