This is spin off from #1151286: Proposed Timeline for D7 Port. In the current proposal, it is suggested that the D6 schema will need to be changed to match the D7 schema, this implies writing update functions to move the data around so Neologism D7 can understand it. This can be pretty tedious, and since Neologism has the export/import vocabulary functionality, I wonder if this could be used as a migration mechanism. In other words, instead of spending time writing update functions to match the schema of external module (evoc, RDFx, etc..) whose schema might change over time, this effort could be spent towards polishing the import/export functionalities, which are very important features useful beyond the migration process.

One thing to check is to make sure all the data from the vocabularies are exported properly, if I recall properly the last time I worked on Neologism (a few years back now!), there were still a few things which were not in RDF (like some of the documentation), but that might have changed. If it's still the case, maybe it could be added to the export data so each vocabulary is fully described in RDF (no data Neologism specific).

CommentFileSizeAuthor
#7 Import external RDF vocabulary Intelligent system Prototype - Mozilla Firefox_2011-05-14_06-22-46.png17.18 KBmayankkandpal

Comments

cygri’s picture

cygri commented

There are things in a vocabulary that should not be in the import/export, and these will only become more numerous as we go along (e.g., access permissions, configuration about which parts of the documentation to show or to hide, layout of the diagram). So I don't think it's a feasible option to do version upgrades by exporting and importing again.

An update function that runs SQL queries to do the schema changes is indeed tedious, but it's less brittle and it's the Drupal way of doing things AFAICT.

scor’s picture

scor commented

I didn't mean to include site permissions and other non-neologism settings + data, these would be taken care of by Drupal core or other contrib module (e.g. CCK/content_migrate).

Also, talking about Drupal way of doing things, if there is evoc data to be migrated to D7, it would be the job of evoc-7.x (part of RDFx) to migrate that data (since neologism is building its dependency on the external evoc module).

cygri’s picture

cygri commented

I was talking about Neologism-specific settings that shouldn't be in the RDF output.

As you know, Neologism on D6 uses a combination of CCK and a fork of evoc-6.x to store its data. Having the code for migrating from that to evoc-7.x in evoc-7.x doesn't make any sense IMO.

Anonymous’s picture

Anonymous (not verified) commented

I think I agree that this should be handled using the API instead of direct database manipulation.

In the update functions everything would be the same except that the SQL code would be replaced with a call to the api. It makes us consumers of our own API, which is a Drupal best practice. If our APIs are as solid as they should be, I think that this would actually be the less brittle way to handle import to the new schema. It also means that if Evoc's schema does change, Neologism doesn't have to know about it. It can just rely on Evoc's import API to handle everything during new migrations, and rely Evoc's update functions to update installations that have already migrated.

Any additional Neologism-specific settings could also be handled by the update functions. This may require direct database manipulation, I'm not sure what other settings are involved in Neologism. I don't think that using the API for migrating the vocabulary data itself will affect the way that settings are migrated, but I may be wrong about that.

To do this, we would need to include and rename the D6 export code in the D7 module. The renamed export function would then be called right before the call to the import function. These could be included in the install file as "private" helper functions (private with quotation marks because they would just use the convention of starting with an underscore in order to designate that they are private).

This will only work if the module is enabled during the upgrade. The module upgrade handbook on drupal.org used to advise people to disable modules during upgrade. However, this recommendation was updated in the handbook somewhat recently since some modules use their own functions in the update functions (the person who complained about this is merlinofchaos, creator of Views and Panels).

cygri’s picture

cygri commented

So the idea would be to do the following in the Neologism update code:

* use SQL queries and whatever else is needed to migrate Neologism settings
* use a version of Neologism's D6 export code, plus the evoc D7 import API, to migrate the actual vocabulary data (class/property stuff)

Uhm, does evoc currently have an import API? Where can we find documentation for that?

Anonymous’s picture

Anonymous (not verified) commented

It has been a long time since I've touched that code, so I had to look back. It looks like evoc_import_vocabulary currently only supports importing from the Web. It seems that I did have the idea to add the import from file functionality that is a part of Neologism, but that that feature hasn't been added yet. That would need to be added for this to work, but that would be pretty trivial.

mayankkandpal’s picture

mayankkandpal commented
StatusFileSize
new Import external RDF vocabulary Intelligent system Prototype - Mozilla Firefox_2011-05-14_06-22-46.png17.18 KB

There is a way to import and register the vocabulary through evoc without hosting it on the web.

Evoc supports importing from file provided that we provide the location of the file in URI format.
Suppose I have the RDFS file of my vocabulary, named "LearningObject", inside a folder called "vocab", within my project "test". In that case, I would need to provide the URI in this format:

http://localhost/test/vocab/LearningObject.rdfs#

Thus, as Lin said, the import from file functionality can be easily added. We would just need to upload the file provided by the users to, say, the vocab folder, whose URI is known to us (http://localhost/test/vocab/), append the name of the file uploaded by the user to the URI and call evoc_import_vocabulary.

cygri’s picture

cygri commented

@mayankkandpal, that would certainly work but seems unnecessarily complicated. It would be easier to just add an API function to evoc. Then we could pass the RDFS representation of the vocabulary directly to evoc, without saving it as a file somewhere first. That also avoids all sorts of possible problems, such as disk write permissions, and stale files being left on the disk in case something goes wrong in the process.

(And for what it's worth, being able to import only from the web *might* actually be sufficient. Think about it -- currently if I have Neologism running at http://localhost/neologism/ and have a vocabulary at http://localhost/neologism/vocab1#, then I can import from that URL to a new vocabulary vocab2 and it works just fine. But I'm not suggesting that this approach would be a good idea for the D6->D7 upgrade process.)

Anonymous’s picture

Anonymous (not verified) commented
Version: 6.x-1.x-dev » 7.x-2.x-dev

Just changing the version to 7.x.