Although conneg is not so much of a requirement anymore with RDFa, it would be nice to have it regardless. Looks like this would be fairly easy to achieve with hook_page_delivery_callback_alter.

Comments

Anonymous’s picture

We can also look at how Richard and Guido got conneg working for Neologism.

scor’s picture

The RESTful Web Services already handles the RDF/XML output for entities, and we should try to plug the other RDF serialization formats.

Anonymous’s picture

Title: Support for content negotiation in Drupal 7 » Non-RDFa serializations and content negotiation in Drupal 7

This makes a lot of sense. The RDF/XML it outputs needs some work, here is what it looks like:

<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description xmlns:site="http://localhost/rdf-contrib/" xmlns:dc="http://purl.org/dc/terms/" xmlns:sioc="http://rdfs.org/sioc/ns#" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:og="http://ogp.me/ns#" rdf:about="http://localhost/rdf-contrib/node/1">
    <rdf:type rdf:resource="http://rdfs.org/sioc/ns#Item"/>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Document"/>
    <site:nid xmlns:site="http://localhost/rdf-contrib/">1</site:nid>
    <site:vid xmlns:site="http://localhost/rdf-contrib/">1</site:vid>
    <site:is_new xmlns:site="http://localhost/rdf-contrib/"></site:is_new>
    <site:type xmlns:site="http://localhost/rdf-contrib/">article</site:type>
    <dc:title xmlns:dc="http://purl.org/dc/terms/">Foobar</dc:title>
    <site:language xmlns:site="http://localhost/rdf-contrib/">und</site:language>
    <site:url xmlns:site="http://localhost/rdf-contrib/">http://localhost/rdf-contrib/node/1</site:url>
    <site:edit_url xmlns:site="http://localhost/rdf-contrib/">http://localhost/rdf-contrib/node/1/edit</site:edit_url>
    <site:status xmlns:site="http://localhost/rdf-contrib/">1</site:status>
    <site:promote xmlns:site="http://localhost/rdf-contrib/">1</site:promote>
    <site:sticky xmlns:site="http://localhost/rdf-contrib/">0</site:sticky>
    <dc:date xmlns:dc="http://purl.org/dc/terms/" rdf:datatype="xsd:dateTime">1303674405</dc:date>
    <dc:modified xmlns:dc="http://purl.org/dc/terms/" rdf:datatype="xsd:dateTime">1303674405</dc:modified>
    <site:author xmlns:site="http://localhost/rdf-contrib/">
      <rdf:Description rdf:about="http://localhost/rdf-contrib/user/1"/>
    </site:author>
    <site:log xmlns:site="http://localhost/rdf-contrib/"></site:log>
    <site:revision xmlns:site="http://localhost/rdf-contrib/"></site:revision>
    <site:comment xmlns:site="http://localhost/rdf-contrib/">2</site:comment>
    <sioc:num_replies xmlns:sioc="http://rdfs.org/sioc/ns#" rdf:datatype="xsd:integer">0</sioc:num_replies>
    <site:comment_count_new xmlns:site="http://localhost/rdf-contrib/">0</site:comment_count_new>
    <content:encoded xmlns:content="http://purl.org/rss/1.0/modules/content/">
      <rdf:Description>
        <site:value xmlns:site="http://localhost/rdf-contrib/">&lt;p&gt;here is the body text&lt;/p&gt;</site:value>
        <site:summary xmlns:site="http://localhost/rdf-contrib/"></site:summary>
        <site:format xmlns:site="http://localhost/rdf-contrib/">filtered_html</site:format>
      </rdf:Description>
    </content:encoded>
    <dc:subject xmlns:dc="http://purl.org/dc/terms/">
      <rdf:Description>
        <site:item xmlns:site="http://localhost/rdf-contrib/">
          <rdf:Description rdf:about="http://localhost/rdf-contrib/taxonomy_term/1"/>
        </site:item>
      </rdf:Description>
    </dc:subject>
    <og:image xmlns:og="http://ogp.me/ns#">
      <rdf:Description>
        <site:alt xmlns:site="http://localhost/rdf-contrib/"></site:alt>
      </rdf:Description>
    </og:image>
  </rdf:Description>
</rdf:RDF>

I will be posting issues in that queue for clean-up. We should move to a dependency on RestWS for all of the RDF/XML (and other serializations) as soon as we can.

Since we would have to depend on Entity and RestWS, I think it would make sense to have RDF output be a separate module in the package. This way, modules can depend on RDFx without having to enable Entity and RestWS.

Anonymous’s picture

Title: Non-RDFa serializations and content negotiation in Drupal 7 » Use RestWS for non-RDFa serializations and content negotiation in Drupal 7

The RestWS module routes all entity menu router items through its own callback, restws_page_callback, where it checks for the format extension. Because of this, there is no menu router item for the different serializations. This means that they aren't part of the menu system, so we can't create tabs for them unless we do a redirect from node/%/rdf to node/%.rdf

I think no tabs is a better solution than a redirect.

scor’s picture

No RDF tab is not an issue, since we're eventually planning to have the conneg at the entity URI anyways. Most ppl will probably not care for these alternate formats anyways, and the RDF geeks can always advertise these formats through links elsewhere on the page.

Anonymous’s picture

Status: Active » Needs review
StatusFileSize
new25.67 KB

OK, here is a Turtle format for RestWS. It builds an array suitable for ARC2's Turtle serializer.

I had to move away a little bit from the algorithm used by the RDF format in two places: when the property is a multivalue field and when the property has a metadata structure that Entity preserves, as in the case of textareas. There are comments in the code that explain.

Currently there is no statement for images. I believe this is a bug in Entity's handling of image metadata, there is no URI given for the image. You'll see it is also left out in the RDF/XML above.

The patch also puts module files directly in the RDFx directory. This is how we started doing it in SPARQL module. We should probably figure out which way we want to go and make sure both packages do it that way.

Anonymous’s picture

StatusFileSize
new25.68 KB

Whoops, forgot to change the MIME type. Prefix.cc uses text/turtle, but I think application/x-turtle is the standard one to use.

Anonymous’s picture

Status: Needs review » Needs work

I realized that in my haste to go get lunch, I forgot to add datatype, language, and callback.

scor’s picture

I don't think the RDF model building logic should be moved out of the main rdfx module... many RDF related modules will use this in-memory RDF model without the need to output RDF (first example which comes to mind is the SPARQL endpoint module). Building the RDF model of an entity should not require the RestWS module, it's only if you want to do conneg that you should require it. I suspect there will be other modules which will want to build the in-memory RDF model for entities and do things with it, without requiring conneg or any kind of serialization even.

Solutions I can think of:
1. place any conneg related code into rdf_output.module, which requires RestWS for the conneg handling but keep the generic ARC2 RDF model in rdfx.
2. have the RestWS integration inside rdfx, but make the conneg available only if the RestWS is installed. If RestWS is not there, it will just always return the RDFa page, if it's there, then RestWS conneg kicks in and might return some other serialization.

Anonymous’s picture

Building the RDF model does require Entity, this implementation would be impossible without Entity... and the implementation is made much easier with RestWS. If you look at the code, you will see conditionals such as if ($property instanceof EntityValueWrapper). This is because the view function is actually a method of a RestWS class and it has Entity objects passed to it.

Entity gets the properties with their values from the database. This is definitely preferable to the way the module currently does it, because it doesn't require a switch statement and it handles the language and access controls.

There is no conneg related code necessary, AFAIK. I haven't tested it, but it seems that conneg is supposed to be a part of RestWS.

As I see it, the options are:

  1. Have all of this code in RDFx and introduce the dependency on RestWS and Entity
  2. Keep all this in RDF Output
  3. Stick with the current way of building the model

I personally dont' see #3 as a real option. Having to use a case in a switch statement for each kind of field is not sustainable and not modular. It means that new field types cannot introduce their RDF into the model. We need to use Entity instead.

Anonymous’s picture

In thinking about it, we could probably extract all the model building and not have the dependency on RestWS. We would still have the dependency on Entity, but I think that's ok since a lot of modules will.

We would need to duplicate the logic from restws_property_access_filter into a function in RDFx, but it's not a very complicated a function, and implement a new version of restws_resource_uri. I think this is good anyway because we could make the resource URIs match the ones used in the RDFa.

The final version would have a function like rdfx_get_model($type, $id). That would return an array suitable for ARC2.

I can take care of detangling the model building code and move that to RDFx. Then RDF Output will just have a simple call to rdfx_get_model, which means all the different format implementations become smaller and cleaner.

scor’s picture

I'm all for the dependency on entity, I think it can save us a lot of code... and like you said, entity is very likely to become a very common dependency in the contrib world. many modules have this dependency already such as sparql_views.

tourendal’s picture

Subscribing... (nice to see that you you are bringing the LoD principles to Drupal)

Anonymous’s picture

Project: Resource Description Framework (RDF) » RDF Extensions

Moving this to project/rdfx, the contrib RDF module's new home

scor’s picture

Status: Needs work » Needs review
StatusFileSize
new30.96 KB

This patch implements the option 2 of comment #9: have the RestWS integration inside rdfx, but make the conneg available only if the RestWS is installed. There is no dependency on RestWS but only on entity. I was able to reuse some of Lin's code pretty much as is with some modifications. So this is what we have now:
- include datatype on objects
- disable automatic export of all properties (see comment).
- overrides RestWS' RDF/XML formatter to use ARC2 serializer
- serialization formats supported: RDF/XML, Turtle, NTriples, RDFJSON via ARC2.
- add dependency on entity
- add site: prefix to rdfx_rdf_namespaces()

So, for example, to get the NTriples export of node/1, just type: curl -H "Accept: text/plain" http://localhost/d7/node/1

scor’s picture

created #1164132: Support for wildcards and q-values conneg for follow up once this issue is fixed.

Anonymous’s picture

Status: Needs review » Needs work

Exciting! It looks good to me and will be great to have these formats and the model building code... just a few nitpicks.

+++ b/rdfx.infoundefined
@@ -4,11 +4,12 @@ package = RDF
+dependencies[] = entity

How do you handle dependency checking if RDFx is already enabled on a site? I used git to update the code on an existing site and there was no warning.

+++ b/rdfx.moduleundefined
@@ -108,456 +90,232 @@ function rdfx_help($path, $arg) {
+          // When addToTurtle is used recurssively, EntityListWrapper inserts

No longer have addToTurtle function, comment should be changed.

+++ b/rdfx.moduleundefined
@@ -108,456 +90,232 @@ function rdfx_help($path, $arg) {
+function rdfx_property_access_filter($wrapper) {

Would rdfx_property_access make sense? That seems to be the naming convention in other things, like node_access and Views' access method.

Powered by Dreditor.

milesw’s picture

Patch #15 won't apply for me. Can you re-roll?

scor’s picture

Status: Needs work » Needs review
StatusFileSize
new30.68 KB

How do you handle dependency checking if RDFx is already enabled on a site?

The best thing I can think of is throwing a message with an update function, and add a note in the next release.

Would rdfx_property_access make sense? That seems to be the naming convention in other things, like node_access and Views' access method.

They do different things. node_access take a node as parameter and returns whether you have access or not to that node. rdfx_property_access_filter take a list of existing properties and returns a list of properties to which you have access, hence the _filter in the name of the function.

Anonymous’s picture

Status: Needs review » Needs work

I see, and RestWS uses that naming convention as well, so I guess we aren't making up our own convention at least. Makes sense to me.

Update function with message makes sense to me. We'll need to add that to this patch.

scor’s picture

Status: Needs work » Needs review

I'm not sure we actually need an update function here. update.php was fixed in D7 to warn users when a dependency is missing, so by visiting update.php you will naturally get a warning that entity is missing.

Anonymous’s picture

Status: Needs review » Needs work

I'm testing this on a site that has entity disabled when I apply the patch. Going to update.php doesn't give me any message, it says "No pending updates". We might just need to add an empty update function to trigger the warning.

scor’s picture

Status: Needs work » Needs review
StatusFileSize
new31.11 KB

Thanks for testing that use case, I only tested when there entity module was not present on the server. I added an update function which enabled the entity module. The initial dependency checking of update.php takes care of the case where the entity is not present on the server.

Anonymous’s picture

Works like a charm :)

I'm going to leave this until tomorrow to see if anyone else wants to review.

milesw’s picture

Works great for me also. Very, very cool to see this functionality.

So the dependency on Entity API is necessary even if not using conneg via Rest WS? I guess this is not so bad since, as mentioned before, Entity is a commonly installed module.

I did encounter some notices when testing this patch. Using Firefox Modify Headers plugin I was changing the media type and refreshing a single node view. Occasionally, when I would change the Accept type to something Drupal wasn't familiar with, I'd see the following notice repeated twice...

Notice: Trying to get property of non-object in entity_metadata_user_access() (line 539 of /Users/miles/Sites/seven.dev/sites/all/modules/entity/modules/callbacks.inc).

It was inconsistent and only happened after requesting unrecognized media types. Definitely a minor issue, but figured I would report.

scor’s picture

Thanks for testing Miles! Yes, entity is required to build the RDF model in memory which is useful for other modules such as the SPARQL endpoint. The bug you found needs to be investigated, but let's commit that beast first (Lin?), and deal with that minor issue (which might be an entity module issue) in a separate issue.

Anonymous’s picture

Fixed with c29136c.

Anonymous’s picture

Status: Needs review » Fixed

I meant fixed!

scor’s picture

Updated project page to reflect the new dependency on entity the optional use of restws.

Follow ups:
#1175590: Add option to allow all entity properties to be exported in RDF
#1171030: Advertise the various formats supported by RestWS
#1164132: Support for wildcards and q-values conneg

@milesw, want to file a new issue for the notices you encountered with Firefox Modify Headers plugin?

adorsk’s picture

Just a small caveat related to the use of Entity:

If you are using a custom field and you want it to appear in RDF serializations, you will need to also define the 'property_type' key, in your field definition's hook_field_info function, per the entity API.

Otherwise your field will not appear in the serialized RDF.

scor’s picture

thanks for the tip, Alex. Could you paste an example here, so we can keep track of that and document it somewhere?

adorsk’s picture

Sure thing.

Say you are defining a field and you have an implementation of hook_field_info(), like this:


/**
 * Implements hook_field_info().
 *
 */
function my_field_field_info() {
  return array(
    'my_field' => array(
      'label' => t('My Field'),
      'description' => t('My totally sweet field!'),
      'default_widget' => 'some_widget',
      'default_formatter' => 'some_formatter',
    ),
  );
}

If you want Entity API to include your field in the list of properties serialized for RDF, You will also need to add a key 'property_type'. For example,

/**
 * Implements hook_field_info().
 *
 */
function my_field_field_info() {
  return array(
    'my_field' => array(
      'label' => t('My Field'),
      'description' => t('My totally sweet field!'),
      'default_widget' => 'some_widget',
      'default_formatter' => 'some_formatter',
      // HERE IS THE PROPERTY TYPE KEY
      'property_type' => 'text',
    ),
  );
}

See the comments for entity_hook_field_info() in entity/entity.api.php in the code for http://drupal.org/project/entity .

This requirement bit me when I was defining my own field and wondered why it wasn't appearing in the RDF serialization. It took a bit of digging through the Entity API code to figure out what was going on.

A related caveat is that Entity API currently serializes only the first column of a field's stored values by default. Look for the line like $values[$delta] = $data[$columns[0]]; in entity_metadata_field_property_get() in entity/modules/callbacks.inc .

It would be nice if Entity API had a GUI that let people choose a formatter to use for property values, but I think that's more of an Entity issue and isn't in the scope of rdfx.

scor’s picture

A related caveat is that Entity API currently serializes only the first column of a field's stored values by default.

seems like an entity bug to me. do you want to file it in the entity issue queue?

adorsk’s picture

Sure thing, I'll add an issue in the Entity issue queue.

fago’s picture

> A related caveat is that Entity API currently serializes only the first column of a field's stored values by default.
That's not the case.

My reply there:

The default callback handles only the first column, every other field-type with multiple columns needs to do its own callback, see the examples for the core field types like formatted-text, files or images.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.