Problem/Motivation

Drupal 7 includes default RDF mappings from quite a few namespaces (FOAF, Dublin Core, SIOC, SKOS, etc). These were the popular vocabularies back in 2009. From the feedback we got, it seems that the amount of namespaces in Drupal 7 confused people. Since then, schema.org was launched with a broad set of types and properties all under the same namespace, and it is backed by the major search engines.

Proposed resolution

List the types and properties from schema.org which are relevant for the types of data we have in core (taxonomy term, user, file, their fields, etc) and in the standard install profile. There might also be some terms from schema.org we could use to "suggest" mappings when a user creates a new content type or field depending on the field type. The notion of "default" mapping should go away in Drupal 8, and be replaced with suggested mappings so that users are not forced into using a particular default mapping without knowing it. This approach was used by Lin Clark in the microdata module.

Remaining tasks

  • Add test for user mapping and user page.
  • Add test for term page.
  • Add test for teasers in node listing.
  • Add the mapping for page comments.

User interface changes

None. There is currently no UI in core to choose RDF mappings.

 

 Property Drupal 7 mapping Drupal 8 mapping Notes

 

 # node.page      

 

 type foaf:Document schema:WebPage  

 

 title dc:title schema:name  

 

 created dc:date, dc:created
datatype => xsd:dateTime
callback => date_iso8601		 schema:dateCreated should use the HTML5 <time> element if rendered

 

 changed dc:modified
datatype => xsd:dateTime
callback => date_iso8601		 schema:dateModified should use the HTML5 <time> element if rendered

 

 body content:encoded schema:text
datatype => rdf:HTML		  

 

 uid sioc:has_creator (rel) schema:author  

 

 name foaf:name NA this should no longer need to be explicitly in comment, but only as a mapping to user

 

 comment_count sioc:num_replies
datatype => xsd:integer		 schema:interactionCount  

 

 last_activity sioc:last_activity_date
datatype => xsd:dateTime
callback => date_iso8601		    

 

        

 

 # node.article      

 

 type sioc:Item, foaf:Document schema:Article Article is more generic than BlogPosting or NewsArticle, and given that we don't know kind of article end user will use this content type for, I think it's safer to just use the generic schema:Article

 

 title (same as node.page) (same as node.page)  

 

 created (same as node.page) (same as node.page)  

 

 changed (same as node.page) (same as node.page)  

 

 body (same as node.page) (same as node.page)  

 

 uid (same as node.page) (same as node.page)  

 

 name (same as node.page) NA this should no longer need to be explicitly in comment, but only as a mapping to user

 

 comment_count (same as node.page) (same as node.page)  

 

 last_activity (same as node.page) (same as node.page)  

 

 field_image og:image, rdfs:seeAlso (rel) schema:image  

 

 field_tags dc:subject (rel) schema:keywords ?  

 

        

 

 # node.forum      

 

 type sioc:Post, sioct:BoardPost schema:Discussion Not an official type yet, but there is a proposal for this type

 

 title (same as node.page) (same as node.page)  

 

 created (same as node.page) (same as node.page)  

 

 changed (same as node.page) (same as node.page)  

 

 body (same as node.page) (same as node.page)  

 

 uid (same as node.page) (same as node.page)  

 

 name (same as node.page) NA this should no longer need to be explicitly in comment, but only as a mapping to user

 

 comment_count (same as node.page) (same as node.page)  

 

 last_activity (same as node.page) (same as node.page)  

 

 taxonomy_forums sioc:has_container (rel)    

 

        

 

 # comment (all)      

 

 type sioc:Post, sioct:Comment schema:Comment  

 

 title (same as node.page) schema:name  

 

 created (same as node.page) schema:dateCreated  

 

 changed (same as node.page) schema:dateModified  

 

 comment_body content:encoded schema:text  

 

 pid sioc:reply_of (rel)    

 

 uid sioc:has_creator (rel) schema:author  

 

 name (same as node.page) NA this should no longer need to be explicitly in comment, but only as a mapping to user

 

        

 

        

 

 #taxonomy_vocabulary (all)      

 

 type skos:ConceptScheme    

 

 name dc:title    

 

 description rdfs:comment    

 

        

 

 # taxonomy_term (all)      

 

 type skos:Concept schema:Thing  

 

 name rdfs:label, skos:prefLabel schema:name  

 

 description skos:definition schema:description  

 

 vid skos:inScheme (rel)    

 

 parent skos:broader (rel)    

 

        

 

 # taxonomy_term.forums      

 

 type sioc:Container, sioc:Forum    

 

 name (same as taxonomy_term.all)    

 

 description (same as taxonomy_term.all)    

 

 vid (same as taxonomy_term.all)    

 

 parent (same as taxonomy_term.all)    

 

        

 

 # user (no bundle)      

 

 type sioc:UserAccount schema:Person  

 

 name foaf:name schema:name  

 

 homepage foaf:page (rel) NA I would suggest to no longer the homepage field because it is only available for anonymous users and its value cannot be as trustable as an authenticated user (spam). People can create an homepage field if they want to on the user entity type
CommentFileSizeAuthor
#46 1784234-46-schema-mapping.patch36.13 KBlinclark
#44 1784234-45-schema-mapping_includes1941286.patch38.63 KBlinclark
#44 interdiff.txt3.92 KBlinclark
#37 1784234-37-schema-mapping.patch36.17 KBlinclark
#33 1784234-33-schema-mapping.patch36.22 KBlinclark
#33 interdiff.txt9.11 KBlinclark
#31 1784234-31-schema-mapping.patch36.39 KBlinclark
#31 interdiff.txt2.4 KBlinclark
#27 1784234-27-schema-mapping.patch35.25 KBlinclark
#27 interdiff.txt3.26 KBlinclark
#25 1784234-24-schema-mapping.patch33.11 KBlinclark
#25 interdiff.txt5.72 KBlinclark
#23 1784234-22-schema-mapping.patch29.92 KBlinclark
#23 interdiff.txt6.13 KBlinclark
#21 1784234-21-schema-mapping.patch524.02 KBlinclark
#21 interdiff.txt18.72 KBlinclark
#20 1784234-20-schema-mapping.patch29.63 KBlinclark
#20 interdiff.txt12.46 KBlinclark
#19 1784234-19-schema-mapping.patch27.86 KBlinclark
#19 interdiff.txt5.65 KBlinclark
#18 interdiff.txt2.13 KBlinclark
#18 1784234-17-schema-mapping.patch25.57 KBlinclark
#16 1784234-16-schema-mapping.patch24.42 KBlinclark
#16 interdiff.txt1.2 KBlinclark
#14 1784234-14-schema-mapping.patch23.22 KBlinclark
#14 interdiff.txt7.51 KBlinclark
#12 1784234-12-schema-mapping.patch18.19 KBlinclark
#10 1784234-10-schema-mapping_includes1869600.patch162.08 KBlinclark
#10 1784234-10-schema-mapping-do-not-test.patch18.18 KBlinclark

Comments

Anonymous’s picture

Anonymous (not verified) commented

I can attest to this. In surveys I performed, half listed SEO as the primary reason they used RDF module, but only 1 of the 30 respondents had made alterations to the core mappings (which would be required to get SEO results).

chx’s picture

chx commented

Talked to dale42 about this today, I recommended typed_data()->getDefinitions() as the starting point.

Anonymous’s picture

Anonymous (not verified) commented

Usually, we map terms like Schema.org to fields and content types. While Schema.org does have it's own datatypes, they aren't frequently used. Datatypes are often covered by XSD and other vocabularies. What specifically was dale42 interested in doing?

chx’s picture

chx commented

No idea :) do you mean field types? dale42 was also very confusing :/ but anyways, field types and content types are not the same "level" so to speak. Nor are invidual fields... I was trying to give some guidance on D8, anyways.

Anonymous’s picture

Anonymous (not verified) commented

These are the kinds of mappings that we'll need to figure out:

  1. Replacements for the existing mappings on the default install profile's bundles and field instances. For example, what is now sioc:Post in core should probably be schema:Article or schema:BlogPosting.
  2. Defaults for particular field types, which the user can accept when adding a new field instance. There are unlikely to be many sensible defaults for core field types, since they are so flexible. The more special purpose fields that are in contrib, like AddressField and Fivestar, are more likely to have sensible defaults.
  3. Possibly, mappings for data types (e.g. the primitives defined in system_data_type_info). We may decide to use Schema.org datatypes for these, but I think it's more likely we'll want to use xsd and rdf. The discussion in #1831286: Provide machine-readable description of entity/field/property is related to this.
judahtanthony’s picture

judahtanthony commented

FYI, I just got notice from Google that they are going to start looking at the itemprop="logo".

We could do something like:

      <?php if ($logo): ?>
-        <a href="<?php print $front_page; ?>" title="<?php print t('Home'); ?>" rel="home" id="logo">
-          <img src="<?php print $logo; ?>" alt="<?php print t('Home'); ?>" />
-        </a>
+        <div itemscope itemtype="http://schema.org/Organization" id="logo">
+          <a itemprop="url" href="<?php print $front_page; ?>" title="<?php print t('Home'); ?>" rel="home">
+            <img itemprop="logo" src="<?php print $logo; ?>" alt="<?php print t('Home'); ?>" />
+          </a>
+        </div>
       <?php endif; ?>

We may have to do some style changes to accomodate, but the idea remain.

scor’s picture

scor commented

The logo property in question is for the Organization type. This would make sense if we were shipping with an organization content type, I don't think core will ever have such content type, but distributions may.

Anonymous’s picture

Anonymous (not verified) commented

Topping. This should be handled ASAP if we're going to get it in before July 1.

scor’s picture

scor commented
Assigned: Unassigned » scor

yes, I'm going to take this one on.

Anonymous’s picture

Anonymous (not verified) commented
Status: Active » Needs review
StatusFileSize
new 1784234-10-schema-mapping-do-not-test.patch18.18 KB
new 1784234-10-schema-mapping_includes1869600.patch162.08 KB

I think it will actually reduce the duplication of effort if this patch is finished before #1778410: Throw exception when RDF namespaces collide, so I've started on it.

The patch includes mappings and tests for Article and Page. I had two thoughts on the mappings you suggested for those:

  • comment_count: this could map to interactionCount
  • field_tags: this could map to about, since we don't yet have anything in place to change output from resource to literal.

This patch includes a concept called the SchemaOrgDataConverter. This would provide callbacks for formatting values to schema.org's standards... for example, interactionCount as "UserComment:5". However, I haven't written the tests for the front page yet (which is where comment count is displayed), so I haven't made it work yet.

scor’s picture

scor commented
Status: Needs review » Needs work

This needs to be rerolled now that #1869600: Refactor RDF mappings to be inline with the new Entity Field API has been committed.

Let's keep going in the direction described in #15, I agree with all of your points, Lin.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
Issue tags: -Novice
StatusFileSize
new 1784234-12-schema-mapping.patch18.19 KB

Rerolled to account for the refactor RDF patch and the taxonomy term API switching to entity reference.

Anonymous’s picture

Anonymous (not verified) commented
Assigned: scor » linclark

Reassign.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new interdiff.txt7.51 KB
new 1784234-14-schema-mapping.patch23.22 KB

I've added the mapping for article comments.

We currently do not have a mapping which relates the node to the comment (which would be schema:comment). This should be easy once #1907960: Helper issue for "Comment field" is in.

In rdf_preprocess_comment, we were accessing the mapping for 'title' instead of 'subject', which is what comment calls its title. I changed this, but it might break something else. I was surprised that our CommentAttributeTest didn't catch it.

Status: Needs review » Needs work

The last submitted patch, 1784234-14-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt1.2 KB
new 1784234-16-schema-mapping.patch24.42 KB

Fixing the test.

Anonymous’s picture

Anonymous (not verified) commented

I suggest that we don't even worry about mapping vocabularies... if the closest we can get is schema:Thing, than there isn't anything useful that could really be done with the information anyway.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new 1784234-17-schema-mapping.patch25.57 KB
new interdiff.txt2.13 KB

Changed the mapping for tags.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new interdiff.txt5.65 KB
new 1784234-19-schema-mapping.patch27.86 KB

Changed mapping for users.

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new interdiff.txt12.46 KB
new 1784234-20-schema-mapping.patch29.63 KB

Moved the user mapping to user module (since it is independent of standard profile) and made some code style fixes.

I also changed the mappings for forum nodes and terms. I didn't add tests for this since it isn't part of the standard profile and we don't have tests for them at this point. I don't think it is worth the additional effort of testing them, since we are already testing the rendering of RDFa in nodes and terms.

Left to do:

  • Add test for user mapping and user page.
  • Add test for term page.
  • Add test for teasers in node listing.
  • Add the mapping for page comments.
Anonymous’s picture

Anonymous (not verified) commented
Issue summary: View changes

initial draft

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new interdiff.txt18.72 KB
new 1784234-21-schema-mapping.patch524.02 KB

In this patch, I make it possible to use methods on classes as datatype callbacks, add a test for this, and add tests for the front page display.

I also rearranged the test assertions so that mappings that are the same (e.g., the shared mappings between page and article) are tested by the same function (e.g. _testCommonNodeProperties).

Status: Needs review » Needs work

The last submitted patch, 1784234-21-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt6.13 KB
new 1784234-22-schema-mapping.patch29.92 KB

Whoops, forgot to merge.

This patch fixes that and also removes the test for the mapping config... this is already tested indirectly by testing the output.

Status: Needs review » Needs work

The last submitted patch, 1784234-22-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt5.72 KB
new 1784234-24-schema-mapping.patch33.11 KB

The last patch will fail because #1996714: Convert FileItem and ImageItem to extend EntityReferenceItem got in. I changed fid to target_id to fix that.

I also added tests for the user page and removed the hardcoded relationship between the user and the account.

Status: Needs review » Needs work

The last submitted patch, 1784234-24-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt3.26 KB
new 1784234-27-schema-mapping.patch35.25 KB

So it turns out we were hard-coding the placement of the content attribute for comment count and didn't have a datatype callback to handle it. I've added a callback for raw values.

I've also added a test for the taxonomy term page display. For some reason, the description isn't getting exposed. I assume we test for this in the taxonomy term test, so I'm not sure what's going on here. Will be looking in to it.

Anonymous’s picture

Anonymous (not verified) commented

It turns out we weren't testing for the description in our TaxonomyAttributesTest.

@scor, do you have any idea why the description attributes wouldn't be placed.

scor’s picture

scor commented

I need to review the whole patch, but here is a reply to your question.

@scor, do you have any idea why the description attributes wouldn't be placed.

We couldn't place the attributes because the term description was not a field, and adding markup around it required extra functionality that didn't make it in D7. I believe the right solution is #569434: Remove taxonomy term description field; provide description field for forum taxonomy (which afaict would not break any API and could be committed after July 1st). I'll try to get someone to review and push for this issue.

Status: Needs review » Needs work

The last submitted patch, 1784234-27-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt2.4 KB
new 1784234-31-schema-mapping.patch36.39 KB

Ah, that makes sense. Ok, I've removed that test and added a todo to add tests once it is a field.

I've also added the mapping for page comments. I figure a test is probably overkill, but we could add it if necessary.

I'll be going over it once more for code style, but the content should be ready.

scor’s picture

scor commented
Status: Needs review » Needs work

This is looking great so far!

+++ b/core/modules/rdf/lib/Drupal/rdf/Tests/StandardProfileTest.php
@@ -0,0 +1,491 @@
+    // @todo Once the title data is output consistently between full and teaser
+    // view modes, move this to _testCommonNodeProperties().
...
+  function _testCommonNodeProperties($graph, $node, $message_prefix) {
+    // Article.
+    $uri_info = $node->uri();
+    $uri = url($uri_info['path'], array('absolute' => TRUE));

Not sure what you mean with the comment "Article". Looks like you are setting the URI here.

+++ b/core/modules/rdf/lib/Drupal/rdf/Tests/TaxonomyAttributesTest.php
@@ -77,5 +77,7 @@ function testTaxonomyTermRdfaAttributes() {
+    // @todo Add test for term description once it is a field.

you might want to add a link to the issue here, to show it's being take care of.

+++ b/core/modules/rdf/rdf.module
@@ -559,6 +546,8 @@ function rdf_preprocess_comment(&$variables) {
+  // @todo When comments are turned into fields, this should be changed.
+  // Currently there is no mapping relating a comment to its node.

link to issue?

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt9.11 KB
new 1784234-33-schema-mapping.patch36.22 KB

This patch fixes code style issues and addresses the comments from scor.

scor’s picture

scor commented
Title: Decide which types and properties from schema.org are relevant for core » Use schema.org types and properties in RDF mappings
Status: Needs review » Reviewed & tested by the community

Reviewed and tested the patch locally. It's working great and I can easily tweak the mappings by editing the YAML files.

This patch brings a truck load of new schema.org types and properties into Drupal 8 core. The article, forum and page nodes, as well as users, comments and taxonomy terms are now using schema.org for all their metadata and field instances defined by core. This patch also includes a small API addition for supporting a schema.org data conversion, and tests for the standard profile schema.org mappings. The @todos are waiting for API-cleanup issues to be resolved.

scor’s picture

scor commented
Issue tags: -schema.org

#33: 1784234-33-schema-mapping.patch queued for re-testing.

unfortunately this will need to be rerolled because of #1941286: Remove the process layer (rdf module).

Status: Reviewed & tested by the community » Needs work
Issue tags: +schema.org

The last submitted patch, 1784234-33-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new 1784234-37-schema-mapping.patch36.17 KB

It seems that that patch may have broken the output of the comment created date. To late in Ireland to debug tonight.

Status: Needs review » Needs work

The last submitted patch, 1784234-37-schema-mapping.patch, failed testing.

Anonymous’s picture

Anonymous (not verified) commented

So as I explain in https://drupal.org/node/1941286#comment-7595467, the patch in that issue removed the placement of certain comment attributes. @scor, since you were involved in that issue can you debug that?

Once that regression is fixed, this patch should be good to go.

scor’s picture

scor commented

yes, let me take care of that...

scor’s picture

scor commented

As explained in #1941286-40: Remove the process layer (rdf module), this fails because the standard profile uses Bartik. Could you try to switch the Standard profile theme to stark in setup() to see if it makes any difference? This is taken from testCommentLinks():

    // Bartik theme alters comment links, so use a different theme.
    theme_enable(array('stark'));
    config('system.theme')
      ->set('default', 'stark')
      ->save();
Anonymous’s picture

Anonymous (not verified) commented

You're right, it does fail because the standard profile uses Bartik, but also because of the change in #1941286: Remove the process layer (rdf module). The test worked with Bartik in #31.

scor’s picture

scor commented

Could you try the patch #1941286-42: Remove the process layer (rdf module)?

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt3.92 KB
new 1784234-45-schema-mapping_includes1941286.patch38.63 KB

Scor's patch from the other issue fixes these test fails. I have included that patch and also fixed a typo.

scor’s picture

scor commented
Status: Needs review » Needs work

Follow up for #1941286: Remove the process layer (rdf module) was committed, so this patch can now be rerolled.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review
StatusFileSize
new 1784234-46-schema-mapping.patch36.13 KB

Rerolled.

scor’s picture

scor commented
Status: Needs review » Reviewed & tested by the community

Compared patch #33 (which was RTBC'ed) and #46 in my IDE and verified the changes are the same.

Back to RTBC (see #33).

yannickoo’s picture

yannickoo
Location Berlin
commented

Great work!

dries’s picture

dries commented
Status: Reviewed & tested by the community » Fixed

Committed to 8.x. This is exciting. Thanks.

Anonymous’s picture

Anonymous (not verified) commented
Title: Use schema.org types and properties in RDF mappings » Change notice: Use schema.org types and properties in RDF mappings
Priority: Normal » Critical
Status: Fixed » Needs work

Since this changes the HTML data that is exposed, we should probably provide a change notice. I'll work on that.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs work » Needs review

OK, change notice added.

Anonymous’s picture

Anonymous (not verified) commented
Issue summary: View changes

Updated remaining tasks.

bleen’s picture

bleen commented

for reference: https://drupal.org/node/2034127

scor’s picture

scor commented
Status: Needs review » Fixed

This record looks good to me. Thanks Lin.

tim.plunkett’s picture

tim.plunkett
he/him
Primary language English
Location Philadelphia
commented

I encountered this test while debugging another issue.
It declares none of its properties and uses _test as a prefix for methods (which we're trying to avoid), among other things.

I've opened #2036765: Drupal\rdf\Tests\StandardProfileTest needs clean up as a follow-up, please review those changes as I've seen these inconsistencies in several RDF tests before (#1869600-123: Refactor RDF mappings to be inline with the new Entity Field API, for example).

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

(not verified) commented
Issue summary: View changes

Updated mapping table to reflect what was in the patch.

jneubert’s picture

jneubert commented

Just came across to adopt Drupal 7 mappings in ZBW Labs to the future standard schema.org mappings, and spotted

field_tags schema:keywords ?

I suppose the question mark's there because "keywords" has "Text" as the expected type. Perhaps it would be more appropriate to use schema:about, which expects schema:Thing?

from https://schema.org/CreativeWork:

about   Thing   The subject matter of the content.
jneubert’s picture

jneubert commented

Just another one: Re node.article

body schema:text

would perhaps be better schema:articleBody (from https://schema.org/Article)

articleBody   Text   The actual body of the article.
jneubert’s picture

jneubert commented

At node.page, the "Drupal 8 mapping" column for last_activity is empty - by accident?

I couldn't find a schema property for this - perhaps stick with sioc:last_activity_date?