With this option, users (who may edit a node AND may set this option) can set if and how a node is listed in robot.txt.
This involves following changes in the GUI:
1) Add to '{host}/admin/user/permissions' an option "Allow setting robot.txt directives" with values: "No" (selected by default) and "Yes" per user role.
2) Add to '{host}/admin/content/types/add' an option "Allow setting robot.txt derictive" with values: "No" (selected by default) and "Yes".
3) Add to '{host}/node/[nid]/create' and '{host}/node/[nid]/edit' an option "Robot.txt directive" with values:
3a) "Allow",
3b) "Disallow",
3c) "Do not list and do NOT filter" (selected by default) and
3d) "Do not list and DO automatically filter (remove any listing of the node in the robot.txt)".
NB: since Drupal installations with clean URLs enabled allow nodes access via '{host}/node/[nid]' AND '{host}/{automated URL alias}', both (!) paths should be handled with this feature.
Comments
Comment #1
not_Dries_Buytaert commentedMerged with this feature request: http://drupal.org/node/821410