By plik on

Dear all,

Can someone point me out some rational reason to use XHTML instead of HTML 4.01 Strict?

I'm sure everyone here knows that we cannot serve the pages with the correct MIME type (application/xhtml+xml) for XHTML without breaking IE compatibility, and that any content served with text/html will be parsed as HTML by any browser out there. So if the content is not parsed as XML, what's the point of using XHTML at all?

At least, shouldn't drupal let the user choose, on a theme by theme basis

  1. the content-type header it sends
  2. the use of HTML or XHTML

As a side note, I already modified my local drupal code to behave just like that, but I'm really interested in hearing your arguments about this question...

More info on this topic can be read here: http://hixie.ch/advocacy/xhtml

Regards,

-l

Comments

treksler’s picture

treksler commented

Drupal does let the user choose on a theme by theme basis!
You can make your themes HTML or XHTML and send it as whatever you want

It just so happens that, for the reasons mentioned above, XHTML 1.0 Strict, sent as text/html, is somewhat of a de facto standard at the moment and sicking to HTML 4.01 Strict (ha!) isn't going to help bring closer the day when XHTML 1.0 can be sent as text/xml.

Porting HTML 4.01 to XHTML is a pain.
XHTML is the future
Need I say more?

plik’s picture

plik commented

You can make your themes HTML or XHTML and send it as whatever you want

Actually, that's not totally true. You cannot have a valid and conformant html code, because of a few minor issues in Drupal core (sending ' />' for link elements, for example). It only took me a few minutes to fix this and send conformant code to browsers, but I would have loved to have the choice at the beginning...

XHTML is the future

Are you really sure? It seems to me that some browser vendors could take the HTML5 route, because of a few recurring problems with XHTML. What do you think?

Don't get me wrong, though: I use XML, XSL and others every day, so I'm quite convinced it can help us a lot. I just can't see the point sending invalid XHTML markup, with a text/html MIME type, so it will be parsed by the Tag Soup processor of any browser in the market (and for any foreseeable future). You surely know that the current markup of the main theme would miserably fail to render, would it be sent with the correct application/xhtml+xml MIME type, and thus parsed by XML processors. So how is that future proof?

ceejayoz’s picture

ceejayoz commented

I just can't see the point sending invalid XHTML markup, with a text/html MIME type, so it will be parsed by the Tag Soup processor of any browser in the market (and for any foreseeable future).

Because a) it works just fine and b) can be turned valid 30 seconds after IE fixes its issues.

freixas’s picture

freixas commented

Can XHTML pages be turned valid 30 seconds after IE fixes its issues?

Actually, you can do it in less than 30 seconds if you use some code I posted elsewhere. You check the browser headers to see what content types it accepts. If it accepts application/html+xml, you use that, otherwise you use text/html.

If you make this change *today*, then as soon as you view your pages on Firefox or any browser that supports application/html+xml, you will know whether those pages are valid XHTML. If they aren't, they won't work and only an error message will be displayed.

Running your code through the W3C validator does *not* guarantee your code will get through the XHTML parser. As I've stated several other places in this discussion, JavaScript will probably the the problem. If you use TinyMCE, your pages aren't XHTML (unless the TinyMCE people have fixed things since I tried it).

What I think is more likely is that once IE starts accepting XHTML, people are going to start discovering that their code still needs some work.

The next thing you will discover (also mentioned elsewhere) is that any syntax error in creating or editing a post or comment will generate a page that only contains an error message. Fixing this will require adding a tool like HTML Tidy to ensure that the syntax of the posting is correct before posting or previewing.

This should keep you busy for more than 30 seconds. If you don't agree, start using application/html+xml for browsers that accept it, use Firefox for viewing your site and see what happens.

In fact, if the Drupal team is committed to XHTML, there is no reason the Drupal core shouldn't put the content type code in the Drupal core.

--
Tony Freixas

mikhailian’s picture

mikhailian commented

AFAIU, what Drupal developers are worried about is well-formedness of the pages in the XML sense, without all of the SGML minimisation trickery that HTML 4.01 Strict allows.

They are just following the common belief that insuring the XMLish well-formedness is enough to be able to serve an XHTML web page.

Now, on the matter of user choice.

You can override the theme_page function to insert any DOCTYPE you want. Check $DRUPAL_HOME/themes/chameleon/chameleon.theme to see how it works.

The content-type is configurable in php.ini with the default_mimetype directive.

As you see, there is no need to modify the Drupal code to return a correct DOCTYPE and Content-Type.

plik’s picture

plik commented

AFAIU, what Drupal developers are worried about is well-formedness of the pages in the XML sense, without all of the SGML minimisation trickery that HTML 4.01 Strict allows.

This is a real point I totally agree with. But still, when well-formedness is achieved through a variety of checks with XML tools, I would really love to serve valid, conformant and semantic HTML (not X) to browsers. I simply think it's more elegant, future proof and consistent with what real browsers do (parsing as HTML).

So my only concern is that I cannot do that without modifying a bit of code in Drupal Core (' />' must be changed to '>'). I know how to modify the mime type and the doctype, but it will not make my code perfect, and that's what I'm worried about...

It's not a really big issue, but I'm always interested in knowing how the decision was made to go the XHTML route, despite several warnings of different actors. Was it really a conscious decision? Has it been discussed here?

Regards,

-l

mikhailian’s picture

mikhailian commented

So my only concern is that I cannot do that without modifying a bit of code in Drupal Core (' />' must be changed to '>'). I know how to modify the mime type and the doctype, but it will not make my code perfect, and that's what I'm worried about...

You may succede in convincing Drupal core developers to leave the decision about outputting SGML or XML to the theme. Make a patch that fixes Drupal code to be *ML-agnostic and create a bug report with the patch attached. I will definitely raise my voice for promoting such a patch into the HEAD.

sepeck’s picture

sepeck commented

http://drupal.org/principles

Standards-based. Drupal supports established and emerging standards. Specific target standards include XHTML and CSS.

The several pages found here are a good read for anyone to start with on the community.

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

plik’s picture

plik commented

AFAIK, HTML 4 is as much a standard as XHTML is, am I wrong? So my question would be:

About emerging standards, is XHTML the right one for Drupal? Did the community consider alternatives?

In other words, what specific features of possible use with XHTML does Drupal use or will use in the near future:

  • Namespaces?
  • XForms?
  • SVG?

As I guess the answer is: none. Why use it at all? When will these features be natively implemented in browsers, compared to WF2? What about the total failure of XHTML 2? Do the community know the real future of XHTML?

Just asking some basic questions :)

Regards,

-l

dmf-1’s picture

dmf-1 commented

Do the community know the real future of XHTML?

I might humbly suggest the community may possibly have some impact on the real future of XHTML. When most communities continue to push its' adoption, usage, and support, then they're actively creating their future rather than throwing up their hands in what could be simplified as a "IE does not currently support this! Oh Noes!!" self-defeating surrender to being held hostage by a single vendor.

Just another point of view on active versus passive.

plik’s picture

plik commented

I strongly agree that drupal should be active promoting emerging technologies, I'm just not convinced XHTML 2 is really the one to promote! The only technology that could be available in a foreseeable future is, IMO, (X)HTML5. So let's start promoting that emerging technology! It's really not a IE/not IE dilemma (IE could go his own route with XAML).

By the way, I think it's best coding practices that needs promotion, and it doesn't involve XHTML at all. It includes

  • valid markup
  • semantic markup and CSS layout
  • degrades gracefully
  • accessibility
  • etc.

And serving XHTML with a text/html mime type will definitely not fit into my best coding practices handbook. That's why I'm willing to promote either:

  • XHTML with content negotiation and correct mime type
  • stick to the basis, serve HTML!

More here: http://blog.whatwg.org/html-vs-xhtml

Regards,

-l

sepeck’s picture

sepeck commented

You asked why, I provided you with the the why. These discussions happened quite a while ago and have been accepted directions for a very long time in keeping with our develop towards the future and stay flexible to adapt to the future you can't see philosophy.

I am not interested in debating this as you seem to have come prepared to launch your issue. If you wish to make HTML Strict theme's then please do so and feel free to contribute them. It's certainly possible.

There is always ongoing work to separate the content from presentation (HTML). Core is not bad as is and is doing better all the time. Contrib modules are the usual mixed bag but community knowledge of how to leverage the tools to provide better separation is getting even better as well.

We've also been following other emerging standards such as OpenID. This isn't debated so much as someone actually works on it, contributes code and others in the community like it.

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

freixas’s picture

freixas commented

Hi,

I'm just starting to work with Drupal and I had the same question as 'plik'. As he pointed out, the article to read regarding this issue is http://hixie.ch/advocacy/xhtml.

Some things to note:

  • If you don't send the right MIME type, you are not writing XHTML, just syntactically invalid HTML (since HTML parsers are forgiving, the code works anyway).
  • There are significant differences in the way Javascript is handled by XHTML. Some of the details are in the article referred to above.

My own belief is that XHTML use among many PHP tools is more a fad than a reasoned decision and that HTML 4.01 would do an equivalent job of supporting standards (or perhaps a better job, since the code would actually be valid).

But I'm more interesting in deciding what I should do in working with Drupal:

  1. Should I write valid XHTML code and invalid Javascript to match the current Drupal implementation?
  2. Should I try to convert the Drupal core and any modules I use to HTML?
  3. Should I make Drupal (and modules) output true XHTML for browser that can accept it and invalid HTML for those that can't?
  4. Should I only output valid XHTML?

Option 1 is the path of least resistance.

It sounds like Option 2 might not be too much work--it may even be possible to automate (a global replace of /> to > might take care of it).

I have a site that I'm thinking of converting to use Drupal and which uses Option 3. The amount of work needed to extend Option 3 to Drupal depends mostly on the amount of Javascript used by the core and the modules. A quick scan for the word "javascript" in my Drupal build, which contains just the standard modules that come with the base installation, didn't turn up much, so this might not be too difficult. All Javascript would need to be reviewed and the process probably can't be easily automated.

Has anyone ever tried running Drupal as XHTML (by sending the correct MIME type)? If I were to run the experiment are the Drupal developers interested in the results? If so, is this forum the best place to post the info?

Option 4 is not really viable: neither IE6 or IE7 accepts XHTML.

--
Tony Freixas

mrael2’s picture

mrael2 commented

Please do run the experiment of serving Drupal as application/xhtml+xml. And yes, this is probably a good place to post the results of your experiment.

If possible, please try to use option 4. It's true that neither IE6 nor IE7 support application/xhtml+xml, but it's expected that IE8 will support this MIME type. Once it does, content negotiation (option 3) will become increasingly less used. Content negotiation is also a bit of a pain to implement.

I run a site (http://www.slvdweller.com) that uses no content negotiation and just serves XHTML as application/xhtml+xml. We get hundreds of visitors per day, so people will willingly convert to Firefox or some other browser if the content is compelling enough.

IE8 is not very far off.

freixas’s picture

freixas commented

I converted my existing site (www.tigerheron.com) to use Drupal.

Note: the Drupal version runs only on my PC at this point. If you visit my site, you will not be seeing Drupal in action (just yet).

The site serves application/xhtml+xml to any browser that says it can handle it. It serves text/html to the rest. The DOCTYPE changes as well, as does the some of the Javascript code. About the only thing I don't do is try to change /> to > for text/html.

The Drupal version of the site works fine as XHTML 1.0 Strict served as application/xhtml+xml. I've just started trying out the blog features of Drupal and my few blog pages also work. Finally, I use a different theme for administration. The admin pages that don't use the admin theme (like the content-creation pages) came up fine.

At mrael2's request I tried option 4 with IE7. This isn't right, since I am sending a MIME type which the browser has not said it understands, but I wanted to see what IE7 did. Well, it offered to download the page, but not display it. I noticed that mrael2's site re-directs the visitor to a page recommending that they use Firefox.

I don't have much more than the core modules installed. I'll continue to feed xhtml+xml to Firefox. If I run into any interesting problems, I'll report back here.

--
Tony Freixas

mrael2’s picture

mrael2 commented

It was great to hear that you have successfully served Drupal as application/xhtml+xml! I'd like to do the same at some point in the future on a new web site.

Would you be able to contribute your code to the Drupal project so that others such as myself could take advantage of it? The page at http://drupal.org/contribute/development seems to be the place where you can apply for a CVS account if you don't already have one.

The work you've already done will help the XHTML community a lot, so I'm hoping you can commit your code as a developer to the Drupal project.

It would be nice to be able to switch among the various options -- option 3 for most folks and option 4 for stalwarts like me.

freixas’s picture

freixas commented

Just a quick update: I installed TinyMCE which uses document.write(). For those not in the know, document.write() is not allowed in XHTML.

My options are to fall back to HTML tag soup (e.g. HTML 4.01), fix TinyMCE or not use it. I chose option 1.

As for mrael2's request above, I'm not sure what you're asking for. To serve your pages as application/xhtml_xml, you could use this code snippet:

if (@$useXHTML &&
    strpos(@$_SERVER['HTTP_ACCEPT'], 'application/xhtml+xml') !== false ||
    strpos($_SERVER['HTTP_USER_AGENT'], 'W3C_Validator') === 0) {
	header('Content-type: application/xhtml+xml; charset=utf-8');
}
else {
    header('Content-type: text/html; charset=utf-8');
    $useXHTML = false;
}

The useXHTML variable let's you control the option. Only browsers that accept XHTML will receive XHTML.

The following code might also be handy:

/**
 * Print out the appropriate DOCTYPE.
 *
 * This method depends on the global value $useXHTML. If the global is
 * set to true, then an XHTML DOCTYPE is printed.
 */

function printDocType()
{
    global $useXHTML;
    if ($useXHTML) {
	echo
            '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" ' .
            '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">';
    }
    else {
	echo
            '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" ' .
            '"http://www.w3.org/TR/html4/strict.dtd">';
    }
}

/**
 * Print out the appropriate HTML tag attributes
 *
 * This method depends on the global value $useXHTML. If the global is
 * set to true, then the HTML XML attributes are printed.
 */

function printHtmlAttributes()
{
    global $useXHTML;
    if ($useXHTML) {
	echo 'xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"';
    }
}

/**
 * Include the correct Meta content-type, mainly for use by the W3C
 * validator.
 *
 * This method depends on the global value $useXHTML. If the global is
 * set to true, then the content is application/xhtml+xml; otherwise,
 * it is text/html.
 */

function printMetaContentType()
{
    global $useXHTML;
    if ($useXHTML) {
	echo
            '<meta http-equiv="Content-Type" ' .
            'content="application/xhtml+xml; charset=iso-8859-1" />';
    }
    else {
	echo
            '<meta http-equiv="Content-Type" ' .
            'content="text/html; charset=iso-8859-1">';
    }
}

All this is executed as part of my theme code. I made no changes to the Drupal core. Even if I had time to contribute to the Drupal CVS, I have no idea where I would place this code. And since it breaks certain modules, I'm not sure it would be a good idea. Forcing Drupal to use the XHTML parser when available would at least highlight anything that is not XHTML compatible, but I don't think its for a decision for a Drupal newbie like me to make.

--
Tony Freixas

freixas’s picture

freixas commented

To continue my sad tale...

I dropped TinyMCE for the moment and switched my system back to using XHTML and serving it as application/xhtml+xml to the browsers that can handle it.

I ran into another problem with this scenario. I was entering a post and had an error in my XHTML code. This results in a screen with a single XML parser error message (usually cryptic) and a line number that doesn't correspond to any source file. Luckily, Firefox's View Source command displays a useful listing and the Go To Line command got me to the error.

The problem is that without fixing the error, it's hard to fix the error—since the page doesn't display, you can't use the entry field to correct the error and resubmit. So I used the back key to get me to a page where I could fix the problem.

If I could flip a switch and have everyone's Drupal installations serve XHTML properly to any browser that handled it, I think I would quickly have a lynch mob after me. This begs the question of why Drupal is written in XHTML if XHTML is not really an option, even for browsers that can handle it.

The solution for posting would be to run all posts through a validating XHTML parser and reject previewing or accepting invalid code. This involves some work, both in generating the code (which could include PHP statements) and generating useful error messages.

It seems simpler to stick with a forgiving parser—that is what HTML 4 is for, as well as the proposed HTML 5. For those who don't know, the reason for creating HTML 5 is because the current roadmap of HTML 4 -> XHTML 1 -> XHTML 2 is pretty crazy. From HTML 4 to XHTML 1, you have to modify the HTML code, any Javascript and the way the file is served.

Apparently, the proposed XHTML 2 standard is incompatible with XHTML 1, so that path also requires a rewrite. I don't know the details.

The proposed path from HTML 4 -> HTML 5 is much does not require a rewrite and you retain a forgiving parser.

Anyway I turned off XHTML again and generated an HTML 4.01 doctype. I also had to go through all the Drupal code and convert '/>' to '>', so I could validate the code.

To the Drupal gods, I suggest that you revisit this issue. HTML 4.01 Strict might be the way to go for Drupal 6. That may sound like blasphemy, particularly since the Drupal core appears to be valid XHTML (at least, as far as the modules I used). But to get really use XHTML, you would have to solve the posting problem. And then once that was solved, you have nowhere to go without a rewrite. With HTML 4, you can be less concerned about posting errors and you have a compatible upgrade path to HTML 5.

Just my 2 cents.

P.S. It seems crazy to me that TinyMCE generates XHTML code but isn't itself valid XHTML.

--
Tony Freixas

AdrianB’s picture

AdrianB commented

I think plik makes good points. I wouldn't pretend to know the ins and outs of this, but from what I've read recently (by Smart People™ :) it seems like HTML5 has a bright future. I don't think it will happen, but it would nice if it was a choice to use HTML 4.01 Strict in Drupal.

wvperegrine’s picture

wvperegrine commented

I agree with plik and AdrianB in some regards. I think the comment in the source header of Drupal.org is a cop-out, although I hasten to add that the community's work on this project has been extraordinary and I appreciate Drupal very much.

I urge the developers to implement Drupal in a way that delivers valid HTML/XHTML output. That simply means specifying a doctype and delivering code that adheres to that standard of HTML or XHTML (whatever standard is agreed upon). This would be a fairly small fix that would enhance the whole project's credibility.

The invalid code on Drupal.org is trivial and fairly easy to fix, and I challenge anyone to explain how it adds to the user experience in any way. Part of it ... <wbr /> ... is simply a typo, and some here have mentioned that they are creating standards-compliant sites using Drupal by editing the source code after the fact. Small price to pay, you might say, considering how well Drupal works in other ways!

I propose that we make that post-production editing unnecessary. Adhering to W3C standards goes a long way toward establishing solid credibility in the world of web development, so it is important to implement this better. I've participated in the W3C's working group on HTML5 development. It will be a very robust standard and I think it will improve the user experience in significant ways. But XHTML 1.0 Transitional and XHTML 1.0 Strict are good, too (I don't like frames, but that's my problem). And for now, as others have mentioned, there may be good reasons to prefer HTML 4.01.

It's also been mentioned that you can change your doctype preference in Drupal. I'm simply saying that, when you do, Drupal should deliver fully compliant code. The key thing is to tell the user agent (browser, screen reader, PDA ...) what you're delivering, then deliver it correctly. Right now Drupal does not quite do that, and this really should be fixed.

Also, as standards-lazy as they are, IE6 and IE7 will generally render XHTML pages well in "quirks mode" if you leave off the initial line in the XHTML doctype specification: <?xml version="1.0" encoding="utf-8"?> ... Technically, that's a bit of a hack, but it validates and renders reasonably well on all browsers if you've done everything else right. And don't worry ... Microsoft will catch up with the real world eventually, even if we have to drag them kicking and screaming!

freixas’s picture

freixas commented

Just a reminder for anyone wanting to produce valid XHTML. XHTML imposes requirements on Javascript. Right now, I don't remember if Javascript produced for XHTML will work with HTML. When I tackled the problem some time ago, I wrote one version of Javascript code for XHTML and one for HTML, which implies that I thought there were some incompatibilities.

In other words, producing Web pages that will work both when using an XML parser and an HTML parser is much more difficult than you might think if Javascript is involved. The biggest irritation is the removal of document.write() but there are others.

See http://hixie.ch/advocacy/xhtml for details.

In my comment above, I also point out the extra work need to handle syntax errors in postings when using XHTML.

--
Tony Freixas

JStarcher’s picture

JStarcher commented

plik, I'm with you.

Here are some very interesting and informative articles you may like to see:

http://www.webdevout.net/articles/beware-of-xhtml
http://www.webdevout.net/articles/validity-and-well-formedness
http://www.goer.org/Journal/2003/04/index.html#results

I wrote a quick fix to make Drupal HTML 4.01 Strict valid. Change your $head in your template file to this:

 $html_head = str_replace(" />", ">", $head);
        print $html_head

and that will do the trick.

-Jordan

Toe’s picture

Toe commented

Why do people keep saying/implying that XHTML sent as text/html is incorrect? I assure you, it's quite explicitly allowed by the XHTML 1.0 strict standard.

XHTML Documents which follow the guidelines set forth in Appendix C, "HTML Compatibility Guidelines" may be labeled with the Internet Media Type "text/html" [RFC2854], as they are compatible with most HTML browsers.

http://www.w3.org/TR/xhtml1/#media

I'm perfectly well aware that there are disadvantages to it, I read Ian Hickson's posting a long time ago. I can also understand if you put it as something to avoid on your 'best practices' list. But don't call me wrong when I'm following the spec to the letter.

freixas’s picture

freixas commented

Interesting point! I continue to keep learning more about this issue every day.

Appendix C has some interesting limitations. One problem in taking this path is: how do you validate your XHTML code against Appendix C?

One possibility, I suppose, would be to use Firefox and serve the files as application/xhtml+xml. That's one approach, since section 5.1 does not exempt you from producing valid XHTML. But the XHTML parser won't catch all the additional requirements in Appendix C. How exactly do you know if you are "following the spec to the letter"?

Also, while section 5.1 may grant permission to send XHTML to an HTML parser, the XHTML code is still invalid HTML (at least as I understand it). I mention this only because many of the postings above refer to "invalid" rather than "incorrect".

--
Tony Freixas