URL filter fails on URLs surrounded by HTML entities

This patch adds   to the list in the regular expression in _filter_url($text, $format) to solve the first issue.

Hereby

Drupal 7 patch against HEAD with test.

HEAD is broken

This patch succeeds testing but doesn't address the example you use.
The test fixes   at the beginning of a URL but not at the end of a URL.

I'm not convinced URL filter should break processing on a trailing  . For example http://www.example.com/?foo=x =y seems to be a valid URL?

Sure thing. :) So we just fix the prefix, as the existing patch here seems to do already?

Worse in my opinion is that \s doesn't matches the unicode non-breaking space (U+00A0). We should add that to the list too.

Re: #10
Your bring up a good point.
However, certainly the   in the OP is not meant to be interpreted as part of a top level domain!

BTW, here's the W3C spec that defines semi-colon as a reserved character:
http://www.w3.org/Addressing/URL/url-spec.txt

#13 Thanks for the link to the spec. I actually was thinking to link to it too (or even read it) but since I was working on another bug I just left a quickie comment.

You say ; is a "reserved character" (it seems to be at least in the WAIS protocol). But reserved doesn't mean it cannot be used in a URL, on the contrary it means it will be used in a URL. (Just like / is also a reserved character to denote levels of hierarchy, ie directories.)

Re: top-level domain. I see this issue as similar to how we strip of trailing dots or other punctuation. For a string like "http://example.com/index.html." there is no way to absolutely know whether the dot is part of the URL or just there because we are at the end of the sentence. Our experience (or you could do a real statistical study) just shows that in 99+% of cases it is punctuation, not part of the URL.

This issue is similar, the nbsp; can be non-breaking space, or it can be part of the URL, there is no way for the filter to know. Personally I think both are rare cases, so my strong preference is to vote for the alternative that makes the code simpler and not complicate the code for a corner cases that is open to debate. (A well defined corner case, like the leading nbsp; is ok.)

Is http://www.example.com/?foo=x =y really a valid URL?

- & denotes a new query parameter. To be part of the x-value, it would have to be escaped.

- ; and = are reserved characters. ; must appear after the query string, or otherwise needs to be escaped.

I've put my expectations into attached patch.

The same problem with 6.x
Example:

<p>Read more:&nbsp;http://en.wikipedia.org/wiki/Internet_forum</p><p>&nbsp;</p>