Andrew & group -- We've benefitted heavily from Linkchecker *without* this feature thus far, but I was wondering:

Is Linkchecker supposed to be able to change an anchor tag in the body of a node (a link....) to reflect a new location (after getting n 301s)?

We've had the "Update Address" feature selected and running for a couple of weeks, but it isn't automagically updating inline anchor tags. I wasn't sure whether that was because I was expecting magic, or perhaps something is wrong.

If this is supposed to work, please change the status of this to bug report and I'll provide all the information you desire.

Thanks!

TC

Comments

hass’s picture

hass commented
Priority: Minor » Normal
Status: Postponed (maintainer needs more info) » Fixed

Do you mean a fragment like #jumphere in a link http://example.com/foo#jumphere?

How good that I've thought about this only 2 days ago... :-). The module save the links with fragment and should test the links with fragment, but there is no technical way to solve such a redirection in a standard way and I haven't verified yet if drupal_http_request will send the fragment when checking. Not sure - we need to investigate. If a server would provide you a 301 redirect for a request with a fragment - yes the link will be updated, but I would not expect that someone will send you this redirect :-(. Do you have an example URL for testing?

So on the end of the day I expect that you need to manually fix it in most if not all cases... the module cannot really guess where it should send a user - if there is no redirection.

hass’s picture

hass commented
Priority: Normal » Minor
Status: Active » Postponed (maintainer needs more info)

I'm changing the status for investigation, but give it a low prio.

ifbyphone1’s picture

ifbyphone1 commented
Priority: Normal » Minor
Status: Fixed » Postponed (maintainer needs more info)

Andrew, my issue is more basic. Say I have a link in the body of a node. Nodetype = page. Input type = Full HTML or Filtered HTML. Example below:
Lorem ipsum dolor sit, visit <a href="http://www.foo.com/bar">here</a>

And in this example, www.foo.com/bar is returning a 301, redirecting to www.foo.com/barbar. This 301 shows up in the Broken Links report, and after receiving a 301 n times (n = 3 for us), then *should* Linkchecker update the link in the body of my node to read:
Lorem ipsum dolor site, visit <a href="http://www.foo.com/barbar">here</a>

If it should, it is not doing so. We have 300 - 400 links that are returning 301, and have been 301 for a month with no change to the link.

If it should not update the link, then I'll plan to make corrections manually.

Thanks!

TC

hass’s picture

hass commented

This need to work and was tested...

Ah - maybe the issue I've documented in the readme. There is a known bug in core and additional you need to verify the remote link if the remote server is properly configured / working properly. Please provide me 2-3 real links I can debug... I saw many buggy web servers myself within the last months :-(((. I don't know why there are so many NooB admins out there unable to configure their servers properly...

ifbyphone1’s picture

ifbyphone1 commented

Andrew,

I recall looking at that issue when first installing Linkschecker, and thinking "well, our servers will behave nicely, so...." I didn't think it would be a problem.

I very much appreciate your offer to debug -- THANK YOU! Here are a few links that are appearing in the list (I just checked these with wget, and they are resolving to a 200).

http://www.ifbyphone.com
http://public.ifbyphone.com/verifymenow
http://public.ifbyphone.com/services/developer-api
http://www.smithonvoip.com/voip-news/ifbyphone-continues-to-impress/

...and one more that generates a 301 to a 302 to 200:

http://www.nytimes.com/2008/08/11/business/media/11auto.html?ref=business

Best,

TC

hass’s picture

hass commented
Title: Pls Clarify: Does LinkChecker Modify Anchor Tags? » Permanently moved links not updated
Category: support » bug
Priority: Minor » Normal
Status: Postponed (maintainer needs more info) » Fixed

How annoying - you are affected by a very small nobrainer bug... it's caused by a small change I made after all my testing to make the automatic update more fails positive save. The check needs to get a status code 200 on the redirected page to allow the change :-(. So - please install latest DEV or apply the patch http://drupal.org/cvs?commit=194120 and let cron run once on the failed links. Then all 301 links are automatically fixed. Afterwards we need to take a look again what's left and that may need a manual fix.

The module will not automatically update redirects like "301 to a 302 to 200" as it expects a 200 after a 301. Not requiring this 200 after a 301 would result in fails changes! So better be save - than sorry. You need to fix these buggy redirects by hand.

ifbyphone1’s picture

ifbyphone1 commented

Andrew, thanks very much for the diagnosis and the patch. I cheated and just edited linkchecker.module to make the one character change. A few hours later (about 60 or so cron runs later), I got an ominous entry in my logs:
linkchecker Apr 7, 2009 6:33pm Wiped out obsolete links
Is there a table I can check to get a little more detail on what linkchecker did? I'm running the patch on a DEV box right now, and if I can confirm it is making all the right changes, I'll happily promote the patch to prod.

Gratefully,

TC

hass’s picture

hass commented

Only to note, my name is not Andrew.

This is the clean-up process that runs once per week by default. It wipes out dead and no longer used links. You need to wait until your links are rechecked. You can only force this by reducing the re-check interval to 1 day or so, but you could also wait if you can wait...

ifbyphone1’s picture

ifbyphone1 commented

My apologies, mate: thought it was your first name! Sorry!

Links just started changing moments ago -- things are looking good!

At the risk of hijacking my own thread (happy to start another): could linkchecker be modified to check & change Only local images are allowed. tags as well? We are always finding broken links back to old artwork...

Thanks again for the patch,

TC

hass’s picture

hass commented

Open a new case, please and give EXAMPLES - I do not understand what you are talking about if you talk about "tags".

ifbyphone1’s picture

ifbyphone1 commented

Sorry. I wrote:
<IMG> tags
Drupal filtered to:
tags

I'll open another -- this one is closed.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Anonymous (not verified) commented
Version: 6.x-2.1 » 6.x-2.x-dev
Status: Closed (fixed) » Active

Hi Alexander, Relating to permanently moved 301. before trying to understand cause for my other issues i thought i would get as many 301's fixed as i could first

I have had single node set to be cheked, 18 checks later still 301. I noticed this thread a few days ago and to latest

// $Id: linkchecker.module,v 1.7.2.61 2009/04/07 21:16:07 hass Exp $

The link in question is http://www.forbiddenplanet.com

settings User Agent Drupal
Check Interval 1day
Cleanup 1 day

Update permanently moved links after one failed check

Anonymous’s picture

Anonymous (not verified) commented

Furtehr info which seems to indicate that i am affected by your comment in the second paragraph of #6. which is ironic that out of 1 link 1 is affected :)

#1 Server Response: http://www.forbiddenplanet.com
HTTP Status Code: HTTP/1.1 301 Moved Permanently
Server: nginx/0.6.32
Date: Sat, 02 May 2009 20:07:17 GMT
Content-Type: text/html
Content-Length: 185
Connection: close
Location: http://forbiddenplanet.com/
Redirect Target: http://forbiddenplanet.com/

#2 Server Response: http://forbiddenplanet.com/
HTTP Status Code: HTTP/1.1 302 Moved Temporarily
Server: nginx/0.6.32
Date: Sat, 02 May 2009 20:07:17 GMT
Connection: close
Location: http://www.forbiddenplanet.com/fp
Content-Length: 0
Redirect Target: http://www.forbiddenplanet.com/fp

#3 Server Response: http://www.forbiddenplanet.com/fp
HTTP Status Code: HTTP/1.1 301 Moved Permanently
Server: nginx/0.6.32
Date: Sat, 02 May 2009 20:07:17 GMT
Content-Type: text/html
Content-Length: 185
Connection: close
Location: http://forbiddenplanet.com/fp
Redirect Target: http://forbiddenplanet.com/fp

#4 Server Response: http://forbiddenplanet.com/fp
HTTP Status Code: HTTP/1.1 200 OK
Server: nginx/0.6.32
Date: Sat, 02 May 2009 20:07:18 GMT
Content-Type: text/html;charset=UTF-8
Connection: close

hass’s picture

hass commented
Status: Active » Closed (fixed)

Good analysis :-) - this is for sure the issue I described in #6 part 2.

I'm sorry, but automatically updating such a buggy redirect looks tooooo risky to me. You need to verify this link by hand and correct it... better save than sorry.

Anonymous’s picture

Anonymous (not verified) commented
Status: Closed (fixed) » Active

Hi Alexander, after fixing the above link i decided to allow some more to be added, this time internal

Cron has run and I have a couple of entries in my Broken Links report

node/2536 has an internal link to node/4187 returning a 301

Same settings as above.

if i select edit node 2536 and look at the link in question it shows node/4187 The page shows a Revision for today coinciding with the cron run.

In the recent Log entries i have the following message which relates to the nodes in question

Changed permanently moved link in /raymond-feist/faq/2536/i-ve-heard-that-i-can-mail-my-books-to-ref-to-get-them-signed-how-do-i-do-thi from http://www.crydee.com/node/4187 to http://www.crydee.com/raymond-feist/whats-happening/other-events.

So even though it is reporting that a change has taken place it appears not to have. the second 301 link processed is in the same situation

please advise

hass’s picture

hass commented
Status: Active » Closed (fixed)

This issue has been solved, please do not re-open it.

Internal links have never been tested with 301 auto-update function and are not yet supported as it's ultra risky to change them. This may change after #451456: 301 auto-update could break links is solved.

swduncan’s picture

swduncan commented

I'm not sure if I should use this issue, or start a new one, but my permanently moved links are not being updated either. I followed the instructions above, but still no change.

I started with the stable release, and then upgraded to the dev release when I found this thread. I did a database update afterward.

The only entries I'm finding in the log for Link Checker are "Link X has changed an needs to be updated", "Broken Link found" and "Wiped out obsolete links"

Some 301-error links, from the very long list:
www.garmin.com
www.hoovers.com
http://www.mobi-systems.com/
http://www.suse.com/

I can send more if needed.

The threshold for changing 301's is 3 times, they have failed 5 times.

Any ideas? Do I have something set wrong?

hass’s picture

hass commented

Yeah - there was a bug that has been fixed in DEV after the 2.1 release. Therefore try DEV version, please.

swduncan’s picture

swduncan commented

The version I'm using is 6.x-2.x-dev, downloaded on 22 June. The page for Links Checker shows the dev release as 14 May. Is the dev release date not showing properly? If it is, I've already got and have been using the latest dev. I just confirmed on my 'modules' page that it's 6.x-2.x-dev.

hass’s picture

hass commented

Latest commit was on 14.5 so you have the latest version.

swduncan’s picture

swduncan commented

Ok. Is there anything I can do to help debug this?

hass’s picture

hass commented

Not sure if there is anything to debug

swduncan’s picture

swduncan commented

I'd like to figure outwhy my permanently redirected links are not getting updated.

hass’s picture

hass commented

Try to figure it out, but use DEV, please.

hass’s picture

hass commented

Maybe of interest #1875602: Check URL fragment identifiers in content.