I have been confused over the issue of deleted pages and search engines for some time now.

I have a good amount of News Feeds coming in, I age them for 6 weeks then I use the "auto delete" feature of Feed API for news over 6 weeks old.

My question is, how do the search engines treat a site that is always deleting content?

Will XML Sitemap keep me out of trouble?

Suggestions and comments welcomed.
thanks

Comments

Anonymous’s picture

Anonymous (not verified) commented

Search engines love changing content especially Google. A sitemap.xml file is what keeps your links fresh in the search engines. I use xmlsitemap on my sites that change often with no issues.

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented

Using this module will definitely help the search engines find your new content and help de-index your deleted content since it will not be contained in the sitemap.

avpaderno’s picture

avpaderno
he/him
Primary language Italian
Location Brescia, 🇮🇹 🇪🇺
commented

I noticed that Google doesn't seem to de-index the content of my site; I keep to get an error message about a page not found, when the link is not anymore in the sitemap.
Probably it's just Google to do so, but it should be reported.

avpaderno’s picture

avpaderno
he/him
Primary language Italian
Location Brescia, 🇮🇹 🇪🇺
commented
Title: question to group re: delketed pages & search engines » Question to group: deleted pages & search engines
green monkey’s picture

green monkey commented

hey gang ... thanks

I have also seen #3 above .... much to often.... pages not removed - so I'm doing everything I can to redirect to home page

I have one more question

I see that default is set up to add Nodes to site, and that Tax Terms is turned off by default

Is this so, that there are "not" dupe pages being indexed?

thanks

dave reid’s picture

dave reid
he/him
Primary language English
Location Nebraska USA
commented
Status: Active » Fixed

Node pages are included by default since most users of the module will always want their content indexed. Not all users of the module want their taxonomy terms indexed. Anyway, I'm marking this as fixed since all questions have gotten an answer. Aside from keeping current content (no deleted content) in the sitemap.xml and submitting the sitemap to search engines when changed, there's not much else that the module can do.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.