I have installed robotstxt module and also removed robots.txt file from root directory. After installing, this module is not generating robots.txt file. If I try to access /robots.txt Iam getting 404 Not found error. Also changes done in /admin/config/search/robotstxt is not reflecting.

Comments

Nikhilesh Gupta created an issue. See original summary.

naresh_bavaskar’s picture

naresh_bavaskar
Location Pune, Maharashtra
CreditAttribution: naresh_bavaskar as a volunteer and at QED42 commented

might be you have installed a fast404 module which causes an issue normally to robots txt module. if yes then you need to configure fast404 configuration first.

nikhilesh gupta’s picture

nikhilesh gupta CreditAttribution: nikhilesh gupta as a volunteer and at Infosys commented

@naresh_bavaskar Fast 404 is not installed for my site.

mdixoncm’s picture

mdixoncm CreditAttribution: mdixoncm at ComputerMinds commented

Have you maybe got fast404 configured in your settings file? The default configuration is to fast404 on missing .txt files - of which robots.txt would be one ...

ccjjmartin’s picture

ccjjmartin CreditAttribution: ccjjmartin as a volunteer and at Four Kitchens commented
Status: Active » Closed (cannot reproduce)

Nikhilesh, can you provide additional details on how to reproduce this issue and reopen? It is working on my environment with only core and robotstxt installed.