Search engines indexing settings

Posted by Community Admin on 04-Aug-2018 17:25

Search engines indexing settings

All Replies

Posted by Community Admin on 24-Apr-2014 00:00

Is there a site-wide setting to disallow search engines to index pages during different phases of development Sitefinity 6.3) ?

In an enterprise environment with multiple phases of deployment (development, staging, QA, etc.) and where sometime IP's need to be public, we don't want to have SE's crawl our site.

I know there's an option for this at the page level, but for a site with dozens of pages, disabling (meta robots noindex) and then enabling this before production, is not a very efficient approach.

Please advise. Thanks.

Posted by Community Admin on 24-Apr-2014 00:00

Hi AG,

Just put a robots.txt file in the root of your site. That can block all spiders from all pages as explained here. Don't ever forget to adjust the file when publishing the live website. That could cause a lot of trouble.

Posted by Community Admin on 24-Apr-2014 00:00

Hi Arno,

Thanks for your super quick reply. Yes, that's an option if you have access to the project or deployment server. Another option would be to temporarily code the meta in your custom master page/template and leave the page level option for SE's checked (as is per default = allow indexing).

This thread is closed