Search engines indexing settings
Is there a site-wide setting to disallow search engines to index pages during different phases of development Sitefinity 6.3) ?
In an enterprise environment with multiple phases of deployment (development, staging, QA, etc.) and where sometime IP's need to be public, we don't want to have SE's crawl our site.
I know there's an option for this at the page level, but for a site with dozens of pages, disabling (meta robots noindex) and then enabling this before production, is not a very efficient approach.
Please advise. Thanks.
Hi AG,
Just put a robots.txt file in the root of your site. That can block all spiders from all pages as explained here. Don't ever forget to adjust the file when publishing the live website. That could cause a lot of trouble.
Hi Arno,
Thanks for your super quick reply. Yes, that's an option if you have access to the project or deployment server. Another option would be to temporarily code the meta in your custom master page/template and leave the page level option for SE's checked (as is per default = allow indexing).