Can you please point me to any references on how to exclude robots and search engines from indexing an unfinished, draft site? Would it be strictly through robots.txt? Is there anything in the Newscast package that can be used?
Also, is it useful to disallow a crawl of any of the WordPress / Newscast directories, and is there anything prepackaged for that in the Newscast download package?
I believe this can help you. It’s not advised to deny robots/crawlers since they generate traffic, lots of it. I believe WordPress has a built in script so drafts can’t be seen. To my knowledge Newscast doesn’t offer these kind of functions.
Viewing 2 posts - 1 through 2 (of 2 total)
The topic ‘Exclusion of robots / search engines til site is finished’ is closed to new replies.