Comments on: How to use Robots.txt file effectively http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/ A latest Technology Buzz Thu, 04 Mar 2010 15:33:47 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Tech Maish http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/comment-page-1/#comment-32931 Tech Maish Mon, 05 Oct 2009 16:49:21 +0000 http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/#comment-32931 If we just add this code in our robots.txt file, then it is ok or not. (For Wordpress) User-agent: * Disallow: / OR we have to add some more tags If we just add this code in our robots.txt file, then it is ok or not. (For Wordpress)

User-agent: *
Disallow: /

OR we have to add some more tags

]]> By: Marilyn http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/comment-page-1/#comment-29846 Marilyn Wed, 10 Jun 2009 22:02:17 +0000 http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/#comment-29846 Thank you for this post, it was very informative, even two years later. I'm interested to know about the effect of disallow. Do you know if the robots will stop coming back to check on your url if it finds a robots.txt like: User-agent: * Disallow: / I am trying to come up with a beta release strategy. I'd like to prevent indexing during the private beta phase, then turn indexing on when the site goes public. I'd love to hear your thoughts on this matter. Thank you for this post, it was very informative, even two years later. I’m interested to know about the effect of disallow. Do you know if the robots will stop coming back to check on your url if it finds a robots.txt like:

User-agent: *
Disallow: /

I am trying to come up with a beta release strategy. I’d like to prevent indexing during the private beta phase, then turn indexing on when the site goes public. I’d love to hear your thoughts on this matter.

]]> By: PPC Coach Review http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/comment-page-1/#comment-24286 PPC Coach Review Thu, 05 Feb 2009 08:46:39 +0000 http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/#comment-24286 Great article. I'll definitely be back. All the best, Leonel Great article. I’ll definitely be back. All the best, Leonel

]]> By: 31 Days to Building a Better Blog « Sztuka Spekulacji http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/comment-page-1/#comment-10334 31 Days to Building a Better Blog « Sztuka Spekulacji Tue, 05 Aug 2008 19:37:28 +0000 http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/#comment-10334 [...] How to use Robots.txt file effectively by Vijay Shinde [...] [...] How to use Robots.txt file effectively by Vijay Shinde [...]

]]> By: What Everybody Ought to Know About Blogging - 97 Blog Tips : Opportunities for Life http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/comment-page-1/#comment-112 What Everybody Ought to Know About Blogging - 97 Blog Tips : Opportunities for Life Thu, 23 Aug 2007 16:38:53 +0000 http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/#comment-112 [...] How to use Robots.txt file effectively by Vijay Shinde [...] [...] How to use Robots.txt file effectively by Vijay Shinde [...]

]]> By: What Everybody Ought to Know About Blogging - 97 Blog Tips http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/comment-page-1/#comment-111 What Everybody Ought to Know About Blogging - 97 Blog Tips Thu, 23 Aug 2007 12:49:20 +0000 http://www.etechbuzz.com/how-to-use-robotstxt-file-effectively/#comment-111 [...] How to use Robots.txt file effectively by Vijay Shinde [...] [...] How to use Robots.txt file effectively by Vijay Shinde [...]

]]>