Back To Other Search Engines & Directories

A question on robots.txt file!

  • 3DProf4online
  • Beginner
  • Beginner
  • No Avatar
  • Joined: May 19, 2008
  • Posts: 41
  • Status: Offline

Post October 8th, 2008, 12:38 am

Hey guys, I just would like to discuss with what benefits can be gained from robots.txt file when promoting website? Could you please explain how to upload this file to hosting? Is it uploaded to public_html root directory as well?
  • Anonymous
  • Bot
  • No Avatar
  • Joined: 25 Feb 2008
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post October 8th, 2008, 12:38 am

  • Don2007
  • Web Master
  • Web Master
  • No Avatar
  • Joined: Nov 21, 2006
  • Posts: 4924
  • Loc: NY
  • Status: Offline

Post October 8th, 2008, 5:44 am

The purpose of a robots.txt file is to stop bots from crawling parts of your site that you want to keep private. However, at the same time, it's telling the world where those private things are located. From a security point of view, there are people who look for the robots.txt file if they have some interest in compromising your site. I wouldn't use it.
How do you know when a politician is lying? His mouth is moving.

Page 1 of 1

Post Information

  • Total Posts in this topic: 2 posts
  • Users browsing this forum: No registered users and 22 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
cron
 

© 2011 Unmelted, LLC. Ozzu® is a registered trademark of Unmelted, LLC.