List of bad Robots

  • David26
  • Banned
  • Banned
  • David26
  • Posts: 29

Post 3+ Months Ago

I have put together a list of known bots that you do not want to spider your site ;) You can view/download the list here:

List of BAD ROBOTS
  • Anonymous
  • Bot
  • No Avatar
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post 3+ Months Ago

  • ATNO/TW
  • Super Moderator
  • Super Moderator
  • User avatar
  • Posts: 23456
  • Loc: Woodbridge VA

Post 3+ Months Ago

I can't say as I disagree as to your reasoning or most of those I see on your list, but could you please expound on it a bit. Obviously your link gives some reasons in your post, but people here that don't click the link have no idea why they should. Not to mention some small instructions for the nOObs on how to create a robots.txt file to ban unwanted bots.
  • David26
  • Banned
  • Banned
  • David26
  • Posts: 29

Post 3+ Months Ago

I figured that the topic that is a sticky in this forum that helps explain more about robots.txt:

http://www.ozzu.com/other-search-engines/common-search-engine-terms-t24816.html

Would help anyone who has any questions about how to create a robots.txt file. Not to mention that some noobs have asked how or where to place the robots.txt and I have answered this.
  • ATNO/TW
  • Super Moderator
  • Super Moderator
  • User avatar
  • Posts: 23456
  • Loc: Woodbridge VA

Post 3+ Months Ago

I suppose it wasn't so much how to create a robots.txt file, but rather why those are bad bots. And I did read the post in your link,, obviously....but nOObs reading here for the first time won't have a clue why you are telling us to ban those bots unless they click your link and understand what you are saying. Even your post at the link doesn't give an overall comprehensive reason. Mostly I'm just pointing out that you posted a "do this because it's bad" kinda post without giving much of an explanation as to the why....

catch my drift?

//edit -- for the record, I think I pretty much agree with the bots on your list.
  • David26
  • Banned
  • Banned
  • David26
  • Posts: 29

Post 3+ Months Ago

Well if anyone has a question then they could ask. I had posted that topic a long time ago and noone has ever asked why each bot is considered bad.

Post Information

  • Total Posts in this topic: 5 posts
  • Users browsing this forum: No registered users and 12 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
cron
 

© 1998-2014. Ozzu® is a registered trademark of Unmelted, LLC.