Disallow: /application?if_this=yes

  • joebert
  • Sledgehammer
  • Genius
  • No Avatar
  • Joined: Feb 10, 2004
  • Posts: 13455
  • Loc: Florida
  • Status: Offline

Post November 28th, 2006, 6:04 pm

Any ideas to get Google to ignore pages depending on a GET variable ?

Would somthing along the lines of this work in robots.txt ?

Code: [ Select ]
Allow: search.php
Disallow: search.php?mode=nothing_g_wants_to_see
  1. Allow: search.php
  2. Disallow: search.php?mode=nothing_g_wants_to_see
Strong with this one, the sudo is.
  • Anonymous
  • Bot
  • No Avatar
  • Joined: 25 Feb 2008
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post November 28th, 2006, 6:04 pm

  • meman
  • Web Master
  • Web Master
  • User avatar
  • Joined: Aug 03, 2004
  • Posts: 3432
  • Loc: London Town , Apples and pears and all that crap
  • Status: Offline

Post November 28th, 2006, 10:54 pm

Yeah that should work. That would be the best way to do it too, With a bots text file.

You might also want to consider putting a noindex meta tag in the pages you don't google to index.
Code: [ Select ]
<meta name="ROBOTS" content="NOINDEX">


Any other method (user agent or IP) might get you into trouble.
GodBeGone - Atheist Blog
  • joebert
  • Sledgehammer
  • Genius
  • No Avatar
  • Joined: Feb 10, 2004
  • Posts: 13455
  • Loc: Florida
  • Status: Offline

Post November 28th, 2006, 11:03 pm

I'm trying to keep search engines out of the sites search on specific modes for the purpose of reducing load, so anything that requires the bot to actually fetch the page to make work isn't what I'm looking for.

I need to review all links to the search & make sure there's nowhere that the order of variables would differ, then I'll try the ~bots.txt

You know, now that I think about it, I may need to review my rel="nofollow", I could have sworn I marked any links to this resource in that mode with them. :scratchhead:
Strong with this one, the sudo is.

Post Information

  • Total Posts in this topic: 3 posts
  • Users browsing this forum: No registered users and 30 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
 

© 2011 Unmelted, LLC. Ozzu® is a registered trademark of Unmelted, LLC.