robots

  • tastysite
  • Proficient
  • Proficient
  • User avatar
  • Posts: 349
  • Loc: Brighouse, West Yorkshire, England

Post 3+ Months Ago

I have just upload my site to the web, yay, im so proud, I know ive not done much but never mind!
Anyhow ive just thorght that there a few pages that realy dont need to be indexed now i know about the
Code: [ Select ]
<meta name="robots" content="noindex" />
but my site is run through php to get the pagers so and the robots code has to be in the head of the site so how do I get the code to show when I want it to and not when I don't Its probley very simple and im probley missing something fundamental here but just cope with me, thanks.
  • Anonymous
  • Bot
  • No Avatar
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post 3+ Months Ago

  • dhanjit
  • Newbie
  • Newbie
  • dhanjit
  • Posts: 13
  • Loc: India

Post 3+ Months Ago

Make a file and name it as http://www.yoursite.com/robots.txt and upload it on the server. In the Robots.txt file write the pages which you don't want search engines to crawl. Use these text-

User-agent: *
Disallow: /pagename
Disallow: /pagename
  • ramone_johnny
  • Graduate
  • Graduate
  • ramone_johnny
  • Posts: 107
  • Loc: Brisbane, Australia

Post 3+ Months Ago

Yep, what he said. Setting robot no follows within individual pages makes for nothing more than an un-manageable mess.
  • thomas_09
  • Novice
  • Novice
  • thomas_09
  • Posts: 19

Post 3+ Months Ago

dhanjit wrote:
User-agent: *
Disallow: /pagename
Disallow: /pagename


yep... This is the best way to prevent specific pages to get crawled by search engiens.
  • UPSGuy
  • Lurker ಠ_ಠ
  • Web Master
  • User avatar
  • Posts: 2733
  • Loc: Nashville, TN

Post 3+ Months Ago

Are you two the yes-men or what? Solution! Concur! Yep!

Post Information

  • Total Posts in this topic: 5 posts
  • Users browsing this forum: No registered users and 4 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
 

© 1998-2014. Ozzu® is a registered trademark of Unmelted, LLC.