Meta Tag to block pages from being scanned by spiders/bots

Is there a way to block email harvesters/spiders/bot thingys from accessing a page? I am thinking this because maybe we could use it only on the pages where email addresses are held (if you have dedicated pages for contact info)

Anyone know?

Well you could make it so your page only accepts certain browsers that are passed through the User-Agent field. However most harvestors I think simply fake this field to pass it off as if they are just a regular IE browser which means you might limit some of the harvestors, but many would still get through like normal.

mmm bummer! stupid spammers!!

I'm assuming you're probably not going to want to do this, but you could have a contact page that is only accessible by registered members using passwords.

I'm also thinking, you could make life difficult for spiders/bots by using unconventional code (like using contact info inside iframes), but I'm assuming you want to do this for convenience, and anyway that wouldn't catch all spiders out.

Looks like you'll probably have to stick with contact info as images --- you could write a PHP script or something that takes text as an input and automatically places an image containing the text!?

mmm the image thing sounds good but then I would have to have the email in the Source code

Oooh, I read somewhere about the Document.write .... and doing Email addresses so Spiders miss them as random text and not addy's

I will find it out.

I don't quite understand what you mean -- I was refering to simply doing an <img src="emailpic.png">. Rather than reading a file containing a list of email addresses as text to display, you could read a list of images that display email addresses as png's or something. The source code wouldn't need to contain any 'email@address.com' at all...

oh! i missunderstood

yes I suppose I could do that

ROSE...

I found it, register to bravenet. they give out good Webmasters tips



that bit of code when tweaked fools any Bot or spider

rose hunni, u thought about a robots.txt file?

you can include/exclude the files and directories you want bots/spiders to have access to

there's a short tutorial here http://www.searchengineworld.com/robots ... torial.htm

not sure if it'll work, but it's worth a try

Ok, i just found some useful info about Atomic.

They use the robots.txt to block search engines. as they were searching the database and slowing the System to a halt.

anyways, my Idea for Emails.

They could be held in the Document.write thingy, in a email.js file. then just set the Variables in your Document.



that is the HTML code

this is the JS script.


If you place the above .Js script into a email.js file you can call it after setting the variables as shown above. then this will insert an email address without letting Spiders see em cos they are built from two seperate files.

Its 8:30 in the morning and I havent finished my Corn Flakes yet - I will tackle this later today - dont the email harvesters read text off the actual website not just the code?

xox for your input!

I know DD had to do that - it went through every user and slowed everything down - in fact one week i got so many through mine it counted them as visits and put me in the top ten !!

Ok rose hun,

Lol@ DD getting top spot.

the bit of code and the variables make the Emails from totally seperate documents, eg the Variables are set in the page and the email address assembled In the Email.JS so the spider or Robot never realises it found an email just some more Code and a file that writes something, which is called for in a page.

It would have to be a smart Spider to assemble the code and write the document before It searched.

I should be here in the "Morning" Since I'm off work with a bad toothache... eg I had tooth pulled out today and am in ickle pain. No sympathy please it my fault. << Katie will complain if you give me sympathy she says I been milking it LOL....

I am going to try to do it later for sure !!!
Anything to stop those dumb spam thingies!!!

well,

I should be back in the morning. So if you need any help and i'm in MSN. Just shout