Does google spider forums?

  • Willy
  • Graduate
  • Graduate
  • Willy
  • Posts: 199

Post 3+ Months Ago

I am curious as to google spidering forums such as ozzu.

I run a PHBB forum, my mainsite has a PR4 but I never get any google hits to any of the forum pages. There are enough random comments in there that i'd expect one. I was wondering if I need to configure anything special in phpbb or any special meta tags to get the spiders to index everything?
  • Anonymous
  • Bot
  • No Avatar
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post 3+ Months Ago

  • Axe
  • Genius
  • Genius
  • User avatar
  • Posts: 5739
  • Loc: Sub-level 28

Post 3+ Months Ago

Yes, Google does spider forums...

One of the sites I setup (http://forum.reptilerap.com), running vBulletin, currently has 703 threads.

Each thread is shown by the forum as a regular pag (there's no "print view" that I'm aware of like there is on the PNphpBB forum on ReptileRooms.com), and I also installed an "archive" add-on, which produces text views of all the threads on the forum (http://forum.reptilerap.com/archive/).

Google knows of all the pages on the regular vBulletin site, and on the text view, without all the fancy extras, but the text views usually seem to turn up higher in appropriate searches.

With only 703 total threads, Google knows of 1750 pages on the site (you can see from the results it gives back that the text-based pages are being listed first).
http://www.google.com/search?q=allinurl ... 8&filter=0

Also, on ReptileRooms.com, which does have many other pages besides the forums, Google knows of 3340 pages right now. The forums don't show up til about page 18 in the results,but they are there (both in regular & print views)...
http://www.google.com/search?q=allinurl ... 8&filter=0

But, I've done heavy modifications to PostNuke & PNphpBB there to make Google like it a bit more.
  • digitalMedia
  • a.k.a. dM
  • Genius
  • User avatar
  • Posts: 5149
  • Loc: SC-USA

Post 3+ Months Ago

http://www.google.com/search?sourceid=navclient&ie=UTF-8&oe=UTF-8&q=ozzu+digitalMedia+profile
  • Axe
  • Genius
  • Genius
  • User avatar
  • Posts: 5739
  • Loc: Sub-level 28

Post 3+ Months Ago

And here's one I should've probly checked before :)

33K3 Pages
http://www.google.com/search?q=allinurl ... 8&filter=0
  • I, Brian
  • Novice
  • Novice
  • I, Brian
  • Posts: 17
  • Loc: Yorkshire, UK

Post 3+ Months Ago

Most forums (including phpbb and vBulletin 2.x) require the application fo a simple mod/hack to kill the sessions IDs for the Google bots. after that, they are open for indexing. :)

With vB3 it's automatic, though I've noted serious concerns about the duplication of content in vBulletin 3's dragging down the PR of a site. An issue I'm watching.
  • nuclei
  • Graduate
  • Graduate
  • User avatar
  • Posts: 147
  • Loc: On a mountain

Post 3+ Months Ago

I, Brian wrote:
Most forums (including phpbb and vBulletin 2.x) require the application fo a simple mod/hack to kill the sessions IDs for the Google bots. after that, they are open for indexing. :)

With vB3 it's automatic, though I've noted serious concerns about the duplication of content in vBulletin 3's dragging down the PR of a site. An issue I'm watching.


Hi Brian, You know me as WilliamC on Phils board. I have to agree on this point as I have seen other forums go down slightly in pr before. Not sure why google would think this way tho, as the more content, the more help people are likely to get.
  • alansmith
  • Newbie
  • Newbie
  • alansmith
  • Posts: 5
  • Loc: BA

Post 3+ Months Ago

Does anybody know where to get the hack for phpbb. I have a bb running that isnt getting spidered by google.
  • Axe
  • Genius
  • Genius
  • User avatar
  • Posts: 5739
  • Loc: Sub-level 28

Post 3+ Months Ago

phpbb mod_rewrite

vBulletin mod_rewrite

PostNuke mod_rewrite
  • suzie
  • Guru
  • Guru
  • User avatar
  • Posts: 1134
  • Loc: England

Post 3+ Months Ago

Excuse me guys n gurls..suzie is laughin her ed off here... :P

How does one know ones been googled lmaooooooooooooo

Actually a serious q !!!

8)
  • nuclei
  • Graduate
  • Graduate
  • User avatar
  • Posts: 147
  • Loc: On a mountain

Post 3+ Months Ago

look in your domains access logs
  • pompei
  • Graduate
  • Graduate
  • pompei
  • Posts: 117

Post 3+ Months Ago

You can also see which keywords were used to find your page:

look for something like this:

http://www.google.com?search

and what follows should be more details of the search. Bear in mind though not all log files are setup to include a list of referrers.

You should also see googlebot somewhere in your files, which is google's spider that indexes your files.
  • suzie
  • Guru
  • Guru
  • User avatar
  • Posts: 1134
  • Loc: England

Post 3+ Months Ago

Thankyou everyone,

pompei, i got this from your link:

Not Found
The requested URL /?search was not found on this server.

A few minutes ago the comp crashed <--- again...--> :evil:..and i got a different message about reporting the error to Ozzu !!

Anyway thanks again...i understand about google.

:wink:
  • Willy
  • Graduate
  • Graduate
  • Willy
  • Posts: 199

Post 3+ Months Ago

Its been a month from this initial post and still no spidering, I jsut came back and saw the phpbb hacks thread.

How exactly are hacks set up to be run?


My phpbb board is version 2.0.5



i ran the mod and it did not work :(
  • beefcakejcc
  • Graduate
  • Graduate
  • beefcakejcc
  • Posts: 103
  • Loc: Atlanta,GA

Post 3+ Months Ago

I found this article on how to set-up a forum that will be spidered. I haven't tried it yet but hopefully you'll find it helpfull.

http://www.darwinmag.com/read/120103/forums.html
  • phaugh
  • Professor
  • Professor
  • User avatar
  • Posts: 796

Post 3+ Months Ago

Is it possible that google is viewing the web forums as blogs? They have a very similar structure.
  • CazpianXI
  • Proficient
  • Proficient
  • User avatar
  • Posts: 285

Post 3+ Months Ago

I just did a quick search on Google:

http://www.google.com/search?q=%2Bthe+s ... 8&oe=utf-8

and it sure showed up all of (or nearly all of) Ozzu's threads.
  • SEO Guy
  • Beginner
  • Beginner
  • User avatar
  • Posts: 47
  • Loc: Canada

Post 3+ Months Ago

Willy email me your forum I'll take a look as long as it doesnt promote SEO or Kiddy porn I'll do the mod for free :-)
  • nuclei
  • Graduate
  • Graduate
  • User avatar
  • Posts: 147
  • Loc: On a mountain

Post 3+ Months Ago

nice offer guy :)
  • suzie
  • Guru
  • Guru
  • User avatar
  • Posts: 1134
  • Loc: England

Post 3+ Months Ago

Quote:
and it sure showed up all of (or nearly all of) Ozzu's threads

Yea i found my posts on google via ozzu lolololol....and i put in google some other word!
:)
Suzie.
  • nacho45
  • Born
  • Born
  • nacho45
  • Posts: 1

Post 3+ Months Ago

My hack was huge for my board. Initially I wasn't getting spidered at all, but about a week after the hack was installed I was getting spidered like crazy and indexed. I'd recommend it.
  • ccb056
  • Graduate
  • Graduate
  • User avatar
  • Posts: 189

Post 3+ Months Ago

yeah, google spiders my forums pretty well -> linky
  • bdemay
  • Born
  • Born
  • bdemay
  • Posts: 1

Post 3+ Months Ago

That's cool, I always thought forums were excluded. I guess it depends on whether the pages are static or not, or use of sitemaps etc?

BD
  • meman
  • Web Master
  • Web Master
  • User avatar
  • Posts: 3432
  • Loc: London Town , Apples and pears and all that crap

Post 3+ Months Ago

If a browser can view the page there is little reason a spider bot wouldn't be able to.
They may spider dynamic content a little slower that static pages, But this is because dynamic content put's more of a load on the server and the last thing google want to be known for is lagging peoples server.

Google has indexed 618,000 pages from phpbb.com, And they have the standard forum runninig as thier support forum (no mod_rewrite) with variiables in URL's and sessions.
  • corduroy090
  • Beginner
  • Beginner
  • corduroy090
  • Posts: 53

Post 3+ Months Ago

If Google spiders forums...then is it possible for forums to rank high on target keywords (without actually typing 'forum' in your search string)?
  • meman
  • Web Master
  • Web Master
  • User avatar
  • Posts: 3432
  • Loc: London Town , Apples and pears and all that crap

Post 3+ Months Ago

Of course. Forums turn up all the time on search results, If google thinks that page can give you the best result for your search.

Like if you were to search for some phpbb related problem, It is likely that the phpbb support forum will be high in those results.
  • recep
  • Beginner
  • Beginner
  • recep
  • Posts: 40
  • Loc: www.loanus.info

Post 3+ Months Ago

seo for web forums and mostly search engine friendly urls are very important issues...
  • meman
  • Web Master
  • Web Master
  • User avatar
  • Posts: 3432
  • Loc: London Town , Apples and pears and all that crap

Post 3+ Months Ago

recep wrote:
seo for web forums and mostly search engine friendly urls are very important issues...
----------------

Removed


No it's not. Google has no problem spidering dynamic pages.

edit: Why do you keep posting warez links in every post, Is it to get around ozzus signature rules? You do know the forum has "nofollow" attributes on links, don't you?
  • Paramore
  • Born
  • Born
  • Paramore
  • Posts: 2

Post 3+ Months Ago

Can Google still crawl your forums if it's set that only registered users can see and post? Does Google get past that, or do I need to change it so that it's visible to everyone?
  • meman
  • Web Master
  • Web Master
  • User avatar
  • Posts: 3432
  • Loc: London Town , Apples and pears and all that crap

Post 3+ Months Ago

How would google get passed it? And even if they could get passed it why would they want to? Google wouldn't want to send searchers to login pages.

If an unregistered users can't see it then neither can google.
If you want all of your pages to be spidered and indexed you will have to let google spider and index them.
  • subet
  • Newbie
  • Newbie
  • subet
  • Posts: 7

Post 3+ Months Ago

Willy wrote:
I am curious as to google spidering forums such as ozzu.

I run a PHBB forum, my mainsite has a PR4 but I never get any google hits to any of the forum pages. There are enough random comments in there that i'd expect one. I was wondering if I need to configure anything special in phpbb or any special meta tags to get the spiders to index everything?


Yes, it defiatelly spider forums. Even better then other websites. I have a forum site http://www.tradenec.com. It is an old forum. Even the information on the forum is very old. It is a useless website. It has a pagerank 4 and 809 pages are spidered by google. Which means all pages are indexed. Near this, i have a game website. http://www.zopcuk.eu It is very nice designed and has a daily updated content. It is pagerank 0 and has 165 pages indexed. Which means about half of the website indexed.
  • Anonymous
  • Bot
  • No Avatar
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post 3+ Months Ago

Post Information

  • Total Posts in this topic: 34 posts
  • Users browsing this forum: No registered users and 3 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
 

© 1998-2014. Ozzu® is a registered trademark of Unmelted, LLC.