PHP/MySQL and Spiders

  • RBeau
  • Student
  • Student
  • RBeau
  • Posts: 69
  • Loc: Aurora, CO

Post 3+ Months Ago

Okay lets say I have a database, and my "meat" of my web content is in the database. When a spider visits my site to index it on lets say Google or another search engine. does the spider have access to the database content, and if not, is there a script that would allow crawling of the contents of my database? or is there a formal name for a script that would allow this to happen that I could google and find out how it would work?

The reason for this is if the content of the database is what would bring considerable search engine traffic to my site, and if the spider does not have access to the content... I would be up the creek with no paddle right?

Robert
  • dEfEndEr
  • Newbie
  • Newbie
  • dEfEndEr
  • Posts: 14

Post 3+ Months Ago

Do you have web pages that display the data ?

If so, (like a phpBB) the page that pulls the data is added to the search engine.

phpBB holds all the post's in a database, these are displayed to the users 9you and me) on request) a BOT will also make the same request when it spiders the site.

The hardest part is getting them to keep comming back and indexing the page, I use a create date meta tage in the html parts of my site, and with a little php the date can be changed for every visit by the spider, so it indexs the whole page again :)
  • RBeau
  • Student
  • Student
  • RBeau
  • Posts: 69
  • Loc: Aurora, CO

Post 3+ Months Ago

I will have some pages that parts of the data are on it. Most of the data will be displayed by initiating a search to display records, but that gives me an idea, I could probably create a page that displays an index of all the records in the database, that way the spider will be able to use that information to index my site. That is a good idea about the date changing script, I will have to look into that further. I am in the "OOP" stage of building this site, so I am trying to get concepts down and making a collage of code to include in the website. Thank you very much for your reply, very useful information!

Post Information

  • Total Posts in this topic: 3 posts
  • Users browsing this forum: No registered users and 95 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
 

© 1998-2014. Ozzu® is a registered trademark of Unmelted, LLC.