Server Overloaded with requests for files that don't exist

I'm running a dedicated server for my site. I was hosting about 55,000 small images, but the load on the server was too much. I've partnered up with ImageShack, who is going to host all of the images free. I've transloaded all of the 55,000 images over to them and I have their new locations and where they used to be locally.

I put all the redirects in a file for apache to read and redirect the requests, but the server is still overloaded. It hangs up within 5-10 seconds of starting httpd. All of the images are in a specific directory, and they are what is getting overloaded with hits.

Is there any solution you can think of to the problem? I'm not that knowledgible about Apache or Linux...or really even how servers work. Something to the likes of setting up a limit on Apache handling the requests or using another http daemon that can listen on the same port (80) and redirect the requests would probably work.

Please help ASAP!

I am going to try and make heads or tails of your situation, so bear with me.

Firstly, the problem is not serving 55k small images; it's obviously threads.
The first thing I would do is tune the apache config, it sounds as though it's been set up with defaults.

Secondly, do not try and parse a file for 55k images named individually; that is going to keep threads open even longer.

You could use either a rewrite rule or a redirect in the apache config.

A redirect will simply send any request off to the alternate location, as in this redirectmatch statement:


You may also use rewrite rules, though there's a certain bit of voodoo associated with mod_rewrite and will not necessarily free up your threads unless performed adequately.


I am presuming a local dir of images being 'images' of course.

Either way, YMMV.

Thanks...I already fixed the problem as follows:

Setup a rewrite rule to redirect all requests coming to the banner directory. They all rewrite to a PHP script that pulls the "to" url from a mysql database based on the "from" name of the image.

ImageShack has hundreds of servers and adds a 3 character random add-on to the name of every file uploaded, making something simple like just redirecting all requests straight to them impossible.

This system seems to be working quite well. The server's been up for a good 36 hours or so and has a minimal load. Additionally, I've only transfered about 2GB in that time VS the 20-50GB I would have transferred.

=)