mod_asis in combination with mod_deflate

  • joebert
  • Fart Bubbles
  • Genius
  • User avatar
  • Posts: 13504
  • Loc: Florida

Post 3+ Months Ago

gzip compressed files, good.
CPU usage added by compressing files with mod_deflate, bad.

What if I compress copies of static files with gzip before hand and send those with some fancy mod_rewrite rules that check the Acept-Encoding HTTP header for gzip though ?

One way I can think to do this is to generate the files with something like wget in combination with a --header= switch to force the server to send me something I can use with mod_asis to send those files with the headers already attached. I'm not sure if that will complicate things like mod_expires though.

Another thought is just to save *.gz files and hope Apache and mod_rewrite will send the correct headers along with that compressed file. I have a feeling that's not going to work out though.

I could rewrite all requests to a PHP/CGI script that generates the headers and does a simple passthru() with the files content, but I have a feeling that's going to defeat the purpose of avoiding the much simpler mod_deflate.

Anyone ever do anything like this before, any ideas ?
  • joebert
  • Fart Bubbles
  • Genius
  • User avatar
  • Posts: 13504
  • Loc: Florida

Post 3+ Months Ago

It looks like the easiest thing to do is going to be the one I didn't think was going to work.

It seems to be working. I have to reset the mime-type to the original type in order to get Firefox to play nice with it though. I haven't had a chance to look at it in anything other than FF and Opera on Ubuntu yet. I've tested it in Opera, FF, IE8, and Chrome on Vista with the same results as Opera/FF on Ubuntu.

APACHE Code: [ Select ]
<IfModule headers_module>
   <IfModule rewrite_module>
 
      <FilesMatch "\.(js|css|png)\.gz$">
         Header set Content-Encoding gzip
      </FilesMatch>
 
      <FilesMatch "\.js\.gz$">
         AddType application/javascript gz
      </FilesMatch>
 
      <FilesMatch "\.css\.gz$">
         AddType text/css gz
      </FilesMatch>
 
      <FilesMatch "\.png\.gz$">
         AddType image/png gz
      </FilesMatch>
     
      RewriteEngine On
      RewriteCond %{HTTP:accept-encoding} gzip
      RewriteCond %{REQUEST_FILENAME}.gz -f
      RewriteRule \.(js|css|png)$ %{REQUEST_URI}.gz
   </IfModule>
</IfModule>
  1. <IfModule headers_module>
  2.    <IfModule rewrite_module>
  3.  
  4.       <FilesMatch "\.(js|css|png)\.gz$">
  5.          Header set Content-Encoding gzip
  6.       </FilesMatch>
  7.  
  8.       <FilesMatch "\.js\.gz$">
  9.          AddType application/javascript gz
  10.       </FilesMatch>
  11.  
  12.       <FilesMatch "\.css\.gz$">
  13.          AddType text/css gz
  14.       </FilesMatch>
  15.  
  16.       <FilesMatch "\.png\.gz$">
  17.          AddType image/png gz
  18.       </FilesMatch>
  19.      
  20.       RewriteEngine On
  21.       RewriteCond %{HTTP:accept-encoding} gzip
  22.       RewriteCond %{REQUEST_FILENAME}.gz -f
  23.       RewriteRule \.(js|css|png)$ %{REQUEST_URI}.gz
  24.    </IfModule>
  25. </IfModule>
  • joebert
  • Fart Bubbles
  • Genius
  • User avatar
  • Posts: 13504
  • Loc: Florida

Post 3+ Months Ago

This does not play nice if you already have mod_deflate compressing the targeted content globally on the server. As soon as that AddType directive comes in to play it will trigger the output filter and corrupt the output.

Symptoms include binary data being dumped to the browser screen and identical HTTP headers as you would get on a server without mod_deflate enabled.

So far disabling mod_deflate and restarting Apache is the only thing I've had fix the issue. The RemoveOutputFilter didn't appear to work.

// Edit - FilterChain appears to be the solution, it requires mod_filter to be loaded though.
  • joebert
  • Fart Bubbles
  • Genius
  • User avatar
  • Posts: 13504
  • Loc: Florida

Post 3+ Months Ago

I did some benchmarking with three setups. One that uses a static HTML cache, another that has mod_deflate enabled for that static cache, and a third that uses the pre-generated gz files.

Here's the command line from ab. ab was executed from a workstation on the same internal network as the test server.

BASH Code: [ Select ]
ab -n 1000 -c 10 -H "Accept-Encoding: gzip,deflate" http://domain.com/file.html


Here are the results.
I ran each multiple times and selected the one that best reflected the average for each setup.

Static HTML (no deflate)


Code: [ Select ]
Document Length:    7406 bytes

Concurrency Level:   10
Time taken for tests:  3.004 seconds
Complete requests:   1000
Failed requests:    0
Write errors:      0
Total transferred:   7724717 bytes
HTML transferred:    7413406 bytes
Requests per second:  332.92 [#/sec] (mean)
Time per request:    30.037 [ms] (mean)
Time per request:    3.004 [ms] (mean, across all concurrent requests)
Transfer rate:     2511.43 [Kbytes/sec] received

Connection Times (ms)
       min mean[+/-sd] median  max
Connect:    0  1  3.4   0   19
Processing:   6  28 16.3   21   110
Waiting:    3  28 16.4   20   110
Total:     9  30 17.1   22   120

Percentage of the requests served within a certain time (ms)
 50%   22
 66%   30
 75%   32
 80%   40
 90%   50
 95%   59
 98%   98
 99%  110
100%  120 (longest request)
  1. Document Length:    7406 bytes
  2. Concurrency Level:   10
  3. Time taken for tests:  3.004 seconds
  4. Complete requests:   1000
  5. Failed requests:    0
  6. Write errors:      0
  7. Total transferred:   7724717 bytes
  8. HTML transferred:    7413406 bytes
  9. Requests per second:  332.92 [#/sec] (mean)
  10. Time per request:    30.037 [ms] (mean)
  11. Time per request:    3.004 [ms] (mean, across all concurrent requests)
  12. Transfer rate:     2511.43 [Kbytes/sec] received
  13. Connection Times (ms)
  14.        min mean[+/-sd] median  max
  15. Connect:    0  1  3.4   0   19
  16. Processing:   6  28 16.3   21   110
  17. Waiting:    3  28 16.4   20   110
  18. Total:     9  30 17.1   22   120
  19. Percentage of the requests served within a certain time (ms)
  20.  50%   22
  21.  66%   30
  22.  75%   32
  23.  80%   40
  24.  90%   50
  25.  95%   59
  26.  98%   98
  27.  99%  110
  28. 100%  120 (longest request)


Static HTML (deflate)


Code: [ Select ]
Document Length:    2085 bytes

Concurrency Level:   10
Time taken for tests:  2.924 seconds
Complete requests:   1000
Failed requests:    0
Write errors:      0
Total transferred:   2450448 bytes
HTML transferred:    2087085 bytes
Requests per second:  342.04 [#/sec] (mean)
Time per request:    29.237 [ms] (mean)
Time per request:    2.924 [ms] (mean, across all concurrent requests)
Transfer rate:     818.50 [Kbytes/sec] received

Connection Times (ms)
       min mean[+/-sd] median  max
Connect:    0  1  3.2   0   16
Processing:   5  28 10.3   21   60
Waiting:    5  27 10.4   21   60
Total:     12  29 11.0   27   60

Percentage of the requests served within a certain time (ms)
 50%   27
 66%   30
 75%   32
 80%   40
 90%   50
 95%   50
 98%   53
 99%   60
100%   60 (longest request)
  1. Document Length:    2085 bytes
  2. Concurrency Level:   10
  3. Time taken for tests:  2.924 seconds
  4. Complete requests:   1000
  5. Failed requests:    0
  6. Write errors:      0
  7. Total transferred:   2450448 bytes
  8. HTML transferred:    2087085 bytes
  9. Requests per second:  342.04 [#/sec] (mean)
  10. Time per request:    29.237 [ms] (mean)
  11. Time per request:    2.924 [ms] (mean, across all concurrent requests)
  12. Transfer rate:     818.50 [Kbytes/sec] received
  13. Connection Times (ms)
  14.        min mean[+/-sd] median  max
  15. Connect:    0  1  3.2   0   16
  16. Processing:   5  28 10.3   21   60
  17. Waiting:    5  27 10.4   21   60
  18. Total:     12  29 11.0   27   60
  19. Percentage of the requests served within a certain time (ms)
  20.  50%   27
  21.  66%   30
  22.  75%   32
  23.  80%   40
  24.  90%   50
  25.  95%   50
  26.  98%   53
  27.  99%   60
  28. 100%   60 (longest request)


pre-generated gz files (mod_deflate disabled)


Code: [ Select ]
Document Length:    2085 bytes

Concurrency Level:   10
Time taken for tests:  1.633 seconds
Complete requests:   1000
Failed requests:    0
Write errors:      0
Total transferred:   2423838 bytes
HTML transferred:    2089170 bytes
Requests per second:  612.30 [#/sec] (mean)
Time per request:    16.332 [ms] (mean)
Time per request:    1.633 [ms] (mean, across all concurrent requests)
Transfer rate:     1449.34 [Kbytes/sec] received

Connection Times (ms)
       min mean[+/-sd] median  max
Connect:    0  0  1.3   0   18
Processing:   5  16  6.3   15   51
Waiting:    5  15  6.3   15   51
Total:     6  16  6.2   16   52

Percentage of the requests served within a certain time (ms)
 50%   16
 66%   20
 75%   20
 80%   20
 90%   24
 95%   30
 98%   30
 99%   31
100%   52 (longest request)
  1. Document Length:    2085 bytes
  2. Concurrency Level:   10
  3. Time taken for tests:  1.633 seconds
  4. Complete requests:   1000
  5. Failed requests:    0
  6. Write errors:      0
  7. Total transferred:   2423838 bytes
  8. HTML transferred:    2089170 bytes
  9. Requests per second:  612.30 [#/sec] (mean)
  10. Time per request:    16.332 [ms] (mean)
  11. Time per request:    1.633 [ms] (mean, across all concurrent requests)
  12. Transfer rate:     1449.34 [Kbytes/sec] received
  13. Connection Times (ms)
  14.        min mean[+/-sd] median  max
  15. Connect:    0  0  1.3   0   18
  16. Processing:   5  16  6.3   15   51
  17. Waiting:    5  15  6.3   15   51
  18. Total:     6  16  6.2   16   52
  19. Percentage of the requests served within a certain time (ms)
  20.  50%   16
  21.  66%   20
  22.  75%   20
  23.  80%   20
  24.  90%   24
  25.  95%   30
  26.  98%   30
  27.  99%   31
  28. 100%   52 (longest request)


It looks like mod_deflate reduces bandwidth usage, but it doesn't really do anything for the number of requests that can be served per second. Whereas using pre-generated gz files appears to double the number of requests that can be served per-second.

Post Information

  • Total Posts in this topic: 4 posts
  • Users browsing this forum: No registered users and 11 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
cron
 

© 1998-2014. Ozzu® is a registered trademark of Unmelted, LLC.