Linux as server #7 – Apache benchmark, performance boost with Varnish (proxy) and yslow analysis

March 17, 2014

niagara-218591_1280

This post is part of Tero Karvinen’s course: Linux as server. Even it is related to school mission it will offer usefull information about GNU/Linux!

Before final test of the course Mr. Karvinen introduced solutions to improve server performance with proxy. My last mission was:

  • Do apache`s pressure test with static and dynamic WordPress pages
  • Install Varnish, do pressure test again and examine the results
  • Make .iso images pass varnish
  • Analyse pages with yslow (firefox addon)

All tests made with Xubuntu 12.04 LTS Precise Pangolin 32bit.

Hardware:

  • Motherboard: Asus Z87-C
  • CPU: Intel Core i5-4670K 3.40GHz
  • RAM: 8GB DDR3 1600MHz
  • HDD: 120GB SSD Sata 3.0
  • GPU: Geforce GTX 560 Ti Phantom, 2GB GDDR5 (Gainward)
  • Asus cd/dvd

Preparing for benchmark test

First I needed two sites static and dynamic ones. I already had wordpress installed so I decided to use that as dynamic site. I went to index-page of WordPress with Firefox, right-clicked on site and then pressed “View Page Source”. I had source of singe wordpress-page which I could use as static page. I copied the source to clipboard and went to terminal.

$ cd
$ cd public_html/
$ mkdir static
$ cd static/
$ nano index.html
CTRL+v->CTRL+x->y->ENTER

Then I tested if the pages look similar

$ firefox http://localhost/~tuukka/wordpress/
$ firefox http://localhost/~tuukka/static

Both pages looked exactly the same. What makes them different to each other is that page on http://localhost/~tuukka/wordpress/ is build from mysql database with php, html and css. Static page just contains pure html and css.

Benchmark before varnish

I used apache`s own benchmark tool ab in these forms:

$ ab -c 150 -n 1000 http://localhost/~tuukka/wordpress
$ ab -c 150 -n 1000 http://localhost/~tuukka/static

Here are results in same image:

combination-novarnish

At this point I realised there might be something I am doing wrong since results are so close to each other. I made fresh install of wordress and made new post. I also made static version of that post page. Here is the new page:

benchmark

I made same tests again with new pages.

$ ab -c 150 -n 1000 http://localhost/~tuukka/wordpress/?p=4
$ ab -c 150 -n 1000 http://localhost/~tuukka/static/index.html

combined1

I got totally different kind of results with new setup. 1000 page loads 150 at the same time took 24.157seconds with dynamic page and 0.049 with static page.

Varnish installation

I installed package.

$ sudo apt-get update
$ sudo apt-get install varnish

Then I made apache to listen on port 8080 instead default 80. I edited ports.conf lines NameVirtualHost and Listen.
$ sudoedit /etc/apache2/ports.conf

# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default
# This is also true if you have upgraded from before 2.2.9-3 (i.e. from
# Debian etch). See /usr/share/doc/apache2.2-common/NEWS.Debian.gz and
# README.Debian.gz

NameVirtualHost *:8080
Listen 8080

<IfModule mod_ssl.c>
    # If you add NameVirtualHost *:443 here, you will also have to change
    # the VirtualHost statement in /etc/apache2/sites-available/default-ssl
    # to <VirtualHost *:443>
    # Server Name Indication for SSL named virtual hosts is currently not
    # supported by MSIE on Windows XP.
    Listen 443
</IfModule>

I also made changes to /etc/apache2/sites-enabled/000-default as the third line in ports.conf said.

$ sudoedit /etc/apache2/sites-enabled/000-default
Changed: VirtualHost *:80 -> VirtualHost *:8080

ServerName tuukka-xubuntu
<VirtualHost *:8080>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog ${APACHE_LOG_DIR}/access.log combined

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

</VirtualHost>

Tested if it was working by restarting apache and trying to load page.

$ sudo service apache2 restart
$ firefox localhost
Led me to “Unable to connect page
$ firefox localhost:8080
Worked perfectly!

Then the varnish setup. I made changes to /etc/default/varnish.

$ sudoedit /etc/default/varnish
There was different setups ready to use. By default it was using “Alternative 2” configuration which allows VCL. I decided to use it aswell and changed the port of line starting with: DAEMON_OPTS

...
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.  Use a 1$
# fixed-size cache file.
#
DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"
...

Then I restarted varnish and tried if it was working.

$ sudo service varnish restart
$ firefox localhost
This led me to apaches default “It works” page and I noticed that varnish is working since apache is not listening default port.

Benchmark with varnish

Varnish is now configured and working. I made same tests again.

$ ab -c 150 -n 1000 http://localhost/~tuukka/wordpress/?p=4
$ ab -c 150 -n 1000 http://localhost/~tuukka/static/index.html

With dynamic page the improvement was clear. I noticed that with static page it was not so clear and it was actually slower to load than dynamic page. I do not know the reason. I tried test couple of times and all results were almost same. Here are example results:

combined2

Summary of the benchmark

Dynamic page
Without varnish:Time taken for tests: 24.157 seconds
Longest request: 9931 ms

With varnish:
Time taken for tests: 2.525 seconds
Longest request: 2489 ms

Static page
Without varnish:
Time taken for tests: 0.049 seconds
Longest request: 13 ms

With varnish:Time taken for tests: 3.944 seconds
Longest request: 3927 ms

With these results I would say Varnish is must if there is huge amount of traffic with dynamic pages.

How to make .iso images pass Varnish

First I tried what happens if I am trying to download .iso image through Varnish. I downloaded xubuntu-13.10 .iso and putted it under the public_html. I tried to download it and I got this funny guru meditation page. I am not quite sure what is exact reason for it. I guess it might be something related to memory of Varnish.

$ firefox localhost/~tuukka/xubuntu-13.10-desktop-i386.iso

gurumeditation

I googled a bit and found this Jukka Pentti`s blog. He was tried to make this work by editing /etc/varnish/default.vcl put did not tested it.

$ sudoedit /etc/varnish/default.vcl
I added this:

if (req.url ~“\.iso$”) {
 set req.backend = web;
 pass;
 }

 

$ sudo service varnish restart
At this point I got this message:

 * Stopping HTTP accelerator varnishd                                  [ OK ] 
 * Starting HTTP accelerator varnishd                                  [fail] 
Message from VCC-compiler:
Syntax error at
('input' Line 12 Pos 14)
if (req.url ~“\.iso$”) {
-------------#--------------

Running VCC-compiler failed, exit 1

VCL compilation failed

 

Obviously there is something wrong with the script I just added to default.vcl. I tried some debugging with this but it did not solved. I decided to google again and I found mikkott`s blog where he solved problem by passing all over 200mb files from varnish.

$ sudoedit /etc/varnish/default.vcl
Added this under backend default script:

sub vcl_recv {
 if (req.http.x-pipe && req.restarts > 0) {
  remove req.http.x-pipe;
  return (pipe);
 }
}

sub vcl_fetch {
 if (beresp.http.Content-Length ~ "[0-9]{8,}" ) {
  set req.http.x-pipe = "1";
  return (restart);
 }
}

 

Then I restarted Varnish and tried to download .iso again.

$ sudo service varnish restart

$ firefox localhost/~tuukka/xubuntu-13.10-desktop-i386.iso
After that guru did not meditated again and I was able to download .iso!

download

Page analysis with yslow

Yslow is plugin for firefox. I installed it from firefoxes add-ons manager. It was not working out of the box. Turned out that it might need firebug plugin to work. I installed firebug and reboot firefox. Made inspection with firebug(right clicked somewhere in any page) and after that yslow was also working.

I decided to analyse two different well known finnish sites mtv3.fi and yle.fi. Here are the results in one image:

combined_yslow

Overal scores:

yle.fi
Grade: DOverall performance score 69

mtv3.fi
Grade: D
Overall performance score 60

http://localhost/~tuukka/wordpress/ (this is the page used in varnish test)
Grade: A
Overall performance score 94

Based on these results seems like mtv3.fi is little bit faster to load. Yslow gave lots of information of these two sites. If I would like to make yle.fi load faster I would start from these yslow`s high priority suggestions:

This page has 13 external Javascript scripts. Try combining them into one.
This page has 4 external stylesheets. Try combining them into one.
This page has 25 external background images. Try combining them with CSS sprites.

 

Sources

http://terokarvinen.com/2013/aikataulu-–-linux-palvelimena-ict4tn003-11-ja-12-kevaalla-2014
Varnishin ja Nginx asennus Ubuntu serveriin
http://jukkapentti.wordpress.com/tag/varnish/

3 Comment

  1. […] siirryin asentamaan Varnishia. Käytin tässä Tuukka Meriläisen tekemää ohjetta: http://tuukkamerilainen.com/linux-as-server-7-apache-benchmark-and-varnishproxy/ […]

Leave a Reply