web server - nginx as load balancer to nginx webservers

I am trying to set up a software based load balancer with nginx. Before I install heartbeat and pacemaker, I have created a CentOS virtual machine and installed nginx on it (lb-01), which will serve as my load balancer. I have also created another CentOS virtual machine (web-01) which will serve as my webserver. The above is the simplest way to have something up and running prior to adding more resources to it on the LB level or the web level.

On the load balancer I have nginx setup as:

user                            nginx nginx;
worker_processes                4;
worker_rlimit_nofile            16384;

pid                             /var/run/nginx.pid;

events {
    worker_connections          4096;
}

http {
    include                     mime.types;
    access_log                  /var/log/nginx/access.log main;
    error_log                   /var/log/nginx/error.log  error;


    sendfile                    on;
    ignore_invalid_headers      on;
    reset_timedout_connection   on;
    tcp_nopush                  on;
    tcp_nodelay                 on;
    keepalive_timeout           60;
    keepalive_requests          500;
    send_timeout                30;


    client_body_buffer_size     256k;
    large_client_header_buffers 16 8k;
    client_body_timeout         30;
    client_max_body_size        10m;
    client_header_timeout       30;

    gzip                        on;
    gzip_disable                "MSIE [1-6]\.(?!.*SV1)";

    upstream webservers {

        server 192.168.173.129;
    }

    server {
        listen                  80 default_server;
        location / {
            proxy_pass          http://webservers;
            proxy_set_header    X-Real-IP $remote_addr;
            proxy_next_upstream timeout;
        }

    }
}

The webserver (web-01) is listening on port 80 for requests. On that server I have specified a default_server to just show the hostname, while other directives process various sites configured on the server.

As a test, I have pointed the A record of one of my domains (abc.example.com) to the load balancer IP address. The idea is that the request will go to the load balancer, it will be passed to web-01 which will point it to the correct domain and then it will be served and the data will be returned back to the client.

So when I try to load abc.example.com I see on the logs of the load balancer:

173.86.99.33 - - [20/Mar/2011:22:08:17 -0400] GET / HTTP/1.1 "304" 0 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.151 Safari/534.16" "-" "-"
173.86.99.33 - - [20/Mar/2011:22:08:18 -0400] GET /favicon.ico HTTP/1.1 "404" 201 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.151 Safari/534.16" "-" "-"

and looking at the logs of the web server (web-01) I see errors like the ones below:

2011/03/20 22:17:04 [error] 3657#0: *3917 open() "/var/www/_local/favicon.ico" failed (2: No such file or directory), client: 192.168.173.125, server: chromium.niden.net, request: "GET /favicon.ico HTTP/1.0", host: "webservers"
2011/03/20 22:17:04 [error] 3657#0: *3917 open() "/var/www/_local/404.html" failed (2: No such file or directory), client: 192.168.173.125, server: chromium.niden.net, request: "GET /favicon.ico HTTP/1.0", host: "webservers"

The browser shows the name of the host (which is the default site on the server as mentioned earlier).

The site itself is not passed from the load balancer to the web server (web-01) so the content cannot be returned properly. Therefore instead of the web server returning the content of abc.example.com it produces not found errors and returns the default site.

I tried Google as well as nginx's site but did not have any luck.

Any pointers would be more than appreciated.

Thank you!

Answer

If your backend is using a virtual host and requires the Host header to contain the actual hostname of the site, you will need to add this to your load balancer location:

proxy_set_header Host $host;

This will forward whatever Host: header the client sent to the load balancer on to the back-end. This exact scenario is documented on the nginx wiki.