Nginx only handing domains properly

On my server, all requests to mydomain.com and www.mydomain.com go to my server's IP address correctly and the content is displayed accurately. However, if you go directly to the server's IP, the "Welcome to nginx!" default page is shown. I want requests directly to the server's IP to show the proper content. My server configuration is located below. I should also mention that merely adding the public IP or "0.0.0.0" after "mydomain.com" on the "server_name" line does not change anything. Thank you.

server {
    listen 443 ssl spdy;
    root /var/www;


    server_name mydomain.com www.mydomain.com;
    ssl_certificate /etc/nginx/ssl/ssl.crt;
    ssl_certificate_key /etc/nginx/ssl/ssl.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:ECDH-ECDSA-RC4-SHA:ECDH-RSA-RC4-SHA:ECDHE-RSA-AES256-SHA:RC4-SHA:HIGH:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!CBC:!EDH:!kEDH:!PSK:!SRP:!kECDH;
    ssl_prefer_server_ciphers on;
    keepalive_timeout 70;
    include /var/www/wp-content/plugins/better-wp-security/rules.conf;
    location / {

            try_files $uri $uri/ /index.php?q=$uri&$args;
            index index.php index.html index.htm;
    }

    error_page 404 /404.html;
    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
          root /usr/share/nginx/www;
    }


    location ~ \.php$ {
            fastcgi_pass unix:/var/run/php5-fpm.sock;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
            try_files $uri =404;
    }

}

server {
listen 80;
server_name mydomain.com www.mydomain.com;
rewrite ^ https://$host$request_uri permanent;
}

Answer

Your configuration for domain.com and www.domain.com is fine. What happens is that there is another "default" server defined in nginx (probably in main nginx.conf configuration file) which responds to requests by IP. You need to remove that catch-all server configuration.