Sunday, December 10, 2017

postfix - Can sendmail forward e-mail immediately instead of queueing?



Some of the e-mail passing through my server is forwarded to external accounts.




Unfortunately, my upstream SMTP-server is very picky about spam -- and rejects some of the legitimate messages as such. When this happens to the forwarded mail, I get the bounces (as the postmaster) -- not the originators.



I understand, that this is because sendmail queues the messages locally, disconnects from the relay, and only then proceeds to forward them further. If the further forwarding breaks for any reason -- such as because the next relay misidentifies the message as spam -- my sendmail is left to hold the pieces.



Can things be configured so that the forwarding begins immediately instead (as soon as the forwarding destination is determined)? The status -- success or failure -- can then be communicated directly to the previous relay still on the line...



If sendmail can not do it, can any other MTAs? Thanks!


Answer



No, it's not possible as it's not implemented with any wide spread SMTP software; you would have to program your own SMTP server that supports this kind of behavior, which would be out of scope on Serverfault. In this answer I explain, why all MTAs have implemented the SMTP protocol very similarly, using queue, and how that is the best way to accomplish all requirements of the protocol.




A mail transport agent MTA always either denies a message or accepts and queues it, based on its own settings. Then, it's relayed or delivered from the queue.



That's because




  • there can be both permanent and temporary errors. If the MTA can't connect the nexthop immediate, it'll try again later and bounces only if the delay reaches the limit set. Neither can it wait for another MTA to respond before closing the connection, as it may have other messages to deliver first.


  • there can be several recipients. While a client can simply list all recipients at once with RCPT TO commands, the message can be finally delivered to several other servers, of which some can be available now and some later. Furthermore, the MTA can't open all these connections at once during the initial connection and wait for their responses. There's no practical reason to have totally different workflow for messages with a single recipient.


  • it should always be clear which MTA currently has the responsibility for delivering the message. (This has been explained by examples in MadHatter's answer.)





That's just how SMTP was designed. Rather than syntactical requirement for the connection commands this leads to very similar architectures; Sendmail, Postfix and even MS Exchange has separate components for sending and receiving mail.




  1. The SMTP server component receives mail and adds it to the queue.

  2. Then, separate SMTP client tries to send it further to other MTAs, or if a recipient is local, the message can be saved to a file or passed to a mail delivery agent MDA, e.g. Procmail.



The requirement still comes from the SMTP specification; RFC 5321 2.1 on SMTP model basic structure:





Fully-capable SMTP implementations, including the relays used by
these less capable ones, and their destinations, are expected to
support all of the queuing, retrying, and alternate address
functions discussed in this specification. In many situations and
configurations, the less-capable clients discussed above SHOULD be
using the message submission protocol (RFC 4409) rather than
SMTP.





And a bit further:




In other words, message transfer can occur in a single connection between the original SMTP-sender and the final SMTP-recipient, or can
occur in a series of hops through intermediary systems. In either case, once the server has issued a success response at the end of the
mail data, a formal handoff of responsibility for the message occurs:
the protocol requires that a server MUST accept responsibility for
either delivering the message or properly reporting the failure to do
so (see Sections 6.1, 6.2, and 7.8).




Saturday, December 9, 2017

Why are domain names occasionally inaccessible?



I could not track down the reason why my domain names occasionally are inaccessible to some visitors. With some I mean some, not everyone. I was working on my website, and at some moment I could not access it anymore. I thought that it's a hosting-related problem. I was getting "website not found" error in my browser. In the same time I could still access the website from a different computer (with different ip address). Problem was gone the next day when I woke up, so unfortunately, I let it go and did not try to find out the reason behind it.




Right now myself and some of my course mates were not able to access our website while some others were able to. Website is now inaccessible to people which it was accessible to earlier.



I was looking through error logs and anything that could be of use in Cpanel. But I was kind of lost.



I don't have extensive knowledge about servers. Which means my reasoning can be faulted. I thought that some visitors could theoretically get ip-banned if they download/upload a lot in a short amount of time. I thought that some defensive mechanism to prevent "spamming" could have worked. However, when I looked in Cpanel I saw that no ip addresses are banned.



Using google made me come across the idea that the problem could be caused by ISP and their DNS service. I found out it is possible to use specific DNS servers, instead of "obtaining DNS server addresses automatically". I changed settings of my connection (specifically, Internet protocol version 4 ) and now I can access my website again. I 'm using opendns.com nameservers now.



This means I have found a fix. But I still don't know why the problem happens in the first place. I shouldn't be asking visitors of my website to change settings of their internet connection.




So, the question is:
Why does this happen, and how can I prevent it? If I can?



Edit1:
Records in my registrar look like this:



Type    IP      Hostname        Alias   MX Preference
NS ns3.website.com
NS ns4.website.com


Answer



This sounds like your primary DNS servers are experiencing temporary outages.



When a user requests a DNS record, it gets handed up the chain of DNS servers. First the computer will check if it already has that record cached. If not, it will ask it's DNS server (usually the router). If it has it cached, it returns it, otherwise it asks it's DNS Server - usually the ISP's Server (or OpenDNS in your case). The DNS servers keep passing the request on until (assuming none of them has it cached) it hits your primary DNS Server.



Your primary DNS Server returns either an A-Record (IP Address) or CNAME-Record (other domain name) that points to your server. In your case it's an A Record pointing to 75.126.154.2. The DNS Servers then pass the information back the chain to the user who requested it - caching it for future request.



The cache timeout depends on the server and the TTL you set for your DNS records and is customarily anywhere between 15 Minutes and 12 hours. Your timeout is set to 4 hours.




Now, if your DNS servers go down or are overloaded (this is more common than it should be in private setups or with cheap hosters) it is well possible that a good portion of the Internet will have your DNS Records cached.



Example:



You requested the record before the DNS Server went down. Your router has it cached. Your friend on the same LAN request the page, the record is returned by the router, he can access the page. Another user with the same ISP also request the page, the ISP still has the record cached, he can visit the page.



Then a completely unrelated user request the record. The requests gets passed up from DNS server to DNS server until it finally hits your primary DNS server. Since it is down, the request will time out. The user will not be able to access your page.



Solutions:




1) Move to a more reliable DNS service



2) Host your backup DNS server with a completely different service provider. There are many DNS service providers charging very moderate fees. This way, as long as one of the two stays up, users will be able to access your site.


Thursday, December 7, 2017

debian - How to debug Apache2 virtual hosts

I'm hosting two websites in one server. Each has their own VirtualHost defined. First site works nicely but when I try to access the second I get redirected to the first one. It's like Apache would not recognize the correct VirtualHost by the ServerName and redirects me to the default (first) site.



Is there a log or something where I can see what steps Apache is taking when it handles my request? Does it log anywhere what ServerName it is using when it tries to find a matching VirtualHost?




Update:



I added a VirtualHost section for a subdomain of my second site and that works. It renders page from the correct application server that runs in 8501 port when I enter "interface.domain.com" into my browser. But when I try "www.domain.com" it redirects to my first site.



Below is my virtual host file for the second site.




ServerName www.domain.com
DocumentRoot /home/domain/current/public


RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ http://localhost:8501/$1 [P]



ServerName interface.domain.com
DocumentRoot /home/domain/current/public


RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ http://localhost:8501/$1 [P]



apache2ctl -S



AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 1.2.3.4. Set the 'ServerName' directive globally to suppress this message
VirtualHost configuration:

1.2.3.4:443 www.domainfirst.com (/etc/apache2/sites-enabled/https_www_domainfirst_com.conf:1)
1.2.3.4:80 is a NameVirtualHost
default server domainfirst.com (/etc/apache2/sites-enabled/domainfirst_com.conf:1)
port 80 namevhost domainfirst.com (/etc/apache2/sites-enabled/domainfirst_com.conf:1)
port 80 namevhost www.domainfirst.com (/etc/apache2/sites-enabled/www_domainfirst_com.conf:1)
*:80 is a NameVirtualHost
default server domain.com (/etc/apache2/sites-enabled/domain_com.conf:1)
port 80 namevhost domain.com (/etc/apache2/sites-enabled/domain_com.conf:1)
port 80 namevhost www.domain.com (/etc/apache2/sites-enabled/www_domain_com.conf:1)
port 80 namevhost interface.domain.com (/etc/apache2/sites-enabled/www_domain_com.conf:10)

ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/lock/apache2" mechanism=fcntl
PidFile: "/var/run/apache2/apache2.pid"

Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33
Group: name="www-data" id=33


Solution:



Clear the browser cache. My browser had cached a rule to forward request to the other site.

networking - (D)DOS - tcpdump established connections analysis (too many established connections)



¿How can I debug this problem?




(I've got full tcpdump captures)



I have a TCP server into which many clients establish persistent connections. Normally all these clients behave, and I never reach the 1024 default Linux limit connections (open files) per process.



Yesterday someone (or something) started misbehaving and leaving a lot of open connections, forcing me to restart the server. You can see its behavior on the following munin netstat graph:



munin netstat connections graph



Every time connections reach 1000, I restart the server. Only the fourth time the misbehavior stopped as mysteriously as it started, without any apparent reason. Something similar happened one week ago.




All the bad connections come from the same (sub)network: I can isolate them, but there are some valid connections that come from the same network too (so I can't deny connections from that network).



So far I've used tcpdump, ethereal and ngrep, but I haven't found a way to look at connections that are established, but that don't transfer data.




  • How should I look the tcpdump (pcap) captures to isolate the misbehaving connections and study them?

  • What would you suggest to stop this happening?




Thanks!


Answer



In Wireshark, go to Statistics->Conversations->TCP. Try eyeballing the list to see if anything looks odd, e.g. a host with an abnormally large number of connections, low bytes transferred, or a low transfer rate. If you really need to you can copy the data to a spreadsheet. (You can do something similar on the server side using netstat, e.g. on Linux you could run netstat -nt | sort -n -t . -k5,5 -k6,6 -k7,7 -k8,8 to list connections sorted by client IP address).



If the problem is limited to one or two clients, you can look at their traffic to try to narrow the problem down further.



(And if you really are using Ethereal, you should upgrade to Wireshark immediately. Disclosure: I'm the lead developer.)


Monday, December 4, 2017

port - Unable to connect to remote host: Connection refused

I have created vm instance in gcloud. And setup firewall rule to allow connection from port 5000.I added network tag to my vm instance.



But if i test the connection by teneting to port 5000 I get




telnet: Unable to connect to remote host: Connection refused




Content of netstat -lnp




image

Sunday, December 3, 2017

encryption - Fixing security holes on my web server

Full disclosure: I'm largely ignorant on web security matters. I'm a self-taught webmaster, and I manage lifering.org, a WordPress site, as a volunteer for Lifering Secular Recovery. We would like to make the site HTTPS, but my attempts to bring it up always give me an error that I'm still using TLS 1.0. I know enough to know that won't do. I also know that when we renewed our security certificate, we bought a SHA-2 certificate with 256 bit encryption. Poking around, I found a site, https://www.ssllabs.com/ssltest/analyze.html, which analyzed lifering.org and gave it an F. You can see the detailed analysis by entering the domain on that site, but in summary, it said the certificate is golden, BUT the site has these failings:




  • It doesn't support forward secrecy with all browsers.

  • It accepts the RC4 cipher, but only with older protocol versions.

  • It doesn't support TLS 1.2, only older protocols (1.1 is highest supported).

  • It supports weak Diffie-Hellman key exchange parameters (logjam).


  • It supports 512 bit export suites and may be vulnerable to FREAK attacks.



I need to know how to fix these things. Is this an Apache configuration issue, or what? I'd normally ask my ISP for help, but we have a virtual server hosted by GoDaddy, which means we get zero technical support unless we pay them fees we can't (as a small nonprofit) afford. Fortunately I have enough Unix background to be able to deal with an SSH shell, but I don't know what to do, or where to do it.

lamp - mysql always using maximum connection

I have LAMP server having 4 core CPU and 32 GB RAM.We are running a large website on it. I have following issues now in my server.




  1. When I use Mysqlreport tool to monitor the mysql server i am always seeing the connection usage as below. And the users reporting connection issues in the website.



_





Connections _________________________________________________________



Max used 251 of 250 %Max: 100.40 Total 748.71k
3.5/s




But when I use "show process list" command it will output nothing. We are using MyISAM engine for all our DBs.



My Mysql Config File is pasted below:





######################

[mysqld]



max_connections = 250



set-variable=local-infile=0




datadir=/var/lib/mysql



socket=/var/lib/mysql/mysql.sock



user=mysql



skip-name-resolve



skip-bdb




wait_timeout = 60



thread_cache_size = 100



table_cache = 1024



key_buffer = 384M



log_slow_queries=/mysql-log/mysql-slow.log




query-cache-size=512M



query-cache-type=1



query_cache_limit=20M



[mysqld_safe]



log-error=/var/log/mysqld.log




pid-file=/var/run/mysqld/mysqld.pid



#


Who is using my Mysql connection pool? How can I find it?



And i have another issue.





  1. Sometimes the Load average goes beyond 4-8 range. See below:




13:40:02 up 2 days, 10:39, 0 users, load average: 5.03, 1.68, 0.93




At that time i can see mysql is the top consumer of the CPU. Is there any optimization needed in mysql Server?

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...