Tuesday, May 8, 2018

debian - Sendmail not sending mail as generic alias

Following the advice in this answer and the steps listed here, I've added the following to my /etc/mail/sendmail.mc file and compiled it with m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf. The end of.mc file itself:



dnl # Default Mailer setup
dnl # Masquerading options
FEATURE(`always_add_domain')dnl
MASQUERADE_AS(`debian')dnl
FEATURE(`allmasquerade')dnl

FEATURE(`masquerade_envelope')dnl
FEATURE(`genericstable',`hash -o /etc/mail/genericstable.db')dnl
GENERICS_DOMAIN_FILE(`/etc/mail/generics-domains')dnl
MAILER_DEFINITIONS
MAILER(`local')dnl
MAILER(`smtp')dnl


I created the /etc/mail/generics-domains file with:




mylinodeboxdomain.com
avirtualdomain.com


And created the /etc/mail/genericstable file with:



virtualuser    noreply@avirtualdomain.com


And restarted the sendmail service. Despite that, emails sent by virtualuser still seem to be coming from virtualuser@localhost rather than the desired noreply@avirtualdomain.com:




Received: from www.avirtualdomain.com (virtualuser@localhost)
by mylinodeboxdomain.com (8.15.2/8.15.2/Submit) with SMTP id x6QIOIYY002415
for ; Fri, 26 Jul 2019 14:24:18 -0400


I'm guessing my config is messed up, but as this is all new to me, I'm unsure what to change/remove.

hardware - Upgrading RAID controller to 4kn - do I have to reinitialize the array to use 4kn?




I have an enterprise-grade workstation with an Adaptec 6405E RAID controller in it. According to Adaptec, this RAID controller does not, and will never be able to, natively support 4K disk sectors. If you stick a 4K disk in it, then if it has 512-byte sector emulation mode (512e), it will use 512e. If it doesn't have 512e, the disk just won't work at all.



Connected to my 6405E are four HGST SATA disks that all have 4K native sectors, but they do support 512e. The disks are in RAID10 and the array is working "reasonably well" (performance is fine to good, but not amazing).



Without wiping the data on the disks and re-initializing the array, assuming I plug in an Adaptec 71605E, which does support native 4K sectors, will the controller use 4K sectors when interfacing with my disks? Or is this decision of using 512e or 4Kn baked into the structure of the on-disk format such that I'd have to wipe the disks to do that?



This question is just about whether I have to backup my data and re-initialize the array, or whether the controller can (automatically, or with manual intervention) be asked to "switch over" to Advanced Format 4Kn addressing. I already know for certain that if I did wipe and reinitialize the array, I could definitely set it up from scratch to use 4Kn on all the drives, using this new RAID controller.







Note that I am already quite familiar with the arcconf command line utility, and have previously used it to upgrade this array from RAID0 to RAID10 (yes, I know, I should've never been using RAID0 to begin with, but I got lucky, okay?). If there is some feature of the arcconf utility to "switch over" from 512e to 4Kn on the Adaptec 7-series controllers, I'd love to know about it, so I can use that to avoid having to reformat and temporarily offload the data to a backup location.



In the worst case, I have off-site backups of critical data already, but the system has so much software loaded on it that it would be cheaper (in terms of time spent) for me to do a block-layer copy of the entire array onto another disk -- probably a cheap 4TB disk connected to the mobo via AHCI -- then copy it back over once the logical array is reinitialized. Compared to the prospect of reinstalling everything (a metric ton of proprietary Windows programs with activation and such), that'd actually be cheaper and faster.


Answer



Your disks are either 512e (512 sectors on SAS/SATA interface) or 4k native (4k sectors on SAS/SATA interface), and unfortunately there is no way to change that via software or jumpers etc. You select the transfer mode when you buy the disks. Buy 4k native disk if you have adapter that supports 4k native on interface.



Update: and, once again, the disks never "fallback from 4kn to 512e" etc. The disks are either 512e - it means they will always send data in 512 sized sectors over SAS/SATA interface, or 4kn, it means disks will always send data in 4k sized data over SAS/SATA interface, and it depends only on disk, not on RAID adapter capabilities. The difference between 512n and 512e is that on the physical media sectors are sized as 512 for 512n, and 4k for 512e (disk chip translates each 4k sector on plates into 8 x 512 sectors on interface), on the interface 512e disk will always transmit only sectors at 512 bytes, no matter what adapter it is connected. The part number differ for 512e and 4kn disks, for example:



ST6000NM0014 - 6TB SAS drive with 4k sectors on SAS interface (called 4kn drive);




ST6000NM0034 - 6TB SAS drive with 512 byte sectors on SAS interface (called 512e drive)



both of these have 4k sectors on disk media, so care must be taken about sector write alignment in 512e case.



And you can still buy 512n disks, for example:
ST4000NM0023 - 4TB SAS drive with 512 byte sectors on interface and 512 sectors on media, so no need to care about sector alignment for this drive.



The RAID adapters fall intro 3 categories:
a) the oldest ones that do not know about 4k sectors - they work with 512n and 512e disks, however issues may arise with write performance if writes are not aligned by 8xsector boundaries on 512e drives,

b) the not so old ones that know about 4k internal sectors and about 512e emulation, but only work with 512 sectors on interface - less problems with alignment as controller cares about that,
c) the very new ones that are able to work with 4k sectors on interface. Only these will work with new 4kn disks that pass native 4kb sector as 4kb sector onto SAS/SATA interface.



Also, only Windows 8, 8.1 or later OS support 4kn drives (for server, 2012 or later version). Majority of old utilities that directly work with disks will NOT work properly with 4k sectors as they assume sectors are always 512 byte sized. instead of checking.



So, to avoid any confusion with alignment and get the maximum performance, use new 4kn drives, new 4kn enabled adapters, and new OS.



I think this statement below is not correct:
"certain 4kb-native disks may choose to support 512-byte emulation. If they support 512-byte emulation, they can switch between this mode and 4kn depending on what the disk controller supports; they'll prefer 4kn, but fall back to 512e if they have to".




Sector size is fixed in the factory. I am not aware about any drive that is able to automatically change sector size on interface depending on RAID adapter capabilities. What I see in Seagate order systems are very separate part numbers depending on sector size on interface. Impossible to change sector size after disk is ordered (could be possible by some hacking, changing disk firmware, etc. but not officially supported). So if your drive is 512e it will always send only 512 byte sectors on interface, and never 4k sectors. If your drive is 4kn, it will always send only 4k sectors on interface and never 512 sectors. You decide only when ordering, as its different part numbers.



The possible drive formats are (the number indicates sector size on interface):



512n - 512 on disk, 512 on interface (simple)



512e - 4k on disk, 512 on interface (performance complications possible on old systems)



4kn - 4k on disk, 4k on interface (simple, best interface performance, do not work on old systems)




n or e means if specified sector size on interface is native disk sector size (n), or emulated size (e).



And the answer is: your disks are 512e disks (as they work with adapter that does not support 4kn disks), they are not 4kn disks. Your 512e disks will never use 4k sectors on interface with any RAID adapter. BTW, only the very new 6TB drives from Seagate are possible in 4kn format, and new 6TB and 8TB from HGST also can be ordered as either 512e or 4kn. All drives up to 4TB before were only available in 512e or 512n, I was not able to purchase any 4kn drive for testing before this September.



My personal recommendation is to use LSI adapters. Most compatible with best error reporting from anything I tested, and best performance. With latest firmware release, fully support 4kn disks. I am using many Smart Array adapters from HP also, as they come with HP ProLiant servers, but still no information if and when SmartArray adapters will support 4kn disks. Only host bus adapters mentioned in release notes - very recent firmware update enables support for 4kn disks. So, still 4kn disks are very new.



Hope I helped to make it clear.


Monday, May 7, 2018

dd using all memory and swap



I have a dedicated Centos 6.9 linux server with a company that crashed due to hardware failure and has been restored with the old HDD's on a usb caddy. I am trying to move lvm partitions over to the new internal hard disks. I have created a new volume group and same size logical volumes. The partitions are currently being used by kvm virtual machines. I have taken a snaphot of the lv. Then I am using something like the following to copy the partitions.



dd if=/dev/OldVolGroup/lv__snap of=/dev/NewVolGroup/lv bs=1M


This seems to work well with the following speed:




8192+0 records in
8192+0 records out
8589934592 bytes (8.6 GB) copied, 264.61 s, 32.5 MB/s


However, the ram and then swap slowly starts to get swallowed up. The box has 24GB of RAM and around 4GB spare when I start. It has a 2GB lv for swap.



I was able to turn some kvm machines off to free more memory, around 7-8GB and it just managed to finish the 8GB transfer without using all resources.



Can anyone tell me why it uses up so much memory and then swap? Is this normal?



Answer



It's because dd reads into cache before writing. If you have top running you'll see that it's cache that is being used up before memory. All you can really do is try a larger block size. It's not a good thing to be doing on a running server but obviously it's what you're working with.


Sunday, May 6, 2018

Dell Poweredge 2950 Hard drive size

I recently purchased a Dell Poweredge 2950 at an auction for a very low price, couldn't pass it up.
It didn't come with any hard drives so I ordered 5 SATA caddy trays and I am curious what the maximum size the hard disks could be. I plan on doing RAID 6 so I have plenty of redundancy and I would like to have at least 1TB drives installed. Will the 2950 support 5-6 1TB drives or maybe even 2TB drives? I'm not concerned whether or not dell "supports it". I have already tested a WD Green 500GB SATA drive I had lying around and it worked fine in the SAS bay. Also, will the caddys I ordered still have lights that tell me whether or not a drive is dead or is that just an SAS thing?

Windows Group Policy - Can I alter with VBscript?




First of all I'm not an expert in either Windows or Group Policy scripting but one of my tasks is to configure XP machines which are destined to be NOT network. Consequently I've used VBScript to change Registry settings but now I have some settings in the Group Policy which I currently configure by hand but want to script these or automate it in someway.



Could anyone tell me if I can make changes to the Group Policy using VBscript or do I need to use something else?



I found this page which outlines some possibilities but there's not much detail and I can't yet find follow up articles.



** Additional Information **



Using gpedit.msc I currently manually set the following settings in the GUI:





  1. Computer Configuration > Administrative Templates > System > Logon > Always wait for the network at computer startup and logon

  2. Computer Configuration > Administrative Templates > System > Turn Off Autoplay

  3. Computer Configuration > Windows Settings > Security Settings > User Rights Assignment > Access this computer from the network

  4. Computer Configuration > Windows Settings > Security Settings > User Rights Assignment > Deny access to this computer from the network

  5. Computer Configuration > Windows Settings > Security Settings > User Rights Assignment > Deny logon locally

  6. Computer Configuration > Windows Settings > Security Settings > User Rights Assignment > Logon Locally

  7. Computer Configuration > Windows Settings > Security Settings > Security Options > Network access: Shares that can be accesses from the network




I want to be able to script these changes using VBScript or some other means (.NET perhaps?)



I have found that for a couple of the above settings I can alter them directly in the registry but the changes are not seen in the Group Policy editor (gpedit.msc)


Answer



It's possible, yes.



Not really advisable, because the supported and recommended way to do this is through the gpedit.msc tool (or GPOs on a domain). As a result, doing it with a script is semi-documented at best, and you do run into some oddities trying.



If you're still determined to try, theses settings are really just registry keys. If can figure out which keys are changed, it's trivial to script something up to import all those registry keys and end up with the desired configuration. I'd just use a batch file, honestly.




The problem you'll run into, however, is that these changes need to be saved to %SYSTEM ROOT%\System32\GroupPolicy\User\Registry.pol to be enforced as local group policy, and I don't know of a good scripted solution to that, so the common workaround is to set this up the way you want on a second machine, and copy the %SYSTEM ROOT%\System32\GroupPolicy\User folder and files to your target machines.



I personally find the whole thing to be such a pain that I'd rather set up a small domain than go through that process to apply automate applying local group policy.


Saturday, May 5, 2018

ubuntu - Nginx 404 Deploying Ruby on Rails Application with Phusion Passenger



There are a lot of moving parts here that I have varying levels of familiarity with, so I'm hoping another set of eyes will help me discover why I'm getting a 404 after deploying my Rails app on Nginx running Phusion Passenger.



My app's /public directory currently resides at:



$pwd
>> /root/public_html/photobook/public



My server settings in nginx.conf:



server {
listen 80;
server_name pb.mysite.com;
root /root/public_html/photobook/public; # <--- be sure to point to 'public'!
passenger_enabled on;
}



But after restarting nginx:



sudo /opt/nginx/sbin/nginx


Going to my browser displays a 404 page served up by Nginx.



Are there rules for handling subdomains differently than I'm not aware of? Or is this something else entirely?




Also, since I'm new to Passenger on Nginx, is there a log file I should find that might give me more helpful feedback on the 404, like what page it's looking up?



Thanks!



Edit



First, thanks for the advice on /var/log.



I didn't know about it; this will be a great help in the future.




I'm not sure my syslogs (I have two that appear current and then it looks like .gz backups going back) are recording anything relevant, however:



Jun 23 01:25:01 myusername /USR/SBIN/CRON[11379]: 
(root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jun 23 01:49:01 myusername -- MARK --


Just that, the hourly cron task and whatever -- MARK -- is for, all the way back. Any ideas?


Answer




Check under /var/log. Unless you have an nginx or passenger log, everything will be under syslog.



Your next step will depend on what you see there. If the solution isn't readily apparent, post your log as nginx starts cold so we can get a better clue as to where to go next.



EDIT:



Since you have nginx under /opt, take a look under /opt/nginx/logs/error.log.



Also, you'll want to take your app out of /root as it can cause nginx to mis-read the path if you run nginx as root.




Lastly, consider creating an nginx user to run nginx to limit your exposure to baddies.


centos - Apache doesn't start: syntax error....DocumentRoot must be a directory



I read all the previous answer but I couldn't fix my problem.. I'm changing the server where my website is hosted and I'm setting again apache.. Now, after installing php, fail2ban, etc i have that issue:




service httpd start 
Avvio di httpd: Syntax error on line 292 of /etc/httpd/conf/httpd.conf: DocumentRoot must be a directory
[FALLITO]


I've already disabled selinux and restarted, but nothing happened..



sestatus 
SELinux status: disabled



In other answer I read to post that.. But I don't know what it is, but I saw it is very different from other I saw..



ls -laZ /var/www/html/
drwxr-xr-x root root ? .
drwxr-xr-x root root ? ..
drwxr-xr-x root root ? form



Obviously my httpd.conf has



DocumentRoot "/var/www/html”


The root I need is



DocumentRoot "/var/www/html/form”



Now with the default root doesn't work, but before editing some files (I can't remember which) it started correctly.
I hope you can help me.. I am using cents 6.5 x64..
Thank you very much



Andrea


Answer



As ETL mentioned, your problem is almost certainly the quote character in DocumentRoot "/var/www/html/form” - you need to use DocumentRoot "/var/www/html/form" (or just DocumentRoot /var/www/html/form, which works fine.).



Never edit a configuration file like this in a text editor that replaces "" with “” or -- with —, notably MS Word and Wordpad.


linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...