Monday, March 5, 2018

Warm Standby with Windows Server 2003



We have a primary file server for our Windows Server 2003 (Standard Edition) domain. We're running active directory in a plain vanilla type environment. When we recently migrated from an old server to a new one, I set up a DNS entry as Files.OurDomain.local as an Alias (CNAME) to the real server name. I disabled Strict Name Checking (see MSDN link text). All client machines now refer to \\Files or \\Files.OurDomain.local to get to their files. No one ever uses RealServer.OurDomain.local. Printers also use an alias to get to their server.



I plan to use this setup in the event of a significant server problem to switch the dns alias to another server as a poor man's warm standby. Also it should make server migrations in the future very easy. I tried DFS but it didn't give me the comfort that I had exact control over where files were going. The default TTL (Time to Live) for the DNS entries is 1 hour. That's probably what it would take the tech staff to determine that the server had been lost and we would really need to switch to the backup.



Any concerns or problems (or even accolades) with this approach?



Answer



It seems like you're handicapping yourself by not just using DFS or Clustered File Sharing.



http://support.microsoft.com/kb/284838


windows server 2008 - Newbie one: Virtual Networks - Hyper-V - Remote Destktop - Only one physical NIC



I'll try to explain my situation and I'll apreciate any help:



I have a phisical server (quad core, 4Gb ram, 1TB raid 10, etc) with Win Server 2008 R2 enterprise, running IIS, Printing, etc...



Also, I want to set up 2 virtual Servers with 2008 R2 standart one with SQL Server and the other with Team Foundation.




What i need is:




  1. Being able to access from inside the private phisical network, to Remote Desktops on each of the Virtual and the phisical Servers

  2. Had Access from the outside, using a router and port Forwarding, to the TFS server and the IIS server (one is virtualized, the other is phisical)



This is it, but note that I only have one Phisical Nic. How do I configure this to work. When i set up the hyper-v role, on the wizard something like it showed up but I don't remmember what i choose, and right now, I cannot access none of the servers from remote desktop, not even from the phisical private network.



Can anybody point me, what can i do? Thanks in advance




(sorry 4 my english, i'm a spanish talker and my english isn't that good)


Answer



I've never used Hyper-V, so I'm basing this answer on other hypervisors.



When you create & configure a VM, it usually gives you the option to add a virtual NIC card. Usually, there will be several ways to configure this NIC, and how it interfaces with the physical NIC. The 2 most common options are "bridged networking" and "NAT".



"Bridged networking" bridges your virtual NIC to your physical NIC, and will make the virtual NIC act like a machine on the same subnet as the host machine. "NAT" configures your virtual NIC to use network address translation, and make it look as though it is sitting on a different subnet, behind a virtual router.



From your description, my guess is that your VMs are configured to use NAT, so they don't appear on your host subnet. For you to access them this way, you would need to configure routing between the 2 networks. If this is what you desire, you'll need to figure it out in Hyper-V, configure one of the virtual servers as a router/gateway, or create a virtual router with something like Vyatta.




However, if this is not what you want, then the simplest solution is to change the VM NICs to use bridged networking. If there is no option to change it, add a new NIC with bridged, and delete the old one.


IIS Rewrite using IIS Manager



Hi there I am attempted to do a proper IIS rewrite utilizing the IIS manager credentials my hosting provider has given me. I have no experience with this and the articles I have read online so far have not been too helpful.




Here is the scenario: I have an ASP.net MVC 5 website (reallylongmaindomain.com, for ease of reading) that physically has the files uploaded to it. It is secured with an SSL certificate. The literature we are handing out has a shorter domain (not a URL-shortened domain) that we will call shortdomain.com, which does not have any files. However, if you navigate to shortdomain.com, you get an SSL domain name mismatch error (ERR_CERT_COMMON_NAME_INVALID to be precise). This is what I need to avoid.



In comes the IIS manager. I have successfully logged in to the system and added a URL Rewrite. This is the rule I have: (http(s)?://)?(www.)?shortdomain.com. This handles any http/https and/or www that the user may put in the address bar. The action properties are set to "Redirect" (Permanent 301) and the Redirect URL is set to https://reallylongmaindomain.com, but I am still getting the SSL certificate error and the URL in the window is still shortdomain.com.



I am sure I am just missing a simple thing or two. Do I also need to add the rule into my Web.Config file? I was under the impression the IIS manager rules superseded the Web.Config rules. EDIT: The rule was added to my web.config, here it is:



    










Thanks!


Answer



Turns out this is the rule I was looking for:
















After applying this rule, navigating to shortdomain.com resolves perfectly fine to https://reallylongmaindomain.com.



Hopefully this helps someone else out!


Saturday, March 3, 2018

linux - Why is this SSD drive failing with bad sectors, and was it predictable?



Note: this question was previously closed as off-topic. You can read the discussion. My reasons for asking it here are:





  • This drive is in an offline content cache server for schools in rural Zambia.

  • The servers are created from disk images, and all the content is replaceable.

  • It has to be cheap because Zambian schools are budget-limited and there will be a lot of them.

  • It also has to be reliable because it might be 8 hours each way on bad roads to replace.

  • I'm not allowed to ask here what drives are not "ultra-cheap crap".

  • So we're doing our own research and experimentation on drives that meet these criteria.

  • My inability to repair the bad sectors by overwriting them (auto reallocate) defied my assumptions and I wanted to know why.

  • I thought maybe a SECURITY ERASE might fix the bad sectors, but wanted opinions of others before I trash the drive.

  • I thought I might have missed something in the SMART data that could have predicted the failure.




This is a Kingston 240GB SSD disk that was working fine on site for about 3 months, and has suddenly developed bad sectors:



smartctl 5.41 2011-06-09 r3365 [i686-linux-3.2.20-net6501-121115-1cw] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model: KINGSTON SVP200S3240G
Serial Number: 50026B7228010E5C

LU WWN Device Id: 5 0026b7 228010e5c
Firmware Version: 502ABBF0
User Capacity: 240,057,409,536 bytes [240 GB]
Sector Size: 512 bytes logical/physical
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: ACS-2 revision 3
Local Time is: Tue Mar 5 17:10:24 2013 CAT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled


=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x02) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever

been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.

Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine

recommended polling time: ( 48) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x0021) SCT Status supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 084 084 050 Pre-fail Always - 10965286670575

5 Reallocated_Sector_Ct 0x0033 100 100 003 Pre-fail Always - 16
9 Power_On_Hours 0x0032 000 000 000 Old_age Always - 46823733462185
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 127
171 Unknown_Attribute 0x0032 000 000 000 Old_age Always - 0
172 Unknown_Attribute 0x0032 000 000 000 Old_age Always - 0
174 Unknown_Attribute 0x0030 000 000 000 Old_age Offline - 131
177 Wear_Leveling_Count 0x0000 000 000 000 Old_age Offline - 1
181 Program_Fail_Cnt_Total 0x0032 000 000 000 Old_age Always - 0
182 Erase_Fail_Count_Total 0x0032 000 000 000 Old_age Always - 0
187 Reported_Uncorrect 0x0032 000 000 000 Old_age Always - 49900

194 Temperature_Celsius 0x0022 033 078 000 Old_age Always - 33 (Min/Max 21/78)
195 Hardware_ECC_Recovered 0x001c 120 120 000 Old_age Offline - 235163887
196 Reallocated_Event_Count 0x0033 100 100 003 Pre-fail Always - 16
201 Soft_Read_Error_Rate 0x001c 120 120 000 Old_age Offline - 235163887
204 Soft_ECC_Correction 0x001c 120 120 000 Old_age Offline - 235163887
230 Head_Amplitude 0x0013 100 100 000 Pre-fail Always - 100
231 Temperature_Celsius 0x0013 100 100 010 Pre-fail Always - 0
233 Media_Wearout_Indicator 0x0000 000 000 000 Old_age Offline - 363
234 Unknown_Attribute 0x0032 000 000 000 Old_age Always - 208
241 Total_LBAs_Written 0x0032 000 000 000 Old_age Always - 208

242 Total_LBAs_Read 0x0032 000 000 000 Old_age Always - 1001

SMART Error Log not supported
SMART Self-test Log not supported
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing

5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.


Now I get bad blocks in certain places on the disk:



root@iPad2:~# badblocks /dev/sda -v
Checking blocks 0 to 234431063

Checking for bad blocks (read-only test): 8394752 done, 1:15 elapsed
8394756 done, 1:21 elapsed
8394757 done, 1:23 elapsed
8394758 done, 1:24 elapsed
8394759 done, 1:27 elapsed
...
190882871one, 29:49 elapsed
190882888one, 29:53 elapsed
190882889one, 29:54 elapsed
190882890one, 29:56 elapsed

190882891one, 29:58 elapsed
done
Pass completed, 80 bad blocks found.


They appear to be repeatable, and auto reallocation fails, so they can't be fixed by writing to them:



root@iPad2:~# badblocks /dev/sda -wvf 8394756 8394756
/dev/sda is apparently in use by the system; badblocks forced anyway.
Checking for bad blocks in read-write mode

From block 8394756 to 8394756
Testing with pattern 0xaa: 8394756
done
Reading and comparing: done
Testing with pattern 0x55: done
Reading and comparing: done
Testing with pattern 0xff: done
Reading and comparing: done
Testing with pattern 0x00: done
Reading and comparing: done

Pass completed, 1 bad blocks found.


And I get errors like this in the system logs:



ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
ata1.00: irq_stat 0x40000000
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/08:00:08:30:00/00:00:01:00:00/40 tag 0 ncq 4096 in
res 51/40:08:08:30:00/00:00:01:00:00/40 Emask 0x409 (media error)

ata1.00: status: { DRDY ERR }
ata1.00: error: { UNC }
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] Unhandled sense code
sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 0:0:0:0: [sda] Sense Key : Medium Error [current] [descriptor]
Descriptor sense data with sense descriptors (in hex):
72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
01 00 30 08
sd 0:0:0:0: [sda] Add. Sense: Unrecovered read error - auto reallocate failed

sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 00 30 08 00 00 08 00
end_request: I/O error, dev sda, sector 16789512
Buffer I/O error on device sda, logical block 2098689
ata1: EH complete


Now I don't understand why auto reallocate is failing on this disk. The smartctl output all looks fine to me. Only 16 sectors have been reallocated, that's not many at all. I can't see any legitimate reason why this drive refuses to reallocate sectors. Is this model of SSD just broken or badly designed?



Notes:





  • attribute 174 is "Unexpected Power Loss" according to Kingston's docs.

  • 131 unexpected power losses is quite bad.

  • attribute 187 (Reported_Uncorrect) is 49900 out of a possible maximum of 65535

  • highest temperature ever is quite high at 78'C



The most interesting SMART counters are hidden by Kingston on this drive. But we can infer the number of spare sectors from from attribute 196. Reallocated_Event_Count, which has the following formula for the normalised value:



100 -(100* RBC / MRC)

RBC = Retired Block Count (Grown)
MRE = Maximum reallocation count


Since the normalised value is 100, this implies that RBC << MRE, so we are nowhere close to having exhausted all available sectors for reallocation.


Answer



Cheap SSDs seem to have serious quality issues. You will find lots of users that have issues with your particular drive. However I think that the vendors also sell different drives (e.g. with other NAND chips/controllers) under the same label. So each drive may behave differently.



The SMART values give no indication that the drive would fail soon. In my experience its the same: Suddenly drive errors occur and then the disk fails.




What are your reasons why you use SSDs? I see the advantages of SSDs as there are no mechanical parts and as they are dustproof and produce less heat. However I also see a lot of disadvantages.



e.g. the number of writes to a single memory cell that even with wear-leveling may be reached quickly on a busy volume e.g. when you are using a filesystem with journaling.



And the electronics is also affected by high humidity or high temperatures - the same as with conventional hard drives.



Why not use cheaper conventional hard drives instead and (if raid is not required) ship the server with spare drives that are not connected until they are needed as a replacement (already present in the server case or mounted in a hot-swap cage so that the disk may be used in different servers). Then they could get prepared by a script in the field or from remote (if possible).



As long as conventional hard drives are not powered up, the transport to their destination can be rough...




If there are multiple school servers / permanent clients and a reliable/redundant network, maybe a distributed filesystem could also help to create a failsafe cache server (e.g. by using glusterfs).


ntp - Linux ntpd leap second preparation

There will be a leap second on June 30, 2015 at 23:59:60 UTC.



I have a Linux based stratum 1 NTP server which uses GPS as a reference clock. The server is using the "default" ntp.org ntpd.



How should I prepare for the leap second to ensure that the correct time is available at and after the leap second?



Will the ntpd server behave correctly?




If uncertain, is there a way to test this beforehand?

Friday, March 2, 2018

networking - Part of LAN network connected to 1 switch freezes up periodically

I'm managing an office network which is spread across 2 buildings. (see network setup below)



In building 2, we are getting a problem where the network slows down and then completely cuts off. The only way to bring it back is to power cycle the switch, then before it slows down again, go into the GUI dashboard and power cycle it from the dashboard.



The issue has happened 4 times (and is becoming more frequent). It first happened 2 weeks ago, then again yesterday, and twice today.



I'm trying to work out what could be causing it. I've believe I've narrowed it down the switch as the only network devices in building 2 are the switch and the WiFi access point. When the switch is down, the WiFi access point is still running.




When building 2 is down, if i go to building 1, I can still connect to the network fine.



Any idea what might be causing the above? My 2 guesses are:




  1. network storm - but this would be odd as we haven't recently added notable new devices to the network. So why has it come about all of a sudden, plus why is it getting worse on each reset?


  2. faulty switch (needs replacing under warranty)








Network is configured as follows :



Building 1




  • Draytek 2860 router (connected to fibre modem)

  • Netgear 24 gigabit port smart switch

  • Ubiquity UAP wifi access point (mostly used by mobiles / tablets.. low traffic)


  • 4 wired LAN computers (heavier use, as we all use dropbox on LAN sync mode for heavy files)

  • printers and other devices

  • all cabling cat6



Building 2 (connected to building 1 by cat6 c. 30m cable length)




  • Netgear 48 gigabit port smart switch (Netgear GS748T)

  • Ubiquity UAP wifi access point (used mostly by 2 laptops, heavier usage that building 1, again most traffic is used from dropbox LAN sync.)


Thursday, March 1, 2018

Multiple Enterprise-level Western Digital hard drives failing



In 2007, I built a 2.5TB server with 5 500GB WDC RE2 hard drives in a RAID5 configuration. Since these are Western Digital's Enterprise-level drives, they come with a 5 year warranty, but I have been seeing these drives die much sooner than that. I lost one drive in 2009 and last year I had two drives fail close together. Since I had to rebuild the server from backups, I decided to rebuild it with only 4 drives using RAID6 since we didn't need all that space. All replacement drives have been WDC RE3 drives and have had no failures as of yet. 2 weeks ago, the fourth original RE2 drive failed with a serious hardware malfunction. It sounded very sickly and was completely non-responsive. It did not register on the system so I couldn't run any S.M.A.R.T. tests or even pull off it's serial number. Two days ago the final RE2 drive failed, but was a little less serious of a crash. It had am unrecoverable sector read error and was unable to reallocate it, but otherwise was fully functional. S.M.A.R.T. even claimed the drive was still healthy. This does not seem normal to loose five Enterprise-level drives with a year left to spare on their warranty.




When I lost the two drives last year, I added additional fans to enhance the cooling. Currently, the three responsive drives report their temperatures at 30°C which is well withing their 0-60°C operating range. Someone mentioned that I might have an under-rated power supply. It's a 500W supply, but I am using the old-style molex for all but two of the SATA drives as it only came with two SATA power connectors. Could that be a problem?



Unfortunately, the new RE3 drives only have a SATA power connector so I will have to either get a splitter or a different power supply once I replace this next disk.



Or is it possible I might just have purchased 5 drives from a bad batch?


Answer



I must say that sadly this pretty much backs our experience with WD RE2 drives. Out of our little drive park of 10 drives, We had to replace 6 in the first two years. And some the replacements sent to us by WD failed rather quickly as well. I believe the "bad batch" spans across the entire RE2 series, as none of our other installed drive types show failure numbers even in the same magnitude as the WD RE2 ones.


linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...