Sunday, April 7, 2019

Struggling to install Windows 2008 Server (x64)



I got a brand new workstation on Friday that I am trying to install with Windows 2008 Server (64-bit) for our development box. The workstation is an Intel i7, 6GB RAM, with 3 x HDDs (SATA) and we've got an MSDN subscription (also new, although the disk I'm using is dated May 2008).




The box was already built and had a trial version of Vista installed (presumably so that the supplier could test everything). First attempt at booting the workstation returned an error about starting Vista (I honestly forget what exactly) but choosing "last known good" got me into this Vista installation. I inserted the windows 2008 DVD and ran through the setup - I didn't note this error when it failed (although after this point I started taking notes) but I presumed it was related to moving the existing Vista files into a windows.01 folder and Setup restored the Vista installation.



So I ran setup again (booted from CD) and at the point of choosing where to install Windows I created a partition on the first drive, formatted the partition and proceeded with the install.



Windows copied the files very quickly, but expanding took a few minutes while the progress meter sat on 0%, was still on 0% when the server rebooted (unexpectedly, none of the usual Windows needs to reboot. 10, 9, 8...) but then I got



Bootmgr is missing
Press ctrl-alt-del to restart



Rebooted from CD and went into Repair. It didn't acknowledge that anything was installed on the drive but from a command prompt I could see the following folders:



$WINDOWS.~BT
$WINDOWS.^LS
PerfLogs
Program Files
Program Files (x86)
Users
Windows



Those folders had subfolders, but search for .exe or .dll files I only found the setup.exe in $WINDOWS.~BT so clearly the copying/expanding isn't working.



Formatted the drive (from the command prompt), then I ran setup. This time it completed the installation ok (the copying files to hard-drive was very quick, the expanding files task only took a couple of minutes but it showed progress) and then server rebooted gracefully. All working.



Only for some reason this time it installed 2008 Server Core! I swear I selected full installation but I thought we've got a result here - it only took 7 minutes to install (core) - so I formatted the drive again, then booted from CD, ran setup. Definitely selected Standard full installation (x-64) this time, installed onto partition 1 on disk 0. Only I got the 0% expanding and then bootmgr is missing error. Inspect the contents of C: - same as before...



I've been stuck in this loop since yesterday. Is it my MSDN DVD, or something about my hardware setup? Any suggestions?


Answer




Last time I had anything similar it was a bad installation CD, though it did at least give an error message while expanding files.


monitoring - Monitor Apache: Parse status page?



I want to monitor an apache server. Primary goal is to detect if MaxClients gets reached, but later other values, too.



I looked at the nagios plugins, but most plugins seem to parse the HTML of the status page. I think this is an ugly solution. Is there any better way to get the server status?




Log file checking is not part of this question.


Answer



You don't need to parse HTML - Apaches mod_status is capable of producing output well-suitable for regex parsing when called with the ?auto parameter (i.e. http://your-server/server-status/?auto). Depending on what monitoring system you need this to work with, there may or may not be a working / well-tested collector for this kind of thing.



As an example, take a look at how it is done in OpenNMS: https://wiki.opennms.org/wiki/Monitoring_Apache_with_the_HTTP_collector


Running a MS-SQL Server Stored Procedure from MS Access?



I have an Access 2010 front-end to an SQL-Server 2008 database. I'd like to use the stored procedures already on the server via the front-end!



Is this even possible?



(Bit of a newb so go easy on me (-: )


Answer



What kind of connection do you have? Does it let you pass T-SQL statements? If so, you should be able to just EXEC the sprocs... Unfortunately, I'm not familiar with Access, only SQL Server.



networking - Revealing the real IP address behind an AWS EC2 load balancer



I have a web-application that contains events that I'm interested in reading from the internet.



To do this, I have an AWS EC2 load balancer that sits in front of two EC2 instances. I have an application that tracks events on both of these servers.



The setup can be visualized like this:
Instance 1 and 2 -> Load Balancer -> Event Monitor




The problem I'm seeing is that each event appears to be coming from the load balancer IP and not from the EC2 instance IP. I'm convinced that when the load balancer forwards traffic, it puts its IP on the packet.



Is there a setting in AWS on the load balancer that I can change to forward packets with the actual IP address instead of the load balancer IP?


Answer




The problem I'm seeing is that each event appears to be coming from
the load balancer IP and not from the EC2 instance IP. I'm convinced
that when the load balancer forwards traffic, it puts its IP on the
packet.





You're convinced of this because yes, that is precisely how load balancers are designed to work.




Is there a setting in AWS on the load balancer that I can change to
forward packets with the actual IP address instead of the load
balancers IP?




Think that through for a minute. Sure, an LB could theoretically change the IP headers so that the source IP matches the backend server. There are several problems with this, though:





  1. Your backend servers have private IPs. Which your clients aren't going to be able to hit.

  2. When your clients hit your ELB, they first have to stand up a TCP connection before any HTTP can happen. So they have a TCP connection with the ELB, not with the backend server. Therefore when the client sees a packet with a source IP that doesn't match a current connection, it's going to discard it.

  3. You likely don't want to do this anyway, as backend instances change frequently and your clients shouldn't need to worry about that, nor should they be able to discover your private infrastructure.



Why do you want this anyway? If you really want your clients to be able to discover this information, you can possibly put the private IP into an HTTP response header, which your client has access to.


linux - Apache2 quietly breaking, won't make apache2.pid



When I run
# /etc/init.d/apache2 restart
It happily claims to have started the program, but hasn't actually done so. The output goes a little something like this.




www ~ # /etc/init.d/apache2 restart
* Stopping apache2 ...
[Sun Oct 25 23:50:38 2009] [warn] Useless use of AllowOverride in line 63 of /etc/apache2/vhosts.d/30_subversion_ssl_vhost.conf.
[Sun Oct 25 23:50:38 2009] [warn] NameVirtualHost --removed--:443 has no VirtualHosts
[Sun Oct 25 23:50:38 2009] [warn] NameVirtualHost --removed1--:8443 has no VirtualHosts
httpd (no pid file) not running [ ok ]
* Starting apache2 ...
[Sun Oct 25 23:50:38 2009] [warn] Useless use of AllowOverride in line 63 of /et
c/apache2/vhosts.d/30_subversion_ssl_vhost.conf.
[Sun Oct 25 23:50:38 2009] [warn] NameVirtualHost --removed--:443 has no VirtualHosts
[Sun Oct 25 23:50:38 2009] [warn] NameVirtualHost --removed1--:8443 has no VirtualHosts

[ ok ]


As you can see, no httpd process was ever running, despite claiming to have started before, as it does once again here. Verifying by hand, /var/run/apache2/apache2.pid doesn't exist. The run/apache2/ directory wasn't even there before I added and chowned it by hand. Nothing is appended to /var/log/apache2/error_log, nor to any of the domain specific logs for any of my vhosts.



However, running with the --verbose and --debug options, it does seem to get hung up on a for loop involving the location /var/run/apache2/apache2.pid




+ i=0
+ '[' '!' -e /var/run/apache2.pid ']'
+ '[' 0 -lt 10 ']'
+ sleep 1
++ expr 0 + 1
+ i=1
+ '[' '!' -e /var/run/apache2.pid ']'
+ '[' 1 -lt 10 ']'
+ sleep 1
++ expr 1 + 1
...



I'm running apache 2.2 on the latest build of Gentoo Linux. The vhosts.d config files have been imported from a previous setup, fyi, but the httpd.conf and associated files are all brand new.



edit:
on further investigation, the apache2 init script calls

/usr/sbin/apache2 -D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST \
-D LANGUAGE -d /usr/lib/apache2 -f /etc/apache2/httpd.conf -k start
If I remove the -D SSL part, the pid creates itself fine... however, I would like ssl... any ideas?

edit2:



start() {
checkconfig || return 1


[ -f /var/log/apache2/ssl_scache ] && rm /var/log/apache2/ssl_scache

ebegin "Starting ${SVCNAME}"
${APACHE2} ${APACHE2_OPTS} -k start

i=0
while [ ! -e "${PIDFILE}" ] && [ $i -lt ${TIMEOUT} ]; do
sleep 1 && i=$(expr $i + 1)
done


test $i -le ${TIMEOUT}
eend $?
}


If this provides any further insight, let me know



edit3:




if, in my ssl vhosts, i comment out SSLEngine on, apache successfully starts...



edit4:



Just a look at my apache build options




www ~ # emerge -av apache

These are the packages that would be merged, in order:


Calculating dependencies |... done!
[ebuild R ] www-servers/apache-2.2.11-r2 USE="sni ssl -debug -doc -ldap (-s
elinux) -static -suexec -threads" APACHE2_MODULES="actions alias asis auth_diges
t authz_host autoindex cache dav deflate dir env expires filter headers include
info log_config logio mime negotiation proxy rewrite setenvif speling status use
rdir usertrack -auth_basic -authn_alias -authn_anon -authn_dbd -authn_dbm -authn
_default -authn_file -authz_dbm -authz_default -authz_groupfile -authz_owner -au
thz_user -cern_meta -charset_lite -dav_fs -dav_lock -dbd -disk_cache -dumpio -ex
t_filter -file_cache -ident -imagemap -log_forensic -mem_cache -mime_magic -prox

y_ajp -proxy_balancer -proxy_connect -proxy_ftp -proxy_http -substitute -unique_
id -version -vhost_alias" APACHE2_MPMS="-event -itk -peruser -prefork -worker" 0
kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

Would you like to merge these packages? [Yes/No]


And also verifying checksums





www ~ # equery check apache
[ Checking www-servers/apache-2.2.11-r2 ]
!!! /etc/apache2/vhosts.d/00_default_vhost.conf has incorrect md5sum
!!! /etc/conf.d/apache2 has incorrect md5sum
!!! /etc/apache2/vhosts.d/00_default_ssl_vhost.conf has incorrect md5sum
* 403 out of 406 files good


edit5:




Woo! Edit 5!



Startup output after changing the corresponding line in /etc/conf.d/apache2 to
APACHE2_OPTS="-X -e debug -D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE -D PHP5



 * Starting apache2 ...
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module actions_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module alias_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module asis_module

[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module authz_host_modul
e
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module autoindex_module

[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module cgi_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module deflate_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module dir_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module env_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module expires_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module filter_module

[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module headers_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module include_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module info_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module log_config_modul
e
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module logio_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module mime_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module negotiation_modu
le
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module rewrite_module

[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module setenvif_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module speling_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module ssl_module
[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module usertrack_module

[Tue Oct 27 23:15:19 2009] [debug] mod_so.c(246): loaded module php5_mod [ ok ]


No hints yet. Also, not sure how to verify ssl config, if that is, indeed, the problem


Answer




Hmm, I am pretty suspicious of the config file



/etc/apache2/vhosts.d/30_subversion_ssl_vhost.conf


When you remove the '-D SSL' you will cause all parts of the configuration files that are enclosed in ... to be skipped. The default SSL vhost file on my Gentoo box is wrapped in that tag so I wonder if, by removing the '-D SSL', you are preventing the config in 30_subversion_ssl_vhost.conf from being run at all and if that is what is allowing Apache to start.



If you temporarily remove the file 30_subversion_ssl_vhost.conf from /etc/apache2/vhosts.d does Apache run? Are there any other SSL related vhost.conf files in vhosts.d? My reasonably fresh/unused Apache install's vhosts.d directory looks like this:



# pwd && ls

/etc/apache2/vhosts.d
00_default_ssl_vhost.conf 00_default_vhost.conf default_vhost.include


edit 1:



So much for that theory :) I am now wondering if the problem is with the Apache SSL setup itself. I apologize if I am covering ground you have already checked but I am wondering if you could do the following to help verify your Apache install.



On my Apache install with working SSL the use flags are as follows:




# emerge -av apache

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild U ] www-servers/apache-2.2.11-r2 [2.2.11] USE="ssl -debug -doc -ldap (-selinux) -sni -static -suexec -threads" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias -asis -auth_digest -authn_dbd -cern_meta -charset_lite -dbd -dumpio -ident -imagemap -log_forensic -proxy -proxy_ajp -proxy_balancer -proxy_connect -proxy_ftp -proxy_http -substitute -version" APACHE2_MPMS="-event -itk -peruser -prefork -worker" 64 kB


In particular do you have the 'ssl' USE flag set?




Also, could you use equery to verify the integrity of your apache2 install? If you do not have the equery command you can install it by running 'emerge -av gentoolkit'. The following command should verify the integrity of your apache install:



equery check apache


On my server the above command gives the following output:



[ Checking www-servers/apache-2.2.11 ]
!!! /etc/apache2/vhosts.d/00_default_ssl_vhost.conf has wrong mtime (is 1256620928, should be 1246793824)
!!! /etc/apache2/modules.d/00_default_settings.conf has wrong mtime (is 1246796304, should be 1246793824)

!!! /etc/conf.d/apache2 has incorrect md5sum
* 429 out of 432 files good


edit 2:



Well the install looks good to me, so much for theory 2. I am wondering if we can coax Apache into giving some more information on startup. In /etc/conf.d/apache2 if you change your APACHE2_OPTS line from:



APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE"



to



APACHE2_OPTS="-X -e debug -D DEFAULT_VHOST -D INFO -D SSL -D SSL_DEFAULT_VHOST -D LANGUAGE"


and then start Apache (/etc/init.d/apache2 start) the daemon should stay in the foreground (the -X flag) and output debuging messages as it starts (the -e debug option). Maybe this will give a clue as to why it is dying on startup.


Friday, April 5, 2019

linux - FTP and Apache permission issues

Im having issues as to which user should own my www directory - ftp or apache?
When set to the ftp user, the user can add, remoe and easily modify files but php file system actions generate permission denied errors (ofcourse because they require the user to be apache). If however, the www directory is chown to apache, the ftpuser wont be able to perform some actions like file modification and deletion.

Any one ever encountered similar issue? What's the fix?
Thanks

Thursday, April 4, 2019

apache 2.2 - MongoDB PHP EC2 Setup Configuration



I am new to web development and server set up. I am looking for some advice or a link to a tutorial on setting up a production system up. Right now, I have a server (Ubuntu, Apache, MongoDB, and PHP). It receives a request, PHP queries Mongo, and PHP sends out the requested data. How do I make that work with more servers? I've read that you can make a cluster of a primary and two slave nodes which work as separate servers running Mongo, but do those also run PHP? Or is the primary the only one running the PHP?
I have read some docs on Mongo site and a video of someone from 10gen going through it, but they are geared towards people that seem to already understand this stuff, I have no idea and need to start from a beginning stage.




If anyone can help me understand where PHP (Acting as my API) lives in these clusters, that would be greatly appreciated! Thanks in advance for any help!


Answer



When you scale out, you generally break out your components. You end up with something like this (these are not standard names, nor do you have to do this, but this is common):




  • Application Server(s) (Linux, Apache, PHP)

  • Database Servers(s) (Linux, MongoDB)



How many you need of each will depend on your application needs. On the MongoDB side of things, for production you will want to run at least a replica set. This would constitute one primary, one secondary and an arbiter as a minimum (see the docs, but as a quick guideline, the primary and secondary should be identical and are generally dedicated hosts, the arbiter has no data and is lightweight, could be run on a shared host).




The applications server will take requests from clients, be they human or some other app calling your application. The application will use the PHP driver to communicate with the MongoDB replica set over the network (rather than locally, which is what you have probably been doing up until now).



Eventually, if you continue to grow, you may need to expand to a sharded MongoDB set up. From a PHP perspective, the only difference will be that you talk to a mongos process instead of directly to the replica sets. That sounds like it is down the road for you in this case though, so be aware of it but it should not be necessary in the immediate future.


linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...