monitoring - how to monitor web app availability with splunk?

I've recently downloaded and installed a Splunk 4.0.4 Server Enterprise trial (running on Windows Server 2008 if that matters), and now I want to set it up to monitor a few web pages in addition to the logs I'm already watching with Splunk.

I installed the "Web Page Monitor" app from the Splunk website, but now I'm baffled about how to set it up to start monitoring some web pages.

I poked around the file system and found \Program Files\Splunk\etc\apps\webping\ which (according to the MANIFEST file) has the friendly name of "Web Page Monitor". (No idea why Splunk isn't showing the friendly name on the Manager/Apps page-- it lists "webping" instead.)

Ideally, I'd like to have it check a particular URL every minute, and send me an email if it returns anything other than a 200 status or takes longer than 10 seconds to return.

Any idea how to do this?

UPDATE: I know that Splunk's monitoring/alerting support is pretty limited (and I'm trying out other monitoring tools in parallel) but that said my monitoring/alerting needs are not complex. Using multiple management tools, even if each is best-of-breed for its task, comes with a cost-- so while I'm trying out Nagios, etc. I'd also like to understand what Splunk can do (even if it's limited compared with other tools), so I can make an informed decision about whether to run a separate monitoring tool or not.

sata - Best performance: 1tb vs larger drives in app using only sequential read/writes

I have a custom app that is multi-threaded; each thread runs on its own logical core (the workstation is a dual xeon, with 12 physical and 24 logical cores). So there are 24 threads running simultaneously.

I've been researching the multitude of storage options over the last 2 days, and my head is spinning at about 15k rpm.

The app has 2 modes, and they are exclusive: read data or write data; by this I mean that they won't be doing reads/writes interleaved. Each thread will just be doing long sequential reads or writes. The total storage I will need is huge: over 50 tbs (if you are reading this in the year 2016 you are probably having a good chuckle right now over the word "huge".)

Each thread will be reading or writing a file that is about .8tb

I am going to go with jbod, because if a drive fails, all I need to do is swap it, and the app will re-create the data in about 10 minutes.

I will be putting the drives into an external tower or rack, using a SATA OR SAS controller (haven't figured out the +/- of those yet).

So, my question: am I correct in assuming that using 1tb drives for this particular app would be better performance wise than using drives 2, 3 or 4 times that size? It would seem that, unless a 3 tb drive has sequential read/write throughput that is 3x that of a 1 tb drive, the smaller drive is the way to go.

Obviously, using 3 tb drives reduces the number of drives I need to worry about by 1/3, but that would only be a consideration if I could achieve performance that is in the same ballpark as the 1tb drives.

Answer

Most importantly, for best performance you need 24 drives (or more, bear with me), because you have 24 threads. If there are less disks than threads, you do not have sequential operation. Considering two threads on a single disk, it will have some seeks; each seek is say 10 ms, so a loss of about 1 MB of transfer opportunity. With only 10 seeks per second, you have 90 MB/s (2 x 45) instead of 1 x 100 MB/s.

I think you would be better with 48 drives 1 TB each. Drives would be paired, to get 24 stripped (aka RAID0) groups. You can assume that stripping done on OS level will have an unnoticeable impact, so effectively each thread gets double throughput of a 1 TB drive.

I see no possible benefit of 3 TB drives performance-wise.

Still the biggest performance benefit would be something totally different. Just make sure that the application streams the data effectively - that it seriously feeds the queue on the HDD with those I/O commands. If it is written in a way that it clumsily waits for some I/O to get completed before queueing another I/O, then it would severe the throughput.

• Best to fix it in the application itself or...



• To partly alleviate this, you would need to invest a lot of money into a disk array with a huge extra cache.

PS. Love the remark about the 2016, hi there folks! Did you get those hover boards already?

dig - bind not able to query some servers / domains

We have been experiencing problems with our company's DNS server when trying to resolve only certain domains, we are running BIND 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 on a CentOS 6.5 server. We are autoritative for some zones and our internal clients and mail system resolve using this server. One of the domains we are having trouble resolving is www.dhl.com, here's what we get when querying using dig:

[root@serverx etc] dig www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> www.dhl.com
;; global options: +cmd
;; connection timed out; no servers could be reached

and

[root@serverx etc]# dig +trace www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +trace www.dhl.com
;; global options: +cmd
.           517419  IN  NS  g.root-servers.net.
.           517419  IN  NS  a.root-servers.net.
.           517419  IN  NS  h.root-servers.net.

.           517419  IN  NS  m.root-servers.net.
.           517419  IN  NS  f.root-servers.net.
.           517419  IN  NS  b.root-servers.net.
.           517419  IN  NS  l.root-servers.net.
.           517419  IN  NS  j.root-servers.net.
.           517419  IN  NS  k.root-servers.net.
.           517419  IN  NS  e.root-servers.net.
.           517419  IN  NS  i.root-servers.net.
.           517419  IN  NS  d.root-servers.net.
.           517419  IN  NS  c.root-servers.net.

;; Received 496 bytes from 192.168.X.X#53(192.168.X.X) in 11 ms

com.            172800  IN  NS  a.gtld-servers.net.
com.            172800  IN  NS  d.gtld-servers.net.
com.            172800  IN  NS  k.gtld-servers.net.
com.            172800  IN  NS  b.gtld-servers.net.
com.            172800  IN  NS  j.gtld-servers.net.
com.            172800  IN  NS  c.gtld-servers.net.
com.            172800  IN  NS  h.gtld-servers.net.
com.            172800  IN  NS  l.gtld-servers.net.

com.            172800  IN  NS  e.gtld-servers.net.
com.            172800  IN  NS  g.gtld-servers.net.
com.            172800  IN  NS  m.gtld-servers.net.
com.            172800  IN  NS  f.gtld-servers.net.
com.            172800  IN  NS  i.gtld-servers.net.
;; Received 489 bytes from 202.12.27.33#53(202.12.27.33) in 6128 ms

dhl.com.        172800  IN  NS  ns4.dhl.com.
dhl.com.        172800  IN  NS  ns6.dhl.com.
dig: couldn't get address for 'ns4.dhl.com': no more

And when I do dig using google's dns server:

dig @8.8.4.4 www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> @8.8.4.4 www.dhl.com
; (1 server found)
;; global options: +cmd
;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11325
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.dhl.com.           IN  A

;; ANSWER SECTION:
www.dhl.com.        1619    IN  CNAME   ngw.dhl.com.edgesuite.net.
ngw.dhl.com.edgesuite.net. 8520 IN  CNAME   a1085.g.akamai.net.
a1085.g.akamai.net. 19  IN  A   23.74.2.113

a1085.g.akamai.net. 19  IN  A   23.74.2.120

;; Query time: 229 msec
;; SERVER: 8.8.4.4#53(8.8.4.4)
;; WHEN: Thu Dec  4 14:47:56 2014
;; MSG SIZE  rcvd: 129

No problem!, and from the same server...

When you look to "/var/log/messages" nothing gets logged!!!. Again, this only happens with certain domains and this server was working ok a couple of days ago, we also have disabled selinux for testing purposes.

This is our named.conf file (named is runnig in a chroot environment):

options {
//      listen-on port 53 { 127.0.0.1;192.168.xx.x; };
//      listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";

        memstatistics-file "/var/named/data/named_mem_stats.txt";
//      allow-query     { localhost; };
        allow-query     { any; };
        recursion yes;
        allow-recursion { recursive-clients; };
//      query-source address * port 53;
//      dnssec-enable yes;
        dnssec-enable no;
//      dnssec-validation yes;
        dnssec-validation no;

        dnssec-lookaside auto;

        /* Path to ISC DLV key */
        bindkeys-file "/etc/named.iscdlv.key";

        managed-keys-directory "/var/named/dynamic";
        allow-transfer { xxx.x.x.x; xxx.x.x.x; xxx.x.x.x; 127.0.0.1; };
        allow-update { 192.168.xx.xx; };
//        forwarders { 8.8.4.4; };
};


acl recursive-clients { xxx.x.x.x/24; 127.0.0.1; xxx.xxx.xx.x/24; xx.xxx.xxx.xxx/29; xxx.xxx.xx.x;};

logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };

};


zone "." IN {
        type hint;
        file "named.ca";
};


zone "domain.com.xx" IN {
        type master;
        file "db.domain.com.xx";

        allow-transfer { xxx.xxx.xx.xx; xxx.xxx.xxx.x; };
};

zone "xx.xxx.xxx.in-addr.arpa" IN {
        type master;
        file "db.xxx.xxx.xx";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

Any idea guys???... I've been doing tests and researching since yesterday and I cannot figure what is happening!!!

Thanks in advance for any help or idea.

Here it the result using dig +trace +additional www.dhl.com:

dig +trace +additional www.dhl.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +trace +additional www.dhl.com
;; global options: +cmd
.           518340  IN  NS  h.root-servers.net.
.           518340  IN  NS  l.root-servers.net.
.           518340  IN  NS  e.root-servers.net.
.           518340  IN  NS  k.root-servers.net.
.           518340  IN  NS  i.root-servers.net.
.           518340  IN  NS  m.root-servers.net.
.           518340  IN  NS  b.root-servers.net.
.           518340  IN  NS  c.root-servers.net.

.           518340  IN  NS  g.root-servers.net.
.           518340  IN  NS  f.root-servers.net.
.           518340  IN  NS  d.root-servers.net.
.           518340  IN  NS  a.root-servers.net.
.           518340  IN  NS  j.root-servers.net.
k.root-servers.net. 518345  IN  A   193.0.14.129
k.root-servers.net. 518345  IN  AAAA    2001:7fd::1
b.root-servers.net. 518345  IN  A   192.228.79.201
b.root-servers.net. 518345  IN  AAAA    2001:500:84::b
c.root-servers.net. 518345  IN  A   192.33.4.12

c.root-servers.net. 518345  IN  AAAA    2001:500:2::c
i.root-servers.net. 518345  IN  A   192.36.148.17
i.root-servers.net. 518345  IN  AAAA    2001:7fe::53
f.root-servers.net. 518345  IN  A   192.5.5.241
f.root-servers.net. 518345  IN  AAAA    2001:500:2f::f
h.root-servers.net. 518345  IN  A   128.63.2.53
h.root-servers.net. 518345  IN  AAAA    2001:500:1::803f:235
a.root-servers.net. 518345  IN  A   198.41.0.4
;; Received 508 bytes from 192.168.x.x#53(192.168.x.x) in 11363 ms


com.            172800  IN  NS  d.gtld-servers.net.
com.            172800  IN  NS  l.gtld-servers.net.
com.            172800  IN  NS  b.gtld-servers.net.
com.            172800  IN  NS  m.gtld-servers.net.
com.            172800  IN  NS  c.gtld-servers.net.
com.            172800  IN  NS  k.gtld-servers.net.
com.            172800  IN  NS  i.gtld-servers.net.
com.            172800  IN  NS  a.gtld-servers.net.
com.            172800  IN  NS  f.gtld-servers.net.
com.            172800  IN  NS  g.gtld-servers.net.

com.            172800  IN  NS  j.gtld-servers.net.
com.            172800  IN  NS  h.gtld-servers.net.
com.            172800  IN  NS  e.gtld-servers.net.
a.gtld-servers.net. 172800  IN  A   192.5.6.30
b.gtld-servers.net. 172800  IN  A   192.33.14.30
c.gtld-servers.net. 172800  IN  A   192.26.92.30
d.gtld-servers.net. 172800  IN  A   192.31.80.30
e.gtld-servers.net. 172800  IN  A   192.12.94.30
f.gtld-servers.net. 172800  IN  A   192.35.51.30
g.gtld-servers.net. 172800  IN  A   192.42.93.30

h.gtld-servers.net. 172800  IN  A   192.54.112.30
i.gtld-servers.net. 172800  IN  A   192.43.172.30
j.gtld-servers.net. 172800  IN  A   192.48.79.30
k.gtld-servers.net. 172800  IN  A   192.52.178.30
l.gtld-servers.net. 172800  IN  A   192.41.162.30
m.gtld-servers.net. 172800  IN  A   192.55.83.30
a.gtld-servers.net. 172800  IN  AAAA    2001:503:a83e::2:30
;; Received 489 bytes from 202.12.27.33#53(202.12.27.33) in 4335 ms

dhl.com.        172800  IN  NS  ns4.dhl.com.

dhl.com.        172800  IN  NS  ns6.dhl.com.
ns4.dhl.com.        172800  IN  A   165.72.192.16
ns6.dhl.com.        172800  IN  A   199.40.254.166
dig: couldn't get address for 'ns4.dhl.com': no more

Output from dig +tcp www.redhat.com

dig +tcp www.redhat.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +tcp www.redhat.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62110
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 8, ADDITIONAL: 8

;; QUESTION SECTION:
;www.redhat.com.            IN  A

;; ANSWER SECTION:

www.redhat.com.     11  IN  CNAME   wildcard.redhat.com.edgekey.net.
wildcard.redhat.com.edgekey.net. 20484 IN CNAME wildcard.redhat.com.edgekey.net.globalredir.akadns.net.
wildcard.redhat.com.edgekey.net.globalredir.akadns.net. 2485 IN CNAME e1890.b.akamaiedge.net.
e1890.b.akamaiedge.net. 20  IN  A   172.229.164.152

;; AUTHORITY SECTION:
b.akamaiedge.net.   2874    IN  NS  n2b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n3b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n4b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n1b.akamaiedge.net.

b.akamaiedge.net.   2874    IN  NS  n7b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n5b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n6b.akamaiedge.net.
b.akamaiedge.net.   2874    IN  NS  n0b.akamaiedge.net.

;; ADDITIONAL SECTION:
n5b.akamaiedge.net. 6917    IN  A   201.144.215.107
n1b.akamaiedge.net. 4917    IN  A   23.61.206.74
n6b.akamaiedge.net. 2917    IN  A   201.144.215.108
n2b.akamaiedge.net. 6917    IN  A   192.204.11.244

n4b.akamaiedge.net. 4917    IN  A   201.144.215.110
n7b.akamaiedge.net. 4917    IN  A   201.144.215.113
n0b.akamaiedge.net. 2917    IN  A   23.61.206.68
n3b.akamaiedge.net. 2917    IN  A   201.144.215.114

;; Query time: 132 msec
;; SERVER: 192.168.x.x#53(192.168.x.x)
;; WHEN: Thu Dec  4 16:03:03 2014
;; MSG SIZE  rcvd: 463

Other tests:

traceroute -U -p 53 165.72.192.16
traceroute to 165.72.192.16 (165.72.192.16), 30 hops max, 60 byte packets
 1  192.168.17.10 (192.168.17.10)  0.724 ms  0.718 ms  0.681 ms
 2  168.243.205.74 (168.243.205.74)  4.188 ms  4.677 ms  3.945 ms
 3  172.26.64.21 (172.26.64.21)  179.831 ms  180.282 ms  181.633 ms
 4  172.24.0.13 (172.24.0.13)  182.433 ms  182.585 ms  179.870 ms
 5  172.24.0.9 (172.24.0.9)  180.654 ms  183.023 ms  180.876 ms

 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *

And:

ping 165.72.192.16

PING 165.72.192.16 (165.72.192.16) 56(84) bytes of data.
64 bytes from 165.72.192.16: icmp_seq=1 ttl=240 time=356 ms
64 bytes from 165.72.192.16: icmp_seq=2 ttl=240 time=325 ms
64 bytes from 165.72.192.16: icmp_seq=3 ttl=240 time=291 ms
64 bytes from 165.72.192.16: icmp_seq=4 ttl=240 time=260 ms

traceroute -U -p 53 199.40.254.166
traceroute to 199.40.254.166 (199.40.254.166), 30 hops max, 60 byte packets
 1  192.168.17.10 (192.168.17.10)  0.710 ms  0.683 ms  0.764 ms
 2  168.243.205.74 (168.243.205.74)  3.136 ms  3.875 ms  4.191 ms

 3  172.26.64.21 (172.26.64.21)  19.367 ms  18.988 ms  19.698 ms
 4  172.24.0.177 (172.24.0.177)  4.657 ms  6.608 ms  7.088 ms
 5  172.24.0.9 (172.24.0.9)  5.126 ms  7.412 ms  5.518 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *

ping 199.40.254.166
PING 199.40.254.166 (199.40.254.166) 56(84) bytes of data.

64 bytes from 199.40.254.166: icmp_seq=1 ttl=237 time=287 ms
64 bytes from 199.40.254.166: icmp_seq=2 ttl=237 time=280 ms
64 bytes from 199.40.254.166: icmp_seq=3 ttl=237 time=286 ms

UPDATE - 5 Dic 2014

Well, I have installed bind in another server on the same subnet, same OS version and same bind release, and it works just fine!!!, the only thing that changes is the IP address and I can't change that to test because the server with the problem is in production... so I think the IDS theory that Andrew sugests is true... I'll talk to our ISP and investigate our external IP if its blacklisted and post how it goes...

UPDATE - 6 Dic 2014

The server's IP address is not listed in any black list that I checked on the Internet...

reverse dns - Hotmail wants me to modify my SPF record

This is my current SPF record. It works well with gmail and yahoo but hotmail was ignoring. So i used their submit system and now they replied me with the text below.

This is my current spf

v=spf1 a mx ptr ~all

Now this is hotmail message

We have successfully added your domain to the Sender ID program. This may take up to 2 business days to be fully replicated in our systems. If you have any questions regarding this please let me know.

We reviewed your SPF record and note that it includes the "ptr" or reverse DNS lookup mechanism. The specification for SPF records (RFC 4408) discourages use of "ptr" for performance and reliability reasons. This is especially important for Windows Live Mail, Hotmail and other large ISPs as a result of the very high volume of mail we receive each day. We highly recommend you remove the "ptr" mechanism from your SPF record and, if necessary, replace it with other SPF mechanisms that do not require a reverse DNS lookup, such as "a", "mx", "ip4" and "include." This will help ensure that Sender ID validation is performed as accurately as possible, maximizing your email deliverability while protecting your domain from spoofing.

So my question is simple. What is the corrected way of it to make it as hotmail wants. Thank you.

Hotmail received email from my server headers

Authentication-Results: hotmail.com; 
sender-id=temperror (sender IP is 85.17.154.139) 
header.from=noreply@monstermmorpg.com; 
dkim=pass 
header.d=monstermmorpg.com; 
x-hmca=pass
X-Message-Status: n:0:n

X-SID-PRA: MonsterMMORPG 

Answer

In your case, the same string you're using minus the ptr should do the trick.

v=spf1 a mx ~all

amazon web services - AWS ECS: Unable to place task

I am trying to set up an AWS service with autoscaling. I have created a cluster with an application loadbalancer and created a task using a docker image that should be open on port 8080 for use. I have created a service based on that task for which I have set minimum and desired number of instances to 1 and maximum to 10, and created rules for scaling up and down. However, no new instances are created and all I get in the list of events at regular intervals is:

service microrecieverservice was unable to place a task because no container instance met all of its requirements. The closest matching container-instance 97d97ce9-967d-49ad-83ad-f4f904aae1f6 is already using a port required by your task. For more information, see the Troubleshooting section.

I have not been able to find anything relevant in the troubleshooting section. I have been able to manually add another instance to the cluster, but with no change in the events given. I could ssh into this instance, and there were two docker images: one was amazon-ecs-agent:latest and the other was my task definition. At this point I tried sending a REST request to the server to see if it would go through but got Connection Refused. At about the same time, the docker image restarted.

The container is not running anything else that would use port 8080 and when I do netstat -lntp the process using port 8080 is that of the my docker image.

git, nagios and hooks, corrupted git repo

Background

We're using nagios to monitor our infrastructure. We don't have the nagios configs under version control at the moment, and there are two of us that manage nagios configuration. As such, I'm working to get our nagios config into a central git repo, using some hooks to do syntax checking and then if the configs look good, make them "active". I'm using this guy's post as a starting point.

The general workflow I'm trying to implement is:

1. Edit local git repo of nagios config. Add edited files, commit locally.


2. git push origin master to the remote repo.


3. Push is intercepted by the pre-receive hook, which takes the files, moves them to a temporary directory on the server, and runs them through the nagios syntax checker.


4. If the syntax checker passes, accept the push, then use the post-commit hook to git pull the new code into the live nagios configuration directory and then restart nagios.


5. If the syntax checker fails, reject the push, showing the nagios syntax error to the user.

I'm running into an odd behavior, though, when I reject a git push due to syntax errors in the nagios config. What I expect to happen is that if I reject the hook, the attempted push should leave the repository just how it was, untouched. That doesn't appear to be the case, though. Below are the details of what I'm seeing:

Problem

I edit the nagios config locally, intentionally including a syntax error, add, then commit locally:

host:nagios erik$ vi nagios.cfg
host:nagios erik$ git add nagios.cfg
host:nagios erik$ git commit -m "syntax error"

[master da71aed] syntax error
 1 files changed, 1 insertions(+), 0 deletions(-)

Now I push those changes to the master repo. This will be rejected due to the syntax error:

host:nagios erik$ git push origin master
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.

Writing objects: 100% (3/3), 12.74 KiB, done.
Total 3 (delta 1), reused 2 (delta 1)
remote: Previous HEAD position was 3ddc880... removed syntax error
remote: HEAD is now at da71aed... syntax error
remote: Nagios Config Check Exit Status: 254
remote: Your configs did not parse correctly, there was an error. Output follows.
remote:
remote: Nagios Core 3.2.3
remote: Copyright (c) 2009-2010 Nagios Core Development Team and Community Contributors
remote: Copyright (c) 1999-2009 Ethan Galstad

remote: Last Modified: 10-03-2010
remote: License: GPL
remote:
remote: Website: http://www.nagios.org
remote: Reading configuration data...
remote: Error in configuration file '/tmp/nagiosworkdir/nagios.cfg' - Line 23 (NULL value)
remote:    Error processing main config file!
remote:
remote:
remote:

remote: ***> One or more problems was encountered while processing the config files...
remote:
remote:      Check your configuration file(s) to ensure that they contain valid
remote:      directives and data defintions.  If you are upgrading from a previous
remote:      version of Nagios, you should be aware that some variables/definitions
remote:      may have been removed or modified in this version.  Make sure to read
remote:      the HTML documentation regarding the config files, as well as the
remote:      'Whats New' section to find out what has changed.
remote:
To git@remote-server.example.com:nagios

 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@remote-server.example.com:nagios'

This shouldn't have touched the remote repo, but it did. If I change to another local temp directory and try to clone the repo, I get:

host:temp erik$ git clone git@remote-server.example.com:nagios
Cloning into nagios...
remote: Counting objects: 30, done.
remote: Compressing objects: 100% (29/29), done.

remote: Total 30 (delta 12), reused 0 (delta 0)
Receiving objects: 100% (30/30), 29.81 KiB, done.
Resolving deltas: 100% (12/12), done.
error: Trying to write ref HEAD with nonexistant object da71aedfde2e0469288acd9e45bb8b57a6e5a7b3
fatal: Cannot update the ref 'HEAD'.

Now I go back to the original work directory, fix the syntax error, add, commit, and push:

host:nagios erik$ vi nagios.cfg

host:nagios erik$ git add nagios.cfg
host:nagios erik$ git commit -m "removing syntax error, push should succeed this time"
[master f147ded] removing syntax error, push should succeed this time
 1 files changed, 0 insertions(+), 2 deletions(-)
host:nagios erik$ git push origin master
Counting objects: 6, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 487 bytes, done.
Total 4 (delta 2), reused 0 (delta 0)

remote: Previous HEAD position was 4c80d45... syntax error
remote: HEAD is now at f147ded... removing syntax error, push should succeed this time
remote: Nagios Config Check Exit Status: 0
remote: Your configs look good and parsed correctly.
To git@remote-server.example.com:nagios
   3ddc880..f147ded  master -> master

At this point, the repository is fine, and I'm able to change to a temporary directory and clone the repo again:

host:temp erik$ git clone git@remote-server.example.com:nagios
Cloning into nagios...
remote: Counting objects: 34, done.
remote: Compressing objects: 100% (33/33), done.
remote: Total 34 (delta 14), reused 0 (delta 0)
Receiving objects: 100% (34/34), 30.22 KiB, done.
Resolving deltas: 100% (14/14), done.

Here is the pre-receive hook I'm using.

I'm using git v1.7.5.4 on the client, and v1.7.2.3 on the server.

So, to the question: why is the repository being left in an inconsistent state when I reject the push? Is something awry with my git hook or perhaps my understanding of git is lacking?

Answer

You're doing:

export GIT_WORK_TREE=/tmp/nagiosworkdir
/usr/bin/git checkout -f $NEW_SHA1

in your hook. Although it's not touching your usual working-copy it is updating references in the git-dir (specifically the HEAD reference), as shown in your error:

...
remote: HEAD is now at da71aed... syntax error
...

Your hook is doing exit 1 to reject the update, but it's not (re)resetting the HEAD reference after failure.

I think you need to update the failure branch in your hook like so:

...
if [ "$NAGIOS_CHECK_STATUS" -ne 0 ]
   then
   echo "Your configs did not parse correctly, there was an error. Output follows."
   cat $GIT_WORK_TREE/check.out
   /usr/bin/git reset --hard $OLD_SHA1    # <-- Add This
   exit 1
else

   ...

networking - Dedicated IP for webserver through pfsense and a draytek 2860

We have a leased line which has 8 public IP addresses. This connects to our Draytek 2860 router.

From here we have an internal network which we use for our office computers and another network for our datacentre (which hosts a few customer servers).

We have installed a pfsense firewall which we'd like to use to manage NAT and routing for the datacentre

I would like to create a web server in our datacentre, behind the pfsense firewall on one of the free public static IP addresses I have been assigned. How do I configure this?

I'm thinking I should set up DMZ's for all the IP's I'd like to use within the datacentre on the draytek, then set it up as a 1:1 NAT on my pfsense, but not sure.

I need all outgoing traffic on my internal network, datacentre network and webserver to come from their own dedicated public IP addresses

Answer

You really shouldn't put a public-facing webserver behind even one NAT (a busy webserver is a really quick way to fill up the NAT connections table).

If you're using the Draytek in NAT mode rather than Routing mode, its DMZ feature is really a 1:1 NAT mapping from one of your public IP addresses to a specific internal IP address.

I don't have enough detail of your specific requirements, but assuming a technical solution would be acceptable this is possibly how I would approach the problem.

               +---------+       +---------+       +------------+
               |         | <===> | pfSense | <===> | Datacentre |
               |         |       +---------+       +------------+
               #         |
Internet <===> # Draytek |       +----------------+

               #         | <===> |                |
               |         | <===> | Office network |
               |         | <===> |                |
               +---------+       +----------------+

I've used # to denote the NAT boundary.

1. Set up a separate LAN on the Draytek for the Datacentre.



2. Use the pfSense firewall in Routing or Transparent mode rather than NAT (I'm not familiar with pfSense specifically, but I assume this is possible).


3. Have the Draytek manage routing between the Datacentre and Office network, using its internal firewall as appropriate.


4. Punch as many of the public IPs as necessary through to the Datacentre LAN, either with specific Port Forwarding or as a DMZ host.

---

If I've misinterpreted your network topology, please do update your question with a suitable schematic. For example, you might mean your leased line terminates on the Draytek and on pfSense, so they're "side by side". But I didn't think that's what you meant.