I set up a new Server with CentOS 6.4 final as successor for an old mysql server and I'm facing some problems with it. From time to time mysql connections are being disconnected. Furthermore the transfer of the large backup tar files to a ftp-storage sometimes fails. Both not reproducible.
While analyzing I found some strange messages that I cannot interpret in /var/log/messages.
Mar 30 13:09:24 s16838172 kernel: swapper: page allocation failure. order:1, mode:0x20
Mar 30 13:09:24 s16838172 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-358.0.1.el6.x86_64 #1
Mar 30 13:09:24 s16838172 kernel: Call Trace:
Mar 30 13:09:24 s16838172 kernel: [] ? __alloc_pages_nodemask+0x757/0x8d0
Mar 30 13:09:24 s16838172 kernel: [] ? kmem_getpages+0x62/0x170
Mar 30 13:09:24 s16838172 kernel: [] ? fallback_alloc+0x1ba/0x270
Mar 30 13:09:24 s16838172 kernel: [] ? cache_grow+0x2cf/0x320
Mar 30 13:09:24 s16838172 kernel: [] ? ____cache_alloc_node+0x99/0x160
Mar 30 13:09:24 s16838172 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Mar 30 13:09:24 s16838172 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Mar 30 13:09:24 s16838172 kernel: [] ? sk_clone+0x22/0x2e0
Mar 30 13:09:24 s16838172 kernel: [] ? inet_csk_clone+0x16/0xd0
Mar 30 13:09:24 s16838172 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Mar 30 13:09:24 s16838172 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Mar 30 13:09:24 s16838172 kernel: [] ? tcp_check_req+0x226/0x460
Mar 30 13:09:24 s16838172 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Mar 30 13:09:24 s16838172 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Mar 30 13:09:24 s16838172 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Mar 30 13:09:24 s16838172 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Mar 30 13:09:24 s16838172 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Mar 30 13:09:24 s16838172 kernel: [] ? ip_local_deliver+0x98/0xa0
Mar 30 13:09:24 s16838172 kernel: [] ? ip_rcv_finish+0x12d/0x440
Mar 30 13:09:24 s16838172 kernel: [] ? ip_rcv+0x275/0x350
Mar 30 13:09:24 s16838172 kernel: [] ? __netif_receive_skb+0x4ab/0x750
Mar 30 13:09:24 s16838172 kernel: [] ? netif_receive_skb+0x58/0x60
Mar 30 13:09:24 s16838172 kernel: [] ? napi_skb_finish+0x50/0x70
Mar 30 13:09:24 s16838172 kernel: [] ? napi_gro_receive+0x39/0x50
Mar 30 13:09:24 s16838172 kernel: [] ? e1000_receive_skb+0x5b/0x90 [e1000e]
Mar 30 13:09:24 s16838172 kernel: [] ? e1000_clean_rx_irq+0x241/0x4c0 [e1000e]
Mar 30 13:09:24 s16838172 kernel: [] ? e1000e_poll+0xbd/0x380 [e1000e]
Mar 30 13:09:24 s16838172 kernel: [] ? e1000_put_txbuf+0x6a/0xa0 [e1000e]
Mar 30 13:09:24 s16838172 kernel: [] ? net_rx_action+0x103/0x2f0
Mar 30 13:09:24 s16838172 kernel: [] ? hrtimer_get_next_event+0xc3/0x100
Mar 30 13:09:24 s16838172 kernel: [] ? __do_softirq+0xc1/0x1e0
Mar 30 13:09:24 s16838172 kernel: [] ? handle_IRQ_event+0x60/0x170
Mar 30 13:09:24 s16838172 kernel: [] ? call_softirq+0x1c/0x30
Mar 30 13:09:24 s16838172 kernel: [] ? do_softirq+0x65/0xa0
Mar 30 13:09:24 s16838172 kernel: [] ? irq_exit+0x85/0x90
Mar 30 13:09:24 s16838172 kernel: [] ? do_IRQ+0x75/0xf0
Mar 30 13:09:24 s16838172 kernel: [] ? ret_from_intr+0x0/0x11
Mar 30 13:09:24 s16838172 kernel: [] ? intel_idle+0xde/0x170
Mar 30 13:09:24 s16838172 kernel: [] ? intel_idle+0xc1/0x170
Mar 30 13:09:24 s16838172 kernel: [] ? cpuidle_idle_call+0xa7/0x140
Mar 30 13:09:24 s16838172 kernel: [] ? cpu_idle+0xb6/0x110
Mar 30 13:09:24 s16838172 kernel: [] ? rest_init+0x7a/0x80
Mar 30 13:09:24 s16838172 kernel: [] ? start_kernel+0x424/0x430
Mar 30 13:09:24 s16838172 kernel: [] ? x86_64_start_reservations+0x125/0x129
Mar 30 13:09:24 s16838172 kernel: [] ? x86_64_start_kernel+0xfa/0x109
This kind of message blocks appearing about 2-10 times in 5 minutes, after that they are gone for a few hours.
Can somebody help me with that? I hope its not a hardware problem.
Update:
Seems to be reproducible by transferring big files over network (backups to ftp-storage). The ftp upload fails/aborts after a few GB and the stuff above appears in /var/log/messages
Thanks!
No comments:
Post a Comment