Re: Unexplained Halt

#47175

jeremya
Member

I have been running the B11 system for several days now. On average it stays up for around a day before failing.

Monitoring on the serial port it appears to be running out of memory. I get a lot of messages such as below. In the end I have to restart it.

I did not get this type of failure in B10.

Sample logs:


syslogd invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Pid: 25989, comm: syslogd Tainted: P 2.6.25.17 #1
[] ? printk+0x18/0x21
[] oom_kill_process+0x75/0x1f0
[] ? badness+0x148/0x1c0
[] out_of_memory+0x19e/0x1e0
[] __alloc_pages+0x34f/0x380
[] __do_page_cache_readahead+0x111/0x220
[] do_page_cache_readahead+0x4b/0x70
[] filemap_fault+0x318/0x3f0
[] __do_fault+0x56/0x3a0
[] ? __wait_on_bit_lock+0x5c/0x70
[] ? sync_page+0x0/0x60
[] ? __lock_page+0x7d/0x90
[] handle_mm_fault+0xf4/0x590
[] do_page_fault+0x2b9/0x790
[] ? sys_sendto+0xa5/0xd0
[] ? copy_from_user+0x35/0x130
[] ? rw_copy_check_uvector+0x58/0x100
[] ? do_readv_writev+0x184/0x1c0
[] ? pipe_write+0x0/0x510
[] ? sys_recv+0x36/0x40
[] ? sys_socketcall+0x160/0x260
[] ? do_page_fault+0x0/0x790
[] error_code+0x72/0x78
[] ? pfkey_release+0xf0/0x100
=======================
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Normal per-cpu:
CPU 0: hi: 90, btch: 15 usd: 52
Active:25 inactive:397 dirty:0 writeback:90 unstable:0
free:734 slab:51510 mapped:15 pagetables:112 bounce:0
DMA free:1076kB min:124kB low:152kB high:184kB active:0kB inactive:16kB present:16256kB pages_scanned:27 all_unreclaimable? yes
lowmem_reserve[]: 0 238 238 238
Normal free:1860kB min:1908kB low:2384kB high:2860kB active:100kB inactive:1572kB present:243840kB pages_scanned:1690 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1076kB
Normal: 1*4kB 6*8kB 5*16kB 2*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1860kB
425 total pagecache pages
Swap cache: add 724956, delete 724814, find 705499/849580
Free swap = 126460kB
Total swap = 131064kB
Out of memory: kill process 23609 (wifi-manager) score 42 or a child
Killed process 23609 (wifi-manager)
syslogd invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Pid: 25989, comm: syslogd Tainted: P 2.6.25.17 #1
[] ? printk+0x18/0x21
[] oom_kill_process+0x75/0x1f0
[] ? badness+0x148/0x1c0
[] out_of_memory+0x19e/0x1e0
[] __alloc_pages+0x34f/0x380
[] __do_page_cache_readahead+0x111/0x220
[] do_page_cache_readahead+0x4b/0x70
[] filemap_fault+0x318/0x3f0
[] __do_fault+0x56/0x3a0
[] ? __wait_on_bit_lock+0x5c/0x70
[] ? sync_page+0x0/0x60
[] ? __lock_page+0x7d/0x90
[] handle_mm_fault+0xf4/0x590
[] do_page_fault+0x2b9/0x790
[] ? sys_sendto+0xa5/0xd0
[] ? copy_from_user+0x35/0x130
[] ? rw_copy_check_uvector+0x58/0x100
[] ? do_readv_writev+0x184/0x1c0
[] ? pipe_write+0x0/0x510
[] ? sys_recv+0x36/0x40
[] ? sys_socketcall+0x160/0x260
[] ? do_page_fault+0x0/0x790
[] error_code+0x72/0x78
[] ? pfkey_release+0xf0/0x100
=======================
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Normal per-cpu:
CPU 0: hi: 90, btch: 15 usd: 78
Active:6 inactive:398 dirty:0 writeback:89 unstable:0
free:734 slab:51510 mapped:18 pagetables:112 bounce:0
DMA free:1076kB min:124kB low:152kB high:184kB active:0kB inactive:16kB present:16256kB pages_scanned:60 all_unreclaimable? yes
lowmem_reserve[]: 0 238 238 238
Normal free:1860kB min:1908kB low:2384kB high:2860kB active:24kB inactive:1576kB present:243840kB pages_scanned:1786 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1076kB
Normal: 1*4kB 6*8kB 5*16kB 2*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1860kB
408 total pagecache pages
Swap cache: add 724956, delete 724827, find 705499/849582
Free swap = 126500kB
Total swap = 131064kB
Out of memory: kill process 23610 (lgwatcher) score 38 or a child
Killed process 23610 (lgwatcher)
wifi-manager invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Pid: 23768, comm: wifi-manager Tainted: P 2.6.25.17 #1
[] ? printk+0x18/0x21
[] oom_kill_process+0x75/0x1f0
[] ? badness+0x148/0x1c0
[] out_of_memory+0x19e/0x1e0
[] __alloc_pages+0x34f/0x380
[] ? swap_info_get+0x54/0xd0
[] __do_page_cache_readahead+0x111/0x220
[] do_page_cache_readahead+0x4b/0x70
[] filemap_fault+0x318/0x3f0
[] __do_fault+0x56/0x3a0
[] handle_mm_fault+0xf4/0x590
[] do_page_fault+0x2b9/0x790
[] ? do_notify_resume+0x494/0x810
[] ? task_new_fair+0x98/0x150
[] ? vfs_read+0x99/0x140
[] ? do_sync_read+0x0/0x110
[] ? do_page_fault+0x0/0x790
[] error_code+0x72/0x78
[] ? pfkey_release+0xf0/0x100
=======================
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Normal per-cpu:
CPU 0: hi: 90, btch: 15 usd: 53
Active:21 inactive:395 dirty:0 writeback:85 unstable:0
free:736 slab:51508 mapped:24 pagetables:103 bounce:0
DMA free:1076kB min:124kB low:152kB high:184kB active:0kB inactive:36kB present:16256kB pages_scanned:159 all_unreclaimable? yes
lowmem_reserve[]: 0 238 238 238
Normal free:1868kB min:1908kB low:2384kB high:2860kB active:84kB inactive:1544kB present:243840kB pages_scanned:1206 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1076kB
Normal: 1*4kB 7*8kB 5*16kB 2*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1868kB
422 total pagecache pages
Swap cache: add 724988, delete 724869, find 705508/849600
Free swap = 126524kB
Total swap = 131064kB
Out of memory: kill process 23525 (sleep) score 30 or a child
Killed process 23525 (sleep)
ath_mgtstart: discard, no xmit buf
ath_mgtstart: discard, no xmit buf
ath_mgtstart: discard, no xmit buf
ath_mgtstart: discard, no xmit buf
ath_mgtstart: discard, no xmit buf
Out of memory: kill process 23589 (sleep) score 30 or a child
Killed process 23589 (sleep)
/root/kerbynet.cgi/scripts/ddns2: line 65: 23589 Killed sleep 600
/root/kerbynet.cgi/scripts/chec

[/quote]