Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bmcd crash if flashing modules several times #178

Open
camrossi opened this issue Jan 26, 2024 · 1 comment
Open

bmcd crash if flashing modules several times #178

camrossi opened this issue Jan 26, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@camrossi
Copy link

I have been experimenting with Talos on RK-1 and had to re flesh them a few times.
After a few flashing operations (say 6+) bmcd crashed due to OOM.

As a work around I just restarted the bmcd (/etc/init.d/S94bmcd restart) and all works fine

Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486229] [<c010e390>] (unwind_backtrace) from [<c010a850>] (show_stack+0x10/0x14)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486247] [<c010a850>] (show_stack) from [<c05e432c>] (dump_stack+0x78/0x94)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486264] [<c05e432c>] (dump_stack) from [<c0196118>] (dump_header+0x68/0x288)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486278] [<c0196118>] (dump_header) from [<c0195634>] (oom_kill_process+0x84/0x2b4)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486291] [<c0195634>] (oom_kill_process) from [<c0195fe4>] (out_of_memory+0x330/0x388)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486307] [<c0195fe4>] (out_of_memory) from [<c01c4d2c>] (__alloc_pages_nodemask+0x7d0/0x8e4)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486321] [<c01c4d2c>] (__alloc_pages_nodemask) from [<c0192850>] (pagecache_get_page+0xf8/0x1a0)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486334] [<c0192850>] (pagecache_get_page) from [<c0192d38>] (filemap_fault+0x1b8/0x408)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486348] [<c0192d38>] (filemap_fault) from [<c01b0304>] (__do_fault+0x50/0xa8)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486363] [<c01b0304>] (__do_fault) from [<c01b316c>] (handle_mm_fault+0x534/0x788)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486377] [<c01b316c>] (handle_mm_fault) from [<c010f204>] (do_page_fault+0x19c/0x2e8)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486392] [<c010f204>] (do_page_fault) from [<c010f540>] (do_PrefetchAbort+0x38/0x88)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486407] [<c010f540>] (do_PrefetchAbort) from [<c0102808>] (ret_from_exception+0x0/0x18)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486413] Exception stack(0xc5aabfb0 to 0xc5aabff8)
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486423] bfa0:                                     b5f2e0b8 b5cfd670 b5cfd670 000000cc
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486434] bfc0: b5cfd630 b5f2e0b8 00000000 b5f2e0a0 b5cfd6a0 b5cfd6a8 00008001 b5cfd8f4
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486444] bfe0: b5f1edf0 b5cfd618 004fb4ac 00500510 20070010 ffffffff
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486461] Mem-Info:
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486480] active_anon:18755 inactive_anon:22 isolated_anon:0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486480]  active_file:46 inactive_file:76 isolated_file:0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486480]  unevictable:0 dirty:1 writeback:0 unstable:0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486480]  slab_reclaimable:1129 slab_unreclaimable:4512
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486480]  mapped:31 shmem:23 pagetables:146 bounce:0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486480]  free:1272 free_pcp:12 free_cma:0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486493] Node 0 active_anon:75020kB inactive_anon:88kB active_file:184kB inactive_file:304kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:124kB dirty:4kB writeback:0kB shmem:92kB writeback_tmp:0kB unstable:
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486511] Normal free:5088kB min:5436kB low:5768kB high:6100kB active_anon:75020kB inactive_anon:88kB active_file:144kB inactive_file:84kB unevictable:0kB writepending:4kB present:131072kB managed:113768kB mlocked:0kB kerne
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486515] lowmem_reserve[]: 0 0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486523] Normal: 7*4kB (U) 76*8kB (UE) 271*16kB (UE) 1*32kB (U) 1*64kB (E) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 5068kB
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486559] Free pages count per migrate typeat order:     0      1      2      3      4      5      6      7      8      9     10
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486582] zone   Normal, type    Unmovable      7     73    259      1      0      0      0      0      0      0      0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486756] zone   Normal, type      Movable      0      0      0      0      0      0      0      0      0      0      0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486777] zone   Normal, type  Reclaimable      0      3     12      0      1      0      0      0      0      0      0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486821] zone   Normal, type          CMA      0      0      0      0      0      0      0      0      0      0      0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486853] zone   Normal, type   HighAtomic      0      0      0      0      0      0      0      0      0      0      0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486874] zone   Normal, type      Isolate      0      0      0      0      0      0      0      0      0      0      0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486895] 153 total pagecache pages
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486900] 0 pages in swap cache
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486906] Swap cache stats: add 0, delete 0, find 0/0
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486909] Free swap  = 0kB
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486913] Total swap = 0kB
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486917] 32768 pages RAM
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486920] 0 pages HighMem/MovableOnly
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486924] 4326 pages reserved
Jan 26 03:28:45 turingpi kern.warn kernel: [79860.486927] 2048 pages cma reserved
Jan 26 03:28:45 turingpi kern.info kernel: [79860.486944] Tasks state (memory values in pages):
Jan 26 03:28:45 turingpi kern.info kernel: [79860.486948] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Jan 26 03:28:45 turingpi kern.info kernel: [79860.486967] [   1045]     0  1045      684       22    10240        0             0 syslogd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.486986] [   1049]     0  1049      684       22    10240        0             0 klogd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.486996] [   1065]     0  1065      716       67    10240        0             0 mdev
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487013] [   1118]     0  1118      684       21    10240        0             0 udhcpc
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487023] [   1123]   101  1123     2866       54    12288        0             0 chronyd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487032] [   1129]   100  1129      705       91    10240        0             0 avahi-daemon
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487042] [   1130]   100  1130      648       36    10240        0             0 avahi-daemon
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487051] [   1134]     0  1134     1619      145    12288        0         -1000 sshd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487061] [   1138]     0  1138      582       62    10240        0             0 collectd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487070] [   1151]     0  1151    31957    17524   129024        0             0 bmcd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487080] [   1155]     0  1155      684       12     8192        0             0 getty
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487091] [   1590]     0  1590     1716      233    12288        0             0 sshd
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487101] [   1594]     0  1594      684       34    10240        0             0 sh
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487114] [   1719]     0  1719     5656      394    26624        0             0 tpi
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487126] [   1745]     0  1745      684       21     8192        0             0 sh
Jan 26 03:28:45 turingpi kern.info kernel: [79860.487143] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),task=bmcd,pid=1151,uid=0
Jan 26 03:28:45 turingpi kern.err kernel: [79860.487212] Out of memory: Killed process 1151 (bmcd) total-vm:127828kB, anon-rss:70096kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:126kB oom_score_adj:0
Jan 26 03:28:45 turingpi kern.info kernel: [79860.525661] oom_reaper: reaped process 1151 (bmcd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
@camrossi camrossi added the bug Something isn't working label Jan 26, 2024
@svenrademakers
Copy link
Collaborator

related to #162

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants