[Comm] глючит ядро ovz-el
Slava Dubrovskiy
slava на tangramltd.com
Пн Июл 4 19:20:25 UTC 2011
Здравствуйте уважаемая редакция.
Серверов с ядром ovz-el на серверах становится все больше и больше. И
багов соответственно.
Есть один сервер на котором периодически (1-2 раза в неделю) происходит
ребут в результате kernel panic.
Перед ребутом LA уходит в космос - выше 200.
Ядро 2.6.32-ovz-el-alt24, бранч t6.
Стоит 2 SATA диска в raid1 на mdadm на котором установлена система и все
служебные VPS
И 2 SAS диска из которых собран raid1 и с него работает одна рабочая VPS.
Диски по smart живые, ошибок нет.
В логе вот такое перед ребутом вижу:
Jul 4 21:05:51 ua83 kernel: [293760.725062] INFO: task flush-253:1:1294
blocked for more than 120
seconds.
Jul 4 21:05:59 ua83 kernel: [293760.725069] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this
message.
Jul 4 21:06:26 ua83 kernel: [293760.725074] flush-253:1 D
ffff8801dcb82c90 0 1294 2
0x00000000
Jul 4 21:07:33 ua83 kernel: [293760.725083] ffff8801d5a0ba10
0000000000000046 0000000000000000
ffffffffa02cc12c
Jul 4 21:07:33 ua83 kernel: [293760.725091] ffff88013696e5a8
ffffffff811b3e70 ffffffffa00d3c40
00000001117b8ec4
Jul 4 21:07:33 ua83 kernel: [293760.725098] ffff8801dcb83270
ffff8801d5a0bfd8 000000000000f7c8
ffff8801dcb83270
Jul 4 21:07:33 ua83 kernel: [293760.725106] Call
Trace:
Jul 4 21:07:33 ua83 kernel: [293760.725132] [<ffffffffa02cc12c>] ?
dm_table_unplug_all+0x5c/0xd0
[dm_mod]
Jul 4 21:07:33 ua83 kernel: [293760.725155] [<ffffffff811b3e70>] ?
end_buffer_async_write+0x0/0x190
Jul 4 21:07:33 ua83 kernel: [293760.725185] [<ffffffffa00d3c40>] ?
ext4_bh_delay_or_unwritten+0x0/0x30
[ext4]
Jul 4 21:07:33 ua83 kernel: [293760.725195] [<ffffffff813ff403>]
io_schedule+0xa3/0x110
Jul 4 21:07:33 ua83 kernel: [293760.725202] [<ffffffff8111a6e0>] ?
sync_page+0x0/0x50
Jul 4 21:07:45 ua83 kernel: [293760.725207] [<ffffffff8111a71d>]
sync_page+0x3d/0x50
Jul 4 21:08:27 ua83 kernel: [293760.725212] [<ffffffff813ffc8f>]
__wait_on_bit+0x5f/0x90
Jul 4 21:08:29 ua83 kernel: [293760.725218] [<ffffffff8111a8d3>]
wait_on_page_bit+0x73/0x80
Jul 4 21:08:38 ua83 kernel: [293760.725225] [<ffffffff81091a30>] ?
wake_bit_function+0x0/0x50
Jul 4 21:08:38 ua83 kernel: [293760.725234] [<ffffffff81131355>] ?
pagevec_lookup_tag+0x25/0x40
Jul 4 21:08:38 ua83 kernel: [293760.725239] [<ffffffff8111ac9b>]
wait_on_page_writeback_range+0xfb/0x190
Jul 4 21:08:38 ua83 kernel: [293760.725245] [<ffffffff8111ad5f>]
filemap_fdatawait+0x2f/0x40
Jul 4 21:08:38 ua83 kernel: [293760.725252] [<ffffffff811ab418>]
writeback_single_inode+0x238/0x2e0
Jul 4 21:08:38 ua83 kernel: [293760.725258] [<ffffffff811ab706>]
writeback_sb_inodes+0xf6/0x1b0
Jul 4 21:08:38 ua83 kernel: [293760.725263] [<ffffffff811abae0>]
wb_writeback+0x170/0x400
Jul 4 21:08:38 ua83 kernel: [293760.725268] [<ffffffff813feb4c>] ?
thread_return+0x4e/0x862
Jul 4 21:08:38 ua83 kernel: [293760.725276] [<ffffffff8107c7ba>] ?
del_timer_sync+0x2a/0x40
Jul 4 21:08:38 ua83 kernel: [293760.725281] [<ffffffff811abe35>]
wb_do_writeback+0xc5/0x250
Jul 4 21:08:38 ua83 kernel: [293760.725286] [<ffffffff8107bd20>] ?
process_timeout+0x0/0x10
Jul 4 21:08:38 ua83 kernel: [293760.725292] [<ffffffff811ac027>]
bdi_writeback_task+0x67/0x1c0
Jul 4 21:08:38 ua83 kernel: [293760.725297] [<ffffffff810918c7>] ?
bit_waitqueue+0x17/0xc0
Jul 4 21:08:38 ua83 kernel: [293760.725305] [<ffffffff81141556>]
bdi_start_fn+0x86/0x100
Jul 4 21:08:38 ua83 kernel: [293760.725310] [<ffffffff811414d0>] ?
bdi_start_fn+0x0/0x100
Jul 4 21:08:38 ua83 kernel: [293760.725315] [<ffffffff81091680>]
kthread+0xc0/0xe0
Jul 4 21:08:38 ua83 kernel: [293760.725321] [<ffffffff8100c2ca>]
child_rip+0xa/0x20
Jul 4 21:08:38 ua83 kernel: [293760.725326] [<ffffffff810915c0>] ?
kthread+0x0/0xe0
Jul 4 21:08:38 ua83 kernel: [293760.725331] [<ffffffff8100c2c0>] ?
child_rip+0x0/0x20
Jul 4 21:08:38 ua83 kernel: [293760.725336] INFO: task jbd2/dm-1-8:1306
blocked for more than 120
seconds.
Jul 4 21:09:58 ua83 kernel: [293760.725340] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this
message.
Jul 4 21:10:37 ua83 kernel: [293760.725344] jbd2/dm-1-8 D
ffff8801da9ea640 0 1306 2
0x00000000
Jul 4 21:10:38 ua83 kernel: [293760.725351] ffff8801d4951b00
0000000000000046 ffff8801d4951ac0
ffffffffa02cc12c
Jul 4 21:10:38 ua83 kernel: [293760.725358] ffff880119e48808
ffff880119eddb40 ffff8801d4951ab0
ffffffff8109c569
Jul 4 21:10:38 ua83 kernel: [293760.725365] ffff8801da9eac20
ffff8801d4951fd8 000000000000f7c8
ffff8801da9eac20
Jul 4 21:10:38 ua83 kernel: [293760.725372] Call
Trace:
Jul 4 21:10:38 ua83 kernel: [293760.725381] [<ffffffffa02cc12c>] ?
dm_table_unplug_all+0x5c/0xd0
[dm_mod]
Jul 4 21:10:38 ua83 kernel: [293760.725388] [<ffffffff8109c569>] ?
ktime_get_ts+0xa9/0xe0
Jul 4 21:10:38 ua83 kernel: [293760.725394] [<ffffffff8109c569>] ?
ktime_get_ts+0xa9/0xe0
Jul 4 21:10:38 ua83 kernel: [293760.725399] [<ffffffff813ff403>]
io_schedule+0xa3/0x110
Jul 4 21:10:38 ua83 kernel: [293760.725404] [<ffffffff8111a6e0>] ?
sync_page+0x0/0x50
Jul 4 21:10:38 ua83 kernel: [293760.725409] [<ffffffff8111a71d>]
sync_page+0x3d/0x50
Jul 4 21:10:38 ua83 kernel: [293760.725414] [<ffffffff813ffc8f>]
__wait_on_bit+0x5f/0x90
Jul 4 21:10:38 ua83 kernel: [293760.725419] [<ffffffff8111a8d3>]
wait_on_page_bit+0x73/0x80
Jul 4 21:10:38 ua83 kernel: [293760.725425] [<ffffffff81091a30>] ?
wake_bit_function+0x0/0x50
Jul 4 21:10:38 ua83 kernel: [293760.725430] [<ffffffff81131355>] ?
pagevec_lookup_tag+0x25/0x40
Jul 4 21:10:38 ua83 kernel: [293760.725436] [<ffffffff8111ac9b>]
wait_on_page_writeback_range+0xfb/0x190
Jul 4 21:10:38 ua83 kernel: [293760.725443] [<ffffffff811b7c9b>] ?
bio_alloc_bioset+0x5b/0xf0
Jul 4 21:10:38 ua83 kernel: [293760.725449] [<ffffffff8111ad5f>]
filemap_fdatawait+0x2f/0x40
Jul 4 21:10:38 ua83 kernel: [293760.725462] [<ffffffffa00a7ec0>]
jbd2_journal_commit_transaction+0x7f0/0x1490
[jbd2]
Jul 4 21:10:38 ua83 kernel: [293760.725469] [<ffffffff810919f0>] ?
autoremove_wake_function+0x0/0x40
Jul 4 21:10:38 ua83 kernel: [293760.725480] [<ffffffffa00ad978>]
kjournald2+0xb8/0x220
[jbd2]
Jul 4 21:10:38 ua83 kernel: [293760.725486] [<ffffffff810919f0>] ?
autoremove_wake_function+0x0/0x40
Jul 4 21:10:38 ua83 kernel: [293760.725496] [<ffffffffa00ad8c0>] ?
kjournald2+0x0/0x220
[jbd2]
Jul 4 21:10:38 ua83 kernel: [293760.725501] [<ffffffff81091680>]
kthread+0xc0/0xe0
Jul 4 21:10:38 ua83 kernel: [293760.725506] [<ffffffff8100c2ca>]
child_rip+0xa/0x20
Jul 4 21:10:38 ua83 kernel: [293760.725511] [<ffffffff810915c0>] ?
kthread+0x0/0xe0
Jul 4 21:10:38 ua83 kernel: [293760.725516] [<ffffffff8100c2c0>] ?
child_rip+0x0/0x20
и т.д.
Подскажите что делать? Куда еще можно посмотреть?
--
WBR,
Dubrovskiy Viacheslav
----------- следующая часть -----------
Было удалено вложение не в текстовом формате...
Имя : smime.p7s
Тип : application/pkcs7-signature
Размер : 6229 байтов
Описание: S/MIME Cryptographic Signature
Url : <http://lists.altlinux.org/pipermail/community/attachments/20110704/21ac8a1d/attachment-0001.bin>
Подробная информация о списке рассылки community