[ 1800.973963][ T12] ------------[ cut here ]------------ [ 1800.974350][ T12] conntrack cleanup blocked for 60s [ 1800.974352][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack], CPU#3: kworker/u16:0/12 [ 1800.974849][ T12] Modules linked in: nf_flow_table(+) cls_flower sch_ingress xt_conntrack nft_chain_nat xt_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 geneve bonding psample macsec vxlan ip6_gre ip_gre gre cls_u32 sch_htb chacha libchacha chacha20poly1305 libpoly1305 tls xfrm_user xt_policy ip6t_REJECT nf_reject_ipv6 nft_compat nf_tables [last unloaded: netdevsim] [ 1800.975849][ T12] CPU: 3 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.19.0-rc1-virtme #1 PREEMPT(full) [ 1800.976138][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1800.976323][ T12] Workqueue: netns cleanup_net [ 1800.976492][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 1800.976694][ T12] Code: 00 48 8b 54 24 18 48 8b 05 b3 38 80 f1 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 0e 75 f0 e9 25 fd ff ff 48 8d 3d 46 06 e4 ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df [ 1800.977127][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297 [ 1800.977289][ T12] RAX: 000000010016e56d RBX: ffa00000000c7af8 RCX: 0000000000000001 [ 1800.977482][ T12] RDX: 000000010016e56c RSI: 0000000000000004 RDI: ffffffffc084e030 [ 1800.977663][ T12] RBP: 0000000000000001 R08: ffffffffc0a0d7ba R09: ffe21c0000b3ff6c [ 1800.977846][ T12] R10: ffe21c0000b3ff6d R11: 0000000000000001 R12: dffffc0000000000 [ 1800.978034][ T12] R13: ff110000059ffb60 R14: fffffbfff810b607 R15: ff1100001d8fc400 [ 1800.978220][ T12] FS: 0000000000000000(0000) GS:ff110000b8b73000(0000) knlGS:0000000000000000 [ 1800.978816][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1800.978976][ T12] CR2: 00007f8a34688e88 CR3: 0000000030d48002 CR4: 0000000000771ef0 [ 1800.979201][ T12] PKRU: 55555554 [ 1800.979329][ T12] Call Trace: [ 1800.979451][ T12] [ 1800.979536][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 1800.979744][ T12] ? kfree+0x119/0x580 [ 1800.979865][ T12] ? lock_acquire+0x10a/0x150 [ 1800.980033][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 1800.980230][ T12] ops_undo_list+0x2d7/0x8f0 [ 1800.980355][ T12] ? netns_install+0x280/0x280 [ 1800.980489][ T12] cleanup_net+0x3bc/0x890 [ 1800.980609][ T12] ? process_one_work+0xd16/0x1390 [ 1800.980731][ T12] ? net_passive_dec+0x190/0x190 [ 1800.980857][ T12] ? rcu_is_watching+0x15/0xd0 [ 1800.980982][ T12] ? process_one_work+0xd16/0x1390 [ 1800.981102][ T12] ? lock_acquire+0x10a/0x150 [ 1800.981223][ T12] ? rcu_is_watching+0x15/0xd0 [ 1800.981345][ T12] process_one_work+0xd57/0x1390 [ 1800.981481][ T12] ? pwq_dec_nr_in_flight+0x700/0x700 [ 1800.981602][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 1800.981726][ T12] ? assign_work+0x152/0x380 [ 1800.981848][ T12] worker_thread+0x4d6/0xd40 [ 1800.981978][ T12] ? process_one_work+0x1390/0x1390 [ 1800.982100][ T12] kthread+0x355/0x5b0 [ 1800.982193][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1800.982315][ T12] ? __lock_release.isra.0+0x59/0x170 [ 1800.982439][ T12] ? rcu_is_watching+0x15/0xd0 [ 1800.982559][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1800.982680][ T12] ret_from_fork+0x3fb/0x510 [ 1800.982803][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 1800.982959][ T12] ? __switch_to+0x53c/0xd00 [ 1800.983081][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1800.983211][ T12] ret_from_fork_asm+0x11/0x20 [ 1800.983338][ T12] [ 1800.983437][ T12] irq event stamp: 15613601 [ 1800.983560][ T12] hardirqs last enabled at (15613609): [] __up_console_sem+0x5a/0x70 [ 1800.983782][ T12] hardirqs last disabled at (15613616): [] __up_console_sem+0x3f/0x70 [ 1800.984003][ T12] softirqs last enabled at (15613376): [] handle_softirqs+0x56f/0x840 [ 1800.984231][ T12] softirqs last disabled at (15613371): [] irq_exit_rcu+0xa2/0xf0 [ 1800.984451][ T12] ---[ end trace 0000000000000000 ]--- [ 1967.521992][ T44] INFO: task modprobe:6062 blocked for more than 122 seconds. [ 1967.522414][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 1967.522626][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1967.522833][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 1967.523098][ T44] Call Trace: [ 1967.523186][ T44] [ 1967.523248][ T44] __schedule+0xae2/0x1720 [ 1967.523367][ T44] ? io_schedule_timeout+0x130/0x130 [ 1967.523479][ T44] ? __lock_release.isra.0+0x59/0x170 [ 1967.523593][ T44] schedule+0xd4/0x260 [ 1967.523681][ T44] schedule_preempt_disabled+0x18/0x30 [ 1967.523792][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 1967.523905][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 1967.524025][ T44] ? rcu_is_watching+0x15/0xd0 [ 1967.524134][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 1967.524245][ T44] ? register_pernet_subsys+0x19/0x40 [ 1967.524354][ T44] ? lock_acquire+0x10a/0x150 [ 1967.524463][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 1967.524598][ T44] down_write+0x1c6/0x1f0 [ 1967.524685][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 1967.524822][ T44] register_pernet_subsys+0x19/0x40 [ 1967.524946][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 1967.525088][ T44] do_one_initcall+0xa3/0x2e0 [ 1967.525199][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 1967.525335][ T44] ? kasan_unpoison+0x44/0x70 [ 1967.525448][ T44] do_init_module+0x27b/0x820 [ 1967.525562][ T44] ? free_module+0x450/0x450 [ 1967.525673][ T44] ? backing_file_read_iter+0x50e/0x730 [ 1967.525784][ T44] init_module_from_file+0x15d/0x180 [ 1967.526275][ T44] ? __do_sys_init_module+0x200/0x200 [ 1967.526391][ T44] ? __lock_release.isra.0+0x59/0x170 [ 1967.526500][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 1967.526609][ T44] idempotent_init_module+0x226/0x760 [ 1967.526722][ T44] ? init_module_from_file+0x180/0x180 [ 1967.526833][ T44] ? rcu_is_watching+0x15/0xd0 [ 1967.526951][ T44] ? cap_capable+0x181/0x3f0 [ 1967.527063][ T44] __x64_sys_finit_module+0xcd/0x160 [ 1967.527172][ T44] do_syscall_64+0xbd/0xfc0 [ 1967.527281][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1967.527416][ T44] RIP: 0033:0x7f2c6c2d334d [ 1967.527534][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 1967.527704][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 1967.527909][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 1967.528077][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 1967.528247][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 1967.528412][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 1967.528586][ T44] [ 1967.528672][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 1967.528853][ T44] [ 1967.528853][ T44] Showing all locks held in the system: [ 1967.529112][ T44] 4 locks held by kworker/u16:0/12: [ 1967.529223][ T44] 1 lock held by khungtaskd/44: [ 1967.529334][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1967.529552][ T44] 2 locks held by kworker/u17:3/1010: [ 1967.529658][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 1967.529867][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 1967.530125][ T44] 1 lock held by modprobe/6062: [ 1967.530228][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 1967.530438][ T44] [ 1967.530493][ T44] ============================================= [ 1967.530493][ T44] [ 2090.401986][ T44] INFO: task modprobe:6062 blocked for more than 245 seconds. [ 2090.402287][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2090.402498][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2090.402729][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2090.403047][ T44] Call Trace: [ 2090.403152][ T44] [ 2090.403216][ T44] __schedule+0xae2/0x1720 [ 2090.403354][ T44] ? io_schedule_timeout+0x130/0x130 [ 2090.403490][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.403614][ T44] schedule+0xd4/0x260 [ 2090.403712][ T44] schedule_preempt_disabled+0x18/0x30 [ 2090.403836][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2090.403966][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2090.404096][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.404217][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.404335][ T44] ? register_pernet_subsys+0x19/0x40 [ 2090.404452][ T44] ? lock_acquire+0x10a/0x150 [ 2090.404566][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2090.404713][ T44] down_write+0x1c6/0x1f0 [ 2090.404800][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2090.404954][ T44] register_pernet_subsys+0x19/0x40 [ 2090.405074][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2090.405216][ T44] do_one_initcall+0xa3/0x2e0 [ 2090.405326][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2090.405466][ T44] ? kasan_unpoison+0x44/0x70 [ 2090.405587][ T44] do_init_module+0x27b/0x820 [ 2090.405701][ T44] ? free_module+0x450/0x450 [ 2090.405815][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2090.405926][ T44] init_module_from_file+0x15d/0x180 [ 2090.406045][ T44] ? __do_sys_init_module+0x200/0x200 [ 2090.406170][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.406280][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2090.406392][ T44] idempotent_init_module+0x226/0x760 [ 2090.406508][ T44] ? init_module_from_file+0x180/0x180 [ 2090.406627][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.406736][ T44] ? cap_capable+0x181/0x3f0 [ 2090.406847][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2090.406964][ T44] do_syscall_64+0xbd/0xfc0 [ 2090.407082][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2090.407218][ T44] RIP: 0033:0x7f2c6c2d334d [ 2090.407335][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2090.407507][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2090.407671][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2090.407839][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2090.408010][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2090.408186][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2090.408356][ T44] [ 2090.408442][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2090.408621][ T44] [ 2090.408621][ T44] Showing all locks held in the system: [ 2090.408782][ T44] 5 locks held by kworker/u16:0/12: [ 2090.408896][ T44] 1 lock held by khungtaskd/44: [ 2090.409087][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2090.409305][ T44] 2 locks held by kworker/u17:3/1010: [ 2090.409408][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2090.409626][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2090.409874][ T44] 1 lock held by modprobe/6062: [ 2090.409987][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2090.410199][ T44] [ 2090.410254][ T44] ============================================= [ 2090.410254][ T44] [ 2213.281966][ T44] INFO: task modprobe:6062 blocked for more than 368 seconds. [ 2213.282237][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2213.282428][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.282663][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2213.282967][ T44] Call Trace: [ 2213.283076][ T44] [ 2213.283146][ T44] __schedule+0xae2/0x1720 [ 2213.283285][ T44] ? io_schedule_timeout+0x130/0x130 [ 2213.283407][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.283532][ T44] schedule+0xd4/0x260 [ 2213.283626][ T44] schedule_preempt_disabled+0x18/0x30 [ 2213.283751][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2213.283883][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2213.284008][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.284130][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.284257][ T44] ? register_pernet_subsys+0x19/0x40 [ 2213.284379][ T44] ? lock_acquire+0x10a/0x150 [ 2213.284492][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2213.284631][ T44] down_write+0x1c6/0x1f0 [ 2213.284721][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2213.284870][ T44] register_pernet_subsys+0x19/0x40 [ 2213.284993][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2213.285135][ T44] do_one_initcall+0xa3/0x2e0 [ 2213.285252][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2213.285400][ T44] ? kasan_unpoison+0x44/0x70 [ 2213.285514][ T44] do_init_module+0x27b/0x820 [ 2213.285636][ T44] ? free_module+0x450/0x450 [ 2213.285747][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2213.285862][ T44] init_module_from_file+0x15d/0x180 [ 2213.285979][ T44] ? __do_sys_init_module+0x200/0x200 [ 2213.286096][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.286205][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2213.286323][ T44] idempotent_init_module+0x226/0x760 [ 2213.286441][ T44] ? init_module_from_file+0x180/0x180 [ 2213.286553][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.286664][ T44] ? cap_capable+0x181/0x3f0 [ 2213.286774][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2213.286884][ T44] do_syscall_64+0xbd/0xfc0 [ 2213.286998][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2213.287134][ T44] RIP: 0033:0x7f2c6c2d334d [ 2213.287255][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2213.287420][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2213.287583][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2213.287747][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2213.287918][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2213.288084][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2213.288254][ T44] [ 2213.288335][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2213.288513][ T44] [ 2213.288513][ T44] Showing all locks held in the system: [ 2213.288675][ T44] 3 locks held by kworker/u16:0/12: [ 2213.288782][ T44] 1 lock held by khungtaskd/44: [ 2213.288884][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2213.289189][ T44] 2 locks held by kworker/u17:3/1010: [ 2213.289294][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2213.289513][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2213.289748][ T44] 1 lock held by modprobe/6062: [ 2213.289851][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2213.290066][ T44] [ 2213.290120][ T44] ============================================= [ 2213.290120][ T44] [ 2336.161963][ T44] INFO: task modprobe:6062 blocked for more than 491 seconds. [ 2336.162234][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2336.162439][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.162669][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2336.162970][ T44] Call Trace: [ 2336.163071][ T44] [ 2336.163141][ T44] __schedule+0xae2/0x1720 [ 2336.163279][ T44] ? io_schedule_timeout+0x130/0x130 [ 2336.163410][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.163543][ T44] schedule+0xd4/0x260 [ 2336.163633][ T44] schedule_preempt_disabled+0x18/0x30 [ 2336.163757][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2336.163884][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2336.164015][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.164138][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.164254][ T44] ? register_pernet_subsys+0x19/0x40 [ 2336.164384][ T44] ? lock_acquire+0x10a/0x150 [ 2336.164514][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2336.164667][ T44] down_write+0x1c6/0x1f0 [ 2336.164750][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2336.164887][ T44] register_pernet_subsys+0x19/0x40 [ 2336.165005][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2336.165142][ T44] do_one_initcall+0xa3/0x2e0 [ 2336.165253][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2336.165396][ T44] ? kasan_unpoison+0x44/0x70 [ 2336.165510][ T44] do_init_module+0x27b/0x820 [ 2336.165621][ T44] ? free_module+0x450/0x450 [ 2336.165729][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2336.165841][ T44] init_module_from_file+0x15d/0x180 [ 2336.165956][ T44] ? __do_sys_init_module+0x200/0x200 [ 2336.166070][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.166184][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2336.166294][ T44] idempotent_init_module+0x226/0x760 [ 2336.166403][ T44] ? init_module_from_file+0x180/0x180 [ 2336.166517][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.166624][ T44] ? cap_capable+0x181/0x3f0 [ 2336.166734][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2336.166844][ T44] do_syscall_64+0xbd/0xfc0 [ 2336.166964][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2336.167098][ T44] RIP: 0033:0x7f2c6c2d334d [ 2336.167211][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2336.167375][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2336.167541][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2336.167706][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2336.167867][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2336.168039][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2336.168207][ T44] [ 2336.168288][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2336.168468][ T44] [ 2336.168468][ T44] Showing all locks held in the system: [ 2336.168633][ T44] 4 locks held by kworker/u16:0/12: [ 2336.168738][ T44] 1 lock held by khungtaskd/44: [ 2336.168845][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2336.169148][ T44] 2 locks held by kworker/u17:3/1010: [ 2336.169257][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2336.169466][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2336.169705][ T44] 1 lock held by modprobe/6062: [ 2336.169808][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2336.170028][ T44] [ 2336.170082][ T44] ============================================= [ 2336.170082][ T44] [ 2459.041960][ T44] INFO: task modprobe:6062 blocked for more than 614 seconds. [ 2459.042226][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2459.042422][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.042659][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2459.042951][ T44] Call Trace: [ 2459.043047][ T44] [ 2459.043112][ T44] __schedule+0xae2/0x1720 [ 2459.043248][ T44] ? io_schedule_timeout+0x130/0x130 [ 2459.043374][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.043512][ T44] schedule+0xd4/0x260 [ 2459.043612][ T44] schedule_preempt_disabled+0x18/0x30 [ 2459.043739][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2459.043872][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2459.044005][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.044125][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.044256][ T44] ? register_pernet_subsys+0x19/0x40 [ 2459.044381][ T44] ? lock_acquire+0x10a/0x150 [ 2459.044508][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2459.044661][ T44] down_write+0x1c6/0x1f0 [ 2459.044756][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2459.044901][ T44] register_pernet_subsys+0x19/0x40 [ 2459.045023][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2459.045167][ T44] do_one_initcall+0xa3/0x2e0 [ 2459.045282][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2459.045421][ T44] ? kasan_unpoison+0x44/0x70 [ 2459.045535][ T44] do_init_module+0x27b/0x820 [ 2459.045649][ T44] ? free_module+0x450/0x450 [ 2459.045760][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2459.045873][ T44] init_module_from_file+0x15d/0x180 [ 2459.045990][ T44] ? __do_sys_init_module+0x200/0x200 [ 2459.046109][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.046226][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2459.046336][ T44] idempotent_init_module+0x226/0x760 [ 2459.046445][ T44] ? init_module_from_file+0x180/0x180 [ 2459.046557][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.046667][ T44] ? cap_capable+0x181/0x3f0 [ 2459.046782][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2459.046900][ T44] do_syscall_64+0xbd/0xfc0 [ 2459.047019][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2459.047161][ T44] RIP: 0033:0x7f2c6c2d334d [ 2459.047288][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2459.047452][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2459.047620][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2459.047782][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2459.047945][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2459.048111][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2459.048282][ T44] [ 2459.048368][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2459.048546][ T44] [ 2459.048546][ T44] Showing all locks held in the system: [ 2459.048699][ T44] 4 locks held by kworker/u16:0/12: [ 2459.048806][ T44] 1 lock held by khungtaskd/44: [ 2459.048909][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2459.049228][ T44] 2 locks held by kworker/u17:3/1010: [ 2459.049330][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2459.049541][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2459.049777][ T44] 1 lock held by modprobe/6062: [ 2459.049885][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2459.050103][ T44] [ 2459.050156][ T44] ============================================= [ 2459.050156][ T44] [ 2581.921959][ T44] INFO: task modprobe:6062 blocked for more than 737 seconds. [ 2581.922222][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2581.922422][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2581.922643][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2581.922945][ T44] Call Trace: [ 2581.923041][ T44] [ 2581.923117][ T44] __schedule+0xae2/0x1720 [ 2581.923244][ T44] ? io_schedule_timeout+0x130/0x130 [ 2581.923371][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.923498][ T44] schedule+0xd4/0x260 [ 2581.923592][ T44] schedule_preempt_disabled+0x18/0x30 [ 2581.923713][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2581.923842][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2581.923974][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.924097][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.924210][ T44] ? register_pernet_subsys+0x19/0x40 [ 2581.924321][ T44] ? lock_acquire+0x10a/0x150 [ 2581.924433][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2581.924571][ T44] down_write+0x1c6/0x1f0 [ 2581.924654][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2581.924797][ T44] register_pernet_subsys+0x19/0x40 [ 2581.924911][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2581.925065][ T44] do_one_initcall+0xa3/0x2e0 [ 2581.925177][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2581.925316][ T44] ? kasan_unpoison+0x44/0x70 [ 2581.925428][ T44] do_init_module+0x27b/0x820 [ 2581.925540][ T44] ? free_module+0x450/0x450 [ 2581.925648][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2581.925760][ T44] init_module_from_file+0x15d/0x180 [ 2581.925875][ T44] ? __do_sys_init_module+0x200/0x200 [ 2581.925999][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.926112][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2581.926225][ T44] idempotent_init_module+0x226/0x760 [ 2581.926339][ T44] ? init_module_from_file+0x180/0x180 [ 2581.926450][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.926566][ T44] ? cap_capable+0x181/0x3f0 [ 2581.926679][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2581.926788][ T44] do_syscall_64+0xbd/0xfc0 [ 2581.926896][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2581.927033][ T44] RIP: 0033:0x7f2c6c2d334d [ 2581.927151][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2581.927314][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2581.927480][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2581.927644][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2581.927807][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2581.927972][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2581.928147][ T44] [ 2581.928230][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2581.928405][ T44] [ 2581.928405][ T44] Showing all locks held in the system: [ 2581.928559][ T44] 4 locks held by kworker/u16:0/12: [ 2581.928664][ T44] 1 lock held by khungtaskd/44: [ 2581.928778][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2581.929076][ T44] 2 locks held by kworker/u17:3/1010: [ 2581.929181][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2581.929390][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2581.929628][ T44] 1 lock held by modprobe/6062: [ 2581.929736][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2581.929946][ T44] [ 2581.929998][ T44] ============================================= [ 2581.929998][ T44] [ 2704.801957][ T44] INFO: task modprobe:6062 blocked for more than 860 seconds. [ 2704.802229][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2704.802428][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2704.802654][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2704.802964][ T44] Call Trace: [ 2704.803083][ T44] [ 2704.803154][ T44] __schedule+0xae2/0x1720 [ 2704.803291][ T44] ? io_schedule_timeout+0x130/0x130 [ 2704.803414][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.803539][ T44] schedule+0xd4/0x260 [ 2704.803632][ T44] schedule_preempt_disabled+0x18/0x30 [ 2704.803753][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2704.803878][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2704.804010][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.804137][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.804262][ T44] ? register_pernet_subsys+0x19/0x40 [ 2704.804372][ T44] ? lock_acquire+0x10a/0x150 [ 2704.804482][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2704.804620][ T44] down_write+0x1c6/0x1f0 [ 2704.804705][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2704.804843][ T44] register_pernet_subsys+0x19/0x40 [ 2704.804991][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2704.805132][ T44] do_one_initcall+0xa3/0x2e0 [ 2704.805242][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2704.805378][ T44] ? kasan_unpoison+0x44/0x70 [ 2704.805489][ T44] do_init_module+0x27b/0x820 [ 2704.805598][ T44] ? free_module+0x450/0x450 [ 2704.805719][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2704.805833][ T44] init_module_from_file+0x15d/0x180 [ 2704.805947][ T44] ? __do_sys_init_module+0x200/0x200 [ 2704.806066][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.806175][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2704.806284][ T44] idempotent_init_module+0x226/0x760 [ 2704.806396][ T44] ? init_module_from_file+0x180/0x180 [ 2704.806514][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.806624][ T44] ? cap_capable+0x181/0x3f0 [ 2704.806736][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2704.806846][ T44] do_syscall_64+0xbd/0xfc0 [ 2704.806963][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2704.807101][ T44] RIP: 0033:0x7f2c6c2d334d [ 2704.807219][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2704.807387][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2704.807552][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2704.807715][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2704.807875][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2704.808049][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2704.808223][ T44] [ 2704.808305][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2704.808482][ T44] [ 2704.808482][ T44] Showing all locks held in the system: [ 2704.808640][ T44] 4 locks held by kworker/u16:0/12: [ 2704.808751][ T44] 1 lock held by khungtaskd/44: [ 2704.808856][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2704.809168][ T44] 2 locks held by kworker/u17:3/1010: [ 2704.809272][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2704.809491][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2704.809732][ T44] 1 lock held by modprobe/6062: [ 2704.809834][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2704.810046][ T44] [ 2704.810099][ T44] ============================================= [ 2704.810099][ T44] [ 2827.681959][ T44] INFO: task modprobe:6062 blocked for more than 983 seconds. [ 2827.682239][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2827.682441][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2827.682661][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2827.682969][ T44] Call Trace: [ 2827.683077][ T44] [ 2827.683150][ T44] __schedule+0xae2/0x1720 [ 2827.683290][ T44] ? io_schedule_timeout+0x130/0x130 [ 2827.683426][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.683557][ T44] schedule+0xd4/0x260 [ 2827.683653][ T44] schedule_preempt_disabled+0x18/0x30 [ 2827.683776][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2827.683918][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2827.684052][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.684170][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.684293][ T44] ? register_pernet_subsys+0x19/0x40 [ 2827.684415][ T44] ? lock_acquire+0x10a/0x150 [ 2827.684539][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2827.684677][ T44] down_write+0x1c6/0x1f0 [ 2827.684764][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2827.684908][ T44] register_pernet_subsys+0x19/0x40 [ 2827.685037][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2827.685183][ T44] do_one_initcall+0xa3/0x2e0 [ 2827.685293][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2827.685437][ T44] ? kasan_unpoison+0x44/0x70 [ 2827.685550][ T44] do_init_module+0x27b/0x820 [ 2827.685664][ T44] ? free_module+0x450/0x450 [ 2827.685775][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2827.685888][ T44] init_module_from_file+0x15d/0x180 [ 2827.686007][ T44] ? __do_sys_init_module+0x200/0x200 [ 2827.686122][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.686231][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2827.686345][ T44] idempotent_init_module+0x226/0x760 [ 2827.686464][ T44] ? init_module_from_file+0x180/0x180 [ 2827.686577][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.686686][ T44] ? cap_capable+0x181/0x3f0 [ 2827.686798][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2827.686913][ T44] do_syscall_64+0xbd/0xfc0 [ 2827.687028][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2827.687168][ T44] RIP: 0033:0x7f2c6c2d334d [ 2827.687289][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2827.687456][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2827.687624][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2827.687788][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2827.687965][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2827.688128][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2827.688303][ T44] [ 2827.688384][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2827.688570][ T44] [ 2827.688570][ T44] Showing all locks held in the system: [ 2827.688731][ T44] 3 locks held by kworker/u16:0/12: [ 2827.688840][ T44] 1 lock held by khungtaskd/44: [ 2827.689040][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2827.689256][ T44] 2 locks held by kworker/u17:3/1010: [ 2827.689362][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2827.689579][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2827.689822][ T44] 1 lock held by modprobe/6062: [ 2827.689937][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2827.690146][ T44] [ 2827.690200][ T44] ============================================= [ 2827.690200][ T44] [ 2950.561958][ T44] INFO: task modprobe:6062 blocked for more than 1105 seconds. [ 2950.562239][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2950.562439][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2950.562689][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 2950.563000][ T44] Call Trace: [ 2950.563098][ T44] [ 2950.563165][ T44] __schedule+0xae2/0x1720 [ 2950.563300][ T44] ? io_schedule_timeout+0x130/0x130 [ 2950.563430][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.563562][ T44] schedule+0xd4/0x260 [ 2950.563659][ T44] schedule_preempt_disabled+0x18/0x30 [ 2950.563784][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2950.563909][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2950.564037][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.564163][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.564287][ T44] ? register_pernet_subsys+0x19/0x40 [ 2950.564411][ T44] ? lock_acquire+0x10a/0x150 [ 2950.564535][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 2950.564700][ T44] down_write+0x1c6/0x1f0 [ 2950.564787][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2950.564929][ T44] register_pernet_subsys+0x19/0x40 [ 2950.565052][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2950.565194][ T44] do_one_initcall+0xa3/0x2e0 [ 2950.565306][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2950.565449][ T44] ? kasan_unpoison+0x44/0x70 [ 2950.565564][ T44] do_init_module+0x27b/0x820 [ 2950.565677][ T44] ? free_module+0x450/0x450 [ 2950.565788][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2950.565904][ T44] init_module_from_file+0x15d/0x180 [ 2950.566024][ T44] ? __do_sys_init_module+0x200/0x200 [ 2950.566137][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.566253][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2950.566363][ T44] idempotent_init_module+0x226/0x760 [ 2950.566473][ T44] ? init_module_from_file+0x180/0x180 [ 2950.566585][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.566693][ T44] ? cap_capable+0x181/0x3f0 [ 2950.566805][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2950.566917][ T44] do_syscall_64+0xbd/0xfc0 [ 2950.567039][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2950.567177][ T44] RIP: 0033:0x7f2c6c2d334d [ 2950.567293][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2950.567458][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 2950.567623][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 2950.567790][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 2950.567960][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 2950.568121][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 2950.568290][ T44] [ 2950.568377][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 2950.568557][ T44] [ 2950.568557][ T44] Showing all locks held in the system: [ 2950.568713][ T44] 4 locks held by kworker/u16:0/12: [ 2950.568822][ T44] 1 lock held by khungtaskd/44: [ 2950.568925][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2950.569249][ T44] 2 locks held by kworker/u17:3/1010: [ 2950.569361][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2950.569575][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2950.569815][ T44] 1 lock held by modprobe/6062: [ 2950.569921][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2950.570143][ T44] [ 2950.570195][ T44] ============================================= [ 2950.570195][ T44] [ 3073.441956][ T44] INFO: task modprobe:6062 blocked for more than 1228 seconds. [ 3073.442236][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 3073.442441][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3073.442674][ T44] task:modprobe state:D stack:25136 pid:6062 tgid:6062 ppid:1010 task_flags:0x400100 flags:0x00080000 [ 3073.442981][ T44] Call Trace: [ 3073.443079][ T44] [ 3073.443145][ T44] __schedule+0xae2/0x1720 [ 3073.443270][ T44] ? io_schedule_timeout+0x130/0x130 [ 3073.443393][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.443518][ T44] schedule+0xd4/0x260 [ 3073.443613][ T44] schedule_preempt_disabled+0x18/0x30 [ 3073.443740][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3073.443870][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3073.444005][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.444121][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.444233][ T44] ? register_pernet_subsys+0x19/0x40 [ 3073.444347][ T44] ? lock_acquire+0x10a/0x150 [ 3073.444461][ T44] ? conntrack_mt_exit+0xfb0/0xfb0 [xt_conntrack] [ 3073.444601][ T44] down_write+0x1c6/0x1f0 [ 3073.444687][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3073.444829][ T44] register_pernet_subsys+0x19/0x40 [ 3073.444947][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 3073.445090][ T44] do_one_initcall+0xa3/0x2e0 [ 3073.445210][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3073.445348][ T44] ? kasan_unpoison+0x44/0x70 [ 3073.445459][ T44] do_init_module+0x27b/0x820 [ 3073.445570][ T44] ? free_module+0x450/0x450 [ 3073.445679][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3073.445790][ T44] init_module_from_file+0x15d/0x180 [ 3073.445901][ T44] ? __do_sys_init_module+0x200/0x200 [ 3073.446027][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.446139][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3073.446247][ T44] idempotent_init_module+0x226/0x760 [ 3073.446356][ T44] ? init_module_from_file+0x180/0x180 [ 3073.446467][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.446575][ T44] ? cap_capable+0x181/0x3f0 [ 3073.446685][ T44] __x64_sys_finit_module+0xcd/0x160 [ 3073.446798][ T44] do_syscall_64+0xbd/0xfc0 [ 3073.446906][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3073.447044][ T44] RIP: 0033:0x7f2c6c2d334d [ 3073.447160][ T44] RSP: 002b:00007fff0369d8c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3073.447323][ T44] RAX: ffffffffffffffda RBX: 0000563a0291ccf0 RCX: 00007f2c6c2d334d [ 3073.447491][ T44] RDX: 0000000000000000 RSI: 00005639e2fc0f2e RDI: 0000000000000000 [ 3073.447656][ T44] RBP: 00007fff0369d960 R08: 0000000000000000 R09: 0000000000000000 [ 3073.447817][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005639e2fc0f2e [ 3073.447985][ T44] R13: 0000000000040000 R14: 0000563a0291cca0 R15: 0000563a0291c570 [ 3073.448168][ T44] [ 3073.448257][ T44] INFO: task modprobe:6062 is blocked on an rw-semaphore, but the owner is not found. [ 3073.448432][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 3073.448610][ T44] [ 3073.448610][ T44] Showing all locks held in the system: [ 3073.448764][ T44] 3 locks held by kworker/u16:0/12: [ 3073.448868][ T44] 1 lock held by khungtaskd/44: [ 3073.449069][ T44] #0: ffffffffb25850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3073.449282][ T44] 2 locks held by kworker/u17:3/1010: [ 3073.449384][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 3073.449593][ T44] #1: ffa00000029b7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 3073.449828][ T44] 1 lock held by modprobe/6062: [ 3073.449938][ T44] #0: ffffffffb2c85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3073.450152][ T44] [ 3073.450205][ T44] ============================================= [ 3073.450205][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr