[ 1832.345372][ T12] ------------[ cut here ]------------ [ 1832.345720][ T12] conntrack cleanup blocked for 60s [ 1832.345723][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack], CPU#1: kworker/u16:0/12 [ 1832.346238][ T12] Modules linked in: nf_flow_table(+) sch_ingress xt_conntrack nft_chain_nat xt_nat xfrm_user openvswitch psample nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nsh geneve vxlan act_csum act_pedit cls_flower sch_prio xt_HL nft_compat nf_tables amt [last unloaded: ila] [ 1832.346933][ T12] CPU: 1 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.19.0-rc1-virtme #1 PREEMPT(full) [ 1832.347219][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1832.347389][ T12] Workqueue: netns cleanup_net [ 1832.347525][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 1832.347728][ T12] Code: 00 48 8b 54 24 18 48 8b 05 b3 18 88 c8 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 ee 7c c7 e9 25 fd ff ff 48 8d 3d 46 f6 db ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df [ 1832.348181][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297 [ 1832.348343][ T12] RAX: 0000000100175fd7 RBX: ffa00000000c7af8 RCX: 0000000000000001 [ 1832.348536][ T12] RDX: 0000000100175fd6 RSI: 0000000000000004 RDI: ffffffffc054f030 [ 1832.348721][ T12] RBP: 0000000000000001 R08: ffffffffc078f7ba R09: ffe21c00017543e4 [ 1832.348909][ T12] R10: ffe21c00017543e5 R11: 0000000000000001 R12: dffffc0000000000 [ 1832.349101][ T12] R13: ff1100000baa1f20 R14: fffffbfff80ab807 R15: ff1100000cc8dc00 [ 1832.349284][ T12] FS: 0000000000000000(0000) GS:ff110000e1c73000(0000) knlGS:0000000000000000 [ 1832.349657][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1832.349810][ T12] CR2: 00007f6260be0a48 CR3: 000000003b348005 CR4: 0000000000771ef0 [ 1832.349993][ T12] PKRU: 55555554 [ 1832.350090][ T12] Call Trace: [ 1832.350183][ T12] [ 1832.350250][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 1832.350414][ T12] ? kfree+0x119/0x580 [ 1832.350507][ T12] ? lock_acquire+0x10a/0x150 [ 1832.350631][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 1832.350784][ T12] ops_undo_list+0x2d7/0x8f0 [ 1832.350907][ T12] ? netns_install+0x280/0x280 [ 1832.351033][ T12] cleanup_net+0x3bc/0x890 [ 1832.351155][ T12] ? process_one_work+0xd16/0x1390 [ 1832.351280][ T12] ? net_passive_dec+0x190/0x190 [ 1832.351411][ T12] ? rcu_is_watching+0x15/0xd0 [ 1832.351532][ T12] ? process_one_work+0xd16/0x1390 [ 1832.351651][ T12] ? lock_acquire+0x10a/0x150 [ 1832.351771][ T12] ? rcu_is_watching+0x15/0xd0 [ 1832.351893][ T12] process_one_work+0xd57/0x1390 [ 1832.352019][ T12] ? pwq_dec_nr_in_flight+0x700/0x700 [ 1832.352140][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 1832.352264][ T12] ? assign_work+0x152/0x380 [ 1832.352389][ T12] worker_thread+0x4d6/0xd40 [ 1832.352513][ T12] ? process_one_work+0x1390/0x1390 [ 1832.352633][ T12] kthread+0x355/0x5b0 [ 1832.352725][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1832.352844][ T12] ? __lock_release.isra.0+0x59/0x170 [ 1832.352965][ T12] ? rcu_is_watching+0x15/0xd0 [ 1832.353088][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1832.353209][ T12] ret_from_fork+0x3fb/0x510 [ 1832.353330][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 1832.353485][ T12] ? __switch_to+0x53c/0xd00 [ 1832.353607][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1832.353727][ T12] ret_from_fork_asm+0x11/0x20 [ 1832.353853][ T12] [ 1832.353945][ T12] irq event stamp: 24504127 [ 1832.354070][ T12] hardirqs last enabled at (24504135): [] __up_console_sem+0x5a/0x70 [ 1832.354279][ T12] hardirqs last disabled at (24504142): [] __up_console_sem+0x3f/0x70 [ 1832.354492][ T12] softirqs last enabled at (24503740): [] handle_softirqs+0x56f/0x840 [ 1832.354701][ T12] softirqs last disabled at (24503735): [] irq_exit_rcu+0xa2/0xf0 [ 1832.354915][ T12] ---[ end trace 0000000000000000 ]--- [ 1967.554384][ T44] INFO: task modprobe:24674 blocked for more than 122 seconds. [ 1967.554786][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 1967.555035][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1967.555338][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 1967.555699][ T44] Call Trace: [ 1967.555840][ T44] [ 1967.555954][ T44] __schedule+0xae2/0x1720 [ 1967.556132][ T44] ? io_schedule_timeout+0x130/0x130 [ 1967.556304][ T44] ? __lock_release.isra.0+0x59/0x170 [ 1967.556503][ T44] schedule+0xd4/0x260 [ 1967.556643][ T44] schedule_preempt_disabled+0x18/0x30 [ 1967.556812][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 1967.556981][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 1967.557151][ T44] ? rcu_is_watching+0x15/0xd0 [ 1967.557321][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 1967.557513][ T44] ? register_pernet_subsys+0x19/0x40 [ 1967.557676][ T44] ? lock_acquire+0x10a/0x150 [ 1967.557842][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 1967.558037][ T44] down_write+0x1c6/0x1f0 [ 1967.558174][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 1967.558388][ T44] register_pernet_subsys+0x19/0x40 [ 1967.558552][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 1967.558744][ T44] do_one_initcall+0xa3/0x2e0 [ 1967.558906][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 1967.559100][ T44] ? kasan_unpoison+0x44/0x70 [ 1967.559266][ T44] do_init_module+0x27b/0x820 [ 1967.559471][ T44] ? free_module+0x450/0x450 [ 1967.559765][ T44] ? backing_file_read_iter+0x50e/0x730 [ 1967.559933][ T44] init_module_from_file+0x15d/0x180 [ 1967.560094][ T44] ? __do_sys_init_module+0x200/0x200 [ 1967.560262][ T44] ? __lock_release.isra.0+0x59/0x170 [ 1967.560453][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 1967.560619][ T44] idempotent_init_module+0x226/0x760 [ 1967.560784][ T44] ? init_module_from_file+0x180/0x180 [ 1967.560948][ T44] ? rcu_is_watching+0x15/0xd0 [ 1967.561109][ T44] ? cap_capable+0x181/0x3f0 [ 1967.561272][ T44] __x64_sys_finit_module+0xcd/0x160 [ 1967.561459][ T44] do_syscall_64+0xbd/0xfc0 [ 1967.561625][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1967.561810][ T44] RIP: 0033:0x7f086258634d [ 1967.561979][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 1967.562196][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 1967.562429][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 1967.562647][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 1967.562859][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 1967.563072][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 1967.563293][ T44] [ 1967.563467][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 1967.563718][ T44] [ 1967.563718][ T44] Showing all locks held in the system: [ 1967.563928][ T44] 4 locks held by kworker/u16:0/12: [ 1967.564086][ T44] 2 locks held by kworker/u17:0/35: [ 1967.564238][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 1967.564596][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 1967.564883][ T44] 1 lock held by khungtaskd/44: [ 1967.565035][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1967.565304][ T44] 1 lock held by modprobe/24674: [ 1967.565479][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 1967.565747][ T44] [ 1967.565851][ T44] ============================================= [ 1967.565851][ T44] [ 2090.434377][ T44] INFO: task modprobe:24674 blocked for more than 245 seconds. [ 2090.434731][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2090.435011][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2090.435328][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2090.435749][ T44] Call Trace: [ 2090.435890][ T44] [ 2090.436002][ T44] __schedule+0xae2/0x1720 [ 2090.436185][ T44] ? io_schedule_timeout+0x130/0x130 [ 2090.436374][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.436549][ T44] schedule+0xd4/0x260 [ 2090.436690][ T44] schedule_preempt_disabled+0x18/0x30 [ 2090.436854][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2090.437027][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2090.437195][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.437379][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.437545][ T44] ? register_pernet_subsys+0x19/0x40 [ 2090.437711][ T44] ? lock_acquire+0x10a/0x150 [ 2090.437877][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2090.438077][ T44] down_write+0x1c6/0x1f0 [ 2090.438215][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2090.438425][ T44] register_pernet_subsys+0x19/0x40 [ 2090.438590][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2090.438794][ T44] do_one_initcall+0xa3/0x2e0 [ 2090.438956][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2090.439147][ T44] ? kasan_unpoison+0x44/0x70 [ 2090.439313][ T44] do_init_module+0x27b/0x820 [ 2090.439506][ T44] ? free_module+0x450/0x450 [ 2090.439669][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2090.439832][ T44] init_module_from_file+0x15d/0x180 [ 2090.439997][ T44] ? __do_sys_init_module+0x200/0x200 [ 2090.440171][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.440331][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2090.440508][ T44] idempotent_init_module+0x226/0x760 [ 2090.440670][ T44] ? init_module_from_file+0x180/0x180 [ 2090.440834][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.440995][ T44] ? cap_capable+0x181/0x3f0 [ 2090.441168][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2090.441329][ T44] do_syscall_64+0xbd/0xfc0 [ 2090.441505][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2090.441691][ T44] RIP: 0033:0x7f086258634d [ 2090.441856][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2090.442076][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2090.442293][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2090.442525][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2090.442738][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2090.442952][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2090.443179][ T44] [ 2090.443311][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2090.443569][ T44] [ 2090.443569][ T44] Showing all locks held in the system: [ 2090.443775][ T44] 4 locks held by kworker/u16:0/12: [ 2090.443931][ T44] 2 locks held by kworker/u17:0/35: [ 2090.444089][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2090.444371][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2090.444657][ T44] 1 lock held by khungtaskd/44: [ 2090.444809][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2090.445091][ T44] 1 lock held by modprobe/24674: [ 2090.445244][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2090.445533][ T44] [ 2090.445636][ T44] ============================================= [ 2090.445636][ T44] [ 2213.314375][ T44] INFO: task modprobe:24674 blocked for more than 368 seconds. [ 2213.314690][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2213.314940][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.315207][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2213.315581][ T44] Call Trace: [ 2213.315734][ T44] [ 2213.315856][ T44] __schedule+0xae2/0x1720 [ 2213.316037][ T44] ? io_schedule_timeout+0x130/0x130 [ 2213.316223][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.316410][ T44] schedule+0xd4/0x260 [ 2213.316551][ T44] schedule_preempt_disabled+0x18/0x30 [ 2213.316717][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2213.316887][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2213.317057][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.317232][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.317416][ T44] ? register_pernet_subsys+0x19/0x40 [ 2213.317580][ T44] ? lock_acquire+0x10a/0x150 [ 2213.317744][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2213.317940][ T44] down_write+0x1c6/0x1f0 [ 2213.318077][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2213.318280][ T44] register_pernet_subsys+0x19/0x40 [ 2213.318468][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2213.318660][ T44] do_one_initcall+0xa3/0x2e0 [ 2213.318824][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2213.319014][ T44] ? kasan_unpoison+0x44/0x70 [ 2213.319185][ T44] do_init_module+0x27b/0x820 [ 2213.319380][ T44] ? free_module+0x450/0x450 [ 2213.319544][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2213.319710][ T44] init_module_from_file+0x15d/0x180 [ 2213.319870][ T44] ? __do_sys_init_module+0x200/0x200 [ 2213.320037][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.320200][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2213.320382][ T44] idempotent_init_module+0x226/0x760 [ 2213.320543][ T44] ? init_module_from_file+0x180/0x180 [ 2213.320711][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.320873][ T44] ? cap_capable+0x181/0x3f0 [ 2213.321035][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2213.321200][ T44] do_syscall_64+0xbd/0xfc0 [ 2213.321377][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2213.321564][ T44] RIP: 0033:0x7f086258634d [ 2213.321729][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2213.321946][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2213.322166][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2213.322394][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2213.322604][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2213.322820][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2213.323039][ T44] [ 2213.323171][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2213.323429][ T44] [ 2213.323429][ T44] Showing all locks held in the system: [ 2213.323636][ T44] 4 locks held by kworker/u16:0/12: [ 2213.323797][ T44] 2 locks held by kworker/u17:0/35: [ 2213.323951][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2213.324217][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2213.324520][ T44] 1 lock held by khungtaskd/44: [ 2213.324672][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2213.324943][ T44] 1 lock held by modprobe/24674: [ 2213.325103][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2213.325380][ T44] [ 2213.325481][ T44] ============================================= [ 2213.325481][ T44] [ 2336.194370][ T44] INFO: task modprobe:24674 blocked for more than 491 seconds. [ 2336.194689][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2336.194931][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.195199][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2336.195558][ T44] Call Trace: [ 2336.195717][ T44] [ 2336.195834][ T44] __schedule+0xae2/0x1720 [ 2336.196013][ T44] ? io_schedule_timeout+0x130/0x130 [ 2336.196189][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.196375][ T44] schedule+0xd4/0x260 [ 2336.196515][ T44] schedule_preempt_disabled+0x18/0x30 [ 2336.196683][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2336.196851][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2336.197025][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.197189][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.197370][ T44] ? register_pernet_subsys+0x19/0x40 [ 2336.197531][ T44] ? lock_acquire+0x10a/0x150 [ 2336.197703][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2336.197897][ T44] down_write+0x1c6/0x1f0 [ 2336.198039][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2336.198233][ T44] register_pernet_subsys+0x19/0x40 [ 2336.198406][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2336.198597][ T44] do_one_initcall+0xa3/0x2e0 [ 2336.198765][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2336.198956][ T44] ? kasan_unpoison+0x44/0x70 [ 2336.199120][ T44] do_init_module+0x27b/0x820 [ 2336.199295][ T44] ? free_module+0x450/0x450 [ 2336.199494][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2336.199660][ T44] init_module_from_file+0x15d/0x180 [ 2336.199822][ T44] ? __do_sys_init_module+0x200/0x200 [ 2336.199987][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.200149][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2336.200313][ T44] idempotent_init_module+0x226/0x760 [ 2336.200486][ T44] ? init_module_from_file+0x180/0x180 [ 2336.200656][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.200819][ T44] ? cap_capable+0x181/0x3f0 [ 2336.200981][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2336.201142][ T44] do_syscall_64+0xbd/0xfc0 [ 2336.201302][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2336.201507][ T44] RIP: 0033:0x7f086258634d [ 2336.201677][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2336.201891][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2336.202103][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2336.202314][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2336.202546][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2336.202763][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2336.202982][ T44] [ 2336.203113][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2336.203442][ T44] [ 2336.203442][ T44] Showing all locks held in the system: [ 2336.203654][ T44] 4 locks held by kworker/u16:0/12: [ 2336.203808][ T44] 2 locks held by kworker/u17:0/35: [ 2336.203960][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2336.204221][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2336.204530][ T44] 1 lock held by khungtaskd/44: [ 2336.204690][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2336.204959][ T44] 1 lock held by modprobe/24674: [ 2336.205110][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2336.205388][ T44] [ 2336.205490][ T44] ============================================= [ 2336.205490][ T44] [ 2459.074372][ T44] INFO: task modprobe:24674 blocked for more than 614 seconds. [ 2459.074699][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2459.074975][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.075266][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2459.075639][ T44] Call Trace: [ 2459.075779][ T44] [ 2459.075898][ T44] __schedule+0xae2/0x1720 [ 2459.076070][ T44] ? io_schedule_timeout+0x130/0x130 [ 2459.076239][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.076431][ T44] schedule+0xd4/0x260 [ 2459.076568][ T44] schedule_preempt_disabled+0x18/0x30 [ 2459.076739][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2459.076914][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2459.077083][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.077248][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.077428][ T44] ? register_pernet_subsys+0x19/0x40 [ 2459.077590][ T44] ? lock_acquire+0x10a/0x150 [ 2459.077755][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2459.077956][ T44] down_write+0x1c6/0x1f0 [ 2459.078093][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2459.078285][ T44] register_pernet_subsys+0x19/0x40 [ 2459.078462][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2459.078653][ T44] do_one_initcall+0xa3/0x2e0 [ 2459.078819][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2459.079013][ T44] ? kasan_unpoison+0x44/0x70 [ 2459.079177][ T44] do_init_module+0x27b/0x820 [ 2459.079340][ T44] ? free_module+0x450/0x450 [ 2459.079532][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2459.079694][ T44] init_module_from_file+0x15d/0x180 [ 2459.079862][ T44] ? __do_sys_init_module+0x200/0x200 [ 2459.080027][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.080197][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2459.080373][ T44] idempotent_init_module+0x226/0x760 [ 2459.080534][ T44] ? init_module_from_file+0x180/0x180 [ 2459.080696][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.080864][ T44] ? cap_capable+0x181/0x3f0 [ 2459.081029][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2459.081189][ T44] do_syscall_64+0xbd/0xfc0 [ 2459.081369][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2459.081553][ T44] RIP: 0033:0x7f086258634d [ 2459.081717][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2459.081938][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2459.082150][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2459.082382][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2459.082591][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2459.082807][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2459.083026][ T44] [ 2459.083157][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2459.083418][ T44] [ 2459.083418][ T44] Showing all locks held in the system: [ 2459.083622][ T44] 3 locks held by kworker/u16:0/12: [ 2459.083776][ T44] 2 locks held by kworker/u17:0/35: [ 2459.083934][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2459.084201][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2459.084505][ T44] 1 lock held by khungtaskd/44: [ 2459.084657][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2459.084921][ T44] 1 lock held by modprobe/24674: [ 2459.085074][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2459.085337][ T44] [ 2459.085468][ T44] ============================================= [ 2459.085468][ T44] [ 2581.954373][ T44] INFO: task modprobe:24674 blocked for more than 737 seconds. [ 2581.954683][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2581.954927][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2581.955200][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2581.955578][ T44] Call Trace: [ 2581.955718][ T44] [ 2581.955834][ T44] __schedule+0xae2/0x1720 [ 2581.956007][ T44] ? io_schedule_timeout+0x130/0x130 [ 2581.956178][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.956372][ T44] schedule+0xd4/0x260 [ 2581.956513][ T44] schedule_preempt_disabled+0x18/0x30 [ 2581.956680][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2581.956853][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2581.957021][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.957187][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.957367][ T44] ? register_pernet_subsys+0x19/0x40 [ 2581.957533][ T44] ? lock_acquire+0x10a/0x150 [ 2581.957699][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2581.957896][ T44] down_write+0x1c6/0x1f0 [ 2581.958033][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2581.958224][ T44] register_pernet_subsys+0x19/0x40 [ 2581.958402][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2581.958601][ T44] do_one_initcall+0xa3/0x2e0 [ 2581.958763][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2581.958959][ T44] ? kasan_unpoison+0x44/0x70 [ 2581.959123][ T44] do_init_module+0x27b/0x820 [ 2581.959287][ T44] ? free_module+0x450/0x450 [ 2581.959477][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2581.959641][ T44] init_module_from_file+0x15d/0x180 [ 2581.959810][ T44] ? __do_sys_init_module+0x200/0x200 [ 2581.959982][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.960144][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2581.960306][ T44] idempotent_init_module+0x226/0x760 [ 2581.960483][ T44] ? init_module_from_file+0x180/0x180 [ 2581.960648][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.960809][ T44] ? cap_capable+0x181/0x3f0 [ 2581.960983][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2581.961145][ T44] do_syscall_64+0xbd/0xfc0 [ 2581.961305][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2581.961505][ T44] RIP: 0033:0x7f086258634d [ 2581.961675][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2581.961894][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2581.962111][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2581.962322][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2581.962548][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2581.962760][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2581.962988][ T44] [ 2581.963120][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2581.963377][ T44] [ 2581.963377][ T44] Showing all locks held in the system: [ 2581.963582][ T44] 4 locks held by kworker/u16:0/12: [ 2581.963738][ T44] 2 locks held by kworker/u17:0/35: [ 2581.963897][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2581.964163][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2581.964553][ T44] 1 lock held by khungtaskd/44: [ 2581.964707][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2581.964976][ T44] 1 lock held by modprobe/24674: [ 2581.965133][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2581.965416][ T44] [ 2581.965518][ T44] ============================================= [ 2581.965518][ T44] [ 2704.834369][ T44] INFO: task modprobe:24674 blocked for more than 860 seconds. [ 2704.834687][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2704.834943][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2704.835216][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2704.835615][ T44] Call Trace: [ 2704.835773][ T44] [ 2704.835889][ T44] __schedule+0xae2/0x1720 [ 2704.836060][ T44] ? io_schedule_timeout+0x130/0x130 [ 2704.836233][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.836425][ T44] schedule+0xd4/0x260 [ 2704.836566][ T44] schedule_preempt_disabled+0x18/0x30 [ 2704.836733][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2704.836903][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2704.837070][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.837233][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.837421][ T44] ? register_pernet_subsys+0x19/0x40 [ 2704.837585][ T44] ? lock_acquire+0x10a/0x150 [ 2704.837751][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2704.837944][ T44] down_write+0x1c6/0x1f0 [ 2704.838080][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2704.838281][ T44] register_pernet_subsys+0x19/0x40 [ 2704.838466][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2704.838656][ T44] do_one_initcall+0xa3/0x2e0 [ 2704.838818][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2704.839008][ T44] ? kasan_unpoison+0x44/0x70 [ 2704.839172][ T44] do_init_module+0x27b/0x820 [ 2704.839342][ T44] ? free_module+0x450/0x450 [ 2704.839536][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2704.839720][ T44] init_module_from_file+0x15d/0x180 [ 2704.839885][ T44] ? __do_sys_init_module+0x200/0x200 [ 2704.840055][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.840221][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2704.840405][ T44] idempotent_init_module+0x226/0x760 [ 2704.840571][ T44] ? init_module_from_file+0x180/0x180 [ 2704.840742][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.840904][ T44] ? cap_capable+0x181/0x3f0 [ 2704.841065][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2704.841226][ T44] do_syscall_64+0xbd/0xfc0 [ 2704.841405][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2704.841592][ T44] RIP: 0033:0x7f086258634d [ 2704.841757][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2704.841980][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2704.842191][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2704.842426][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2704.842637][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2704.842850][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2704.843069][ T44] [ 2704.843201][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2704.843465][ T44] [ 2704.843465][ T44] Showing all locks held in the system: [ 2704.843669][ T44] 4 locks held by kworker/u16:0/12: [ 2704.843827][ T44] 2 locks held by kworker/u17:0/35: [ 2704.843982][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2704.844246][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2704.844576][ T44] 1 lock held by khungtaskd/44: [ 2704.844730][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2704.844997][ T44] 1 lock held by modprobe/24674: [ 2704.845150][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2704.845517][ T44] [ 2704.845618][ T44] ============================================= [ 2704.845618][ T44] [ 2827.714368][ T44] INFO: task modprobe:24674 blocked for more than 983 seconds. [ 2827.714673][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2827.714925][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2827.715188][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2827.715543][ T44] Call Trace: [ 2827.715690][ T44] [ 2827.715816][ T44] __schedule+0xae2/0x1720 [ 2827.715998][ T44] ? io_schedule_timeout+0x130/0x130 [ 2827.716169][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.716342][ T44] schedule+0xd4/0x260 [ 2827.716502][ T44] schedule_preempt_disabled+0x18/0x30 [ 2827.716668][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2827.716842][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2827.717017][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.717183][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.717345][ T44] ? register_pernet_subsys+0x19/0x40 [ 2827.717527][ T44] ? lock_acquire+0x10a/0x150 [ 2827.717693][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2827.717888][ T44] down_write+0x1c6/0x1f0 [ 2827.718026][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2827.718222][ T44] register_pernet_subsys+0x19/0x40 [ 2827.718400][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2827.718594][ T44] do_one_initcall+0xa3/0x2e0 [ 2827.718763][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2827.718955][ T44] ? kasan_unpoison+0x44/0x70 [ 2827.719120][ T44] do_init_module+0x27b/0x820 [ 2827.719293][ T44] ? free_module+0x450/0x450 [ 2827.719488][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2827.719652][ T44] init_module_from_file+0x15d/0x180 [ 2827.719821][ T44] ? __do_sys_init_module+0x200/0x200 [ 2827.719986][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.720147][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2827.720307][ T44] idempotent_init_module+0x226/0x760 [ 2827.720486][ T44] ? init_module_from_file+0x180/0x180 [ 2827.720652][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.720820][ T44] ? cap_capable+0x181/0x3f0 [ 2827.720982][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2827.721144][ T44] do_syscall_64+0xbd/0xfc0 [ 2827.721304][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2827.721505][ T44] RIP: 0033:0x7f086258634d [ 2827.721674][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2827.721893][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2827.722107][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2827.722318][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2827.722549][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2827.722768][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2827.722986][ T44] [ 2827.723118][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2827.723376][ T44] [ 2827.723376][ T44] Showing all locks held in the system: [ 2827.723587][ T44] 4 locks held by kworker/u16:0/12: [ 2827.723746][ T44] 2 locks held by kworker/u17:0/35: [ 2827.723900][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2827.724162][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2827.724557][ T44] 1 lock held by khungtaskd/44: [ 2827.724708][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2827.724978][ T44] 1 lock held by modprobe/24674: [ 2827.725135][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2827.725415][ T44] [ 2827.725516][ T44] ============================================= [ 2827.725516][ T44] [ 2950.594369][ T44] INFO: task modprobe:24674 blocked for more than 1105 seconds. [ 2950.594688][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 2950.594936][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2950.595218][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 2950.595569][ T44] Call Trace: [ 2950.595711][ T44] [ 2950.595824][ T44] __schedule+0xae2/0x1720 [ 2950.595998][ T44] ? io_schedule_timeout+0x130/0x130 [ 2950.596171][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.596371][ T44] schedule+0xd4/0x260 [ 2950.596508][ T44] schedule_preempt_disabled+0x18/0x30 [ 2950.596676][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2950.596844][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2950.597013][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.597178][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.597368][ T44] ? register_pernet_subsys+0x19/0x40 [ 2950.597529][ T44] ? lock_acquire+0x10a/0x150 [ 2950.597693][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 2950.597888][ T44] down_write+0x1c6/0x1f0 [ 2950.598025][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2950.598224][ T44] register_pernet_subsys+0x19/0x40 [ 2950.598403][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 2950.598594][ T44] do_one_initcall+0xa3/0x2e0 [ 2950.598755][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2950.598950][ T44] ? kasan_unpoison+0x44/0x70 [ 2950.599114][ T44] do_init_module+0x27b/0x820 [ 2950.599284][ T44] ? free_module+0x450/0x450 [ 2950.599475][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2950.599638][ T44] init_module_from_file+0x15d/0x180 [ 2950.599799][ T44] ? __do_sys_init_module+0x200/0x200 [ 2950.599966][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.600136][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2950.600305][ T44] idempotent_init_module+0x226/0x760 [ 2950.600483][ T44] ? init_module_from_file+0x180/0x180 [ 2950.600649][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.600812][ T44] ? cap_capable+0x181/0x3f0 [ 2950.600973][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2950.601134][ T44] do_syscall_64+0xbd/0xfc0 [ 2950.601308][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2950.601508][ T44] RIP: 0033:0x7f086258634d [ 2950.601673][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2950.601887][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 2950.602100][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 2950.602323][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 2950.602555][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 2950.602767][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 2950.602986][ T44] [ 2950.603117][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 2950.603385][ T44] [ 2950.603385][ T44] Showing all locks held in the system: [ 2950.603588][ T44] 4 locks held by kworker/u16:0/12: [ 2950.603742][ T44] 2 locks held by kworker/u17:0/35: [ 2950.603896][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2950.604160][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2950.604476][ T44] 1 lock held by khungtaskd/44: [ 2950.604629][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2950.604894][ T44] 1 lock held by modprobe/24674: [ 2950.605049][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2950.605320][ T44] [ 2950.605448][ T44] ============================================= [ 2950.605448][ T44] [ 3073.474369][ T44] INFO: task modprobe:24674 blocked for more than 1228 seconds. [ 3073.474688][ T44] Tainted: G W 6.19.0-rc1-virtme #1 [ 3073.474937][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3073.475206][ T44] task:modprobe state:D stack:25136 pid:24674 tgid:24674 ppid:35 task_flags:0x400100 flags:0x00080000 [ 3073.475572][ T44] Call Trace: [ 3073.475716][ T44] [ 3073.475827][ T44] __schedule+0xae2/0x1720 [ 3073.475999][ T44] ? io_schedule_timeout+0x130/0x130 [ 3073.476170][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.476342][ T44] schedule+0xd4/0x260 [ 3073.476506][ T44] schedule_preempt_disabled+0x18/0x30 [ 3073.476678][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3073.476846][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3073.477015][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.477181][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.477344][ T44] ? register_pernet_subsys+0x19/0x40 [ 3073.477530][ T44] ? lock_acquire+0x10a/0x150 [ 3073.477702][ T44] ? ingress_module_exit+0xf20/0xf20 [sch_ingress] [ 3073.477894][ T44] down_write+0x1c6/0x1f0 [ 3073.478031][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3073.478223][ T44] register_pernet_subsys+0x19/0x40 [ 3073.478400][ T44] nf_flow_table_module_init+0x16/0x1000 [nf_flow_table] [ 3073.478593][ T44] do_one_initcall+0xa3/0x2e0 [ 3073.478768][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3073.478957][ T44] ? kasan_unpoison+0x44/0x70 [ 3073.479121][ T44] do_init_module+0x27b/0x820 [ 3073.479285][ T44] ? free_module+0x450/0x450 [ 3073.479478][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3073.479649][ T44] init_module_from_file+0x15d/0x180 [ 3073.479811][ T44] ? __do_sys_init_module+0x200/0x200 [ 3073.479983][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.480145][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3073.480306][ T44] idempotent_init_module+0x226/0x760 [ 3073.480484][ T44] ? init_module_from_file+0x180/0x180 [ 3073.480654][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.480814][ T44] ? cap_capable+0x181/0x3f0 [ 3073.480983][ T44] __x64_sys_finit_module+0xcd/0x160 [ 3073.481145][ T44] do_syscall_64+0xbd/0xfc0 [ 3073.481307][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3073.481514][ T44] RIP: 0033:0x7f086258634d [ 3073.481685][ T44] RSP: 002b:00007ffea1ad76a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3073.481899][ T44] RAX: ffffffffffffffda RBX: 0000559aee6c4cf0 RCX: 00007f086258634d [ 3073.482116][ T44] RDX: 0000000000000000 RSI: 0000559ae337ef2e RDI: 0000000000000000 [ 3073.482329][ T44] RBP: 00007ffea1ad7740 R08: 0000000000000000 R09: 0000000000000000 [ 3073.482561][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559ae337ef2e [ 3073.482779][ T44] R13: 0000000000040000 R14: 0000559aee6c4ca0 R15: 0000559aee6c4570 [ 3073.483000][ T44] [ 3073.483131][ T44] INFO: task modprobe:24674 is blocked on an rw-semaphore, but the owner is not found. [ 3073.483384][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 3073.483626][ T44] [ 3073.483626][ T44] Showing all locks held in the system: [ 3073.483840][ T44] 4 locks held by kworker/u16:0/12: [ 3073.483993][ T44] 2 locks held by kworker/u17:0/35: [ 3073.484150][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 3073.484516][ T44] #1: ffa000000027fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 3073.484807][ T44] 1 lock held by khungtaskd/44: [ 3073.484960][ T44] #0: ffffffff893850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3073.485227][ T44] 1 lock held by modprobe/24674: [ 3073.485394][ T44] #0: ffffffff89a85bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3073.485663][ T44] [ 3073.485766][ T44] ============================================= [ 3073.485766][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr