[ 1930.659676][ T12] ------------[ cut here ]------------ [ 1930.660099][ T12] conntrack cleanup blocked for 60s [ 1930.660102][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack], CPU#2: kworker/u16:0/12 [ 1930.660863][ T12] Modules linked in: nf_nat(+) xt_conntrack nf_conntrack nf_defrag_ipv4 nf_defrag_ipv6 sctp_diag sctp act_gact cls_matchall sch_ingress chacha libchacha chacha20poly1305 libpoly1305 tls xt_HL nft_compat nf_tables amt [ 1930.661602][ T12] CPU: 2 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.19.0-rc3-virtme #1 PREEMPT(full) [ 1930.661986][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1930.662223][ T12] Workqueue: netns cleanup_net [ 1930.662420][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 1930.662727][ T12] Code: 00 48 8b 54 24 18 48 8b 05 b3 a8 0d cf 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 7e 02 ce e9 25 fd ff ff 48 8d 3d 46 56 e3 ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df [ 1930.663254][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297 [ 1930.663467][ T12] RAX: 000000010018dfd4 RBX: ffa00000000c7af8 RCX: 0000000000000001 [ 1930.663728][ T12] RDX: 000000010018dfd3 RSI: 0000000000000004 RDI: ffffffffc056c030 [ 1930.663970][ T12] RBP: 0000000000000001 R08: ffffffffc07367ba R09: ffe21c00009690c4 [ 1930.664207][ T12] R10: ffe21c00009690c5 R11: 0000000000000001 R12: dffffc0000000000 [ 1930.664443][ T12] R13: ff11000004b48620 R14: fffffbfff80af207 R15: ff1100001428bcc0 [ 1930.664698][ T12] FS: 0000000000000000(0000) GS:ff110000db4f3000(0000) knlGS:0000000000000000 [ 1930.664970][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1930.665176][ T12] CR2: 00007fb835545a48 CR3: 000000005f348006 CR4: 0000000000771ef0 [ 1930.665738][ T12] PKRU: 55555554 [ 1930.665883][ T12] Call Trace: [ 1930.666027][ T12] [ 1930.666147][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 1930.666352][ T12] ? kfree+0x119/0x580 [ 1930.666494][ T12] ? lock_acquire+0x10a/0x150 [ 1930.666691][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 1930.666896][ T12] ops_undo_list+0x2d7/0x8f0 [ 1930.667072][ T12] ? netns_install+0x280/0x280 [ 1930.667247][ T12] cleanup_net+0x3bc/0x890 [ 1930.667422][ T12] ? process_one_work+0xd16/0x1390 [ 1930.667600][ T12] ? net_passive_dec+0x190/0x190 [ 1930.667786][ T12] ? rcu_is_watching+0x15/0xd0 [ 1930.667957][ T12] ? process_one_work+0xd16/0x1390 [ 1930.668128][ T12] ? lock_acquire+0x10a/0x150 [ 1930.668300][ T12] ? rcu_is_watching+0x15/0xd0 [ 1930.668472][ T12] process_one_work+0xd57/0x1390 [ 1930.668674][ T12] ? pwq_dec_nr_in_flight+0x700/0x700 [ 1930.668848][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 1930.669022][ T12] ? assign_work+0x152/0x380 [ 1930.669196][ T12] worker_thread+0x4d6/0xd40 [ 1930.669374][ T12] ? process_one_work+0x1390/0x1390 [ 1930.669546][ T12] kthread+0x355/0x5b0 [ 1930.669706][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1930.669878][ T12] ? __lock_release.isra.0+0x59/0x170 [ 1930.670054][ T12] ? rcu_is_watching+0x15/0xd0 [ 1930.670228][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1930.670401][ T12] ret_from_fork+0x3fb/0x510 [ 1930.670582][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 1930.670798][ T12] ? __switch_to+0x53c/0xd00 [ 1930.670972][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1930.671146][ T12] ret_from_fork_asm+0x11/0x20 [ 1930.671330][ T12] [ 1930.671474][ T12] irq event stamp: 7720791 [ 1930.671665][ T12] hardirqs last enabled at (7720803): [] __up_console_sem+0x5a/0x70 [ 1930.671927][ T12] hardirqs last disabled at (7720812): [] __up_console_sem+0x3f/0x70 [ 1930.672190][ T12] softirqs last enabled at (7720298): [] handle_softirqs+0x56f/0x840 [ 1930.672458][ T12] softirqs last disabled at (7720291): [] irq_exit_rcu+0xa2/0xf0 [ 1930.672744][ T12] ---[ end trace 0000000000000000 ]--- [ 2090.448663][ T44] INFO: task modprobe:28745 blocked for more than 122 seconds. [ 2090.449011][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2090.449214][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2090.449456][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2090.449753][ T44] Call Trace: [ 2090.449846][ T44] [ 2090.449911][ T44] __schedule+0xae2/0x1720 [ 2090.450037][ T44] ? io_schedule_timeout+0x130/0x130 [ 2090.450157][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.450282][ T44] schedule+0xd4/0x260 [ 2090.450372][ T44] schedule_preempt_disabled+0x18/0x30 [ 2090.450490][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2090.450608][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2090.450730][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.450842][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.450957][ T44] ? register_pernet_subsys+0x19/0x40 [ 2090.451071][ T44] ? lock_acquire+0x10a/0x150 [ 2090.451185][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2090.451305][ T44] down_write+0x1c6/0x1f0 [ 2090.451392][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2090.451532][ T44] ? static_obj+0x5d/0x90 [ 2090.451615][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2090.451740][ T44] register_pernet_subsys+0x19/0x40 [ 2090.451850][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2090.451960][ T44] do_one_initcall+0xa3/0x2e0 [ 2090.452071][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2090.452210][ T44] ? kasan_unpoison+0x44/0x70 [ 2090.452326][ T44] do_init_module+0x27b/0x820 [ 2090.452525][ T44] ? free_module+0x450/0x450 [ 2090.452634][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2090.452755][ T44] init_module_from_file+0x15d/0x180 [ 2090.452864][ T44] ? __do_sys_init_module+0x200/0x200 [ 2090.452982][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.453090][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2090.453199][ T44] idempotent_init_module+0x226/0x760 [ 2090.453313][ T44] ? init_module_from_file+0x180/0x180 [ 2090.453426][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.453533][ T44] ? cap_capable+0x181/0x3f0 [ 2090.453648][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2090.453757][ T44] do_syscall_64+0xbd/0xfc0 [ 2090.453867][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2090.454001][ T44] RIP: 0033:0x7fee353a334d [ 2090.454119][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2090.454292][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2090.454454][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2090.454624][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2090.454795][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2090.454973][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2090.455143][ T44] [ 2090.455226][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2090.455408][ T44] [ 2090.455408][ T44] Showing all locks held in the system: [ 2090.455567][ T44] 5 locks held by kworker/u16:0/12: [ 2090.455768][ T44] 1 lock held by khungtaskd/44: [ 2090.455871][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2090.456083][ T44] 2 locks held by kworker/u17:1/45: [ 2090.456186][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2090.456399][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2090.456652][ T44] 1 lock held by modprobe/28745: [ 2090.456756][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2090.456965][ T44] [ 2090.457019][ T44] ============================================= [ 2090.457019][ T44] [ 2213.328660][ T44] INFO: task modprobe:28745 blocked for more than 245 seconds. [ 2213.328941][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2213.329148][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.329391][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2213.329710][ T44] Call Trace: [ 2213.329808][ T44] [ 2213.329874][ T44] __schedule+0xae2/0x1720 [ 2213.329993][ T44] ? io_schedule_timeout+0x130/0x130 [ 2213.330115][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.330234][ T44] schedule+0xd4/0x260 [ 2213.330325][ T44] schedule_preempt_disabled+0x18/0x30 [ 2213.330439][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2213.330556][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2213.330685][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.330800][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.330913][ T44] ? register_pernet_subsys+0x19/0x40 [ 2213.331024][ T44] ? lock_acquire+0x10a/0x150 [ 2213.331136][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2213.331251][ T44] down_write+0x1c6/0x1f0 [ 2213.331347][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2213.331488][ T44] ? static_obj+0x5d/0x90 [ 2213.331571][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2213.331698][ T44] register_pernet_subsys+0x19/0x40 [ 2213.331810][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2213.331921][ T44] do_one_initcall+0xa3/0x2e0 [ 2213.332030][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2213.332168][ T44] ? kasan_unpoison+0x44/0x70 [ 2213.332278][ T44] do_init_module+0x27b/0x820 [ 2213.332400][ T44] ? free_module+0x450/0x450 [ 2213.332512][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2213.332622][ T44] init_module_from_file+0x15d/0x180 [ 2213.332737][ T44] ? __do_sys_init_module+0x200/0x200 [ 2213.332851][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.332967][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2213.333076][ T44] idempotent_init_module+0x226/0x760 [ 2213.333185][ T44] ? init_module_from_file+0x180/0x180 [ 2213.333300][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.333407][ T44] ? cap_capable+0x181/0x3f0 [ 2213.333517][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2213.333626][ T44] do_syscall_64+0xbd/0xfc0 [ 2213.333743][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2213.333878][ T44] RIP: 0033:0x7fee353a334d [ 2213.333994][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2213.334157][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2213.334323][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2213.334487][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2213.334651][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2213.334813][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2213.334981][ T44] [ 2213.335062][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2213.335240][ T44] [ 2213.335240][ T44] Showing all locks held in the system: [ 2213.335397][ T44] 3 locks held by kworker/u16:0/12: [ 2213.335503][ T44] 1 lock held by khungtaskd/44: [ 2213.335610][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2213.335908][ T44] 2 locks held by kworker/u17:1/45: [ 2213.336013][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2213.336222][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2213.336464][ T44] 1 lock held by modprobe/28745: [ 2213.336566][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2213.336781][ T44] [ 2213.336835][ T44] ============================================= [ 2213.336835][ T44] [ 2336.208658][ T44] INFO: task modprobe:28745 blocked for more than 368 seconds. [ 2336.208925][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2336.209125][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.209368][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2336.209690][ T44] Call Trace: [ 2336.209799][ T44] [ 2336.209869][ T44] __schedule+0xae2/0x1720 [ 2336.210008][ T44] ? io_schedule_timeout+0x130/0x130 [ 2336.210132][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.210249][ T44] schedule+0xd4/0x260 [ 2336.210341][ T44] schedule_preempt_disabled+0x18/0x30 [ 2336.210457][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2336.210572][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2336.210696][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.210809][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.210922][ T44] ? register_pernet_subsys+0x19/0x40 [ 2336.211036][ T44] ? lock_acquire+0x10a/0x150 [ 2336.211149][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2336.211263][ T44] down_write+0x1c6/0x1f0 [ 2336.211348][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2336.211483][ T44] ? static_obj+0x5d/0x90 [ 2336.211567][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2336.211690][ T44] register_pernet_subsys+0x19/0x40 [ 2336.211800][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2336.211914][ T44] do_one_initcall+0xa3/0x2e0 [ 2336.212024][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2336.212163][ T44] ? kasan_unpoison+0x44/0x70 [ 2336.212273][ T44] do_init_module+0x27b/0x820 [ 2336.212388][ T44] ? free_module+0x450/0x450 [ 2336.212497][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2336.212607][ T44] init_module_from_file+0x15d/0x180 [ 2336.212726][ T44] ? __do_sys_init_module+0x200/0x200 [ 2336.212840][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.212955][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2336.213064][ T44] idempotent_init_module+0x226/0x760 [ 2336.213173][ T44] ? init_module_from_file+0x180/0x180 [ 2336.213285][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.213409][ T44] ? cap_capable+0x181/0x3f0 [ 2336.213518][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2336.213628][ T44] do_syscall_64+0xbd/0xfc0 [ 2336.213745][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2336.213877][ T44] RIP: 0033:0x7fee353a334d [ 2336.213989][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2336.214154][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2336.214315][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2336.214478][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2336.214644][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2336.214806][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2336.214974][ T44] [ 2336.215054][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2336.215229][ T44] [ 2336.215229][ T44] Showing all locks held in the system: [ 2336.215386][ T44] 4 locks held by kworker/u16:0/12: [ 2336.215492][ T44] 1 lock held by khungtaskd/44: [ 2336.215596][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2336.215896][ T44] 2 locks held by kworker/u17:1/45: [ 2336.215999][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2336.216209][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2336.216452][ T44] 1 lock held by modprobe/28745: [ 2336.216554][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2336.216766][ T44] [ 2336.216819][ T44] ============================================= [ 2336.216819][ T44] [ 2459.088660][ T44] INFO: task modprobe:28745 blocked for more than 491 seconds. [ 2459.088930][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2459.089128][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.089353][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2459.089673][ T44] Call Trace: [ 2459.089771][ T44] [ 2459.089842][ T44] __schedule+0xae2/0x1720 [ 2459.089978][ T44] ? io_schedule_timeout+0x130/0x130 [ 2459.090113][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.090249][ T44] schedule+0xd4/0x260 [ 2459.090335][ T44] schedule_preempt_disabled+0x18/0x30 [ 2459.090450][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2459.090565][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2459.090686][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.090800][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.090913][ T44] ? register_pernet_subsys+0x19/0x40 [ 2459.091024][ T44] ? lock_acquire+0x10a/0x150 [ 2459.091136][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2459.091252][ T44] down_write+0x1c6/0x1f0 [ 2459.091337][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2459.091475][ T44] ? static_obj+0x5d/0x90 [ 2459.091558][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2459.091686][ T44] register_pernet_subsys+0x19/0x40 [ 2459.091795][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2459.091906][ T44] do_one_initcall+0xa3/0x2e0 [ 2459.092016][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2459.092153][ T44] ? kasan_unpoison+0x44/0x70 [ 2459.092265][ T44] do_init_module+0x27b/0x820 [ 2459.092376][ T44] ? free_module+0x450/0x450 [ 2459.092488][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2459.092597][ T44] init_module_from_file+0x15d/0x180 [ 2459.092714][ T44] ? __do_sys_init_module+0x200/0x200 [ 2459.092828][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.092935][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2459.093046][ T44] idempotent_init_module+0x226/0x760 [ 2459.093154][ T44] ? init_module_from_file+0x180/0x180 [ 2459.093266][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.093372][ T44] ? cap_capable+0x181/0x3f0 [ 2459.093485][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2459.093593][ T44] do_syscall_64+0xbd/0xfc0 [ 2459.093705][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2459.093839][ T44] RIP: 0033:0x7fee353a334d [ 2459.093953][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2459.094116][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2459.094276][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2459.094444][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2459.094612][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2459.094776][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2459.094945][ T44] [ 2459.095028][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2459.095203][ T44] [ 2459.095203][ T44] Showing all locks held in the system: [ 2459.095365][ T44] 4 locks held by kworker/u16:0/12: [ 2459.095474][ T44] 1 lock held by khungtaskd/44: [ 2459.095579][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2459.095879][ T44] 2 locks held by kworker/u17:1/45: [ 2459.095982][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2459.096197][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2459.096437][ T44] 1 lock held by modprobe/28745: [ 2459.096539][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2459.096752][ T44] [ 2459.096807][ T44] ============================================= [ 2459.096807][ T44] [ 2581.968657][ T44] INFO: task modprobe:28745 blocked for more than 614 seconds. [ 2581.968924][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2581.969140][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2581.969377][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2581.969686][ T44] Call Trace: [ 2581.969781][ T44] [ 2581.969846][ T44] __schedule+0xae2/0x1720 [ 2581.969985][ T44] ? io_schedule_timeout+0x130/0x130 [ 2581.970124][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.970259][ T44] schedule+0xd4/0x260 [ 2581.970355][ T44] schedule_preempt_disabled+0x18/0x30 [ 2581.970498][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2581.970612][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2581.970734][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.970852][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.970963][ T44] ? register_pernet_subsys+0x19/0x40 [ 2581.971078][ T44] ? lock_acquire+0x10a/0x150 [ 2581.971189][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2581.971310][ T44] down_write+0x1c6/0x1f0 [ 2581.971395][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2581.971537][ T44] ? static_obj+0x5d/0x90 [ 2581.971623][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2581.971745][ T44] register_pernet_subsys+0x19/0x40 [ 2581.971853][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2581.971965][ T44] do_one_initcall+0xa3/0x2e0 [ 2581.972073][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2581.972211][ T44] ? kasan_unpoison+0x44/0x70 [ 2581.972322][ T44] do_init_module+0x27b/0x820 [ 2581.972435][ T44] ? free_module+0x450/0x450 [ 2581.972544][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2581.972659][ T44] init_module_from_file+0x15d/0x180 [ 2581.972773][ T44] ? __do_sys_init_module+0x200/0x200 [ 2581.972887][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.972996][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2581.973106][ T44] idempotent_init_module+0x226/0x760 [ 2581.973214][ T44] ? init_module_from_file+0x180/0x180 [ 2581.973327][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.973436][ T44] ? cap_capable+0x181/0x3f0 [ 2581.973547][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2581.973660][ T44] do_syscall_64+0xbd/0xfc0 [ 2581.973771][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2581.973904][ T44] RIP: 0033:0x7fee353a334d [ 2581.974017][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2581.974181][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2581.974348][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2581.974513][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2581.974677][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2581.974839][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2581.975007][ T44] [ 2581.975088][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2581.975264][ T44] [ 2581.975264][ T44] Showing all locks held in the system: [ 2581.975421][ T44] 4 locks held by kworker/u16:0/12: [ 2581.975529][ T44] 1 lock held by khungtaskd/44: [ 2581.975631][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2581.975932][ T44] 2 locks held by kworker/u17:1/45: [ 2581.976035][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2581.976244][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2581.976487][ T44] 1 lock held by modprobe/28745: [ 2581.976591][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2581.976804][ T44] [ 2581.976856][ T44] ============================================= [ 2581.976856][ T44] [ 2704.848657][ T44] INFO: task modprobe:28745 blocked for more than 737 seconds. [ 2704.848923][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2704.849134][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2704.849366][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2704.849685][ T44] Call Trace: [ 2704.849777][ T44] [ 2704.849838][ T44] __schedule+0xae2/0x1720 [ 2704.849958][ T44] ? io_schedule_timeout+0x130/0x130 [ 2704.850079][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.850197][ T44] schedule+0xd4/0x260 [ 2704.850285][ T44] schedule_preempt_disabled+0x18/0x30 [ 2704.850400][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2704.850520][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2704.850651][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.850767][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.850879][ T44] ? register_pernet_subsys+0x19/0x40 [ 2704.850989][ T44] ? lock_acquire+0x10a/0x150 [ 2704.851101][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2704.851214][ T44] down_write+0x1c6/0x1f0 [ 2704.851299][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2704.851436][ T44] ? static_obj+0x5d/0x90 [ 2704.851526][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2704.851654][ T44] register_pernet_subsys+0x19/0x40 [ 2704.851766][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2704.851877][ T44] do_one_initcall+0xa3/0x2e0 [ 2704.851985][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2704.852122][ T44] ? kasan_unpoison+0x44/0x70 [ 2704.852235][ T44] do_init_module+0x27b/0x820 [ 2704.852347][ T44] ? free_module+0x450/0x450 [ 2704.852455][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2704.852570][ T44] init_module_from_file+0x15d/0x180 [ 2704.852685][ T44] ? __do_sys_init_module+0x200/0x200 [ 2704.852799][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.852908][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2704.853019][ T44] idempotent_init_module+0x226/0x760 [ 2704.853129][ T44] ? init_module_from_file+0x180/0x180 [ 2704.853241][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.853349][ T44] ? cap_capable+0x181/0x3f0 [ 2704.853457][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2704.853569][ T44] do_syscall_64+0xbd/0xfc0 [ 2704.853683][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2704.853816][ T44] RIP: 0033:0x7fee353a334d [ 2704.853931][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2704.854092][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2704.854253][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2704.854420][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2704.854586][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2704.854752][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2704.854918][ T44] [ 2704.854998][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2704.855179][ T44] [ 2704.855179][ T44] Showing all locks held in the system: [ 2704.855340][ T44] 4 locks held by kworker/u16:0/12: [ 2704.855445][ T44] 1 lock held by khungtaskd/44: [ 2704.855550][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2704.855849][ T44] 2 locks held by kworker/u17:1/45: [ 2704.855953][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2704.856164][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2704.856398][ T44] 1 lock held by modprobe/28745: [ 2704.856503][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2704.856720][ T44] [ 2704.856772][ T44] ============================================= [ 2704.856772][ T44] [ 2827.728657][ T44] INFO: task modprobe:28745 blocked for more than 860 seconds. [ 2827.728930][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2827.729131][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2827.729368][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2827.729706][ T44] Call Trace: [ 2827.729810][ T44] [ 2827.729882][ T44] __schedule+0xae2/0x1720 [ 2827.730016][ T44] ? io_schedule_timeout+0x130/0x130 [ 2827.730132][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.730248][ T44] schedule+0xd4/0x260 [ 2827.730337][ T44] schedule_preempt_disabled+0x18/0x30 [ 2827.730452][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2827.730572][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2827.730695][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.730808][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.730921][ T44] ? register_pernet_subsys+0x19/0x40 [ 2827.731032][ T44] ? lock_acquire+0x10a/0x150 [ 2827.731142][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2827.731258][ T44] down_write+0x1c6/0x1f0 [ 2827.731345][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2827.731480][ T44] ? static_obj+0x5d/0x90 [ 2827.731562][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2827.731685][ T44] register_pernet_subsys+0x19/0x40 [ 2827.731794][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2827.731905][ T44] do_one_initcall+0xa3/0x2e0 [ 2827.732019][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2827.732154][ T44] ? kasan_unpoison+0x44/0x70 [ 2827.732262][ T44] do_init_module+0x27b/0x820 [ 2827.732376][ T44] ? free_module+0x450/0x450 [ 2827.732484][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2827.732593][ T44] init_module_from_file+0x15d/0x180 [ 2827.732707][ T44] ? __do_sys_init_module+0x200/0x200 [ 2827.732823][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.732932][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2827.733041][ T44] idempotent_init_module+0x226/0x760 [ 2827.733150][ T44] ? init_module_from_file+0x180/0x180 [ 2827.733261][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.733371][ T44] ? cap_capable+0x181/0x3f0 [ 2827.733481][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2827.733592][ T44] do_syscall_64+0xbd/0xfc0 [ 2827.733702][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2827.733834][ T44] RIP: 0033:0x7fee353a334d [ 2827.733948][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2827.734111][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2827.734273][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2827.734439][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2827.734598][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2827.734764][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2827.734931][ T44] [ 2827.735013][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2827.735192][ T44] [ 2827.735192][ T44] Showing all locks held in the system: [ 2827.735349][ T44] 5 locks held by kworker/u16:0/12: [ 2827.735453][ T44] 1 lock held by khungtaskd/44: [ 2827.735557][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2827.735858][ T44] 2 locks held by kworker/u17:1/45: [ 2827.735960][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2827.736169][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2827.736408][ T44] 1 lock held by modprobe/28745: [ 2827.736510][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2827.736721][ T44] [ 2827.736773][ T44] ============================================= [ 2827.736773][ T44] [ 2950.608657][ T44] INFO: task modprobe:28745 blocked for more than 983 seconds. [ 2950.608944][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2950.609148][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2950.609386][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 2950.609695][ T44] Call Trace: [ 2950.609800][ T44] [ 2950.609873][ T44] __schedule+0xae2/0x1720 [ 2950.610011][ T44] ? io_schedule_timeout+0x130/0x130 [ 2950.610145][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.610267][ T44] schedule+0xd4/0x260 [ 2950.610355][ T44] schedule_preempt_disabled+0x18/0x30 [ 2950.610467][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2950.610582][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2950.610704][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.610816][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.610931][ T44] ? register_pernet_subsys+0x19/0x40 [ 2950.611044][ T44] ? lock_acquire+0x10a/0x150 [ 2950.611157][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 2950.611270][ T44] down_write+0x1c6/0x1f0 [ 2950.611358][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2950.611493][ T44] ? static_obj+0x5d/0x90 [ 2950.611582][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 2950.611710][ T44] register_pernet_subsys+0x19/0x40 [ 2950.611819][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 2950.611929][ T44] do_one_initcall+0xa3/0x2e0 [ 2950.612039][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2950.612176][ T44] ? kasan_unpoison+0x44/0x70 [ 2950.612285][ T44] do_init_module+0x27b/0x820 [ 2950.612399][ T44] ? free_module+0x450/0x450 [ 2950.612509][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2950.612618][ T44] init_module_from_file+0x15d/0x180 [ 2950.612730][ T44] ? __do_sys_init_module+0x200/0x200 [ 2950.612845][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.612952][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2950.613060][ T44] idempotent_init_module+0x226/0x760 [ 2950.613169][ T44] ? init_module_from_file+0x180/0x180 [ 2950.613282][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.613393][ T44] ? cap_capable+0x181/0x3f0 [ 2950.613503][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2950.613612][ T44] do_syscall_64+0xbd/0xfc0 [ 2950.613726][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2950.613859][ T44] RIP: 0033:0x7fee353a334d [ 2950.613972][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2950.614137][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 2950.614298][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 2950.614462][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 2950.614623][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 2950.614791][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 2950.614961][ T44] [ 2950.615046][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 2950.615222][ T44] [ 2950.615222][ T44] Showing all locks held in the system: [ 2950.615379][ T44] 4 locks held by kworker/u16:0/12: [ 2950.615487][ T44] 1 lock held by khungtaskd/44: [ 2950.615590][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2950.615886][ T44] 2 locks held by kworker/u17:1/45: [ 2950.615990][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 2950.616200][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 2950.616444][ T44] 1 lock held by modprobe/28745: [ 2950.616546][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2950.616759][ T44] [ 2950.616812][ T44] ============================================= [ 2950.616812][ T44] [ 3073.488658][ T44] INFO: task modprobe:28745 blocked for more than 1105 seconds. [ 3073.488929][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 3073.489143][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3073.489393][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 3073.489701][ T44] Call Trace: [ 3073.489801][ T44] [ 3073.489867][ T44] __schedule+0xae2/0x1720 [ 3073.490009][ T44] ? io_schedule_timeout+0x130/0x130 [ 3073.490139][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.490274][ T44] schedule+0xd4/0x260 [ 3073.490372][ T44] schedule_preempt_disabled+0x18/0x30 [ 3073.490495][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3073.490611][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3073.490733][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.490844][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.490955][ T44] ? register_pernet_subsys+0x19/0x40 [ 3073.491066][ T44] ? lock_acquire+0x10a/0x150 [ 3073.491177][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 3073.491292][ T44] down_write+0x1c6/0x1f0 [ 3073.491382][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3073.491519][ T44] ? static_obj+0x5d/0x90 [ 3073.491600][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 3073.491724][ T44] register_pernet_subsys+0x19/0x40 [ 3073.491832][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 3073.491943][ T44] do_one_initcall+0xa3/0x2e0 [ 3073.492051][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3073.492190][ T44] ? kasan_unpoison+0x44/0x70 [ 3073.492299][ T44] do_init_module+0x27b/0x820 [ 3073.492414][ T44] ? free_module+0x450/0x450 [ 3073.492523][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3073.492632][ T44] init_module_from_file+0x15d/0x180 [ 3073.492747][ T44] ? __do_sys_init_module+0x200/0x200 [ 3073.492861][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.492972][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3073.493080][ T44] idempotent_init_module+0x226/0x760 [ 3073.493189][ T44] ? init_module_from_file+0x180/0x180 [ 3073.493300][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.493412][ T44] ? cap_capable+0x181/0x3f0 [ 3073.493520][ T44] __x64_sys_finit_module+0xcd/0x160 [ 3073.493628][ T44] do_syscall_64+0xbd/0xfc0 [ 3073.493744][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3073.493877][ T44] RIP: 0033:0x7fee353a334d [ 3073.493990][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3073.494157][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 3073.494323][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 3073.494487][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 3073.494658][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 3073.494820][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 3073.494987][ T44] [ 3073.495068][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 3073.495245][ T44] [ 3073.495245][ T44] Showing all locks held in the system: [ 3073.495401][ T44] 4 locks held by kworker/u16:0/12: [ 3073.495508][ T44] 1 lock held by khungtaskd/44: [ 3073.495610][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3073.495909][ T44] 2 locks held by kworker/u17:1/45: [ 3073.496014][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 3073.496223][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 3073.496462][ T44] 1 lock held by modprobe/28745: [ 3073.496565][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3073.496776][ T44] [ 3073.496829][ T44] ============================================= [ 3073.496829][ T44] [ 3196.367665][ T44] INFO: task modprobe:28745 blocked for more than 1228 seconds. [ 3196.367936][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 3196.368135][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3196.368370][ T44] task:modprobe state:D stack:24872 pid:28745 tgid:28745 ppid:45 task_flags:0x400100 flags:0x00080000 [ 3196.368678][ T44] Call Trace: [ 3196.368766][ T44] [ 3196.368827][ T44] __schedule+0xae2/0x1720 [ 3196.368947][ T44] ? io_schedule_timeout+0x130/0x130 [ 3196.369064][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.369184][ T44] schedule+0xd4/0x260 [ 3196.369273][ T44] schedule_preempt_disabled+0x18/0x30 [ 3196.369392][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3196.369510][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3196.369635][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.369752][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3196.369862][ T44] ? register_pernet_subsys+0x19/0x40 [ 3196.369977][ T44] ? lock_acquire+0x10a/0x150 [ 3196.370089][ T44] ? nf_nat_cleanup+0x650/0x650 [nf_nat] [ 3196.370202][ T44] down_write+0x1c6/0x1f0 [ 3196.370286][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3196.370429][ T44] ? static_obj+0x5d/0x90 [ 3196.370513][ T44] ? lockdep_init_map_type+0x5c/0x250 [ 3196.370624][ T44] register_pernet_subsys+0x19/0x40 [ 3196.370752][ T44] nf_nat_init+0xce/0x1000 [nf_nat] [ 3196.370861][ T44] do_one_initcall+0xa3/0x2e0 [ 3196.370969][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3196.371106][ T44] ? kasan_unpoison+0x44/0x70 [ 3196.371214][ T44] do_init_module+0x27b/0x820 [ 3196.371326][ T44] ? free_module+0x450/0x450 [ 3196.371436][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3196.371549][ T44] init_module_from_file+0x15d/0x180 [ 3196.371663][ T44] ? __do_sys_init_module+0x200/0x200 [ 3196.371777][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.371885][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3196.371993][ T44] idempotent_init_module+0x226/0x760 [ 3196.372102][ T44] ? init_module_from_file+0x180/0x180 [ 3196.372213][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.372322][ T44] ? cap_capable+0x181/0x3f0 [ 3196.372432][ T44] __x64_sys_finit_module+0xcd/0x160 [ 3196.372540][ T44] do_syscall_64+0xbd/0xfc0 [ 3196.372652][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3196.372785][ T44] RIP: 0033:0x7fee353a334d [ 3196.372900][ T44] RSP: 002b:00007fffb6fc5858 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3196.373064][ T44] RAX: ffffffffffffffda RBX: 0000559afa2f2dc0 RCX: 00007fee353a334d [ 3196.373226][ T44] RDX: 0000000000000000 RSI: 0000559abfe3df2e RDI: 0000000000000000 [ 3196.373389][ T44] RBP: 00007fffb6fc58f0 R08: 0000000000000000 R09: 0000000000000000 [ 3196.373555][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559abfe3df2e [ 3196.373723][ T44] R13: 0000000000040000 R14: 0000559afa2f2d70 R15: 0000559afa2f2570 [ 3196.373889][ T44] [ 3196.373970][ T44] INFO: task modprobe:28745 is blocked on an rw-semaphore, but the owner is not found. [ 3196.374146][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 3196.374325][ T44] [ 3196.374325][ T44] Showing all locks held in the system: [ 3196.374480][ T44] 4 locks held by kworker/u16:0/12: [ 3196.374584][ T44] 1 lock held by khungtaskd/44: [ 3196.374777][ T44] #0: ffffffff8fb85140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3196.374986][ T44] 2 locks held by kworker/u17:1/45: [ 3196.375090][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 3196.375302][ T44] #1: ffa000000031fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 3196.375543][ T44] 1 lock held by modprobe/28745: [ 3196.375650][ T44] #0: ffffffff90285e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3196.375860][ T44] [ 3196.375912][ T44] ============================================= [ 3196.375912][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr