[ 1790.358299][ T12] ------------[ cut here ]------------
[ 1790.358723][ T12] conntrack cleanup blocked for 60s
[ 1790.358726][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack], CPU#2: kworker/u16:0/12
[ 1790.359322][ T12] Modules linked in: vxlan(+) xt_conntrack nft_chain_nat xt_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 cls_bpf netdevsim psample unix_diag act_gact cls_matchall sch_ingress xfrm_user xt_policy ipt_REJECT nf_reject_ipv4 nft_compat nf_tables
[ 1790.360021][ T12] CPU: 2 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.19.0-rc1-virtme #1 PREEMPT(full)
[ 1790.360349][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 1790.360563][ T12] Workqueue: netns cleanup_net
[ 1790.360774][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack]
[ 1790.361032][ T12] Code: 00 48 8b 54 24 18 48 8b 05 b3 08 2d cc 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 de 21 cb e9 25 fd ff ff 48 8d 3d 46 c6 e7 ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df
[ 1790.361563][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297
[ 1790.361786][ T12] RAX: 000000010016bbde RBX: ffa00000000c7af8 RCX: 0000000000000001
[ 1790.362032][ T12] RDX: 000000010016bbdd RSI: 0000000000000004 RDI: ffffffffc05bd030
[ 1790.362291][ T12] RBP: 0000000000000001 R08: ffffffffc07407ba R09: ffe21c0000a986c4
[ 1790.362530][ T12] R10: ffe21c0000a986c5 R11: 0000000000000001 R12: dffffc0000000000
[ 1790.362775][ T12] R13: ff110000054c3620 R14: fffffbfff80b9407 R15: ff11000013b20400
[ 1790.363017][ T12] FS: 0000000000000000(0000) GS:ff110000de2f3000(0000) knlGS:0000000000000000
[ 1790.363312][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1790.363904][ T12] CR2: 000055cb74ca41f8 CR3: 0000000033148004 CR4: 0000000000771ef0
[ 1790.364145][ T12] PKRU: 55555554
[ 1790.364305][ T12] Call Trace:
[ 1790.364452][ T12]
[ 1790.364569][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 1790.364795][ T12] ? kfree+0x119/0x580
[ 1790.364940][ T12] ? lock_acquire+0x10a/0x150
[ 1790.365124][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 1790.365350][ T12] ops_undo_list+0x2d7/0x8f0
[ 1790.365537][ T12] ? netns_install+0x280/0x280
[ 1790.365725][ T12] cleanup_net+0x3bc/0x890
[ 1790.365900][ T12] ? process_one_work+0xd16/0x1390
[ 1790.366079][ T12] ? net_passive_dec+0x190/0x190
[ 1790.366255][ T12] ? rcu_is_watching+0x15/0xd0
[ 1790.366452][ T12] ? process_one_work+0xd16/0x1390
[ 1790.366627][ T12] ? lock_acquire+0x10a/0x150
[ 1790.366811][ T12] ? rcu_is_watching+0x15/0xd0
[ 1790.366989][ T12] process_one_work+0xd57/0x1390
[ 1790.367170][ T12] ? pwq_dec_nr_in_flight+0x700/0x700
[ 1790.367355][ T12] ? lock_acquire.part.0+0xbc/0x260
[ 1790.367534][ T12] ? assign_work+0x152/0x380
[ 1790.367716][ T12] worker_thread+0x4d6/0xd40
[ 1790.367897][ T12] ? process_one_work+0x1390/0x1390
[ 1790.368072][ T12] kthread+0x355/0x5b0
[ 1790.368227][ T12] ? kthread_is_per_cpu+0xe0/0xe0
[ 1790.368417][ T12] ? __lock_release.isra.0+0x59/0x170
[ 1790.368595][ T12] ? rcu_is_watching+0x15/0xd0
[ 1790.368778][ T12] ? kthread_is_per_cpu+0xe0/0xe0
[ 1790.368959][ T12] ret_from_fork+0x3fb/0x510
[ 1790.369138][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 1790.369356][ T12] ? __switch_to+0x53c/0xd00
[ 1790.369536][ T12] ? kthread_is_per_cpu+0xe0/0xe0
[ 1790.369720][ T12] ret_from_fork_asm+0x11/0x20
[ 1790.369900][ T12]
[ 1790.370044][ T12] irq event stamp: 8647063
[ 1790.370220][ T12] hardirqs last enabled at (8647073): [] __up_console_sem+0x5a/0x70
[ 1790.370503][ T12] hardirqs last disabled at (8647084): [] __up_console_sem+0x3f/0x70
[ 1790.370778][ T12] softirqs last enabled at (8646570): [] handle_softirqs+0x56f/0x840
[ 1790.371044][ T12] softirqs last disabled at (8646563): [] irq_exit_rcu+0xa2/0xf0
[ 1790.371332][ T12] ---[ end trace 0000000000000000 ]---
[ 1967.545327][ T44] INFO: task modprobe:23603 blocked for more than 122 seconds.
[ 1967.545806][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 1967.546036][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1967.546319][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 1967.546674][ T44] Call Trace:
[ 1967.546851][ T44]
[ 1967.546969][ T44] __schedule+0xae2/0x1720
[ 1967.547151][ T44] ? io_schedule_timeout+0x130/0x130
[ 1967.547339][ T44] ? __lock_release.isra.0+0x59/0x170
[ 1967.547518][ T44] schedule+0xd4/0x260
[ 1967.547658][ T44] schedule_preempt_disabled+0x18/0x30
[ 1967.547828][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 1967.548003][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 1967.548179][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.548366][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 1967.548540][ T44] ? register_pernet_subsys+0x19/0x40
[ 1967.548712][ T44] ? lock_acquire+0x10a/0x150
[ 1967.548886][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 1967.549089][ T44] down_write+0x1c6/0x1f0
[ 1967.549225][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 1967.549440][ T44] register_pernet_subsys+0x19/0x40
[ 1967.549625][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 1967.549798][ T44] do_one_initcall+0xa3/0x2e0
[ 1967.549963][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 1967.550151][ T44] ? kasan_unpoison+0x44/0x70
[ 1967.550326][ T44] do_init_module+0x27b/0x820
[ 1967.550884][ T44] ? free_module+0x450/0x450
[ 1967.551049][ T44] ? backing_file_read_iter+0x50e/0x730
[ 1967.551210][ T44] init_module_from_file+0x15d/0x180
[ 1967.551380][ T44] ? __do_sys_init_module+0x200/0x200
[ 1967.551547][ T44] ? __lock_release.isra.0+0x59/0x170
[ 1967.551707][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 1967.551865][ T44] idempotent_init_module+0x226/0x760
[ 1967.552031][ T44] ? init_module_from_file+0x180/0x180
[ 1967.552194][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.552368][ T44] ? cap_capable+0x181/0x3f0
[ 1967.552537][ T44] __x64_sys_finit_module+0xcd/0x160
[ 1967.552696][ T44] do_syscall_64+0xbd/0xfc0
[ 1967.552856][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1967.553039][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 1967.553216][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 1967.553448][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 1967.553671][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 1967.553883][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 1967.554105][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 1967.554329][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 1967.554549][ T44]
[ 1967.554683][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 1967.554959][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 1967.555294][ T44] Workqueue: netns cleanup_net
[ 1967.555452][ T44] Call Trace:
[ 1967.555589][ T44]
[ 1967.555694][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 1967.555894][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.556052][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 1967.556216][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 1967.556419][ T44] ? lock_acquire+0x10a/0x150
[ 1967.556584][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.556741][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.556898][ T44] ? trace_contention_end+0xd6/0x110
[ 1967.557056][ T44] ? __mutex_lock+0x285/0x1f20
[ 1967.557217][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 1967.557425][ T44] ? ww_mutex_lock+0x160/0x160
[ 1967.557591][ T44] ? __lock_acquire+0x577/0xc10
[ 1967.557753][ T44] ? __mutex_unlock_slowpath+0x14b/0x540
[ 1967.557913][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 1967.558073][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 1967.558261][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 1967.558467][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 1967.558696][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 1967.558911][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 1967.559096][ T44] ? kfree+0x119/0x580
[ 1967.559226][ T44] ? lock_acquire+0x10a/0x150
[ 1967.559400][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 1967.559609][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 1967.559774][ T44] ? netns_install+0x280/0x280
[ 1967.559934][ T44] ? cleanup_net+0x3bc/0x890
[ 1967.560092][ T44] ? process_one_work+0xd16/0x1390
[ 1967.560251][ T44] ? net_passive_dec+0x190/0x190
[ 1967.560425][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.560592][ T44] ? process_one_work+0xd16/0x1390
[ 1967.560756][ T44] ? lock_acquire+0x10a/0x150
[ 1967.560914][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.561074][ T44] ? process_one_work+0xd57/0x1390
[ 1967.561234][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 1967.561405][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 1967.561573][ T44] ? assign_work+0x152/0x380
[ 1967.561733][ T44] ? worker_thread+0x4d6/0xd40
[ 1967.561894][ T44] ? process_one_work+0x1390/0x1390
[ 1967.562054][ T44] ? kthread+0x355/0x5b0
[ 1967.562188][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 1967.562363][ T44] ? __lock_release.isra.0+0x59/0x170
[ 1967.562531][ T44] ? rcu_is_watching+0x15/0xd0
[ 1967.562691][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 1967.562850][ T44] ? ret_from_fork+0x3fb/0x510
[ 1967.563008][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 1967.563197][ T44] ? __switch_to+0x53c/0xd00
[ 1967.563371][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 1967.563536][ T44] ? ret_from_fork_asm+0x11/0x20
[ 1967.563702][ T44]
[ 1967.563837][ T44]
[ 1967.563837][ T44] Showing all locks held in the system:
[ 1967.564053][ T44] 4 locks held by kworker/u16:0/12:
[ 1967.564211][ T44] 1 lock held by khungtaskd/44:
[ 1967.564379][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1967.564653][ T44] 2 locks held by kworker/u17:2/4541:
[ 1967.564806][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 1967.565065][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 1967.565383][ T44] 1 lock held by modprobe/23603:
[ 1967.565545][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 1967.565815][ T44]
[ 1967.565920][ T44] =============================================
[ 1967.565920][ T44]
[ 2090.425312][ T44] INFO: task modprobe:23603 blocked for more than 245 seconds.
[ 2090.425656][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2090.425904][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2090.426182][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2090.426534][ T44] Call Trace:
[ 2090.426677][ T44]
[ 2090.426792][ T44] __schedule+0xae2/0x1720
[ 2090.426974][ T44] ? io_schedule_timeout+0x130/0x130
[ 2090.427149][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2090.427347][ T44] schedule+0xd4/0x260
[ 2090.427489][ T44] schedule_preempt_disabled+0x18/0x30
[ 2090.427655][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2090.427827][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2090.428004][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.428191][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2090.428368][ T44] ? register_pernet_subsys+0x19/0x40
[ 2090.428534][ T44] ? lock_acquire+0x10a/0x150
[ 2090.428702][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2090.428911][ T44] down_write+0x1c6/0x1f0
[ 2090.429053][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2090.429261][ T44] register_pernet_subsys+0x19/0x40
[ 2090.429456][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2090.429628][ T44] do_one_initcall+0xa3/0x2e0
[ 2090.429793][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2090.429984][ T44] ? kasan_unpoison+0x44/0x70
[ 2090.430149][ T44] do_init_module+0x27b/0x820
[ 2090.430329][ T44] ? free_module+0x450/0x450
[ 2090.430487][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2090.430651][ T44] init_module_from_file+0x15d/0x180
[ 2090.430809][ T44] ? __do_sys_init_module+0x200/0x200
[ 2090.430973][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2090.431132][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2090.431312][ T44] idempotent_init_module+0x226/0x760
[ 2090.431473][ T44] ? init_module_from_file+0x180/0x180
[ 2090.431643][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.431803][ T44] ? cap_capable+0x181/0x3f0
[ 2090.431963][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2090.432128][ T44] do_syscall_64+0xbd/0xfc0
[ 2090.432309][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2090.432491][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2090.432654][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2090.432872][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2090.433086][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2090.433314][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2090.433526][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2090.433740][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2090.433955][ T44]
[ 2090.434084][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2090.434377][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2090.434692][ T44] Workqueue: netns cleanup_net
[ 2090.434849][ T44] Call Trace:
[ 2090.434977][ T44]
[ 2090.435081][ T44] ? rcu_read_lock_sched_held+0x3c/0x70
[ 2090.435247][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.435418][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2090.435576][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.435768][ T44] ? lock_acquire+0x10a/0x150
[ 2090.435926][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.436083][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.436247][ T44] ? trace_contention_end+0xd6/0x110
[ 2090.436423][ T44] ? __mutex_lock+0x285/0x1f20
[ 2090.436586][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.436774][ T44] ? ww_mutex_lock+0x160/0x160
[ 2090.436939][ T44] ? find_held_lock+0x2b/0x80
[ 2090.437098][ T44] ? __lock_acquire+0x577/0xc10
[ 2090.437260][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2090.437433][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.437617][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.437803][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2090.438026][ T44] ? schedule+0xd4/0x260
[ 2090.438157][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2090.438390][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2090.438578][ T44] ? kfree+0x119/0x580
[ 2090.438711][ T44] ? lock_acquire+0x10a/0x150
[ 2090.438873][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2090.439060][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2090.439230][ T44] ? netns_install+0x280/0x280
[ 2090.439404][ T44] ? cleanup_net+0x3bc/0x890
[ 2090.439561][ T44] ? process_one_work+0xd16/0x1390
[ 2090.439720][ T44] ? net_passive_dec+0x190/0x190
[ 2090.439877][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.440034][ T44] ? process_one_work+0xd16/0x1390
[ 2090.440198][ T44] ? lock_acquire+0x10a/0x150
[ 2090.440377][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.440536][ T44] ? process_one_work+0xd57/0x1390
[ 2090.440699][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2090.440859][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2090.441019][ T44] ? assign_work+0x152/0x380
[ 2090.441180][ T44] ? worker_thread+0x4d6/0xd40
[ 2090.441362][ T44] ? process_one_work+0x1390/0x1390
[ 2090.441524][ T44] ? kthread+0x355/0x5b0
[ 2090.441658][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2090.441821][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2090.441980][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.442138][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2090.442317][ T44] ? ret_from_fork+0x3fb/0x510
[ 2090.442480][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2090.442665][ T44] ? __switch_to+0x53c/0xd00
[ 2090.442828][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2090.442987][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2090.443151][ T44]
[ 2090.443301][ T44]
[ 2090.443301][ T44] Showing all locks held in the system:
[ 2090.443507][ T44] 4 locks held by kworker/u16:0/12:
[ 2090.443661][ T44] 1 lock held by khungtaskd/44:
[ 2090.443816][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2090.444085][ T44] 2 locks held by kworker/u17:2/4541:
[ 2090.444247][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2090.444524][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2090.444813][ T44] 1 lock held by modprobe/23603:
[ 2090.444966][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2090.445227][ T44]
[ 2090.445341][ T44] =============================================
[ 2090.445341][ T44]
[ 2213.305304][ T44] INFO: task modprobe:23603 blocked for more than 368 seconds.
[ 2213.305630][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2213.305884][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2213.306158][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2213.306516][ T44] Call Trace:
[ 2213.306655][ T44]
[ 2213.306767][ T44] __schedule+0xae2/0x1720
[ 2213.306946][ T44] ? io_schedule_timeout+0x130/0x130
[ 2213.307124][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2213.307316][ T44] schedule+0xd4/0x260
[ 2213.307461][ T44] schedule_preempt_disabled+0x18/0x30
[ 2213.307629][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2213.307799][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2213.307975][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.308145][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2213.308336][ T44] ? register_pernet_subsys+0x19/0x40
[ 2213.308511][ T44] ? lock_acquire+0x10a/0x150
[ 2213.308681][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2213.308889][ T44] down_write+0x1c6/0x1f0
[ 2213.309035][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2213.309241][ T44] register_pernet_subsys+0x19/0x40
[ 2213.309430][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2213.309597][ T44] do_one_initcall+0xa3/0x2e0
[ 2213.309760][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2213.309954][ T44] ? kasan_unpoison+0x44/0x70
[ 2213.310121][ T44] do_init_module+0x27b/0x820
[ 2213.310308][ T44] ? free_module+0x450/0x450
[ 2213.310470][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2213.310635][ T44] init_module_from_file+0x15d/0x180
[ 2213.310793][ T44] ? __do_sys_init_module+0x200/0x200
[ 2213.310967][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2213.311126][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2213.311297][ T44] idempotent_init_module+0x226/0x760
[ 2213.311459][ T44] ? init_module_from_file+0x180/0x180
[ 2213.311621][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.311788][ T44] ? cap_capable+0x181/0x3f0
[ 2213.311953][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2213.312114][ T44] do_syscall_64+0xbd/0xfc0
[ 2213.312293][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2213.312479][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2213.312641][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2213.312857][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2213.313076][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2213.313299][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2213.313509][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2213.313720][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2213.313945][ T44]
[ 2213.314075][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2213.314364][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2213.314679][ T44] Workqueue: netns cleanup_net
[ 2213.314836][ T44] Call Trace:
[ 2213.314970][ T44]
[ 2213.315073][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.315264][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.315437][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2213.315593][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.315780][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.315950][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.316112][ T44] ? trace_contention_end+0xd6/0x110
[ 2213.316285][ T44] ? __mutex_lock+0x285/0x1f20
[ 2213.316444][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.316631][ T44] ? ww_mutex_lock+0x160/0x160
[ 2213.316789][ T44] ? __lock_acquire+0x577/0xc10
[ 2213.316958][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2213.317124][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2213.317299][ T44] ? nf_ct_iterate_cleanup+0x8e/0x350 [nf_conntrack]
[ 2213.317485][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.317673][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.317859][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2213.318084][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2213.318315][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2213.318501][ T44] ? kfree+0x119/0x580
[ 2213.318632][ T44] ? lock_acquire+0x10a/0x150
[ 2213.318791][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2213.318981][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2213.319149][ T44] ? netns_install+0x280/0x280
[ 2213.319322][ T44] ? cleanup_net+0x3bc/0x890
[ 2213.319485][ T44] ? process_one_work+0xd16/0x1390
[ 2213.319644][ T44] ? net_passive_dec+0x190/0x190
[ 2213.319807][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.319970][ T44] ? process_one_work+0xd16/0x1390
[ 2213.320130][ T44] ? lock_acquire+0x10a/0x150
[ 2213.320302][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.320466][ T44] ? process_one_work+0xd57/0x1390
[ 2213.320627][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2213.320784][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2213.320950][ T44] ? assign_work+0x152/0x380
[ 2213.321112][ T44] ? worker_thread+0x4d6/0xd40
[ 2213.321292][ T44] ? process_one_work+0x1390/0x1390
[ 2213.321450][ T44] ? kthread+0x355/0x5b0
[ 2213.321588][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2213.321747][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2213.321904][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.322070][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2213.322230][ T44] ? ret_from_fork+0x3fb/0x510
[ 2213.322400][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2213.322582][ T44] ? __switch_to+0x53c/0xd00
[ 2213.322745][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2213.322907][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2213.323078][ T44]
[ 2213.323208][ T44]
[ 2213.323208][ T44] Showing all locks held in the system:
[ 2213.323419][ T44] 4 locks held by kworker/u16:0/12:
[ 2213.323573][ T44] 1 lock held by khungtaskd/44:
[ 2213.323726][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2213.324003][ T44] 2 locks held by kworker/u17:2/4541:
[ 2213.324153][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2213.324426][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2213.324719][ T44] 1 lock held by modprobe/23603:
[ 2213.324876][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2213.325137][ T44]
[ 2213.325236][ T44] =============================================
[ 2213.325236][ T44]
[ 2336.184294][ T44] INFO: task modprobe:23603 blocked for more than 491 seconds.
[ 2336.184639][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2336.184912][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2336.185200][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2336.185594][ T44] Call Trace:
[ 2336.185740][ T44]
[ 2336.185861][ T44] __schedule+0xae2/0x1720
[ 2336.186040][ T44] ? io_schedule_timeout+0x130/0x130
[ 2336.186222][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2336.186410][ T44] schedule+0xd4/0x260
[ 2336.186555][ T44] schedule_preempt_disabled+0x18/0x30
[ 2336.186729][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2336.186905][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2336.187083][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.187255][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2336.187442][ T44] ? register_pernet_subsys+0x19/0x40
[ 2336.187614][ T44] ? lock_acquire+0x10a/0x150
[ 2336.187785][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2336.187992][ T44] down_write+0x1c6/0x1f0
[ 2336.188132][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2336.188354][ T44] register_pernet_subsys+0x19/0x40
[ 2336.188527][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2336.188706][ T44] do_one_initcall+0xa3/0x2e0
[ 2336.188872][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2336.189057][ T44] ? kasan_unpoison+0x44/0x70
[ 2336.189220][ T44] do_init_module+0x27b/0x820
[ 2336.189395][ T44] ? free_module+0x450/0x450
[ 2336.189556][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2336.189716][ T44] init_module_from_file+0x15d/0x180
[ 2336.189890][ T44] ? __do_sys_init_module+0x200/0x200
[ 2336.190057][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2336.190215][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2336.190387][ T44] idempotent_init_module+0x226/0x760
[ 2336.190547][ T44] ? init_module_from_file+0x180/0x180
[ 2336.190710][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.190881][ T44] ? cap_capable+0x181/0x3f0
[ 2336.191040][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2336.191199][ T44] do_syscall_64+0xbd/0xfc0
[ 2336.191373][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2336.191556][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2336.191721][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2336.191939][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2336.192154][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2336.192383][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2336.192596][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2336.192830][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2336.193053][ T44]
[ 2336.193184][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2336.193468][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2336.193778][ T44] Workqueue: netns cleanup_net
[ 2336.193947][ T44] Call Trace:
[ 2336.194076][ T44]
[ 2336.194179][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.194383][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.194539][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2336.194696][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.194889][ T44] ? lock_acquire+0x10a/0x150
[ 2336.195050][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.195205][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.195380][ T44] ? trace_contention_end+0xd6/0x110
[ 2336.195539][ T44] ? __mutex_lock+0x285/0x1f20
[ 2336.195699][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.195892][ T44] ? ww_mutex_lock+0x160/0x160
[ 2336.196051][ T44] ? __lock_acquire+0x577/0xc10
[ 2336.196213][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2336.196390][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.196577][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.196764][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2336.196984][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2336.197199][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2336.197403][ T44] ? kfree+0x119/0x580
[ 2336.197535][ T44] ? lock_acquire+0x10a/0x150
[ 2336.197698][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2336.197899][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2336.198061][ T44] ? netns_install+0x280/0x280
[ 2336.198223][ T44] ? cleanup_net+0x3bc/0x890
[ 2336.198396][ T44] ? process_one_work+0xd16/0x1390
[ 2336.198561][ T44] ? net_passive_dec+0x190/0x190
[ 2336.198723][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.198890][ T44] ? process_one_work+0xd16/0x1390
[ 2336.199047][ T44] ? lock_acquire+0x10a/0x150
[ 2336.199205][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.199374][ T44] ? process_one_work+0xd57/0x1390
[ 2336.199536][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2336.199698][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2336.199864][ T44] ? assign_work+0x152/0x380
[ 2336.200024][ T44] ? worker_thread+0x4d6/0xd40
[ 2336.200186][ T44] ? process_one_work+0x1390/0x1390
[ 2336.200358][ T44] ? kthread+0x355/0x5b0
[ 2336.200492][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2336.200650][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2336.200822][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.200981][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2336.201142][ T44] ? ret_from_fork+0x3fb/0x510
[ 2336.201314][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2336.201500][ T44] ? __switch_to+0x53c/0xd00
[ 2336.201659][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2336.201823][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2336.201986][ T44]
[ 2336.202119][ T44]
[ 2336.202119][ T44] Showing all locks held in the system:
[ 2336.202335][ T44] 3 locks held by kworker/u16:0/12:
[ 2336.202486][ T44] 1 lock held by khungtaskd/44:
[ 2336.202641][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2336.202910][ T44] 2 locks held by kworker/u17:2/4541:
[ 2336.203060][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2336.203336][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2336.203628][ T44] 1 lock held by modprobe/23603:
[ 2336.203782][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2336.204045][ T44]
[ 2336.204150][ T44] =============================================
[ 2336.204150][ T44]
[ 2459.064294][ T44] INFO: task modprobe:23603 blocked for more than 614 seconds.
[ 2459.064613][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2459.064864][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2459.065130][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2459.065498][ T44] Call Trace:
[ 2459.065651][ T44]
[ 2459.065780][ T44] __schedule+0xae2/0x1720
[ 2459.065963][ T44] ? io_schedule_timeout+0x130/0x130
[ 2459.066136][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2459.066321][ T44] schedule+0xd4/0x260
[ 2459.066461][ T44] schedule_preempt_disabled+0x18/0x30
[ 2459.066637][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2459.066818][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2459.066989][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.067158][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2459.067333][ T44] ? register_pernet_subsys+0x19/0x40
[ 2459.067501][ T44] ? lock_acquire+0x10a/0x150
[ 2459.067679][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2459.067889][ T44] down_write+0x1c6/0x1f0
[ 2459.068029][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2459.068229][ T44] register_pernet_subsys+0x19/0x40
[ 2459.068410][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2459.068579][ T44] do_one_initcall+0xa3/0x2e0
[ 2459.068752][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2459.068946][ T44] ? kasan_unpoison+0x44/0x70
[ 2459.069106][ T44] do_init_module+0x27b/0x820
[ 2459.069282][ T44] ? free_module+0x450/0x450
[ 2459.069441][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2459.069605][ T44] init_module_from_file+0x15d/0x180
[ 2459.069772][ T44] ? __do_sys_init_module+0x200/0x200
[ 2459.069937][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2459.070100][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2459.070258][ T44] idempotent_init_module+0x226/0x760
[ 2459.070433][ T44] ? init_module_from_file+0x180/0x180
[ 2459.070597][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.070765][ T44] ? cap_capable+0x181/0x3f0
[ 2459.070927][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2459.071087][ T44] do_syscall_64+0xbd/0xfc0
[ 2459.071251][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2459.071447][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2459.071609][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2459.071826][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2459.072036][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2459.072251][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2459.072475][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2459.072687][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2459.072913][ T44]
[ 2459.073042][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2459.073333][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2459.073644][ T44] Workqueue: netns cleanup_net
[ 2459.073803][ T44] Call Trace:
[ 2459.073932][ T44]
[ 2459.074035][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.074228][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.074401][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2459.074557][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.074748][ T44] ? lock_acquire+0x10a/0x150
[ 2459.074909][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.075065][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.075224][ T44] ? trace_contention_end+0xd6/0x110
[ 2459.075408][ T44] ? __mutex_lock+0x285/0x1f20
[ 2459.075572][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.075768][ T44] ? ww_mutex_lock+0x160/0x160
[ 2459.075926][ T44] ? __lock_acquire+0x577/0xc10
[ 2459.076085][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2459.076242][ T44] ? net_generic+0x1f/0x1e0 [nf_conntrack]
[ 2459.076442][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.076630][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.076822][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2459.077035][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2459.077247][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2459.077446][ T44] ? kfree+0x119/0x580
[ 2459.077578][ T44] ? lock_acquire+0x10a/0x150
[ 2459.077743][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2459.077935][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2459.078096][ T44] ? netns_install+0x280/0x280
[ 2459.078259][ T44] ? cleanup_net+0x3bc/0x890
[ 2459.078432][ T44] ? process_one_work+0xd16/0x1390
[ 2459.078600][ T44] ? net_passive_dec+0x190/0x190
[ 2459.078770][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.078927][ T44] ? process_one_work+0xd16/0x1390
[ 2459.079085][ T44] ? lock_acquire+0x10a/0x150
[ 2459.079242][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.079413][ T44] ? process_one_work+0xd57/0x1390
[ 2459.079577][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2459.079741][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2459.079905][ T44] ? assign_work+0x152/0x380
[ 2459.080064][ T44] ? worker_thread+0x4d6/0xd40
[ 2459.080227][ T44] ? process_one_work+0x1390/0x1390
[ 2459.080405][ T44] ? kthread+0x355/0x5b0
[ 2459.080536][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2459.080694][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2459.080858][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.081016][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2459.081180][ T44] ? ret_from_fork+0x3fb/0x510
[ 2459.081353][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2459.081543][ T44] ? __switch_to+0x53c/0xd00
[ 2459.081702][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2459.081866][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2459.082029][ T44]
[ 2459.082157][ T44]
[ 2459.082157][ T44] Showing all locks held in the system:
[ 2459.082376][ T44] 4 locks held by kworker/u16:0/12:
[ 2459.082529][ T44] 1 lock held by khungtaskd/44:
[ 2459.082677][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2459.082941][ T44] 2 locks held by kworker/u17:2/4541:
[ 2459.083093][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2459.083367][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2459.083651][ T44] 1 lock held by modprobe/23603:
[ 2459.083806][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2459.084062][ T44]
[ 2459.084161][ T44] =============================================
[ 2459.084161][ T44]
[ 2581.945295][ T44] INFO: task modprobe:23603 blocked for more than 737 seconds.
[ 2581.945634][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2581.945887][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2581.946153][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2581.946512][ T44] Call Trace:
[ 2581.946655][ T44]
[ 2581.946772][ T44] __schedule+0xae2/0x1720
[ 2581.946950][ T44] ? io_schedule_timeout+0x130/0x130
[ 2581.947126][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2581.947315][ T44] schedule+0xd4/0x260
[ 2581.947459][ T44] schedule_preempt_disabled+0x18/0x30
[ 2581.947630][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2581.947808][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2581.947986][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.948162][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2581.948339][ T44] ? register_pernet_subsys+0x19/0x40
[ 2581.948510][ T44] ? lock_acquire+0x10a/0x150
[ 2581.948681][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2581.948904][ T44] down_write+0x1c6/0x1f0
[ 2581.949044][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2581.949243][ T44] register_pernet_subsys+0x19/0x40
[ 2581.949421][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2581.949589][ T44] do_one_initcall+0xa3/0x2e0
[ 2581.949765][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2581.949958][ T44] ? kasan_unpoison+0x44/0x70
[ 2581.950129][ T44] do_init_module+0x27b/0x820
[ 2581.950306][ T44] ? free_module+0x450/0x450
[ 2581.950465][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2581.950625][ T44] init_module_from_file+0x15d/0x180
[ 2581.950791][ T44] ? __do_sys_init_module+0x200/0x200
[ 2581.950955][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2581.951119][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2581.951300][ T44] idempotent_init_module+0x226/0x760
[ 2581.951459][ T44] ? init_module_from_file+0x180/0x180
[ 2581.951625][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.951788][ T44] ? cap_capable+0x181/0x3f0
[ 2581.951948][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2581.952107][ T44] do_syscall_64+0xbd/0xfc0
[ 2581.952280][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2581.952475][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2581.952639][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2581.952861][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2581.953071][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2581.953293][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2581.953508][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2581.953716][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2581.953932][ T44]
[ 2581.954065][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2581.954364][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2581.954676][ T44] Workqueue: netns cleanup_net
[ 2581.954838][ T44] Call Trace:
[ 2581.954967][ T44]
[ 2581.955070][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.955260][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.955436][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2581.955596][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.955786][ T44] ? lock_acquire+0x10a/0x150
[ 2581.955945][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.956107][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.956282][ T44] ? trace_contention_end+0xd6/0x110
[ 2581.956445][ T44] ? __mutex_lock+0x285/0x1f20
[ 2581.956615][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.956813][ T44] ? ww_mutex_lock+0x160/0x160
[ 2581.956973][ T44] ? __lock_acquire+0x577/0xc10
[ 2581.957135][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2581.957314][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.957499][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.957689][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2581.957912][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2581.958132][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2581.958331][ T44] ? kfree+0x119/0x580
[ 2581.958462][ T44] ? lock_acquire+0x10a/0x150
[ 2581.958622][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2581.958826][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2581.958986][ T44] ? netns_install+0x280/0x280
[ 2581.959148][ T44] ? cleanup_net+0x3bc/0x890
[ 2581.959317][ T44] ? process_one_work+0xd16/0x1390
[ 2581.959474][ T44] ? net_passive_dec+0x190/0x190
[ 2581.959632][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.959798][ T44] ? process_one_work+0xd16/0x1390
[ 2581.959962][ T44] ? lock_acquire+0x10a/0x150
[ 2581.960123][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.960300][ T44] ? process_one_work+0xd57/0x1390
[ 2581.960465][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2581.960622][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2581.960786][ T44] ? assign_work+0x152/0x380
[ 2581.960950][ T44] ? worker_thread+0x4d6/0xd40
[ 2581.961114][ T44] ? process_one_work+0x1390/0x1390
[ 2581.961293][ T44] ? kthread+0x355/0x5b0
[ 2581.961424][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2581.961581][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2581.961738][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.961904][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2581.962063][ T44] ? ret_from_fork+0x3fb/0x510
[ 2581.962221][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2581.962426][ T44] ? __switch_to+0x53c/0xd00
[ 2581.962583][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2581.962740][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2581.962909][ T44]
[ 2581.963038][ T44]
[ 2581.963038][ T44] Showing all locks held in the system:
[ 2581.963239][ T44] 4 locks held by kworker/u16:0/12:
[ 2581.963402][ T44] 1 lock held by khungtaskd/44:
[ 2581.963557][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2581.963822][ T44] 2 locks held by kworker/u17:2/4541:
[ 2581.963975][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2581.964233][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2581.964537][ T44] 1 lock held by modprobe/23603:
[ 2581.964687][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2581.964956][ T44]
[ 2581.965055][ T44] =============================================
[ 2581.965055][ T44]
[ 2704.825292][ T44] INFO: task modprobe:23603 blocked for more than 860 seconds.
[ 2704.825625][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2704.825877][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2704.826156][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2704.826509][ T44] Call Trace:
[ 2704.826653][ T44]
[ 2704.826765][ T44] __schedule+0xae2/0x1720
[ 2704.826951][ T44] ? io_schedule_timeout+0x130/0x130
[ 2704.827127][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2704.827317][ T44] schedule+0xd4/0x260
[ 2704.827471][ T44] schedule_preempt_disabled+0x18/0x30
[ 2704.827640][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2704.827818][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2704.827991][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.828167][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2704.828351][ T44] ? register_pernet_subsys+0x19/0x40
[ 2704.828514][ T44] ? lock_acquire+0x10a/0x150
[ 2704.828680][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2704.828900][ T44] down_write+0x1c6/0x1f0
[ 2704.829035][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2704.829235][ T44] register_pernet_subsys+0x19/0x40
[ 2704.829415][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2704.829589][ T44] do_one_initcall+0xa3/0x2e0
[ 2704.829757][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2704.829952][ T44] ? kasan_unpoison+0x44/0x70
[ 2704.830121][ T44] do_init_module+0x27b/0x820
[ 2704.830307][ T44] ? free_module+0x450/0x450
[ 2704.830467][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2704.830636][ T44] init_module_from_file+0x15d/0x180
[ 2704.830801][ T44] ? __do_sys_init_module+0x200/0x200
[ 2704.830965][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2704.831126][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2704.831298][ T44] idempotent_init_module+0x226/0x760
[ 2704.831458][ T44] ? init_module_from_file+0x180/0x180
[ 2704.831625][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.831790][ T44] ? cap_capable+0x181/0x3f0
[ 2704.831953][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2704.832114][ T44] do_syscall_64+0xbd/0xfc0
[ 2704.832291][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2704.832477][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2704.832641][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2704.832866][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2704.833075][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2704.833298][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2704.833507][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2704.833722][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2704.833945][ T44]
[ 2704.834076][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2704.834363][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2704.834677][ T44] Workqueue: netns cleanup_net
[ 2704.834840][ T44] Call Trace:
[ 2704.834968][ T44]
[ 2704.835070][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.835259][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.835432][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2704.835591][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.835789][ T44] ? lock_acquire+0x10a/0x150
[ 2704.835944][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.836106][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.836265][ T44] ? trace_contention_end+0xd6/0x110
[ 2704.836437][ T44] ? __mutex_lock+0x285/0x1f20
[ 2704.836600][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.836793][ T44] ? ww_mutex_lock+0x160/0x160
[ 2704.836956][ T44] ? __lock_acquire+0x577/0xc10
[ 2704.837120][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2704.837297][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.837486][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.837676][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2704.837900][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2704.838124][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2704.838320][ T44] ? kfree+0x119/0x580
[ 2704.838451][ T44] ? lock_acquire+0x10a/0x150
[ 2704.838611][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2704.838810][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2704.838970][ T44] ? netns_install+0x280/0x280
[ 2704.839135][ T44] ? cleanup_net+0x3bc/0x890
[ 2704.839307][ T44] ? process_one_work+0xd16/0x1390
[ 2704.839466][ T44] ? net_passive_dec+0x190/0x190
[ 2704.839625][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.839788][ T44] ? process_one_work+0xd16/0x1390
[ 2704.839946][ T44] ? lock_acquire+0x10a/0x150
[ 2704.840105][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.840281][ T44] ? process_one_work+0xd57/0x1390
[ 2704.840447][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2704.840605][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2704.840769][ T44] ? assign_work+0x152/0x380
[ 2704.840935][ T44] ? worker_thread+0x4d6/0xd40
[ 2704.841098][ T44] ? process_one_work+0x1390/0x1390
[ 2704.841257][ T44] ? kthread+0x355/0x5b0
[ 2704.841402][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2704.841565][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2704.841726][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.841891][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2704.842050][ T44] ? ret_from_fork+0x3fb/0x510
[ 2704.842208][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2704.842409][ T44] ? __switch_to+0x53c/0xd00
[ 2704.842568][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2704.842731][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2704.842903][ T44]
[ 2704.843034][ T44]
[ 2704.843034][ T44] Showing all locks held in the system:
[ 2704.843240][ T44] 4 locks held by kworker/u16:0/12:
[ 2704.843405][ T44] 1 lock held by khungtaskd/44:
[ 2704.843557][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2704.843825][ T44] 2 locks held by kworker/u17:2/4541:
[ 2704.843982][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2704.844243][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2704.844543][ T44] 1 lock held by modprobe/23603:
[ 2704.844694][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2704.844962][ T44]
[ 2704.845062][ T44] =============================================
[ 2704.845062][ T44]
[ 2827.705292][ T44] INFO: task modprobe:23603 blocked for more than 983 seconds.
[ 2827.705609][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2827.705855][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2827.706128][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2827.706496][ T44] Call Trace:
[ 2827.706636][ T44]
[ 2827.706743][ T44] __schedule+0xae2/0x1720
[ 2827.706924][ T44] ? io_schedule_timeout+0x130/0x130
[ 2827.707095][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2827.707292][ T44] schedule+0xd4/0x260
[ 2827.707433][ T44] schedule_preempt_disabled+0x18/0x30
[ 2827.707600][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2827.707764][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2827.707938][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.708102][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2827.708265][ T44] ? register_pernet_subsys+0x19/0x40
[ 2827.708444][ T44] ? lock_acquire+0x10a/0x150
[ 2827.708607][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2827.708800][ T44] down_write+0x1c6/0x1f0
[ 2827.708940][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2827.709131][ T44] register_pernet_subsys+0x19/0x40
[ 2827.709305][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2827.709468][ T44] do_one_initcall+0xa3/0x2e0
[ 2827.709633][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2827.709827][ T44] ? kasan_unpoison+0x44/0x70
[ 2827.709990][ T44] do_init_module+0x27b/0x820
[ 2827.710158][ T44] ? free_module+0x450/0x450
[ 2827.710336][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2827.710497][ T44] init_module_from_file+0x15d/0x180
[ 2827.710660][ T44] ? __do_sys_init_module+0x200/0x200
[ 2827.710836][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2827.710997][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2827.711157][ T44] idempotent_init_module+0x226/0x760
[ 2827.711332][ T44] ? init_module_from_file+0x180/0x180
[ 2827.711495][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.711656][ T44] ? cap_capable+0x181/0x3f0
[ 2827.711828][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2827.711991][ T44] do_syscall_64+0xbd/0xfc0
[ 2827.712151][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2827.712346][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2827.712508][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2827.712720][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2827.712939][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2827.713148][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2827.713372][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2827.713586][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2827.713812][ T44]
[ 2827.713949][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2827.714223][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2827.714543][ T44] Workqueue: netns cleanup_net
[ 2827.714704][ T44] Call Trace:
[ 2827.714886][ T44]
[ 2827.714994][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.715189][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.715365][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2827.715524][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.715709][ T44] ? lock_acquire+0x10a/0x150
[ 2827.715874][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.716036][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.716193][ T44] ? trace_contention_end+0xd6/0x110
[ 2827.716367][ T44] ? __mutex_lock+0x285/0x1f20
[ 2827.716526][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.716714][ T44] ? ww_mutex_lock+0x160/0x160
[ 2827.716879][ T44] ? __lock_acquire+0x577/0xc10
[ 2827.717040][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2827.717205][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.717404][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.717590][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2827.717805][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2827.718028][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2827.718219][ T44] ? kfree+0x119/0x580
[ 2827.718363][ T44] ? lock_acquire+0x10a/0x150
[ 2827.718522][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2827.718712][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2827.718877][ T44] ? netns_install+0x280/0x280
[ 2827.719045][ T44] ? cleanup_net+0x3bc/0x890
[ 2827.719204][ T44] ? process_one_work+0xd16/0x1390
[ 2827.719375][ T44] ? net_passive_dec+0x190/0x190
[ 2827.719537][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.719699][ T44] ? process_one_work+0xd16/0x1390
[ 2827.719862][ T44] ? lock_acquire+0x10a/0x150
[ 2827.720023][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.720185][ T44] ? process_one_work+0xd57/0x1390
[ 2827.720360][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2827.720518][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2827.720680][ T44] ? assign_work+0x152/0x380
[ 2827.720843][ T44] ? worker_thread+0x4d6/0xd40
[ 2827.721007][ T44] ? process_one_work+0x1390/0x1390
[ 2827.721163][ T44] ? kthread+0x355/0x5b0
[ 2827.721310][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2827.721470][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2827.721628][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.721792][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2827.721954][ T44] ? ret_from_fork+0x3fb/0x510
[ 2827.722113][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2827.722312][ T44] ? __switch_to+0x53c/0xd00
[ 2827.722471][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2827.722632][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2827.722796][ T44]
[ 2827.722933][ T44]
[ 2827.722933][ T44] Showing all locks held in the system:
[ 2827.723139][ T44] 4 locks held by kworker/u16:0/12:
[ 2827.723307][ T44] 1 lock held by khungtaskd/44:
[ 2827.723463][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2827.723728][ T44] 2 locks held by kworker/u17:2/4541:
[ 2827.723884][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2827.724150][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2827.724447][ T44] 1 lock held by modprobe/23603:
[ 2827.724598][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2827.724860][ T44]
[ 2827.724959][ T44] =============================================
[ 2827.724959][ T44]
[ 2950.585291][ T44] INFO: task modprobe:23603 blocked for more than 1105 seconds.
[ 2950.585603][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2950.585847][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2950.586109][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 2950.586475][ T44] Call Trace:
[ 2950.586624][ T44]
[ 2950.586743][ T44] __schedule+0xae2/0x1720
[ 2950.586919][ T44] ? io_schedule_timeout+0x130/0x130
[ 2950.587085][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2950.587254][ T44] schedule+0xd4/0x260
[ 2950.587407][ T44] schedule_preempt_disabled+0x18/0x30
[ 2950.587574][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2950.587739][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2950.587911][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.588073][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2950.588232][ T44] ? register_pernet_subsys+0x19/0x40
[ 2950.588409][ T44] ? lock_acquire+0x10a/0x150
[ 2950.588577][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 2950.588779][ T44] down_write+0x1c6/0x1f0
[ 2950.588919][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2950.589106][ T44] register_pernet_subsys+0x19/0x40
[ 2950.589263][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 2950.589443][ T44] do_one_initcall+0xa3/0x2e0
[ 2950.589603][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2950.589789][ T44] ? kasan_unpoison+0x44/0x70
[ 2950.589959][ T44] do_init_module+0x27b/0x820
[ 2950.590121][ T44] ? free_module+0x450/0x450
[ 2950.590294][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2950.590458][ T44] init_module_from_file+0x15d/0x180
[ 2950.590618][ T44] ? __do_sys_init_module+0x200/0x200
[ 2950.590783][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2950.590949][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2950.591116][ T44] idempotent_init_module+0x226/0x760
[ 2950.591288][ T44] ? init_module_from_file+0x180/0x180
[ 2950.591451][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.591614][ T44] ? cap_capable+0x181/0x3f0
[ 2950.591773][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2950.591940][ T44] do_syscall_64+0xbd/0xfc0
[ 2950.592099][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2950.592299][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 2950.592462][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2950.592675][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 2950.592890][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 2950.593098][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 2950.593326][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 2950.593535][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 2950.593750][ T44]
[ 2950.593884][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2950.594165][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2950.594488][ T44] Workqueue: netns cleanup_net
[ 2950.594644][ T44] Call Trace:
[ 2950.594773][ T44]
[ 2950.594883][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.595073][ T44] ? rcu_read_lock_sched_held+0x3c/0x70
[ 2950.595234][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.595401][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2950.595560][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.595746][ T44] ? lock_acquire+0x10a/0x150
[ 2950.595903][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.596067][ T44] ? trace_contention_end+0xd6/0x110
[ 2950.596231][ T44] ? __mutex_lock+0x285/0x1f20
[ 2950.596404][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.596591][ T44] ? ww_mutex_lock+0x160/0x160
[ 2950.596750][ T44] ? __lock_acquire+0x577/0xc10
[ 2950.596917][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2950.597077][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.597262][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.597467][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2950.597682][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2950.597906][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2950.598093][ T44] ? kfree+0x119/0x580
[ 2950.598223][ T44] ? lock_acquire+0x10a/0x150
[ 2950.598396][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2950.598589][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2950.598749][ T44] ? netns_install+0x280/0x280
[ 2950.598918][ T44] ? cleanup_net+0x3bc/0x890
[ 2950.599073][ T44] ? process_one_work+0xd16/0x1390
[ 2950.599230][ T44] ? net_passive_dec+0x190/0x190
[ 2950.599399][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.599555][ T44] ? process_one_work+0xd16/0x1390
[ 2950.599718][ T44] ? lock_acquire+0x10a/0x150
[ 2950.599880][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.600039][ T44] ? process_one_work+0xd57/0x1390
[ 2950.600201][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2950.600369][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2950.600530][ T44] ? assign_work+0x152/0x380
[ 2950.600692][ T44] ? worker_thread+0x4d6/0xd40
[ 2950.600866][ T44] ? process_one_work+0x1390/0x1390
[ 2950.601023][ T44] ? kthread+0x355/0x5b0
[ 2950.601157][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2950.601332][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2950.601493][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.601653][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2950.601812][ T44] ? ret_from_fork+0x3fb/0x510
[ 2950.601977][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2950.602166][ T44] ? __switch_to+0x53c/0xd00
[ 2950.602341][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2950.602500][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2950.602661][ T44]
[ 2950.602789][ T44]
[ 2950.602789][ T44] Showing all locks held in the system:
[ 2950.602994][ T44] 4 locks held by kworker/u16:0/12:
[ 2950.603147][ T44] 1 lock held by khungtaskd/44:
[ 2950.603314][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2950.603574][ T44] 2 locks held by kworker/u17:2/4541:
[ 2950.603724][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 2950.603989][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 2950.604294][ T44] 1 lock held by modprobe/23603:
[ 2950.604448][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2950.604715][ T44]
[ 2950.604813][ T44] =============================================
[ 2950.604813][ T44]
[ 3073.465293][ T44] INFO: task modprobe:23603 blocked for more than 1228 seconds.
[ 3073.465605][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 3073.465843][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3073.466107][ T44] task:modprobe state:D stack:25072 pid:23603 tgid:23603 ppid:4541 task_flags:0x400100 flags:0x00080000
[ 3073.466445][ T44] Call Trace:
[ 3073.466584][ T44]
[ 3073.466701][ T44] __schedule+0xae2/0x1720
[ 3073.466883][ T44] ? io_schedule_timeout+0x130/0x130
[ 3073.467061][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3073.467229][ T44] schedule+0xd4/0x260
[ 3073.467382][ T44] schedule_preempt_disabled+0x18/0x30
[ 3073.467543][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 3073.467712][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 3073.467888][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.468052][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3073.468213][ T44] ? register_pernet_subsys+0x19/0x40
[ 3073.468402][ T44] ? lock_acquire+0x10a/0x150
[ 3073.468564][ T44] ? vxlan_cleanup_module+0xe10/0xe10 [vxlan]
[ 3073.468761][ T44] down_write+0x1c6/0x1f0
[ 3073.468904][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 3073.469100][ T44] register_pernet_subsys+0x19/0x40
[ 3073.469259][ T44] vxlan_init_module+0x16/0x1000 [vxlan]
[ 3073.469440][ T44] do_one_initcall+0xa3/0x2e0
[ 3073.469597][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 3073.469788][ T44] ? kasan_unpoison+0x44/0x70
[ 3073.469954][ T44] do_init_module+0x27b/0x820
[ 3073.470117][ T44] ? free_module+0x450/0x450
[ 3073.470295][ T44] ? backing_file_read_iter+0x50e/0x730
[ 3073.470454][ T44] init_module_from_file+0x15d/0x180
[ 3073.470614][ T44] ? __do_sys_init_module+0x200/0x200
[ 3073.470777][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3073.470942][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 3073.471104][ T44] idempotent_init_module+0x226/0x760
[ 3073.471283][ T44] ? init_module_from_file+0x180/0x180
[ 3073.471443][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.471601][ T44] ? cap_capable+0x181/0x3f0
[ 3073.471761][ T44] __x64_sys_finit_module+0xcd/0x160
[ 3073.471926][ T44] do_syscall_64+0xbd/0xfc0
[ 3073.472085][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 3073.472282][ T44] RIP: 0033:0x7f1ff1d6f34d
[ 3073.472444][ T44] RSP: 002b:00007fff5cf152a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 3073.472660][ T44] RAX: ffffffffffffffda RBX: 000055b84d911b10 RCX: 00007f1ff1d6f34d
[ 3073.472879][ T44] RDX: 0000000000000000 RSI: 000055b835c15f2e RDI: 0000000000000000
[ 3073.473089][ T44] RBP: 00007fff5cf15340 R08: 0000000000000000 R09: 0000000000000000
[ 3073.473315][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055b835c15f2e
[ 3073.473529][ T44] R13: 0000000000040000 R14: 000055b84d911c60 R15: 000055b84d911570
[ 3073.473744][ T44]
[ 3073.473880][ T44] INFO: task modprobe:23603 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 3073.474153][ T44] task:kworker/u16:0 state:R running task stack:26032 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 3073.474474][ T44] Workqueue: netns cleanup_net
[ 3073.474633][ T44] Call Trace:
[ 3073.474761][ T44]
[ 3073.474871][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.475063][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.475220][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3073.475390][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.475580][ T44] ? lock_acquire+0x10a/0x150
[ 3073.475737][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.475901][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.476061][ T44] ? sched_clock_cpu+0x69/0x620
[ 3073.476220][ T44] ? trace_contention_end+0xd6/0x110
[ 3073.476392][ T44] ? __mutex_lock+0x285/0x1f20
[ 3073.476556][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.476748][ T44] ? ww_mutex_lock+0x160/0x160
[ 3073.476914][ T44] ? __lock_acquire+0x577/0xc10
[ 3073.477076][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3073.477235][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.477434][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.477623][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 3073.477841][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 3073.478062][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 3073.478254][ T44] ? kfree+0x119/0x580
[ 3073.478400][ T44] ? lock_acquire+0x10a/0x150
[ 3073.478560][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 3073.478747][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 3073.478918][ T44] ? netns_install+0x280/0x280
[ 3073.479080][ T44] ? cleanup_net+0x3bc/0x890
[ 3073.479237][ T44] ? process_one_work+0xd16/0x1390
[ 3073.479408][ T44] ? net_passive_dec+0x190/0x190
[ 3073.479569][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.479726][ T44] ? process_one_work+0xd16/0x1390
[ 3073.479888][ T44] ? lock_acquire+0x10a/0x150
[ 3073.480052][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.480213][ T44] ? process_one_work+0xd57/0x1390
[ 3073.480391][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 3073.480548][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 3073.480708][ T44] ? assign_work+0x152/0x380
[ 3073.480872][ T44] ? worker_thread+0x4d6/0xd40
[ 3073.481038][ T44] ? process_one_work+0x1390/0x1390
[ 3073.481199][ T44] ? kthread+0x355/0x5b0
[ 3073.481344][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3073.481502][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3073.481660][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.481816][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3073.481983][ T44] ? ret_from_fork+0x3fb/0x510
[ 3073.482141][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 3073.482339][ T44] ? __switch_to+0x53c/0xd00
[ 3073.482498][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3073.482655][ T44] ? ret_from_fork_asm+0x11/0x20
[ 3073.482816][ T44]
[ 3073.482951][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
[ 3073.483180][ T44]
[ 3073.483180][ T44] Showing all locks held in the system:
[ 3073.483480][ T44] 4 locks held by kworker/u16:0/12:
[ 3073.483635][ T44] 1 lock held by khungtaskd/44:
[ 3073.483787][ T44] #0: ffffffff8cd850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 3073.484056][ T44] 2 locks held by kworker/u17:2/4541:
[ 3073.484206][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 3073.484479][ T44] #1: ffa000000b057cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 3073.484770][ T44] 1 lock held by modprobe/23603:
[ 3073.484927][ T44] #0: ffffffff8d485bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 3073.485186][ T44]
[ 3073.485301][ T44] =============================================
[ 3073.485301][ T44]
WAIT TIMEOUT stderr
Ctrl-C stderr
Ctrl-C stderr