[ 1857.602024][T24613] pktgen: Packet Generator for packet performance testing. Version: 2.75 [ 1888.923837][ T12] ------------[ cut here ]------------ [ 1888.924247][ T12] conntrack cleanup blocked for 60s [ 1888.924251][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack], CPU#2: kworker/u16:0/12 [ 1888.924886][ T12] Modules linked in: pktgen(+) xt_conntrack nf_conntrack nf_defrag_ipv4 nf_defrag_ipv6 sctp_diag sctp geneve vxlan ip6_gre ip_gre gre act_gact cls_matchall cls_flower sch_prio xt_mark sch_ingress act_mirred cls_basic sch_fq_codel xfrm_user xt_policy xt_HL nft_compat nf_tables amt [ 1888.925691][ T12] CPU: 2 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.19.0-rc3-virtme #1 PREEMPT(full) [ 1888.926035][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1888.926255][ T12] Workqueue: netns cleanup_net [ 1888.926472][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 1888.926736][ T12] Code: 00 48 8b 54 24 18 48 8b 05 b3 88 50 d8 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 5e 45 d7 e9 25 fd ff ff 48 8d 3d 46 66 d6 ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df [ 1888.927259][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297 [ 1888.927499][ T12] RAX: 0000000100183cdc RBX: ffa00000000c7af8 RCX: 0000000000000001 [ 1888.927747][ T12] RDX: 0000000100183cdb RSI: 0000000000000004 RDI: ffffffffc066f030 [ 1888.928001][ T12] RBP: 0000000000000001 R08: ffffffffc09087ba R09: ffe21c00014de67c [ 1888.928243][ T12] R10: ffe21c00014de67d R11: 0000000000000001 R12: dffffc0000000000 [ 1888.928499][ T12] R13: ff1100000a6f33e0 R14: fffffbfff80cf807 R15: ff11000009558a00 [ 1888.928744][ T12] FS: 0000000000000000(0000) GS:ff110000d1ef3000(0000) knlGS:0000000000000000 [ 1888.929332][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1888.929546][ T12] CR2: 00005610ac3b7600 CR3: 000000001ff48002 CR4: 0000000000771ef0 [ 1888.929780][ T12] PKRU: 55555554 [ 1888.929939][ T12] Call Trace: [ 1888.930083][ T12] [ 1888.930200][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 1888.930419][ T12] ? kfree+0x119/0x580 [ 1888.930565][ T12] ? lock_acquire+0x10a/0x150 [ 1888.930742][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 1888.930960][ T12] ops_undo_list+0x2d7/0x8f0 [ 1888.931138][ T12] ? netns_install+0x280/0x280 [ 1888.931318][ T12] cleanup_net+0x3bc/0x890 [ 1888.931496][ T12] ? process_one_work+0xd16/0x1390 [ 1888.931671][ T12] ? net_passive_dec+0x190/0x190 [ 1888.931861][ T12] ? rcu_is_watching+0x15/0xd0 [ 1888.932033][ T12] ? process_one_work+0xd16/0x1390 [ 1888.932206][ T12] ? lock_acquire+0x10a/0x150 [ 1888.932385][ T12] ? rcu_is_watching+0x15/0xd0 [ 1888.932562][ T12] process_one_work+0xd57/0x1390 [ 1888.932743][ T12] ? pwq_dec_nr_in_flight+0x700/0x700 [ 1888.932932][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 1888.933107][ T12] ? assign_work+0x152/0x380 [ 1888.933285][ T12] worker_thread+0x4d6/0xd40 [ 1888.933469][ T12] ? process_one_work+0x1390/0x1390 [ 1888.933644][ T12] kthread+0x355/0x5b0 [ 1888.933790][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1888.933982][ T12] ? __lock_release.isra.0+0x59/0x170 [ 1888.934156][ T12] ? rcu_is_watching+0x15/0xd0 [ 1888.934329][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1888.934509][ T12] ret_from_fork+0x3fb/0x510 [ 1888.934684][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 1888.934900][ T12] ? __switch_to+0x53c/0xd00 [ 1888.935073][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1888.935250][ T12] ret_from_fork_asm+0x11/0x20 [ 1888.935435][ T12] [ 1888.935578][ T12] irq event stamp: 16154931 [ 1888.935751][ T12] hardirqs last enabled at (16154941): [] __up_console_sem+0x5a/0x70 [ 1888.936034][ T12] hardirqs last disabled at (16154952): [] __up_console_sem+0x3f/0x70 [ 1888.936298][ T12] softirqs last enabled at (16154438): [] handle_softirqs+0x56f/0x840 [ 1888.936576][ T12] softirqs last disabled at (16154431): [] irq_exit_rcu+0xa2/0xf0 [ 1888.936859][ T12] ---[ end trace 0000000000000000 ]--- [ 2090.431860][ T44] INFO: task modprobe:24613 blocked for more than 122 seconds. [ 2090.432277][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2090.432475][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2090.432707][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2090.433034][ T44] Call Trace: [ 2090.433161][ T44] [ 2090.433231][ T44] __schedule+0xae2/0x1720 [ 2090.433367][ T44] ? io_schedule_timeout+0x130/0x130 [ 2090.433494][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.433621][ T44] schedule+0xd4/0x260 [ 2090.433721][ T44] schedule_preempt_disabled+0x18/0x30 [ 2090.433849][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2090.433978][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2090.434106][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.434231][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.434355][ T44] ? register_pernet_subsys+0x19/0x40 [ 2090.434471][ T44] ? lock_acquire+0x10a/0x150 [ 2090.434588][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2090.434708][ T44] down_write+0x1c6/0x1f0 [ 2090.434797][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2090.434949][ T44] register_pernet_subsys+0x19/0x40 [ 2090.435062][ T44] pg_init+0x29/0x1000 [pktgen] [ 2090.435177][ T44] do_one_initcall+0xa3/0x2e0 [ 2090.435292][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2090.435436][ T44] ? kasan_unpoison+0x44/0x70 [ 2090.435548][ T44] do_init_module+0x27b/0x820 [ 2090.435666][ T44] ? free_module+0x450/0x450 [ 2090.435775][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2090.435893][ T44] init_module_from_file+0x15d/0x180 [ 2090.436004][ T44] ? __do_sys_init_module+0x200/0x200 [ 2090.436450][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.436564][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2090.436676][ T44] idempotent_init_module+0x226/0x760 [ 2090.436795][ T44] ? init_module_from_file+0x180/0x180 [ 2090.436918][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.437027][ T44] ? cap_capable+0x181/0x3f0 [ 2090.437138][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2090.437248][ T44] do_syscall_64+0xbd/0xfc0 [ 2090.437358][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2090.437495][ T44] RIP: 0033:0x7f4985ad034d [ 2090.437615][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2090.437789][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2090.437956][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2090.438120][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2090.438285][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2090.438452][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2090.438622][ T44] [ 2090.438712][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2090.438949][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2090.439216][ T44] Workqueue: netns cleanup_net [ 2090.439327][ T44] Call Trace: [ 2090.439411][ T44] [ 2090.439470][ T44] ? asm_common_interrupt+0x26/0x40 [ 2090.439582][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2090.439693][ T44] ? rcu_read_lock_sched_held+0x3c/0x70 [ 2090.439803][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.439915][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.440026][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2090.440166][ T44] ? lock_acquire+0x10a/0x150 [ 2090.440275][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.440382][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.440492][ T44] ? trace_contention_end+0xd6/0x110 [ 2090.440602][ T44] ? pick_task_fair+0x7d/0x1e0 [ 2090.440715][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2090.440857][ T44] ? mark_held_locks+0x40/0x70 [ 2090.440965][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.441073][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2090.441209][ T44] ? __lock_acquire+0x577/0xc10 [ 2090.441317][ T44] ? common_interrupt+0x92/0xf0 [ 2090.441428][ T44] ? asm_common_interrupt+0x26/0x40 [ 2090.441538][ T44] ? nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 2090.441707][ T44] ? trace_contention_end+0x30/0x110 [ 2090.441823][ T44] ? nf_ct_iterate_cleanup+0x94/0x350 [nf_conntrack] [ 2090.441966][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2090.442103][ T44] ? schedule+0xd4/0x260 [ 2090.442187][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2090.442355][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2090.442493][ T44] ? kfree+0x119/0x580 [ 2090.442584][ T44] ? lock_acquire+0x10a/0x150 [ 2090.442698][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2090.442841][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2090.442952][ T44] ? netns_install+0x280/0x280 [ 2090.443063][ T44] ? cleanup_net+0x3bc/0x890 [ 2090.443174][ T44] ? process_one_work+0xd16/0x1390 [ 2090.443282][ T44] ? net_passive_dec+0x190/0x190 [ 2090.443392][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.443499][ T44] ? process_one_work+0xd16/0x1390 [ 2090.443607][ T44] ? lock_acquire+0x10a/0x150 [ 2090.443717][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.443831][ T44] ? process_one_work+0xd57/0x1390 [ 2090.443946][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2090.444054][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2090.444166][ T44] ? assign_work+0x152/0x380 [ 2090.444275][ T44] ? worker_thread+0x4d6/0xd40 [ 2090.444388][ T44] ? process_one_work+0x1390/0x1390 [ 2090.444497][ T44] ? kthread+0x355/0x5b0 [ 2090.444580][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2090.444692][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.444804][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.444918][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2090.445027][ T44] ? ret_from_fork+0x3fb/0x510 [ 2090.445137][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2090.445273][ T44] ? __switch_to+0x53c/0xd00 [ 2090.445381][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2090.445495][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2090.445611][ T44] [ 2090.445703][ T44] [ 2090.445703][ T44] Showing all locks held in the system: [ 2090.445871][ T44] 4 locks held by kworker/u16:0/12: [ 2090.445981][ T44] 1 lock held by khungtaskd/44: [ 2090.446086][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2090.446310][ T44] 1 lock held by modprobe/24613: [ 2090.446416][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2090.446625][ T44] [ 2090.446683][ T44] ============================================= [ 2090.446683][ T44] [ 2213.311854][ T44] INFO: task modprobe:24613 blocked for more than 245 seconds. [ 2213.312140][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2213.312337][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.312570][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2213.312863][ T44] Call Trace: [ 2213.312962][ T44] [ 2213.313030][ T44] __schedule+0xae2/0x1720 [ 2213.313171][ T44] ? io_schedule_timeout+0x130/0x130 [ 2213.313298][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.313429][ T44] schedule+0xd4/0x260 [ 2213.313523][ T44] schedule_preempt_disabled+0x18/0x30 [ 2213.313644][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2213.313763][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2213.313893][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.314018][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.314140][ T44] ? register_pernet_subsys+0x19/0x40 [ 2213.314264][ T44] ? lock_acquire+0x10a/0x150 [ 2213.314389][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2213.314513][ T44] down_write+0x1c6/0x1f0 [ 2213.314605][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2213.314748][ T44] register_pernet_subsys+0x19/0x40 [ 2213.314863][ T44] pg_init+0x29/0x1000 [pktgen] [ 2213.314974][ T44] do_one_initcall+0xa3/0x2e0 [ 2213.315084][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2213.315222][ T44] ? kasan_unpoison+0x44/0x70 [ 2213.315334][ T44] do_init_module+0x27b/0x820 [ 2213.315446][ T44] ? free_module+0x450/0x450 [ 2213.315561][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2213.315675][ T44] init_module_from_file+0x15d/0x180 [ 2213.315784][ T44] ? __do_sys_init_module+0x200/0x200 [ 2213.315903][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.316012][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2213.316122][ T44] idempotent_init_module+0x226/0x760 [ 2213.316231][ T44] ? init_module_from_file+0x180/0x180 [ 2213.316345][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.316455][ T44] ? cap_capable+0x181/0x3f0 [ 2213.316569][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2213.316678][ T44] do_syscall_64+0xbd/0xfc0 [ 2213.316787][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2213.316926][ T44] RIP: 0033:0x7f4985ad034d [ 2213.317041][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2213.317208][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2213.317370][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2213.317535][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2213.317697][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2213.317867][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2213.318033][ T44] [ 2213.318116][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2213.318344][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2213.318611][ T44] Workqueue: netns cleanup_net [ 2213.318719][ T44] Call Trace: [ 2213.318800][ T44] [ 2213.318860][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.319001][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.319109][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.319219][ T44] ? update_curr+0x411/0x670 [ 2213.319333][ T44] ? sched_clock_cpu+0x69/0x620 [ 2213.319444][ T44] ? trace_contention_end+0xd6/0x110 [ 2213.319555][ T44] ? __mutex_lock+0x285/0x1f20 [ 2213.319663][ T44] ? mark_held_locks+0x40/0x70 [ 2213.319770][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.319912][ T44] ? mark_held_locks+0x40/0x70 [ 2213.320022][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.320129][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2213.320267][ T44] ? common_interrupt+0x92/0xf0 [ 2213.320379][ T44] ? asm_common_interrupt+0x26/0x40 [ 2213.320492][ T44] ? handle_bug+0x1de/0x2d0 [ 2213.320600][ T44] ? trace_contention_end+0x30/0x110 [ 2213.320710][ T44] ? nf_ct_iterate_cleanup+0x94/0x350 [nf_conntrack] [ 2213.320850][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.320987][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2213.321149][ T44] ? schedule+0xd4/0x260 [ 2213.321230][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2213.321393][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2213.321532][ T44] ? kfree+0x119/0x580 [ 2213.321613][ T44] ? lock_acquire+0x10a/0x150 [ 2213.321724][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2213.321867][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2213.321977][ T44] ? netns_install+0x280/0x280 [ 2213.322088][ T44] ? cleanup_net+0x3bc/0x890 [ 2213.322196][ T44] ? process_one_work+0xd16/0x1390 [ 2213.322305][ T44] ? net_passive_dec+0x190/0x190 [ 2213.322412][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.322525][ T44] ? process_one_work+0xd16/0x1390 [ 2213.322630][ T44] ? lock_acquire+0x10a/0x150 [ 2213.322738][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.322853][ T44] ? process_one_work+0xd57/0x1390 [ 2213.322966][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2213.323072][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2213.323182][ T44] ? assign_work+0x152/0x380 [ 2213.323294][ T44] ? worker_thread+0x4d6/0xd40 [ 2213.323409][ T44] ? process_one_work+0x1390/0x1390 [ 2213.323518][ T44] ? kthread+0x355/0x5b0 [ 2213.323599][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2213.323707][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.323820][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.323929][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2213.324037][ T44] ? ret_from_fork+0x3fb/0x510 [ 2213.324151][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2213.324286][ T44] ? __switch_to+0x53c/0xd00 [ 2213.324396][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2213.324507][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2213.324622][ T44] [ 2213.324714][ T44] [ 2213.324714][ T44] Showing all locks held in the system: [ 2213.324875][ T44] 3 locks held by kworker/u16:0/12: [ 2213.324986][ T44] 1 lock held by khungtaskd/44: [ 2213.325089][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2213.325308][ T44] 1 lock held by modprobe/24613: [ 2213.325411][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2213.325626][ T44] [ 2213.325680][ T44] ============================================= [ 2213.325680][ T44] [ 2336.191852][ T44] INFO: task modprobe:24613 blocked for more than 368 seconds. [ 2336.192138][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2336.192338][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.192564][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2336.192876][ T44] Call Trace: [ 2336.192972][ T44] [ 2336.193039][ T44] __schedule+0xae2/0x1720 [ 2336.193173][ T44] ? io_schedule_timeout+0x130/0x130 [ 2336.193304][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.193440][ T44] schedule+0xd4/0x260 [ 2336.193539][ T44] schedule_preempt_disabled+0x18/0x30 [ 2336.193673][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2336.193799][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2336.193928][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.194054][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.194180][ T44] ? register_pernet_subsys+0x19/0x40 [ 2336.194304][ T44] ? lock_acquire+0x10a/0x150 [ 2336.194431][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2336.194562][ T44] down_write+0x1c6/0x1f0 [ 2336.194658][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2336.194807][ T44] register_pernet_subsys+0x19/0x40 [ 2336.194930][ T44] pg_init+0x29/0x1000 [pktgen] [ 2336.195040][ T44] do_one_initcall+0xa3/0x2e0 [ 2336.195150][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2336.195289][ T44] ? kasan_unpoison+0x44/0x70 [ 2336.195406][ T44] do_init_module+0x27b/0x820 [ 2336.195519][ T44] ? free_module+0x450/0x450 [ 2336.195627][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2336.195742][ T44] init_module_from_file+0x15d/0x180 [ 2336.195855][ T44] ? __do_sys_init_module+0x200/0x200 [ 2336.195969][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.196077][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2336.196186][ T44] idempotent_init_module+0x226/0x760 [ 2336.196295][ T44] ? init_module_from_file+0x180/0x180 [ 2336.196412][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.196524][ T44] ? cap_capable+0x181/0x3f0 [ 2336.196636][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2336.196746][ T44] do_syscall_64+0xbd/0xfc0 [ 2336.196861][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2336.196994][ T44] RIP: 0033:0x7f4985ad034d [ 2336.197111][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2336.197277][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2336.197443][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2336.197605][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2336.197768][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2336.197934][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2336.198104][ T44] [ 2336.198188][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2336.198418][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2336.198681][ T44] Workqueue: netns cleanup_net [ 2336.198789][ T44] Call Trace: [ 2336.198876][ T44] [ 2336.198934][ T44] ? __lock_acquire+0x577/0xc10 [ 2336.199044][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2336.199152][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.199291][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.199401][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.199511][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.199647][ T44] ? lock_acquire+0x10a/0x150 [ 2336.199756][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.199869][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.199976][ T44] ? trace_contention_end+0xd6/0x110 [ 2336.200086][ T44] ? __mutex_lock+0x285/0x1f20 [ 2336.200194][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.200336][ T44] ? ww_mutex_lock+0x160/0x160 [ 2336.200445][ T44] ? __lock_acquire+0x577/0xc10 [ 2336.200556][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.200665][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.200802][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.200942][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2336.201111][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2336.201275][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2336.201414][ T44] ? kfree+0x119/0x580 [ 2336.201497][ T44] ? lock_acquire+0x10a/0x150 [ 2336.201609][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2336.201747][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2336.201864][ T44] ? netns_install+0x280/0x280 [ 2336.201975][ T44] ? cleanup_net+0x3bc/0x890 [ 2336.202081][ T44] ? process_one_work+0xd16/0x1390 [ 2336.202191][ T44] ? net_passive_dec+0x190/0x190 [ 2336.202299][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.202409][ T44] ? process_one_work+0xd16/0x1390 [ 2336.202519][ T44] ? lock_acquire+0x10a/0x150 [ 2336.202627][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.202737][ T44] ? process_one_work+0xd57/0x1390 [ 2336.202853][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2336.202962][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2336.203072][ T44] ? assign_work+0x152/0x380 [ 2336.203184][ T44] ? worker_thread+0x4d6/0xd40 [ 2336.203297][ T44] ? process_one_work+0x1390/0x1390 [ 2336.203410][ T44] ? kthread+0x355/0x5b0 [ 2336.203495][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2336.203603][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.203711][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.203824][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2336.203934][ T44] ? ret_from_fork+0x3fb/0x510 [ 2336.204045][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2336.204181][ T44] ? __switch_to+0x53c/0xd00 [ 2336.204292][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2336.204405][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2336.204519][ T44] [ 2336.204615][ T44] [ 2336.204615][ T44] Showing all locks held in the system: [ 2336.204774][ T44] 4 locks held by kworker/u16:0/12: [ 2336.204887][ T44] 1 lock held by khungtaskd/44: [ 2336.204992][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2336.205213][ T44] 1 lock held by modprobe/24613: [ 2336.205316][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2336.205530][ T44] [ 2336.205583][ T44] ============================================= [ 2336.205583][ T44] [ 2459.071854][ T44] INFO: task modprobe:24613 blocked for more than 491 seconds. [ 2459.072139][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2459.072342][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.072571][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2459.072884][ T44] Call Trace: [ 2459.072979][ T44] [ 2459.073042][ T44] __schedule+0xae2/0x1720 [ 2459.073174][ T44] ? io_schedule_timeout+0x130/0x130 [ 2459.073305][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.073443][ T44] schedule+0xd4/0x260 [ 2459.073541][ T44] schedule_preempt_disabled+0x18/0x30 [ 2459.073673][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2459.073796][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2459.073925][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.074045][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.074165][ T44] ? register_pernet_subsys+0x19/0x40 [ 2459.074279][ T44] ? lock_acquire+0x10a/0x150 [ 2459.074399][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2459.074515][ T44] down_write+0x1c6/0x1f0 [ 2459.074600][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2459.074741][ T44] register_pernet_subsys+0x19/0x40 [ 2459.074857][ T44] pg_init+0x29/0x1000 [pktgen] [ 2459.074971][ T44] do_one_initcall+0xa3/0x2e0 [ 2459.075084][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2459.075221][ T44] ? kasan_unpoison+0x44/0x70 [ 2459.075338][ T44] do_init_module+0x27b/0x820 [ 2459.075451][ T44] ? free_module+0x450/0x450 [ 2459.075560][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2459.075673][ T44] init_module_from_file+0x15d/0x180 [ 2459.075782][ T44] ? __do_sys_init_module+0x200/0x200 [ 2459.075904][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.076019][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2459.076127][ T44] idempotent_init_module+0x226/0x760 [ 2459.076237][ T44] ? init_module_from_file+0x180/0x180 [ 2459.076351][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.076459][ T44] ? cap_capable+0x181/0x3f0 [ 2459.076573][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2459.076682][ T44] do_syscall_64+0xbd/0xfc0 [ 2459.076790][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2459.076931][ T44] RIP: 0033:0x7f4985ad034d [ 2459.077053][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2459.077219][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2459.077389][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2459.077552][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2459.077716][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2459.077884][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2459.078055][ T44] [ 2459.078139][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2459.078383][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2459.078651][ T44] Workqueue: netns cleanup_net [ 2459.078762][ T44] Call Trace: [ 2459.078851][ T44] [ 2459.078907][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.079050][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.079161][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.079269][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.079408][ T44] ? lock_acquire+0x10a/0x150 [ 2459.079516][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.079628][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.079737][ T44] ? trace_contention_end+0xd6/0x110 [ 2459.079851][ T44] ? __mutex_lock+0x285/0x1f20 [ 2459.079961][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.080099][ T44] ? ww_mutex_lock+0x160/0x160 [ 2459.080208][ T44] ? __lock_acquire+0x577/0xc10 [ 2459.080319][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2459.080491][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.080601][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.080736][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.080876][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2459.081040][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2459.081206][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2459.081344][ T44] ? kfree+0x119/0x580 [ 2459.081426][ T44] ? lock_acquire+0x10a/0x150 [ 2459.081537][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2459.081675][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2459.081786][ T44] ? netns_install+0x280/0x280 [ 2459.081902][ T44] ? cleanup_net+0x3bc/0x890 [ 2459.082012][ T44] ? process_one_work+0xd16/0x1390 [ 2459.082124][ T44] ? net_passive_dec+0x190/0x190 [ 2459.082232][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.082341][ T44] ? process_one_work+0xd16/0x1390 [ 2459.082449][ T44] ? lock_acquire+0x10a/0x150 [ 2459.082555][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.082664][ T44] ? process_one_work+0xd57/0x1390 [ 2459.082776][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2459.082892][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2459.083003][ T44] ? assign_work+0x152/0x380 [ 2459.083112][ T44] ? worker_thread+0x4d6/0xd40 [ 2459.083226][ T44] ? process_one_work+0x1390/0x1390 [ 2459.083336][ T44] ? kthread+0x355/0x5b0 [ 2459.083420][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2459.083529][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.083640][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.083749][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2459.083863][ T44] ? ret_from_fork+0x3fb/0x510 [ 2459.083973][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2459.084108][ T44] ? __switch_to+0x53c/0xd00 [ 2459.084217][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2459.084329][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2459.084444][ T44] [ 2459.084544][ T44] [ 2459.084544][ T44] Showing all locks held in the system: [ 2459.084703][ T44] 4 locks held by kworker/u16:0/12: [ 2459.084821][ T44] 1 lock held by khungtaskd/44: [ 2459.084924][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2459.085145][ T44] 1 lock held by modprobe/24613: [ 2459.085248][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2459.085467][ T44] [ 2459.085520][ T44] ============================================= [ 2459.085520][ T44] [ 2581.951855][ T44] INFO: task modprobe:24613 blocked for more than 614 seconds. [ 2581.952142][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2581.952339][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2581.952574][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2581.952899][ T44] Call Trace: [ 2581.953006][ T44] [ 2581.953079][ T44] __schedule+0xae2/0x1720 [ 2581.953220][ T44] ? io_schedule_timeout+0x130/0x130 [ 2581.953348][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.953482][ T44] schedule+0xd4/0x260 [ 2581.953574][ T44] schedule_preempt_disabled+0x18/0x30 [ 2581.953697][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2581.953827][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2581.953953][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.954074][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.954195][ T44] ? register_pernet_subsys+0x19/0x40 [ 2581.954312][ T44] ? lock_acquire+0x10a/0x150 [ 2581.954428][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2581.954546][ T44] down_write+0x1c6/0x1f0 [ 2581.954631][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2581.954771][ T44] register_pernet_subsys+0x19/0x40 [ 2581.954886][ T44] pg_init+0x29/0x1000 [pktgen] [ 2581.954998][ T44] do_one_initcall+0xa3/0x2e0 [ 2581.955108][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2581.955248][ T44] ? kasan_unpoison+0x44/0x70 [ 2581.955361][ T44] do_init_module+0x27b/0x820 [ 2581.955476][ T44] ? free_module+0x450/0x450 [ 2581.955584][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2581.955694][ T44] init_module_from_file+0x15d/0x180 [ 2581.955801][ T44] ? __do_sys_init_module+0x200/0x200 [ 2581.955920][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.956030][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2581.956138][ T44] idempotent_init_module+0x226/0x760 [ 2581.956246][ T44] ? init_module_from_file+0x180/0x180 [ 2581.956357][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.956468][ T44] ? cap_capable+0x181/0x3f0 [ 2581.956577][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2581.956685][ T44] do_syscall_64+0xbd/0xfc0 [ 2581.956795][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2581.956932][ T44] RIP: 0033:0x7f4985ad034d [ 2581.957045][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2581.957210][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2581.957374][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2581.957537][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2581.957702][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2581.957869][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2581.958037][ T44] [ 2581.958119][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2581.958349][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2581.958612][ T44] Workqueue: netns cleanup_net [ 2581.958720][ T44] Call Trace: [ 2581.958802][ T44] [ 2581.958862][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.959003][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.959114][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.959220][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.959355][ T44] ? lock_acquire+0x10a/0x150 [ 2581.959466][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.959571][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.959677][ T44] ? trace_contention_end+0xd6/0x110 [ 2581.959786][ T44] ? __mutex_lock+0x285/0x1f20 [ 2581.959903][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.960038][ T44] ? ww_mutex_lock+0x160/0x160 [ 2581.960147][ T44] ? __lock_acquire+0x577/0xc10 [ 2581.960260][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.960372][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.960507][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.960644][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2581.960809][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2581.960980][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2581.961115][ T44] ? kfree+0x119/0x580 [ 2581.961197][ T44] ? lock_acquire+0x10a/0x150 [ 2581.961307][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2581.961449][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2581.961557][ T44] ? netns_install+0x280/0x280 [ 2581.961668][ T44] ? cleanup_net+0x3bc/0x890 [ 2581.961773][ T44] ? process_one_work+0xd16/0x1390 [ 2581.961887][ T44] ? net_passive_dec+0x190/0x190 [ 2581.961993][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.962101][ T44] ? process_one_work+0xd16/0x1390 [ 2581.962208][ T44] ? lock_acquire+0x10a/0x150 [ 2581.962318][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.962428][ T44] ? process_one_work+0xd57/0x1390 [ 2581.962539][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2581.962647][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2581.962761][ T44] ? assign_work+0x152/0x380 [ 2581.962878][ T44] ? worker_thread+0x4d6/0xd40 [ 2581.962990][ T44] ? process_one_work+0x1390/0x1390 [ 2581.963098][ T44] ? kthread+0x355/0x5b0 [ 2581.963180][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2581.963287][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.963402][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.963509][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2581.963616][ T44] ? ret_from_fork+0x3fb/0x510 [ 2581.963724][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2581.963865][ T44] ? __switch_to+0x53c/0xd00 [ 2581.963977][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2581.964084][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2581.964196][ T44] [ 2581.964300][ T44] [ 2581.964300][ T44] Showing all locks held in the system: [ 2581.964460][ T44] 4 locks held by kworker/u16:0/12: [ 2581.964569][ T44] 1 lock held by khungtaskd/44: [ 2581.964672][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2581.964896][ T44] 1 lock held by modprobe/24613: [ 2581.964999][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2581.965217][ T44] [ 2581.965270][ T44] ============================================= [ 2581.965270][ T44] [ 2704.831855][ T44] INFO: task modprobe:24613 blocked for more than 737 seconds. [ 2704.832140][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2704.832333][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2704.832569][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2704.832877][ T44] Call Trace: [ 2704.832977][ T44] [ 2704.833044][ T44] __schedule+0xae2/0x1720 [ 2704.833187][ T44] ? io_schedule_timeout+0x130/0x130 [ 2704.833321][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.833456][ T44] schedule+0xd4/0x260 [ 2704.833547][ T44] schedule_preempt_disabled+0x18/0x30 [ 2704.833663][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2704.833786][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2704.833916][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.834039][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.834179][ T44] ? register_pernet_subsys+0x19/0x40 [ 2704.834306][ T44] ? lock_acquire+0x10a/0x150 [ 2704.834433][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2704.834553][ T44] down_write+0x1c6/0x1f0 [ 2704.834635][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2704.834774][ T44] register_pernet_subsys+0x19/0x40 [ 2704.834888][ T44] pg_init+0x29/0x1000 [pktgen] [ 2704.835001][ T44] do_one_initcall+0xa3/0x2e0 [ 2704.835112][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2704.835250][ T44] ? kasan_unpoison+0x44/0x70 [ 2704.835362][ T44] do_init_module+0x27b/0x820 [ 2704.835479][ T44] ? free_module+0x450/0x450 [ 2704.835588][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2704.835698][ T44] init_module_from_file+0x15d/0x180 [ 2704.835809][ T44] ? __do_sys_init_module+0x200/0x200 [ 2704.835926][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.836034][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2704.836142][ T44] idempotent_init_module+0x226/0x760 [ 2704.836250][ T44] ? init_module_from_file+0x180/0x180 [ 2704.836362][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.836474][ T44] ? cap_capable+0x181/0x3f0 [ 2704.836588][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2704.836696][ T44] do_syscall_64+0xbd/0xfc0 [ 2704.836806][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2704.836946][ T44] RIP: 0033:0x7f4985ad034d [ 2704.837061][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2704.837225][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2704.837390][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2704.837553][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2704.837712][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2704.837877][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2704.838052][ T44] [ 2704.838136][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2704.838364][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2704.838629][ T44] Workqueue: netns cleanup_net [ 2704.838737][ T44] Call Trace: [ 2704.838823][ T44] [ 2704.838883][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.839023][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.839130][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.839239][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.839374][ T44] ? lock_acquire+0x10a/0x150 [ 2704.839483][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.839594][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.839703][ T44] ? trace_contention_end+0xd6/0x110 [ 2704.839820][ T44] ? __mutex_lock+0x285/0x1f20 [ 2704.839930][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.840067][ T44] ? ww_mutex_lock+0x160/0x160 [ 2704.840175][ T44] ? __lock_acquire+0x577/0xc10 [ 2704.840286][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.840398][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.840540][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.840676][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2704.840844][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2704.841007][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2704.841146][ T44] ? kfree+0x119/0x580 [ 2704.841229][ T44] ? lock_acquire+0x10a/0x150 [ 2704.841339][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2704.841479][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2704.841588][ T44] ? netns_install+0x280/0x280 [ 2704.841697][ T44] ? cleanup_net+0x3bc/0x890 [ 2704.841804][ T44] ? process_one_work+0xd16/0x1390 [ 2704.841920][ T44] ? net_passive_dec+0x190/0x190 [ 2704.842033][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.842139][ T44] ? process_one_work+0xd16/0x1390 [ 2704.842250][ T44] ? lock_acquire+0x10a/0x150 [ 2704.842357][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.842468][ T44] ? process_one_work+0xd57/0x1390 [ 2704.842578][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2704.842687][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2704.842797][ T44] ? assign_work+0x152/0x380 [ 2704.842910][ T44] ? worker_thread+0x4d6/0xd40 [ 2704.843023][ T44] ? process_one_work+0x1390/0x1390 [ 2704.843130][ T44] ? kthread+0x355/0x5b0 [ 2704.843212][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2704.843319][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.843430][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.843539][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2704.843646][ T44] ? ret_from_fork+0x3fb/0x510 [ 2704.843755][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2704.843894][ T44] ? __switch_to+0x53c/0xd00 [ 2704.844003][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2704.844112][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2704.844225][ T44] [ 2704.844337][ T44] [ 2704.844337][ T44] Showing all locks held in the system: [ 2704.844500][ T44] 4 locks held by kworker/u16:0/12: [ 2704.844608][ T44] 1 lock held by khungtaskd/44: [ 2704.844711][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2704.844935][ T44] 1 lock held by modprobe/24613: [ 2704.845038][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2704.845258][ T44] [ 2704.845311][ T44] ============================================= [ 2704.845311][ T44] [ 2827.711855][ T44] INFO: task modprobe:24613 blocked for more than 860 seconds. [ 2827.712143][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2827.712346][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2827.712582][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2827.712884][ T44] Call Trace: [ 2827.712979][ T44] [ 2827.713045][ T44] __schedule+0xae2/0x1720 [ 2827.713182][ T44] ? io_schedule_timeout+0x130/0x130 [ 2827.713315][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.713452][ T44] schedule+0xd4/0x260 [ 2827.713548][ T44] schedule_preempt_disabled+0x18/0x30 [ 2827.713669][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2827.713787][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2827.713916][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.714033][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.714147][ T44] ? register_pernet_subsys+0x19/0x40 [ 2827.714262][ T44] ? lock_acquire+0x10a/0x150 [ 2827.714381][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2827.714504][ T44] down_write+0x1c6/0x1f0 [ 2827.714591][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2827.714736][ T44] register_pernet_subsys+0x19/0x40 [ 2827.714853][ T44] pg_init+0x29/0x1000 [pktgen] [ 2827.714964][ T44] do_one_initcall+0xa3/0x2e0 [ 2827.715075][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2827.715216][ T44] ? kasan_unpoison+0x44/0x70 [ 2827.715328][ T44] do_init_module+0x27b/0x820 [ 2827.715444][ T44] ? free_module+0x450/0x450 [ 2827.715561][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2827.715677][ T44] init_module_from_file+0x15d/0x180 [ 2827.715790][ T44] ? __do_sys_init_module+0x200/0x200 [ 2827.715910][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.716020][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2827.716136][ T44] idempotent_init_module+0x226/0x760 [ 2827.716246][ T44] ? init_module_from_file+0x180/0x180 [ 2827.716362][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.716474][ T44] ? cap_capable+0x181/0x3f0 [ 2827.716585][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2827.716696][ T44] do_syscall_64+0xbd/0xfc0 [ 2827.716806][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2827.716946][ T44] RIP: 0033:0x7f4985ad034d [ 2827.717061][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2827.717230][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2827.717399][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2827.717565][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2827.717727][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2827.717898][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2827.718069][ T44] [ 2827.718153][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2827.718381][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2827.718648][ T44] Workqueue: netns cleanup_net [ 2827.718757][ T44] Call Trace: [ 2827.718845][ T44] [ 2827.718901][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.719046][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.719155][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.719264][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.719404][ T44] ? lock_acquire+0x10a/0x150 [ 2827.719518][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.719625][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.719734][ T44] ? trace_contention_end+0xd6/0x110 [ 2827.719848][ T44] ? __mutex_lock+0x285/0x1f20 [ 2827.719958][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.720095][ T44] ? ww_mutex_lock+0x160/0x160 [ 2827.720207][ T44] ? __lock_acquire+0x577/0xc10 [ 2827.720318][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.720429][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.720565][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.720701][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2827.720873][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2827.721039][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2827.721179][ T44] ? kfree+0x119/0x580 [ 2827.721261][ T44] ? lock_acquire+0x10a/0x150 [ 2827.721373][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2827.721512][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2827.721622][ T44] ? netns_install+0x280/0x280 [ 2827.721736][ T44] ? cleanup_net+0x3bc/0x890 [ 2827.721847][ T44] ? process_one_work+0xd16/0x1390 [ 2827.721955][ T44] ? net_passive_dec+0x190/0x190 [ 2827.722063][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.722169][ T44] ? process_one_work+0xd16/0x1390 [ 2827.722281][ T44] ? lock_acquire+0x10a/0x150 [ 2827.722387][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.722503][ T44] ? process_one_work+0xd57/0x1390 [ 2827.722616][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2827.722724][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2827.722842][ T44] ? assign_work+0x152/0x380 [ 2827.722952][ T44] ? worker_thread+0x4d6/0xd40 [ 2827.723065][ T44] ? process_one_work+0x1390/0x1390 [ 2827.723174][ T44] ? kthread+0x355/0x5b0 [ 2827.723257][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2827.723367][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.723479][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.723586][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2827.723694][ T44] ? ret_from_fork+0x3fb/0x510 [ 2827.723806][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2827.723945][ T44] ? __switch_to+0x53c/0xd00 [ 2827.724053][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2827.724164][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2827.724277][ T44] [ 2827.724390][ T44] [ 2827.724390][ T44] Showing all locks held in the system: [ 2827.724550][ T44] 4 locks held by kworker/u16:0/12: [ 2827.724657][ T44] 1 lock held by khungtaskd/44: [ 2827.724759][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2827.724983][ T44] 1 lock held by modprobe/24613: [ 2827.725089][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2827.725309][ T44] [ 2827.725363][ T44] ============================================= [ 2827.725363][ T44] [ 2950.591855][ T44] INFO: task modprobe:24613 blocked for more than 983 seconds. [ 2950.592144][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 2950.592339][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2950.592567][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2950.592874][ T44] Call Trace: [ 2950.592968][ T44] [ 2950.593038][ T44] __schedule+0xae2/0x1720 [ 2950.593168][ T44] ? io_schedule_timeout+0x130/0x130 [ 2950.593304][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.593434][ T44] schedule+0xd4/0x260 [ 2950.593531][ T44] schedule_preempt_disabled+0x18/0x30 [ 2950.593655][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2950.593777][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2950.593910][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.594023][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.594136][ T44] ? register_pernet_subsys+0x19/0x40 [ 2950.594248][ T44] ? lock_acquire+0x10a/0x150 [ 2950.594362][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 2950.594477][ T44] down_write+0x1c6/0x1f0 [ 2950.594565][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2950.594708][ T44] register_pernet_subsys+0x19/0x40 [ 2950.594824][ T44] pg_init+0x29/0x1000 [pktgen] [ 2950.594937][ T44] do_one_initcall+0xa3/0x2e0 [ 2950.595049][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2950.595188][ T44] ? kasan_unpoison+0x44/0x70 [ 2950.595298][ T44] do_init_module+0x27b/0x820 [ 2950.595409][ T44] ? free_module+0x450/0x450 [ 2950.595523][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2950.595634][ T44] init_module_from_file+0x15d/0x180 [ 2950.595743][ T44] ? __do_sys_init_module+0x200/0x200 [ 2950.595861][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.595970][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2950.596078][ T44] idempotent_init_module+0x226/0x760 [ 2950.596189][ T44] ? init_module_from_file+0x180/0x180 [ 2950.596300][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.596409][ T44] ? cap_capable+0x181/0x3f0 [ 2950.596524][ T44] __x64_sys_finit_module+0xcd/0x160 [ 2950.596633][ T44] do_syscall_64+0xbd/0xfc0 [ 2950.596742][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2950.596882][ T44] RIP: 0033:0x7f4985ad034d [ 2950.596999][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2950.597167][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 2950.597329][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 2950.597489][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 2950.597655][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 2950.597825][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 2950.597993][ T44] [ 2950.598076][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2950.598306][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2950.598575][ T44] Workqueue: netns cleanup_net [ 2950.598684][ T44] Call Trace: [ 2950.598767][ T44] [ 2950.598828][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.598971][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.599080][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.599188][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.599388][ T44] ? lock_acquire+0x10a/0x150 [ 2950.599494][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.599605][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.599713][ T44] ? trace_contention_end+0xd6/0x110 [ 2950.599827][ T44] ? __mutex_lock+0x285/0x1f20 [ 2950.599936][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.600074][ T44] ? ww_mutex_lock+0x160/0x160 [ 2950.600182][ T44] ? __lock_acquire+0x577/0xc10 [ 2950.600295][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.600406][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.600545][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.600680][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2950.600851][ T44] ? nf_conntrack_cleanup_net_list+0x1aa/0x550 [nf_conntrack] [ 2950.601016][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2950.601152][ T44] ? kfree+0x119/0x580 [ 2950.601234][ T44] ? lock_acquire+0x10a/0x150 [ 2950.601346][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2950.601483][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2950.601595][ T44] ? netns_install+0x280/0x280 [ 2950.601709][ T44] ? cleanup_net+0x3bc/0x890 [ 2950.601822][ T44] ? process_one_work+0xd16/0x1390 [ 2950.601932][ T44] ? net_passive_dec+0x190/0x190 [ 2950.602041][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.602150][ T44] ? process_one_work+0xd16/0x1390 [ 2950.602258][ T44] ? lock_acquire+0x10a/0x150 [ 2950.602366][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.602479][ T44] ? process_one_work+0xd57/0x1390 [ 2950.602597][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2950.602705][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2950.602822][ T44] ? assign_work+0x152/0x380 [ 2950.602933][ T44] ? worker_thread+0x4d6/0xd40 [ 2950.603045][ T44] ? process_one_work+0x1390/0x1390 [ 2950.603160][ T44] ? kthread+0x355/0x5b0 [ 2950.603246][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2950.603355][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.603464][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.603575][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2950.603685][ T44] ? ret_from_fork+0x3fb/0x510 [ 2950.603794][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2950.603933][ T44] ? __switch_to+0x53c/0xd00 [ 2950.604043][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2950.604153][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2950.604266][ T44] [ 2950.604384][ T44] [ 2950.604384][ T44] Showing all locks held in the system: [ 2950.604545][ T44] 4 locks held by kworker/u16:0/12: [ 2950.604653][ T44] 1 lock held by khungtaskd/44: [ 2950.604756][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2950.604983][ T44] 1 lock held by modprobe/24613: [ 2950.605085][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2950.605307][ T44] [ 2950.605361][ T44] ============================================= [ 2950.605361][ T44] [ 3073.472858][ T44] INFO: task modprobe:24613 blocked for more than 1105 seconds. [ 3073.473131][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 3073.473320][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3073.473537][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 3073.473847][ T44] Call Trace: [ 3073.473947][ T44] [ 3073.474020][ T44] __schedule+0xae2/0x1720 [ 3073.474151][ T44] ? io_schedule_timeout+0x130/0x130 [ 3073.474273][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.474399][ T44] schedule+0xd4/0x260 [ 3073.474491][ T44] schedule_preempt_disabled+0x18/0x30 [ 3073.474618][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3073.474743][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3073.474876][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.475001][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.475144][ T44] ? register_pernet_subsys+0x19/0x40 [ 3073.475262][ T44] ? lock_acquire+0x10a/0x150 [ 3073.475377][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 3073.475491][ T44] down_write+0x1c6/0x1f0 [ 3073.475575][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3073.475715][ T44] register_pernet_subsys+0x19/0x40 [ 3073.475831][ T44] pg_init+0x29/0x1000 [pktgen] [ 3073.475943][ T44] do_one_initcall+0xa3/0x2e0 [ 3073.476056][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3073.476194][ T44] ? kasan_unpoison+0x44/0x70 [ 3073.476304][ T44] do_init_module+0x27b/0x820 [ 3073.476415][ T44] ? free_module+0x450/0x450 [ 3073.476524][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3073.476639][ T44] init_module_from_file+0x15d/0x180 [ 3073.476749][ T44] ? __do_sys_init_module+0x200/0x200 [ 3073.476873][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.476982][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3073.477092][ T44] idempotent_init_module+0x226/0x760 [ 3073.477201][ T44] ? init_module_from_file+0x180/0x180 [ 3073.477313][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.477420][ T44] ? cap_capable+0x181/0x3f0 [ 3073.477532][ T44] __x64_sys_finit_module+0xcd/0x160 [ 3073.477639][ T44] do_syscall_64+0xbd/0xfc0 [ 3073.477749][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3073.477892][ T44] RIP: 0033:0x7f4985ad034d [ 3073.478008][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3073.478174][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 3073.478338][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 3073.478501][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 3073.478663][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 3073.478832][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 3073.479009][ T44] [ 3073.479093][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 3073.479325][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 3073.479587][ T44] Workqueue: netns cleanup_net [ 3073.479698][ T44] Call Trace: [ 3073.479782][ T44] [ 3073.479844][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.479987][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.480095][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.480203][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.480338][ T44] ? lock_acquire+0x10a/0x150 [ 3073.480446][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.480556][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.480664][ T44] ? trace_contention_end+0xd6/0x110 [ 3073.480776][ T44] ? pick_task_fair+0x7d/0x1e0 [ 3073.480895][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.481032][ T44] ? ww_mutex_lock+0x160/0x160 [ 3073.481141][ T44] ? __lock_acquire+0x577/0xc10 [ 3073.481252][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.481365][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.481501][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.481636][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 3073.481802][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 3073.481975][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 3073.482115][ T44] ? kfree+0x119/0x580 [ 3073.482198][ T44] ? lock_acquire+0x10a/0x150 [ 3073.482312][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 3073.482450][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 3073.482561][ T44] ? netns_install+0x280/0x280 [ 3073.482674][ T44] ? cleanup_net+0x3bc/0x890 [ 3073.482783][ T44] ? process_one_work+0xd16/0x1390 [ 3073.482900][ T44] ? net_passive_dec+0x190/0x190 [ 3073.483011][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.483118][ T44] ? process_one_work+0xd16/0x1390 [ 3073.483224][ T44] ? lock_acquire+0x10a/0x150 [ 3073.483332][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.483440][ T44] ? process_one_work+0xd57/0x1390 [ 3073.483551][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 3073.483660][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 3073.483772][ T44] ? assign_work+0x152/0x380 [ 3073.483889][ T44] ? worker_thread+0x4d6/0xd40 [ 3073.484001][ T44] ? process_one_work+0x1390/0x1390 [ 3073.484108][ T44] ? kthread+0x355/0x5b0 [ 3073.484192][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3073.484300][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.484410][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.484521][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3073.484630][ T44] ? ret_from_fork+0x3fb/0x510 [ 3073.484739][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 3073.484882][ T44] ? __switch_to+0x53c/0xd00 [ 3073.484992][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3073.485100][ T44] ? ret_from_fork_asm+0x11/0x20 [ 3073.485212][ T44] [ 3073.485332][ T44] [ 3073.485332][ T44] Showing all locks held in the system: [ 3073.485497][ T44] 4 locks held by kworker/u16:0/12: [ 3073.485605][ T44] 1 lock held by khungtaskd/44: [ 3073.485707][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3073.485934][ T44] 1 lock held by modprobe/24613: [ 3073.486037][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3073.486263][ T44] [ 3073.486317][ T44] ============================================= [ 3073.486317][ T44] [ 3196.352856][ T44] INFO: task modprobe:24613 blocked for more than 1228 seconds. [ 3196.353147][ T44] Tainted: G W 6.19.0-rc3-virtme #1 [ 3196.353346][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3196.353573][ T44] task:modprobe state:D stack:25136 pid:24613 tgid:24613 ppid:1 task_flags:0x400100 flags:0x00080002 [ 3196.353889][ T44] Call Trace: [ 3196.354001][ T44] [ 3196.354087][ T44] __schedule+0xae2/0x1720 [ 3196.354231][ T44] ? io_schedule_timeout+0x130/0x130 [ 3196.354360][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.354486][ T44] schedule+0xd4/0x260 [ 3196.354574][ T44] schedule_preempt_disabled+0x18/0x30 [ 3196.354694][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3196.354825][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3196.354949][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.355073][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3196.355200][ T44] ? register_pernet_subsys+0x19/0x40 [ 3196.355326][ T44] ? lock_acquire+0x10a/0x150 [ 3196.355447][ T44] ? pg_cleanup+0x80/0x80 [pktgen] [ 3196.355561][ T44] down_write+0x1c6/0x1f0 [ 3196.355644][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3196.355783][ T44] register_pernet_subsys+0x19/0x40 [ 3196.355898][ T44] pg_init+0x29/0x1000 [pktgen] [ 3196.356015][ T44] do_one_initcall+0xa3/0x2e0 [ 3196.356126][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3196.356266][ T44] ? kasan_unpoison+0x44/0x70 [ 3196.356378][ T44] do_init_module+0x27b/0x820 [ 3196.356490][ T44] ? free_module+0x450/0x450 [ 3196.356601][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3196.356711][ T44] init_module_from_file+0x15d/0x180 [ 3196.356826][ T44] ? __do_sys_init_module+0x200/0x200 [ 3196.356940][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.357051][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3196.357159][ T44] idempotent_init_module+0x226/0x760 [ 3196.357264][ T44] ? init_module_from_file+0x180/0x180 [ 3196.357376][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.357483][ T44] ? cap_capable+0x181/0x3f0 [ 3196.357595][ T44] __x64_sys_finit_module+0xcd/0x160 [ 3196.357703][ T44] do_syscall_64+0xbd/0xfc0 [ 3196.357820][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3196.357954][ T44] RIP: 0033:0x7f4985ad034d [ 3196.358073][ T44] RSP: 002b:00007fff32ccc058 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3196.358239][ T44] RAX: ffffffffffffffda RBX: 0000563378237a00 RCX: 00007f4985ad034d [ 3196.358402][ T44] RDX: 0000000000000000 RSI: 000056335e038f2e RDI: 0000000000000005 [ 3196.358565][ T44] RBP: 00007fff32ccc0f0 R08: 0000000000000000 R09: 0000000000000000 [ 3196.358726][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000056335e038f2e [ 3196.358893][ T44] R13: 0000000000040000 R14: 0000563378237b20 R15: 0000563378237460 [ 3196.359066][ T44] [ 3196.359148][ T44] INFO: task modprobe:24613 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 3196.359378][ T44] task:kworker/u16:0 state:R running task stack:24784 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 3196.359638][ T44] Workqueue: netns cleanup_net [ 3196.359749][ T44] Call Trace: [ 3196.359838][ T44] [ 3196.359894][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.360040][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.360146][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3196.360256][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.360392][ T44] ? lock_acquire+0x10a/0x150 [ 3196.360498][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.360607][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.360716][ T44] ? trace_contention_end+0xd6/0x110 [ 3196.360829][ T44] ? __mutex_lock+0x285/0x1f20 [ 3196.360939][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.361078][ T44] ? ww_mutex_lock+0x160/0x160 [ 3196.361186][ T44] ? __lock_acquire+0x577/0xc10 [ 3196.361296][ T44] ? __mutex_unlock_slowpath+0x14b/0x540 [ 3196.361406][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3196.361512][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 3196.361626][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.361760][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.361900][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 3196.362069][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 3196.362236][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 3196.362371][ T44] ? kfree+0x119/0x580 [ 3196.362454][ T44] ? lock_acquire+0x10a/0x150 [ 3196.362568][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 3196.362706][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 3196.362822][ T44] ? netns_install+0x280/0x280 [ 3196.362934][ T44] ? cleanup_net+0x3bc/0x890 [ 3196.363045][ T44] ? process_one_work+0xd16/0x1390 [ 3196.363155][ T44] ? net_passive_dec+0x190/0x190 [ 3196.363263][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.363369][ T44] ? process_one_work+0xd16/0x1390 [ 3196.363476][ T44] ? lock_acquire+0x10a/0x150 [ 3196.363582][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.363698][ T44] ? process_one_work+0xd57/0x1390 [ 3196.363810][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 3196.363924][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 3196.364037][ T44] ? assign_work+0x152/0x380 [ 3196.364146][ T44] ? worker_thread+0x4d6/0xd40 [ 3196.364259][ T44] ? process_one_work+0x1390/0x1390 [ 3196.364367][ T44] ? kthread+0x355/0x5b0 [ 3196.364451][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3196.364558][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.364665][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.364776][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3196.364890][ T44] ? ret_from_fork+0x3fb/0x510 [ 3196.365001][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 3196.365136][ T44] ? __switch_to+0x53c/0xd00 [ 3196.365245][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3196.365354][ T44] ? ret_from_fork_asm+0x11/0x20 [ 3196.365466][ T44] [ 3196.365548][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 3196.365768][ T44] [ 3196.365768][ T44] Showing all locks held in the system: [ 3196.365931][ T44] 4 locks held by kworker/u16:0/12: [ 3196.366044][ T44] 1 lock held by khungtaskd/44: [ 3196.366150][ T44] #0: ffffffff99185140 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3196.366368][ T44] 1 lock held by modprobe/24613: [ 3196.366470][ T44] #0: ffffffff99885e50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3196.366691][ T44] [ 3196.366744][ T44] ============================================= [ 3196.366744][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr